Defeating Hackers and Malware With Disorder

Entropy — the measure of disorder or randomness — isn’t always desirable in the world of IT security. Kinda, sorta patching your IT systems sometimes, for example, would be a bad thing. At times, though, entropy can be a powerful tool, as in the case of well-chosen passwords that are difficult to crack. A fast-growing SIEM (security incident event managment) company called Vigilant is using entropy in an innovative way that warrants a closer look: Its anomaly-detection service identifies malicious threats based on entropy.

First, a quick primer: Entropy, often measured in bits, is the technical measurement of the randomness of the next piece of data in a string. If you see a sequence of letters with a clear pattern, such as “ABABABABA,” you would logically predict the next letter in the series will be B. Because the answer is fairly certain, the entropy would be 0. If you’re flipping a coin, the predicted outcome, heads or tails, is considered to be 1 bit of entropy. If a native English speaker is shown a sequence of standard English text and is asked to predict the next letter, he or she could guess it with great accuracy. English text is considered to have an entropy of 0.6 to 1.5 bits.

One more example: A truly random 8-character password, using every possible character on a standard keyboard, could have 52 bits of entropy. Unfortunately, most people use common words as part of their passwords, so most have only 18 bits of entropy. Thus, a password cracker doesn’t need to work through every permutation, just the most likely ones in the range of 2¹⁸, which is far easier than 2⁵². (Read Appendix A and Table A.1 of NIST Special Publication 800-63 for all the nerdy calculations and details.)

In the realm of IT security, antispam services have long used entropy and its link to anomaly detection to sniff out unwanted messages. If the service detects a single email account sending out messages to thousands of seemingly random and unconnected addresses, the provider will usually examine the message for other indicators that it’s spam. Entropy and its link in anomaly detection isn’t new, but Vigilant has extended it further to fighting malware.

Malware often generates random file names when writing Trojans to the system directory. When I perform light forensic analysis on a Windows machine, my first clue that it’s been compromised is one or more very strange-looking file names, such as vx3kjngq.exe or bb9[qamz.exe, in the Windows/System32 folder. Many malware programs use nonrandom names, but enough use randomly generated names that high-entropy filenames are a fairly good — if only rudimentary — indicator of something bad. (If you want to perform an experiment on your system, look for high-entropy file names in unexpected places, such as Windows/System32 or a root folder. Not every extremely weird name is a sign of maliciousness — but it’s a consistent characteristic.)

Vigilant has figured out that the average domain name has from 2.5 to 3.9 bits of entropy per byte. The company’s analysis algorithms flag DNS domains with more than 4.0 bits of entropy per byte; it also looks for less frequently occurring top-level domains, such as .biz and .info, with less entropy.

Vigilant’s services also looks for high-entropy file names and network connections to unexpected locations. But perhaps my favorite Vigilant check is for high entropy within HTTP content. Most HTTP content is close to the English language (or whatever native language is used) and should have low entropy. Encryption, on the other hand, has — or should have — high entropy. In fact, good crypto should have such high entropy that its encrypted ciphertext is indistinguishable from “noise.” Vigilant knows that advanced persistent threats often send out victims’ data in encrypted form using HTTP versus the normally expected encrypted HTTPS. If Vigilant’s service sees an outbound data stream using HTTP with high entropy, it raises a red flag.

Again, Vigilant isn’t the first company to use randomness in its anomaly detection, but it’s the first company of which I am aware that looks for, measures, and alerts of entropy characteristics. Sure, the company does all the traditional anomaly detection, but I like the fact that Vigilant uses mathematics and expected probabilities to add in another type of measurement.

Continue Reading»

Malware Surpasses 75 Million Unique Pests in a Year

Despite McAfee predicting that unique malware samples would hit 75 million in 2011, the security vendor actually found that the real number actually surpassed that estimate.

The vendor’s latest report, McAfee Threats Report: Fourth Quarter 2011, finds that while new malware slowed in Q4, mobile malware was on the rise and experienced its busiest period to date.

McAfee Labs senior vice-president, Vincent Weafer, found the thread landscape in 2011 highly evolved, with a change in the motivation typical for cyber attacks.

“Increasingly, we’ve seen that no organization, platform or device is immune to the increasingly sophisticated and targeted threats,” he said.

While the good news in the report was that PC-based malware was found to have declined throughout Q4 of 2011, reaching a level that was in fact significantly lower than the same quarter a year earlier, the fact is unique malware samples exceeded 75 million.

McAfee found that Q4 2011 was the busiest period for mobile malware, with the victim in this case being the Android platform due to loopholes found by hackers in the open source OS .

“On a global basis, we are conducting more of our personal and business transactions through mobile devices, and this is creating new security risks and challenges in how we safeguard our commercial and personal data,” Weafer said.

An average of 9300 new malicious sites come up every day in Q4, an increase from 6500 in the earlier quarter. The majority originated from the United States (73 per cent), followed by the Europe-Middle East (more than 17 per cent) and Asia-Pacific (7 per cent) regions.

The downfall of spam in 2011 has been well documented by the media and vendors alike, and McAfee found spam levels to be at their lowest in countries such as the UK, Brazil, Argentina and South Korea.

Botnet growth was found to have rebounded in November and December, with the current “spearphishing” and spam being more sophisticated than ever before.

Confirming what provacyrights.org has been saying all along, the report confirmed that data breaches via hacking, malware, fraud and insiders have more than doubled since 2009, reaching 40 publicly reported breaches in Q4 alone.

Continue Reading»