How does a web page harm thee? Let me count the ways....

As the web grows and diversifies, so do the number of ways in which web pages can harm users (for example). To help counter this threat, Google, Norton, and McAfee maintain "blacklists" that warn users away from potentially unsafe webpages. But what exactly does it mean to get blacklisted by these services? Do the lists agree on which sites are harmful?

When analyzing a blacklist, the primary fact to keep in mind is that the term "bad" is not well defined. As such, each list operates under different criteria. Google, for example, focuses on technical threats such as drive-by downloads. McAfee and Norton take a broader view, and will flag a site based on things like "annoyance factors" or "excessive popups." McAfee also signs up for any mailing lists it finds and records the amount of spam generated. Google limits itself to a binary response marking a site as potentially harmful (e.g., "This site may harm your computer") or not, while McAfee and Norton label websites as "Safe," "Caution," "Warning," or "Untested."

Given these divergent criteria, the first thing that is immediately apparent is that it would be much too simplistic to call all blacklisted sites "malicious," while calling all non-blacklisted sites "safe." To see how similar or different these blacklists are in practice, we took a set of relatively popular domains on the Internet and queried all three lists.

Of the sites that were flagged by at least one of the blacklists:

• 57% were marked as potentially harmful or "Warning," with the other 43% marked as "Caution."


• Google flagged 5.2% with a "This site may harm your computer" label.

• Norton flagged 16.3% of the sites with a rating of "Warning," and another 14.4% with a rating of "Caution."


• McAfee flagged 38.6% of the sites with a rating of "Warning," and another 32% with a rating of "Caution."

Per the statistics above, Google's list was by far the smallest, reflecting its focus on technical threats. Norton had far more "untested" sites than McAfee, partially explaining Norton's lower numbers.

When we compare which sites were blacklisted, however, the results become far more interesting. Of the sites that Google blacklisted:

• Norton labeled less than half with "Warning," half with "Safe," and the remaining ones as "Untested." None were labled with "Caution."


• McAfee labeled a quarter with "Warning," a quarter were "Untested," and the remaining half were safe. None were labled with "Caution."

• McAfee's users lodged complaints about more than half of the sites.

Norton and McAfee's blacklists also didn't agree with each other often. Of the sites flagged between them, only 4% were on both lists. Amazingly, the overlap between all three lists was less than 1%.

Of the sites flagged "Warning" or "Harmful" by at least one list, 61%
were flagged only by McAfee. Only 1% were flagged by all three lists.

These discrepancies shouldn't be surprising, given the fact that the lists employ different criteria and techniques for evaluating sites. But there are also deeper reasons for the lack of overlap: For one thing, the frequency and timing of testing can have a significant impact on the rating a site receives -- if a site is compromised after it's tested by a service, and that service doesn't test it again for another day, week, or even month, that site could end up infecting a significant number of its users while still being marked as "safe" by the service. For another, the diagnostics employed by these services aren't necessarily infallible -- web-based malware is sometimes masked to prevent its detection by some testing services.

So what does all this mean for protecting your business? How should you deal with the fact that there are so many blacklists out there, testing your site on several different criteria and with varying levels of effectiveness? At Dasient, we believe the answer is being proactive about monitoring and protecting your site from web-based malware. To learn how we can help you do that, click http://wam.dasient.com.

Attackers infect websites via ad networks, widgets

According to reports on Twitter as well as on ZDNet over the weekend, visitors to several high-profile websites were blocked from accessing parts of the sites because an advertising partner, Eyewonder, suffered a malware attack.



Malicious advertisements are increasingly being used by attackers as a vector for distributing malware via legitimate sites. In this case, what happens is that the malicious code that the website ends up serving to users is being sourced in from an advertising partner. The website itself has not been compromised by attackers -- rather, the ad network used by the site has been compromised. Attackers often use malicious ads to achieve scale and avoid detection. It would have been difficult for the attackers to infect a large number of high-profile websites directly; instead, they were able to leverage the trusted relationship between the websites and their ad network to get malicious content (in this case an ad) served to the sites' end users. In some cases, Dasient has added certain ad networks to its internal blacklist to inform its customers where there is a risk that ads may result in infecting their users.

In addition to malware coming in through ads, we have also seen cases where malicious code comes into a website via content mash-ups or third-party widgets. For example, third-party widgets such as traffic counters have been used to infect websites (see section 4.4 of "The Ghost in the Browser"). We have spoken to website owners who explained that their sites were infected not through attackers exploiting a vulnerability in the website, but because they included a plug-in or widget that ended up being malicious. In some cases, the widget is benign for a period of time (even years), but then drastically changes behavior to become malicious (either because the widget provider was itself malicious or, more likely, because the widget provider's servers were hacked).

Attackers will continue to find network and web application vulnerabilities in websites that they can exploit to directly plant malicious code. However, it is clear from the Eyewonder incident that the attackers will also seek ways of exploiting the trusted relationships between websites and their third-party advertising or content partners to create the same effect. The nature of the open web encourages websites to mash up best-of-breed content (and ads) from various sources. To reduce risk, it is important for websites to perform due diligence on all third-party content and ad providers, as well as employ automated detection and remediation services.