Planet Antispam

Researchers at the University of Alabama say almost all of the websites advertised through spam are hosted in China on servers protected by bulletproof hosting. That means that the ISPs who provide hosting to spammers and malicious domains simply don’t care and ignore abuse complaints and take down orders.

The researchers reviewed millions of spam message and found that over 69,000 unique domains hosted the websites found in the spams and of those, 70% were located in China, making it a definite spam haven.

“It is very normal that more than one-third of the domain names we see each day in spam messages come from China,” wrote Gary Warner, director of research in computer forensics at the university. “When one also considers the many ‘.com’ and ‘.ru’ domain names which are also hosted in China, the problem is much worse.”

The so-called bulletproof providers actively recruit spammers and cybercriminals, going as far as to post ads on the underground websites where they are known to socialize. These hosts ignore take down requests and abuse reports and even make IP addresses hard to trace. A Chinese domain name can be had for a mere 15 cents, which only adds to the problem.

The researchers aren’t sure all the providers hosting the spam domains are bulletproof however. They speculate that a few may simply not have the resources or understanding to deal with the problem. Curiously enough, while the Chinese government had made headlines and waves with its increasing attempts to censor the Internet in the name of fighting porn, they have had nothing to say about the spam problem. It’s not known if they are even aware that there is one!

July 03, 2009 08:10 AM

Yesterday I said that the original motivations for adding new TLDs were to break Verisign's monopoly on .COM, and to use domain names as directories. Competitive registrars broke the monopoly more effectively than any new domains, and the new domains that tried to be directories have failed. So what could a new TLD do?

---

Get rich quick: the new domains with the most registrations are .BIZ and .INFO, clones of .COM and .ORG for people who missed out the first time. Despite vigorous marketing and, for .INFO, price cutting, neither is more than a pale shadow of the original, and both are plagued with sleazy registrants. Nonetheless, we can expect a few more clones like .WEB, who will make their money from defensive trademark registrations, domain squatters, speculators, and a few suckers who think that SAUERKRAUT.WEB can be the gateway to a mail-order fortune.

Idealists: Another unpersuasive theory says that a TLD enables communities. The best example to date is .CAT for Catalonia, which is modestly successful but doesn't tell us much since Barcelona is a rich sophisticated city that would be awash in Internet content with or without a domain. On the other hand .MUSEUM is a noble failure, with only about 200 registrants, a lot of dead links, and negligible visibility. Two pleasant young men have been trying to get .BERLIN through ICANN for years, and there are other candidates like .ECO, but it's hard to see why anyone would switch from their existing domain in .DE or .ORG or whatever since they haven't for any of the community domains we have now. I've heard claims that tiny language groups in danger of dying out need their own TLD, but it seems to me that if they could raise the $185K that a TLD application costs, they'd be a lot better off hiring linguists and programmers to compile dictionaries and adapt text and web tools to work in the language.

Certification: sponsored TLDs are supposed to ensure that all of their registrants meet specific requirements, so you know that a domain in, say, .COOP is an actual co-operative. The flaw in this theory so far is that none of the sponsored TLDs so far have been in areas where there's a problem with fakes, nor do they have any process to verify that registrants remain eligible. The little poultry packer that registered CHICKEN.COOP sold out to a larger company, but nobody noticed they weren't a co-op any more until I wrote to .COOP management and told them. They thanked me and encouraged me to report any more violations I saw, so I guess I volunteered to be the compliance department. The number of registrations in .COOP is on the order of 1% of the co-ops in the world, so it appears that the other 99% of co-ops are getting along fine without a special domain.

The .PRO domain is supposed to be just for licensed doctors, lawyers, accountants and maybe other licensed professionals (the web site is a bit vague), who have to present their licenses to register, but a combination of mismanagement and financial problems have allowed in large numbers of speculators and other registrants who clearly don't meet the criteria, so it doesn't tell us anything useful. I could imagine that a .BANK domain that carefully vetted its registrants to be sure they were real banks with government banking licenses might help tell real from fake bank web sites and mail, but that certification niche seems to be taken already by green bar SSL certificates.

Branding: The new rules allow single owner domains, so we can expect Apple to get .MAC and probably other companies will register their name like .IBM or brand names. Marketers are doubtless salivating, but for regular users, it's hard to see why you'd want to be BOB.MAC and rent your identity to your computer vendor.

Non-English languages: This is the only one that has any urgency at all. China really wants .中国 in addition to .CN, and a lot of other countries with non-Roman writing would also like localized domains. ICANN has a separate process for non-ASCII TLDs, so I'll ignore them for now.

So running down this list, where's the compelling argument? Does anyone (ignoring those with vested interests) really think that more TLDs will break the .COM monopoly? That more "community" TLDs will be any more of a success than the failures to date? That anyone will use a TLD rather than a search engine as a directory?

The only unambigous beneficiary of new TLDs is ICANN, whose cash flow will increase by $185,000 per application, and all of the consultants they've hired to do the evaluations because ICANN's many highly paid staff evidently can't do it themselves. Since a lot of the new TLDs will be run by organizations with little or no experience as a registry, we can expect them to learn slowly and painfully about all the sleazy tricks that crooked registrants pull.

In sum, neither of the two classic arguments for new domains, competition and directories, have worked in the past decade, and there's no reason to think they will in the future. Other than support for non-English languages, all of the other rationales strike me as wishful thinking, not business models. So I look forward to .中国 and its ilk, but other than that, they're all going to fail, very expensively.

July 03, 2009 02:11 AM

Jack Goldsmith in the New York Times: Defend America, One Laptop at a Time: “…the private sector owns and controls most of the networks the government must protect. …the firms that build and run computer and communications networks focus on increasing profits, not protecting national security. They invest in levels of safety that satisfy their own purposes, and tend not to worry when they contribute to insecure networks that jeopardize national security. This is a classic market failure that only government leadership can correct. The tricky task is for the government to fix the problem in ways that do not stifle innovation or unduly hamper civil liberties.”

July 02, 2009 06:03 PM

Exact Target: The Tipping Point Between Inbox and Spambox:

“Many in our industry don’t help the matter at all because they prefer to create fear, uncertainty and doubt (FUD) around email delivery because they feel that obfuscation will serve them well and buy them customers. The outcome is confusion and distrust.”

Not sure why Chip didn’t also mention that most email marketing blogs (and twits) just repeat what other people in the industry said the day before.

July 02, 2009 05:03 PM

Spam Wars: More on the URL Shorteners: “What a great way for a company to build an online brand presence—by hiding behind a URL shortener. WTF?”

July 02, 2009 04:03 PM

ComputerWorld: Registrars under fire in domain disputes:

“Are domain registrars making money from cybersquatters at the expense of legitimate brands? If so, why isn’t ICANN stopping it?”

Perhaps because the fees trickle upwards, and thus ICANN is making money from this abusive behavior too?

July 02, 2009 03:02 PM

Wired Threat Level: This Just In: Fake News Sites Are Great!: “There’s something icky about the fake news ads showing up on genuine news sites like Salon, Slate and Huffington Post. … It turns out there’s a whole fake-media empire pushing the story of the massive profits to be made by gaming Google from home…. Consumers who have signed up for the $2 trial have reported being hit with surprise charges on their credit cards ranging from $70 to $80.”

July 02, 2009 02:02 PM

I was intrigued by a story in the Daily Telegraph about a new electric motor created by Dyson. The DC motor apparently rotates at 104,000 RPM and is to be used in a portable vacuum cleaner.

The motor technology itself is switched reluctance. Essentially, the motor works by turning on and off electromagnets at just the right time to keep the rotor inside the motor spinning.

My immediate thought was 'how fast is the outside edge of the rotor moving if it's spinning at 104,000 RPM?' And shortly after that, 'how close is that to the speed of sound?'

In Electronic Weekly there's an article which states that the motor is 55.8mm across. Now, that's probably not the diameter of the rotor, but given that Dyson is attaching an impeller to the rotor anyway I'm going to take that as the diameter and work my calculations from there.

So the distance travelled in one rotation is π * 55.8mm and there are 104000 / 60 rotations per second. So, the outside is moving at 304ms^-1.

The speed of sound at sea level is 340ms^-1.

So the impeller is likely operating at near the speed of sound. I wonder if there are any nasty effects of rotating at that speed and if Dyson is close to the theoretical limit of what he can do.

There are two patent applications from Dyson that I believe cover this invention: 20070252551 and 20070278983. Neither mentions the speed of sound.

July 02, 2009 01:41 PM

A British security vendor has discovered that the ZBot Trojan has harvested the FTP credentials of over 68,000 websites including Bank of America, the BBC, Amazon, Cisco, Monster.com and most of the major anti-spam software makers. The credentials could allow hackers to compromise legitimate sites with malicious code and drive by downloads.

To make matters worse the list of FTP credentials is stored on a server in China in plain text, making it available to anyone who stops by. Experts say they were all stolen within the past 2 weeks and most are still valid.

The ZBot Trojan has also been spotted in several email attacks masquerading as everything from a ticket confirmation from Delta Airlines to a critical update for Microsoft Outlook. If downloaded it steals personal information using a keylogger.

It’s crucial to make sure any unused FTP credentials on your website are disabled and that active ones have their passwords changed regularly. As we saw recently when hundreds of government sites in the UK were compromised and redirected visitors to internet pharmacies selling Viagra or porn sites, hackers are eager to infect legit sites. If they hit yours it could be a real nightmare for you and your customers, so stay alert and keep an eye on your servers and FTP logins!

July 02, 2009 12:23 PM

The "The Business News" spammer who uses URL shortening services (noted here and here) has shown me another shortening service that doesn't give a crap about spam abuse reports — even though they solicit such reports directly on their home page.

I am now adding hurl.ws to my destructo spam filters. It's too bad, because on the surface they look like they want to do the right thing. Moreover, the outfit appears to be run by bluespark.co.nz, a fellow iPhone app developer (yeah, that's sort of been my day job recently). They advertise the service thusly:

Hurl is a url shortening service with a difference, ....

I guess the difference is that they turn a deaf ear to abuse complaints.

In return, my email server turns a deaf ear to any email message (from a non-whitelisted sender) whose body contains a hurl.ws URL. What's Maori for "Adios, amigos"?

July 02, 2009 07:11 AM

The following is an excerpt from Investor's Business Daily:

Microsoft (MSFT), the software giant, increased its market share in U.S. Web searches to 8.23% in June from 7.81% in May, thanks to its new Bing search site, according to tracking firm StatCounter. Web search king Google (GOOG) lost share slightly, dipping to 78.48% from 78.72%.

Figures like these really annoy me.  Why?  Because they are using statistics inaccurately.  Look at Google's "loss" of search share - a drop of 0.24%.  How could they possibly measure that?

In statistics, there is always a margin of error known as the confidence interval.  If you were to survey a group of users and 75% of them reported the same answer, then you cannot straight out extrapolate that to the rest of the population.  If you sampled ~1000 people, then you can say that 75% of the population, +/- 4% would give the same answer.  At a 95% confidence level, then you would say that you are 95% confident that between 71% - 79% of the population would give that answer.

Surveys work by doing random sampling.  Yet, in order to get the responses above, we have to make sure that the margin of error is less than the difference.  For example, in my above example, suppose you asked 1000 people what kind of widget they liked best and 67% of them said Widget A.  Next month, you ask 1000 people the same question and and 65% of them say Widget A.  Does that mean there was a drop of 2%?  No, because the 2% drop is within the 4% margin of error from the previous month.  You cannot be certain of anything.

In order for Google to have experienced actual market share loss, the original number had to be 78.72% +/- 0.11%, while the second number has to be 78.48% +/- 0.11%.  Why?  Because we have to have non-overlapping margins of error:

78.72 - 0.11 = 78.61%
78.48 + 0.11 = 78.59%

Those two do not overlap and thus we can be confident that real market share has been lost by Google.  So, how many people would the survey have to interview in order to get that confidence interval?  About 735,000.  I somehow doubt that this surveying company actually asked that many people what their favorite search engine is (or however they did their sampling).  In order for Microsoft to have gained their market share, they would need to have sampled 213,000 people.  Sounds unlikely to me unless they have some automated way of culling out all of this data.

People need to know how to use statistics properly.

July 02, 2009 01:52 AM

ICANN's Sydney meeting has come and gone, with the promised flood of new top-level domains claimed to be ever closer to reality. Does the world need more TLDs? Well, no.

---

Way back in the mid 1990s, it seemed obvious that Internet users would use the DNS as a directory, particularly once early web browsers started to add .COM to words typed in the address bar. This led to the first Internet land rush, with heavy hitters like Procter and Gamble registering diarrhea.com in 1995.

Everyone wanted to get into .COM, since that was the de-facto directory for the Internet. Network Solutions, the predecessor to Verisign, had a monopoly on registrations in .COM and that was a problem. Many people thought the solution was to add more TLDs with different monopoly registrars (often themselves.) I believe that I was the first to propose breaking the registration monopoly by splitting registries and registrars in December 1996. One of ICANN's undeniable successes is the competitive registrar market, which (as I predicted) as allowed a wide variety of sales models, and a lot of bundling of low-cost domains with web hosting and other services.

Since 1996 we've learned two things about TLDs: TLDs make a lousy directories, and users don't use the DNS for directories anyway. Several of the new TLDs introduced by ICANN since 2000 were intended to be structured as directories. The .AERO domain reserved two letter domains for airlines and three letter domains for airports, using standard industry codes, which was a clever idea, but not one that interested many airlines or airports. The .MUSEUM domain tried very hard to be a directory, with names organized both by the type of museum (metropolitan.art.museum) and the location (vam.london.museum) but that didn't work either.

A huge change in the Net since the late 1990s is that everyone uses search engines to find what they're looking for, to the extent that many non-technical users don't know the difference between the address and search boxes in their browsers. (Sometimes they'll type a search term into the address box, which keeps domain squatters in business.)

So if TLDs aren't useful as directories, what could they be useful for? We'll address the possibilities tomorrow.

July 02, 2009 12:11 AM

Wired Threat Level: Filtering Companies Can’t Be Sued By Blacklisted Firms, Court Rules

July 01, 2009 10:04 PM

Howard Rheingold on SFGate: Crap Detection 101: ‘“Crap detection,” as Hemingway called it half a century ago, is more important than ever before, now that the automation of crapcasting has generated its own word: “spamming.” Unless a great many people learn the basics of online crap detection and begin applying their critical faculties en masse and very soon, I fear for the future of the Internet….’

July 01, 2009 07:54 PM

Another one who let go too early.
Fresh spam. Well relatively fresh, from one inbox yesterday (parts of it):

Subject: %SI_subj

What if you could %SI2_rnd10 your desire and %SI2_rnd11 by just %SI2_rnd12 %SI2_rnd13 step?
What if this step was %SI2_rnd14, %SI2_rnd15 and side-effect-free?

There is %SI2_rnd16 solution!
%SI2_rnd17 %SI2_rnd18 use %SI2_rnd20 to give their %SI2_rnd20 %SI2_rnd21 night fire!

If there are no %SI2_rnd22, why refusing to take one pilule before %SI2_rnd23?

%SI2_rnd24 of men did it – You can do it too!

read more

July 01, 2009 07:52 PM

Well, look what the spam cat just dragged in - address book importing spam from Bebo. Why is it that these various social networking and other sites seem to simply consider the incidental spam attendant to address book importing (if they think about it at all) to be the cost of ...

July 01, 2009 07:41 PM

39819 patterns, 11412 right anchor strings, 172127 test IPs.

Contribs from yesterday, plus more from a CBL list.txt I recently
resolved down to PTRs. This release matches 99.995% of the PTRs in
that CBL zone.

Was asked to start tracking couplets (pattern class and tech, taken
together as a sort of meta-identifier); there are no new couplets in
this release.

Download them here:

sendmail:
http://enemieslist.com/downloads/sendmail_access_db
http://enemieslist.com/downloads/rightanchors

postfix:
http://enemieslist.com/downloads/postfix_regexp_table
http://enemieslist.com/downloads/postfix_regexp_table-20090701

exim:
http://enemieslist.com/downloads/exim_hosts
http://enemieslist.com/downloads/exim_hosts-20090701

July 01, 2009 07:41 PM

New York Times: U.S. and Russia Differ on a Treaty for Cyberspace: “United States officials say the disagreement over approach has hindered international law enforcement cooperation, particularly given that a significant proportion of the attacks against American government targets are coming from China and Russia.”

July 01, 2009 06:54 PM

I've previously complained about poor technical support that I received from Hewlett-Packard. That particular incident isn't over yet... the issue has been escalated a couple of times, HP has told me they are end-of-lifeing the product, ... I'll write that up when it comes to a resolution.

But it's not all moaning! Two companies that have provided excellent customer service recently are Apple and Bugaboo. I dealt directly with Apple myself, a friend with small children told me about the Bugaboo goodness.

First off, Apple. I own a MacBook Pro that I bought in mid-2007. Unfortunately, it suddenly started to suffer from the NVIDIA GeForce 8600M GT problem a couple of months ago. The upshot was that my machine would boot but couldn't find a display adapter (or at least it found the Intel display adapter, not the NVIDIA one).

I verified that I could ssh into the machine and ran System Profiler on the command-line. A quick search by serial number showed that my machine was susceptible to this problem and that Apple offered free service.

So, I called AppleCare. I never bought AppleCare for this machine and for this problem I didn't need it. I described my problem in detail to the technician including the steps that I'd taken to try to resolve it (including resetting the PRAM and SMC) and he did something great. He completely avoided going through any script, realized that I knew what I was talking about and immediately set the machine up for repair.

Next step was an appointment with the Genius Bar. This was the most annoying part because Apple's Concierge software is poorly designed. But once at the Genius Bar I got my appointment in about 10 minutes of the allotted time. The technician immediately verified that I had the NVIDIA problem and that I was eligible for a motherboard replacement.

While I was chatting with him I mentioned that my iPhone headphones had a fault and I wanted to buy some new ones. He asked me how long I'd had the iPhone (about 3 months) and simply went and got me a new pair, for free, just like that.

Then he told me to expect that my MacBook Pro would take about a week to repair. I left the Apple Store and went into work. That evening Apple called me to tell me the laptop was ready.

Nice.

Now Bugaboo. My friend Bill has two small kids and one of them is always in a Bugaboo Cameleon stroller. These are really high-end and expensive bits of kit. But they are very, very well made.

Now Bill's Bugaboo's brakes had developed a fault. They didn't always work and it was a minor annoyance. Little did Bill know that Bugaboo had identified this as a common fault and recalled the Cameleon.

Happily, Bill had filled out the warranty card for the stroller and sent it back when he bought it. One day a small package arrived unannounced containing a kit to fix the brakes. The kit worked perfectly.

Nice.

In both cases, Apple and Bugaboo, we were dealing with premium brands and got premium support. Apple's ability to just give me new headphones made my experience wonderful, and Bugaboo simply sending the repair kit to Bill made him a loyal customer for life (he just needs to have some more kids).

July 01, 2009 05:25 PM

Nate Anderson reports for ARS Technica: "A federal judge yesterday found Usenet.com liable for just about every copyright infringement claim on the books: direct infringement, inducement of infringement, contributory infringement, and (just for good measure) vicarious infringement. Not content to be loud and proud about its pro-pirate agenda, Usenet.com also resorted to stonewalling legal

July 01, 2009 05:23 PM

I was discussing a spam problem with a customer recently and they mentioned to me that one of their biggest problems is spam sent to their email distribution lists.  The problem had come about due to two things - firstly the email addresses for some of their distribution lists are very easy to guess (eg, the “All Staff email group has an email address of allstaff[at]company.com), and secondly there had been occasions in the past where staff exposed the email addresses by CC’ing them on emails sent outside the company.

Over time the problem has grown to the point where it is now very frustrating for their staff.  They’ve asked me for some suggestions on how to fix this problem, so I presented them with these options.

Requiring Authentication for Exchange Server 2007 Distribution Groups

The default behavior for newly created distribution groups in Exchange Server 2007 is to require that all senders be authenticated, or the message is simply rejected.  This is useful, however, for a vast majority of Exchange Server 2007 organisations their distribution groups existed prior to the upgrade to Exchange Server 2007.  In these cases the authentication requirement is not enabled.To require authentication for a distribution group simply open the group properties, navigate to the Mail Flow Settings tab, open the Message Delivery Restrictions and then tick the box marked “Require that all senders are authenticated”.

While this solution has the desired effect of preventing spam from reaching the distribution group, it also prevents other legitimate outside email from reaching the list.

Filtering Distribution Groups by Sender

The authentication requirement will prevent legitimate outside email from reaching important distribution groups.  To resolve this through the same Message Delivery Restrictions you can instead control which senders are permitted to send to the distribution group.

This method causes some extra administrative burden for the email server admins because each permitted sender must first be added as an Exchange Contact.  Furthermore if you want the distribution group to receive emails from internal staff you need to ensure they are also added to the list, either directly or via a group.

Obscuring Distribution Group Email Addresses

One method that most email admins will try at least once in their career is to obscure the email address of distribution groups to make it harder to guess, or to make it impossible to send to from outside the organization.  In Exchange Server 2007 this is achieved by using Email Address Policies that apply only to distribution group objects.

For example, the policy may apply a string of characters to the email address to make it harder to guess, such as allstaff_ksf2ui2[at]company.com.  While this does have the effect of making it nearly impossible to guess it does nothing to prevent exposure of the email address if it were included in an email sent outside the organization.

A second technique is to use an SMTP domain that is invalid outside of the organization.  For example allstaff[at]groups.company.com or allstaff[at]company.local.  This has the effect of nullifying any exposure of the email address outside the organization but similar to the earlier filtering techniques it prevents legitimate outside email from reaching the group.

Implementing an Anti-Spam Solution

Although the customer was seeking a free solution once I explained each of the options above it became clear to them that these techniques would either be ineffective, require too much effort to maintain, or would prevent legitimate business use of their distribution groups.

Instead they agreed to trial an anti-spam solution, which satisfied them by preventing spam and other unwanted emails in an effective and easy to manage way, and which they ultimately purchased and are now happily getting on with their business without the constant hassle of spam.

July 01, 2009 12:41 PM

A new malware attack is lurking behind emails made to look like Outlook updates sent by Microsoft. The messages look authentic and include a link that looks like it points to update.microsoft.com but actually points to a malicious domain. If clicked the link activates a download which contains the Zbot Trojan. Zbot steals usernames, passwords and banking information and installs a rootkit that could allow a hacker access to any network the infected computer is attached to.

Zbot even contains a list of specific sites to monitor including Facebook, MySpace, Bank of America, Amazon, HSBC, Paypal, Blogger, and just about every bank you can think of. This Trojan means business. Once a user on an infected machine accesses one of the sites on the list, a built in keylogger is activated and records their information. The stolen information is then uploaded to a remote server.

Zbot has been spotted in several previous attacks. One pretended to be a notice from UPS, another a ticket confirmation from Delta Airlines and a third a notice from Western Union. The gang behind the attacks is said to be hiding out in Russia.

To protect yourself and your users, remember that common sense is a hacker’s worst enemy. They are hoping people will trust that it a real update from Microsoft even though it’s well known that Microsoft pushes their patches through on the second Tuesday of each month only and never ever sends them via email. If you get an update from anywhere other than the Microsoft Update console, chances are it’s fake. Make sure you have a policy in place regarding software installation. It’s probably best to restrict everyone but the IT department from doing any at all.

July 01, 2009 12:11 PM

For years now, I’ve been collecting bookmarks at delicious.com/jm — nearly 7000 of them by now. I’ve been scrupulous about tagging and describing each one, so they’re eminently searchable, too. I’ve frequently found this to be a very useful personal reference resource.

I was quite pleased to come across the Delicious Search Results on Google Greasemonkey userscript, accordingly. It intercepts Google searches, adding Delicious tag-search results at the top of the search page, and works pretty well. Unfortunately though, that searches all of delicious, not specifically my own bookmarks.

So here’s a quick hack fix to do just that:

my_delicious_search_results.user.js - My Delicious Search Results on Google

Shows tag-search results from my Delicious account on Google search pages, with links to more extensive Delicious searches. Use ‘User Script Commands‘ -> ‘Set Delicious Username‘ to specify your username.

Screenshot:

Enjoy!

July 01, 2009 09:58 AM

I'm going to attempt to summarize a blocklist without going to the article on Wikipedia.  I'll be doing this straight off the top of my head.

Motivation

A blocklist is essentially a shortcut to spam filtering.  Assume that you have a content filter that is doing all of the work of filtering, faithfully executing and flagging messages as spam.  Everything is great except that the spam filter is doing a lot of work and occasionally, the odd spam message or two slips through.  You can live with this if all you are filtering is 10,000 messages per day.

But imagine you are filtering 10 million messages per day.  Suddenly bandwidth becomes an issue because most of your bandwidth is being taken up by useless data (spam).  In addition, if your filter is "only" 99% effective, 100,000 spams are still leaking through to end users.  If your organization has 10,000 users (a good size company), then that's about 10 spams per day to the end user.

You need a way to make this work better. 

Methods

You sit down one day and start pouring through your spam samples that your end users are submitting to you.  "What's this?" you say out loud to no one in particular.  You observe that while the spams have no particular pattern, you do notice that they seem to be coming from a narrow set of IPs.  Let's say that out of 100 messages, you see the following pattern (I'm using hypothetical IPs):

IP	Spam Count
292.144.16.11	16
292.144.16.17	15
292.144.16.19	22
292.144.16.22	18
292.144.16.27	29

"That's odd," you say again.  "There seems to be a lot of IPs in that range."  You do a quick WHOIS lookup of that IP and you find that the IP space is owned by the organization Canadian Pharmaspammers.  "Well," you exclaim, "if these guys own those IPs, I should flat out block them all!  It is very unlikely that they will ever send out anything legitimate."  How do you know this?  Spammers never change their spots.  If a spammer sends out this much spam from these IPs, at that level of volume (100 messages randomly sampled) then you can safely conclude that they will never send out anything else.

You decide to add all five of those IPs to your own blocklist.  Anything that hits your network that comes from those IPs you will reject (how this works we'll get to in a future post).  You've now saved your end-users from getting spam from these IPs.

Refinements

You wipe your hands and assume the problem is solved.  But it's not; users are still getting Canadian Pharmaspam!  Once again, you start to grab the spam samples and looking at the connecting IP.  The content is all different -- again -- but the IPs look familiar:

IP	Spam Count
292.144.16.12	19
292.144.16.14	17
292.144.16.18	18
292.144.16.21	20
292.144.16.26	27

Those IPs look similar to the IPs you previously blocklisted.  You have no spam from those other IPs, but lots of spam from its sister IPs.  Once again, you decide to do a WHOIS look up on that IP and notice something you didn't see before.  It's listed to Canadian Pharmaspammers, but they also own the netblock 292.144.16.0/27 -- a netblock of 32 IPs.  You decide to get pre-emptive; you go into your personal blocklist and remove the previous five IPs and instead insert 292.144.16.0/27.  You have now listed the entire range of IPs.  You only have evidence from 10 different IPs but strongly suspect that spam is coming out of all of them, and therefore you engage in a pre-emptive strike.  You list the IP range, cross your fingers and hope for the best.

The next day you check your spam stats and notice something; rather than content filtering 10 million messages per day at the content filter, your upstream IP filter has cut that down to 1 million per day!  Gah!  That's a reduction of 90%!  Your content filter is flying!  Furthermore, the amount of spam complaints has gone down from 100 per day to 20 per day, a reduction of 80%.  By adding these IPs to the blocklist, you have accomplished two things:

1. Users are seeing less spam in their inboxes because while your filters are good, there may be gaps.  This blocklist fills in those gaps.
   
   
2. You have saved a good chunk on bandwidth and spending precious resources on less and less junk.

Those are the two basic uses of blocklists.  A third would be spam filter automation and leveraging the work of others, but we'll get to that in a future post.  But by and large, these impacts are immediately noticeable by everyone using the service and therefore, the use of blocklists eventually becomes indispensable if you want to run a filtering service.

July 01, 2009 04:37 AM

Just a heads-up; a variant of the Spanish Prisoner scam has been on the rise lately.

To recap: in the Spanish Prisoner scam, someone writes to you claiming to be a prisoner in a Spanish prison (the scam is said to goes back to the 1500's). If you send bail money, riches will be yours once he returns to freedom.

In the modern variant, the offer either arrives via random spam, or targeted directly to you through the compromised email account of a friend.

The latter form is the most insidious. The email actually comes from someone you know, claiming to be in dire straights of some sort or another. Typically your friend is traveling abroad, the email will say, and has been robbed of cash, credit cards, and ID. You are begged to send cash as quickly as possible so your poor friend doesn't wind up jailed as a vagrant or some other terrible thing.

If you're sharp, you might notice that your friend isn't calling you by your name. Or you might remember that your friend isn't traveling anywhere at all, and in fact you had poker night with them just last night.

If you're a little bit slow on the uptake, you might actually send some money. If that happens, expect to get requests for more (oops, too late, he got arrested for vagrancy and now needs bail money).

The requests for money will continue until you catch on or run out of money to send.

For a good account of the scam, read Gadi Evron's article Facebook Scam: I'm Stranded In London. Send Money!

So remember to be on the lookout. If you get email from a friend asking for emergency money, always double-check via some other channel. A phone call is best.

And if you're the one whose email, facebook, or other account has been used for a scam like this, be sure to contact everybody on your contacts list and warn them. Chances are, the scammer has been hitting every name on the list.

July 01, 2009 12:30 AM

We have our own home-grown sendmail antispam filters here, which use a fairly broad brush to score incoming mail, but which have been remarkably effective for us for over six years.

One of the data points we check is of course whether the sending host has a generic PTR, via the enemieslist DNSBL. But we also find it useful to check the TCP fingerprint of the sending host, to see if the box on the other end is running some form of Windows - particularly certain highly vulnerable releases and patchlevels, like Windows XP Service Pack 1. We also check to see whether the message in question is in multipart/alternative format, or "HTML email", because in our experience it's rare to see spam that is in plain text format.

Each of these conditions (HTML, Windows, generic PTR) scores a fairly low spam score, because of course it's perfectly normal for mail to be in HTML format, and there are many Windows boxes running MSExchange and other legitimate Windows-based mail server software. And of course, there are many small businesses with generic addressing on their static netspace. The problem is when we see all three together.

As a default, all of our local accounts here have a spam score threshold of 4, which is sufficient to keep out the vast majority of the inbound spam - especially if the local scoring has been tweaked to give high scores to generic HELOs and low to generic PTRs - and which lets almost all normal mail traffic through. For historical reasons, the scoring is all done in integers, so we don't have the fine-tuning capabilities available in SpamAssassin, for example, where an HTML message might get a 1.7 just for containing HTML and no text part. Here, by default, HTML email scores a point, any Windows system scores a point, and any other issue is usually enough to dump it into the quarantine. A static generic PTR gets 2 points. So, the Trifecta is 4 points, enough to reject on for most accounts.

Pretty much the only time we ever have to whitelist anyone here is when the sender has hit the Trifecta outlined above. HTML-only email, sending from a Windows box, with a generic (almost always static) PTR. What's sad about this isn't that we have to make up for their IT consultants' failure to bother to request a custom PTR, or that some people run MTA software that spits out HTML-only email. No, that's pretty much par for the course in any industry without a need for a full-time IT person or team. Lawyers, galleries, non-profits, small businesses of many kinds are subject to the pressure to conform - and to pay lots of money for Exchange (when they could use free, high-performance Unix-based mail server software). And for the skills needed to install it (poorly), maintain it (poorly) and patch and upgrade it (rarely). OK, enough Unix bigotry. For now.

Some will complain that we shouldn't be blocking (or even scoring discriminately) on known "statics". The problem is that there are a lot more statically assigned IPs out there that have unfiltered access to the rest of the Internet, and are vulnerable to infection by the botnets, than there are legitimate mail servers with generic PTRs.

For example, yesterday we blocked 349 messages sent from static generics out of 8810 total rejected messages, or 4% of our total rejections, with one false positive (the message that spurred on this post). Of those, 117 were from .com or .net hosts, with the rest coming from ccTLDs we rarely have legitimate traffic from, so we can't just accept from static generics with .com or .net TLDs.

To effectively work around the infected statics problem while avoiding the occasional Trifecta-as-FP problem will take some more analysis, or, some more widespread clue among WIndows IT consultants. And we're not going to reduce our overall filter effectiveness by 4% daily just because of a once-a-quarter FP due to a lack of care on the part of someone else. So we need to tweak, and tune, our policies on this end without compromising our perimeter defenses, or adding to my quarantine watch workload.

Our system usually generates what, to our biased minds, are perfectly useful and informative error messages, especially in response to particular problems. The problem with the Trifecta is that we're blocking based on a score, not a specific set of problems, so the error looks like this:

554 5.7.1 HISCORE Contact postmaster@hesketh.net if this is in error, but your message was rejected as spam; it simply failed too many tests. (threshold: 4; score: 4)

There's a token (for our stats), immediately followed by a contact email address that is more or less unfiltered, a rationale, and a score/threshold. The problem is that many Exchange servers either truncate the error message, rendering it less useful, or explain that the remote system did not provide a reason - often including the complete error message beneath! - which most people don't bother to read. So we get phone calls to the effect that our system is blocking their mail. Which it is, and in many cases these are actual false positives. So we whitelist their IP address, and they can send again. (Incidentally, of the 349 messages we rejected, six had a 4/4 threshold/score; one of those was the false positive. Two had a 4/5, two had a 4/6, three had a 4/7. So, one way to deal with this is to raise our default threshold to 5, thereby letting in 7 more spams a day in order to prevent a quarterly FP. This on a system where userbase-wide we see about 3 or 4 spams/day make it through the filters, and maybe a couple 419 scams and phishing scams. So, a difficult choice - how tolerant do we become, and how low do we sink in order to accommodate these arguably at-fault systems?)

What's even more annoying is that once we've whitelisted the sending IP address of one of these poor victims, they'll go home and try to send from Outlook Web Access, which many IT consultants set up on yet another IP address, also with a generic static PTR. So we go through the whole rigamarole again, only this time with their OWA IP address.

The real problem here is two-fold: the failure of IT consultants to have even the most basic understanding of the nature of deliverability and its relationship to the generic PTR question, and the continuing acceptance of such a low standard of compliance with email community norms. (And yes, there's a third factor, namely, my reluctance to raise the default spam score threshold just to accommodate these edge cases.)

So let me close with a plea to any IT consultant tasked with setting up a Windows-based mail system: please, for the love of all that is good and holy, ask your customers' ISPs for custom reverse DNS for any system legitimately sending mail. We'll tolerate your HTML-only email, and your choice of Windows, if you'll do your part and signal to us with a custom PTR that this is a system that is intended to send mail, rather than an infected end-user system or NAT or insecure LAN.

June 30, 2009 08:08 PM

39710 patterns, 11430 right anchor strings, 171622 test IPs.

Contribs from yesterday, plus more from a CBL list.txt I recently
resolved down to PTRs. This release matches 99.995% of the PTRs in
that CBL zone.

Was asked to start tracking couplets (pattern class and tech, taken
together as a sort of meta-identifier); there are no new couplets in
this release.

Download them here:

sendmail:
http://enemieslist.com/downloads/sendmail_access_db
http://enemieslist.com/downloads/rightanchors

postfix:
http://enemieslist.com/downloads/postfix_regexp_table
http://enemieslist.com/downloads/postfix_regexp_table-20090630

exim:
http://enemieslist.com/downloads/exim_hosts
http://enemieslist.com/downloads/exim_hosts-20090630

June 30, 2009 07:26 PM

Not all URL shorteners are created equal when it comes to handling abuse complaints. Yesterday's flood continues. I went back to see how my abuse reports faired. Of the services I contacted, the only one that seems truly diligent about stomping out spam abuse of their service is is.gd. Four gold stars for them!

The one that looks to be the most problematic is kl.am, which appears to be run by a Tennessee "online marketing" firm called Sitening LLC. Unlike the responsive shorteners, kl.am does not have an abuse reporting link on their main page...or anywhere. Moreover, the main page is titled:

Shorten URL with URL Shortener for Internet Marketers

In other words, they seem to be encouraging the use of URL shorteners by commercial emailers. What a great way for a company to build an online brand presence—by hiding behind a URL shortener. WTF?

OTOH, it makes it easy for me to handle any company that takes advantage of this shortening service for spam purposes. From hereon, any email message body that contains a kl.am URL arriving from a non-whitelisted address goes straight to dev/null. See y'all!

(Tinyurl may be next.)

June 30, 2009 04:15 PM

OK, it wasn't really, but I thought I'd run the Scacco/Beber analysis on that election and see what it comes up with. Guess what.

If you look at the non-adjacent, non-repeated digits in the last two places in the votes counts by state for Roosevelt and Dewey you discover that 59.38% of the votes are non-adjacent, non-repeated. If the numbers were truly random you'd expect 70%. That's way worse than the 62.07% in the Iranian election.

If you then do the old Z-Test you get a Z value of -2.49 with a p-value of 0.013. That's well below the 0.05 critical value so you can reject the null hypothesis. The final digits are not random.

Is this fraud?

Is there any suggestion that the state-level numbers in the 1944 US election were invented by people?

If not, how can anyone claim that this test indicates fraud in the Iranian election?

Now run the other bit of their test looking at the frequencies of the last digit. You get 'too many' 7s (expected 10%, got 16%) and 'too few' 1s (expected 10%, got 5%).

I'm telling you, man, what's the chance of that happening, and the non-adjacent, non-repeating digits thing? (It's about 0.17% according to simulation) I mean, come on, that's gotta be fraud.

Oh, wait, it's not.

June 30, 2009 03:04 PM

The Sydney Morning Herald reported yesterday that a new scam is making the rounds in the land down under. A perpetrator of a phishing scam has created an email scam, claiming to be the Australian Tax Office (ATO). The email promises Aussie taxpayers a $250 bonus with their tax return, and sends them to an online form that asks for their tax information, along with their bank account data.

The web site containing the form then asks the victim to mail a printed copy of the form to an address. The print-and-send is just a ruse though, the data is actually captured through a hack when the victim presses the “print” button. The email, like many such scams, attempts to create a sense of false security, by claiming the print-and-send routine is being done for the victim’s safety.

Officials still have not been able to trace the source of the fraudulent email sender, who is using a bot network to send the emails. The ATO recommends that people delete emails like this immediately, and advises that they do not ask people to provide personal information by email. The same holds true for most, if not all, tax collecting agencies in other countries.

June 30, 2009 01:36 PM