Sorting SPAM

28 Feb 2007

I been using SpamAssassin for a while to help identify SPAM. About a week ago, I started seeing all messages that were being flagged as SPAM by SpamAssassin show up in my Inbox instead of in my SPAM folder.

Well, it irritated me enough a moment ago to actually take a look at the full headers of just such a message. Here are the headers added by SpamAssassin:

X-Spam-Flag: YES
X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on
X-Spam-Level: ***********************
X-Spam-Status: Yes, score=23.0 required=4.0 tests=BAYES_80,DRUGS_ERECTILE,
       URIBL_JP_SURBL,URIBL_SBL,URIBL_SC_SURBL,VIA_GAP_GRA autolearn=no version=3.1.8
       *  2.5 VIA_GAP_GRA BODY: Attempts to disguise the word 'viagra'
       *  2.0 BAYES_80 BODY: Bayesian spam probability is 80 to 95%
       *      [score: 0.8180]
       *  0.0 HTML_MESSAGE BODY: HTML included in message
       *  1.6 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in
       *      [Blocked - see <http ://>]
       *  1.6 URIBL_SBL Contains an URL listed in the SBL blocklist
       *      [URIs:]
       *  3.8 URIBL_AB_SURBL Contains an URL listed in the AB SURBL blocklist
       *      [URIs:]
       *  4.1 URIBL_JP_SURBL Contains an URL listed in the JP SURBL blocklist
       *      [URIs:]
       *  4.5 URIBL_SC_SURBL Contains an URL listed in the SC SURBL blocklist
       *      [URIs:]
       *  2.4 DRUGS_ERECTILE_OBFU Obfuscated reference to an erectile drug
       *  0.5 DRUGS_ERECTILE Refers to an erectile drug

(Now that’s one spammy piece of SPAM!)

OK, so I took a look at my ~/.mailfilter file on the server:

### SPAM
if ( /^X-Spam-Flag: *(yes|YES) / )
   to "$HOME/mail/.SPAM/"

Many of my readers may be eagle-eyed enough to spot the problem right away. If you said, “Hey, you’ve got a superfluous space after your closing parenthesis in your regular expression there,” then you got it.

That regex would match either “yes” or “YES” (they are case sensitive). I did this because at some point long ago, I had a rule on a system that used “yes”, but SpamAssassin today produces “YES” and I just didn’t want to have it missing stuff because of something like that.

I decided to further improve this regex so that it might be less likely I’ll have to “fix” it again:

### SPAM
if ( /^X-Spam-Flag: *[yY][eE][sS]/ )
   to "$HOME/mail/.SPAM/"

Problem solved.

BTW: the term SPAM originally came to be used in the computer world because of the Monty Python Spam sketch.

WordPress 2.1 Upgrade

10 Feb 2007

There were a few minor difficulties updating this blog to WordPress 2.1 Ella (reffering to Ella Fitzgerald). Over the past couple of weeks, I’ve slowly worked on fixing things. It looks like everything is the way it should be now.

There were some databases changes that weren’t handled very cleanly by the update script. I had to drop and recreate the DB from my backup (taken just before starting the upgrade process).

One of things that changed was the way that links are managed. The wp_linkcategories table was “replaced” with a new wp_link2cat table. Unfortunately, the upgrade script didn’t complete the conversion process.

Most of the work required to fix up my links was done within the management interface in my web browser. Not hard, but it took a few moments. One of the categories didn’t make it through the upgrade. Several others (but not all of them) from the categories list now show up, too. Upon further examination I discovered that the only categories that appeared were those which were not assigned to any posts, though, it appears in the UI that they intend all categories to be simultaneously usable for posts, pages & links. However, when I tried to assign a category which is in use for several posts to a link, that link no longer rendered.

The change from a separate set of categories for links to a unified categories system was certainly the right direction. It merely appears that testing of that part of the upgrade code wasn’t very thourough.

Writing this post, I discovered what appears to be a new auto-save feature. Every couple of minutes while the focus is in the post body input field, the auto-save kicks in and a text marker next to the save/publish buttons updates to say “Saved at h:mm:ss” (the time on this notebook). Nice!

On Global Warming

4 Feb 2007

I have to agree with Fozz in his recent post about global warming.

I remember (vaguely) when Mt. St. Hellens blew its top (we even have a jar of ash from our front yard that we kept). That one event released more “greenhouse gases” than all industrial and automotive emissions since man started industrializing. Yet, the environment overcame it (and fairly quickly, too). I read an article about a year ago about how man didn’t clean up all the ash (and the environment is still “self-cleaning”) in one valley.

Reading Fozz’s comments, the thought returned to me that any attempt to state that the quantity of greenhouse gases that man produces are responsible would require hard numbers on the total amount of energies across the spectrum that reach the Earth (reaching the upper atmosphere, penetrate partway and reach the surface). I’ve never heard or read about any good continuous measurements being taken on an ongoing basis. Maybe you have; if so, I’d love to hear about them.

In all the reading that I’ve done, I find it highly unlikely that the puny quantity of gases we produce could account for the observed changes. We just don’t have numbers going back long enough to determine whather this is just part of a normal cycle or not. We have some evidence that goes back a few hundred years that let us make some guesses, but not enough to really extrapolate a pattern with enough certainty to begin answering such questions.

In the 50′s & 60′s, there was a significant concern around the world about global cooling. It wasn’t until the mid 70′s and early 80′s that we reached the point where those fears were gone, only to be replaced with the current global warming concern.

Here’s my theory about man’s activities and global warming: I think that direct heating, thermal-punping and surfacing account for the vast majority of man’s contribution to global temperatures. Burning fuel to produce heat for the home, energy to move a car and so forth produce a lot of heat. Covering large areas with concrete drastically alters the thermodynamics of the Earth’s surface.

Another thing to remember is that almost all the data is about surface temperatures, as it should be. The temperatures at the surface are responsible for the climate that we have to live with. That’s the part we experience. Again, if there’s been a study with correlating data about temperatures aloft.

Overall, I just don’t think we have enough data or enough understanding of how the planet works to be able to properly asses the state of the Earth nor the true impact man has upon it. Scientists want to take the numbers and produce an equation that explains it all, but we can’t. We don’t have the data and even when we finally do, I think the equations will be far more complex than anyone today would ever conceive.

It seems that “global warming” is poised to become the next warm-n-fuzzy (no pun intended) political issue. I believe that we will be hearing a lot of politicians glom onto this “issue” over the next couple of years.