Schonhose

This week the guys and girls from Defensio have released the 2.0 version of their API. Not only a SPAM filter, but Defensio can now eliminate malware and other unwanted or risky content from your blog. The best part is Pixelpost supports the latest API version through a new release of the Defensio addon for Pixelpost.

A major difference is the way content is evaluated. The old version depended on an instant results after querying Defensio, sometimes resulting in comments not being processed when the service was hammered with request. Therefore the content is now evaluated asynchronous, sending the content to Defensio which will provide a callback when the processing is done.

Initially all comments for Pixelpost make it into the Defensio Quarantine. These comments can basically have two statuses: FAIL and PENDING. The later means that the results through the callback are not in yet (you can issue a query to fetch them right away). The status FAIL means that for some reason Defensio could not be contacted (either the service is down or the API key is invalid or..). In that case you can send these comments to Defensio for an evaluation.

The major drawback of the 1.0 version of the addon was that each comment that had failed had to be rechecked manually. This issue has been resolved with the addition of a new button that processes every comment with a status of FAIL or PENDING for the last two weeks. Obviously the buttons to send either HAM or SPAM to Defensio remain in full effect.

For more information about the new Defensio checkout the “What’s new with Defensio 2.0?” or the video below. You can also download the latest plugin for Pixelpost.

Over the last few months several threads have been posted on the Pixelpost forum regarding SPAM comments. In some of these threads the author boldly claims that Pixelpost isn’t stopping any SPAM. Well, since December 28th, 2006 I have installed a SPAMlog addon on my blog. Basically, what this addon does is keeping track if and why a comment is blocked.

First I think I have to elaborate on my settings, so here they are. I use the <TOKEN> setting from Pixelpost, along with a 30 seconds SPAMflood protection setting and a maximum of three URLS in a comment. Besides that I use the http://BL addon (more info about this addon) and I have installed Defensio addon for Pixelpost.

So now we know the configuration let’s show some stats from the last 743 days shall we?

It seems my photoblog received a total of 31897 comments (43 comments per day). A total of 2457 comments actually made it through the defensive lines of both Pixelpost and the http://BL addon. Defensio managed to catch 2256 comments as SPAM, so this leaves out 201 comments. It turns out that 70 of these comments were SPAM, but these slipped through before the Defensio addon was installed (The Defensio addon was installed a few months after the initialization of the SPAMlog).

These are the numbers, but what happened to the initial 31897 – 2457 = 29440 comments which were blocked by both Pixelpost and the http://BL addon?

Let me break that number down: Pixelpost internal measures took care of 17194 of these comments while the http://BL addon took care of the other 12246 comments. Basically this says that the measures in Pixelpost are capable of catching at least 50% of all SPAM comments.

So what is the internal Pixelpost method that stops most of the SPAM?

• 887 comments contained words listed in ban or moderation list
• 10379 comments used an incorrect token
• 2 commenters waited too long before posting (30 minutes)
• 156 comments were posted to rapidly in succession (SPAM flood)
• 5403 comments contained too many URLs (3)
• 114 comments contained an URL on the blacklist
• 210 comments were not allowed (commenting disabled)
• 41 commenters used an invalid e-mail address

As can be seen the token protection is responsible for 60% of the SPAM stopped by Pixelpost own defensive measures. Feel free to comment on my analysis.

Finally the (long awaited) update for the Defensio addon for Pixelpost is here. As mentioned in this blogpost earlier it contains the fix that slowed down a lot of photoblogs earlier this month.

Thanks to Carl Mercier of the Defensio team I was able to add one more bugfix. Carl was looking at the logs over at Defensio and noticed that sometimes the article-date is not passed to the service. The date is a required parameter and from the very first version I made sure the addon passed it along with the other variables.

This was kind of odd since it was clearly a Pixelpost blog and the date was not passed. I have to say I was stumped on this one, but Carl managed to figure it out. The Defensio addon allows comments to be posted on images that don’t exists yet. Once we (Carl) identified the problem a fix was a matter of a few lines. If the article-date is not in the database, the Defensio addon will throw a nice 404 error.

So there you have it, the best just got better!

Since yesterday Defensio reported they were having technical difficulties. About the same time some of the forum members over at Pixelpost were complaining about slow loading times of their site. John Hess identified the Defensio addon as the cause of this slow loading.

As the author of the Defensio plugin I’ve conducted a small investigation myself. It didn’t take much time to connect the problem with the downtime of the Defensio Service. However, finding the cause of the problem took some more time.

The majority of the code is only executed when needed. So basically Defensio is only activated when a comment is called. However, I managed to identify the source of the problem. The problem lies within the code for the widget.

That particular piece of code has a cache of two hours. If the stats are older than two hours it will try to pull down new stats and store it in the database. However, if the service is down for more than two hours it will repeatedly try to fetch the stats (failing each time). The addon will do this on every call to the page, slowing down the loading of the page in question.

The good news is I’ll fix it asap, but you can easily fix it yourself (assuming you have the latest version of the Defensio addon):

Open up admin_defensio.php and look at line 94 (or about):


// widget support
$defensio_widget = defensio_counter($defensio_conf);
$tpl = ereg_replace("", $defensio_widget, $tpl);


Change this code to:


// widget support
//$defensio_widget = defensio_counter($defensio_conf);
//$tpl = ereg_replace("", $defensio_widget, $tpl);

This will disable the widget code while storing unprocessed comments in the Defensio queue (so your spam won’t get through).

Look for the official patch in a few days (as I am totally swamped this weekend).

UPDATE:the problems with the Defensio service have been solved. All should be working again.

Today I’ve released the 1.2.0 version for the Pixelpost Defensio addon. You can download the new version at the Pixelpost Extend and at Defensio.

Changelog:
Version 1.2 (14 December 2007)

• Added information about the learning status to the stats overview
• Fixed a small bug with an if statement (use == to test and not =… >_< )
• Unprocessed comments are placed in the Defensio queue and displayed with a blue color. These comments can be reprocessed by clicking the appropriate link.
• Cleanup of the code (Removed some testing code as well)
• Replaced the signup url with the correct one

Please update your addon to use the new features.