This post no longer reflects the state of the art. See here .
CAPTCHAs – “Completely Automated Public Turing test to tell Computers and Humans Apart”s – are known as the foremost means to stop registrations by SPAM programs, so-called “Bots”. In phpBB, a visual confirmation CAPTCHA is used.
The key here is the “Completely Automated” part, meaning that the software – phpBB – creates the question and the correct answer without interaction by a user. This has the drawback that computers are usually able to find the answer as well, given time to adjust. This is an active field in research. In the end it is an arms race. A new CAPTCHA will usually buy a few months of peace, before the major Bot vendors adjust their products.
This article is about presenting some reasons behind our CAPTCHAs; it is not intended to be a case for or against CAPTCHAs in general or particular. It is not about other means to combat SPAM, but only about CAPTCHAs.
CAPTCHAs first became part of the phpBB package with version 2.0.10, when the need for some means to stop registration of Bot users became obvious. (Actually it was bundled from 2.0.5 onwards, but not installed) For that purpose, the well known CAPTCHA from the 2.1 branch (which later became 3.0) was backported to 2.0. We all know that the CAPTCHA was very simple to beat, but it did the job at the time.
However, as the development of what was later to become 3.0 proceeded, it became clear that it would not do to for a future version and so Robert “Xore” Hetzler took the task of developing an unique new set of CAPTCHAs for the development version.
The concept was having a powerful plugin system to allow a vast array of different CAPTCHAs, with a large number of configuration options. However, as development picked up again for the beta versions, it was decided that the system was too ambitious for the 3.0 line and should be pushed back.
In its stead, a Free Type based 3rd party CAPTCHA was bundled with the Beta versions. However, NeoThermic soon found out that it was broken by default.
… that was too easy.
As a result, the CAPTCHA code by Xore was resurrected and surveyed for CAPTCHA implementations that were compatible with subsilver2 and prosilver, while being unique enough to keep Bots away. Two such algorithms were developed to work with the downsized CAPTCHA code, one of which was finally selected to become the default CAPTCHA of Olympus.
We selected it, because it was very unlike existing implementations; naturally we are and always were aware of ways to break it – but that’s a different story. To this day, the bundled implementation works reasonably well. We are always working on possible replacements, should that change.
The major issue with a CAPTCHA bundles with a software like phpBB is that it has to satisfy a very restrictive set of requirements.
The first is very important. Any approach that requires users to register to a service like recaptcha or to set up questions and answers is not acceptable. Users can switch to such a solution later on, but for the initial setup that would be very bad indeed.
2. is often overlooked, but very important nonetheless: the CAPTCHA has to work regardless of language and cultural background.
3. is rather obvious, but can be surprisingly difficult. Not everyone is able to see 3d effects in 2d images; some people are unable to identify emotions, some can’t tell red and green apart and so on. Finally, there is 4.; meaning that almost any existing solution won’t work.
A CAPTCHA just created for your site is a different story altogether. Here you can use tricks specific to your audience, like questions or images specific to the topic of you page. In fact, even very simple approaches will work as long as they are unique. Most custom CAPTCHAs would be broken in a matter of hours, should they be included in a package like phpBB. But on your unique page, a custom CAPTCHA can be an excellent defense.
The most frequently asked questions regarding CAPTCHAs are “Why don’t you use a Q&A approach?” and “What about kitten auth?”. In fact those are excellent questions. “Question & Answer” subsumes approaches where a somewhat not-trivial question is asked upon registration, with the user having to provide an answer within certain parameters. The problems with this approach are manifold. For starters, google will answer many questions, so that many good questions are not effective. Then, the program has to be able to verify that the entered solution was correct. What about typos? What about writing “two” instead of “2″? What about internationalisation? What to do once a bot has a correct answer? When doing multiple choice – how to prevent brute forcing? The list goes on. The main issue certainly is: the approach is not a CAPTCHA at all, but merely a challenge where the questions and answers are very finite. Any given set of questions and answers bundled with the software or downloadable will be known by bots instantly.
An attempt to sidestep the language issue is kitten auth. Kitten auth presents the user with a set of images, some of which satisfy a question (show kittens). This has the drawback that – like with Q&A – the number of questions is very finite. Any freely available set of images bundled can be hashed by bots, meaning that only unique picture sets will provide any barrier at all. Simple distortions are not even remotely enough to throw image identification algorithms off.
So the answer to all questions about kitten-auth and Q&A has to be: go for it if it works for you; it wouldn’t if we were to use it.
Internationalisation is almost impossible; letters are pronounced differently.
Sometimes there are attempts to use flash or java to display CAPTCHAs. This approach has an underlying problem: the java or flash program that displays the CAPTCHA runs on the client computer and thus needs to know the code it is supposed to display. Bots can just extract that seed and don’t bother with the flash/java shell around it. Also it requires the client to have the particular platform installed, which is not a good assumption either.
The current 3.0 CAPTCHA implementation is lacking plug ability, something that had to be done to get it working in finite time. However, even so it is possible to use CAPTCHA classes other than the ones bundled without having to adjust code outside the CAPTCHA class.
The following code shall provide an example for a simple Q&A CAPTCHA that can be installed simply by replacing the existing captcha_gd file. Then the user would have to provide a question by altering the language entries and a set of images satisfying the question and another set not doing so. The images have to be in the “correct” respectively “incorrect” directories.
The code is very simple and not fleshed out to the degree one would expect from a MOD or so; it is merely given as an example.
As an example, the class used with “Enter the letters underneath smiling emoticons” (which is a very bad idea, so don’t do that at home) and the 3.0 smilies.
An early prototype for the phpBB 3.1 CAPTCHA plugins can now be found in our SVN repository.
You guys have got to get support for reCAPTCHA instead of rolling your own.
Well Jeffery, we covered why supporting reCAPTCHA is not only difficult, but problematic in this blog post (look for the heading “CAPTCHA Services”).
I don’t buy it. Sure, you have to sign up for an API key, but that is part of the security of the service. Since when does phpbb care about being 100% functional completely out of the box with zero configuration?
Of course you need to install imagemagick, set up your smtp server, configure your database, etc. to unlock various features of the software, so why is it so hard to ask the user to enter an API key if they want to use recaptcha, like WordPress does for akismet? If they don’t, default back to the standard captcha.
reCaptcha is pretty much perfect. Once reCaptcha is “broken”, that doesn’t mean that the reCaptcha has been broken, it means that computers have equaled humans in terms of OCR capabilities and we will have to move on to cat analysis and stuff. The beauty of it is that it doesn’t use an algorithm to generate images, it uses text that the cutting edge OCR software failed to parse.
Basically, don’t waste time rolling your own captcha which can certainly be broken trivially, just add support for recaptcha.
Furthermore, there are obvious benefits to having one centralized captcha provider. For example, one IP can no longer hit 2000 servers at once. The centralized captcha server would deny access to it. With a phpbb generated captcha, a robot can try to break captchas on 2000 servers concurrently and the phpbb installation is none the wiser.
I understand the logic in not using something that doesn’t work out of the box like reCAPTCHA, but it would be great if phpBB could have official support for it as an option in case the administrator does decide he wants to go that route later.
CAPTCHAs are cracked and will be cracked in future, too. Or not?
So IMO it’S not really a good idea to use it.
1. Guest Posting: Generally it’s not a good idea to use it.
2. Account activation: My personal tipp: Create a required custom profile and give this field a crazy or random internal name. E.g. I’m using the first name for this. SPAM account bots don’t know what’s the name of this field or whether it exists at all. 🙂 On idea for core pbpBB: Perhaps all profile fields should use random generated internal names?
All is only my personal opinion!
– well, what if that stops working?
– what if they have to shut down for some reason?
– and as mentioned in the entry, what if the bots break reCaptcha?
So it can really only check you on one word.
I’d _really_ like to use Mollom (http://mollom.com). They do content analysis like Akismet does, but also serve image and audio CAPTCHAs.
A plugin for reCaptcha is naturally something that will turn up, but we can’t require users to subscribe to a centralized service. Many users do not want to use such services, others can’t. The default CAPTCHA thus has to be something else.
Moreover, reCaptcha does not equate computers equalling humans. The logic works the other way round: if computers can OCR like humans, then they can solve reCaptcha. But they can solve it without that step too.
The thing is: you do not need 100%; 20% correct guesses is fine for a bot. Recaptcha itself is not particularly difficult, no matter how cool the idea looks on paper. That’s why they introduced more distortion and the line to make separation harder.
Use what works for you, services are a good idea, but can’t be default.
Also, I recommend having discussions about blog posts to the forums. Comments are fine for short responses, but constructive discussion is something the forums would benefit from.
Perhaps setting up a dedicated topic in phpBB Discussion for each blog post so that people can discuss, share ideas, and so on? It seems a bit redundant to have both the bulletin board and the blog for commenting.
@Jeffrey Rosen – phpBB doesn’t require imagemagick be installed and by default, phpBB uses php’s mail() function, which alievates the need to confgigure an SMTP server.
The only thing you need to configure is the database and that, right there, already disuades users from installing phpBB, no doubt. By that I mean… lots of users still opt for flat file message boards because they don’t want to deal with the hassle of a MySQL database.
And plus, if you do it for reCaptcha, what’s next? Maybe Akismet! Maybe something else! Maybe in order to install phpBB, you’ll have to click Next 200 times because that’s how many services phpBB is asking you to enter API keys for! Why spam your little service when phpBB will do it for you?
I’ve had best luck with preventing spam registrations by creating a question that users must answer to register.
Naturally, a bot can’t answer that question, but a human can. Since I implemented this on a few forums I run, I’ve never once had any trouble with spam.
On one of my larger forums, I’ve just tweaked the default phpBB3 CAPTCHA settings with good success – but remember, no matter how good a visual CAPTCHA is, there’ll eventually be an OCR system that can solve it with a reasonable degree of accuracy.
Some sort of an audio captcha would be very good, if it were very easy to use.
Yes, easy for spammers, hard for humans.
This information will be helpful for when I get my forum running again but I see somethings I want to comment on. There seems to be audio captchas that people can get through using bots but why don’t we use hybrids. Some people have forums that use image captchas and if someone can’t understand then they can “listen to the letters” or get the audio captcha. this gives bots 2 methods of getting into your forum to spam but what if you took the questions and answers and twist them up. Image captchas ask you to identify letters from obeservation, audio captchas ask for the same but through listening. What I’ve only seen from one other person is a private custom captcha that doesn’t allow bots to register or seperates them. Some admins can learn when there is a bot only after they’ve registered, admins have the option of grouping them up and deleting them or taking away their permissions. This only goes for some of us and only with MySQL (I think). If we stick to captchas however, then when giving a captcha the question should be one unrelated to the captcha and with a random answer count. This way brutes can’t get through and bots can’t go through your database for an answer. I know from sight that this method can and has been done but I don’t know how to manually do it. The best way to protect yourself is look at yourself as a bot and see what you can do to penetrate yourself and take the right measures to become less helpful to the bot. You won’t ever be completely safe but you will get close. Recaptcha seems like a good option but can be very annoying so rather than using it you can take its scripts and apply them to your own cutom made captcha. There is always the illusion where you make it so one would have to use logic to get through which a bot can’t use. Put up a captcha which will not need an answer but will ask you for one. Bots always try to get around or will try filling the blanks so they can’t complete the registration. They will be led to an infinite random answer loop. There are few ther methods fellow c++ programmers have sought but I myself can’t justify all of them.
Very informative article, thank you for that.
By the way: on my forum, I made an extra custom profile field (which doesn’t show in the profile). It tells to enter a specified text (provided, so easy to copy-paste) and is required at registration. The field checks to be the length of the text provided, so when bots just random fill something in, the registration fails.
So far, for about 10 months, this works perfectly. No single bot has registerd yet.
Right now, there seems to be a big spike in SPAM BOT problems, vBulletin owners have been having some serious issues with the new spam bots of late. And I even noticed IPB are using “reCapture” on their own offical forum.
You like captchas don’t you? Seems like the latest marketing garbage where many believe it improves your real human visitors experience with your site.
I found it pathetic to say the least, to bloat forms with crossed-out images and mysterious active scripts and other ajax and anti-ocr methods to identify if it’s a bot or a human who submits a form.
Forms content enrichment? no thanks, forms are complex enough.
In reality you only need some extra html with the forms and some extra form processing none of which is visible to your visitors and you can efficiently protect your form pages from automatic submission.
how about recapcha addon ?
Is it possible to deny the use of captcha altogether by initiating a registration completion by email method? Is this a viable option inside of phpbb?
Most of the spam I have been receiving on my forum seems to come from spoofed email addresses and bogus identities in Arabia. My Co-Admin spends an enormous amount of time proofing the user list for fakes.
I have a blind person trying to join the forum, and she cannot because of the captcha. An audio option would be great.
It appears CAPTCHA accessibility for blind and visually impaired users has not yet been addressed by PHPBB. What does PHPBB intend to do in order to stop locking us out? If PHPBB can’t or doesn’t want to roll out an audio CAPTCHA, there are now a number of options.
I tried Captchas in their several permutations and have come to the irrevocable conclusion that this idea deserves to die.
Captchas are inconvenient and annoying to humans, but seem to present no barrier to spambots, which started flooding my phpBB 3.0 site from around the world.
I dropped the Captcha, substituted the very simplest of questions instead, thereby stumping spambots but allowing painless entry by any human.
The blog post is based on a past state and is as of 3.0.6 no longer reflecting the state of things in phoBB3.
Yes, simple questions are in there too.
Can we use the captcha and then have the user answer a question? Is there an easy way to implement this for someone who can barely inderstand “html for idiots”?, ie me?
Yes, 3.0.6 has an easy-to-use dialog to set up question without using html.
Congrats, I attempted to register to a phpbb hosted blog and was totally stumped by the captchas GD provided by your site. You succeeded in wasting a bunch of my time and locked me out. Not being a computer wizz I am unaware of the sevear problems that the spamming community is wrecking on the web. I would like to know more.