What sort of languages would be able to handle writing your own web crawler? Could PHP handle this? I'm quite good with PHP (following best practices etc). But I'd like a good reason to learn a new language if I need to.The idea is to crawl one specific website that has multiple entries, much like an RSS feed, but they don't offer that an RSS feed of the site...
Roll your own web crawler to crawl one specific website that has multiple entries
php;web crawler
Any language that has good built-in HTTP facilities for downloading and parsing (badly formed) web documents would be fine.  I'd recommend a scripting language like Python, because a webcrawler generally spends most of its time waiting on network I/O (DNS or HTTP responses), so performance is generally not much of a concern.  Python also has decent libraries for managing HTTP connections and parsing HTML.  You need a library specifically designed to handle badly formed HTML, because the web is mostly crappy HTML.  If you choose Python, you can use something like libxml2dom.