title: Screen-scraping and SQL database
body: <p>I'm thinking of starting a small project involving screen-scraping various websites, the data of which I would like to store in an SQL database on the server.</p>
  
  <p>I did this project a few years ago using C# and software called 'screen-scraper' which did most of the hard work for me, but I don't think this solution was as good as it could of been despite it working eventually so I would like to do it better this time. For reasons I can't remember now I could never run screen-scraper on the server and always had to leave another computer of my own on to do the scraping itself, then the data would be uploaded to the server from there..(maybe I was just being a moron).</p>
  
  <p>I've heard very good things about Python and Beautiful Soup for this task although I've got no previous Python experience.</p>
  
  <p>I would like ultimately everything to be able to run on the server side, so python script runs 24/7 on some server updating the SQL database with some frequency. Other code will also live on the server and be called upon by the user the manipulate the data from database etc.</p>
  
  <p>I have more experience with C based languages, but I don't mind learning some Python if it's worth it. I just want to make sure I am heading in the right direction..</p>