Running background tasks in Meteor.js
Question: This is my scenario:

    
    
    1. Scrape some data every X minutes from example.com
    2. Insert it to Mongodb database
    3. Subscribe for this data in Meteor App.
    

Because, currently I am not very good at Meteor this is what I am going to do:

    
    
    1. Write scraper script for example.com in Python or PHP.
    2. Run script every X minutes with cronjob.
    3. Insert it to Mongodb.
    

Is it possible to do it completely with Meteor without using Python or PHP?
How can I handle task that runs every X minutes?


Answer: There are Cron like systems such as [percolate:synced-
cron](https://github.com/percolatestudio/meteor-synced-cron) for Meteor.
There, you could register a job using
[Later.js](http://bunkat.github.io/later/) syntax similar to this example
taken from the percolate:synced-cron readme file:

    
    
    SyncedCron.add({
      name: 'Crunch some important numbers for the marketing department',
      schedule: function(parser) {
        // parser is a later.parse object
        return parser.text('every 2 hours');
      }, 
      job: function() {
        var numbersCrunched = CrushSomeNumbers();
        return numbersCrunched;
      }
    });
    

If you want to rely on an OS level cron job, you could just provide an HTTP
endpoint in your Meteor.js application [that you could then access through
curl at the chosen time](http://stackoverflow.com/questions/17850967/run-a-
curl-command-using-cron-jobs).

