Ask HN: Is wanting to be a VC a red flag for a startup hire? - <AUTHOR> I've noticed a fair number of people trying to work in the startup space are quite set on going into VC ("eventually"). Have you ever hired such a person? How did it go? <comment><USER2> To me, this sounds more or less like 'and then after high school I'll go to college' mental model of what it means to have a career. Working at a startup, then having a startup, then becoming a VC is how a guidance counselor might describe the career of 'entrepreneur'.

My advice, trust your gut. What matters is if it is a red-flag for _you_.

Good luck.

</comment><comment><USER3> A red flag for what danger? Sounds like they're pretty dedicated to making themselves a lot of money to achieve that goal, presumably via working for your startup. I don't see why that would be a red flag at all; rather, they're probably going to be willing to work hard to realize that goal.

<comment><AUTHOR> Ah, I guess I wasn't clear. They want to work for a VC firm.

The danger is that they aren't in it for the long haul, are just using this as a resume padder, and are looking to jump ship to a VC firm (or potentially a better resume padder) asap.

<comment><USER4> Honestly, the fact that you use the words "in it for the long haul" the way you do would be a red flag for me if I was a prospective hire.

I want to work at a place where the incentives are aligned in such a way that, if I really am some approximation of a rational economic agent, I would have no choice but to "be in it for the long haul", without having that be my intent on day one.

In other words: it is _your job_ as a company to make a prospect stay there for the long haul; it is not their job to come in with that psychological state.

And yes, I'd like to work at a VC firm eventually.

<comment><AUTHOR> I think where I disagree with you is that I don't think it's effective to "incentivise" someone to do a job that they fundamentally don't want to do.

First I want to find the right fit. Then I want to incentivise them. Is someone who wants a bit of "startup experience" before working for a VC firm the right fit?

<comment><USER4> Yea, sorry, I don't think we will be able to come to any sort of productive discussion here.

'I don't think it's effective to "incentivise" someone to do a job that they fundamentally don't want to do.'

I don't even know what this means. I don't see how anyone "fundamentally" would want to do anything to help you with your company unless you incentivize them.</comment></comment></comment></comment></comment>


Ask HN: Who are the best journalism start-ups? - <AUTHOR>

Any examples that people are using would be great. But founders who want to plug their products, that's fine too. <comment><USER2> 'Pitch Me' is a brilliant example of innovation in the publishing field!

</comment><comment><USER3> What do you mean by "journalism startups"? Does MuckRack count?

<comment><AUTHOR> Sure. I'm not a definition Nazi; you can define it how you want.

I'd say something like "any start-up focused on innovating in journalism."</comment></comment>


Cryptocurrency: Why Stratis may be the next big thing - <AUTHOR> https://email@example.com/why-stratis-may-be-the-next-big-thing-4629c0038c73 <comment><USER2> My first thought why choose C#? The CLR supports floating point. Problematic if you want deterministic execution. Which you do for a smart contract. How would you deal with the halting problem?</comment>


Introduction to MCollective slides - <AUTHOR> http://www.slideshare.net/PuppetLabs/mcollectiveintroductionsf

<comment><AUTHOR> [http://www.devco.net/archives/2013/06/14/introduction-to- mco...](http://www.devco.net/archives/2013/06/14/introduction-to-mcollective- deck.php) announces the slide deck.</comment>


The NSA must rely on a lot of open source software... - <AUTHOR> http://daniel.morrione.net/?p=213

<comment><USER3> > Why couldn't open source licenses be amended to forbid use in surveillance > devices?

Because otherwise, it would be non-free.

If you look at the Free Software definition (from [http://www.gnu.org/philosophy/free- sw.html](http://www.gnu.org/philosophy/free-sw.html) ), the first freedom is : "The freedom to run the program, for any purpose (freedom 0).". "For any purpose" is really clear, you can't forbid someone to use it in a surveillance device.

The DFSG (Debian Free Software Guidelines, [http://www.debian.org/social_contract#guidelines](http://www.debian.org/social_contract#guidelines) ) are even clearer : "No Discrimination Against Fields of Endeavor - <USER2> license must not restrict anyone from making use of the program in a specific field of endeavor. For example, it may not restrict the program from being used in a business, or from being used for genetic research.".

<comment><USER4> So in other words, you are totally free to remove the freedom others.</comment></comment>


Git: fetch and merge, don't pull - <AUTHOR> http://longair.net/blog/2009/04/16/git-fetch-and-merge/

<comment><USER2> I don't quite follow what he is trying to say.

From the manpage for git pull: git-pull - Fetch from and merge with another repository or a local branch

Git pull is identical to git fetch && git merge $remote_branch. He's saying that it prevents you from learning how branching works. So, if he were explicit, he should say 'Don't use git until you know how branching works at some level.', which is good advice. Not using git pull is a pretty sensational way to get at that nugget.

<comment><USER3> His issues seem to be mostly:

* If you `git pull`, you don't get the chance to diff between your local branch and what you just fetched and check if you really want to do that merge, using `git fetch` and `git merge` makes it more likely you'll do that check

* Getting used to `git fetch` means if you need to diverge from the straight `git fetch; git merge` workflow (because you want to rebase, or you have a high number of active topic branches floating around) you're already halfway there.

</comment></comment><comment><USER4> Very nice description of a number of use cases. I was just a bit annoyed that the title was described in a very hand-waving way (do it because it's more useful sometimes). With so many other examples I really expected a detailed description of some situation where fetch/pull makes a huge difference.

</comment><comment><USER5> No mention of "git pull --rebase"?

<comment><USER6> I like it so much, I even have `autosetuprebase = remote` in my gitconfig.

<comment><USER7> I don't think that's a good idea in general. It might work for your particular use case, but if you're doing work on a branch you regularly update from a remote, then you will also need to test every time you do a pull. you might introduce new bugs because the code you rebased does not necessarily make sense on top of the new base, and you have no easy way to reset to the old base either. Merges can be undone easily but rebases are much trickier.

I think the best approach is to code on a separate branch while keeping the base unchanged and do the rebase (or merge) when the code is complete. That way, you at least have a branch of code that you know is working, and when you merge, you immediately know that any issues are caused by something that changed between the old and the new base. And you still can do mid-development rebases if there is a need for them.

<comment><USER8> I think you're missing the point. Rebasing work you haven't pushed _disentangles_ your work from the remote branch.

Here are some advantages:

1\. Work you happened to do after a merge doesn't spuriously appear to depend on that merge.

2\. You can do conflict resolution incrementally during the rebasing process (commit-by-commit instead resolving a merge conflict all at once).

3\. Linearizing more of your history improves the precision of git bisect. You're much more likely to have the result of a bisect point to a simple commit rather than a complex merge when there are fewer merges.

The two major disadvantages I see are:

1\. Pulling and rebasing is more complex than pulling and merging (more steps and more decisions).

2\. You have to be careful not to rebase a branch you have "published".

<comment><USER7> This reply is a bit late but...

You're not supposed to do work on the same feature after a merge. I'm not advocating pulling master (or whatever the base branch is) often and merging whenever, so that you end up with loads of pointless merges. Once your feature is done, you merge once. This makes point 3 pretty much moot, since bisect handles merges just fine.

Sometimes you need to update your code on top of a new base because of some important bugfix or something, and then it's fine to rebase, but I don't think it should be the default.

I'll accept point 2, though I think git's merging is good enough that it's not noticeable in most cases. If you rebase you also lose information about the conflicts which may be undesirable. Someone might want to review your merge to see if you have solved the conflicts correctly.

</comment></comment></comment></comment></comment><comment><USER9> Not sure if it warrants an article but I've never found any use for git-pull either. When I want to fetch and merge/rebase, I can _do just that_ ; I never figured why would I want to use git-pull for it. But on the other hand, I've only used git for about five years and I've only learned the most important low-level commands yet.

</comment><comment><USER10> This is one of the things that Mercurial got right. (In Mercurial, you pull and then merge separately, usually.)

</comment><comment><USER11> git is deep. And you can do the same thing using different steps and different number of steps. I think this post provided a good tip on how to go about merging remote changes into your local. If one finds that git pull requires a lot of cleanup activity, git fetch and git merge is a good way to go..

</comment><comment><USER12> Interesting to learn. In Mercurial (which is what I use primarily) pull does not merge unless you ask it to. Is there a list anywhere of gotchas in git for people coming from other DVCS? (i.e. things like commands with the same name that do subtly different things, such as this)

<comment><USER3> > In Mercurial (which is what I use primarily) pull does not merge unless you > ask it to.

Actually, in Mercurial `pull` does not _ever_ merge. It can _update_ if you ask it to (with `-u`), but it won't merge.

`hg fetch` (standard extension) can merge (it's basically "pull and update, if update fails merge"), but it might go the way of the dodo pretty soon[0]

[0] [http://www.selenic.com/pipermail/mercurial- devel/2011-August...](http://www.selenic.com/pipermail/mercurial- devel/2011-August/033764.html)

<comment><USER12> Mea culpa, absolutely correct.

</comment></comment></comment><comment><USER13> The article mentions:

        $ git fetch
        $ git checkout master
        $ git merge origin/master

On the third (last) line of code above, is there a more short-hand way to say "merge the remote tracked branch related to this current branch"?

<comment><USER14> git pull?

</comment></comment><comment><USER15> Git pull is nice, but it _is_ good to know what's going on underneath the hood.</comment>


We Don't Run Cron Jobs (2016) - <AUTHOR> https://engblog.nextdoor.com/we-don-t-run-cron-jobs-at-nextdoor-6f7f9cc62040 <comment><USER2> Cron works great when you don't need to guarantee execution, e.g., if a server goes down. Unfortunately, all the alternatives are pretty heavyweight, e.g., Jenkins, Azkaban, Airflow. I've been working a job scheduler that strives to work like a distributed cron. It works with very little code, because it leans heavily on Postgres (for distributed locking, parsing time interval expressions, configuration storage, log storage) and PostgREST (for the HTTP API). The application binary (~100 lines of haskell), polls for new jobs, then checks out and execute tasks. The code is here if you're interested:

[https://github.com/finix-payments/jobs](https://github.com/finix- payments/jobs)

It compiles to machine code, so deploying the binary is easy. That said, I'd like to add some tooling to simplify deploying and configuring Postgres and PostgREST.

<comment><USER3> Jenkins is "heavyweight" but Postgres isn't?

<comment><USER4> As someone who has administrated both, I'd rather manage 10 Postgres instances than one Jenkins box. No question.

Edit: I should expound. Jenkins seems like it has a lot of clunky moving parts. It all works, and I'd rather use it than anything else, but it's kind of like IKEA furniture: you use it because you have to, not necessarily because you want to.

It's also incredibly difficult to automate. I can configure Postgres with a config file or two and easily use Ansible to get the exact same instance every time. Jenkins has to be dragged into Automation Alley kicking and screaming. I partially blame this on the fact that Jenkins has nontrivial amounts of configuration that's done via GUI. I approach a long-running Jenkins instance with the same fear and dread I approach a Windows box that hasn't been restarted in six months. I.e. the box is now a snowflake and trying to make it reproducible and automated is going to be a bad time.

I could go on, but as a devops critter, Postgres wins every time.

<comment><USER5> Jenkins is probably 100 times easier to deploy, configure, operate, and scale than Postgres+whatever, which is a horrifying statement, but still true. If what you have to do is schedule and run arbitrary jobs on any kind of machine, it's no contest. They aren't even related. Postgres is a relational database, and Jenkins is a single Java process that stores flat files on local disk, connects to remote nodes with SSH, and has a thousand plugins.

It's like comparing a missile with an airplane. One gets where it needs to go faster and more efficiently, and the other one transports people.

<comment><USER2> Here's a fairly objective metric for comparing the complexity of deploying Jenkins vs Postgres:

[https://github.com/geerlingguy/ansible-role- jenkins](https://github.com/geerlingguy/ansible-role-jenkins)

[https://github.com/geerlingguy/ansible-role- postgresql](https://github.com/geerlingguy/ansible-role-postgresql)

Two ansible roles by the same author, supporting both Centos and Ubuntu. Not hugely different in complexity IMO. Installing Postgres on FreeBSD, though, is little more than

        pkg install postgresql10-server
        sysrc postgresql_enable="YES"
        service postgresql initdb
        service postgresql start

<comment><USER5> I get that they install differently, but that's not the point. The point is that they do very different things, in different ways. Postgres is not a Jenkins replacement. Postgres is storage and querying. Its equivalent on Jenkins is XML files.

</comment></comment></comment></comment></comment></comment><comment><USER6> The last company I worked at used Jenkins for all cron jobs. It has great reporting, supports complex jobs, has a million plugins. It worked really great.

<comment><USER7> Interesting, isn't Jenkins mostly used for internal builds? I have a hard time imagining using it for things like marketing email cron jobs.

<comment><USER8> The beauty of jenkins is that you can use it for pretty much anything.

<comment><USER9> I tend to feel the opposite about things you can use for pretty much anything. I much prefer tools that are good for a single thing. I really disliked maintaining a Jenkins instance awhile back; maybe this "jack of all trades" ethos is part of the reason I felt that way.

<comment><USER10> I tend to prefer simplified stack, especially for tools that don't need to be touched very often.

<comment><USER9> I can't tell: are you saying that using Jenkins results in the simplified stack you prefer? If so, I don't agree; I think Jenkins is a very un-simple way to run from jobs.

<comment><USER10> When I say simplified stack I mean fewer tools that must be learned and managed by people. Jenkins simplifies in that way.

<comment><USER9> Interesting. I still don't see it that way. It seems less like a single tool when used in this way than multiple tools running alongside one another. Different strokes for different folks I suppose!

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> > Here is an example of a typical oncall experience: 1) get paged with the > command line of the failed job; 2) ssh into the scheduler machine; 3) copy & > paste the command line to rerun the failed job

No doubt this works if it's an established procedure, but if I was approaching a system I wasn't familiar with I would never do (3) because the environment can differ wildly between crond and a login shell. It is safer to edit the cron schedule and duplicate the entry with a time set for few minutes in the future. (And clean that up afterwards).

<comment><USER12> You could even write a cron job to clean up the crontab

<comment><USER13> Or you could use at(1) or batch(1) to invoke the job once. That seems significantly easier and less error prone than trying to clean up crontab.

<comment><USER14> at(1) is great, but you'll still have the problem of a different environment to cron (it copies most of your environment variables and working directory).

</comment></comment></comment></comment><comment><USER15> If you're already using cron like millions of us, here are a couple things that can help you:

[https://crontab.guru](https://crontab.guru)

[https://cronitor.io/docs/cron-troubleshooting- guide](https://cronitor.io/docs/cron-troubleshooting-guide)

</comment><comment><USER16> I worked at a company which wrote their own scheduler and it was fraught with bugs. Dealing with time and date is HARD. Really, really hard. Your custom scheduler will break and at a worst possible time.

If Cron doesn't work, get an open source or commercial solution. And who cares what tech the scheduler is written in? Scheduler's job is to provide run your programs and provide API and a nice GUI if you desire.

<comment><USER17> Yeah exactly. I don't understand why they wanted the scheduler to be written in Python, since the scheduler should be decoupled from the jobs they are running anyway.

<comment><USER18> If your jobs are written in Python, there is nothing wrong with a Python-based scheduler. It can actually be quite convenient.

</comment></comment></comment><comment><USER19> I think they have a task worker system built back in 2014[1], so they need to have something custom working with it as well. Back then I think they really didn't have many options, but if they were to do it again now, I think either AWS Lambda or AWS Batch will serve this type of scheduled job cases very well.

[1]: [https://engblog.nextdoor.com/nextdoor-taskworker-simple- effi...](https://engblog.nextdoor.com/nextdoor-taskworker-simple-efficient- amp-scalable-ac4f7886957b)

</comment><comment><USER20> NMS engineer at an enterprise telecom here. At my company, we've been switching over to Jenkins for job scheduling. Most of what used to be cronjobs have been fully Dockerized, and now we have Jenkins run periodic "builds" via pipelines. The pipelines themselves just run a docker image.

The single biggest advantage this has gotten us is centralized logging. I can check on the console output of any cronjob just by going to Jenkins and clicking on the job.

Moving to Jenkins to cron wasn't my idea, but the implementation is mine. I've built a few base Docker images as bases. One is just the standard Python 3.6 docker image. Another is the CentOS image equipped with Python 3.6 and Oracle RPMs for jobs that need database access. Another is the aforementioned image plus a number of Perl dependencies for jobs that need to call into our legacy Perl scripts.

For many scripts, I can use identical Dockerfiles. I just copy the directory containing the script, requirements.txt, Dockerfile, and Jenkinsfile, then I change out the script, edit the Jenkinsfile to reference the new script's name, and make any needed changes to the requirements.txt.

</comment><comment><USER21> I've had good experiences using celery-beat to replace crond. It lets you use all the good parts of celery without much work. [http://docs.celeryproject.org/en/latest/userguide/periodic-t...](http://docs.celeryproject.org/en/latest/userguide/periodic- tasks.html)

<comment><USER22> Ditto, especially in combination with Python/Django, as used by Nextdoor. Ironically, they had already removed Celery from their stack a few years prior. [https://engblog.nextdoor.com/nextdoor-taskworker-simple- effi...](https://engblog.nextdoor.com/nextdoor-taskworker-simple-efficient- amp-scalable-ac4f7886957b)

</comment></comment><comment><USER23> Since nobody mentioned Anacron, I will. [https://en.wikipedia.org/wiki/Anacron](https://en.wikipedia.org/wiki/Anacron) Deals with the server possibly being down for a period.

</comment><comment><USER24> That was a difficult read for me, the blog post starts out with the four main problems with cron (their use wasn't scalable, editing the text file was hard, running their jobs are complex, and they didn't have any telemetry.)

That's great, what does that have to do with cron?

As a result what I read was:

"We don't understand what cron does, nor do we understand how job scheduling is supposed to work, and we don't understand how to write 'service' based applications, so somebody said 'Just use cron' and we did some stuff and it didn't work how we liked, and we still haven't figured out really what is going on with schedulers so we wrote our own thing which works for us but we don't have any idea why something as broken as cron has persisted as the way to do something for longer than any of us has been alive."

I'm not sure that is the message they wanted to send. So lets look at their problems and their solution for a minute and figure out what is really going on here.

First problem was 'scalability'. Which is described in the blog post as "cron jobs pushed the machine to its limit" and their scalability solution was to write a program that put a layer between the starting of the jobs and the jobs themselves (sends messages to SQS) and they used a new scheduler (AP scheduler to implement the core scheduler).

So what was the real win here? (since they have recreated cron :-)) The win is that instead of forking and execing as cron does allowing things like standard in and what not to be connected to the process, their version of cron sends a message to another system to actually start jobs. Guess what, if they wrote a bit of python code that all it did was send a message to SQS and exit that would run pretty simply. If they did it in C or C++ so they weren't loading an entire interpreter and its run time everytime it would be lighting fast and add no load to the "cron server", this is basically being unaware of how cron works so not knowing what would be the best way to use it.

Their second beef was that cron tabs are hard to edit reliably. Their solution was to write a giant CGI script in a web server that would read in the data structure used by their scheduler for jobs, show it as a web page, let people make changes to it, and then re-write the updated data structure to the scheduler. Guess what, the cron tab is just the data structure for cron in a text form so you can edit it by hand if necessary. Or you can use crontab -e which does syntax checking, or you could even write a giant CGI script that would read in the cron file, display it nicely on a web page, and then re- write as syntactically correct cron tab when it was done.

Problem three was that their jobs were complex and _failed often_. This forced their poor opsen to log in, cut and paste a complex command line and restart the job. The real problem there is _jobs are failing_ which is going to require someone to figure out why they failed. If you don't give a crap about why they failed the standard idiom is a program that forks the child to do the thing you want done and if you catch a signal from it that it has died you fork it again[1]. But really what is important here is that you have a configuration file _under source code control_ that contains the default parameters for the jobs you are running so that starting them is just typing in the jobname if you need to restart or maybe overriding a parameter like a disk that is too full if that is why it failed. Again, nothing to do with cron and everything to do with writing code that runs on servers.

And finally their is no telemetry, no way to tell what is going on. Except that UNIX and Linux have like a zillion ways to get telemetry out, the original one is syslog, where jobs can send messages that get collected over the network even, of what they are up to, how they are feeling and what, if anything, is going wrong. There are even different levels like INFO, or FATAL which tell you which ones are important. Another tried and true technique is to dump a death rattle into /tmp for later collection by a mortician process (also scheduled by cron).

At the end of the day, I can't see how _cron_ had anything to do with their problems, their inability to understand the problem they were trying to solve in a general way which would have allowed them to see many solutions, both with cron or with other tools that solve similar problems, would have saved them from re-inventing the wheel yet again, giving them the opportunity to experience the bugs that those other systems have fixed given their millions (if not billions) of hours of collective run time.

[1] Yes this is lame and you get jobs that just loop forever restarting again and again which is why the _real_ problem is the failing not the forking.

<comment><USER25> This rant seems out of character, but personally I appreciate it. People are always suggesting getting rid of cron, but I have always liked and trusted it. I prefer using tools that are old, battle-tested, and standard, but I do try to appreciate the advantages of new things. Cron seems to be a favorite target of NIH, for as long as I remember but especially in these days of "serverless", so it's easy to second-guess my appreciation for it. Thanks for clarifying where their problems really were. There seems to be a common temptation to think a new tool will solve your problems, when really many are in the irreducible specificity of your own code or systems.

EDIT: Oh by the way, Ruby folks struggling with cron might appreciate this: [https://github.com/pjungwir/cron2english/](https://github.com/pjungwir/cron2english/)

<comment><USER24> Cron2English looks very helpful for folks who struggle to read lines in a crontab.

I recognize that it is a sore spot for me when people re-invent the wheel when it seems clear they didn't have to and even clearer that the energy spent re- inventing the wheel would have been better (in terms of using the wheel) spent learning why the wheel is the way it is.

It is sort of the Chesterson's Fence of computer science. Don't tell me its wrong, tell me how it is right for what it does and where it falls short for what you want it to do. Cron gets a bad rap here, as did sendmail for that matter. When taken to extremes you replace working systems like init with systems that are borked like systemd. No doubt cron is on the list of things to be absorbed by the systemd borg at some point. Yes, it makes me grumpy.

<comment><USER26> > cron ... absorbed by the systemd borg

This has already happened, right? [https://wiki.archlinux.org/index.php/Systemd/Timers](https://wiki.archlinux.org/index.php/Systemd/Timers)

Not a systemd fan, myself.

<comment><USER27> IMO cron is one of the parts that a good process management system should solve. With systemd timers I can do stuff like require other processes to run for a timer to run, activate a process either on a timer or on a socket, specify that it runs in a private chroot/netns/cgroup. These are all common things that I might usually want a process or service to do, and putting timed, manual, socket and path invocation of a service in the process management system makes sense to me.

I understand some of the systemd criticism, but I don't at all understand why you want something else to manage your processes than the process management system (which is basically always the init system)?

<comment><USER28> > I understand some of the systemd criticism, but I don't at all understand > why you want something else to manage your processes than the process > management system (which is basically always the init system)?

Perhaps you don't understand one of the fundamental, philosophical criticisms, which is that it violates the Unix tenet of "do one thing and one thing well".

Timing, and scheduling are difficult and (arguably, of course), deserve their own, separate utility.

Cron isn't a process manager, any more than and interactive shell [1]. It just starts processes (and passes output to a local email client, though that's, understandably, brittle).

I wouldn't object to moving the process-starting functionality out of cron and leaving it with only the ability to tell init to "start job named XYZ", but I do object to just integrating its core competency of scheduling into that init.

[1] Arguably less than, considering modern shells' job control and signal facilities

</comment></comment></comment></comment></comment></comment><comment><USER29> We schedule jobs on our Elixir cluster. Nice to not need anything on top of what you're already running

</comment><comment><USER30> > Second, editing the plain text crontab is error prone

Doesn't every crontab in existence have a comment line giving the order of the time columns? I know I always rely on such.

Although the article didn't touch on it, this point reminded me of yesterday's discussion about manpages and command-line options. I think it's still the case today that `cron -e` is _the_ way to edit the crontab whilst `cron -r` (the key _right nextdoor_ in case this part needed stressing!) removes it altogether.

<comment><USER14> I got tired of writing the time specs manually, so now I keep this script in my PATH:

        #!/bin/sh
        # print a crontab entry for 2 minutes into the future

        date -d "+2 minutes" +"%M %H %d %m * /path/to/command"

</comment></comment><comment><USER31> Using Elixir we just scheduled work using a simple genserver, the initial naive version has no tracking of jobs done/failed/etc, but you can append those easily since they are just language constructs.

</comment><comment><USER32> Kubernetes CronJobs resource would be my go to for this.

<comment><USER33> Good answer in 2018 but in 2016 the development of ScheduledJob (now CronJob) had just begun around November/December of that year so it was not an option for these guys.

</comment></comment><comment><USER34> The CPU comparison are kinda funny to see. At first glance the low CPU usage looks good but to me that's wasted resources. Good to see a more efficient system though. Hopefully those instances get allocated to different problem sets.

</comment><comment><USER35> I use Apache Airflow with BashOperator for tons of stuff like this, simple web UI for logs/retries, supports dependencies between jobs and when tasks get more complex it's Python and it supports extensions.

<comment><USER36> Apache Airflow is a great way to future proof your cron jobs. Existing cron jobs can be easily migrated and with that you'll get access to built in logging, distributed execution, connection management, web ui for simple monitoring and task retries and more.

</comment></comment><comment><USER37> A little tangential, but I recently created a small tool called "tsk" (pronounced same as task)

[https://www.tsk.io](https://www.tsk.io)

I'm calling it a "speed-dial for your APIs". I used to have a VPS that would run out of memory, so easiest way to resolve this was to restart the server. But even that was annoying, so I created a button that would call the API to restart the box. Now I just have to click the button whenever I want to reboot.

Something similar to CRON, I'm currently building a feature in tsk to schedule the tasks.

<comment><USER38> My scraping VPS keeps running out of memory too and I restart it every couple of days.

> so I created a button that would call the API to restart the box

Been thinking about building a 'button' that does this as well. Will check out TSK to see how well it addresses this. Sound idea for sure.

<comment><USER37> Thanks! Would love to hear your feedback on it!

</comment></comment></comment><comment><USER39> We switched from Heroku Scheduler (very limited cron) to a system called Sidekiq Cron[1] (if we used Sidekiq Enterprise we would use the built-in scheduler). All Sidekiq Cron does is drop a job into the queue on a given interval. We also use HireFire to auto-scale our workers as necessary to keep things running.

[1]: [https://github.com/ondrejbartas/sidekiq- cron](https://github.com/ondrejbartas/sidekiq-cron)

[2]: [https://hirefire.io](https://hirefire.io)

<comment><USER40> We use Sidekiq's scheduled jobs[1] to replace the cron dependency in our codebase. Did you try it before opting for Sidekiq Cron?

[1]: [https://github.com/mperham/sidekiq/wiki/Scheduled- Jobs](https://github.com/mperham/sidekiq/wiki/Scheduled-Jobs)

</comment></comment><comment><USER41> Where I work we have a similar product where we run all scheduled tasks on our Mesos cluster. Same idea as a uni cCron. You have a task that is executed every N minutes/hours/days, it runs on a box, and does its thing.

It doesn't replace every cron job, but it is distributed, fault tolerant, and only breaks when the hadoop cluster backs up. The product mentioned here seems like a good solution for a team that needs to execute linux crons without much overhead.

</comment><comment><USER42> A bit of a side-topic but, has anyone tried APS (Advanced Python Scheduler - [https://apscheduler.readthedocs.io/en/latest/](https://apscheduler.readthedocs.io/en/latest/)) in production?

I've been evaluating it as it seems to provide fault-tolerance, but IMO the documentation could be much better with more examples (e.g. mixing different triggers, configs,.etc)

Can anyone comment on it?

</comment><comment><USER43> Job Scheduling.. We're using jenkins heavily at one of my clients sites for this (it's already used for builds, so scheduling jobs didn't seem that far afield..) The thing I'm missing, and would love to know if exists, is a calendar like view of all of the scheduled jobs, ala google calendar. Surely a product must exist that provides this view?

<comment><USER44> [https://github.com/jenkinsci/calendar-view- plugin](https://github.com/jenkinsci/calendar-view-plugin)

</comment></comment><comment><USER45> Here it's Quartz and its JDBC persistence baked into a Dropwizard application with a custom resource implementing job and trigger CRUD ops, and a Spring- based JobFactory for dependency injection stuff. Quartz has a few funny behaviors that could be better, but on the whole it's been working out nicely for scheduling across a multi-node stateless cluster.

</comment><comment><USER46> For JVM (Java, Scala, Clojure, Groovy, etc), Quartz with JDBC (backed by DB instead of in-memory) works best for us.

</comment><comment><USER47> It's worth noting that SQS promises "at least once" delivery. In practice, this means some messages will be delivered multiple times.

After every job is successfully finished, it should be noted in your db. Every time a worker starts processing a job, it should check the db to see if this job has already been run.

</comment><comment><USER48> The old fogeys among us would point out that this is an incomplete implementation of the batch queues that VMS has had for 3 or 4 decades. The new fogeys would point out that 60% of this functionality (the hard parts) is present in PowerShell and all the rest is a boring CRUD app.

</comment><comment><USER49> Based on the CPU usage graph it looks like there's a lot more opportunity to downsize the scheduler hardware. I'd be curious to see how their TaskWorker cluster absorbs spikes in load when big cron jobs start.

</comment><comment><USER50> Or they could have just used Hashicorp's Nomad....

[https://www.nomadproject.io/](https://www.nomadproject.io/)

It's dead simple to set up and use.

<comment><USER51> Nomad was started about 39 months ago. The post in question is from 32 months ago when they wrote about a thing they had already built.

"We've been using Nextdoor Scheduler for over 18 months and we are extremely happy with it."

That means that what they built pre-dates Nomad by at least 11 months.

</comment></comment><comment><USER52> They basically built sidekiq enterprise. Way to go.

</comment><comment><USER53> I'm the author of this blog post and the original developer of this scheduler system.

Glad to see a lot of interesting and insightful comments in this thread!

Some contexts here:

0\. Like any piece of software, this scheduler system is not perfect for every company -- legacy code, # of engineers, skillsets of existing employees, engineering culture, tech stack, business,...

1\. Every engineer in Nextdoor needs to do on-call rotation -- 50+ engineers back in 2014 (100+ now?), when the scheduler was built. It's important to run a system that all 50+ engineers have confidence to debug on a Friday night if things go south. There are some blackbox-type alternatives of cron, which may be great for a small team of backend experts to operate, but may not be a good fit for 50~100 engineers with very diverse skillsets & backend experience. But why every engineer needs to be on call? Well, that may deserve a new thread of discussions :)

2\. In 2014, there were ~200 production jobs with very different computing characteristics (450+ now). Jobs are owned by different teams and different people with different expertise. Jobs are frequently (every few weeks?) added, removed, and updated. Outages most likely happen during code deployment. It's important to run a system that works well during deployment (and rollback), e.g., always run the RIGHT version of code for hundreds of jobs.

3\. The core scheduler system was pretty tiny, which could be easily understood by any engineer in the company. I remembered it took me a few days (probably less than a week) to finish most of the code. The hard part was productionization, e.g., carefully migrating 200 production jobs to the new system, logging, monitoring/alerting at job level, deployment & rollback process, oncall-related things (documentation/training)… With this simple system, we could easily enumerate various failure situations, so oncall engineers have confidence how to respond.

4\. I'm not with Nextdoor any more. But turns out this scheduler system is still working well now: [https://github.com/Nextdoor/ndscheduler/issues/33#issuecomme...](https://github.com/Nextdoor/ndscheduler/issues/33#issuecomment-387954933) I think the ROI is pretty good -- a 3-man-month project (from design to run all 200+ jobs on the new system) => running 4 years and counting & easy oncall situation for 50~100+ engineers.

5\. Why not Jenkins or other open source alternatives? We investigated a bunch of alternatives. Jenkins is great. Back in 2014, Nextdoor used Jenkins for CI (not sure if it's still true now). We ruled out Jenkins (and Rundeck or the like) for operational reasons, e.g., challenges to integrate with existing code deployment process, operational complexity for 50+ engineers with different backgrounds/expertise... Open source alternatives? Well, in 2014, we couldn't find a good Python-based project. It's important to limit the number of languages/external technologies in the tech stack.

</comment><comment><USER54> cloudwatch rules ??

<comment><USER55> Bingo! Those work great paired with a AWS Lambda function

</comment></comment><comment><USER56> Have they heard of crontab -e? It checks that your cronjobs have valid syntax. There are a crazy number of edge cases related to time. Dealing with time sucks, let the smart people who battled tested and built cron deal with it.

<comment><USER57> What does that have to do with the problems that they dealt with? (randomly failing jobs, resource management, monitoring, etc)

<comment><USER58> I think the comment is in reference to the second problem listed, "editing the plaintext file is difficult."

</comment></comment></comment><comment><USER59> So, if not cron, what is better?

</comment><comment><USER60> Aurora? Nomad?

</comment><comment><USER61> We use Jenkins when we need something more complicated than cron can provide.

</comment><comment><USER62> jenkins?

</comment><comment><USER63> They really ought just have learnt to use Cron properly

</comment><comment><USER62> I was expecting everything is 'reactive'( i.e react to events in real time) so there is no need for cron jobs.</comment>


European Parliament Member: Get Your Money Out While You Can [video] - <AUTHOR> https://www.youtube.com/watch?v=a0OCcjz4lfU&t=184

<comment><AUTHOR> Actual "get your money out" quote around 2:02.

Note: I know "world news" is usually not threshold, but the Cypriot bailout seems like a fresh rubicon for the EU (albeit one of several recently).</comment>


Minneapolis ends single family zoning - <AUTHOR> https://slate.com/business/2018/12/minneapolis-single-family-zoning-housing-racism.html <comment><USER2> While I certainly agree that single family zoning has led to awful commutes and an untenable housing shortage, and while I approve of eliminating single family housing restrictions, I politely assert that protecting neighborhoods from incompatible higher density development was a legitimate motivation in addition to the motivation to exclude "undesirables". Higher density housing can destroy a neighborhood's quality of life. What is needed is thoughtful development not unlicensed development in the name of social justice. This is not an easy thing to accomplish. One hopes the authorities can set aside divisive rhetoric and deal thoughtfully with the conflicting aspects of the problem.

<comment><USER3> This isn't the right solution for every place, but it's the right solution for Minneapolis. There's no shortage of land and quasi-affordable housing in the suburbs / exurbs - people who want that can move out and get the house they want for the money they'll collect for their more-valuable Minneapolis house.

But Minneapolis has been very dense for awhile and other bordering cities have done an excellent job at managing density and quietness - for example, the two adjacent cities Edina and Colombia Heights have been offering peace and quiet to Minneapolis residents since the freeways went in.

<comment><USER4> FYI it looks like you've been shadow banned since your first posts in the thread "A female engineer's opinion on why there are fewer...".

Your posts since then have been substantive so I vouched for this one. You should contact the mods and ask them to reconsider the ban. It looks like others have vouched specific comments multiple times in the past and the mods are pretty reasonable.

<comment><USER5> Does the "show dead" setting override the shadow ban so one can view dead comments?

Interesting you can vouch for comments at ~4k karma and I cannot at ~31k.

<comment><USER4> Yes, showdead showed the comment and I wondered why it was dead because it was not too controversial.

Are you sure you can't vouch? Click on the time at the top of the comment and it should show on the page that loads. Its intentionally "hidden".

<comment><USER5> I was wrong, it appears I can vouch but as you mentioned you have to dig for the option. Mea culpa.</comment></comment></comment></comment></comment></comment>


Solution to the Journalism Crisis - <AUTHOR> I believe that quality writing deserves to be rewarded. I am happy to pay the New York Times, the Wall Street Journal, The Guardian, etc. an affordable amount for a specific article (50¢, $1.00 ?). However, flat monthly subscriptions are not practical.
For example, NYT is $50/yr, WSJ is $100/yr, Bloomberg $420/yr, LA Times $98/yr, The Guardian £119/yr, WaPo ~$80/yr. These are all fine publications, and I have no doubt each is well worth the price. However, I do not have time to take full advantage of all the channels if I were to subscribe (which I can't afford), and I don't want to limit myself to one paper. I want to pick and choose random articles from any publication.
What if I could pay on a per-article basis? I wouldn't have a problem paying 50¢ or $1.00 for an article I was interested in. It's not practical for each publisher to set up micro-payments. But, if there were an intermediate agent that accepted and managed payments for individual random articles, the money could be aggregated and remitted in a lump sum to the publishers. Sort of like an old-fashioned news stand.
The middleman could accept PayPal payments, or I could open an account with my credit card, and pay once a month for all the individual articles I have read.
Sounds like a business opportunity. Is anyone doing this? <comment><USER2> I honestly giggled a bit when I read this. I think your heart is in the right place but adding another level of "Middle Men" to take a share of the money probably isn't going to solve things.

What might work is a "middle-man" between publishers offering a multi- pulbisher subscription system similar to what cable television providers currently do, though that model seems to be dying too (think cord cutters) and really doesn't solve the crisis either but might be a buisness opportunity.

Personally I think what has happened to journalists and is currently happening to television personalities is the elimination of a class of people(for good or bad the 4th estate is not as valuable in a connected world), the other night I watched SNL attempt to create their show with clips from the staff at home, and compared to what I already watch on Youtube it was OK at best.

Is it possible journalists have just had it too good for too long? Similar but to a leser extent to how the music record industry had it Pre-Napster?

</comment><comment><USER3> afaik blendle.com is trying this, with mixed success.

<comment><AUTHOR> Sounds interesting. They appear to link to NYT, WSJ, The New Yorker. However, the interface is in Dutch, no English option. Are they offering full access, or just insipid subset?

I tried to create account, but the prompts soon became incomprehensible.

<comment><USER3> oh weird. Seems like [https://launch.blendle.com/](https://launch.blendle.com/) is their starting page for the US market?

As far as I remember you can read everything from partner publications, but need to go there through their site or app.

<comment><AUTHOR> Excellent. Thanks for the link. They are in Beta so I can't evaluate yet. If they're doing what I think they're doing, that's exactly what I'm looking for. However, if the original newspapers only allow Blendle access to a subset of their material, this won't cut it. I need to be able to go directly from a HN link to any NYT article, or at least access the same article through my Blendle subscription.

Time magazine failed in this respect with Roku. I got all excited until I saw what dreck Time was restricting us to.</comment></comment></comment></comment>


Android phones look to expand app offerings - <AUTHOR> http://www.sfgate.com/cgi-bin/article.cgi?f=/c/a/2010/09/02/BU381F6GOA.DTL <comment><USER2> > Trip Hawkins, the founder of Electronic Arts Inc., who now runs mobile- > gaming company Digital Chocolate. He calls Google's policy of letting people > use paid apps for as long as 24 hours and return them for a full refund > "senseless and lazy."

I think this is interesting. There is a definite divide between game programmers and "utility" programmers on this issue.

As a developer of a utility app, I am in favor of the 24 hour policy. It means people will try my app, see that it does what they need and decide to keep it. Of course this only works because it offers ongoing utility. (Shameless plug: <http://www.appidio.com/apps/gas-oil-mix/>)

Given the typical quality of mobile apps in general, I know I would personally be much less inclined to buy one without the possibility of a refund. Nobody likes being ripped off.

However, games are typically a fleeting distraction. You download a game, play it for a half hour and may very well never use it again. I'm quite sure the 24 hour refund policy hurts game sales.

Perhaps what Google needs to do is allow developers to choose whether or not they allow refunds. I do not mind at all if someone returns my app. It obviously wasn't worth their $0.99 - that's my fault. But there are plenty of apps that have a very short anticipated lifespan or single plausible use event. Making them subject to the 24 hour return policy really hurts those product categories.

<comment><USER3> I think a better solution might be more along the lines of system-supported (as in linked to the full app) trial applications that can be upgraded in place, along with the application provider being able to specify if refund is allowed or not. Maybe not shareware style where the license server unlocks the app since that's usually pretty easy to crack open, but two different apps that the system treats the same.

It's frustrating trying to find the free vs. paid application and then having to upgrade and potentially lose settings defined in the free one. This was the case when I had an iPhone and is now the case with Android.

But an ability for trying an app is essential. On the iPad I find it incredibly frustrating how many apps I am interested in that don't seem to have a trial program. So I'm being asked to spend $6-10 for an application that I have absolutely no clue if it's good or not. At $1 I can justify that, but $10 is a meal and isn't so easily just tossed aside.</comment></comment>


The Secret to getting really good press - <AUTHOR> http://go.danielodio.com/press <comment><USER3> Writer for TechCrunch here. There's some good advice in the linked article, and ilamont's comment is great.

That said, I disagree with the linked article's advice on how to personalize a story for the writer you're targeting. Their example is to use a subject line like:

"Cindy – Loved your article on Buena's robot makers + story idea"

This would be a great idea if it wasn't already abused to death. Unfortunately we constantly get pitches that begin in this format, and I now find it more annoying than anything — the compliments just seem really artificial. The worst offenders are the ones that craft subject lines to be misleading (i.e. making it sound like they have additional news to add to a story I recently wrote when their pitch is entirely unrelated).

All of that said, it's still great if you identify that a writer has expertise in a certain area and say that's why you directed your pitch to them (just hold off on complimenting them on some random article).

<comment><USER4> Even if all is done right there are slim chances for reporters at well known newspapers/blogs to write about you...unless, you actually know that reporter from real life (conferences, etc)...

If right now I would send a story to the TC reporter (Jason K) which has all the right characteristics (and be interesting) I bet 90% he won't write about my story. And that because not that he doesn't care or something but because he already has tons of good stories to write about AND knows personally a lot of people who wants to tell their stories. Who is getting the first chance? me or those people?

</comment></comment><comment><USER5> This, for once, was a great article on self promotion. Not something like "make something awesome and then everybody will want to write about it".

</comment><comment><USER6> Editor of Xconomy San Francisco here. For all you startup types, Daniel makes one point that I would absolutely agree with and underscore. Don't send us press releases. It's true, we get hundreds a day. They have a magical way of making a great company sound boring. Have them ready as background material, but don't lead with them.

</comment><comment><USER7> Great point about aiming for coverage in local newspapers first.

Now I just have to explain to reporters in my hick town what a "startup" is.

<comment><USER8> small company? that should suffice for most people.

<comment><USER7> Not with zero employees.

<comment><USER9> I know for a fact that you have at least one employee, yourself. I'm in a rural area and the local paper loves to do stories on any small business happenings. A single-person startup is still economic news in a small town.

</comment></comment></comment></comment><comment><USER10> Founder of SutherlandGold Group here. I would just add that every start up really needs to pitch a story that is relevant to things that are already in the news. As reporters and editors here already pointed out, the media is just too busy covering trends of the day to drop everything for every start up pitch that comes in the door.

The secret to great coverage in the media, is really building a great story about your company, yourself and your product before you ever pitch anything to the media. Founders really need to focus on how what they are doing or building is disruptive and powerful and delivers a really worth value proposition to the intended audience -- consumers, advertisers, publishers or whoever will buy your product.

Almost 10 years ago, I led the team that launched Honest Tea. Our story then was about a founder who was mentored by an MBA professor and the two joined together to build a great company that was recently acquired. Ten years later that story is still being written about the company.

So think about all the elements of "your" story before you pitch anything.

</comment><comment><USER11> Former technology editor (IDG) here. The problem most reporters and bloggers face is a lack of time to research and write. There are too many potential stories, and not enough bandwidth to cover it all. This problem has been exacerbated by industry layoffs and Internet-related factors that make it much easier for startups to contact news organizations directly, as opposed to going through PR people. The result: A flood of emailed pitches and inquiries (many of them off-topic) that can't be properly followed-up.

There is also tremendous pressure to write articles that will get lots of page views. This makes it more difficult to justify writing straight-up product stories or reviews, unless it's something that's truly innovative or game- changing for the public. It's much easier to simply write about some minor Apple rumor, Microsoft's latest screw-up, or Facebook's umpteenth design tweak, add an irresistible headline, and perform a little "SMO" (social media optimization -- submitting to Digg, Twitter, etc.). Look at Techmeme on any given day to see how prevalent this type of writing is.

Nevetheless, I think if you personalize the pitch and identify likely writers (like TFA says) it will help differentiate it from the pack of press releases that form the bulk of many writers' email inboxes. Also, follow up nicely a few days later, asking if the writer has had time to review the earlier email or clarifying whether or not they are the right person to cover this type of product. Some will still ignore you, but others will respond, even if it's only to say, "sorry, not interested. Try _____."

One other piece of advice if you're looking for publicity: If you get an offer to speak at a conference or participate at a panel, do it. These days, there's a good chance that _some_ blogger will be in the audience and writing about the interesting things he/she heard, even if it's local event or meetup. If it's a big technology conference, and assuming news organizations have a travel budget to send people there, there will probably be journalists or pro bloggers (paid) writing about the sessions. Conferences are great for them, because they (usually) get to hear interesting tidbits about interesting products or trends that are not moderated by a press release or PR drone, and they can crank out many stories/posts in a single day without having to deal with scheduling/interview overhead. I have really positive memories of conferences such as Webby Connect and MIT's EmTech, as well as a YCombinator open house when they were still in Cambridge. At all of these events, there was lots of great stuff to write about, including from companies/people that I didn't know much about before I went in.

EDIT: Added background and "off-topic" to first graf. It's one of the biggest irritants to technology reporters to get a pitch that has absolutely nothing to do with what they or their publication covers.

EDIT 2: Added last graf about events

<comment><USER12> ilamont, great feedback; can I repost your comment in the blog? would love it to be there for others to read. - <USER2>

</comment></comment><comment><USER13> > And then I would write her a one or two sentence email with my pitch

I am curious what kind of short email you write. The subject hints at a short pitch, but do you have a specific example?

This was the subject of the email listed in the article: "Cindy – Loved your article on Buena's robot makers + story idea"

<comment><USER12> I did a related post w/ a video answering exactly your question: <http://go.Danielodio.com/wsj>

Here's a sample email - note that he'd written an article about iPhone tethering, which I referenced in the subj. line:

Subject: Re: Chris - re: iPhone Tethering + AppMakr

Chris , just saw your write-up about iPhone tethering. I've always wanted to be able to do this!

Thought you might be interested in a related topic: A self-service tool we just launched in Jan called AppMakr that lets anyone build an app for as little as $199. We've been covered by WSJ, TechCrunch & many others, press at <http://go.AppMakr.com/press> and you can see a video and details of the service at www.AppMakr.com/learn_more

Let me know if you'd like to tell your readers about it!

Regards, <USER2>
____________________________________________________

Daniel Ruben Odio-Paez Co-Founder & Chief Operating Officer, PointAbout.com Creators of www.AppMakr.com

</comment></comment><comment><USER14> so in summary.. seek out reporters who have previously written on related topics, send a personalised email to them and try to build up a good relationship..... focus on the smaller fish to then get picked up by larger ones.

simple and practical advice. i like it, and i will put it into practise.

</comment><comment><USER15> You could do all this work to get your product into the press. Or you can just make something that people actually care about, and the press will come to you.

Within three months of launch, SproutRobot has been on TechCrunch, LifeHacker, and on CyberGuy on 40 TV stations across the U.S., and we didn't lift a finger for any of it. We just built something people were excited about and the press found us.

<comment><USER16> It took me about a decade to learn the lesson of eighth grade dances: you can be a wonderfully witty, intelligent, charming gentleman, and if you stand there in the corner waiting for a young lady to recognize this, in all probability you're going to be a very lonely witty, intelligent, charming gentleman.

</comment></comment><comment><USER17> i've found it's helpful to start building relationships with people BEFORE you need the coverage. for example, i'm always on the lookout for stories i can send over as a tip to TechCrunch. sometimes they cover them, sometimes they don't. the important thing is now they know my name and will read my emails.</comment>


Give your macropad a new 'brain' with my drag-and-drop automation software - <AUTHOR> https://imgur.com/gallery/NsnumSd <comment><AUTHOR> Just to clear things up, this is a desktop app that you install and run by double clicking, just like any other app. It's designed to work with your numpad (either the numeric keypad built into your main keyboard, or a separate USB one).

You drag and drop commands like "type text" and "open folder" onto each of the buttons of your numpad. When you then press that numpad button, your automation is triggered!

However, to let people hook up their macropads, I made it also listen out for the keyboard shortcuts ctrl + alt + 1, ctrl + alt + 2 etc. as numpad button 1, numpad button 2 etc. respectively.

So if you can make your macropad do the keyboard shortcut ctrl + alt + 1, then my software will behave exactly the same as if you pressed numpad button 1 and will trigger the associated automation.

Let me know if you have any questions about your specific use case!</comment>