Cryptocat: webchat with client-side encryption features - <AUTHOR> https://crypto.cat <comment><USER2> Took me a little while to realise you set the password in the blue box above the text entry. Interesting concept though.

For those interested it uses crypto-js (<http://code.google.com/p/crypto- js/>).

Might want to increase the random entropy source, from what I can see it's just DateTime.

Also I'd use a better hashing algorithm than SHA1 with AES-256.

I understand the need for PBDKF2, but I'm not so sure about using the password as the key and salt.

<comment><USER3> They aren't using SHA1; they're using HMAC-SHA1. Not the same thing.

That said: this is an idiosyncratic (rekeying AES every 500ms?) and not particularly resilient cryptosystem. As a commenter said below: don't use this for anything serious. You're equally well off with a server that simply sends "plaintext" messages over TLS.

<comment><AUTHOR> AFAIK, the server does not rekey AES every 500ms - and your suggestion to use a server over TLS opens you up to the problem of the server itself being able to read all plaintext.

<comment><USER3> The server can read all the plaintext anyways.

<comment><AUTHOR> No it can't, that's the whole point! Are you trolling?

<comment><USER3> No, what's happening here is that that you don't actually understand the threat model for encryption code running in Javascript fed by a server, and you're getting more and more frustrated by the fact that I'm not handholding you through it.

Reasoning about security systems (click my name to see that I do that for a living) is frustrating. To see what's happening in this case, steel yourself for a headache, sit down, and think hard through all the different things that attackers can potentially influence. Then ask yourself, "what could an attacker accomplish by manipulating that thing?".

</comment></comment></comment></comment></comment></comment><comment><USER4> Made a test room, 'hn' (<https://crypto.cat/?c=hn>). Set a key, 'password'. Will leave it open in background and peek in occasionally if others want to try.

</comment><comment><USER5> I was thinking of building something similar. The benefits of client-side encryption is that the message is entirely encrypted until it reaches the destination. Even if the cryptocat server was compromised, the message is still "safe".

<comment><USER3> No, if the server is compromised, you are thoroughly boned. In fact, there are very common vulnerabilities that don't come close to compromising the server that still destroy the security of this scheme.

Javascript cryptography is almost always a terrible idea.

<comment><AUTHOR> "If the server is compromised [insert disastrous result here]" applies more so to many other servers serving you crypto - I feel there's the advantage of being able to verify the .js files yourself in your browser here:

<!-- cryptocat uses the crypto-js library - <http://code.google.com/p/crypto- js/> \--> <!-- [http://crypto-js.googlecode.com/files/2.3.0-crypto- sha1-hmac...](http://crypto-js.googlecode.com/files/2.3.0-crypto-sha1-hmac- pbkdf2-blockmodes-aes.js) \--> <script type="text/javascript" src="js/crypto.js"></script>

<comment><USER3> You can't just verify the .js file! You have to verify every line of JS code that ever hits the JS interpreter, whether it's specified directly in the source code that the author told you about or eval'd later in some random event handler. That code could come from an explicit <script> tag in the original DOM source; it could come from an event handler specified in the DOM, or set later by any other piece of JS; it could come from an Ajax request handler; in many browsers, it can even come from CSS. It can easily hide itself after it does its job, and it isn't going to look like "<!-- zero out AES key here -->".

The browser JS runtime does not provide the tools you need to verify the integrity of a cryptosystem. Full stop.

Actually, browser JS runtimes don't even provide the tools you need to safely _run_ a cryptosystem. How do you know what intermediate values are cached by any given JS? How do you know if any given crypto function is leaving footprints in browser memory? You don't; nobody does; nobody has published the exhaustive analysis of _any_ browser JS to support an argument either way.

Not that that matters for crypto.cat; they don't seem to care much about side channel stuff; for instance, their HMAC compare is the JS string "!=" operator.

As for "applies to so many other servers": I don't know what you're saying. You made a claim about the security properties of this application. That claim was wrong. It doesn't become right when you point out some other app that has the same flaw.

<comment><USER6> What about loading Java applet (and communicating from JS) that handles all encrypting/decrypting? Would that take care of at least the latter concern?

<comment><USER3> Yes; you could just use Bouncycastle's Java PGP implementation, which is probably the safest option.

But people hate Java, and it's increasingly disabled in browsers.

<comment><USER6> Well I thought the question was interesting because if that'd work, so would a native Web API.

<comment><USER3> A native web API would work! I think it'd be great if everyone could agree on some basic, high-level crypto functions, implemented in browser C, to expose to JS. You'd want the interface to look much more like Keyczar than crypto-js, though.

<comment><USER6> I was also wondering — I hope you don't mind — would it be considered safe to just verify signed messages using cryptojs? For example, let's say the environment (webapp) is bootstrapped using TLS, this includes the public key. Now could this application receive signed messages through different, unsecure channel and verify them safely using the public key received through the secure channel?

<comment><USER3> If it can be safe to verify the integrity of a JS cryptosystem at all, I think it's going to turn out to be tricky to do it. Like I said elsewhere on this thread, the number of ways a web page can influence the Javascript runtime is huge, and a lot of those ways are cached.

<comment><USER6> Well the integrity of the JS cryptosystem would be no worse than the integrity of the webapp itself.

I understand that one cannot trust the JS crypto for handling the private key and doing any singing & encryption because of the possibility of leaking information through side channels. That's why I was wondering if it'd be possible to use JS crypto in only one way — that is to verify messages against public key received through the secure channel.

If the JS crypto only does verification, it wouldn't matter if the _public_ key is cached, as long as it can't be arbitrarily changed by another process. If the server is taken over or there's an XSS or other client-side vulnerability the bidirectional TLS would be similarly useless anyway.

<comment><USER3> It's not the "math" of the crypto that concerns me; it's the actual integrity of the code performing it. So for instance, if you're trying to do RSA operations in JS, how do you know that none of the bindings in the current Javascript runtime have been exposed to attacker-controllable content? How do you know nothing in the runtime was fed by cached content read over straight HTTP? Or from an HTTPS server that has a DOM corruption flaw?

<comment><USER6> If any of those things can be used to affect the crypto part, then they could be also used to affect the web application itself, in the first place. I'm only looking for the level of integrity, that I can get from launching the web application over HTTPS anyway.

<comment><USER3> Is this a point that we're dancing around here? Let's resolve it: for the most part†, you're right: the things that break clientside JS web crypto are things that break (in various degrees) the web app itself. The problem with accepting this is that the point of clientside crypto is to inoculate users from the downside risks of the simple security flaws that most applications have. What's the point of sophisticated clientside crypto if it's undone by any trivial XSS flaw?

† _There_ are _issues that damage clientside JS crypto that don't blow up the app itself, but they are not the core issues I'm concerning myself with here._

<comment><USER6> I am mostly interested in replacing bi-directional TLS. There are things that need to be encrypted — authentication, any authorized operations, fetching authorized content — however the majority of content that application uses is public, so the encryption is very much redundant.

Unfortunately I can't just bootstrap the application over secure channel and fetch any content over not secure channels, as it'd be trivial to then poison the application using the unencrypted communication. Take a Hacker News-like application: Let's say that news and comments can be accessed over unsecure channel and voting and authentication are performed over TLS. Even though the authorized operations are safe, the data could be poisoned over unsecure channel — for example the user-id could be manipulated and the user would end up voting securely for someone else's post. To mitigate it, all data would have to be fetched over secure channel. However currently this is all or nothing, by fetching public data over TLS, one has to pay the encryption tax — in both CPU time, as well as in disabling any caching middleware.

Of course singing HTTP messages wouldn't be just TLS-light, since the messages could be stored and kept as an evidence that the webserver has produced such a message, which is different to TLS session-based guarantees, but it should be fine for what I need. This is why I was interested to know if such an approach would not degrade the security of the web application below what bi- directional TLS could offer.

I suppose this is not the most exciting use for client-side cryptography right now…

I think the web-application are increasingly going in the direction of reinventing Java applets anyway. There are few components that are needed for that and that will be gradually introduced: 1) Application that can be installed and launched locally (e.g. more sophisticated version of Chrome Webappstore). Once you have that, you basically have RPM in browser. 2) Capabilities system that controls what and how an application can access. All the pieces are already in there: web extensions, mobile apps, etc. 3) Pluggable storage to eliminate the need for storing data on the remote server. There is LocalStorage and IndexedDB already, now it's just a matter of time till Chrome adds Google Storage, Apple adds iCloud, Ubuntu adds UbuntuOne etc.

Once you have that, the Java[Script] applets are basically back. With NaCL in the mix you could even actually run the JVM in the browser once again. So yeah, I think in the long run the client cryptography will come around. But for now I just wanted to make use of what's currently available.

<comment><USER3> This is generally a terrible idea. TLS 1.2 is the product of almost 20 years of sustained attention and adversarial testing effort, 20 years of missteps, 20 years of design decisions argued over by people clearly not limited to but including some of the best cryptosystem thinkers in the field, and it _still_ has missteps in it.

The idea of replacing it with ad-hoc encryption at the application layer carries with it the notion that the basic problem of "taking some bundle of application data and securing it so that it is encrypted in flight" is somehow simple, and that TLS 1.2 is overkill for it. But that's not true. To a significant extent, everything TLS 1.2 does that you don't do is a vulnerability in your system. You won't see it or fully understand it; extremely competent people have gone _years_ staring terrible vulnerabilities in TLS in the face, even after those flaws were _pointed out to them_ , and let them pass because the real-world practical impact wasn't obvious at the time.

And this is all before you get to the point where, to avoid the overhead and complexity and inflexibility (and whatnot) of TLS 1.2, you choose to build _your_ cryptosystem in the most hostile possible environment for cryptography: the language runtime where no operation, function, or binding can be trusted, where every symbol table is potentially corrupted by any of 10s of content sources, many of them under the control of attackers, in a runtime that is being aggressively redesigned, optimized, JIT'd, cached, and transformed for performance.

So you can see that this goal puts you out in the wilderness on _two_ vectors: one of attempting to start from scratch on secure transport protocols, when 20 years has shown us that the simplest service models for secure transport are surprisingly hard; and the other vector being "trying to implement secure crypto in a language environment that's hostile to the concept".

Skip it. Work on something that will make people's lives better in the immediacy, instead.

<comment><USER6> > The idea of replacing it with ad-hoc encryption at the application layer

Yes, I understand that this would be a terrible idea. That is why I only intended to _verify_ signed, _public_ content using _static_ key. I don't indent do any encryption in client-side JS, not even to generate session keys!

<comment><USER3> Verifying signatures under public keys is not an easier problem than encrypting data; it's actually a harder one: more things can go wrong, not less.

<comment><USER6> What do you mean? I thought if you eliminate any of the things I've mentioned previously all that is left is just checksum algorithm and some math. Note that the public key would come from the secure TLS channel, so there's no issue of obtaining and verifying one.

<comment><USER3> We're too far to the right on this thread to keep going with this, but take my word for at least this: public key signature verification is advanced number theoretic cryptography. It is actually _less_ straightforward than somehow applying a block cipher to variable-length arrays of user data.

<comment><USER6> So does it just mean that the implementation is hard to get right? I'm all for killing off ideas, especially when it comes to cryptography, but I'd like to at least learn something in the process. Is there somewhere I can learn about everything that can go wrong with that specific process, or would I have to just try to implement this algorithms myself, to understand the risks? BTW. Thanks for everything so far; I trust you're a busy man.

<comment><USER3> A great place to start is (Google) [nigel cryptography]; for your particular question, start with the chapter "attacks on public key cryptography".

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER7> Seems useful.

<comment><USER8> Except that the encryption is handled with javascript (obviously). I wouldn't use this for anything serious.

<comment><AUTHOR> Why not? AES has been implemented in javascript half a dozen times.

<comment><USER3> And? What is that evidence of?

<comment><USER9> That the implementation of an encryption library in Javascript is not a reason to mistrust services which use that library?

<comment><USER3> It isn't evidence of that. Implementation in _browser_ JS _is_ in fact a reason to distrust a cryptosystem.</comment></comment></comment></comment></comment></comment>


The Economist names Paul Buchheit 2011 Computing and Telecom Award winner - <AUTHOR> http://www.economistconferences.co.uk/press-release/economist-names-paul-buchheit-its-2011-computing-and-telecommunications-innovation-awa

<comment><USER2> Well deserved. I think the list of achievements speaks for itself.

Congrats Paul!

</comment><comment><USER3> Congratulations!</comment>


Ask HN: Is there any web apps that finds your site's security flaws - <AUTHOR>

My first post here, just wondering if something like this exists or even possible? (Hopes it makes sense) <comment><USER2> The web apps I've seen that do this all look pretty scammy. I wouldn't want them poking around my site. Then again, it is a public site, so...

Anyhow, since there are several major attack vectors, there are also several different types of scanners. I recommend you read the book "breaking web software" to get a better understanding of the types of attacks.

As far as tools, if you have a PHP install I recommend <https://chorizo- scanner.com/>

No recommendation, but <http://www.acunetix.com/cross-site- scripting/scanner.htm> looks like they can help.

IBM has a good article/series on web app vulnerabilities at [http://www.ibm.com/developerworks/web/library/wa- appsecurity...](http://www.ibm.com/developerworks/web/library/wa-appsecurity/)

And finally, wapiti is a vulnerability checker written in python that will do scans of web apps. COmmand line, not web, but pretty good - <http://wapiti.sourceforge.net/>

</comment><comment><USER3> I'm sure there are web apps which will find security flaws; but you shouldn't trust their results. Finding security flaws is equivalent to determining whether a Turing machine will halt -- i.e., there is no algorithm which can guarantee to give you the right answer.

</comment><comment><USER4> No web app, but a lot of individuals who'd love to offer this as a service. For general guidelines on writing secure web code, refer to <http://www.owasp.org></comment>


Joel Spolsky: Facebook not living up to "developer love" motto - <AUTHOR> http://meta.stackoverflow.com/a/121688/143965

<comment><USER2> Facebook has always been, and is still notoriously bad at developer support.</comment>


Hundreds of Pterosaur Eggs Found in Record-Breaking Fossil Haul - <AUTHOR> https://news.nationalgeographic.com/2017/11/largest-pterosaurs-eggs-discovered-embryos-fossils-paleontology-science/ <comment><USER2> Yesterday discussion (different link): [https://news.ycombinator.com/item?id=16040273](https://news.ycombinator.com/item?id=16040273)

<comment><AUTHOR> I missed that, thank you!</comment></comment>


Rewards for Opinions Privacy Policy - <AUTHOR> https://www.rewardsforopinions.com/privacy.html <comment><AUTHOR> "Six months following your inactivity with the panel Sites and/or uninstalling the Application/Services, RN may transfer your PII to a third party data broker and/or data management platform for purposes of resale/reuse by such third party."

I feel like this is a great example of how people can easily and unwittingly hand over their personal data for resale. I was going to sign-up for this in order to earn some Southwest Rapid Rewards points, but then I read the privacy policy. To their credit, they provide an opt-out. But, the sheer length of the policy is off-putting.</comment>


Are WhatsApp group a real pain? - <AUTHOR> <comment><USER2> They actually are if the mute for a long time was not available. I have muted multiple groups as I can't keep up and all good. but be careful when exiting a group, that might make you look like a 2nd class citizen when you catch up in person after that!

<comment><AUTHOR> Very true, even exiting is a pain</comment></comment>


Let's stop calling Alan Turing a "codebreaker" - <AUTHOR> http://ali.vc/post/25807311540/alan-turings-100th-birthday

<comment><USER2> Oh even the Gordon Brown public apology mostly talked about him as a 'codebreaker' in World War II

<comment><AUTHOR> Yeah, it's mostly the British press but I've seen US newspapers talk about his AI or codebreaking works with no mention of e.g., Turing Machines</comment></comment>


Moral dilemma in participating in an App Challenge - <AUTHOR> https://navigator-app-challenge.com/ <comment><AUTHOR> The dilemma : The key evaluation point for the challenge is "How the app will promote the use of office paper and the benefits for the user". Even though this is just a challenge, I'm worried if my idea takes off, it would result in increased paper production which will result in increased cutting of trees.</comment>


Running Tips - <AUTHOR> https://ruiper.es/2019/12/31/running-tips/ <comment><USER2> There's interestingly no correlation between reduction in injuries and stretching. Unless you have a training incident that leads to an injury, 'wear and tear' injuries likely come from muscle imbalances and activation dysfunction than lack of stretching.

A decent warm up routine may happen to involve some active stretching but that's through necessity of moving in a good range.

An example being people that flair their lower leg outward from the knee as they run being correlated to acl problems. People talk about tight calves, tight quads etc but ultimately its caused by a dysfunction of glute control which can't be improved via stretching.

Also a related side note, you should almost never stretch your hamstrings, especially if you are sat a lot. That compounds the issue you'll (and most of us) likely already have with long loose hams. Couch stretch hip flexors and strength hams instead.

<comment><USER3> I get into a lot of fights with people about how stretching is a farce since yoga is so popular.

Another running myth that gets brought up in the post is the "lactate threshold." The misconception comes from the "acid" part of "lactic acid". We've come to believe that your muscles producing acid will cause them to feel like they're burning. They're burning because they're tired and hitting their limits, nothing more. Lactic acid is just an extra source of energy. The way you train probably doesn't need to change, but science behind "lactate threshold" is wrong.

<comment><USER4> What are you saying? One can be more specific, but there certainly is a physiologic definition of lactate threshold. It is the output (running, cycling, rowing, skiing, whatever, can be defined as performance as % of VO2max) at which blood lactate begins to rise above baseline. More importantly, maximal lactate steady state is the rate at which lactate production equals clearance. Above that and you'll blow up soon. Yes lactate can be converted back to pyruvate and used for energy, but above MLSS, it accumulates more than is reconverted, and energy stores will run out.

What science do you think is "wrong"?

</comment></comment></comment><comment><USER5> I've been a lifelong runner and a great psychological trick I've shared with people to help get you into your shoes and out the door is this:

"Have you ever regretted going on a run?"

Invariably, the answer is no. It's amazing how effective this is at getting you to just suck it up, no matter how cold it is outside, and get running. You'll be glad you did.

<comment><USER6> That may be so for passionate and experienced runners.

I have certainly regretted going for a run (and broken off the run after a few hundred meters). Not often, but at least twice. It's no surprise that I stopped running some time after that.

<comment><USER7> Yup, that works only for those who have run before and like running. For anyone starting off and/or not really fit enough to run for a few minutes without difficulty, it's easy to give up quickly. Walking is a less strenuous activity for such people (though getting out to walk may itself be a difficult action to take).

</comment></comment></comment><comment><USER8> What do runners do when it's -15degC outside and roads are coated with slippery ice and snow? I don't get enough exercise during winter :( Running is nice because you can start at your front door, but most winter sports seem to require hauling gear to an appropriate venue (thinking about skiing and skating as the most obvious examples).

<comment><USER9> [https://www.moosejaw.com/product/stabil-icers-run- crampon_10...](https://www.moosejaw.com/product/stabil-icers-run- crampon_10368717)

no affiliation and i've never used those but there you go. aside from the shoes very warm gloves and a balaclava.

warm glove hack: get dish gloves and wear them under whatever mittens. your hands will sweat and the dish gloves will contain the warm sweat. it's quite gross but quite effective.

<comment><USER10> can confirm but I use latex or nitrile disposable gloves. they def stay warm but also your fingers will wrinkle from the sweat.

<comment><USER9> >will wrinkle from the sweat

that's true but benefits far outweigh that cost; primarily that you don't need to constantly wash your mittens.

</comment></comment></comment></comment><comment><USER9> i used to run 15 miles a day 3 days a week (7.5 to work and 7.5 back). best piece of advice i can give to someone starting out 1) don't use music and 2) don't run for time but run for distance instead.

my reasoning for the first is you will have an easier time drifting away and enjoying the running for its own sake (rather than as a chore) if you do not try to distract yourself.

my reasoning for the second is running for time is like a prison sentence that you need to bear while running for distance is a goal you can accomplish (or fail to accomplish). my runs to work used to take ~1 hour and on the days that i couldn't make them because of inclement weather i ran on a treadmill. i never made it the full hour on a treadmill.

these two tips help you overcome the greatest (and most important) hurdle to becoming a runner: enjoyment.

edit: i guess one corollary of the running for distance thing is that you don't need to stress about speed; i often tell myself "you don't have to be fast you just have to finish". so when i get gassed in the second half of a run i slow down to compensate. that finish spurs you on the next time and naturally you'll get faster (well to a certain point - if you're really trying to get good then you need sprints/intervals/etc but then you don't need my advice).

<comment><USER11> Yeah, obviously YMMV (quite literally, in this case), but I definitely relate to both of those (especially not listening to music).

</comment></comment><comment><USER12> A couple missing from the list:

Be cognizant of your turnover (stride rate): [https://www.runnersworld.com/training/a20787354/increase- you...](https://www.runnersworld.com/training/a20787354/increase-your-stride- rate/)

Engage your glutes and abs when running

</comment><comment><USER13> Those are some nice tips!

If anyone here is looking to start running as part of a New Year goal, I'd suggest not getting too discouraged with all of the stuff that seems complicated about running. It takes a while to get your body used to it.

When I was 17, I made it a goal to try and run a mile without stopping at the track. Once I hit that goal, I stopped because I thought other people were judging me. It took me ten more years to go for another run, but by that point I'd stopped caring as much about what other people thought. Now I've made it a habit, and enjoy all aspects of the sport.

</comment><comment><USER14> The best running tip I've ever had is "run tall". Most people tend to slouch forward at the hip which prevents some muscle groups from engaging. Made a world of difference for me.

<comment><USER15> My version of "run tall" is "pretend you're riding a pony". Your hips and legs are the pony, and your torso is riding it, and it works for me as a cue to remember that my torso should stay tall and upright.

Also when I get really tired, it helps me dissociate myself from the feeling in my legs because instead of thinking "ow my legs hurt", I can think "thank you, pony, for being so strong and working so hard for me!"

</comment></comment><comment><USER16> 0\. Do less than you think you're capable of, especially if you are a novice, or if you are coming back from injury or a long layoff. You will hurt less and enjoy yourself more... so you'll keep running!

</comment><comment><USER17> 22\. If you get injured - apply pressure. Have an elastic band aid in your training bag or pocket at all times. Applying pressure on an ankle tear will shorten healing time to 1-2 days vs weeks if you let it bleed and swell.

23\. Running technique: you want to distribute forces on all 3 joints. Use elasticity from tendons and binding structures to capture breaking forces and bounce back.

24\. Take it very easy when you start out, change ground, or shoes.

</comment><comment><USER18> I do a lot of running with 1-2 marathons a year. Two recent things had a big impact on my running:

1.) Understanding the importance of rest. The FIRST method described in "Run less, run faster" really changed how I structured my weekly training regimen and drastically reduced my injuries

2.) Getting a Stryd footpod to accurately measure my pace even during fast changing interval sessions. It's still magic to me how a footpod can be that accurate

</comment><comment><USER19> If you just want to get started running, you don't have to pay attention to any of this. Just download a couch 2 5k app and spend $100 on running shoes. I run a 10k in an hour with moderate effort, 90% of the tips listed don't matter to me (though I've heard of most of them).

</comment><comment><USER20> The only running tip I have is to think of miles run like a bank. Every time you go out you're putting money into the bank. When you skip a day you're taking money out. Everyday you skip it's a little bit harder when you go out the next day. So be jealous of the opportunities to go out. Don't cheat yourself by skipping but put a few more pennies in the bank instead.

I'm not super gung-ho though - I think I run 20 miles a week maybe - just 5 or 6 days. And I always take a day off. But it works for me.

<comment><AUTHOR> This is a great analogy!

</comment></comment><comment><USER21> Oh, it's tips about running. I'm embarrassed to admit that I assumed it was going to be a blog post about running some software I hadn't heard of yet called Tips.

</comment><comment><USER22> I am testing out if running can help me improve my focus.

Recently I have signed up for longer races and slowed my pace down. I find it is good practice for impulse control. When I get bored at work or at home, I quickly open a new tab or pull my phone out of my pocket. I would rather just be bored for a moment and stay on task.

I think running long distances can help me a lot because I do get really really bored out there, but there is not much to do but keep running.

</comment><comment><USER23> A bunch of us do the Slowtwitch 100 Runs in 100 Days challenge in the winter. You just have to run a minumum of 30 minutes or 3 miles (whichever is the shorter), one run for every day. If you miss a day, you can do 2 runs on another day.

The focus on consistency and slowly gaining a relatively large training base often puts people in a much better place with their running as the spring comes around.

</comment><comment><USER24> > Have more than one pair of running shoes with different drops.

I am skeptical of this claim. Most of us have heard about running in different shoes though.

</comment><comment><USER25> Tip 0) especially for older people: Get an ok from a physician before starting on a running program. I horked both Achilles pretty badly a few years ago by trying a walk-->run progression I found on the web. Didn't rupture but came pretty close.

GP and later a podiatrist both said they wouldn't have recommended running had I asked. Been dealing with tendonosis ever since.

</comment><comment><USER26> Nice tips ! I have been running for a couple of years now averaging 1000km / year.

I have a problem with keeping motivated though. I can prepare for a race for 3 months, then right after, I rest for 2 months, basically losing all progress. Anyone dealt with this successfully ?

<comment><USER27> I don't think there's any silver bullet. But what helps me is pre-scheduling my runs or exercise in my calendar at the beginning of the week/month. That kind of tips "default-mode" into just listening to the calendar notification and doing the run.

</comment></comment><comment><USER28> The foam roller is a tool I wish I had used 20 years ago. Thanks for sharing these tips!

</comment><comment><USER29> A question for all you runners:

How many of you have ever gotten what's called a "runner's high"?

<comment><USER30> I get the runner's high most days when I run (and I typically run 6 or 7 days a week), starting about a half hour in. I feel very comfortable physically and mildly euphoric mentally. I hear more in the music I'm listening to than I normally do and I tend to feel very good about people (both others and myself).

FWIW, I do most (80%?) of my running at a 130bpm heart-rate and typically don't get the runner's high when I'm working out more intensely (the other ~20% of the time).

I don't know that I would run as much as I do if I didn't get the runner's high, although I didn't get it regularly when I began. One big takeaway I got from reading one of Lydiard's books was that when you're building your base you basically can't run too slowly, but it's very easy to run too quickly.

If you run too quickly you risk injury and burn-out which interferes much more with making progress as a runner than if perhaps you're not running quite fast enough to get the maximum adaptation for your effort.

</comment></comment><comment><USER31> The best advise I got was "it's better to run 300 meters every day, than 5km once a month".

<comment><USER32> Interesting. Is there any evidence for that? It definitely makes sense, but I wonder if 300meters is enough to make a difference.</comment></comment>


Life Without Cars - <AUTHOR> http://krugman.blogs.nytimes.com/2014/07/15/life-without-cars/

<comment><USER2> The problem is that these kinds of car services depend on a supply of people who have cars to "crowd source" to. Notably, if there's a shortage of people in the area able and willing to drive, the price will reflect the supply.

The real car-free solution involves changing cities and public transportation infrastructure to reduce dependency on single-occupant vehicles.

</comment><comment><USER3> Surge pricing is nothing new. You already pay a big price in time and money (lower fuel economy) when you choose to drive during rush hour.</comment>


A Cell Tower of Your Very Own - <AUTHOR> http://online.wsj.com/article/SB10001424052748703636404575353153350315146.html?mod=WSJ_hps_sections_tech

<comment><USER2> It's not really very much of a win for me, is it? I mean, basically I'm paying $250 for the privilege of paying AT&T dollars per gigabyte for traffic that goes through my own broadband connection that I'm already paying for.

I can't even begin to understand... why I would do that.

<comment><USER3> I have one. I recently moved to an area with very poor AT&T coverage. I am under contract with AT&T, and I wasn't pleased with the prospect of having to go outside every time I wanted to make or receive a phone call from my house, so this seemed good for me.

I bought one for $150 (not $250). It works, but not very well. On some calls I get fairly severe audio distortion that makes it hard to understand people. I'm not sure how to fix this yet. Still, though, it's better than not being able to make or receive any phone calls at home.

Data usage isn't an issue because I have wifi, and call time isn't an issue because I never get close to my limit anyway.

<comment><USER4> $150 is probably more than the early termination fee on your AT&T contract, after proration.

</comment></comment></comment><comment><USER5> Heh,

So they all have the same range... _Provides signal for data and voice up to 5,000 square feet_ and none of them want to admit it's ~40 feet from the base station. (40^2 * pi > 5000)

</comment><comment><USER6> I'm coming up on 3 years of use for my T-Mobile Blackberry Curve 8320 w/ UMA support. Still waiting for an all-around better device and network setup for my usage pattern (messaging/emailing a lot, talking a lot, maps, occasional web browsing).

While the T-Mobile coverage has sucked plenty in certain areas, I've had nearly 3 years of satisfaction that if there's a wifi network, I have coverage -- in various apartments where coverage sucked, deep inside work buildings (where other service providers also can't penetrate), at other people's houses where there isn't great coverage... and it costs me nothing. Unlimited calls/data/messaging w/ UMA included: ~$90/month. Plus I can tether!

Anyone know if T-Mo is baking UMA support into any upcoming Android devices?

<comment><USER7> I've read rumors about a targeted Q3 release, though I'm not sure how they'd push that out.

</comment></comment><comment><USER8> What's going on with magic jack's femtocell device? It was getting hyped this past January... based on the PR, it's a femtocell that can hijack nearby GSM phones and bridge them to magicjack's VOIP network. seemed bizarre.

<comment><USER9> It's illegal according to FCC regulations to broadcast on the same frequencies that 3G carriers are licensed to use. You could probably buy some shady device like this and import it from Asia, but you're going to be stepping all over your neighbor's cell phone reception, or worse, sharing it out with them, if it even worked in the first place.

<comment><USER8> Do you know if this was the case specifically with the magicjack device?

An average press clipping from mid-january: "The company said its device is legal, arguing the spectrum licenses of GSM carriers such as AT&T Mobility and T-Mobile USA do not extend into the home." ( [http://www.fiercewireless.com/ceslive/story/magicjack- using-...](http://www.fiercewireless.com/ceslive/story/magicjack-using-gsm- femtocell-technology-voip/2010-01-08) )

It's an interesting tack to take, and I'm very curious about whether the stalled product launch was due to regulation/legal issues, or if magicjack abandoned it for another reason.

</comment></comment></comment><comment><USER10> I find this whole microcell thing amazing. Your operator tells you that since its service is sub-par and you can't get reception, you get the privilege of a) paying $250, b) supplying your own broadband, c) still paying your crappy operator, and d) being unable to move from within the cell to outside it without dropping the call.

I mean — seriously, what kind of a deal is that?

</comment><comment><USER11> For all the author's sarcasm, it was not much a relief from the marketing- speak that the cell companies are plagued by. But I am still glad they are selling these. To me it is a symbolic gesture that says they actually care about solving the many technical problems that they still face.

<comment><USER9> To me it is a symbolic gesture that AT&T doesn't want to pay to upgrade their terrible network, so they would rather the customers pay to buy and install their own femtocells.

I, for one, will refuse to buy one. If AT&T on the other hand wants to provide one free of charge, and pay me a reasonable fee of $25 per month for 2 gigabytes of bandwidth, I will be happy to host it for them.

</comment></comment><comment><USER12> "And because the device relies on a global positioning system to confirm its location in a service area, it needs a clear line of sight to the sky. "

So, it doesn't work in a basement, or a garage, or really any place you might actually need it?

</comment><comment><USER13> San Diego city proper has complete M-Cell coverage. 5 bars and snappy latency wherever I roamed. Still seems expensive when it's such a commitment to be with an operator, anyway.

</comment><comment><USER14> I live overseas in an area with spotty wireless reception... any chance I could use a similar technology to improve signal?

<comment><USER15> Sure! Cell phone repeaters and boosters are widely available. <http://www.google.com/webhp?#q=cell+phone+repeater>

</comment></comment><comment><USER16> <http://en.wikipedia.org/wiki/Cell_On_Wheels>

</comment><comment><USER17> I got one. Voice goes over M-Cell. Data goes over my wifi. 5 bars in entire house. Getting rid of the land-line covers the $20/month unlimited usage (and that gets you $100 rebate).

<comment><USER9> I'll give you a better idea: Instead of spending $149 to pay for an AT&T femtocell, buy an Ooma VoIP phone for about the same money, plug it into your cable modem, and never pay for a land line again. I bought Ooma 2 years ago and it paid for itself in about 6 months of phone bills.

<comment><USER18> If it's for the same money, I'd prefer only having one number. I love the fact that my cellphone is My Phone no matter where I am.

<comment><USER9> Google Voice is your friend...</comment></comment></comment></comment>


I never realized how useful netcat is - <AUTHOR> http://en.wikipedia.org/wiki/Netcat#Examples <comment><USER2> Not sure if I should share this, but here it goes.

One nice trick I used a lot when in college is that some postscript network printers would print raw postscript sent to them on a specific port (which I can't remember sorry). This allowed me to use netcat toprint my documents bypassing the college's print quota system and everyone who would be waiting in the queue.

<comment><USER3> Usually it's port 9100. Bash (and for sure also tcsh or zsh, not to start any flamewar) has a nice builtin tcp-socket interface by redirecting to the "pseudo" tcp device, so you don't even need nc for it on most current unices.

cat file.ps >/dev/tcp/10.2.3.4/9100

<comment><USER4> Wow, I had no idea about the bash device. This is great.

<comment><USER5> Note that it's disabled in Debian because the package maintainer has some stupid notion that it's a security hole.

<comment><USER6> Which is epically obnoxious when you have an open shell on a dying remote box that's lost /lib/libc.so.6 and you're scrounging for a way to restore it. I was halfway through writing a uudecode implementation using only bash builtins when the hosting company got me a serial line on the box.

I despise their policy of diddling with every upstream package -- disabling default build options, removing config files, separating the peas from the carrots, splitting pieces off into new packages, and all that anal-retentive bullshit. It's much worse on systems where there are fewer people keeping an eye on the maintainers -- ever try to use fink?

<comment><USER5> Well, Debian is the worst-maintained distribution out there, except for all the other ones.

<comment><USER6> That's just the problem -- they do massively more _maintenance_ than any other software project ever to exist.

They waste nearly all of their effort doing the fussiest of tasks. The root issue is the pointless 'version' freezes -- they end up forking _every single_ upstream project, being careful to never backport features (only 'bugfixes', which are never really so when isolated from context), and aim for a zealous consistency in the wholesale removal of any features which conflict with their ideology.

I used to get pissed off when the Debian people quoted astronomical figures about how many LoC are 'in Debian', but I got over it -- they really do wear the _hair shirt_.

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> I recommend the even more capable Ncat (<http://nmap.org/ncat/>), which is a reimplementation of netcat by "The Nmap Project".

<comment><USER8> I recommend checking out socat (<http://www.dest-unreach.org/socat/>) as well.

<comment><USER9> I recommend checking out lolcat, too (<http://en.wikipedia.org/wiki/Lolcat>).

<comment><USER10> I actually thought you were recommending a piece of software that had a punny name...

</comment></comment></comment></comment><comment><USER11> Indeed. I threw together this dead simple HTTP server written in shell + netcat the other day, for debugging some CGI scripts:

<http://github.com/tlrobinson/wwwoosh/blob/master/wwwoosh.sh>

(only handles CGI scripts at the moment, could be modified to serve files, etc)

<comment><USER12> Nice, that's really cool :)

</comment></comment><comment><USER4> netcat + ssh's -R and -L = unstoppable

ssh -D is also awesome for when you need a quick http tunnel.

Another trick I picked up is basically:

      nc -l [port] | named_pipe | nc [other_host] [port]

That will tunnel things between two netcat instances, effectively setting up a tunnel. Replace 'named_pipe' with a shell and you have a quick and dirty remote shell.

<comment><USER13> Can you explain this further. There are three machines A,B,C A can connect to B and B can connect to C but A cannot connect to C How can netcat/ssh be used to forward a port on B such that when A connects to that it connects to C [on a specific port]. basically B acts as a proxy.

<comment><USER4> This is from memory so take it with a grain of salt:

2048 is the port you're forwarding.

Machine B:

      machine_b$ mkfifo /tmp/tunn
      machine_b$ nc -k -l 2048 < /tmp/tunn | nc [machine_c] 2048 > /tmp/tunn &

You should now have a working tunnel. Everything sent to B on 2048 will be sent to C:2048

</comment></comment></comment><comment><USER14> Then just wait until you discover ngrep! ;-)

</comment><comment><USER15> NC allows good quick hacks but beware that it could open security issues as well. e.g. never do "nc -l -p 1234 -e /bin/bash" beyond your local machines.

</comment><comment><USER16> I discovered some awesome in Netcat while playing with BashReduce (<http://github.com/erikfrey/bashreduce/>).

That said BashReduce uses the Debian version of Netcat which is different from every other version I could find. I ended up running Debian in VMWare to get BashReduce up and running on my Mac.

</comment><comment><USER17> I like using netcat together with dd for cloning hard disks or volumes over a network. Great for cloning laptops, especially out and about, where it's often nontrivial to attach both drives to the same system simultaneously. Inserting gzip/gunzip into the pipe helps over slow connections.

</comment><comment><USER18> Last.fm has a pretty nifty notification system that uses netcat and an IRC bot. <http://www.metabrew.com/article/how-we-use-irc-at-lastfm/>

</comment><comment><USER19> One of my favorites for quickly copying a directory between hosts is...

dsthost# nc -l 1234 | tar -xvzf - -C .

srchost# tar -xzf - . | nc dsthost 1234

<comment><USER20> Heh, I used to use rsh (or was it rlogin?) for the same purpose back in the day, before rsync came along.

<comment><USER21> and we used to use ttcp (for it was more efficient with slinging around large files.)

</comment></comment></comment><comment><USER22> In most of those examples, you'd use telnet or scp or nmap.

<comment><USER23> yes that is the point. power == doing what would take 3 other programs to do.

<comment><USER22> , poorly.</comment></comment></comment>


YC's list of organizations helping to improve racial justice - <AUTHOR> https://docs.google.com/spreadsheets/d/1hHGenlt9lBrM_56ueSPY7gCR0rTEdqcUKNmAwfOYqqg/edit#gid=0 <comment><USER2> What makes this YC's? I don't think these are all YC companies and I don't see anywhere that a YC employee is curating this list.

<comment><USER3> Announced on YC's blog: [https://blog.ycombinator.com/next- steps/](https://blog.ycombinator.com/next-steps/)</comment></comment>


Neglected machine learning ideas - <AUTHOR> https://scottlocklin.wordpress.com/2014/07/22/neglected-machine-learning-ideas/ <comment><USER2> I think this article exemplifies the difference between the kinds of things you see codified in books and on the internet, versus what's active research and well known folk lore in academia. And maybe it highlights the substandard search mechanisms for published research, or the difficulty of learning from published research papers. But it's definitely not about neglect, at least not for most of the topics listed in the article.

For example, the recent AAAI 2014 conference had a bunch of papers on online algorithms for various problems. [1] Likewise, COLT had five or so papers on online learning. [2] Same with KDD [3], SODA [4], and the many other conferences this year that accept papers about ML.

And learning in the presence of noise? Unsupervised learning? Feature engineering? I am literally doing multiple research projects in all of these areas right now! The only way I can imagine that you think they're neglected is that you just don't know where to look for them, because these topics are all over the place in my world. For example, one common term for "feature engineering" is "representation learning," and this was a big topic at this year's SDM conference, specifically w.r.t. data mining in networks.

Why can't you find a book you like for topic X? Maybe it's because researchers have little incentive to write books. You folks in industry could fix that. What with all your ridiculous market valuations of various mobile apps, surely you could scrape together enough funding to convince the experts in their field to write a book.

[1]: [http://www.aaai.org/Conferences/AAAI/2014/aaai14accepts.php](http://www.aaai.org/Conferences/AAAI/2014/aaai14accepts.php) [2]: [http://orfe.princeton.edu/conferences/colt2014/the- conferenc...](http://orfe.princeton.edu/conferences/colt2014/the- conference/accepted-papers) [3]: [http://www.kdd.org/kdd2014/program.html](http://www.kdd.org/kdd2014/program.html) [4]: [http://www.siam.org/meetings/da14/da14_accepted.pdf](http://www.siam.org/meetings/da14/da14_accepted.pdf)

</comment><comment><USER3> One often neglected idea is transduction. The idea is that you can learn to label data better by augmenting your labeled dataset with unlabeled data.

The unlabeled data helps discover the structure of the data which in turns helps the supervised learning.

There is a lot of 2-phase approach to this problem, by first discovering feature with unsupervised learning and then using those features in supervised learning, but in many cases it's possible to do both at the same time.

If you have a generative model for your data, you can treat the labels as potentially missing values and learn the joint distribution.

<comment><USER4> Afaik what you're talking about is more commonly referred to as "semi- supervised learning". Transduction is a more specific case of semi-supervised learning where you know the test set, i.e. you know which data points the model will have to make predictions for. That means you can exploit this data for training the model, for example by unsupervised pre-training or with a pseudo-labeling approach.

<comment><USER3> Bayesians don't need a test set 8)

<comment><USER5> Why is that?

<comment><USER3> Because the prior on your parameters smooths out the prediction.

Most cookbook techniques such as ridge regressions, cross-validation, etc have a Bayesian interpretation as a prior on the parameter.

Bayesian techniques allow you to use all the data available.

That said, sometimes they are computationally expensive, and it's better to approximate them by using a test set.

<comment><USER6> Unless you have an infinite regress on priors for your priors, and uncomputable Komolgorov penalties on the structure of your model, I think you need a test set. (This means you need a test set.)

<comment><USER7> There's an interesting chapter in MacKay's book on Occam's razor. I'm not sure how I feel about it, but it's very thought-provoking.

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> These are great topics that are often neglected by machine learning textbooks. Some of the reason has to do with machine learning textbook writers not really doing research in Reinforcement Learning or Time Series. For things like Online Learning the author cites a great book but nothing for a more mainstream audience has been written yet.

Much of this stuff is being actively worked on though. If I could give one practical tip. Read KDD conference papers. Those are very applied and usually very accessible demonstrations of what techniques are out there, what problems they are typically applied to and importantly how well they worked.

Excellent post.

</comment><comment><USER9> Really good summary. I find it interesting that Reinforcement Learning, Online Learning, and Time Series modeling are all in the neglected category. They are all methods that seem to fit very well in both autonomous robotics and finance. I would venture to guess that they aren't really neglected, just hidden behind the firewall of Intellectual Property.

<comment><USER10> I agree. I wouldn't consider online learning neglected. The article mentions vowpal rabbit which is being used at Microsoft

</comment></comment><comment> dj-wonk > "online learning" ... and the subject is unfortunately un-googleable

That's not a surprise; "online learning" is a pretty misleading name. Here are some better alternatives:

      * incremental learning / training
      * stream-based learning / training

So, let's use these terms instead -- then it will be easier to find in your favorite search engine.

Here is one example using the term "stream-based learning":

[http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.1.79...](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.1.7903)

<comment><USER11> I wouldn't say its "misleading", it is a correct usage of the word online (especially contrasted with offline learning) and is the name of the sub-area. You won't find many (most?) papers in the area with those search terms unfortunately.

<comment> dj-wonk These so-called "online" techniques do not require being online (as in connected to the internet); they simply require being exposed to a stream of ongoing training data. They learn incrementally. This is why I say the term "online" is misleading.

See: [http://en.wikipedia.org/wiki/Online_machine_learning](http://en.wikipedia.org/wiki/Online_machine_learning)

> Online machine learning is a model of induction that learns one instance at > a time.

<comment><USER12> Online learning existed before the internet, dammit.

</comment></comment></comment></comment><comment><USER13> This is actually a great overview/review of some of the things that I've encounter regularly at work, but never seen discussed seriously in a book or class.

</comment><comment><USER14> Hierarchical Temporal Memory based algorithms such as the Cortical Learning Algorithm (CLA) implemented by NuPIC [1] should also be on this list (unsupervised, online, realtime)

[1] [http://numenta.org/](http://numenta.org/)

<comment><USER15> I think most people would classify those as AI rather than machine learning.

</comment></comment><comment><USER16> I would love to see more books/articles/blogs on unsupervised learning and ensemble techniques. E.G. - can I use k-means clustering as input to train a naïve Bayes classifier?

<comment><USER9> Interesting...I should write a blog post about some techniques I've used that are similar to your example (modulo specific algorithms). Is there something specific you are trying to accomplish?

<comment><USER16> I would read it! I work with a specific kind of high(er) dimensional medical imaging data, and I think unsupervised learning could be used for classification and foreground separation. K-means is giving me some promising preliminary results, but I'd like to assign samples continuous probabilities rather than binary classifications. I'm relatively new to ML but trying to incorporate it into my research, so I apologize if any of that doesn't make sense!

<comment><USER17> K-means clustering has been successfully used to extract features in a "deep learning" style architecture (with good results at image recognition). You'll probably find this useful: [http://web.stanford.edu/~acoates/papers/coatesng_nntot2012.p...](http://web.stanford.edu/~acoates/papers/coatesng_nntot2012.pdf)

</comment></comment></comment></comment><comment><USER18> A lot of these ideas, with the possible exception of conformal prediction, have quite a lot of academic literature. They are hardly neglected.

<comment><USER19> I've scanned a lot of literature on various AI fields over the last few years (And Jeesh, we should say AI if we're talking getting superb, actually- intelligent algorithms as opposed to the work-a-day, reliable algorithms that "machine learning" arguably already has).

I would contend that there can be a "significant" seeming amount of literature on field X but field X may still wind-up not pursued in the larger scheme of things.

Often what happens is a single individual or small circle, gets interested in a given field and researches it among things for as long as the funding persists and then once the funding dries up they move on. Or one person has tenure, keeps researching but everyone else moves on because it doesn't look like a way to keep getting funding.

Even more, as the author mentions, a big question is what approaches are taught as _the way to do it_ (and I guess it again comes down whether you're aiming for just machine-learning/a-better-heuristic-statistics-for-big-data or if you are aiming for moving towards intelligent algorithms, even if intelligence means just flexible adaptivity).

Yes, you can find lots of results if you search for "online learning", say. Otoh, for whatever given algorithm that has mindshare currently, is there a quest to find an online version? My sampling of the literature says no and I happen to agree with the author that online processing could be an important piece of artificial intelligence advances.

</comment></comment><comment><USER20> Was anyone else taken with the images interspersed throughout that blog post? Does anyone know where they are from?

Edit: found them thanks to Google image search: [http://www.darkroastedblend.com/2014/01/machines-alive- whims...](http://www.darkroastedblend.com/2014/01/machines-alive-whimsical- art-of-boris.html)

<comment><USER21> I saw a nice attribution at the end of the post. (But possibly it was added belatedly, e.g. after seeing your comment.)

<comment><USER22> > Images by one of my heroes, the Ukrainian-American artist Boris > Artzybasheff. You can find more of it here.

Agreed, very nice attribution. It wasn't added in response, I noticed it 8 hours ago (last night for me).

</comment></comment></comment><comment><USER23> The invisible hands re-ranking HN posts continue to astound me with their reach..

<comment><USER24> We're experimenting. Expect to see more of this.

[https://news.ycombinator.com/item?id=8122403](https://news.ycombinator.com/item?id=8122403)

<comment><USER23> Interesting, I'm glad to see more thought applied to rankings. What kinds of posts are you hoping to see more often?

<comment><USER24> Substantive posts that would otherwise have fallen through the cracks.</comment></comment></comment></comment>


Sploj: Project Euler meets Brainf*ck - <AUTHOR> http://www.spoj.pl/

<comment><USER2> I'm not sure what the submitter implies by "Project Euler meets Brainfck". While some "fun" problems on _SPOJ_ involve coding exclusively in esoteric languages such as BF and Whitespace, most others are the standard algorithmic challenges. This site is pretty well-known in the programming contest community.

<comment><USER3> That, and the title is typo'd.

<comment><USER2> Hence my italicization in the grandparent comment.

</comment></comment></comment><comment><USER4> "Project Euler meets Brainf*ck" is a pretty terrible description. It looks like a website with programming problems to solve, similar to Project Euler, but with less of a direct emphasis on math.

It also looks like you submit your source and the site operators test it, and there are time limits.</comment>


AWS Lambda now supports Synchronous Invoke for scalable mobile back ends - <AUTHOR> http://aws.amazon.com/blogs/aws/aws-lambda-update-production-status-and-a-focus-on-mobile-apps/

<comment><AUTHOR> It's kinda buried in all the announcements today but before this AWS Lambda could only be used for asynchronous calls. So if you were developing a mobile backend you could only process events asynchronously and had no way other than out of band push notifications to send data back to the client.

Now with Synchronous Invoke you can have a backend for a mobile app live fully on AWS Lambda. The benefit being you pay by the millisecond for compute time used vs having to keep EC2 nodes up and paying for underutilized instances.</comment>


Ask HN: What should I do with my side project? - <AUTHOR> A few years ago, I developed & launched www.last10k.com as a side project while working full time to fulfill a need for individual investors to easily use SEC Filings / financial disclosure documents for researching stock investments.
Over the years I added some unique features including sentiment analysis but aspects in my personal life have now changed where my job + traffic + being newly married is where my time is spent now so not much has been put into the site lately.
The site is fully automated and is receiving just under 3K users a day via SEO with an average time on page at 5 min. It used to make over $1K/month in ad sense revenue but Google just started penalizing us for invalid click activity but the site could use a redesign anyways, particularly mobile.
What are your thoughts on what I should with this side project? Should I leave the site as-is, sell it, quit my job (which I'm not happy with) and re-launch it or just work on something else? Any insight on this is appreciated. Thanks! <comment><USER2> Well... $1K/month probably isn't enough to live on, so the question I would have would be "what can you do to scale the thing up, or find additional ways to monetize it"? And does the answer to that get you to a place where it makes enough money to support you? If so, I'd be thinking hard about the "quit and work on it full-time" thing, although that would obviously depend on a lot of other factors (savings, etc.)

Other things I'd be asking:

Do you have other features or aspects in mind that could make the site even more useful / appealing? Are you planning to say purely ad-supported, or do you have any kind of subscription model in mind? Are there any useful partnerships you could explore? Is an "acqui-hire" scenario a possibility?

<comment><AUTHOR> Thank you for your quick reply! Yes, I do have a few features in the backlog such as comparison functions but it will take some time to develop. I was also thinking of sticking to the ad-revenue model as similar sites are paid subscription-based.

In terms of the job, the company I'm at just had layoffs so working on the site instead of looking for another job crossed my mind. Thankfully savings are plentiful but I don't want to burn through it.

To generate revenue to make a living would require scaling the site to at least 5 times daily volume it's at now so was thinking about investing in marketing. There are some business development opportunities I would like to pursue as well but I have found doing that with my day job difficult.</comment></comment>


Ask HN: What do you use to draw graphs? - <AUTHOR>

In this case I mean plots, not vertex-edge type graphs.
I don't often have to draw plots, and when I do they don't usually have to be very complication.  Sometimes I just drop into Excel (or the LibreOffice equivalent) and plot things there[0], sometimes I use python to generate ppm files "by hand" and then use "convert" to convert them into png files[1].
What do you use? <comment><USER2> I use Seaborn [-> 0, 1] for both purposes from iPython notebook. I found it makes for good publishable graphics that can be fine-tuned. I usually use my own color palette.

[0, 1] [http://stanford.edu/~mwaskom/software/seaborn/](http://stanford.edu/~mwaskom/software/seaborn/)

</comment><comment><USER3> Ploticus[1].

SVG examples: [http://backup.frottage.org/rjp/owngoals.svg](http://backup.frottage.org/rjp/owngoals.svg) [http://backup.frottage.org/rjp/tmp/gryntemp.svg](http://backup.frottage.org/rjp/tmp/gryntemp.svg) [http://backup.frottage.org/rjp/tmp/sunrisesunset.svg](http://backup.frottage.org/rjp/tmp/sunrisesunset.svg)

[1] [https://en.wikipedia.org/wiki/Ploticus](https://en.wikipedia.org/wiki/Ploticus)

</comment><comment><USER4> I see [http://www.highcharts.com/](http://www.highcharts.com/) come up now and then.

D3.js is a great tool for data visualization, but that might be an overkill if your needs are simple.

<comment><USER5> There is also dimple.js [0] that is built on top of d3 and very easy to use, or TauChart [0], that was submitted 3 weeks ago on HN. I never used TauChart but I will give it a shot next time I need to do some graph because it looks really great.

[0] [http://dimplejs.org/](http://dimplejs.org/)

[1] [http://blog.taucharts.com/taucharts-data-focused-charting- li...](http://blog.taucharts.com/taucharts-data-focused-charting-library/)

</comment></comment><comment><USER6> For web, d3.js[0] is a great choice. It's used widely and nicely documented. [0][http://d3js.org/](http://d3js.org/)

</comment><comment><AUTHOR> Examples:

[0] [http://www.solipsys.co.uk/new/SpamGraphs.html](http://www.solipsys.co.uk/new/SpamGraphs.html)

[1] [http://www.solipsys.co.uk/new/TalksInSpaceTime.html?HN_20150...](http://www.solipsys.co.uk/new/TalksInSpaceTime.html?HN_20150819)</comment>


DIY Computer Science Degree - <AUTHOR> https://github.com/iamryandrake/DoItYourselfCSDegree/blob/master/DegreeOutline.md

<comment><USER2> I think this list is missing some important parts of computer science.

Here are the books that our university uses for first-year students combined with books that I found to be useful:

Introduction to Programming (using Eiffel) [1]

Mathematics for Computer Science (or: Discrete Mathematics) [2]

Introduction to Datastructures and Algorithms [3]

Introduction to Digital Design [4]

Parallel Programming (using Java) [5]

Optional but highly recommended, you'll probably find it completely out of scope:

Real Analysis I [6]

Real Analysis II [7]

Introduction to Linear Algebra [8]

Introduction to Physics [9]

[1]: [http://www.amazon.com/Touch-Class-Learning-Program- Contracts...](http://www.amazon.com/Touch-Class-Learning-Program- Contracts/dp/3540921443/ref=sr_1_3?ie=UTF8&qid=1396688827&sr=8-3&keywords=touch+of+class)

[2]: [http://ocw.mit.edu/courses/electrical-engineering-and- comput...](http://ocw.mit.edu/courses/electrical-engineering-and-computer- science/6-042j-mathematics-for-computer-science- fall-2010/readings/MIT6_042JF10_notes.pdf)

[3]: [http://www.amazon.de/Introduction-Algorithms-Thomas-H- Cormen...](http://www.amazon.de/Introduction-Algorithms-Thomas-H- Cormen/dp/0262533057/ref=sr_1_1?ie=UTF8&qid=1396689336&sr=8-1&keywords=introduction+to+algorithms)

[4]: [http://www.amazon.com/Digital-Design-Computer- Architecture-E...](http://www.amazon.com/Digital-Design-Computer-Architecture- Edition/dp/0123944244)

[5]: [http://www.amazon.com/Java-Concurrency-Practice-Brian- Goetz/...](http://www.amazon.com/Java-Concurrency-Practice-Brian- Goetz/dp/0321349601/ref=sr_1_1?ie=UTF8&qid=1396689424&sr=8-1&keywords=java+concurrency+in+practice)

[6]: [http://www.amazon.com/Analysis-Texts-Readings-Mathematics- No...](http://www.amazon.com/Analysis-Texts-Readings-Mathematics- No/dp/8185931941/ref=sr_1_1?ie=UTF8&qid=1396689443&sr=8-1&keywords=terence+tao+analysis)

[7]: [http://www.amazon.com/Analysis-II-Texts-Readings- Mathematics...](http://www.amazon.com/Analysis-II-Texts-Readings- Mathematics/dp/818593195X/ref=sr_1_1?ie=UTF8&qid=1396689499&sr=8-1&keywords=analysis+ii+terence+tao)

[8]: [http://www.amazon.com/Introduction-Linear-Algebra-Fourth- Gil...](http://www.amazon.com/Introduction-Linear-Algebra-Fourth- Gilbert/dp/0980232716/ref=sr_1_1?s=books&ie=UTF8&qid=1396689522&sr=1-1&keywords=gilbert+strang)

[9]: [http://www.amazon.com/Fundamentals-Physics-Extended-David- Ha...](http://www.amazon.com/Fundamentals-Physics-Extended-David- Halliday/dp/0470469080/ref=sr_1_1?ie=UTF8&qid=1396689544&sr=8-1&keywords=physics+halliday)

<comment><AUTHOR> Thanks so much for this list! What's your Twitter handle I can follow you on?

Have added this list to my project!

<comment><USER2> Unfortunately I don't use Twitter anymore. Good luck with your journey!

</comment></comment></comment><comment><USER3> If you want something closer to what you'd expect of CS from a college check out MIT OpenCourseware, it should have everything you need for undergrad CS

<comment><AUTHOR> Yep, working on incorporating more of those videos and course notes into my own structured plan! Thank you :)

</comment></comment><comment><USER4> A bit of advice from someone who is trying to do the same:

\- Learn one language/ platform at a time. Also, think about what you want to learn. Web development? Mobile? CS fundamentals? It is very easy to waste alot of time when your are not focussed on a particular goal.

\- Try not to get tricked into spending to much time on those fancy looking tutorials. Building something yourself is much more valuable than obtaining meaningless 'badges' and achievements.

\- Ruby on Rails is complex and I would not recommend it for beginners. I wasted alot of time doing tutorials that covered random parts of rails that I never used in projects. Again, if you want to learn Rails, start your own project as soon as possible and select learning materials based on that.

\- Stick with the basics. Try not to get distracted by frameworks, tools and the overwhelming amount of tutorials available.

for javaScript, have a look at this:

[http://javascriptissexy.com/how-to-learn-javascript- properly...](http://javascriptissexy.com/how-to-learn-javascript-properly/)

Ruby / Python:

[http://learncodethehardway.org/](http://learncodethehardway.org/) [https://www.udacity.com/course/cs101](https://www.udacity.com/course/cs101)

rails [http://guides.rubyonrails.org/getting_started.html](http://guides.rubyonrails.org/getting_started.html)

good luck

<comment><AUTHOR> Agreed Thomas! Thanks for your input.

I am learning for the sole intent of then being able to immediately put into action what I've learnt within the startup environment and also contribute to open source projects where I can :)

What are you learning?

</comment></comment><comment><USER5> Programming != CS.

<comment><USER6> This. After finishing a BS in Maths, and with coding experience, I started a degree in Computer Sciences, thinking that I would know the most of it because I was coding since I was 15.

The truth is nope. You see a lot of things besides coding, with seems to be more important than to do a a while, compile a program or use a library. You learn the WHY, which most of the time is more important than the HOW.

<comment><AUTHOR> That's totally true.

I agree, so I'm also going to do some of the MOOC CS courses as well, coupled with working at a startup.

I'm trying :)

</comment></comment></comment><comment><USER7> This comes across as more DIY Web Dev.

CS degrees are not so much about programming, as some people have stated. More so theoretical understanding, logic(which is important IMO), best practices on projects, approaches(agile etc.). The programming that is touched on will be an intro to an OO language. Generally, most CS students are expected to learn programming in their own time on their own accord in order to complete assignments etc.

</comment><comment><USER8> I am 3,000 hours into the DIY phase myself, so I appreciate your list.

I think Effective Objective-C and Effective JavaScript (as well as the "Good Parts" and "Secrets of a JS Ninja") were good books.

Also, the C book, since it is difficult to understand Objective-C without a firm grip on C. Although, there are certainly more recent and easier books on C to read.

<comment><AUTHOR> Thank you!

Which book is "THE C Book" you refer to?

And how have you enjoyed your DIY journey so far?

<comment><USER8> I'm sure it would be easier if I had a mentor or at least a friend...but I'm a loner so I'm taking my lumps the "hard way". It's been cool I love learning this even at my advanced age.

</comment></comment></comment><comment><USER9> Not even a mention of Data structures and Algorithms

<comment><AUTHOR> Depends on what you ultimately end up doing at the end of your study, I suppose!

I'm going to go through the MIT OpenCourseware stuff on this to begin with.

What have you found most helpful? Do you deal with algorithms everyday?

<comment><USER10> The point is that you are conflating "slinging code" with "computer science". To paraphrase an old truism, you're talking about pointing a telescope and we're talking about astronomy.</comment></comment></comment>


Leading journalists join call for EU copyright reform - <AUTHOR> https://phys.org/news/2018-08-journalists-eu-copyright-reform.html <comment><USER2> This seems to be the actual letter: [https://cyprus- mail.com/2018/08/28/neighbouring-rights-a-que...](https://cyprus- mail.com/2018/08/28/neighbouring-rights-a-question-of-life-or-death/)

Which is very light on policy and heavy on the emotion.

If any form of link tax is passed you can probably expect the same result as in Spain where internet orgs simply refuse to link to content that it applies to and shutdown products like Google News that cannot function without the content.</comment>


Public APIs: A collective list of free APIs - <AUTHOR> https://github.com/toddmotto/public-apis <comment><USER2> There is a version of the internet of APIs that I wish came to fruition, where it was a series of pipes of raw, live information that you could use to display interesting things about the world or ingest into your own software. Think a personal dashboard of easy to set up and reliable tidbits of information from all around the web.

APIs tend to be hyper specialized and actually far more about using the platform within the platform boundaries rather than interconnecting different APIs or interfaces and making a modular program.

I tried to build the "display live information" part of the idea but I found that it takes a really specific type of information to make it interesting and there are very few free APIs that provide that kind of thing. It needs to be "live" data in chronological order, and it needs to be about the world, not about some platform. For example "Number of astronauts currently in orbit" is fantastic and fun to show in a live graph. "Stars on my github repo" is kind of cool too, if you have a popular repository. There just aren't many free endpoints like that.

</comment><comment><USER3> Aside: why do listicles need a sponsor? According to Github, 452 people have contributed to the list. Are all of them getting some kind of share of the sponsorship dollars?

<comment><USER4> I wondered why too, so I reverted the commit that added the add, pulled in some PRs, and pushed it here. I plan to keep it up to date if there's any interest. [https://notabug.org/anomie/public- apis](https://notabug.org/anomie/public-apis)

<comment> Anthony-G Thanks for doing something about this. I've bookmarked your fork of the project for future use. That ad was obnoxious and completely gratuitous – unlike the more tasteful Digital Ocean logo at the bottom of [https://api.publicapis.org/](https://api.publicapis.org/) and the README page for [https://github.com/davemachado/public- api](https://github.com/davemachado/public-api) In this case, it's justifiable as it's clear that Digital Ocean provide hosting for operating the API server.

</comment></comment></comment><comment><USER5> I used to love the Web and APIs. Still think it's great.

But I think it's far superior to download software (Merkle trees) onto your own computer automatically and run it in a sandbox with your own data, while doing end to end encryption. That way you don't worry if the API goes down or starts charging more or the database is hacked and you don't have to jump through holps to get 1000 keys for each new project. Also you can resell your project to regular people who can host it themselves without them jumping through all those hoops. And finally, hosting should consist of provisioning space in a giant abstract cloud that holds encrypted data, no need to set up Linux instances or whatever.

Which platforms are doing that?

<comment><USER6> If you get rid of "merkle tree" requirement, then you have just described database files hosted on github.

It does plenty of advantages -- it scales good and if done right, it is immune to hacking or site going down.

But it has some disadvantages as well -- the biggest is that client needs to do non-trivial, complex work, so you cannot just insert AJAX call onto your webpage. Also, you lose all control of data, so a competitor can trivially clone your service.

<comment><USER5> Well yeah, I am not talking about Capitalism in the traditional sense anymore (business builds a moat, provides a service to you and charges fees). I am talking about a collaboration economy (open source, science style). You may still be able to sell stuff, but not the stuff whose marginal cost to copy is zero. Information would be free. Asked who would generate it, the answer is increasingly "everyone". News and Maps can be crowdsourced (see openstreetmap for example). The hard part is in the consensus model, ie data sets will be honeypots for people who want to alter the "truth as we know it" (like Stephen Colbert and Elephants on Wikipedia). I am not convinced that a for-profit top- down corporation is necessarily better for that than a flat, democratic crowd, provided we have interesting game mechanics built in (like high stakes challenges where you can lose a significant portion of your accumulated reputation if you don't win by a large margin).

</comment></comment></comment><comment><USER7> Aparently there's no free NFL results API :-(

<comment><USER8> [https://www.thesportsdb.com/league/4391](https://www.thesportsdb.com/league/4391)

<comment><USER7> thanks</comment></comment></comment>


Square says its employees might get too rich - <AUTHOR> https://www.washingtonpost.com/news/the-switch/wp/2015/10/14/square-says-its-employees-might-get-too-rich/ <comment><USER2> This headline is not pointing to super unusual language in these filings - look at Twitter's S-1, for example: "Our growth strategy also depends on our ability to expand and retain our organization with highly skilled personnel. Identifying, recruiting, training and integrating qualified individuals will require significant time, expense and attention. In addition to hiring new employees, we must continue to focus on retaining our best employees. Many of our employees may be able to receive significant proceeds from sales of our equity in the public markets after this offering, which may reduce their motivation to continue to work for us." [http://www.sec.gov/Archives/edgar/data/1418091/0001193125133...](http://www.sec.gov/Archives/edgar/data/1418091/000119312513390321/d564001ds1.htm)

And Alibaba's F-1: "In addition, we have a number of employees, including many members of management, whose equity ownership in our company could give them a substantial amount of personal wealth following our initial public offering. As a result, it may be difficult for us to continue to retain and motivate these employees, and this wealth could affect their decisions about whether or not they continue to remain with us. If we are unable to motivate or retain these employees, our business may be severely disrupted and our prospects could suffer." [https://www.sec.gov/Archives/edgar/data/1577552/000119312514...](https://www.sec.gov/Archives/edgar/data/1577552/000119312514184994/d709111df1.htm)

Etc.</comment>


TicketsNow Circa 1999 - <AUTHOR>

http://web.archive.org/web/19990117003816/http://www.ticketsnow.com/
---
As many of you know, tickets now was acquired Ticketmaster for $265 million in Feb of this year.
What many of you may not know that it was started with only $1000. What started as a one-man operation in a one-bedroom apartment has grown into a multimillion dollar business with approximately 300 full-time employees.
Look at how fucking ugly that first release was, back in 1999. Only persistence and dedication to a vision would have made that company successful, and that's exactly what happened.
To all fellow entrepreneurs out there, just release. Perfection may follow speed, but speed never follows perfection.
Just a dose of inspiration for y'all :). <comment><USER2> Or maybe the goodness of their website wasn't an important facet of their business a decade ago...

<comment><AUTHOR> lol. In 1999 I am sure not many websites looked any good, the "goodness" you speak of is relative. Take a good look around. In 2019 none of the websites you visit today will look any good either.</comment></comment>


Show HN: Simple end-to-end email testing - <AUTHOR> https://testmail.app <comment><USER2> Webpage blank without JavaScript enabled.

<comment><AUTHOR> It's built with React - requires JavaScript.</comment></comment>


How the Web OS has begun to reshape IT and business - <AUTHOR> http://blogs.zdnet.com/Hinchcliffe/?p=771

<comment> old-gregg Ever wondered how "business types" who sell enterprise software come up with impressive looking "bullshit diagrams"? Yeah, those where a SQL Server instance becomes a "data cloud"?

This one certainly beats them all: it includes every single buzzword known to humanity and is shaped to resemble a flying saucer emitting religious looking beams from the middle:

<http://i.zdnet.com/blogs/web_os_2009_small.png>

And yeah, it features Web 3.0 too, just to make sure you're looking at the latest and greatest.

<comment><AUTHOR> Good points.

SaaS and the "Cloud" are definitely a growing phenonmen, but I think this adds more hype than information and that its prediction that the Cloud will take over all computing is a little premature to say the least.</comment></comment>