Predicting Stock Performance with Natural Language Deep Learning - <AUTHOR> https://www.microsoft.com/developerblog/2017/12/04/predicting-stock-performance-deep-learning/ <comment><USER3> This is not very convincing. It reads like a semester project for an undergrad or a summer intern. In particular, the prediction task used for evaluation is not very meaningful, and the intended use case (predicting biotech underperformers) is not very meaningful without predicting _by how much_ they would underperform, i.e. just using the output of this model as yet another regression factor with questionable significance in the end.

But in that case, it's very unlikely a heavy CNN text model would be better suited than simpler methods like a classifier using LSA vectors, or even just extracting the gloVe vectors and doing no convolution. Especially given all the metaparameters they mention needing to tweak.

For example, an ablation study to figure out that you need Lecun initialization seems both overkill and requiring way more specialization in deep learning than a typical firm would be looking to have, all just to squeeze out what _might_ be some slight efficacy in one industry group.

When I previously worked in quant finance, I used to be very passionate about trying to apply the latest & greatest methods.

But over time my feeling is most of it is inapplicable to finance, because it ends up being a lot of work for pretty much no efficacy, like in this post. I found much more value in simpler regression and tree models, and using simple bag of words models with text. The only "advanced" stuff that ever seemed to add significant efficacy was Bayesian hierarchical regression, but only for helping overcome limitations of classical random effects models, not for adding any greater complexity.

The post author deserves plenty of credit for skills in deep learning, but the _analysis_ seems unconvincing if it's supposed to market for using MSFT cloud for deep learning in finance.

<comment><USER4> Agreed. Also noted three potential flows:

1\. They applied pandas.qcut over the entire returns (i.e. training and validation set) when generating the target performance labels, which could compromise the validation.

2\. The ordering of news is actually important when it comes to forecasting the movements of the market; their training/validation split was done after shuffling the entire dataset, which means the model will have access to information that shouldn't be available to it during training.

3\. It makes more sense to use a training-validation-testing setting rather than a training-validation split when reporting the model performance in order to avoid inflated results due to all the hyper-parameter tuning.

<comment><USER5> What do you mean by training-validation-testing vs training-validation split? Also why would pandas.qcut compromise the validation?

<comment><USER3> If the quantile assignments took into account data from both the training set and the test set, then it means the training set quantile label itself contains lookahead information about what the distribution of test set values is.

This can lead to artificially increased test set accuracy since by optimizing the model during training, you are partially optimizing based on information you gained by knowing about the test set in advance.

Training-validation-testing generally means keeping a portion of the training set held out for evaluating accuracy and convergence after one full iteration of training updates. So you get some information per each training update regarding whether the model had overfitting, or performs unrealistically well on the validation set (which it's not trained on), or for early stopping criteria.

Then, after all that, you move on to checking the final accuracy and metrics on a fully held out test set.

This validation approach is common in deep learning because of the many diagnostics you need to get information about during training. Without some information outside the training set itself, it can be hard tonunderstand how the learning rate is affecting you, how likely overfitting is, whether there is a vanishing gradient problem.

Waiting until all the way to the end of training to get any feedback on these is sometimes just too inefficient or too risky that you'll discover an issue only after a huge time sink.

</comment></comment></comment></comment><comment><USER6> Methodology seems to be poor, like with most publications on finance.

1\. How did they split the data for train/validation/test sets? If all of the sets had the data from the same time period, or the same companies end up in multiple sets, it is a major flaw. For example, if outperformance was persistent for the companies over the time period considered, the model may simply learn to identify specific companies by their filings.

2\. What is the variance of the out-of-sample performance? Given that their dataset is very small, and the model performed badly at predicting high returns and reasonably well at predicting good returns, what are the chances of getting those results by luck alone?

3\. How has the model performed since then, on the most recent filings?

4\. Why use a convnet? Would gradient boosted trees not perform just as well/better but be more interpretable? Methods like the ones in the eli5 package can help get an idea of why the model makes a particular prediction, which could help sanity check the model.

<comment><USER7> 1\. Are you really assuming that researchers at MSFT screwed up the training and validation sets?? 4\. In what way is GBT more interpretable than an ANN??

<comment><USER6> I don't think that was MS researchers - I think they just publicised this bit of research.

With GBT, you can check why a particular prediction was made - roughly - <USER2> navigating down each tree for a particular sample and summing up the influence of each feature on the final score. Then you can see if something weird is having a large effect on your score. Can you do something similar with ANNs?

</comment></comment></comment><comment><USER8> They're attempting something fantastically difficult to do effectively (arguably beyond human capability) and arguing that their slightly-better- than-chance results indicate that their approach is promising. More likely, the system is picking up on some "low hanging-fruit" indicators in the data (perhaps that corporate filings with certain buzzwords are likely to fail, or that reference projects beyond a certain age, or similar) which would already be easily picked up on by human analysts, and that anything much beyond that would be impossible with anything remotely resembling the present method.

I want to posit a general hypothesis (perhaps it's already been said); better- than-chance performance by a classifier on some dataset is _not evidence_ that a similar (or any) classifier can perform better on the same data.

<comment><USER9> I was thinking about weather prediction.

Now climate is changing you see that all the models are becoming less and less acurate.

So yeah some models might be performing better but overall they are all performing worse.

<comment><USER10> Have there been any research into the decline of weather prediction models?

Sounds really interesting.

<comment><USER11> Weather models are _constantly_ being validated and improved. They are getting better as computation power and grid resolution have increased. The climate is changing but the laws of physics are not. Also, climate is different from weather.

</comment></comment></comment></comment><comment><USER12> A few years ago I spoke to a data scientist who, after a bunch of NLP work, had hit on a very simple technique for quickly identifying good news and bad news.

How to tell that it's good news: there are numbers near the top.

How to tell that it's bad news: the figures are buried as deeply as legally permissible.

<comment><USER13> Ironically, I figured the authors of the article weren't very impressed with their results since they weren't outlined in the summary...

</comment></comment><comment><USER14> I have found that if the earnings comments are easy to understand, the stock is going to go up. If there are lots of qualifiers and they invent new ways to measure: sales, costs and profit, then the stock is going down.

If they mention the weather during the last quarter - run!

<comment><USER15> Weather really is last refuge of scoundrels.

</comment></comment><comment><USER16> They lose a lot of information artificially classifying a continuous variable (% change) into 3 bins (high/medium/low) instead of trying to do regression on percent change. This incorrect practice is surprisingly prevalent in deep learning.

<comment><USER17> With financial data you might be better off separating out the side of the bet from the size of the bet and thus it makes sense to just make three bins. The features to use for direction and volatility are different.

<comment><USER18> Choosing bins for stock returns isn't easy. You could use two bins for positive vs. negative returns but any other thresholds are arbitrary and results could depend on the knowledge available when the threshold was chosen.

<comment><USER17> It's arbitrary if you make it arbitrary. Depending on the data you might target various time intervals to find the optimal way to trade on that information just like any hyperparameter and the bins are a function of that time, on average in a 30 minute period bitcoin moves by 0.2% on high liquidity exchanges.

<comment><USER18> But as I understand they used the whole sample to determine bins, including validation sets. That alone is a big no-go. I didn't mean that you can't find a proper way to choose bins but there is a lot of potential for errors.

And in the end you want to invest based on the model. Arbitrary bins aren't that helpful if some of them combine desirable and undesirable (e.g. positive and negative) results.

<comment><USER17> >whole sample to determine bins, including validation sets.

well thats not random or arbitrary just wrong.

>And in the end you want to invest based on the model.

I'm not quite sure what you mean here. Either you determine a model and hyperparameters (which includes the bins) is correct enough of the time via testing on out of sample data or even synthetic data or you are talking about a determining what to do given a single observation, which I would assume you give it to the model and ask it what it thinks and the bins are part of the determination (as done here in the article with softmax output) of what to do and given you've done the testing you should have a level of confidence about the outcome of acting upon the models output. The bins aren't a post processing step, the post processing you might do to trade recall for accuracy might be to expect the bin to have a stronger signal (class > 0.5 or something maybe and otherwise ignore), all of this is "the model".

>Arbitrary bins aren't that helpful if some of them combine desirable and undesirable results

Correct me if I'm wrong but it sounds like you are making the case that there are models and bins which would have only good outcomes (and those are the only useful ones)? Am I misunderstanding something here?

</comment></comment></comment></comment></comment></comment><comment><USER19> So if something like this came into common use, the obvious follow-on would be "earnings writers" who specialize in phrasing and vocabulary that tickles the AI. (It would be easy to practice, just feed your draft into the system, get a score, tweak some words, repeat.) So, reproducing the "SEO" frenzies of a decade ago.

<comment><USER3> This technique generally (applying NLP to 10K's and other corporate filings) has been in practice for many years. One common, highly commoditized text signal is earnings surprise: measuring sentiment in earnings call transcripts, and comparing sentiment scores with similar metrics from analyst publications in reaction to the earnings announcement.

The arms race effect you mention absolutely exists. Corporate officers are often coached extensively on key phrases to repeat in earnings calls, and phrases to avoid, and how to steer caller questions away from SEO-like speech patterns they don't want.

</comment></comment><comment><USER20> I truly wonder when we'll get some deep learning models running on Enterprise Exchange Servers trying to find meaning in the deluge of email's that is sent across all large corporates, building fancy Dashboard with "actionable" warnings.

I bet someone is going to make a fortune based on the pitch alone.

<comment><USER21> Weren't there some bots doing sentiment analysis on twitter? Also, these were discovered then exploited at one point to cause a rapid drop in the stock markets.

<comment><USER22> Any links with more info would be appriciated :)

<comment><USER21> [https://www.forbes.com/sites/davidleinweber/2013/04/24/so- mu...](https://www.forbes.com/sites/davidleinweber/2013/04/24/so-much-for- fund-mining-twitter-sentiment-for-picking-stocks-but-ok-at-the- sec/#590bdc2580df)

[http://www.slate.com/articles/business/moneybox/2015/04/bot_...](http://www.slate.com/articles/business/moneybox/2015/04/bot_makes_2_4_million_reading_twitter_meet_the_guy_it_cost_a_fortune.html)

[http://fortune.com/2015/12/07/dataminr-hedge-funds- twitter-d...](http://fortune.com/2015/12/07/dataminr-hedge-funds-twitter- data/)

</comment></comment></comment></comment><comment><USER23> Training Set includes, old stock data? How is that corelated with anything? Steam-Engines where spectacular stakes, until the car and eletricity came along.

I wouldn't train a AI on stakes- i would train it on history books/ old newspapers to find hidden needs and gaps. People do not use half of the day, due to darkness? People fighting, due to starvation, or reduced chances to mate and procreate?

A NN could find these needs, and hand it to a diffrent NN, that is trained to find solutions/ companys likely to develop solutions.

<comment><USER24> Except a NN could not find these needs. They are very poor at the understanding part of language.

</comment></comment><comment><USER25> It would be interesting alongside estimating general "consensus" on stock performance development using NLP processing, sentiment analysis etc. to utilize game theory to find a contrarian strategy utilized by a few players that has the highest probability of yield, being one step ahead of moving average of dominant expectations, and mastering equilibrium where transacted amount doesn't affect the dominant trend to keep surfing the wave as long as possible.

</comment><comment><USER26> related work from 2009 for those interested...

Financial forecasting using character n-gram analysis and readability scores of annual reports. by Matthew Butler and Vlado Keselj

</comment><comment><USER27> This feels like the type of thing that if widespread could be easily abused - at least in the short term. Something along the likes of SEO, but for SEC filings - used to boost a stock price in the short term.

<comment><USER28> Black Hat SEC?

There was a model to predict up votes on Hacker News based on title that made its rounds a way back. Fortunately we don't have a bunch of submissions with the title "YC YC YC YC YC YC golang is better than Rust" [1]

[1] [https://news.ycombinator.com/item?id=14400603](https://news.ycombinator.com/item?id=14400603)

</comment></comment><comment><USER29> This is really only my 2c and I've only really dabbled in btc/alt coin trading. In my experience, there is too much entropy generated by the collective masses transacting, to reliably predict most ups and downs without cheating or using some external "special" knowledge of the market in question. We will need to wait til the days of strong quantum computers before we can achieve that level of precision with ai.

<comment><USER30> By that point the other traders will also have quantum computers, so they'll be just as hard to predict.

</comment></comment><comment><USER31> This person was on this years ago! [https://medium.com/@symbols/mayweather-vs- pacquiao-the-fight...](https://medium.com/@symbols/mayweather-vs-pacquiao-the- fight-from-twitter-and- facebook-77bc547020b9?source=linkShare-4e17da25b976-1527807605)

</comment><comment><USER32> JP Morgan would have a nice quote for these things: "The Market Fluctuates".

I don't really think we can predict the stock market because the market fluctuates every day based on news and things like that.

Stock market is like a legal gamble, in the words of Wrren Buffet, "The intelligent earn money off the fools".

<comment><USER33> Warren Buffet has literally made billions by exactly "predicting the stock market" \- the thing that you write is "impossible". So what do you mean?

<comment><USER32> Well that isn't "prediction". He invests in sound businesses which are poised to grow. His investment returns used to take like decades to gove a good return.

He did not spit out " 100 stocks which will go up tomorrow using my fancy algorithm"

He researched the people behind the company the finances etx

Wildly different from what algorithms do

</comment></comment></comment><comment><USER34> I started doing this with 10k reports, the different formatting between different years was too much of a pain in the ass. They started embedding Excel documents in what I think is base64 into text files? I don't know. In the 90s the tables were in plain text.

<comment><USER5> Yes, XBRL (XML) files. However they only give you the financials. What OP wanted was management summary, risk factors, etc which still needs to be parsed. Luckily many companies now will put anchors to those sections in the HTML.

</comment></comment><comment><USER35> Here's a meta question about all such "predicting stock market performance using X"-type publications: if you have a method that does better than average, why are you publishing it, and not, say, investing all your money and making a killing?

<comment><USER36> This is a pretty common response so I think its worth pointing out that there are a lot of reasons why you might have a model you think might be able to predict the stock market but are not actively using it to make money.

* You may not be eligible to trade that instrument because of where you live or who you are

* You may have found an edge based on historical data in one instrument and expect it could work for another but lack the data necessary to confirm

* You may not have the capital necessary to trade at sufficient scale

* You may not have the programming skills to turn a mathematical model into an build and maintain a high-frequency trading bot with adequate risk controls (waaaaaaaaaay different skillset)

* You may have found a prediction who's error rate is within the transaction costs and thus needs to be traded at higher volume (lower tx costs) to confirm

And before anyone thinks "why not just negotiate to sell it to someone who does" they get hundreds of random people doing that every day so your chances of getting noticed are between slim and none.

Now, what is an effective way to benefit from a market prediction model is to publish it. Those people may hire you to continue to develop it further or to refine it for other markets you didn't think of. No model lasts forever, so having demonstrated the ability to find one makes it more likely you will do so in the future, thus should attract big money salaries from people with the capital for your next one.

Not to say that these are legit, just pointing out its not hand-wavvy dismissive is all. I wish authors would start a paper with something like this just to address this inevitable comment. "We aren't trading this because we can't trade SLURM futures" would really help readers out if they also can't.

</comment></comment><comment><USER37> The Rockwell Retroencabulator ...

<comment><USER38> which one, the original or the turbo?

</comment></comment><comment><USER39> I don't think this is very useful outside of short term fluctuations. At EOD analysis of the 3 financial statements (Balance sheet, cash flow, Profit/Loss) are what drives price.

<comment><USER40> It turns out you can make a lot of money off those short term fluctuations.

<comment><USER39> Yeah I mean that is true. Almost had an edit to mention that.

</comment></comment></comment><comment><USER41> Anyone have experiences with Azure? Nearly everyone I see personally in industry is doing their research on AWS. This article is obviously trying to market Azure, which makes me curious.

<comment><USER9> Azure is almost as big as AWS. I think Azure is more used for Enterprise Saas.

<comment><USER42> According to who? And what is included? Microsoft has a history of blurring the lines here. I think the only way you get them being equal is by combining internal use and/or their SAAS offerings. I don't think it's a fair comparison if you do that.

<comment><USER9> Maybe you like this article: [https://www.redpixie.com/blog/microsoft-azure- aws-guide](https://www.redpixie.com/blog/microsoft-azure-aws-guide)

I think this is a fair and in depth comparison.

<comment><USER41> > I think this is a fair and in depth comparison.

redpixie is a MS partner, so I don't expect it to be fair.

[https://www.redpixie.com/blog/iaas-paas- saas](https://www.redpixie.com/blog/iaas-paas-saas)

"As a Microsoft Partner, we focus on Microsoft Azure IaaS solutions"

</comment></comment></comment></comment></comment><comment><USER43> Nothing like an article written 6 months ago about a method with 50% accuracy. The dime in my pocket does just as well and is much simpler :)

</comment><comment><USER44> this was being promoted to me an insane amount on twitter - now on HN as well?

</comment><comment><USER45> really interesting article. i imagine the financial markets being one industry a lot of people apply deep learning models to.

</comment><comment> cup-of-tea If you learn how to predict the stock market and reveal your methods then you no longer know how to predict the stock market.

</comment><comment><USER46> This is from December 2017.

<comment><USER47> Yeah, the Github repo on this looks quiet; no responses to the issues posted.

</comment></comment><comment><USER48> This sounds like the plot of a mediocre thriller. Oh wait, it is.[1]

[1][https://www.theguardian.com/books/2011/sep/30/fear-index- rob...](https://www.theguardian.com/books/2011/sep/30/fear-index-robert- harris-review)

<comment><USER49> also [https://en.wikipedia.org/wiki/Pi_(film)](https://en.wikipedia.org/wiki/Pi_\(film\))</comment></comment>


Ask HN: What are programmers favorite sites? - <AUTHOR> I am looking for sites where you go for solutions. <comment> d--b Unfortunately, Stackoverflow is still the quickest way to get good answers.

Expect to be shamed, downvoted, asked "why you would do that in the first place", even if your question is well formed and relevant. Your question may be closed, or marked as duplicate, or put "on hold".

Still, almost always, someone nice eventually helps.

Their "policy" to help with civility didn't work at all. Oh, and complaining about this on meta stack overflow will also get you shamed, downvoted, told that you can't take criticism.

<comment><USER2> I've honestly had a great experience on stackoverflow throughout my life.

Maybe it's because I put time and effort into my questions, but everyone has been nice and informative (at worst they link to a similar question)

<comment><USER3> You're low-key suggesting he hasn't put time and effort in to his questions. I've anecdotally also seen extremely toxic behavior on SO.

<comment><USER2> Ah - my bad. Not trying to suggest anything, just sharing my experience.

</comment></comment></comment></comment><comment><USER4> Try: [http://wiki.c2.com/](http://wiki.c2.com/)

It's an ancient site about software development and design patterns. _Frameworks come and go but engineering methodologies never die_

<comment><USER5> The C2 Wiki is a great place. Incidentally named for Cunningham & Cunningham and also the first ever wiki[0].

It does feel more like a reference library though and not really a place for daily or newly developing information. I always enjoy any time I find myself there.

[0] [https://en.wikipedia.org/wiki/C2.com](https://en.wikipedia.org/wiki/C2.com)

</comment></comment><comment><USER6> [https://dev.to](https://dev.to) has a very interesting culture. It's far less toxic than HN or Reddit for discussions, and more suitable for blogging on tech topics than medium.

<comment><USER7> The interface really puts me off though.

<comment><USER6> Yes, it feels imperfect, but I get used to it.

</comment></comment></comment><comment><USER8> [https://www.freecodecamp.org/](https://www.freecodecamp.org/)

</comment><comment><USER9> [https://www.google.com/](https://www.google.com/)

[https://stackoverflow.com/](https://stackoverflow.com/)

[https://github.com/](https://github.com/)

</comment><comment><USER10> [https://stackoverflow.com/](https://stackoverflow.com/) :)

</comment><comment><USER7> 1\. Search Google.

2\. Go to the Stack Overflow links.

3\. If there are none, you go to the Github links.

4\. If that doesn't help, you take a coffee break and work on something else when you come back.

</comment><comment><USER11> [https://lobste.rs](https://lobste.rs) If you are looking for a great community and cool website. But it's invite-only also there's something called user tree which makes you responsible for invited users actions.

</comment><comment><USER12> planetsourcecode.com</comment>


Unlocking the census (data) - <AUTHOR> https://censuslooker.com/ <comment><USER2> Cool but I cannot believe how long it took to load.

<comment><AUTHOR> Yes. I didn't measure it, but I'm guessing there was a heck of a lot of javascript loading.

Too slow to be more than a curiosity, I'm afraid.</comment></comment>


Google's lost opportunity with Android - <AUTHOR> http://abhijitdhakne.blogspot.com/2012/03/googles-lost-opportunity-with-android.html

<comment><USER2> People are getting better and better with linkbait titles here on HN. Wish we could tag as linkbait....

</comment><comment><USER3> ...for Tablets</comment>


Show HN: Reasync-hooks: A package to track redux async action states - <AUTHOR> https://github.com/KevinZhanglulu/reasync-hooks <comment><AUTHOR> Implement three custom hooks: useIsAsyncPendingSelector, useOnAsyncFulfilled,useOnAsyncRejected</comment>


Languages That Compile to Lua - <AUTHOR> https://andregarzia.com/2020/06/languages-that-compile-to-lua.html <comment><USER2> Here is a more complete list: [https://github.com/hengestone/lua- languages](https://github.com/hengestone/lua-languages)

</comment><comment><USER3> I found this project pretty cool: [https://github.com/roblox-ts/roblox- ts](https://github.com/roblox-ts/roblox-ts)

It's Roblox's dialect of Lua, but it's still Lua. Probably not worth a mention for vanilla Lua though.</comment>


We Should Stop Fetishizing Privacy - <AUTHOR> https://www.nytimes.com/2019/05/23/opinion/privacy-tech-companies.html <comment><USER2> Did anyone else think using the term "fetishizing" was not just nonsensical in selection, but actually demonstrated a level of social myopia?

Getting a little looser with language for a moment; privacy isn't something that one would classify as a fetish, as socially speaking, it's been the sensible default for centuries.

Seems to me that the real subject of fetishization in this piece is _exhibitionism_. Specifically of everyone else. With the receiver of gratification from said visibility being tech companies.

Maybe I'm about as straightforward as a series of hairpin turns, but this article did far better at convincing me that tech investors are desperately trying to convince everyone that it's totally okay, and everyone should walk around with their "personal information" out in the open for them to reap the benefits of, and there's absolutely no reason for anyone to second guess this at all, because _< shiny economic buzzwords>_ without even a mention of second or third order implications of their viewpoint, which to be frank, I see as the most blatant form of intellectual and rhetorical dishonesty.

I'm now stuck with a mental image in which the industry I work in is little more than a gaggle of peeping Toms, Voyeurs, and sexual predators all trying to legitimize what their doing watching everyone else as normal or the "new normal" because it's unthinkable that the future shouldn't allow the parasitic commoditization of everyone's personal data.

I'm not even the most socially adept person out there, but this piece is a shocking illustration of what I consider to be malicious perception management.

A very disconcerting read.

</comment><comment><USER3> I wonder whether it was Facebook or Google that paid for this article?

</comment><comment><USER4> Previously: [https://news.ycombinator.com/item?id=19999127](https://news.ycombinator.com/item?id=19999127)

</comment><comment><USER4> This abysmally poor and fatuously argued opinion piece transcends self parody and exemplifies fractal wrongness.

</comment><comment><USER5> By Heidi Messer. Ms. Messer is a technology investor and entrepreneur.</comment>


I spent all of my millions. This is what I learned - <AUTHOR> https://hackernoon.com/i-spent-all-of-my-millions-this-is-what-i-learned-9975ab94c924 <comment><USER2> I've noticed when people go through very traumatic experiences (divorce, death) they sometimes experience a dramatic change in personality. It's almost like the shock of the event kills your previous self.

This guy said he was bored of his current roles, but all said that probably wasn't the actual issue. He was just looking for a way to cure his depression and ennui, and a change as radical as quitting his former life probably seemed appealing.

Maybe if he didn't pick something with such a notoriously huge requirement of time he would have been better off. Like hell, even just being straight unemployed for a year and doing literally nothing might have been better. As of right now, since he's not planning on being a practicing psychologist, it all seems like an enormous waste.

<comment><USER3> > It's almost like the shock of the event kills your previous self.

This is a very profound statement and I think I may take it a bit further in a "ship of theseus" narrative:

Perhaps we all are literally different people from year to year, month to month, week to week, and we just happen to remember our former selves enough to keep the ship intact?

What if the trauma simply shocked us into forgetting or wanting to forget our recent past selves?

</comment></comment><comment><USER4> " Money is flowing all around us. You make money in proportion to how much you place yourself in the flow of value transfer." \- This article

It's the Great Material Continuum! Ferengi saw the universe as having "millions of worlds, all with too much of one, and not enough of the other", with the Great Continuum flowing through them all like a mighty river, from have to want and back again. As such, it was the force that bound the universe together. To get everything you desire in life, you had to have your ship navigate the Continuum with entrepreneurial skill and grace. \-- [https://memory- alpha.fandom.com/wiki/Great_Material_Continuu...](https://memory- alpha.fandom.com/wiki/Great_Material_Continuum)

</comment><comment><USER5> > I lost a large chunk in the 2008 market crash

Oof.

> ..if I had kept my capital and invested it in index funds, and continued to > work, I would now have so much wealth that I would never have to work again.

I really think this is just a testament that you should hire an FA if you come into money and don't know what you're doing.

<comment><USER6> I think that's actually what he did, from what I got in another article, and the financial advisors he hired actually did stock picking and cost him more money than they made for him.

</comment></comment><comment><USER7> > I suspect that part of me places formal qualifications in the way of my > success.

Haha, I know the issue.

I recognize myself in a lot of these lessons. Except the losing money part, I never had much to lose.

</comment><comment><USER8> Where the idea came that anyone needs PhD in psychology to be motivational speaker? Many if not most successful ones don't even have basic degree.

</comment><comment><USER9> People comparing themselves to others is what causes most of their misery but maybe people need to be miserable to elevate themselves higher (which on an individual scale might sucks but i bet it benefits society).

</comment><comment><USER10> Really enjoyed reading your reflection, well written, engaging and great message. Thanks Dr Duncan.</comment>


Scientists Think the Speed of Light Has Slowed, and They're Trying to Prove It - <AUTHOR> http://motherboard.vice.com/read/light-speed-slowed?utm_source=nextdraft <comment><USER2> Wouldn't we notice by now? GPS would be affected if the speed of light changed. All kinds of speed-critical connections would be affected. Heck, even CPU speeds would change.

<comment><USER3> Read the thing. Speed slowed until it reached a lower bound, and has been stable since then. That's the theory described.

The TL;DR is "the cosmic background noise is too uniform; if light moved FAST in the early universe that could explain the uniformity; there have been theories; here is one."</comment></comment>


Firefox Olympics Doodle about:home - <AUTHOR>

An Olympic pyre is now on about:home. Is that legally OK to do that? Didn't Google patent this Doodles thing? <comment><USER2> Google is small fries compared to the corporate clout behind the IOC and its iron-fisted marketing and control of the Olympics.

So, fuck the Olympics. I feel bad for the athletes, but they need to find a less toxic arena to demonstrate their skills.</comment>


Ask HN: Obtain a domain from a squatter by trademarking - <AUTHOR>

I have hit upon a great domain which is being squatted upon at the moment. Everything except the '.com' domain is available. Thank goodness, I don't need the '.com' now (I am based in Canada, so will use '.ca' for now) but I will need it in the future. I don't want to grow my business and then get extorted. How about trademarking in the mean time, and then forcing the squatter out of the domain. Is this a good strategy? <comment><USER2> That's an unethical and impotent strategy. Simply owning a domain without using it is not squatting. Having a trademark does not give you rights to someone else's property either. See Nissan Motors vs. Nissan.com.

The UDRP only allows you to take a domain for trademark infringement if:

a) the domain name is identical or confusingly similar to a trademark or service mark in which the complainant has rights; and

b) he/she has no rights or legitimate interests in respect of the domain name; and

c) the domain name has been registered and is being used in bad faith

<comment><AUTHOR> The squatter has several names registered, has indicated to me that my particular name of interest is for sale, and he asked me to give him the best possible price. I said $300, and I never heard back from him. If I want to use the name, do you have any suggestions?

<comment><USER3>
        I said $300, and I never heard back from him.
        If I want to use the name, do you have any suggestions?

Bid more than $300?

If you don't know how much the domain is worth, you should do some homework. Here are a couple of sites I've used when attempting to determine the market value of a domain:

<http://www.domaintools.com/buy/sales-history/>

<http://www.domain-prices.com/search/prices/5000>

<comment><AUTHOR> I have tried those services. They never give me any info - you see, nobody has ever used that name before.

<comment><USER4> Is there a way to see how long he has owned the domain? From there, estimate a yearly cost to him (12$ * years registered) + 10% profit and start from there. That seems reasonable.

<comment><USER5> You can use whois to see when the domain was registered. Since all the other extensions weren't registered I'd assume this is more of a brand style name and was registered fairly recently. It is hard to assign value of a domain without knowing what it is. Also, most people who sell domains are looking for the sales to compensate for the rest of their portfolio that don't sell. So registration fees + 10% wouldn't likely work as an offer.

</comment></comment></comment></comment></comment></comment><comment><USER6> If he owns the domain before you setup your business and do the trademarks he can keep the domain. Your logic is very flawed because you don't understand the process.

</comment><comment><USER7> If I were you I wouldn't use a non-geographically neutral extension to start with, unless you only intended to keep your business strictly in Canada.

</comment><comment><USER8> Really tired of people calling legitimate domain investors "squatters"

<comment><USER9> I'm tired of domain investors being called legitimate. It's free and open space; take what you need and leave the rest.

... You're probably the person before the hurricane who buys all the gas in the neighborhood and resells it to people for 1,000 times the price.

<comment><USER10> Or invests in a company that looks like it might hit it big before everyone else does.

</comment></comment></comment><comment><USER11> It's what Groupon did, but I wouldn't recommend it.</comment>


A selection of posts on VC funding overall approach and process - <AUTHOR> http://wallen.typepad.com/wallen/2009/03/vc-funding-overall-approach-and-process-.html

<comment><USER2> Nice set of articles,

But I'm really not digging your UI. This is a set of links, needing a flash overlay is overkill.

<comment><AUTHOR> Thanks for your comment.

It's much more than a set of links. Links are just a list of independent items with no relation between them. You'll notice that there is a relation between the different pearls in this map, that there is the perspective of a VC and the one of an entrepreneurs, or that of a VC that likes exec. summary and another one that doesn't. In other words, this map is not a random "set of articles" but a story of the different dimensions and opinions on the VC funding process. You may not have noticed it but by organizing the map in the way I did, I've basically guided you through the web on this topic and by doing so I've put forward certain ideas and not others.

Now, that being said, to be able to easily manipulate these pearls and create maps, flash is required. You can record and turn your navigation into a map, drag&drop pearls, erase them, switch orders, write comments, change titles, indicate crossings, etc. And without flash, it's not possible. The power of a map is way way way bigger than that of a list: to you use the yellow pages to find your way in a city or do you use a map?

In addition, Pearltrees allows you to see other maps that cross my map on VC funding overall process and approach. In this case there are none (at the moment) but if there were you would see a yellow circle to indicate that an other map is crossing this map. You can then basically navigate from map to map, from gudie to guide and you will discover an entirely new, humanly edited web.</comment></comment>


IE6IFY - <AUTHOR> http://ie6ify.com/ <comment><USER2> This might actually be handy if it really did show what a page would look like in IE6. Instead, it comes across to me as a trivial CSS/JS novelty with a somewhat misleading name. Can I have those 10 seconds back?

<comment><USER3> The name was inspired by <http://cornify.com>, how is it misleading? I'd be happy to refund your 10 seconds.

<comment><USER4> It doesn't actually simulate ie6, it just moves things around randomly (see comments below).

<comment><USER5> which is a pretty good description of how ie6 renders pages.

<comment><USER6> Still, it's not accurate ;-)

edit: I doubt it's possible to be - that would imply duplicating the bugs in IE6, something I doubt even Microsoft could pull off.

<comment><USER7> C'mon! It's just a bit of fun!

</comment></comment></comment></comment></comment></comment><comment><USER3> Oh cool, my half-hour project is on HN! I made this a year ago on my netbook while waiting for the bus. Interesting how my silliest ideas are the most successful ones.

</comment><comment><USER8> It'd be awesome if someone _actually_ reverse engineered the algorithm and created a bookmarklet to reproduce it. This page [1] seems to have a lot of useful info.

[1]: <http://www.satzansatz.de/cssd/onhavinglayout.html>

</comment><comment><USER9> Haha, nice one, this technology will surely be bought by Microsoft to support their anti-IE6 operations.

<http://ie6countdown.com/>

</comment><comment><USER10> Very briefly, it works by randomly repositioning page elements.

<comment><USER11> Not quite. Here's the cleaned up JS from the bookmarklet:

        function () {
        var i = 0,
            r = function (n) {
                return Math.floor(Math.random() * n)
            },
            f = document.getElementsByTagName('body')[0].getElementsByTagName('*'),
            o = function (e) {
                return typeof(e.style) == 'object' && e.tagName != 'SCRIPT'
            },
            s = function () {
                while (!o(e = f[r(f.length)])) {}
                return e.style
            };
        while (i++ < 5) {
            s().display = r(2) ? 'block' : 'inline';
            s().position = r(2) ? 'absolute' : 'relative';
            s().margin = r(2) ? '0' : '1em';
            s().padding = r(2) ? '0' : '1em';
            s().width = r(2) ? '' : 'auto';
        }
        })();

Every time you click the button, it does the following:

1\. Randomly selects an element from the page that isn't an object or a script.

2\. Randomly switches the display property to either block or inline, the position property to absolute or relative, the margin property to 0 or 1 em, the padding property to 0 or 1em, and the width to a blank string or auto.

3\. Repeats steps 1/2 until 5 elements have had these transformations applied.

<comment><USER12> why use

        document.getElementsByTagName('body')[0]

instead of

        document.body

?

(it's a legit, not snarky, question )

EDIT: formating.

<comment><USER11> Because maybe there's more than one body? :P

[Seriously though, I have no idea]

<comment><USER13> You know, you're probably fine if you always assume there's only one body tag.

That said, I've seen this and worse when looking at the output of highly-paid web dev departments at large companies. One site we recently saw had _five_ opening html tags, head sections, and body tags. All the head sections were different, and none were closed, including roughly half the content tags we saw. It's a testament to the "be loose in what you accept" mantra of the web, and to the browser devs themselves, that this monstrosity rendered properly.

I'm not saying "enterprise" developers are bad by definition, but the enterprise sure is large enough for the bad ones to sneak by.

</comment></comment></comment></comment></comment><comment><USER14> Reminds me of my buddy Mats Bryntse's IE6Frame. He presented it at a meetup at Yahoo about a year ago:

[http://mankzblog.wordpress.com/2009/11/13/ie6-frame-to- battl...](http://mankzblog.wordpress.com/2009/11/13/ie6-frame-to-battle- chrome-frame/)

</comment><comment><USER15> More IE6 humor: the IE6 effect! <http://mrdoob.com/lab/javascript/effects/ie6/>

</comment><comment><USER16> It works even better after clicking the button a few times.

</comment><comment><USER17> This would be funny if it wasn't so true.

</comment><comment><USER18> I'm guessing @HipsterIE6 disapproves

<comment><USER19> Why do people use @ mentions in places which don't support them?

<comment><USER20> To indicate they're talking about a Twitter user, without saying "The Twitter user, HipsterIE6", you humourless ninny.

<comment><USER21> I've seen people using it here, when referencing a sibling post in a thread. I think some people have generalized it to mean "I'm talking about someone who I know will read this."

<comment><USER22> <http://vanillaforums.org> use the format as a functional component. @Username in a post will (per target user settings) send them an email with a link to the post. Great way to drag specific people in to a thread.</comment></comment></comment></comment></comment>


Technology Boom Allows the Wizards of Silicon Valley Unusual Freedom - <AUTHOR> http://www.wsj.com/articles/SB907021501829664000 <comment><USER2> Please stop posting paywalled articles on Hacker News.

<comment><USER3> You might notice that there is a "web" link next to the comments link. Click that to get around the paywall. You might also look at the HN rules which specifically discuss this issue

</comment></comment><comment><USER4> Needs a [1998] in the title.

<comment><USER3> Oh man, that explains a bit. Unfortunately it explains how someone who I thought to be at least 50 yrs. old is (was) making that kind of money today (actually 1998), and could find a new job within a day. No doubt there are people doing exactly this, but not too commonly.</comment></comment>


Things I've learned about writing software after 12 years - <AUTHOR> https://medium.com/@landongn/12-years-later-what-i-ve-learned-about-being-a-software-engineer-d6e334d6e8a3 <comment><USER2> There's really one issue here. Budget.

You want two pairs of eyeballs on everything as you write it? Okay. Pay two guys.

You want complete test coverage? Okay. Can take as long as writing the code itself. Maybe more.

You want full documentation? Okay. Guy can't spend as much time on coding.

You want new APIs to be included? Okay. Guy's got to read docs and try samples. Less time.

What I've seen is you always end up with someone noticing that you could just spend all your time coding, and you would still produce an output that was ok. At least for small projects, or the start of large ones. It's as if there's a true productivity that's much lower than the imagined, and whoever is setting priorities doesn't notice it. He only sees code that's been written. He doesn't see technical debt. He doesn't see when a test catches a budding error. He doesn't see what that second pair of eyes is for. He doesn't know why you're always reading about some new thing.

<comment><USER3> which is why a software project requires someone with technical knowledge doing the management, and not an MBA or "project manager" who cares only about on budget and on time at the cost of technical quality.

<comment><USER4> If you know the management is not technically competent, the responsibility also lies on the developer to communicate things on their level. If they ask how long will X take, you don't answer 1 day, and then another day to test it. You answer 2 days. If that's outside their budget then forget about that feature and let sales find a new feature they can sell that can be developed within the budget.

This also boils down to the ego thing, many junior developers are proud to give short estimates to show off how quick and good they are at their work. But those short estimates are always always always just the estimated time to get a quick prototype working, where things are configured as you go in the debugger, not something that is robust and will work together with the rest of the application in a customer deployed environment. Estimation is really where you can see the difference in an experienced engineer and a non experienced one.

</comment></comment></comment><comment><USER5> We get the software we deserve. Apparently, people aren't ready to spend more money on performance, reliablility and security. When they buy software, they pay the price of a giant ball of mud (if they aren't expecting it to be free!) and they get a giant ball of mud. Simple. You call it insanity, I call it market, supply and demand. Of course, the perfectionists are unhappy about that (being one myself I can certainly empathise) but you have to realize that this kind of perfection in software costs huge amounts of money and requires the most skilled workers in the field.

<comment><USER6> _" We get the software we deserve."_

Painfully true. There are techniques for writing better software. They work. They take longer and cost more. They are not widely used outside aerospace.

<comment><USER7> Is there a high level overview somewhere I can read? Don't even know what to google here.

<comment><USER8> [http://spinroot.com/gerard/pdf/P10.pdf](http://spinroot.com/gerard/pdf/P10.pdf) (some are more/less appropriate for non-embedded-systems)

In general, the answer typically involves formal specification and formal methods that check the code against these specifications, combined with testing and coding standards that result in analysable code.

More references:

[https://www.cs.umd.edu/~mvz/cmsc630/clarke96formal.pdf](https://www.cs.umd.edu/~mvz/cmsc630/clarke96formal.pdf)

[http://research.microsoft.com/en- us/um/people/lamport/tla/fo...](http://research.microsoft.com/en- us/um/people/lamport/tla/formal-methods-amazon.pdf)

</comment></comment></comment></comment><comment><USER9> _Unless you've written a buggy program, you don't realize that you're addressing our intellect. This is why I think that every engineer on the planet looks at a bug report and feels a twinge of pain as they read whatever detail that was left to serve as a figurative shame sticker on the report card of their creation. It really sucks when you're just flat out wrong.

Being wrong — rather, being incorrect — is an extremely humbling experience. The catastrophically incorrect, which is when software crashes, money is lost, or the absolute worst, data is stolen, is the kind of thing that makes you question your career choice. It makes you want to curl up into a ball and weep at how completely fucking stupid you were when you've found the problem._

Just had to quote this - I urge people who stopped reading or down't want to read the article to read this anyway. I've read a lot on developping but I think it's the first time I saw someone putting it like this. And oh boy, is he right. At least for me. Every bug report (well, the ones which points to something I obviously fucked up) hurts. What hurts even more is the dreaded _reopened_ because I fucked up again. Especially because sometimes that means the whole set of classes surrounding the bug are just textbook examples of all code smells in the universe. And the _only_ thing that can be done about it is the nearly impossible _write good code 100% of the time_.

<comment><USER10> I have a slightly different take. That used to be my initial reaction and it probably still is but taking a cue from other engineers better than me they seemed to internalize that there will be bugs, period. So might was well get over it and just fix them as they're found rather than get upset over them.

Of course they try to write good code, follow good practices, write tests etc but there's just going to be bugs, period. So don't beat yourself up. Just fix the bug, learn whatever you can from it so hopefully you won't do it again and or if it calls for it adjust your build infrastructure or testing infrastructure so you're more likely to catch them in the future.

<comment><AUTHOR> "Just fix the bug, learn whatever you can from it so hopefully you won't do it again"

The baggage associated with bugs is so prevalent that we can't talk about them without a real sense of shame. "Hopefully you won't do it again".

Bugs are just part of the cycle. I completely agree with your assessment that people who can realize that are happier and more productive because they can plan for the probable case.

</comment></comment></comment><comment><USER11> From Edsger Dijkstra: [https://youtu.be/RCCigccBzIU?t=13m54s](https://youtu.be/RCCigccBzIU?t=13m54s)

> ...just after the first successful moon landing [...] I knew that each > Apollo flight required some 40,000 new lines of code. [...] I was duly > impressed that they got so many lines of code correct. So when I met Joel, I > said, "how did you do that? [...] Getting that software right." [...] He > said that in one of the calculations of the orbit of the lunar module the > moon had been defined as repelling instead of attracting. They had > discovered that error by accident. Imagine, by accident five days before the > shot.

Sound familiar to anyone?

Some 16 years experience myself, the clarity of "the problem of time" has become the foremost factor of bad software that I knowingly write. I imagine NASA has the same problems that we do.

I've been through about 5 cycles that all occur in the same form: 1) We put the under-performing developers on maintaining "legacy" systems and the seniors begin on the next one. 2) Time is purposely estimated shorter than viable, corners get cut in the software, prototypes are put into production once again, and a new legacy system is born. 3) The good programmers leave, some juniors become seniors and others are hired, and the cycle repeats.

There are many additional mismanagement mistakes that accumulate, but limited time always seems to be at the center of the problem for me. I whole-heartily believe that really great software can be authored by just two or three high quality engineers over the span of a few to many years with a clear vision.

<comment><USER12> I agree, but I think "with a clear vision" is the rub here. That is possibly the real impediment in all these cases.

</comment></comment><comment><USER13> Many people here already acknowledged that this text misses some details, but I'll put it even harsher: even though it's interesting to read, in fact it's just completely wrong and somewhat misleading.

Author starts by mentioning huge, complicated and ambitious projects and pointing out how seemingly insignificant mistake makes all difference between "it works!" and "it doesn't work!", which sure will sound familiar to what we experience every day, but on different scale. With this impression he moves on to talking about modern software in the industry and at home, and how we fail to write _correct code_ and why.

But in reality, nobody (except the programmer himself, maybe, if he fails to see the bigger picture) gives a fuck about _correct code_. Glitches and errors are ok, if they don't make software unusable and prevent us from achieving some global business-goal. It's ok for landing-site to have a CSS bug, that makes some button look weird in certain situation, unless it really annoys the customer and we lose sale — the chances are that nobody even notices, and it may be fixed next week or even never. It's ok to put your business-critical back-office software on production and find out that it contains some really nasty bug, which you'll fix in a hurry in the next 20 minutes. It's ok for Ubisoft to spend millions of dollars on production and then ship game with bugs, which will be fixed with the next patch-release. It's ok to have stupid over-complicated networking protocol, that might have been 5 times more efficient, but still is usable and allows people to do something they couldn't have done before.

What is _not ok_ is to spend millions of dollars on development and then never ship the product. It is not ok to let your client to find a solution which works for him, but is cheaper. In a word, whatever actually hurts your business is not ok. And the truth is that cheap solution that is "somewhat ok" is usually better, than the expensive "good solution".

</comment><comment><USER14> There's a sentiment in the article I see a lot that keeps coming up these days: "NASA sent a spaceship to the moon with something 100x slower than my computer, so why does it take so long to load a page"?

I think it's entirely unjustified. What makes modern software engineering difficult is that it's one of the few engineering disciplines where not only are you making a Boeing 747 fly, and all the parts were made by different people in different locations, but you're also replacing the engine _in flight_!

Also the specs on the parts are kind of spotty. They weren't agreed upon by a single group of people, and they certainly weren't decided by you. Some of the parts are decades old. There are parts in there that are older than most of us!

Often, the parts decide to change how they work. They don't ask you whether they should. Sometimes, also, the parts decide to disappear. It's up to you to figure that out.

In fact, there aren't really specs on how the parts go together, come to think of it. None of the people who are building this are talking to each other. You could say that most of it is kind of emergent. There are no top-down quality controls.

There are layers upon layers of abstraction, and no one knows all of it. No one even knows most of it. In fact, most people don't care about any of it other than a cubic centimeter of their own few layers.

And it works - by golly it works. And a person with no training can absentmindedly navigate all this by flicking their finger about while walking down the street and drinking their coffee. IMHO, this is a miracle in proportions that are indescribable. The moon landing really was nothing compared to this. So yes, there's some overhead involved in getting that to work. :-)

<comment><USER15> > The moon landing really was nothing compared to this.

I used to believe that, but I have slowly moved to the other side.

Things are complicated. There is no doubt about that. But, landing on the moon was a fight against nature (gravity, air, distance, etc). Current programming is a fight against stuff someone else dreamed up and no one ever fixed, because "LOL, that's old school" or "You're just doin' it wrong".

Most of the points about parts not being specified correctly, not working as expected, or disappearing happens all the time in other industries. (That's one reason hardware kickstarters fail so often.)

<comment><USER16> Except it's an applies-to-oranges comparison to start with. The mathematics of space-travel is _very_ simple - it's just calculus. You need to be accurate, but it's easy to quantify and well controlled.

Which is why you can go to the moon with a decent graphics calculator, but you definitely can't have a self-driving car with much less then a modern supercomputer.

<comment><USER15> > The mathematics of space-travel is very simple - it's just calculus. You > need to be accurate, but it's easy to quantify and well controlled.

We're not talking about calculating our way to the moon. We're talking about going to the moon.

The space program had much more than a graphing calculator. They had buildings full of people, systems, and realtime communication with the craft.

</comment></comment></comment></comment><comment><USER17> The main thing that resonated with me is keep your software simple. Don't try to be cute or clever. Don't whip out your favorite OO design pattern when a simple function will do. Don't try to optimize the function you spend 1% of your time in. Follow Unix design principles. Simple is easy to maintain, easy to debug, easy to learn, easy to understand. Impress your peers by writing good code that works well rather than code that only you understand (and even then you will have to relearn when you come back to it in six months).

<comment><USER18> If I may dare to add:

... writing good code (that others, including yourself 1 year from now, can maintain easily) that works well ...

</comment></comment><comment><USER19> It's a shame the article is so egotistical, hyperbolic, and poorly edited. Also, not crediting obvious sources[1] for your punchlines smells like plagiarism.

Otherwise, I _really_ _do_ like reading articles like this as touchstones for engineering and development.

[1] [http://blog.codinghorror.com/coding-for-violent- psychopaths/](http://blog.codinghorror.com/coding-for-violent-psychopaths/)

<comment><AUTHOR> Credit where credit is due, that was lifted nearly wholesale from codinghorror. Wonder how you do liner notes with Medium. I'm a terrible writer, so the rest of the items I've stolen or otherwise isn't to stand on the shoulders of giants like Atwood, it's to reinforce a point the best way I knew how.

I lifted all of the comments from stackoverflow, too: [http://stackoverflow.com/questions/184618/what-is-the- best-c...](http://stackoverflow.com/questions/184618/what-is-the-best-comment- in-source-code-you-have-ever-encountered)

I'm curious where the egotistical portions were, though. I wanted to at least prefix some of the more "thou shalt" sections toward the end with a disclaimer that they're personal opinions on how I operate.

Glad you liked it, though!

</comment></comment><comment><USER20> Excellent piece of writing. Very funny at times, witty like a satirical novelist. Particularly enjoyed the anecdotes from NASA and the author's conveying of the powerful emotional context of high stakes programming.

I _pine_ for a golden, future age of programming where documentation is as he describes.

> This image brought to you by preventable catastrophe, budget overruns, > political wrangling, infighting, incalculable complexity, and against it's > best effort, humanity.

Beautiful caption. The dismal history of NASA in one sentence.

<comment><USER21> "dismal" \- that's an absurd word to apply to an organization that has successfully enabled humanity to explore the solar system over the course of a short 50 years.

There is plenty to Monday-morning quarterback, but "dismal" is not the word I would ever use.

<comment><USER20> You're reading me wrong. Dismal means sad, and the caption is meant to be wry or ironic.

</comment></comment></comment><comment><USER22> Budget as an excuse for poor software seems like a bad argument. There are as many people talking about how much more expensive it is to fix bugs late in the game as there are people pressuring teams to deliver shit code on tight timelines. It costs less to write better software. Invest upfront and rejoice that you did later or rush it out the door now and pay to maintain it forever.

</comment><comment><USER23> The ideas behind this article are good. The author gives some guidelines that have helped them write better software. However, it makes me think about how we can make "making software" better. It seems like the best current way of going about it is to implement these zen-like self-discipline techniques that have helped the OP.

What's so inherently wrong about software? Why is it so damn difficult to write software that doesn't explode if you don't hold your tongue at the right angle?

<comment><USER24> "Why is it so damn difficult to write software that doesn't explode if you don't hold your tongue at the right angle?"

Hubris mostly.

<comment><USER25> I don't know, it takes a certain amount of hubris to write any software at all.

</comment></comment></comment><comment><USER26> The Big Ball of Mud is not unique to software. Ever been in a building that's had a lot of renovations and additions? There are weird half-flights of stairs here and there, convoluted paths from one side to the other, different styles of fixtures, etc. Eventually they get to the point where it's easier and cheaper to just build a new building somewhere else, or tear the old one down and start from scratch. Just like we do with software.

<comment><AUTHOR> Absolutely true. Shantytowns are a common parallel to big balls of mud, and I think that would hold true for anything that gets constructed.

</comment></comment><comment><USER27> It was a good rant until he encouraged writing a test after writing the code to be tested :( As a developer also with 12 years experience, I implore you to write your tests _first_. Otherwise your organisation will almost inevitably end up with brittle tests & it will often cost more to patch up the tests when they break than the cost of the possible increase in genuine bugs. I don't necessarily espouse slavishly adhering to TDD (it works for some devs, not everyone) but experience has taught my heart to sink every time I hear someone say "I've implemented the feature, now I just need to write tests."

</comment><comment><USER28> Leave it to me to count the zeroes:

> It contained a defect of effectively point zero zero zero zero one inches. > that's 0.000001.

</comment><comment><USER29> Sean Parent would bring up the fact that given enough scale all systems end up being a network problem. If you look at the internet, you can think of it as one big system. Huge and hideously complex, its pieces are connected but not dependent.

If you can boil your software down to parts that can afford latency (even if they need throughput), you can pass messages between them. That is possibly and probably less efficient, but your macro pieces that pass well defined messages between each other can exist without adding to the complexity of the other pieces.

</comment><comment><USER30> To the point of automation, my credo is to automate all hairy stuff because you spend your focus and frustration budget on that otherwise. The rule of thumb is more like "if you can't make this your daily priority but need to do it repeatedly take a day and make it a nobrainer"

</comment><comment><USER31> Great article. Perhaps because I suck at grammar, I didn't even notice the language issues. Interesting that HN is getting bogged down analyzing grammar rather than the actual content. Were the mistakes so bad that they detracted you away from the message? FWIW I was able to comprehend it just fine.

One interesting issue I see is with specific documentation rather than 'self- documenting code': Considering docs don't compile, I have seen (in my 13 years ;) that docs tend to drift apart from the actual code and become more of a PITA than the code itself. At least once I have caught myself down a wrong path because I trusted the docs.

I don't have anything against docs though - write a bunch myself: I am very circumspect against trusting it completely though.

<comment><USER32> Yes, they really do detract from the message. They're a jarring distraction, like a skip or buzz in a piece of music. And sometimes even when you think you've comprehended it fine, you and author are thinking about different things because you silently filled in a gap with the wrong idea (can't say for sure if there were cases like that in TFA)

</comment></comment><comment><USER33> "It's like creating Frankenstein and then realizing that you didn't bolt the legs on. After you've hit it with lightning and given it life. While it's attempting to walk."

</comment><comment><USER34> Inexperienced developers would benefit most from reading this, but its also harder for them to put any of it into practice ... in your first job you're usually just scrambling around day to day trying to get a handle on the work, and also to fit into the workplace culture.

</comment><comment><USER35> Wow. It feels like I wrote that article I agree with it so much. There are a few things I disagree with:

1\. I'm not a TDD purist but I do think a red/green TDD cycle does strongly encourage all of the following things to happen in a more natural, flowing way: "Tell yourself what you're going to do, and then implement that sentence. When you're finished, take that sentence and write a test, then just move on." Coding before testing isn't the issue as much as the potential for needing to change code just to be more testable (an efficiency issue).

2\. No sale on the idea of documenting every piece of code; I truly feel like doc comments muddy the contents of the file, plus resulting documentation is less likely to be read than code and more likely to fall out of sync with the contents of the code. Completely separate documentation is hard to keep updated. It's often something to maintain with often no tangible benefits unless you are churning developers like crazy. Certainly it might be reasonable to document high-level interactions for complete view of the system but documentation at lower levels seems an increasingly bad value proposition to me. In a perfect world, sure... but in real life there are deadlines and tests+code almost always serve as enough of a developer guide to understand clearly. If not, I'd rather spend time refactoring instead of writing docs. For the few new developers brought on, reviewing the system in a paired programming manner is a good way to get to know them and ramp them up at the same time. Parts of the system that are difficult for them to comprehend might warrant refactoring attention.

3\. "If something is performed more than three times in a 4 hour span" It's difficult to make a wholesale suggestion on things like that. If said task is 1 second clicking the "get mail" button and automation is auto-checking every n minutes, automation can be a net loss in terms of productivity. I think the XKCD[1] chart is probably a better measure ;) Anyhow it's definitely better to not waste loads of time doing things you do often, and the more often you do something the more likely it is that automating the task would pay off; just don't do it with an utter lack of discretion.

Some typos in the post: \- Stray period before "I just call it by what it was: A big ball of mud." \- "Simple isn't sexy." should maybe be "Complex isn't sexy" or "Simple is sexy" \- "internet" (proper noun) \- "explaining just how the interact with" s/the/they \- "english" (proper noun)

Very thought-provoking and fun read (lots of laughs)- I really look forward to future posts!

1\. [http://xkcd.com/1205/](http://xkcd.com/1205/)

</comment><comment><USER36> 12 years? Sorry to break it to you, but that's a short time.

<comment><USER37> 12 years is the middle years for a career programmer. Enough to know what patterns aren't serving you, and enough to know that there is still a bunch to learn.

<comment><USER38> Middle years!? How long are these "middle years" supposed to last? I'm 22 years into my career and there's no way I've hit the mid point yet. It still feels like a joke to have a job title with the word "senior" in it.

<comment><USER39> How long have you been programming full time though? I find too many people on HN are 25 and count "15 years experience" because they started programming at 10. It's just not the same thing, especially when most people only count professional experience (and full time at that). 15 years of programming experience < 10 years full time experience in an enterprise or startup environment.

<comment><USER38> I'm counting from when I dropped out of college and started working full-time. Maybe only 21 years ago, it's a little hazy now. I'd been writing code for seven or eight years at that point.

<comment><USER39> It's pretty impressive, then, and nonetheless, I'm just a lot less impressed when somebody says they have 10 years experience and it equates to 2 years of full time, enterprise or startup experience, and 8 years of "I'm going to school full time and program 3 hours a night." I did program and experiment when I was in my teen years too (late 80s, early 90s) and still cringe when people count that as actual experience. Not that I'm constantly learning and growing at my full time job, but meetings, documentation, paperwork, etc are a large part of the job and count more than not having to bother with it.

</comment></comment></comment></comment></comment></comment><comment><USER40> being wrong is ok! try not to hate it! try not to let the fear of it guide you in any way!

</comment><comment><USER41> Not sure what all the hate is about. He presented some interesting anecdotes about the quirks of software engineering and then tied it all together with some concrete lessons.

Maybe 5K words without any sort of lists or clickbait titles is just too much for the average reader.

<comment><USER42> Or writing at higher level than a 12 or 13 year old :-)

<comment><USER43> Beyond the grammatical errors, misplaced punctuation and misspellings, and despite the relative novelty of the anecdotes, I thought the writing itself was dull and simplistic. Many paragraphs were just a string of blunt statements without any independent clauses.

According to a copy+paste of the text into [http://sarahktyler.com/code/sample.php](http://sarahktyler.com/code/sample.php), the article as a whole reaches Flesch-Kindaid grade level 7.

<comment><USER24> So, it was written such that a 7th grader could read it. How is that a bad thing?

Edit: I pasted a sample of John D MacDonald in there and it showed a level of 6.61. Poor John, no wonder he was never a success. /S

<comment><USER43> Your comment isn't really addressing mine from the context in which it was given. My comment was in reply to someone implying criticism of the piece stems from its reading level being above that of a 12 or 13 year old, which isn't the case.

I'm simply proposing that the criticism is founded on actual flaws with the piece, rather than it being too complex to be understood by its detractors.

<comment><USER42> It was what your comment implied to me - that is some one with a low level of English reading comprehension who needs short simple sentence structure.

</comment></comment></comment></comment></comment></comment><comment><USER44> I got halfway through this article and then gave up without any idea of what the author learned. Wish the author would have edited this post better.

<comment><USER45> I stopped at about 5 words in with the oddly placed comma and capitalization.

<comment><USER46> You stop reading things as soon as you encounter an "oddly placed comma"?

<comment><USER47> Like gcc, one misplaced comma and the whole thing might as well be rubbish.

<comment><USER48> -Wall -Wextra -Werror -pedantic

</comment></comment></comment></comment></comment><comment><USER49> Don't forget to thank nazis, who taught you how to build rockets and soviets who showed you that space travel is possible.</comment>


Rebuttal to hungry judges give harsher sentences - <AUTHOR> http://m.pnas.org/content/108/42/E834.full <comment><USER2> This is more of a "Reply" to the rebuttal; the actual rebuttal is here [https://www.pnas.org/content/108/42/E833](https://www.pnas.org/content/108/42/E833) And this is the original paper [https://www.pnas.org/content/108/17/6889](https://www.pnas.org/content/108/17/6889)

My two cents (as a pre-morning-coffee-biased and non-expert judge of this research) is that sample sizes are pretty small so authors shouldn't be reading too much into the "importance" of factors in logistic regression model.</comment>


The Startup Check List - <AUTHOR> http://spencerfry.com/startup-launch-checklist

<comment><USER2> Great list.

You mentioned that you would model revenue, and that makes sense. You should also model expenses. Subtract expenses from revenue per month and you will get projected profit/(loss).

Add your losses up so you know how much money you need.

Obviously this is all magic, but modeling is not necessarily for use in projecting future profit. Modeling is a great tool to understand your business before things happen.

<comment><USER3> Totally agree. I should probably clarify that point to say that we looked at both revenue and expenses. This is especially important if you plan to bootstrap for some time as we do.

</comment></comment><comment><USER4> This is actually a little more in-depth than some of these types of lists get into. I really liked some of the metrics tracking that they suggest adding in. It's unfortunate that this is getting posted to HN, though, before their actual site is live so that could check it out.</comment>