WebGL State Diagram - <AUTHOR> https://webglfundamentals.org/webgl/lessons/resources/webgl-state-diagram.html <comment><USER2> Looks cool. Is this an example or also a tool to parse any script?

</comment><comment><USER3> It looks great! Specially useful for beginners as learning tool.</comment>


Sure, I trust Google to index the contents of all my files. Why not? - <AUTHOR> http://daringfireball.net/linked/2012/04/24/google-drive <comment><USER2> And that is the "cost" of free services with lots of added value. If you want privacy, pay for it. If you don't care, embrace free stuff!

I have learned to "not care" anymore about privacy stuff like this, if I want the product or service, I'll sign up and just forget about it.

</comment><comment><USER3> Scroogled is why <http://blogoscoped.com/archive/2007-09-17-n72.html></comment>


Startups require potential engineer hires to perform a test? - <AUTHOR>

One of my first jobs out of school was for a startup, and they required me to do a short hack to prove myself. Then I worked for two larger companies and they didn't require any pretest. Now I'm considering working at another startup, and they want me to perform a test. Do just startups do this? What's your experience... I almost get the feeling that it's free labor for them. <comment><USER2> Free labor for them? If interview test code is in anyway a part of their codebase, you should be running the other way.

Programming tests are simply to make sure you can actually code. You'd be surprised how many people interviewing for engineering positions can't.

Startups are generally very small, and can only afford to hire engineers who are going to get a tremendous amount of work done.

</comment><comment><USER3> I work at a large org. And we require a programming test using JavaScript for front end engineers. It's the best way to see how skilled someone is. I would recommend it to anyone hiring for a programming position.</comment>


Ask: Know any good community site about online investment? - <AUTHOR>

I'm looking for HN-like sites or conventional forum that gives us reliable advices on getting up investing online and reviews of online brokers available. I found a few, but most of them flooded with spammy and scam articles. Please suggest a site or an online broker (with review if possible). I know that it may not be relevant to ask this kind of question here, but I'm not very comprehensive in internet and HN is the only site that I think has great community and reliable contents.<p>on a side note: I've been a lurker here for long time, and don't see the search button. Is there something I missed? <comment><USER2> Perhaps this site, mentioned in an post nearly adjacent to this one, might help:

The post: <http://news.ycombinator.com/item?id=382462>

The site: <http://tipd.com/>

</comment><comment><USER3> there's isn't a search feature. try google

"site:<http://news.ycombinator.com> <your_keyword>"

or <http://searchyc.com>

<comment><AUTHOR> hehe good old google :). I wish HN adds [save] link so I can read submissions later.

<comment><USER3> It has saved submissions feature too.

Whenever you upvote a post, it's in your saved list. Just visit your profile and you'll find a saved link there.

</comment></comment></comment><comment><USER4> not quite what you're looking for, but try www.newmogul.com

and www.thinkorswim.com

<comment><USER5> thinkorswim is a brokerage. I suggest investopedia.com/ and books. Figure out what style of investing you want to try: value, technical etc. Then go buy a book on the subject.

Other people have told me that they learned a lot of investing info from just blogs and books

<comment><AUTHOR> I read some books about fundamental and technical analysis. What I need is a community, some place you can share, and compare the outputs whether I'm doing it wrong or I'm in the right track and debate about the best technique or discuss currents news etc. Thanks for all the links.

</comment></comment></comment><comment><USER6> Not exactly what you're looking for, but worth checking out: <http://www.covestor.com>

</comment><comment><USER7> Bogleheads.org has excellent signal/noise ratio once you drill down to categories you are interested in.

</comment><comment><USER8> Have you tried The Motley Fool?

<comment><AUTHOR> Thanks, I read it a while ago, I just found that they have forums too :)

</comment></comment><comment><USER5> trade2win.com/ is a good forum for technical analysis

<comment><AUTHOR> nice!! :) man, i love HN</comment></comment>


HP Slate to cost $549, have 1.6GHz Atom Z530, 5 hour battery? - <AUTHOR> http://www.engadget.com/2010/04/05/hp-slate-to-cost-549-have-1-6ghz-atom-z530-5-hour-battery/ <comment><USER2> Even being an anti-apple person, I'm not sure I would give the advantage to HP for the 1.6Ghz Atom over the 1Ghz A4 without some real benchmarks. Also, how lightweight (or not) the OS is will probably determine more than their relative speeds.

<comment><USER3> You can already buy a normal windows tablet today... it doesn't make sense why manufacturers would try to "re-release" products they already have. You can probably buy a fast dual core tablet PC today that would blow the socks off the 1.6Ghz Atom.

</comment></comment><comment><USER4> 1024x600? Ewwww. I am not a fan of this recent trend of stretching displays as wide and short as possible. Especially for tablets which will often be rotated, the iPad's 4:3 ratio is much better.

<comment><USER5> HP should have gone for a higher resolution display, as 1024x600 is the current standard for netbooks, which is lacking at times.

At $549 and similar specs to a netbook(of course not counting touchscreen) HP may have a harder time positioning their slate than Apple.

</comment></comment><comment><USER6> fairly telling how the slide sells the device in the fashion of a traditional pc and not a personal appliance.

</comment><comment><USER7> The hardware is a bit mediocre as you'd expect from a low end tablet but the big liability is software. Besides HP's launcher and a few of their bundled apps you'll mostly be using traditional Windows software designed for a keyboard & mouse. It's the uncanny valley of computing: Not quite a tablet, not quite a laptop. This is why Windows tablets have always failed in the market. I suppose it would be possible to put x86 Android on here but you'll be stuck mostly with software designed for a SmartPhone sized screen. As we see with the iPad running resolution doubled iPhone apps this is really not a very good experience. ChromeOS is an option but you give up the power and flexibility of a full blown OS -- even in comparison to the iPhone OS.

</comment><comment><USER8> Yah, but why in the world would you ship Windows7 on it? What about chrome OS?

<comment><USER9> have you used windows 7? also, this thing can run chrome, so close enough, maybe?

what about the ability to multi-task, install hulu desktop, etc?

also, if one needs more than 5 hours of battery life, it's probably a good sign that one should get a laptop or, dare I say, a netbook. why? because you're probably going to end up buying the docks anyway. no one can sit through 10 hrs of facing down and typing on a screen. i promise you that.

<comment><USER10> Chrome OS has these wonderful little things called "tabs". They do a bangup job of multi-tasking.

(Sorry, your comment just really struck a nerve)

<comment><USER9> i should clarify, my comment about the mult-tasking was in response to your question of why use windows 7. i should have started a new line. anyway, chrome os to me doesn't make sense, unless such machines are priced well below $200. web apps only? bleh.

</comment></comment></comment></comment><comment><USER11> /me watches video

So, one of their advertising points on UI is that _Cover Flow_ stutters like Porky Pig? Cover Flow in particular is smooth as silk on almost any laptop I've tried it on, and I'll _guarantee_ that the iPad will handle however many thousands of covers you may have without the slightest pause.

And I find it kind of strange that it could handle 1080p without a problem, but be slow with scrolling images.

<comment><USER12> Predictive caching is hard. Let's go shopping!

<comment><USER13> How predictive you need to be to scroll over an ordered list?

<comment><USER12> You'd like to know how fast they're scrolling and in which direction so you can decide which resources to cache given finite disk and memory bandwidth as well as finite texture memory.

Apple's implementation doesn't bother loading the cover if the album in question is going to be past the edge of the screen before the load would complete. Instead it loads covers further down the line and renders a placeholder image.

Deciding which covers to load without knowing what the user is going to do next (stop the flow, go backwards, speed up) is a classic example of an online scheduling problem.

<comment><USER11> Did you actually watch the video? They're stepping through covers _individually_ ~3 times (not watching it again) and getting only a few frames of transition each time.

Caching the next and previous covers that are currently off-screen isn't just easy, it's _highly_ likely that it'll be a frequent operation, and should be optimized. If they had scrolled a few dozen in either direction a bit of lag would be expected, but not when iterating.

To make matters worse, this is in an _advertisement_ , and can be expected to perform _worse_ IRL because everyone optimizes for their ads.

</comment></comment></comment></comment></comment><comment><USER14> Let's not forget Apple gets more footprints into their little tent than HP's retail stores (oh wait, they don't have that kind of success on land).

As iTunes is to iPod, App Store is to iPad. Apple succeeds because of the combo punch, not because its consumers care whether HP has more Ghz or not.

Too bad Compaq didn't make better iPaq's (oh that's right, HP bought them :-p )

</comment><comment><USER15> 5 hour batter. Said enough.

<comment><USER16> Agreed. The only reason I'm even considering the iPad at this point is for its battery life. 5 hours is a deal killer.

</comment></comment><comment><USER13> A device with an ARM-based with a tablet-targeted OS and an x86-based Windows 7 device... Why would anyone think they compete?

How long does it take for Windows 7 to start up on that thing? 30 seconds? Yoy have to log-in every time you resume the device like a notebook?

</comment><comment><USER17> Is it me or is that a really terrible info-graphic.

I can't read the writing, and instead of linking to a larger version it just seems to link back to the page itself.

</comment><comment><USER18> price, battery life and brand name are three most selling points. If I need to buy tablet why exactly will I buy HP tablet :-/</comment>


DIY PIC Micro House Network [PicNET2] - <AUTHOR> http://dev.emcelettronica.com/diy-pic-micro-house-network-picnet2 <comment><AUTHOR> PicNET2 allows to develop a DIY domotic network with easy-to-use and cheap devices. I wanted to make possible the communication between the little devices (the well-known PIC) in a simple, safe and cheap way so the idea was born.</comment>


How machine learning can help to increase the productivity of a digital marketer - <AUTHOR> http://blog.customerlabs.co/post/125255114733/how-machine-learning-can-help-to-increase-the#.Vbd-2HaeByk.hackernews <comment><AUTHOR> Machine learning can save lot of time for us by automating the data analysis and recommending the actions which we would possibly act on. Learn to know more.</comment>


Former Goldman Options Trader Becomes Argentina Taxi King - <AUTHOR> http://www.bloomberg.com/news/2014-07-27/former-goldman-options-trader-moonlights-as-argentina-taxi-king.html?alcmpid=mostpop <comment><USER2> duplicate - [https://news.ycombinator.com/item?id=8103051](https://news.ycombinator.com/item?id=8103051)

this link appears to have been submitted with extra info in the URL so wasn't caught.</comment>


In defense of the corporate jet - <AUTHOR> http://www.nytimes.com/2009/02/01/opinion/01garvey.html?scp=1&sq=mile-high%20office&st=cse

<comment><USER2> If I'm a shareholder, I don't want to see profits (ha!) invested in a corporate jet. Fly commercial or better yet cut out travel altogether.

<comment><USER3> I don't think it's the best use of the CEO's time to wait a couple of hours for the connection every time there's no direct flight from point A to point B, or wait 1h for the checked-in baggage to arrive (more than once if it's an international flight to the US with a connection in the US). I also don't think it would be good for business if they miss meetings every time they miss a connection. I'm not talking about waiting in the check-in line or for the security, because I assume people flying first class don't do that--but there's no way first class can save you from missing connections.

<comment><USER4> You have a point, but then again, I don't think it's the best use of the CEO's time (or the company's money) to fly somewhere just to talk to people (some of whom have also flown there) when there are things like video conferences, emails and IM.

<comment><USER5> Indeed. Why fly, when one can instantly Halo? [http://www.hp.com/halo/pdf/Halo_Collaboration_White_Paper_3_...](http://www.hp.com/halo/pdf/Halo_Collaboration_White_Paper_3_21_06.pdf) [PDF]

 _Travel almost immediately declines, which is why many believe that the decline in travel costs alone constitutes Halo's return on investment. [...]

While Halo has proven to reduce travel costs, it is also reported to bring new levels of group productivity. As informal social networks begin to meet in Halo rooms, they accelerate innovation, problem solving and project completion. [...] Face-to-face interactions that occurred quarterly or semi- annually now occur on a daily basis allowing informal social networks to flourish. Travel time and its physical effects on individual productivity are eliminated. Finally, loss of productivity from being away from the home office is avoided, while improved quality of life is realized, both of which contribute to productivity on the job._

<comment><USER6> > Indeed. Why fly, when one can instantly Halo?

Do you always take a vendor's claims as gospel?

Me - I have a strong bias toward whatever decision is made by someone who has significant skin in the game. Sure, they make mistakes, but they've got far more relevant experence and incentive to get it right than someone in the cheap seats.

If you think that Halo is usefully better, use it and drive the fossils who go face to face out of biz.

<comment><USER5> _I have a strong bias toward whatever decision is made by someone who has significant skin in the game._

The people with skin in the game are using telepresence, instead of flying.

<http://news.ycombinator.com/item?id=470063>

 _High-definition (HD)-based video meeting services will replace 2.1 million airline seats annually over the next three years [...] Gartner was saying the technology would take away $3.5 billion from the airline industry - this year. And that's just airline tickets. The total figure of corporate savings would also include room, board and other travel-related expenses._</comment></comment></comment></comment></comment></comment>


80% of people trust Google more than Facebook - <AUTHOR> http://pickfu.com/C9S6ER

<comment><USER2> Well, I think everybody knows google business model, facebook is a little dubious.

</comment><comment><USER3> "(40 of 50 answers)"

What an impressive sample size they have there.</comment>


Need advice: How to help my co-founder get her energy and confidence back? - <AUTHOR>

My co-founder and I have been working on a bootstrapping project for 4 months. We aren't rushed and can still go without revenue for about a year.
The problem is that she does barely anything for the project. Yes, I could make a 5-item list of the things she's actually done: designed a logo, made a few design mockups she never completed, researched influencers we'll contact later to get the word out... Yet in the meantime I've built an entire mobile web app, and took care of a lot of the design and marketing copy that she was supposed to handle.
We are close friends, and I've seen her accomplish a lot in other areas. I asked her many times if she really enjoyed the project and her role in it, if she wanted to work on other aspects of the project or just give up and do something else. It wouldn't ruin our friendship. She is adamant that she does love the project, and she thinks that she can't work because she's too afraid of failing. It's a difficult project in an area where she has little experience, and it freezes her. We've talked about the issue a lot, both doing some introspection and trying to understand how to make it work, but so far the situation hasn't changed.
I am desperately looking for advice. Is her situation common? Should I just give up and stop here? Is there a specific way that I should handle her which would give her the confidence that she needs? I know, she's my friend, so I should know best, but none of my approaches worked and I'm desperate. <comment><USER2> Re-evaluate the business relationship - she's not really a 'co-founder' any more, or perhaps never was. Revisit the relationship and make her employee #1 with a moderate equity stake, or just an 'advisor' who's given some equity stake in the project. I'm assuming you're 50/50 on this (which I'm not sure I could ever bring myself to do personally) but she's certainly not contributing anywhere near 50% of the initial work, and apparently she realizes it.

Keep the friendship, rework the business relationship to something fairer to all involved.</comment>


Inkscape 1.0 - <AUTHOR> https://inkscape.org/news/2020/05/04/introducing-inkscape-10/ <comment><USER2> Good thread from a few weeks ago, about 1.0RC:

[https://news.ycombinator.com/item?id=22855357](https://news.ycombinator.com/item?id=22855357)

<comment><USER3> An earlier thread about the same release is [https://news.ycombinator.com/item?id=21001969](https://news.ycombinator.com/item?id=21001969).

Edit: Obviously this is a fine project and fine news, and if anyone is wondering why this isn't a big front-page thread, I've explained it in detail here: [https://news.ycombinator.com/item?id=23071428](https://news.ycombinator.com/item?id=23071428).

</comment></comment><comment><USER4> Congrats Inkscape Team! I don't use it enough to really be proficient in it, but when I do need to do something with it I'm so grateful that it exists. Mostly my use has been creating fliers for conferences, and making art for feeding to the laser cutter at the Creatorspace,

<comment><AUTHOR> All my scientific figures are done using inkscape, nowadays. I'm done trying to use tikz for things, this is too much of a hassle.

</comment></comment><comment><USER5> I wonder if Adobe takes inspiration from them now :)

</comment><comment><USER6> Congratulations team :)</comment>


Ask HN: How often does technical debt lead to the failure of a startup? - <AUTHOR> The question here comes from my concern of how some startups can actually survive with high technical debt with customers? Wouldn't that lead to their failure as a product/service sooner or later?
I work as a web architect contracting with startups on smaller complicated pieces of technologies. These days lots of startups I talk with have a very high technical debt in their product(s) and most of them don't seem to care.
My first advice to them is that things like that can kill a product, but I've had some strange feedback after passing on that advice.
So I wonder if technical debt does actually have pitfalls along the way? (Bootstrapping yeah I agree, but not when you have 8 engineers on the team) <comment><USER2> The likely failure of a start-up is market based, not technology. Look at Twitter's fail-whale years. They were on a platform which couldn't scale to their growth and it took many months for them to re-architect and build a working solution. It definitely didn't kill them.

Another example is Instagram, I read recently (but can't find the link) that the founders were learning to code when they built it, and that some of that code is still in production, and apparently, it's embarassingly bad, but it gets the job done.

On the other hand, I worked for a consulting firm and one of our clients sites was absolutely riddled with technical debt. The result was that any changes they wanted to make were significantly more expensive than if they would have been on a better architected system. The thing is, it did't matter to them because the cost of re-creating the system was significantly more than the cost of working with what they had.

I hate technical debt, but have come to realise that it is a debt much like any other, and if you can cover the cost of that debt, it doesn't really matter down the line.

This isn't an excuse for bad programming, I consider that to be something slightly different from technical debt which often works but is not an idea implementation.

<comment><AUTHOR> Thanks, I agree there are a lot of factors for a startup to fail.

Thought in the context that I mentioned, it is about some startups that have surpassed the point to find their first thousands of customer and now they need reliability.

But again there are different interesting tradeoffs of these issues.

Regarding:

> Another example is Instagram, I read recently (but can't find the link) that > the founders were learning to code when they built it, and that some of that > code is still in production, and apparently, it's embarassingly bad, but it > gets the job done.

I was at the OpenAir conference this year where Mike Krieger mentioned the above story. But I'm not sure if the Video is out.

</comment></comment><comment><USER3> I don't know a real number but my guess and personal feeling, having failed and succeeded, is that technical debt accounts for very few failures. Wrong product fit, ignoring customers, failing to market properly, failing to ship etc all are far more likely to cause the demise of a startup.

Also, I would say to much concern over technical debt can cause failure. I personally lost what I conservatively would say was 30% of our net revenue last year letting my prior lead dev address technical debt that never benefitted a single client and that in areas we have now replaced (or are in process of replacing) entirely in the product. I am as much (if not more so) to blame as he was because I approved the time.

<comment><AUTHOR> Indeed, there are lots of factors to lead to a product fail.

Though, I'm really curious, mind if I ask what sorts of technical debt you had in your startup that led to such massive slow-down in the process?

Were the debts among any of these:

* Documentations for on boarding devs. * Lack of tests. * Delayed refactoring. * Lack of alignment to standards.

<comment><USER3> So in my case, I will share that we screwed up last year by addressing things that didn't benefit our clients. We had probably 20 features that should have been developed that would have benefitted our clients. Instead I allowed our team to focus on scalability (which wasn't really an issue at the time), plus we migrated databases, added ElasticSearch for full text search (which wasn't an issue yet), reengineered a number of features, all of which were working, although maybe not ideally.

Also, almost everything you mentioned is on my list of WTF was I thinking. We wasted time adjusting code to meet new standards, adding tests to things we knew we were changing, adding scalability to places we had no scalability issues with (yet), adding new interfaces/code in places where something "might" happen etc. None of these changes benefitted a client directly, so IMO we lost money on every one of those initiatives. The reality is a lot of the code we "brought up to standards" was rewritten anyway when we started adding features and refactoring the old code to interface these features. I also allowed the team to create their own modules instead of extending some existing node.js modules that existed in the community, which to me was a waste when I saw what we did. We would have done better to extend the community modules and open source them back to the community.

As for documentation, honestly, we have pretty good documentation, although I wouldn't say it is without issues.

Overall I considered last year, the year of the "it might happen" screw ups. What really pissed me off is I spend a huge amount of time advising clients not to touch old code unless they know it is causing them a problem now or can be proven to cause an issue when testing against a new requirement. Yet, I still failed to follow my own rules, shit happens.

<comment><AUTHOR> Thanks a bunch! You're a champ.

Yeah sometimes making these sorts of decisions could easily lead to big pitfalls on the way.

Regarding this part:

> "I also allowed the team to create their own modules instead of extending > some existing node.js modules that existed in the community, which to me was > a waste when I saw what we did."

This is technical debt as well by the definition of Wikipedia and some 101 CS books. Having enough contribution from a community could always be better of being used, unless that part of the technology is propriety in the product.

<comment><USER3> You are welcome.

I totally agree I screwed the pooch and having 20 years of experience didn't save me. Ironically I had made sure we were giving back to the community but missed some opportunities that would have saved us time and money. Oh well, this year we will add the code to the modules we use. It is on our to do so we remove unnecessary dependencies we added.

</comment></comment></comment></comment></comment><comment><USER4> I see a lot of answers about how it's not technical debt that kills startups, it's failure to address customer needs. But I don't think these two are so different.

My definition of technical debt would be "problems in the code base that make it harder than it should be to do work," whether that work is adding new features, fixing bugs, scaling up, etc. The startup that can implement new features quickly because their code base isn't a disaster is more likely to succeed. They can try out new features quickly, seeing what customers respond to.

Customers need things like uptime, minimal bugginess, quick bug fixes, etc. Technical debt reduces uptime, increases bugginess, and makes it harder to fix bugs.

I think it would be hard to point at a startup and say "technical debt killed it". In most cases I imagine there are many inter-related causes of failure, but an inability to quickly add new features, fix bugs, and keep the system stable can't be a good thing!

<comment><USER5> _Customers need things like uptime, minimal bugginess, quick bug fixes, etc._

You've basically elevated these items to the top of the list, above all other attributes, and declared them more important to all customers.

Customers will pay/use a product that delivers value. The items you listed subtract from that value but they may or may not eliminate it. It depends on the product, and how valuable those items are to the customers.

Edit: I'll add an example. Let's say you like video games. There's a particular game that you love to play. Do you love that game because "it's stable, has minimal bugginess, and the developers release patches on time"? Probably not. You love it because of the gameplay/storyline/graphics/whatever. If the game is unstable, that's irritating, and makes it less enjoyable.. but that doesn't change what you love (value) about the game.

<comment><USER4> > You've basically elevated these items to the top of the list, above all > other attributes, and declared them more important to all customers.

No I didn't. I'm just saying that these things are _also_ important to customers. Churning out tons of broken features on top of a constantly crashing/unavailable system is not a winning formula for success in many markets.

It might work to some degree for free consumer-facing apps (like Twitter), but for many other sectors reliability and data integrity are very important factors (like any B2B product).

<comment><USER5> _Customers need things like ..._

Definition of need:

 _a situation in which someone or something must do or have something_

Saying "these things are also important" and that it depends on the type (twitter vs b2b) of product is a lesser statement than your original statement. If you want to tone down what you originally said that's fine.

</comment></comment></comment></comment><comment><USER6> I have worked at a dozen startups, some failed, some grew and still exist, some were acquired and people made a shit load of money.

I have _never_ been at a startup that failed for technical/technology reasons. All failures I've seen were due to issue with delivering what customers were interested in paying money for. Most often, product owners and developers were not able to establish good communication, which led to developers basically creating a product that didn't match what customers wanted. Period.

I would be interested in your definition of technical debt though. What some people call "technical debt" or "hack", I often call "the best/fastest way to address the current business concern and deliver immediate value". In a startup environment I think these is a critical workflow.

<comment><USER7> At one startup I worked at, most of the developers were very young and had their first job, nevermind first job with stock options as part of compensation.

When it came time for a round of financing, I was leading a team meeting when the subject of dilution came up: "Are we being screwed?" And I had to explain that a smaller percentage of a company with a higher valuation isn't "being screwed." and attendant to that we discussed cost of capital, classes of shares, preferences, ratchets, etc. You could see the light bulbs coming on as I explained.

Engineers are smart. If you explain to them that "good enough with the money we've got" will make them more money later, they are usually OK with deferring both money and correctness gratification.

Think of it as building empathy for "product owners" who might otherwise be dismissed as beancounters.

<comment><USER6> > If you explain to them that "good enough with the money we've got" will make > them more money later, they are usually OK

That's true. Some engineers won't take the "just make it good enough" approach, but I believe most will - otherwise they don't belong in a startup.

I wasn't mentioning the broken communication between PO and devs in this regard though. I have seen teams where the PO were not able to describe what they wanted, what business value they wanted to add to the product, devs had to come up with their own description of feature and sure enough the market didn't care of that.

PO is such a critical position, and such a difficult one to fill. I can count on one hand the companies where I worked who understood that. A PO is not just moving tickets around in JIRA, they are in charge of describing in actionable terms the business value the team is trying to produce.

<comment><USER7> Agneed: Weak PO or weak product management will sink you.

</comment></comment></comment></comment><comment><USER8> Technical debt isn't usually directly responsible for the failure of a startup. However, it can lead to problems that can cause a startup to fail. For example, technical debt can cause products to ship late, new features being harder to build out without refactoring, poor test/QA systems in place, new engineers taking longer to become familiar with the codebase, etc.

Anything that delays shipping is going to contribute to the failure of a startup.

<comment><AUTHOR> Thanks, that is indeed the on spot point I'm looking for.

Drawing some conclusions to what technical debt could eventually lead to at the end.

> "Anything that delays shipping is going to contribute to the failure of a > startup."

Edit: forgot a word.

</comment></comment><comment><USER5> What does a startup absolutely have to do? It isn't creating the most elegantly designed piece of software in the world. It's creating value for the customer.

Along the way to delivering that value, sometimes a less than perfect solution is chosen (often because of resources--time, money, or lack of human resources) OR the perfect solution is chosen, but the product changes and that solution no longer is perfect.

Technical debt should be managed, but it can't be eliminated... unless you want to give up producing anything new, and just work on polishing your previous work forever.

When will it kill your startup? If it turns out that your product is NOT what your customer actually wants, but the shortcuts (tech debt) you took prevent you from making the necessary adjustments to meet your customers needs.

<comment><AUTHOR> Thanks. Indeed, I agree with you regarding how startups are about "creating value for the customer."

But I think at some point, this might become very difficult due to the technical debt when the startup can't deliver more value than it promised.

<comment><USER5> That's definitely true of technical debt.. There are two issues here:

1) A competitive issue.. a company will less tech debt will be able to improve their product faster. That is a risk, but it depends on the competitive landscape how much risk that amounts to.

2) When you say "startup can't deliver more value than it promised".. this is more a variation on my original point. The issue isn't "delivering more value", but that the value that has been delivered is insufficient to meet customer needs.

Above that, "delivering more value" is a growth issue (your ability to meet the needs of additional segments of the market). And it's certainly true that startups with less tech debt can improve their product faster -- ie: grow faster.

<comment><AUTHOR> Thanks :)! Lots of good clarified points.

</comment></comment></comment></comment><comment><USER9> I don't have a story on startup failing due to tech debt, but I do have a story about technical debt forcing us to pass on potentially acquiring a startup.

A potential acquisition came our way that seemed to be exactly what we were looking for. Customer traction and revenues, the features exactly what we were looking to overhaul. There was very high internal interest in acquiring the technology and the company.

The need for this set of features was so high and the internal development estimated to be so costly (in developer time and delays in other projects) we were willing to overlook that this app was not built on our primary stack and only a few of our developers were familiar with. The language choice made by the company didn't cool our appetite. However, technological choices made us pass on the acquisition.

They didn't use a framework.

This means that our developers would have a very long learning curve. We also saw a lot of code that did what a framework would have taken care off and this means that we would have had to learn, maintain and expand the that code instead of working on the revenue generating features.

We found close coupling of code all over the place. This meant we couldn't quickly extend and modify the features as we wanted without first paying back the technical debt accumulated by the developers.

It wouldn't have mattered which framework they would have chosen as most of them have good documentation and force some sort of standard development practices. However, we couldn't take a chance that all the behind the scenes stuff would need a rewrite if we wanted to expand and scale the platform. We passed.

<comment><USER10> Heh. I once joined an early stage startup that had a big bowl of spaghetti code, no source control and no framework. My first insistence was source control. Then after a year in which we hired several developers, I insisted we move to a framework so the learning curve wouldn't be so steep for new people.

It was a monumental effort that took more than a year to complete, and we did the "brain transplant" (as we called it) at the same time we were moving to a new data center. Launch was shaky but overall it went very well. One of my fondest memories is of the "war room" on launch day as we triaged incoming bug reports. The team worked great together and at the end of the day we felt like we'd really accomplish something.

Sometimes of stress can be very rewarding.

</comment></comment><comment><USER7> I have seen it happen. But I have also seen startups skate by. Cost of capital is a real issue, and spending money on being perfectionist is not worth it if you arrived at a good-enough implementation despite being naive and cheapass.

More often it's not "technical debt" but outright failed implementation when technology kills a startup

See how far it takes you and if you are wise enough not to press your luck, you'll have the chance to do it right with cheaper money.

<comment><AUTHOR> See this is what I suggest most of the time!

My first startup that I built and sold had 0 test written, but I knew on spot what I'm giving up for what. I always say that last line of yours:

Making "the chance to do it right with cheaper money." rather than a whole team to re-do everything from scratch or break down already running product.

Sort of goes back to your point with cost of capital, indeed.

Thanks, good points :)!

</comment></comment><comment><USER11> There is no "TASB" for appraising Technical Debt like there is an "FASB" for Financial. So most measures resemble the parable of the Six Blind Men and the Elephant.

Trade off an extra day to do each release versus 30 days to fix the release process? Trade off an extra week to add a feature versus a month to refactor and a month of customer turmoil on the refactored release? Trade off a month of getting complete test coverage versus saving two days on a future feature development?

My guess would be that the startup founders are doing a better investment calculation than it might appear to you.

<comment><AUTHOR> Indeed, founders usually have a better complete vision the the folks around them.

Though I was just pointing that out for over time. One time around the tech- debt is not that big of a deal, until it becomes a beast and hard to overcome.

</comment></comment><comment><USER12> Try framing it as interest payments:

When you get X customers, you'll have to commit Y resources to fix this one thing.

Taking on technical debt is a smart thing in early stages because it's essentially low-cost, non-dilutive financing. So you should recognize that very real benefit.

However, if they are young startups they might not even realize they are doing this smart thing. You can add value by telling them are smart and giving them the schedule of technical interest payments they will have to pay one way or the other if they are still in business.

<comment><AUTHOR> Hey that is another interesting point, thanks:

>"the schedule of technical interest payments they will have to pay one way or the other if they are still in business."

<comment><USER12> Glad you found it useful.

Another reason technical debt is useful, as touched on elsewhere, is that it's something of a real option.

Options have value - think e.g. Oil futures which are the right but not the obligation to purchase X at Y price in the future.

Real options also have value - certain investment decisions can be delayed (e.g. building a plant or refactoring code) and by delaying them you create an option. Here the option is the right but not obligation to write better code that will actually work :)

I know it's hard to convince clients to change their ways, I have found that telling them they are doing great things they didn't even realize is a good way to start before you "bring the pain". Best of luck!

</comment></comment></comment><comment><USER13> In both the best and worst cases, technical debt gets 100% relieved. In the best case because growth requires an entirely different approach to solving rather different problems, and in the worst case because the company goes bust. Where technical debt matters is in the middle, lifestyle businesses and Enterprise IT and companies that aren't going to come close to monopoly due to market position.

Good luck.

</comment><comment><USER14> Not all problems really count as technical debt, IMO.

To figure out which ones do, I sometimes think about the correctness vs. completeness spectrum.

Correctness is how well the architecture fits the problem, how well edge cases are handled, if it's a service then whether it's built to scale horizontally, etc.

Completeness is how shippable the thing is. Are the important features there, are the bugs not too severe, could the thing live in production without causing severe harm, etc.

Some problem domains live only at the far end of the correctness spectrum, like software that life depends on, flight control, nuclear reactors, etc.

Other domains might be all the way on the completeness side, like the throw- away prototypes, temporary stand-alone marketing sites, etc. If it works at all, you can ship it. Problems there don't count as technical debt.

Obviously most projects are somewhere in the middle.

Most high-growth companies can get away with erring a bit toward the completeness side, and ignoring correctness for many things, because a lot of stuff will be re-written every year or so to handle the changes in the business brought on by customer growth. Since it will be re-written anyway, it might not be useful to think about the lack of correctness as technical debt because you won't ever pay the debt. The real trick is figuring out which things must be perfect now and which can be improved later.

Getting that wrong either way could lead to the failure of your startup. Settle for nothing but perfectly correct architecture for everything, and it might take too long to ship. On the other hand, ignoring severe problems in a critical system might somehow limit growth, cause a legal problem, etc., and eventually sink the company.

<comment><AUTHOR> Thanks, that is a very good metric that you shared with an example:

> "correctness vs. completeness spectrum."

and yes! It's a double edge sword:

>"Getting that wrong either way could lead to the failure of your startup."

</comment></comment><comment> vezzy-fnord It has been asserted that a reason for Friendster's demise was in part internal technical problems that manifested themselves in high page response times, often to the point of being unusable.

Though, much of this was exacerbated my management's insistence on implementing a myriad of side jobs to bring revenue while deliberately ignoring the main part of the website being unresponsive.

<comment><AUTHOR> That's interesting. Thanks for pointing it out, after you mentioned that, I had to look it up:

[http://highscalability.com/friendster-lost-lead-because- fail...](http://highscalability.com/friendster-lost-lead-because-failure- scale)

> "technical difficulties proved too pedestrian for a board of this pedigree. > The performance problems would come up, but the board devoted most of its > time to talking about potential competitors and new features, such as the > possibility of adding Internet phone services, or so-called voice over > Internet protocol, or VoIP, to the site."

</comment></comment><comment><USER15> My previous employer failed after almost ten years of venture funding (mostly because it is pretty hard to make a good return on ten years of investment). We had a lot of technical debt. It did not kill us, per se, although I think it was _correlated_ with problems in the company: a lot of things would have been easily resolved had we been less pressed for engineering time. We never had enough revenue to grow the engineering team significantly, and what engineers were there were often fire-fighting. There may have been indirect causal effects like causing good people to get frustrated and leave, or simply causing good people to get frustrated and avoid working on certain problems. We had a few people who flipped out and started repaying the principal (including myself), but they were taking limited engineering time from things that the company needed day-to-day.

It may be true that technical debt directly kills some products, but if it's bad enough that you can't get certain things done, even if what you do instead is also important to the business, the debt is a sign of bad engineering management (either in the past or the present). And bad engineering management _will_ kill the product.

</comment><comment><USER16> Technical debt is more manageable when the core business data is accesible and in sort of usable state. Otherwise a lot of time is lost on improving the state of the data. Time that could be used to create features people will pay for. That's the only constant I've ever recognized in failing startups

<comment><AUTHOR> Thanks. Right, perhaps having some pieces of a technology solidified could help with moving on to new features.

For example an API that users pay for would only be updated or relied on when it's debt free.

<comment><USER16> Charge money when something is debt free? I'm interested in learning more about that, please.

</comment></comment></comment><comment><USER17> Never. It's almost always failure to find customers or running out of money.

</comment><comment><USER18> I'm still unsure exactly what technical debt is?

<comment><USER19> I haven't looked for a precise definition, but it's what you will have to spend in the future for problems that exist in your technology today. Those problems could be outright bugs, or design choices that will make it more costly to continue development work in the future as the product and business follow a normal course of growth.</comment></comment>


Learn you an Agda - <AUTHOR> http://learnyouanagda.liamoc.net/ <comment><USER2> For enlightenment learn Agda. For transcendence, learn Idris ;) I would like to learn either Idris or Agda for their relationship to mathematics, but I am tempted by Idris also being able to do general programming with its compile targets. How does Agda compare to Idris on this point?

<comment><USER3> Runtime performance for Agda... well, it leaves a lot to be desired, at least with respect to the MAlonzo compiler. For example, operations on the natural numbers à la Data.Nat get very slow very quickly-- but while the representation used by Data.Bin speeds things up a _lot_ , you just can't do the same Peano-style structural induction on it.

</comment></comment><comment><USER4> > _Agda's logic is constructive, which means it's impossible to prove the > statement that all statements are either true or false, . This proposition > is called the law of the excluded middle. This makes it impossible to > perform, for example, a general proof by contradiction, or other similar > "classical" proofs from mathematics. There is a large philosophical debate > among mathematicians (those who follow Hilbert, and those who follow > Brouwer) as to which is better_

Still, for a subset of statements and properties it is possible to construct a law of excluded middle, but it has to be a constructive proof. The whole idea of Constructive Mathematics is that you have to show a proof for both, the statement and it's negation to hold their respective truth values. The impossibility to proof the first statement, that of all statements the negation is implied, is surely tautologic, assuming it would hold true for _most of all_ statements, just not a class the opening one is part of. This class is an empty element.

I would basically assume this means nonsense statements can be constructed, but they can't be used in constructive proofs. It doesn't imply that there has to be a class of nonsense statements where the opposite still holds value. I'm cautios of the definition for statements like there or in the multiple incarnations of the liars paradox (whats the latin plural). IE. a statement should by definition be provably correct either in positive or negative logic, hi and low to use ee terms, and one should imply the other polarity (edit: just not by using negation, because that's the relation you want to prove, without circular logic). The _empty element_ is like null, but what logic set the true ^ negative or just the true statements have to be defined in I dont know.

Interesting to know that there is a tool for this kinda maths, I was under the impression that the debate tends to inductive methods, but here we have intuitionistic (constructive) logic, I take it.

<comment><USER5> I'm not sure everything you're talking about, but it's relatively easy to reason about constructive logic by abandoning the idea that your logic is one of "truth and falsehood"—which are, by modern, western intuition mutually exclusive options—but instead one concerning evidence.

One might believe that the Collatz conjecture is true or false, but we certainly lack evidence for either case today. This isn't exactly three value logic, but instead something else. A logic which incorporates the notion of time and communication of proof.

<comment><USER6> > which incorporates the notion of time and communication of proof.

Certainly constructive logics (typically) incorporate a notion of evidence/proof. But constructive logics don't incorporate notions of time or communication (of proof, or in general).

<comment><USER5> Yeah, to be clear, those notions aren't internalized... they just bring into clarity the need for those processes to happen in the meta level.

</comment></comment></comment></comment><comment><USER3> This was one of my first encounters with Agda, and it's a language I've developed something of a soft spot for. I highly recommend playing with it if you have any interest in Haskell or ~undergraduate-level pure mathematics.

</comment><comment><AUTHOR> I also found the following very useful:

* A Demonstration of Agda ([https://www.youtube.com/watch?v=8WFMK0hv8bE](https://www.youtube.com/watch?v=8WFMK0hv8bE))

* Agda Tutorial ([http://people.inf.elte.hu/divip/AgdaTutorial/Index.html](http://people.inf.elte.hu/divip/AgdaTutorial/Index.html))

</comment><comment><USER7> That was the most enjoyable and accessible introduction to Agda that I have ever seen. If you are the creator, please continue the development of the tutorial.

</comment><comment><USER8> > At the time of writing, it is only really feasible to edit Agda code using > Emacs.

Why?

<comment><USER9> It's really just basic editor support. Agda is quite an interactive programming language, so you need lots of shortcuts, syntax highlighting, autofilling etc in order to get productivity at all (especially since you're constantly moving stuff in and out of a repl)

[https://youtu.be/pP7ynVdVY9A?t=13m](https://youtu.be/pP7ynVdVY9A?t=13m) gives a decent example.

As to why support isn't in other editors, it's probably a combination of: more code to support, no one wants to write it, and the fact that Agda is less amenable to being sped up by efficient usage of editors (as it is a more interactive process with less copy pasting).

<comment><USER10> I have to disagree to that one needs syntax highlighting and autofilling to be productive in Agda. I have written a fair amount of agda code in Acme, which has neither. Rather I find these bells and whistles distracting — and prefer giving my complete attention to the task at hand.

I think that writing agda code interactively in Emacs is fine, but it is certainly not the only way to write good agda code. In fact writing without the interactive support encourages better code structure. One is more likely to factor a complicated line into several help terms (using 'where', for example), which makes the code more readable.

</comment></comment></comment><comment><USER11> Does Agda have any significant proof automation yet compared to say Coq? If not, how practical is it to use?

<comment><USER4> there is an agda shell for interactice proving in emacs, with some automations.

 _convenience offered by agda-mode: automatic generation of case analysis, refinement of holes, autocompletion of code based on types to name a few._ [http://lambda.jstolarek.com/2013/12/idris-first- impressionsi...](http://lambda.jstolarek.com/2013/12/idris-first- impressionsions/)

</comment></comment><comment><USER12> don't forget about [https://www.fstar-lang.org/](https://www.fstar-lang.org/)

</comment><comment><USER13> First step: Learn Haskell :D</comment>


VS2008 Stepping into framework source code - <AUTHOR> http://referencesource.microsoft.com/serversetup.aspx Configuring Microsoft Reference Source Server <comment><AUTHOR> Since January 2008, Microsoft has enabled a public symbol server containing source code for most of the .NET Framework libraries. This means you can step into the source code for System.Web.dll and various other core assemblies, which is extremely useful when you have an obscure problem and not even Google can help. This contains more information than the disassembly you might get from Reflector you get the original source code, with comments.</comment>


Netflix now runs on Linux without tweaks - <AUTHOR> http://www.themukt.com/2014/10/10/breaking-netflix-now-runs-linux-without-tweaks/ <comment><USER2> Closed-source DRM-based spyware-browser from Google still required.</comment>


Takeaways from Effective Altruism Foundation's Hiring Round - barry-cotter https://forum.effectivealtruism.org/posts/d3cupMrngEArCygNk/takeaways-from-eaf-s-hiring-round <comment> barry-cotter This is a great description of a well organised, detailed hiring process that others could profitably look at as a model. I found the referenced article's [1]finding that work sample tests add ~nothing to general mental ability tests especially interesting in light of tptacek's extremely positive experience in using them for hiring in the past.

[1]The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 100 Years...

> It appears that almost all of the validity of work sample tests is captured > by GMA measures, because the incremental validity is essentially zero.

[https://www.testingtalent.net/wp- content/uploads/2017/04/201...](https://www.testingtalent.net/wp- content/uploads/2017/04/2016-100-Yrs-Working-Paper-on-Selection-Methods- Schmit-Mar-17.pdf)</comment>


Programmers vs. Zombies - <AUTHOR>

Recently, I've been trying to learn Ruby on Rails via  Rails for Zombies (www.railsforzombies.org), and it's gotten me wondering: what's the deal with Zombies?  (Yes, I'm going to use a capital Z.)
Our monsters have always described our contemporary fears. I think these are the eight most revealing monsters through the ages:
1. Ancient Mesopotamia  - Golem: fear of the implications of Origin stories
2. Dark Ages, Europe - Werewolf: fear of the dual nature of self
3. Victorian Era - Vampires: fear of blood/rabies/sexuality/nobility
4. Industrial Revolution - Frankenstein: fear of the implications of medical science
5. Pre-World War II - King Kong: fear of the undiscovered/uncharted/unconquered
6. Post-World War II - Godzilla: fear of the discovered/nuclear technology
7. Baby Boom - Robots: fear of rapid technological progress
8. Internet Revolution - Zombies: fear of the lack of technology
Somewhere between #7 and #8 we changed our minds about technology.  We went from being afraid the future would move on without us, to being afraid that we'd lose everything we've built.  And take a look at who creates the monster - it's not the guy tinkering with science anymore, it's usually some prideful oaf abusing the scientist.  Also note whose skills become fearfully useless in the Zombie apocalypse - the programmer.
To me, Zombies teach us:
1. People love technology and fear it's demise.
2. Many people are programmers and would fear the demise of their usefulness.
3. Most people believe programmers and their skills are the future - if they become useless, the future becomes a scary question mark.
Programmers, you are the world's only hope against the coming Zombie apocalypse.  Only you can keep the Machines running to defend us against the onslaught.  Who better to defeat the brainless army lumbering toward us than the tastiest brains on earth?  Some ways to help:
1. Consider designing missile software in your free time that could be used to prevent alien asteroids from reaching earth.
2. Do not let your hubris cause you to join LulzSec, hack secret biological weapons facilities, and release strange puffs of green smoke into the atmosphere. I'll be so pissed.
3. Given the possibility you will not heed #1 or #2, at least become a ninja so you stay alive long enough to engineer us back out of the coming nightmare. <comment><USER2> Personally, I think zombie movies reflect 'fear of societal collapse' more than 'fear over the absence of technology'. (The absence of technology is certainly part of a societal collapse, but societal collapse is more than just the absence of technology.)

Sudden, unpredictable, uncontrollable events - large stock market fluctuations, 9/11, the housing crisis, almost certainly the upcoming sovereign debt crisis - have made people very uneasy.

Zombie movies give people an opportunity to think about what they'd do in a societal breakdown, reassuring them on some level - since unlike a sovereign debt crisis, the zombie apocalypse isn't going to happen. Thinking about barricading the windows and aiming for the head is fun; thinking about total market failure is horrifying.

There's also a certain segment of the population, the underclass that's being left behind, who sees nothing in our society for them and would _love_ a hard reset. For them, zombie movies feed into happy fantasies about their rugged gun-toting individualism. (The hero-protagonist in a zombie movie is never, say, a stockbroker from New York.)

<comment><AUTHOR> Good thoughts. Something else I've been wondering about - why do you think the reset always brings us to Pre-Industrial Revolution (lack of infrastructure) - as opposed, to say, the Dark Ages (lack of scientific method)? Specifically, the apocalypse never seems to impact survivor knowledge, just the speed at which they can rebuild?

<comment><USER3> There are a really huge number of books around - it's unlikely that they'll all be destroyed, unless some doomsday cult starts burning every book they can find, so there would be a large amount of knowledge that could be recovered without much difficulty. The scientific method is a fairly easy thing to know; it doesn't seem likely that it's just going to vanish. Realistically, there would also be a lot of people hiding out in bunkers, mountain cabins, or other areas where the number of zombies would be more manageable. So, while a large amount of knowledge is going to vanish (anything computerized is going to be hard to access unless you have a generator), most of it won't be hard to recover unless you wait too long.

I highly encourage you to read World War Z (written by the same guy who wrote the Zombie Survival Guide); while it doesn't directly address most of those questions, you can read between the lines to find the answers. It is, admittedly, only a type 1 Apocalypse, by the TVTropes scale, but still.

</comment></comment></comment><comment><USER4> :) I like how you outlined the discussion.

Now, there are many different Zombie scenarios, but I can understand why you see Zombies corresponding to a fear of loss of technology. Although I always took the fascination with Zombies as:

(besides being the "post-apocalyptic scenario" where you have to figure out how to survive)

\- a fear of losing individuality/being slave to society, or the sort.

\- people feeling others in society are some sort of "monster" or "animals" consuming each other,

\- which makes people feel frustrated, creating a schism between wanting to see it all destroyed (especially if you're a "forever alone"), but at the same time seeing those that reflect what you are/were and your friends and loved ones, becoming something else; join them or psychically "kill" them ("disconnecting" the bond).

And the end product (hitherto): stories about it (or a proletarian revolution?).

Regarding "Also note whose skills become fearfully useless in the Zombie apocalypse - the programmer."

Dude. If the dead starts walking and your worries are finding a job then maybe your problems lies elsewhere.

... Code. Need to code...

<comment><AUTHOR> I think you're right that the feeling others in society are some sort of "monster" is really key. I heard a podcast of This American Life that interviewed Mr. Rogers, and I remember this moment where he's asking a man why they don't talk with their neighbors, and he has this way of making them open up like they're a kid. And the guy responded something like, "I'm afraid they won't BE like me. Like a monster or something."

And that's where our modern monsters come from (and maybe always have): these unspoken, emotional, real-life fears. Like the fear of 'Others', I would suggest that the fear of finding a job is probably weightier than the fear of Zombies, and if they DO start walking... it will be a relief to stop worrying about losing/finding a job. :)</comment></comment>


Physics Forests - <AUTHOR> http://people.inf.ethz.ch/sjeong/physicsforests/ <comment><USER2> That is... an unusual use of regression, to say the least. When I see something sufficiently off the wall, I always wonder how the authors happened to think of that. It also raises a lot of questions like, how much of physics can be usefully approximated by some random forests? Could you replace most of a physics engine with an appropriate small neural network?

<comment><USER3> Random Forests are part of how the LHC found the higgs, based on my armchair understanding of the slide decks.

Measurement, Monte Carlo and Machine Learning form an interesting triangle. My impression of the LHC pipe is they used monte carlo model sims to train up classifiers that would flag the data relevant for distinguishing alternate models.

<comment><USER4> The major difference there is that LHC data measures processes which are inherently stochastic (because QM), so using these methods is pretty natural. The OP, however, is applying the same methods to highly complex, but nominally deterministic problems.

<comment><USER5> I'm curious about this. A lot of us in the computational world have to deal with able but slow iterative methods for things like fluid dynamics and electromagnetic simulations. I'd be interested to know if this works with MHD, and how well it does.

<comment><USER4> It probably would, but it is harder to get performance guarantees. Seems like a taylor expansion on steroids, but with the taylor expansion you get analytical tractability (to prove correct convergence) along with speed, which you don't get in this case.

</comment></comment></comment></comment></comment><comment><USER6> If it isn't based on the Navier Stokes equations, can I actually use this to reliably simulate fluid behavior? Or is this more useful for creating pseudo- realistic water scenes?

<comment> daniel-levin The main goal of mechanics is finding solutions for important variables of interest, from which every other dynamical variable can be easily computed. In fluid mechanics, the main objective for a given problem is finding a velocity field (once you have this, you're done). Since the NS equations are non-linear partial differential equations, finding exact solutions is impossible in most cases. Some notable non-trivial exceptions exist [1]. Navier-Stokes is extremely successful in describing flows in many real physical situations. It is believed, but has not been experimentally confirmed, that Navier-Stokes successfully describes turbulent flow. Most physical theories have a fairly well known domain in which they are applicable and not applicable (Newtonian physics breaks down at velocities near the speed of light). Currently, we don't even know if solutions to NS always exist, never mind make physical sense [2]. With this in mind, we have to acknowledge the uncertainty of the correctness of NS in describing _an arbitrary flow_. It is backed by well- grounded theory (parts of the equations can be derived from conservation laws) and works under many different conditions. This comment landed up longer than I expected, but my main point is that the question can we "reliably simulate fluid behavior" for an arbitrary fluid - is largely up in the air [3].

However, we should restrict our attention to flows that we very strongly believe are correctly described by NS. From the original paper:

>> The main problem of our method is the same as the weakness of all machine learning approaches; the learning methods are not capable to extrapolate the model far outside the data observed during training.

The training data are existing numerical simulations of specific fluid flows. The learning algorithm _learned fluid flow dynamics_ from simulations. This is extremely impressive. It also achieves a significant speed up in simulation time - which is also very impressive. However, as the authors say, it does not generalise well. So, in short the answer to your first question is probably "sometimes yes, but in general not really".

edit: I should add that the simulations in the paper are decidedly based on Navier-Stokes.

[1] [https://en.wikipedia.org/wiki/Hagen%E2%80%93Poiseuille_flow_...](https://en.wikipedia.org/wiki/Hagen%E2%80%93Poiseuille_flow_from_the_Navier%E2%80%93Stokes_equations)

[2] [https://en.wikipedia.org/wiki/Navier%E2%80%93Stokes_existenc...](https://en.wikipedia.org/wiki/Navier%E2%80%93Stokes_existence_and_smoothness)

[3] [https://en.wikipedia.org/wiki/Turbulence](https://en.wikipedia.org/wiki/Turbulence)

<comment><USER7> I think they are using navies stokes to generate the dataset to learn from.. and the regression forest learns how to reproduce what navier stokes did with a bunch of smaller timesteps, but learns to do it in a single timestep.. which sounds amazing so I'm probably wrong

</comment></comment></comment><comment><USER8> This is probably a stupid question, but could an approach like this be used for more efficient ray-tracing?

<comment><USER9> Not a stupid question at all! Not exactly the same approach, but there's been some recent interesting research on neural net and ML driven global illumination:

[0] [http://www.jiapingwang.com/files/shadebot_sig13.pdf](http://www.jiapingwang.com/files/shadebot_sig13.pdf)

[1] [http://cgg.mff.cuni.cz/~jirka/papers/2014/olpm/On- lineLearni...](http://cgg.mff.cuni.cz/~jirka/papers/2014/olpm/On- lineLearningPPMinLTS.pdf)

</comment></comment><comment><USER10> It would be interesting to see an analysis of the models inferred by this method, to see what kinds of approximations they're learning.

</comment><comment><USER11> Reminds me a lot of various machine learning approaches to molecular dynamics - basically trying to fit a cheap function to the energy landscape. I wonder if the more classical (rather than quantum) nature of these systems makes it more approachable?

</comment><comment><USER12> So in my layman's understanding of this, instead of a normal physics engine, which simulates particle movement in small timesteps, this treats the whole system as more of statistical problem? How does machine learning factor into this?

<comment><USER13> ML is just a must-have nowadays.

</comment></comment><comment><USER14> Can anyone explain very roughly what's going on here? I'm familiar with physics engines, but couldn't make heads or tails of the paper or wikipedia's article on random forests...

<comment><USER15> Particularly, I'm wondering if this method converges to the exact solution under all conditions (for various definitions of "converging").

</comment></comment><comment><USER7> I want to see a webgl shader implementation of this!

</comment><comment><USER16> as one of the authors, I might be able to answer some of the questions raised in this discussion

cba9 - In theory given sufficient amount of data (which can be generated), any physics engine can be approximated by a machine learning algorithm, however it might be tricky to make it faster than an existing solver. We got to the conclusion NN can not beat the fastest solvers (PBF). This might change with the newest development of the hardware.

krapht - The feature vector is based on N-S equations, so there is a potential to approximate the solver to any precision

david_ar - We tried to analyze the model, and as for many learning approaches it seem kinda impossible to see what is going on.

fenomas - Instead of solving diff equations, we train a regression model, which predicts the same thing as a numerical solver.

lifeformed - any regression is a machine learning approach by definition

daniel-levin - at the time we wrote the paper, we were not sure about the generalization properties. We didn't find any failure case either (if we did, we would just include such simulations in the training set). However, in the last week before the conference we tried various different forms of additional forces to simulate sand/ash/whatever and to do so we used the model trained only on water simulations. Surprisingly, it generalized quite well. However, not extrapolating outside the training set has some advantages - our method is stable and simply never diverges even if we try really hard.

irascible - after we do some optimizations (I guess it can be sped up by a factor of 10 or so) and finish/publish all kinds of other materials (smoke, fire, foam, elastic bodies, fractures, cloth, ..), we plan to do library, a plugin for Unity, etc and licence it</comment>


Proving a negative isn't impossible - <AUTHOR> https://medium.com/@jeremie_sharpestminds/you-can-absolutely-prove-a-negative-and-i-can-prove-it-35cfb87a8bc4 <comment><USER2> That was a shitty argument. It already failed to establish what is commonly refered to by "negative" in this context. You could criticise the term, that it is too general to hint at a wrongly infered statement in a special case. But the author does something else without really establishing anything. Going to dig in my garden now.

</comment><comment><USER3> You can't prove a negative because of Hume's "problem of induction", which boils down to this: we don't know that the laws of physics are constant throughout space-time. This is where the idea came from that "all the swans ever observed are white" cannot prove that "no black swans exist".

You should read some Hume and Ayers and try this essay again. It doesn't hold together that well, and this is extremely well-trodden ground in the philosophical world (including your specific premises, though they don't make a lot of sense).

And, as far as your suggestion that nothing can be proved, this is a core topic of almost all philosophical work. It's been argued every possible way, including that nothing is real, only thoughts are real, etc. The umbrella term is ontology.</comment>


Kroger automating its operations to challenge Amazon - <AUTHOR> http://money.cnn.com/2018/05/17/technology/kroger-ocado-grocery-delivery/index.html <comment><USER2> Even with Amazon's logistics and the Whole Foods acquisition, it's interesting to see Walmart and Kroger still hold major market share positions in the U.S. grocery market: [https://www.cnbc.com/2017/06/16/amazon-whole-foods- control-o...](https://www.cnbc.com/2017/06/16/amazon-whole-foods-control-only- sliver-of-the-grocery-market-for-now.html)

</comment><comment><USER3> _The Kroger (KR) deal will see the American grocery giant take a 5% stake in Ocado (OCDGF) with an investment worth about £183 million ($247 million). Ocado said it will begin setting up Kroger with various systems to help it manage warehouse operations, automation, logistics and delivery route planning in the US._

the Amazon vans are all over our neighborhood. still, i look forward to this additional home delivery service.</comment>


Performance experiments with matrix multiplication in Rust - <AUTHOR> https://vorner.github.io/2018/05/12/Mat-perf.html <comment><USER2> I'm always impressed how much faster you can go compared to a naive algorithm - if you know what you are doing. This clocks in at 730x and that might not be the peak.

Compare: The original steam enginge trains went I believe 10km/h, so if it received the same level of speedup, it reaches somehere between mach 5 and 10. You can classify this algorithm as hypersonic ;-)

<comment><USER3> Tangent: There's a square power law for velocity due to wind resistance. People aren't impressed that a pro bicyclist is going "only" twice as fast as them but that's four times as strong, for three times as long.

If modern train engine produced 700 times as much power that's only 26 times faster for the same cargo (don't they pull more now?) And we do have trains that can go 260 km/hr... just not with cargo.

progress, man.

<comment><USER4> I actually find the practicalities of algorithm choice even more fascinating. For example, Strassen's algorithm is asymptotically the fastest known algorithm for dense matrix multiplication in certain situations (medium sized matrices), but it is rarely used because it is complicated to implement and less numerically stable.

<comment><USER5> > Strassen's algorithm is asymptotically the fastest known algorithm for dense > matrix multiplication

Coppersmith–Winograd approaches take the crown for asymptotic scaling in matrix-matrix multiplication [1].

It kind of blows my mind that a tight bound on the complexity of matrix-matrix multiplication still isn't known!

Edit: Some quick googling shows that François Le Gall took the crown in 2014.

[1] [https://en.wikipedia.org/wiki/Computational_complexity_of_ma...](https://en.wikipedia.org/wiki/Computational_complexity_of_mathematical_operations#Matrix_algebra)

<comment><USER6> The problem with the algorithms faster than conventional mm is, that they are defined on mathematical numbers.

E.g. strassens-algorithms assumes something like:

(A+B)C - BC = AC

With the convential floating point representation this becomes a problem if B >> A because of rounding errors.

Most algorithms faster than Strassen assume:

(( fA + B)C - BC)/f = AC

and

(fA + B)C = BC +fAC

f is then choosen in such a way, that fAC is neglegible. But if that is the case fA is much smaller than B and we automatically get numerical problems.

There is a procedure to remove the fAC terms (which would imply that f could be choosen close to 1) which armortizes over enough recursions. But for Coppersmith-Winograd (and derivatives like Le-Galls algorithm). We need to divide the matrix in each recursion step into enough submatrices that the indices can be treated heuristically(lets say a division of each index in each iteration in 100 subindices). This would imply the sides of each matrix must be at least of size 100^n_recusion. We won't need to multiply matrices of that size.

<comment><USER7> Thank you for posting your comment. Very insightful and packed with goodies!

</comment></comment></comment></comment></comment></comment><comment><USER8> When we care about speed, we would use OpenBLAS, MKL or Eigen for matrix multiplication. How is Armadillo compared to them, or does Armadillo actually uses BLAS? In that case, what BLAS implementation is used in this benchmark?

<comment><USER9> Armadillo is built on any BLAS you might like to use. (Eigen is a hand- implemented replacement for BLAS, by the way.) So I, too, am interested in what BLAS implementation is used here. OpenBLAS vs. regular LAPACK/BLAS (or ATLAS) can make quite a large difference.

<comment><USER10> Eigen, Armadillo, Blaze, and ETL all have their own replacement implementations for BLAS but can be linked against any version. By the way, MKL supports AVX512, while OpenBLAS does not as of yet. Benchmarks show a factor of 4 between the two for gemm.

<comment><USER11> It's a factor of three for the large-matrix serial case on KNL -- the OpenBLAS issue on KNL -- whereas you might expect a factor of two by analogy with avx/avx2.

For avx512 (and maybe other x86_64, which is now dynamically dispatched) large BLAS, use BLIS. BLIS also provides a non-BLAS interface. For small matrix multiplication, use libxsmm, of course.

Remember that the world isn't all amd64/x86_64, in which case BLIS is infinitely faster than MKL, and it's probably faster even on Bulldozer/Zen. (I haven't compared on Bulldozer recently, and don't have Zen.)

<comment><USER10> I was looking at [0] for that number. You're right, it's closer to 3 than 4; I must have rounded ~12k down to 10k and 37k up to 40k. I could imagine some other factors speeding it up further, as well. There were a number of missing instructions in AVX2 that they've filled in for AVX512 which could play a role.

Thanks for the heads-up RE: BLIS, I'd forgotten about them; it's probably the best option, especially considering its open source status.

[0] [https://github.com/xianyi/OpenBLAS/issues/991#issuecomment-3...](https://github.com/xianyi/OpenBLAS/issues/991#issuecomment-311343769)

</comment></comment></comment></comment></comment><comment><USER12> Check out MOMMS [1], which is a Rust library for matrix multiplication algorithms inspired by/based on BLIS.

1: [https://github.com/tlrmchlsmth/momms](https://github.com/tlrmchlsmth/momms)

</comment><comment><USER13> Interestingly, given this blog post, i'm 100% sure that there are further improvements that can be made. To my knowledge, strassen's isn't actually used by most high performance matrix libraries because the constants are too high that it outweighs the theoretical gains.

<comment><USER14> [https://www.cs.utexas.edu/~jianyu/papers/sc16.pdf](https://www.cs.utexas.edu/~jianyu/papers/sc16.pdf) (paper) [https://www.cs.utexas.edu/~jianyu/presentations/strassen_sc1...](https://www.cs.utexas.edu/~jianyu/presentations/strassen_sc16.pdf) (slides)

<comment><USER13> If I'm reading page 65 of that presentation correctly, naive Strassen (what this person did) should be approximately 75% as fast as MKL on an 8 core machine for a 4Kx4K matrix, and even the improved algorithm outlined in the paper is only approximately equivalent, and I wouldn't call it "Strassen's".

I stand by what I said (although that paper was a cool read!)

<comment><USER14> So ABC Strassen (one of the variants in the paper) is crossing over at 4000x4000 on 10 cores and ahead on fewer cores. I shared this as the current state of performance engineering for Strassen-like algorithms. It offers modest benefits in more practical regimes than the conventional wisdom. I agree that many applications of dense matrices do not benefit.

</comment></comment></comment></comment><comment><USER15> Worth noting that for many use cases using wide floating point vectors from AVX and on is a bad idea. It can take upwards of 2ms to turn on and has weird effects (this delay is due to having to adjust the voltage the core gets).

Unless you know this is what your consumers want, I wouldn't use it in library code.

See [https://gist.github.com/rygorous/32bc3ea8301dba09358fd2c64e0...](https://gist.github.com/rygorous/32bc3ea8301dba09358fd2c64e02d774) for a writeup about it.

<comment><USER11> Reasons you might avoid the widest SIMD in particular cases, particularly for avx2 v. avx512, which include the number of available SIMD units, clock speed, and number of available registers. However, recommending against avx in library code -- we have dense matrix multiplication here! -- is odd. (Various kernels other then fft and gemm have had non-trivial work to use avxN assembler or intrinsics and doubtless others could benefit.)

There are relevant snippets of information on SIMD trades-off around FFTW, BLIS, and, especially, libxsmm. This stuff definitely isn't simple.

<comment><USER15> Yeah, to be clear, I wasn't saying it's definitely the wrong choice for all library code, or anything so strong. More like, it's worth considering the case your users will be using, since for several of them it is likely not the right choice.

For dense matrix multiplication it's probably worth it, but the article is phrased much more broadly in its conclusions, and does not mention the fact that this issue exists (possibly because the author isn't aware of it — which I don't fault them for).

To be clear, a 2ms stall when calling certain functions has the potential to be devastating for anything that wants to run at speeds approximating realtime.

</comment></comment></comment><comment><USER16> Another approach is to start with the obvious 3 nested loops. Then replace with a single loop of n __3 iterations. Take the bits of this single loop variable and deinterleave them to obtain the 3 indicies of the nested loop variant. In other words treat the loop counter as if it 's bits are a3b3c3a2b2c2a1b1c1a0b0c0 and so on. This is simpler with 2 interleaved values but still not bad. It produces a sort of 3 dimensional z-order traversal. The overhead can be minimized by looping with a stride of some power of 2 and unrolling the inner portion to do that many multiply-accumulates.

If you do this while also storing the matrix in z-order in memory the offsets in the unrolled part become constants independent of the matrix size.

This technique is applicable to anything using nested loops where the indicies take you to a lot of memory repeatedly.

I'm still waiting for a couple with bit interleave and deinterleave instructions, but those wouldn't be usable without intrinsics.

</comment><comment><USER17> Can it reach/exceed GPU performance? Thinking of ML craze for GPU.</comment>


Using Haskell to debug a C chess engine - <AUTHOR> http://www.michaelburge.us/2017/09/27/delta-debugging-in-haskell.html <comment><USER2> Personally, I think an engine written in Haskell would be more interesting. Given how ridiculously strong some engines are when running on commodity hardware, a little performance hit wouldn't be the end of the world.

<comment><USER3> I probably won't do an article on it, but I did have a couple-years old Shogi engine in Haskell:

Implementation: [https://bitbucket.org/MichaelBurge/shogi/src/f2370832b328ae1...](https://bitbucket.org/MichaelBurge/shogi/src/f2370832b328ae16e56681fe4123d6912299b1b4/src/Shogi/Rules.hs?at=master&fileviewer=file- view-default)

Tests: [https://bitbucket.org/MichaelBurge/shogi/src/f2370832b328ae1...](https://bitbucket.org/MichaelBurge/shogi/src/f2370832b328ae16e56681fe4123d6912299b1b4/tests/Tests/Shogi/Rules.hs?at=master&fileviewer=file- view-default)

The code & style are pretty similar between the Haskell and the C. I found the C easier to write: The C engine was written and debugged within about 3 days, while the Haskell Shogi engine has a couple weeks of commits.

<comment><USER4> Why do you think the C was easier to write?

<comment><AUTHOR> I looked a few things up after Michael's post, and it seems there are a lot of resources and codified idioms around writing chess engines in C. (E.g. tons of bit-twiddling hacks, many related to the fortuitous existence of 64-bit integers. [Not that you can't do those in Haskell.])

</comment></comment></comment></comment><comment><USER5> do it with checkers</comment>


Ram air turbine - <AUTHOR> https://en.wikipedia.org/wiki/Ram_air_turbine <comment><USER2> They make an horrific noise when deployed, the blade tips are highly supersonic. I've heard one once as a groud observer and I can only imagine the racket they make in the cabin. For that reason they're not tested in routine service but there are ground-testing rigs that spin them up to a fraction of their maximum rotation speed.

The A320 series have RATs but the 737s ( all series ) don't since they remain manually-flyable without electrical power.

</comment><comment><USER3> I'm confused.

It looks like a turbine powered by the movement of ambient air, in this case due to the aircraft's movement through said air.

But the article says:

 _The RAT generates power from the airstream by ram pressure due to the speed of the aircraft._

How is this dependent on the pressure, rather than just the speed of the air? Would a RAT from an aircraft not work on a stationary object subject to wind? Or is such an object also experiencing ram pressure (just the other way around due to reference frame shifting), i.e. all (wind) turbines work due to ram pressure?

<comment><USER4> Aerospace eng. here, you are right in that is a confusing terminology. Ram pressurein this context is a way to refer to the available kinetic energy due to airflow: 1/2 x density x speed^2 (also called dynamic pressure).

A point to stress is that the energy delivered to the plane systems by this auxiliary turbine is thus varying with the square of the speed of the plane. That means that as you slow down for landing, the RAT delivers less and less energy... and the commanding authority of the hydraulic-actuated control surface lowers, making the plane less and less controllable. Not fun!

<comment><USER5> You'd think the RAT would charge a battery acting as buffer. But who am I to armchair quarterback an entire aerospace industry.

<comment><USER6> Having a sufficiently large battery introduces another point of failure, something undesirable in a situation where you're already in bad shape.

You don't deploy the RAT unless shit is going sideways.

<comment><USER4> > You don't deploy the RAT unless shit is going sideways.

We can't also confirm nor deny that RAT is a hand-picked acronym to make the "rats leaving the ship, we're doomed" joke.

</comment></comment></comment></comment></comment><comment><USER7> Here's one of those RATs in motion on a 777:

[https://www.youtube.com/watch?v=FfB2EP71hqY](https://www.youtube.com/watch?v=FfB2EP71hqY)

</comment><comment><USER8> Be sure to follow the link to Pinnacle Airlines Flight 3701. That's a double Darwin Award, well deserved.

<comment><USER9> Wow, that was quite a legendary chain of fuckups. Link for others:

[https://en.m.wikipedia.org/wiki/Pinnacle_Airlines_Flight_370...](https://en.m.wikipedia.org/wiki/Pinnacle_Airlines_Flight_3701)

<comment><USER6> It takes a lot of gall to double down on your mistakes in a situation like that. These two should've been stripped of their license had they survived that stunt.

<comment><USER9> It seemed like they were hoping the 2nd engine wasn't totally cashed. So they lied to the ground about having only 1 engine failure. I would imagine being alive is worth embarrassment but given the prestige of a pilot, I really can't say more than, bad choice.

</comment></comment></comment></comment><comment><USER10> Deep ocean sailing yachts used to use these for electricity generation (for powering things like navigation equipment and radios) but now it's more common to use battery backed solar power.

<comment><USER11> I would say that boats have Wind Turbines rather than RATs.

[https://en.wikipedia.org/wiki/Wind_turbine](https://en.wikipedia.org/wiki/Wind_turbine)

The power comes much more from the wind than the vehicle moving through the air (though I am very aware that the speed of the boat affects the apparent wind the generator will experience)

<comment><USER12> Actually water turbines (aka hydrogenerators) are also an option, but it's less common.

[http://cdn.streamlike.com/secure/Covers/68/17853.jpg](http://cdn.streamlike.com/secure/Covers/68/17853.jpg)

</comment></comment></comment><comment><USER13> Hooke was this by any chance inspired by a reddit discussion?</comment>


Show HN: Geneder inclusive language – a repo and request for comments - <AUTHOR> https://github.com/joelparkerhenderson/gender_inclusive_language <comment><USER2> Geneder?

Did you mean "Gender"?</comment>


Show HN: A very different alternative to Google Reader - <AUTHOR> https://www.feedsapi.org/index.php <comment><USER2> Yeah, let's just resize three 2000x1200 images down to nothing. That won't look like shit and take forever to load.

<comment><AUTHOR> Thanks for the very valuable feedback Kenoch, will check that and try to make the images look less like shit and load faster.</comment></comment>