The shittiest project I ever worked on (2013) - <AUTHOR> https://blog.plover.com/tech/prudential.html <comment><USER2> Having followed HN and similar dev culture (Dilbert etc.) for some time now, I feel a growing contradiction that I cannot fully resolve.

On the one hand big corporations are presented as these incompetent behemoths that are locked in their internal petty office politics, managers are clueless as to what is being or should be developed, and tons of money is wasted, they are wooed by empty marketing etc.

On the other hand these organizations are immensely wealthy and successful.

It may actually be that they are right, and for non-tech-centered companies this whole software stuff is really peanuts and their mental energy is better spent on other business-related stuff. So they will come up with random low- effort comments on all sorts of details because ultimately they (rightly) don't care. What actually matters is to negotiate deals like referral fees, or coming up with other contract ideas that will bring a lot of revenue in.

Or maybe it's just an equilibrium, a local optimum. You don't have to produce "good" stuff (in the developer's sense) without waste, precisely because the other company that you're targeting is also inefficient in similar ways.

I'm really not sure but I think it's important to look beyond just "haha, stupid managers, they can't make up their mind". There must be deeper reasons.

<comment><USER3> I've had this same thought, and have a pet theory about it. I think the answer is something like momentum is so unbelievably useful in business that having it is enough to be immensely wealthy and successful.

As someone who recently created yet another startup, I am reminded daily that literally _everything_ I do has to be done for the first time. It's like molasses covering every part of the machine. When a client needs a proposal you have to write one from scratch, when someone wants payment terms or to negotiate a contract adjustment you have to figure out contract language. You don't have expense reimbursement forms, you don't have job descriptions, you don't have _anything_ to use as a starting point or template for anything you do.

Big companies have all that. They have clients that are planning on using them again this year. They have meetings they took two years ago that are about to be a sale. They have a guy on the third floor that has seen this problem they're about to have once and knows how to avoid it. People instinctively trust them, so they don't have to explain themselves as much. People instinctively fear them, so their contracts get upheld. People that used to work for them now work at potential customers and call them up.

And on and on and on. Having a slightly busted software development process is, in fact, peanuts compared to all this.

Or, perhaps another way to say it is that there's a lot of different kinds of "software" besides the digital ones-and-zeroes kind we think about here. A group of humans all organized around a business purpose with tons of experience and momentum is a high functioning machine as well.

<comment><USER4> This idea of momentum reminds me of Jim Collins Flywheel Effect

[https://www.jimcollins.com/concepts/the- flywheel.html](https://www.jimcollins.com/concepts/the-flywheel.html)

I have to disagree with the broad statement that a busted software development process is peanuts. In highly regulated industries like finance, bad software can cost you billions in fines from regulators.

In these industries, they have to invest in technology and good processes if they want to stay in business.

<comment><USER5> > In highly regulated industries like finance, bad software can cost you > billions in fines from regulators.

You're assuming the regulators are low-level enough to identify a flawed system. A lot of regulations are applied at a policy level (aka process level) and auditors (internal or external) are not always able to identify severe gaps between what's documented in a policy and reality of day to day. Good people and technology matter way more than process.

<comment><USER6> A good process is supposed to protect you from not having good people (or, at least people who can't be good every day).

Sure, if we hire perfect people there's no reason to even have a process. In reality, when that breaks down you have to rely on your process to close those gaps. But, like you said, a policy/process is only as good as the paper it's printed on if nobody follows it.

<comment><USER5> _edit_ I'm using security as an example of a regulated aspect of software/IT systems. I am aware there are others.

100% agree - My point was more that regulators tend to use policy and process artifacts as proof of security compliance. The idea that more highly regulated/audited IT sectors are inherently better at security is probably false. The audits just aren't stringent/detailed enough and there's aren't enough incentives for executives/boards to do more than the minimum. Which is why the values/opinions/ethics of those decision makers becomes key.

The most secure 'company' I worked for was a large research university because it was a priority from the director and we were constantly getting probed by overseas IP addresses. On the other hand, the worst company I worked at in terms of security was in the payment processing world. We were PCI DSS compliant and audited almost continuously by various interested parties (banks, card brands, third parties, etc.). I don't think any of our people were bad/inept but at the end of the day our policy artifacts made us sound/appear way more compliant than we were. The problem was upper management didn't care (allow us time or $$) about closing the gaps in implementation unless we failed audits or pen testing. We didn't fail but no one in IT thought we were doing enough. We didn't have enough auditing and regularly had inexplicable 'bugs' reported that honestly could have been evidence of network intrusion but we had no proof either way.

</comment></comment></comment></comment></comment></comment><comment><USER7> Not me, but one of my favourites would be when a guy joined _our_ team. We were an agency and he'd been hired by the client to eventually take over. On the first morning I walked him through the codebase and showed him how it all worked, how to extend it etc. After a while I said "any questions?" He replied, "yeah, just one - am I meant to be a developer?"

Turned out that when the startup had brought him in as CTO they failed to mention that he was eventually going to be the sole developer.

<comment><USER8> Gotta ask -- how did the next few weeks/months pan out?

<comment><USER7> To his credit he just kinda sucked it up and got on with it. I was happy because eventually I didn't have to work on it anymore. Surprisingly, all the money they wasted on buying rack servers and Oracle enterprise licences didn't net them more than 10 visitors a day and it feel apart. Once it all went pop, him and I crossed paths on other projects too, got to trade stories about the absurdity of it all.

</comment></comment></comment><comment><USER9> I think anyone who has done any consulting with a big organisation probably has one of these stories. As one of the other commenters says, if the end result is something that the client is happy with, and they actually pay you, you should basically consider this a success!

I spent six months building a single-page app (back when that was still an exciting sounding thing) for a major US financial institution; by the time we actually delivered it, the result was so watered down from the initial proposal that I could've just taken some screenshots from what my designer had produced six months earlier and put them in a PowerPoint deck and said "Done!" \- because this was, in fact, what the VP at the company ended up doing, except they were screenshots from the web app we built that no one other than the VP himself ever used.

On the plus side, I became good friends with the VP and the project paid off a significant chunk of the mortgage on my first house.

<comment><USER10> Good thing there are no real problems left to work on in this world or this would be a gross mis-allocation of macro resources.

(No judgement on you OP this is the situation we're in)

<comment><USER11> Ah but if someone paid for it, the problem must be exactly that valuable to them! And if it's valuable to someone, it must be a Real Problem!

<comment><USER8> When it comes down to it, we're not paid to code. We're paid to solve problems with code and if paying you $75/hr as a contractor to build me three webpages with CI/CD capabilities is it, then off you go.

<comment><USER12> $75? That's about half of what I would charge myself, and an agency is going to be $300 minimum.

</comment></comment></comment></comment></comment><comment><USER13> This... this is just my life right now. Get a contract, spend all of my time doing anything other than actually accomplish anything, and then after wasting time adhering to project requirements that don't make sense, wind up delivering something that could have been done in 10% of the time if people listened to me at the start of the project.

<comment><USER14> >wind up delivering something that could have been done in 10% of the time if people listened to me at the start of the project.

I felt that way until I eventually got to a point where people did listen - and it turned out to be false. I underestimate the effort required on first glance and I oversimplify things in hindsight. Also I'm a developer - not a business guy - I can guess a lot of things on the requirement side but a lot of it is outside of my domain. This is something that I have to be aware of now when I'm in a position to give such estimates.

Don't get me wrong - I think there's a lot of room for improvement in almost everything I worked on before and after me - but that gut feeling of how I could have done things was never on point for me. With the article in question - too many people involved in the decision - no way to know the final result for sure until people see it have a bunch of meetings, see the iterations and decide on this - rarely do you have the person with deciding power know/understand these things clearly from the start. And you need to account for bureaucracy BS time-waste - it's just a fact of life when dealing with such clients.

<comment><USER15> Your comment was something like eye - opener for me.

</comment></comment></comment><comment><USER16> My shittiest project was a hostile takeover from another development team.

In the interview, the manager told me a bunch of BS that turned out to be completely false: that it was a new project, that lots of new development was expected, that there where performance challenges to be tackled.

When I got there, I found out on day one that the goal of the project was to take over the code base from an existing development team, with which the customer had lost trust because among other things they refused to use their new shitty trouble ticket system (which was awful), and instead insisted in using JIRA.

Also, they had a server running in Tomcat that they didn't want to migrate to Websphere.

Turns out that we didn't even have access to the code, until we accidentally found that the SVN repo was listed in an infrastructure powerpoint, and we did have a VPN connection set up.

We were at the taking over consulting firm offices and not at the customer, reverse engineering the existing 7-year-old codebase without any assistance from the development team, which didn't even know that they were getting replaced.

After months of reverse engineering and producing useless documentation so that the client manager could say in some meeting that the system was well documented, we ended moving in and mostly doing production support instead of development.

Our managers were afraid that we broke something, so they did everything to make sure that we didn't spend our time coding.

After a few months, I just asked out because it was not development work and this type of work was actually harming my career and even had my consulting manager actually shout to me on the phone and tell me that he would blacklist me at his company.

Software consulting is some of the shadiest businesses out there, beware.

The amount of lies that gets told to candidates just to get them signed at the dotted line to contracts without exit clauses (which was the case with me), the use of outdated job descriptions, the lack of information that you have about the actual job even if you ask a lot of questions in interviews, you really never know what you are getting into until day one.

Here is a tip from what I have learned: ask a ton of questions about what the job will actually be, if they answer evasively or strangely seem like they want to avoid the questions and move on to other aspects of the interview, that is a huge red flag.

<comment><USER17> >> Our managers were afraid that we broke something, so they did everything to make sure that we didn't spend our time coding.

that's like hiring football players and not letting them on the field out of fear they'll score for the other team.

please write a book!

<comment><USER16> Yes its insane LOL Managers were non-coders themselves, and they looked at developers with suspicion.

This happens in a lot of projects, where developers are almost treated like children or assembly-line workers.

I realized through small hints and queues over time that the goal was to take over the maintenance of the project and make sure that we could fix things if something broke, but not add any new features.

They even had a management term for what we were doing. They called it KIR - Keep It Running! LOL

</comment></comment></comment><comment><USER18> > The report I got about the demo was that the real estate people loved it, it > was just what they wanted.

> "But," they said, "how do we collect the referral fees?"

This is why you need to understand what problem a client wants solved and not just build what their suggested solution is. Even if you build their suggested solution perfectly, they're not going to be happy at the end if their original problem isn't solved. Their suggested solution should only be used as a starting point to understanding their requirements.

<comment><USER19> That's a confusion about who the client is. This was a web page, to be used by a public consumer. The real estate people were, in my view, obstructionist old-guard trying to preserve their paper empire. Not the client. They torpedoed an early example of the web removing the middleman and streamlining an industry. Prudential's agents weren't ready for the revolution.

<comment><USER20> Actually, you're very wrong, as I've worked in that trade.

That industry relies very heavily on referral fees to this day. Part of that is matching consumer with broker, and part is brand.

Today it's places like comparethemarket, back then it was Prudential.

So nothing's changed, the web hasn't made any efficiency gains.

The better solution than what they came up with is to have put in a unique forwarding phone number per affiliate, and charge the affiliates a referral fee per call (or perhaps per unique telephone number).

Whether that tech existed in the late 90s I don't know, but it's been available for over a decade now, and was probably available back then in a more analog form. What's nice is that now you get a webhook telling you about each call, so all the billing is automated.

As for when you put your email address into a form on some comparison site? Mortgage brokers bid on those leads. It used to be in the region of £20-50 per lead, pre-2008, and I know someone charging a comparable rate now, plus a cut of commission for subsequent remortgages.

<comment><USER19> Ha! That just shows that web companies have figured out how to put themselves back in as middlemen. So sad.

</comment></comment></comment></comment><comment><USER21> Great story!

Out of curiosity I wondered about the current state of the Prudential website. So I searched, and found Berkshire Hathaway bought the real estate arm in 2012 and now they are Berkshire Hathaway Home Services.

I did a local search on the website, received pictures and basic information (price) for each house... and for each one it lists the same toll-free number as a contact.

So the site he built basically works the same way!

</comment><comment><USER4> As he said he was green, I think this really shows that early on, young developers do not always understand the importance of requirements gathering and problem understanding.

Just yesterday, a junior developer from my team came to me about a request to generate some output from historical data. I went back to the business side and determined it was just a simple switch of several columns in the input data to get the results.

My junior developer at first thought it was some monumental problem that was going to require a lot of work.

<comment><USER22> Yup. Knowing when to go back and clearly define the requirements is a senior type of behavior.

"We don't have time to think, just do!" is a recipe for disaster.

<comment><USER23> They said they wanted everything so the DB displays in a table ... everything.

<comment><USER24> Hey works for me, let me just get my web-scrapper going so I can--hey wait a minute!

</comment></comment></comment></comment><comment><USER25> Oh dear God ... I just finished this exact project this week (ok, 90% similar)[0], everything he says is true.

The insanely messy data (bathroom intergers are a mess!), the firm essentially being just a collection of smaller firms in one building, the massive issues with 'finder fees', the near endless list of VPs that have to pass anything off, the contractors not knowing anything, the 'real' employees also not knowing anything, the hosting of websites, the brokers paying for useless (to them) web-dev, it's all true! Nothing has changed in ~25 years!

[0] I know you're reading this _Jeff_.

</comment><comment><USER26> After working in a web agency for years, my number 1 red flag that the project is going to go badly is the quality of their current website they're hiring you to fix/replace.

If it's solid but just a bit old or needs more features, you're fine. But, if it's crap, look out.

The quality or lack of is never down to the the people who built it (unless it was by the MD's nephew or something), especially if the client blames them for it.

It just shows they can't run a project properly and whatever you end up with will be just a slightly newer pile of crap. And they'll slag you off to the next lot.

Number two red flag is if they say at any point "You tell us, you're the experts!" This can be translated to, "We don't know what we want but we're going to reject whatever you say anyway."

<comment><USER27> > Number two red flag is if they say at any point "You tell us, you're the > experts!" This can be translated to, "We don't know what we want but we're > going to reject whatever you say anyway."

I disagree with this being a red flag. That's exactly why I hired someone to build me a small website. It's not because I don't know HTML / CSS / JS; it's because I'm bad at (visual) design. I wanted someone who knows what they're doing to use _their_ expertise and give me a good product.

<comment><USER26> That's great, you know to hire an expert and let them get on with it.

The problems start when the client either can't or won't explain what they want, then gets upset that what you produce doesn't match what's in their heads. At that point they start trying to undo every decision and things go south rapidly.

</comment></comment></comment><comment><USER28> If he was paid by the hour it would be typical, perfectly reasonable contractor project.

<comment><USER29> Consulting companies know this and they NEVER charge fixed price.

<comment><USER30> However there's a case for fixed pricing and it can work to your advantage: Think of a difficult task or problem that you already solved in similar cases - maybe you already have the heavy foundation of that work ready as a template, maybe you automated complex, repeating steps. Or just think of a simple task that has huge value for your customer. Pricing shouldn't always depend on how long it takes you to finish a project.

</comment></comment></comment><comment><USER31> At least he didn't have to install Solaris on Sun executive's workstations.

[http://www.art.net/~hopkins/Don/unix- haters/slowlaris/worst-...](http://www.art.net/~hopkins/Don/unix- haters/slowlaris/worst-job.html)

<comment><USER32> I feel like I'm missing something in that article. Probably some historical context. Is it really about execs at Sun pranking each other by installing Sun's own flagships OS on each others' computers? Isn't that like "pranking" Satya Nadella by making him use Windows 10?

<comment><USER33> "On September 4, 1991, Sun announced that it would replace its existing BSD- derived Unix, SunOS 4, with one based on SVR4. This was identified internally as SunOS 5, but a new marketing name was introduced at the same time: Solaris 2."

I guess the execs didn't have a high opinion of their new product.

<comment><USER31> The Day SunOS Died

        "Bye, bye, SunOS 4.1.3!
        ATT System V has replaced BSD.
        You can cling to the standards of the industry
        But only if you pay the right fee --
        Only if you pay the right fee . . ."

[http://www.poppyfields.net/filks/00070.html](http://www.poppyfields.net/filks/00070.html)

For context, the guy who wrote "The Worst Job in the World" email was Michael Tiemann, one of "open source's great explainers." ;) Now he's pranking IBM executives by installing RedHat Enterprise Linux on their mainframes.

[https://en.wikipedia.org/wiki/Michael_Tiemann](https://en.wikipedia.org/wiki/Michael_Tiemann)

</comment></comment></comment></comment><comment><USER34> The only thing that's ever worked for me to avoid these scenarios is getting all the relevant stakeholders physically in a room for a couple days and drawing mockups, looking at their systems, learning the domain lingo, running reports over their data, etc as we go. Otherwise the domain knowledge for a proper solution just isn't there, even for a 'simple' project.

Unfortunately I also hated every minute of doing this and have switched to a back-end job where I never have to interact with a customer again!

</comment><comment><USER35> I can say, as a seasoned web apps outsourcing contractor, there are two sides of the medal.

The bright side: clients usually come to me with just an idea, plan or sketches, almost all of my projects were built from the ground up, I totally technically owned them. This is a delight for any coder (and I code a lot, not just manage the team).

The dark side: many startups and green business ventures fail, and the failure rate is around 80%, even larger in the longer run. And I do the apps from the ground up, nurture them.. but sometimes I feel myself a gravedigger.. out of any 10 projects maybe 2 or three are still there after a couple of years.

</comment><comment><USER36> It's an amusing story, I've seen this countless times as well. I think what finally sunk in for me was that if you're focused on saving money or efficiency on small projects then the upper limit of benefit to the company is less than the budget for the project. Instead if you focus on optimizing for the market, the profits can be unbounded essentially, and any wasted money to get to market is simply part of the learning process.

And if you're the contractor having to run around as the plans change, well just make sure you get paid enough :)

<comment><USER37> As contractor I like jobs where I can basically recommend "Just don't build this at all." For example the young startup that wanted credit card payment and automation for essentially a two digit number of invoices per year for a target market where bank transfer would be much more common. Replaced with an excel sheet. Customer happy. I don't want to build crap just for the money that's in building crap. Let me actually add value.

<comment><USER38> Umm... did you earn the kind dof money the pointless project would have gotten you?

<comment><USER39> Probably no but they surely built goodwill with that company and as a result can help them build a good reputation. This is often overlooked but people will remember that.

Or they could just be satisfied with doing the right thing, rather than trying to extract as much money as they could.

</comment></comment></comment></comment><comment><USER40> Thanks for sharing. Good catch at the end. It's easy to forget this as a developer. But usually technically details are just that in the bigger picture - details. They're only important if the bigger picture is somewhat correct.

</comment><comment><USER41> As an aside, "Tell me about a project that failed" is one of my favorite interview questions to ask. Failure is endemic in our industry. A candidate that only tells you about success isn't telling you their whole story.

And a candidate who talks about failures in terms of what everyone else did wrong, but never acknowledges any failures or learning experiences of their own, is telling you something important, too... they're telling you that either they never learn, or they don't tell you the whole story.

</comment><comment><USER42> These are the instances where I feel it is definitely worth keeping some people around who act as glue. (or old timers).

In our company - we have a tech lead role which is supposed to be the technical point of contact for a given team. This person is expected to have answers or have a way to find the right answers about the nuances of the team.

The good ones are invaluable. Whenever new projects come up, the modus operandi is to write a proposal, review it with your current team for sanity (on whether it achieves its purpose) and then with this group of relevant stake holders / tech leads to ensure you're not getting some giant thing wrong that can nuke some other part of the business.

The problem I see with this role though - we (I am the one for analytics and one of the TLs for Data Infrastructure at Snap) have to handle 20-25 hours of meetings around all these coordination AND ensure some work gets done in your own area so that you are in touch with the code bases you are fiddling with. So as much of an alluring title that is, you are giving up deep dive development (like I was able to do building storage engines for DynamoDB). Tradeoffs :/

And there are cases where the tech leads don't have the answers (new or incompetent) OR they think of a weird case couple of months down the line, but this works out overall.

I probably should write a blog about this - but anyway, I don't know how this works out in the non-software world: Why wouldn't you have a knowledge person/council around?

<comment><USER43> > The problem I see with this role though - we (I am the one for analytics and > one of the TLs for Data Infrastructure at Snap) have to handle 20-25 hours > of meetings around all these coordination AND ensure some work gets done in > your own area so that you are in touch with the code bases you are fiddling > with.

If you figure out the answer to this PLEASE share it with me. This describes my current role; whenever I focus on one aspect (eg actually writing a few lines of code) the other (keeping up with planning) suffers.

My suspicion is that "tech lead" in this context basically means "manager," but employers know many devs are scared of the word manager, so we have the highly ambiguous "tech lead" title, which allows dev managers to be in denial about their actual role.

</comment></comment><comment><USER44> There's a lot wrong with this whole thing but the end solution is the wrong way around too - you don't require the client to then take a further step, you get their contact details and then reduce the friction by calling them.

The solution should surely be "enter your phone number and zip code and we'll contact you". Then the central referral agency contact the person immediately, to keep the momentum & check the person's details, reward them for their effort in finding the company, and arrange for the next step -- ideally an at home meeting with the particular agent you booked them with.

Just "right, now call this number" is a bit daft, isn't it?!

</comment><comment><USER45> > "how do we collect the referral fees?"

This is the underlying problem, and the reason why "disruptive" startups can be successful in the first place; making something more beneficial for the consumer in a big organisation can be at the expense of one of the individual units which can prevent it, even if the change would be advantageous for the business as a whole.

</comment><comment><USER46> This is something every freelancer have seen. Who ever introduces you to the job rarely knows what the end-product should look like and it really proves the value of creating a minimum viable product, showing it to customer and pivot from that to where you might need to go. It this case the MVP would have been the final product.

</comment><comment><USER47> Sounds like the OP fessed up to the problem at the top: despite hating large companies, they signed up to a fixed bid project as an independent contractor. Given how big companies work, that's recipe for danger.

Scale is very powerful, but it takes a lot of work to get things done. Large companies have a ton of waste, but getting to "this can work for all of us" has a very big impact.

</comment><comment><USER48> Fun story containing a valuable lesson for developers thinking about going into consulting, especially for larger companies.

</comment><comment><USER49> Do you think this is why developers need to be made aware of user journeys? In my experience, just spending some time on thinking through the user journeys has done wonders to my understanding of features.

Edit : The article doesn't mention anything about user journeys. Hence the thought :-)

<comment><USER50> In an ideal world yes - but in some situations its just not practical. For example, I once worked on a (large) project for a company I worked for where we were the customer and developers were out-sourced about four levels deep - they weren't even clear as to who the top level customer was let alone what the user journey was.

</comment></comment><comment><USER51> In the end, something overpriced but usable was delivered. This project was a success. I wish I could say that of all the projects I have seen.

</comment><comment><USER52> Feels like things like this are super common in the consulting / non-tech company world. A lot of small factions all wanting their little requirement slotted in, and the whole product end up either being completely incoherent or there are large parts that are wasteful.

For me it was a project a while ago with a telecom company. They wanted a place to manage their customers' communications. One group was adamant that we needed to surface customer's emails in this portal, instead of providing say just a link to whatever their email provider is. Essentially they wanted us to build a clone of gmail / hotmail / yahoo mail but only with SMTP / POP / IMAP access to the data. We tried to tell them how expensive and not useful this would be, but no had to build it.

Obviously we couldn't build a gmail clone in the time we had that was better than gmail, and there was pretty much no incentive for users to use our version instead of gmail proper. So after spending more than half of the total project time (and budget) on this email functionality, six months after launch we had a very small handful of users linking their email and most of them didn't do anything with it. Even after being presented with the usage data, we couldn't just get rid of this feature, because that would be admitting failure.

</comment><comment><USER53> Wish the article would point out in the end, that this is a perfect example of why you have to talk to the "final" customer at the start of a project.

</comment><comment><USER54> I just started at a large international firm. ~1 week in I found that a collection of beefy AWS machines had been sitting around doing nothing for a least 5 years (potentially 7 years). The base cost? ~100k USD (we are based in Canada, so ~130k for us). Also, three of the machine (you have to have dev, staging and prod!) where running a (presumably) expensive third party financial software library. No one knew who to talk to about the cost of that lib. When I told my boss, the CTO, they just shrugged and implied that it was peanuts in the grande scheme of things. As far as I can tell the C-Level people just break up their days going to meetings talking and kick all the technical debt down the road. I find such waste upsetting, but, try to focus it as fuel to get my own company going (if successful it too will be disgustingly inefficient in 10 years!). _sigh_ 1/3 of the best years of my brief spark of life on utter nonsense. probably best not to think about that.

</comment><comment><USER55> I've been becoming less cynic about workplace bosses and their decisions. I'm not really talking from raw workplace experience, but from noticing things I've read. I think a boss order comes from a perspective that is _probably_ right. They have an understanding that a developer _might_ not have yet, their instinct could be accurate, maybe most of the time.

</comment><comment><USER56> While it was frustrating, think about all the crucial experience (business more so than technical) you gained.

I would rather face a few bad projects early on to learn how to manage expectations, negotiate, handle sunk costs, and not get overly emotionally invested. I've had a project pulled from production a day after launch. Developing a thick skin is very important. If you are only thinking about what the project means to you and the payoff seeing it go to production, as opposed to the larger business concerns, you are being narcissistic and setting yourself up for failure. Conversely, a lot of decisions are imprudent and bad for the overall business and you have to know when to abandon ship.

</comment><comment><USER57> This is the typical case where starting small and simple helps you

1) save significant amount of time ahead

2) give your customer time to identify what they really want

3) help you iterate quickly on newer versions. If something doesn't work, it's easy to throw stuff away

<comment><USER58> All good advice.

Old army observation. No Battle Plan Survives First Contact With The Enemy. Thus the sooner you get that over with the better.

<comment><USER59> As far as I recall, that's actually a quote: "Kein Plan überlebt die erste Feindberührung." by General Moltke. Your translation is exact.

Just wanted to add a source: [https://en.wikipedia.org/wiki/Helmuth_von_Moltke_the_Elder](https://en.wikipedia.org/wiki/Helmuth_von_Moltke_the_Elder)

<comment><USER60> It's also said, about Moltke the Elder, that he only ever laughed twice in his life. The first time was when his mother-in-law passed away, the other time, when he saw Sweden's Vaxholm fortress.

[https://en.wikipedia.org/wiki/Vaxholm_Fortress](https://en.wikipedia.org/wiki/Vaxholm_Fortress)

</comment></comment></comment></comment><comment><USER61> Big companies are diverse. They have deep roots. Google and Microsoft can afford to indulge in spruce goose's for years and it wouldn't make a difference. As long has google has their search engine, they're racking in money. Same with Office.

In a big company, you are a cog, a little pawn. It doesn't matter though. They have a huge army and they can afford to burn people and it doesn't make a dent. They can just outright buy people. Everyone can adjust their morals if they have extra zeroes at end of their pay check.

</comment><comment><USER62> My shittiest project was conceiving, developing and deploying WorldCup '98 live TV graphics. I worked 100 hours/week for 4 months, put on 10 kgs, lost one tooth, and got like a 150$ bonus for the whole thing.

I've posted parts of the story as Quora comments, here's one:

A company named Compusport had secured the contract to provide visually impressive graphics with statistics for WC98, but had the hubristic project of developing the graphic stack from scratch — and that was too ambitious, and a few short weeks before the World Cup the guy in charge with this part threw the towel in… Uh oh.

My company stupidly proposed to help Compusport (the fools :D ). At the time I was CG operator on French TV, and the only person in Europe comfortable with a high-end, IRIX character generator program called Antero. So I teamed with a colleague to build a PC GUI in Visual Basic to drive Antero through the network, and that proved … tough :)

There was several technical hurdles: Compusport main selling point was advanced statistics (it was supposed to be fully automated, IA powered ball and players tracking. This didn't actually work out, but this is another long, painful and interesting story). So we needed to feed the statistics to the TV screen, coming from a central server. Graphics had to be animated (then a world first at this scale), so the CG system had to be remote controlled from the mixing desk to launch animations and effects. Of course on the different sites there were different mixers (Philips, Sony, GVG, Thomson…) working in subtly different ways. We even had to build some custom electronics to interface some of them with our system… An horribly complex network of various hardware and computers using ethernet, ISDN, serial lines and video… a complete nightmare :)

As a "rehearsal", Compusport has planned to use the French Football Cup finals on May 2, 1998. Oh, by the way he also "sold" the graphic design for this event. With noone in sight to actually do it, you know, like a graphic artist, and no budget either.

So here how the typical April day was: my colleague Eric was building the GUI that remotely controlled Antero, communicated with the video mixer (so that the FX operator could control Antero animations from the main mixing desk) and the statistics server (to grab game data to feed to Antero) from 8 A.M. to 10 P.M. or later. Way later. I helped him on the Antero API ("to display that page send this and this and then wait for ACK then send this and this"), while during the day working with Marc Tatou to set up the graphics and the animations : "make the red card come forward than flip. A bit faster. A bit closer. That's it". Of course there was no GUI to program the animations, I had to write scripts with coordinates and vectors in some weird domain- specific language (apparently similar to the one used on some Ampex digital effect hardware) and proceed by trial and error (that grows your beard).

The rest of the day (and night) was spent building the graphic charter for the French Cup finals myself — I became very proficient in Photoshop quickly…

Finally on May 1st, at 4 A.M., I was done with the graphics, I tapped Eric's shoulder "look, I'm done!". No response. I turned to him. He was sleeping soundly, his forehead firmly stuck on his keyboard. We had lost him. Burnt out without recourse. Fallen in combat.

Obviously the GUI wasn't ready for prime time on national TV. The girl who was supposed to be in charge of CG graphics during the match bailed out in terror (understandably). So without having any sleep for the past 72 hours, I had to manage it during the match, the old way (without any automation whatsoever) which was quite tricky as Antero missed a critical feature, which is the timer (it could only display the system time). I had to write a quick perl script to manage the timer during the match… by resetting the clock at the referee's whistle and switching the display from a static 00:00 page to the rolling counter. Would have been actually easier to use a good old Chyron Max! but hey, that's how it went.

After that, we had to find someone else to develop the remote control program.

To be followed…

<comment><USER63> This is a fantastic story, I remember world cup '98 very well, did you get a chance to enjoy France's victory at all?

<comment><USER62> Not really. To go on with the story, there were 20 stadiums across the country with the whole setup (remote control PCs, SGI Octanes, etc) so every day we worked hard to either correct bugs or implement required features (because we were so late), then I proceeded to push the new program to the central server.

Usually every day, before the match the guy in charge of the setup at the stadium would call me to ask if there was any changes to the setup, programs, etc. One day in Lyon there was to be the first match that could end with extra time. We implemented the required functionality in the remote program during the previous night, so I was home a few minutes before the match was about to begin when I realized that Carlos (the guy in charge) hadn't called me. Uh oh... So while the broadcast began, I called him "hey Carlos, you didn't call! -Yeah, no worry, I know how tired you are I didn't want to bother you, I can manage it... \- Sure, but did you install the new version, the one that support extended time? -Uh, no... -Well, let's pray..."

You should know that the program was several tens of megabytes large, and had to be downloaded through an ISDN line (64k). The very same line had to stay free during the match to transmit statistics data to the central server, so it was absolutely necessary to download the installer at least a full hour before the match...

What do you think happened? I watched the match in anguish. Half-time, still 0-0. 89 minutes, 0-0. God hates us... End of regular time, 0-0, extra time!

Carlos calls back: "\- what should I do now? \- just reset the counter and run the extra time like a regular new match."

The problem was that the program didn't manage the final score display, with the scores for the two half-times and two extra time periods...

So when the match ended, I told Carlos: "-OK, so you have to run the final score card manually. As the program can't send the correct data to fill in the screen, you must enter the data manually"

Fortunately I had sweated long nights on these screens, I knew precisely how many fields there were ... So I said: "-go to the Octane keyboard. On the keypad, press 1001 + enter. Press Tab 4 times. Press 0. Press tab twice. Press 0. Press Tab 3 times. Press 2. Press Tab twice. Press 1. Press ctrl + "+" on the Keypad. Press Ctrl+Enter. Say "On air!" to the director through the intercom".

I was watching my TV, like 1 billion other people. The score card appeared, with the correct values. I can't overstate how relieved I was.

So during the finals, for the very first time, the program was complete. Nothing was missing. Everything went on perfectly smoothly. So sitting next to the Octanes, I spent the whole match playing Unreal on a spare PC. I didn't watch a single second of the finals. I was done with football once and for all. I've never watch a single match ever since.

</comment></comment></comment><comment><USER64> Maybe I've been around too long, but this doesn't seem too bad. The guy made a lot of mistakes (flat fee, not talking to users, etc.) and then suffered the consequences.

As far as the shifting requirements, management nitpicking, "dirty" imported data, and all of that, it sounds like any software project at any medium or large company. It describes a lot of projects at smaller companies, too, for that matter.

</comment><comment><USER65> Reminds me of this: [https://projectfailures.wordpress.com/2008/06/24/project- fro...](https://projectfailures.wordpress.com/2008/06/24/project-from-hell/)

Discussed here before: [https://news.ycombinator.com/item?id=16793884](https://news.ycombinator.com/item?id=16793884)

</comment><comment><USER66> I am embedded within an operational unit. It's a mixed blessing of sorts, but one of the advantages is that, knowing the operational side very well, something like the referral fee issue in the article would occurr to me pretty quickly. Although the downside of a semi decentralized arrangement like this is communication silos that must be actively dealt with to prevent fragmentation of purpose.

</comment><comment><USER67> I decided to quit my job and travel around Asia about a decade ago and didn't want to teach English, so I worked as a remote software contractor instead.

The project was for a person that was running a side-company while working remotely as a project manager for a Fortune 100 company.

The development team was from various parts of India, Guatemala, and Mexico..and I was the only one that spoke English fluently (The project manager was from Guatemala and translated/managed all of the developers from Mexico..they didn't speak any English). I only got the position because I cut my rate in half, as my cost of living was really low, and I didn't want to have a project that was too involved/stressful.

Some highlights:

The company owner was obsessed with the Scrum process, most likely because this is what she used at her main job. We would have 3 hours of meetings almost every day. On Fridays, we would have an extra hour, where we would be required to tell fun stories or jokes. Since we were all from different countries and most didn't even speak the same language, it was just...awkward.

The interview process was a written English test (I had to take one too). The manager wasn't really that experienced and would give repo access immediately. The result was chaos. We spent an entire month with a broken repo because someone was overwriting the master branch constantly. I was paid during this time to just keep checking in changes from one particular bug fix, which would be working when I was finished with everyone's changes..and then the next day, the master branch would be overwritten.

Turnover was high. I was there for a year and we went through over 50 developers. We had one guy come into our meeting room after he was hired, keep interrupting us, and then eventually told us to fuck off. I think at some point, I stayed because I wanted to see where the train wreck was going and simply because I didn't feel like finding another gig.

The project was a way that made filling out government forms easier. There would be major form updates on a monthly basis, which required lots of code changes. The code base was originally written by an outsourced team, which was horrible. There were two, parallel versions running: original and beta. We were all working on the beta, which was supposed to be the better version. However, it was a complete mess of spaghetti code. I found code, written by the current project manager, where he had copy and pasted examples he found online and left the comments in explaining that it was sample code.

When I received a new task to be completed, it was very difficult to figure out what needed to be done. Bug fixes weren't difficult, but most feature requests were tickets from months or even years earlier and some of the requests weren't even relevant anymore. I would call the boss, to try to get clarity, and she had this way of using corporate speak and talking in circles to say lots of words, but never get the information that I need. When I did, it was almost always wrong. I resorted to just completing the task wrong (which I hated) and I would have to correct it later.

We all had to have our web cams on during the meetings. When the boss shared her screen. Every...single...time, she would forget that she had personal emails on the screen..and we would get to see her sexually suggestive emails she was sending to her boyfriend that day. One time I saw everyone's pay in a spreadsheet. Most developers on the team were getting paid $5/hour or less.

We were expected to work all major US holidays and weekends (I just flat out refused to work on any weekend). We had a meeting on July 4th (since I was in Asia, I this didn't really matter to me) one time and she was in the midst of the aftermath of a party. She looked sloppy and there were people passed out on the floor in the background.

When I finally flew back home to the US, she called me up one day and told me that I was too expensive and she could hire 3 people from India for what it costs to pay me.

</comment><comment><USER19> Another takeaway: Prudential skimmed fake 'referral fees' for a task that a web page could do just as well. Of course this was in the old protectionist days when the web was new. But what dense, backward thinking went into this final decision to keep the old phone/paper system in the end.

</comment><comment><USER68> Everybody who believes that private sector is by definition more efficient than public should take note.

</comment><comment><USER69> They could have a contact form with something along the lines of "click here to allow this agent to contact you", and then the affiliates pay for the leads.

They were so worried about getting paid, they forgot to invent a new way to get paid.

</comment><comment><USER70> Working from non-existent or hare-brained specs is the classic front runner of project shittiness metrics. The article exposes an entertaining and realistic example.

</comment><comment><USER71> I love reading stories like these. They are insightful and funny.

</comment><comment><USER72> Discussed at the time: [https://news.ycombinator.com/item?id=6442359](https://news.ycombinator.com/item?id=6442359)

</comment><comment><USER73> This reads like Conway's Law, a novel. I love it.

</comment><comment><USER74> A relative works in a senior role at a medium sized hedge fund.

To a close approximation, software contributes zero to their bottom line.

</comment><comment><USER17> this is just a typical application development story

</comment><comment><USER75> Just a niggle, but "senior web engineer"... in 1995? Uh... given that Netscape Navigator was released in DECEMBER of 1994, can any of us say we were a "senior" "web" ANYTHING in 1995? Sure, I was doing Usenet back on Unix machines in the late 80's in college, but that's not "web." And in 1994, I was dual booting Win3.11 (with Trumpet winsock) and Slackware, and getting on the internet with a free SLIP account, but I was doing Usenet, IRC, FTP, and gopher. Come on, now. "Pretty new" doesn't even begin to describe it. Am I missing something here?

<comment><USER76> Yeah, you're missing that I was there really early.

I was in charge of all the programming and system administration for pathfinder.com, at that time the central web site for all of Time Warner, starting from spring of 1995. At the time I was hired I already had a lot more experience than a lot of other people around; I had been a sysadmin and webmaster for the computer science department of a large university.

I think "pretty new" is accurate. Both the CGI standard and NCSA Mosaic had been released in 1993, with the <FORM> element specified in 1994. I had been running a popular personal web site with innovative features since very early 1994.

</comment></comment><comment><USER77> > Prudential Real Estate is a franchise operation. Prudential does not > actually broker any real estate. Instead, a local franchisee pays a fee for > the use of the name and logo and other services.

I don't know how contracting works, but for corporate project the rule is always "follow the money", then once you figure out the money flow, only then do the requirement gathering bit.

Doing it in the opposite order will result in tears.

</comment><comment><USER78> That is pretty awful. But you know, and maybe this is just me being too fatalistic due to what's happening in my own professional life, but most of what we do is a waste of time in some sense.

Just shovel the shit, get paid, and move on. They want it changed? great, change it. They want a form that does something stupid, fine, build it. A lot of people spend their work day ringing up plastic crap that people don't really need, or whatever. Which is to say it could be worse.

Of course part of what sucks about this story is that the author failed to do the "get paid" part correctly, but I'm speaking generally. I just can't get too attached to outcomes anymore.

<comment><USER79> This is depressing to read. It's still a very in demand line of work. If your day to day is that miserable try finding some place that isn't.

<comment><USER63> I can empathise with his point of view. Sometimes all going the extra mile gets you is a shrug of the shoulders or worse, grief.

If tv didn't exist, people would be more impressed with you for inventing blurry 405 line b&w than high definition colour.

The latter makes it look easy, the former gives them some inkling that it's a technically difficult task.

They'll be more grateful and impressed as a consequence.

</comment></comment></comment><comment><USER80> That was quite the shaggy dog project. I'd say more, but it would spoil the disappointing ending.

</comment><comment><USER81> Wow I searched the comments for agile and no hits. This is basic modern project management - get something in front of the customers and users as soon as possible and iterate from there.

<comment><USER82> Yes, because that never goes wrong...

Maybe it's because I'm a hardware guy, but if anything, agile has normalized a lot of bad practices (like developing before getting any specs) that can only be made to work in software development.

<comment><USER81> Perhaps you're right, but the article here says that is why spec's often dont work. The guy followed the specs, but the it wasn't what the customers wanted.</comment></comment></comment>


Is the 'Pour-Over' Over? Baristas Say Coffee Machines Have Their Perks - <AUTHOR> https://www.wsj.com/articles/is-the-pour-over-over-baristas-say-coffee-machines-have-their-perks-1514913201 <comment><USER2> I use a Melitta pour-over to make my coffee at my desk in the morning. The main advantage over a machine--other than being a cheap piece of plastic--is that most consumer-grade coffee machines just don't get the water hot enough. But I really don't understand why someone would do this in a cafe.</comment>


The Case for Reparations - <AUTHOR> https://www.nytimes.com/2019/03/07/opinion/case-for-reparations.html <comment><USER2> Meh. The idea of original or inherited sin is obnoxious, and ultimately unethical. We're here now, it is what it is--it's day one. How shall we proceed?

I'm white. I grew up in a very liberal household and have never had an unkind word for a black person. Nonetheless, I've been spit on by blacks several times, and otherwise suffered a considerable amount of personal abuse from them, some of which might have gotten me killed had I been less lucky. My response, ultimately, was to simply let it pass. And I think this is really the only answer to this--just let it go.

</comment><comment><USER3> Should we do something for the Union solders what sacrificed there lives to free the slaves as well?

It is sad/horrible but is too late to reimburse those who were enslaved. What isn't too late is to change our society to a more equitable one.</comment>


Roundup found in breakfast cereals - <AUTHOR> https://www.ewg.org/release/roundup-breakfast-part-2-new-tests-weed-killer-found-all-kids-cereals-sampled <comment><USER2> Seems like these results should be taken with a grain of salt, since this group appears to have an agenda they're pushing:

[https://en.wikipedia.org/wiki/Environmental_Working_Group](https://en.wikipedia.org/wiki/Environmental_Working_Group):

> The Environmental Working Group (EWG) is an American activist group that > specializes in research and advocacy in the areas of agricultural subsidies, > public lands, and corporate accountability....

> The accuracy of some claims made by EWG have been criticized by the > scientific community and others.</comment>


Qliiq.com – Save your Bookmarks in Qliiq.com - Seek for Feedback - <AUTHOR> With Qliiq, you can start to organize and connect your favourite bookmarks all in one place. Create your own library and connect with friends. We are web-application in beta version since December 2019. Please Log-in and create your account. Then, leave us feedback in the About us Page. <comment><USER2> Good. I've been thinking about something like this.

You should make the title Qliiq-ible(clickable).

<comment><AUTHOR> Thanks for the feedback. Nice Qliiq-ible! You get the idea.

We are in beta version now and you can already use it and send us feedback. Try to use it, save your bookmarks there and let us know any feedback. We are already the only bookmarks with notes, tags and social aspect at the same time. Let me know if you need more information too. Cheers</comment></comment>


Parent Who Criticized His Son's Math Program Is Sued by Curriculum Company - <AUTHOR> https://www.edweek.org/ew/articles/2019/09/10/parent-who-criticized-his-sons-math-program.html <comment><USER2> I feel like if MVP worked, they could defend themselves using data that shows clear improvements from the students in that curriculum instead of suing a critic.

I joined the facebook group when I first heard about this case. Some parents have complained that at some wake county middle and high schools, the math teachers are not even instructing the kids under penalty of punishment from the WCPSS administration. The kids at those schools are being placed into groups and left to figure out the math entirely on their own. At other schools in the district using the same curriculum teachers are still using traditional direct instruction, but they are all being grouped together when measuring the results county wide.

</comment><comment><USER3> The actual lawsuit is available here:

[https://www.documentcloud.org/documents/6224989-MVP- complain...](https://www.documentcloud.org/documents/6224989-MVP-complaint- against-Blain-Dillard.html)

</comment><comment><USER4> The article seems really thin on the actual details of the suit. He's being sued for defamation not critiques. It's entirely possible that he has been making outrageous defamatory attacks during his campaign.

Parents should be allowed to be involved in their children's curriculum, but that doesn't mean they get a card Blanche to assault any companies involved that they don't like.</comment>


Cops Now Pinning Mugshots to Pinterest - <AUTHOR> http://betabeat.com/2013/01/cops-now-pinning-mugshots-to-pinterest-because-the-police-can-be-aspirational-too/ <comment><USER2> my local cops use facebook. they put up photos that place armed violent offenders alongside people who are late paying fines, and others who have not been convicted of anything. Sometimes there is an 'innocent until guilty' disclaimer, or a note saying they may be witnesses, not criminals.

someone usually replies with recent CCTV images of them beating an innocent man to death ...

Yes I think it's mostly PR. Our local force have certainly been capitalising on the mordich media's reality show</comment>


Ask HN: Does Google maps command traffic? - <AUTHOR> We were in the same car having two phones on Google maps. We had the same destination plugged in.<p>At an intersection the two phones disagreed about the turn. I am wondering whether Google sends people to different routes in order to split the traffic and get everyone home faster. <comment><USER2> My experience is that the quality of directions varies by distance.

Suppose route H -> D passes through intermediate point I.

In cases where most of my concern is directions for the are local to my destination, I've looked at directions H -> D and known they were crap due to knowing that the portion from I -> D contained poor segments. In those cases, getting directions from I -> D tends to improve the poor segments.

As distance increases the search space for routes increases and I strongly suspect Google limits the amount of computation it runs before returning a result. In other words, the route Google provides for A -> B is not idempotent...as is shown by your anecdote.</comment>


Ask HN: The Internet or Cars – Which could you live without? - <AUTHOR>

For me... it's easy... the Internet!<p>Honestly, as I'm in my 40's and remember what things were like before the Internet, I don't think I'd miss the Internet that much.. probably be better for my health as well. <comment><USER2> I live in san fracisco. i walk everywhere. miles a day. It's great for my health, I like the internet but i don't have a cell phone.

I only miss the car when i want to go out of town. i've never missed my cell phone, one time ever..ever... ever.

</comment><comment><USER3> Cars, no contest. I live in an area with excellent mass transit, and between that, walking, and the occasional plane ticket to somewhere that has airport shuttles, Uber, or taxis, I rarely _need_ a car. It's a convenience, not a requirement.

</comment><comment><USER4> Internet. Not just as someone who lives in an area with crappy mass transit, but also as someone who loves to drive around and see new areas a lot. As much as I enjoy the internet, some time away from it would be restful. I'd get by without it.

</comment><comment><USER5> Am I the only one who has to live without, or am I choosing for everyone? Do you mean there's either an internet or automobiles, but the existence of one means the other never did?

</comment><comment><USER6> Internet. I'm trying to get into racing and I have more of a love for things with engines than the internet.

</comment><comment><USER7> Cars, no contest, and I live in an area with TERRIBLE mass transit. I live in a small town just west of San Jose in Silicon Valley, and in order for me to get anywhere, I have to take a 45 min bus ride (the only bus that comes into my town) into Sunnyvale and from there either catch another bus or Caltrain to get where I'm going, and often times another bus, Muni or Bart after that.

That said, I would still rather keep my Internet than gain a car. Safeway delivers for free, I no longer need to go to Blockbuster to get new content, and I can (if I chose) work completely remotely.

Living without a car sucks, especially in California where public transportation is a joke, but its still better than living without internet.

</comment><comment><USER8> Both.</comment>


Show HN: Web Design Stash – designer news & top resources in one place - <AUTHOR> http://www.webdesignstash.com/ <comment><USER2> That is one odd content layout. First there's a search, then a bunch of categories. THEN there's a list of seemingly submitted items. THEN you have the actual purpose of the site being spelled out. Lastly, you have "featured" stuff. Since they are in a grid and at the bottom, they are the opposite of featured.

Then you have a random "submit" button out of nowhere.

So I pressed one of the articles that I wanted to read, and what did I get. This[0]. I mean, what even is this? "Preview or download" ad medium.com. What preview? What download? Why am I not in the article already?

On the side I get some BS "13 Mind-blowing Statistics on UX is another useful resource that we selected for inclusion on Web Design Stash.". If a "for fun" list of 13 random statistics (90% more likely to climb everest than click banner) is something you "choose" to include in your so called resources, then I can only assume that your whole site it just scraping a bunch of websites and mass include stuff for your website.

[http://i.imgur.com/WGeK4Is.png](http://i.imgur.com/WGeK4Is.png)

The tags are also not relevant.

I expected a properly sorted and curated website for actual resources, pretty much the opposite.

<comment><AUTHOR> Some of your feedback is interesting opinion, but the tags are relevant to the UX article. No, it's not "scraping a bunch of websites and mass include stuff". Anyway, thanks for the feedback.

</comment></comment><comment><USER3> The sub headers are too light to be readable before you hover, but my brain is still trying to read them. Also, it be nice if you could click anywhere on the squares, not just the text. Anyways, great content, I found a few things I had never heard of!

</comment><comment><AUTHOR> I'm very interested in knowing if you think the collections or newest resources (votes list) should be shown first on the home page.</comment>


Open source mobile analytics project using node.js and mongodb - <AUTHOR> http://count.ly/dashboard

<comment><USER2> This is very nice. You're definitely beating people on UI, but not yet on functionality. Google's path feature, linking advertising campaigns, etc. Though I'm sure these are things you're thinking about.

Please keep me up to date. Add my email dave AT quipvideo DOT com to a mailing list if you have one.

Also, how are you liking Node + Mongo for this project? We use it at Quip.

<comment><AUTHOR> Thanks a lot. Yeah there are tons of new features on the way and I'm sure Countly will catch up with all the features of Google Analytics in a very short time. So far mongo and node has been great to work with and in terms of performance its quite impressive what you can achieve even with a small VPS.</comment></comment>


Computer viruses: a cautionary tale - <AUTHOR> http://www.camsoftpartners.co.uk/bugs.htm

<comment><USER2> I have a question... "Both sites maintain a database of known spammers and offer spam blocking services."

Every time the spam topic arises, I always ask. Why hasn't those guys been shut down from the internet already?</comment>


FCC: Myths vs. Facts on Net Neutrality [pdf] - <AUTHOR> http://transition.fcc.gov/Daily_Releases/Daily_Business/2017/db1128/DOC-347961A1.pdf <comment><USER2> MYTH: FCC Chaiman Ajit Pai has the best interests of the American consumer in mind.

FACT: FCC Chaiman Ajit Pai's plan to gut net-neutrailty will give ISPs more power over consumers in that they will be able to exercise their monopoly to control what we watch.</comment>


Ants show that emergency exits can work better when they're obstructed - <AUTHOR> http://nautil.us/issue/23/dominoes/want-to-get-out-alive-follow-the-ants-rp

<comment><USER2> I read the book about The Station nightclub fire tragedy on a long flight once. The book had a different reason for people clustering at a single exit. It wasn't because of a heard instinct, it's because in an emergency people rush to the exit they used to enter. In the short amount of time to think in an emergency, people do not look around for new emergency exits. They go with what they know: the way they came in. (I believe this is why flight attendants remind people that their closest emergency exit may be behind them.)

In a situation of panic this leads to overuse of the main entrance and underuse of emergency exits. In extreme cases, it can lead to a crush and cause the exit to become inoperable.

Another issue is that emergency exit signs are posted high, but those are the first areas to be obscured by smoke in a fire.

My take away was to keep egress in the back of your mind when in crowds and to make a mental note of emergency exits.

<comment><USER3> Planning for unreliable alternative exits is one of the fun aspects of cave diving: you virtually always exit by returning to your point of entrance, even if you're nominally closer to another means of egress, since you can't be certain that the "closer" exit is genuinely navigable. When diving a traverse between two points, the best practice involves diving as far as you can one way, leaving a marker, then entering from the other side on a second dive, evaluating your gas situation at the marker, and then either continuing or aborting the dive.

<comment><USER4> "Fun"

</comment></comment></comment><comment><USER5> Anybody able to find a video of the corner exit with no obstruction? Wish they posted that one, being that they claimed it was the fastest.

<comment><USER6> Seems logical that the two walls act to funnel people down towards the opening - a 90 degree-wide corner exit is halfway between a 180 degree-wide funnel towards a single door in a wall, and a 0 degree wide corridor that leads to a door.

Likewise the 'obstructed' wall entrance just turns into two corner exits, side-by side (I imagine the scenario doesn't work out well for the individuals caught directly behind the pillar, as the flow past each side of the pillar into each exit zone prevents them from getting into either flow effectively).

<comment><USER7> I'd bet that if you put another pillar behind each side of the pillar obstructing the exit you'll double the throughput.

<comment><USER8> And thus it follows that, by repeatedly adding pillars and doubling the number and halving the size/spacing at each step, you can approach infinite throughput!

</comment></comment></comment></comment><comment><USER9> no link, but there was a good article or documentary about 10 years ago about how you can improve throughput of an automatic-opening double door by putting a bollard in the centre. iirc it was over 2x increase.

basically people aim for the centre and people cluster up waiting for them to pass instead of having 2 way travel all the time.

</comment><comment><USER10> If you are interested in this stuff you should read [http://www.elsevier.com/books/stochastic-transport-in- comple...](http://www.elsevier.com/books/stochastic-transport-in-complex- systems/schadschneider/978-0-444-52853-7)

Also: [http://www.thp.uni-koeln.de/~as/](http://www.thp.uni-koeln.de/~as/) [https://www.uni-due.de/zlv/portrait/schreckenberg.php](https://www.uni- due.de/zlv/portrait/schreckenberg.php)

</comment><comment><USER11> The TV show Crowd Control did something about this last year ([http://channel.nationalgeographic.com/crowd- control/videos/e...](http://channel.nationalgeographic.com/crowd- control/videos/emergency-exit-experiment/))

</comment><comment><USER12> This is very interesting, but the title is misleading.

The ants don't _do_ anything better than humans.

They just _act_ like panicked humans do, so you can easily test different theories about the best way to build exits.

<comment><USER13> We switched to the subtitle.

<comment><USER14> The new title is still misleading (although I'm not asking you to change it, it's the fault of the article).

They don't do better with it obstructed, they do better when there are channels leading to the exit.

<comment><USER13> I don't doubt you, but in cases like this it is helpful if someone can suggest an accurate, neutral title. We don't have the capacity to absorb all the content well enough to do that in every case.

</comment></comment></comment></comment><comment><USER15> I don't know why they're republishing this. This same source published the same article last year: [http://nautil.us/issue/13/symmetry/want-to-get-out- alive-fol...](http://nautil.us/issue/13/symmetry/want-to-get-out-alive-follow- the-ants)

Interesting, but old news.</comment>


Censorship On Hacker News - <AUTHOR> http://patdryburgh.com/blog/censorship-on-hacker-news/

<comment><USER4> I can see why the original headline might have been edited for clarity - <USER3> mental leap from "porno site" to "productivity porn" is non-obvious.

I'd be more inclined to chalk this one up to miscommunication (the bad headline) and good intentions (editing the headline for clarity) than I would to call this outright censorship.

<comment><AUTHOR> "I can see why the original headline might have been edited for clarity - <USER3> mental leap from "porno site" to "productivity porn" is non-obvious."

Perhaps it is non-obvious reading only the title, but the article did go on to clarify what the intent of using the word "porno" was.

My problem with it is that editing the title in this manner sets up an expectation for the reader, and could cause the reader to become upset when they see the actual title has the word "porno" in it. As I said, the article clarifies what I mean, but if someone is sensitive to the use of that type of language, then they may feel they have been duped into looking at content they wouldn't otherwise look at.

As I pointed out in the article, this doesn't appear to be a common experience. I couldn't find anyone else online complaining about Hacker News censoring their content. But, I wanted to make clear what my intentions were, and hopefully open up a discussion that could possibly lead to needed clarity on exactly what the rules of this community are.

</comment></comment><comment><AUTHOR> I received a great email from Andrew de Andrade, a fellow Hacker News reader, on why what I perceived to be censorship wasn't, and how it was my fault as the link submitter for not providing a title that provided better utility to the community.

Please, accept my sincerest apologies for my misunderstanding and for jumping to conclusions. This article was not written out of malice towards Hacker News, but rather as an attempt to bring clarity to what is and isn't accepted in this community.

I've added a note on the post apologizing for offending anyone, and inviting anyone who would like to discuss the issue further to reach me via email (hello [at] patdryburgh.com)

Thanks for your patience and understanding.

Pat</comment>


Twitter Announces Fire Hose Marketplace: Up to 10k Keyword Filters for 30 Cents - <AUTHOR> http://www.readwriteweb.com/archives/twitter_announces_fire_hose_marketplace_up_to_10k.php

<comment><USER2> 30c... an hour

 _which is ((24x7x52)/12)x$0.3 = $218.40 a month_

<comment><USER3> Or from another perspective...

140 million tweets per day[1] / 24hr / $0.30 = 19.5million tweets per dollar.

_Surely_ there's some valuable data to be gleaned out of 20million tweets?

[1] wildly assuming Techcruch's numbers are connected with reality - [http://techcrunch.com/2011/03/14/new-twitter- stats-140m-twee...](http://techcrunch.com/2011/03/14/new-twitter- stats-140m-tweets-sent-per-day-460k-accounts-created-per-day/)

<comment><USER4> From their pricing calculator it seems the deliver max 2k tweets per hour. Way less than your 6 million, and not that useful.

</comment></comment></comment><comment><USER5> > Thanks to Qwerly integration, when you look at a Twitter @username - > Mediasift sees more than just the Twitter profile. It sees @username who has > bookmarks saved online, plans for public events to attend, photos shared > publicly with friends, check-ins to places around town and much more. Any of > those are like columns in a spreadsheet of Twitter search results. Show me > Tweets with any of the following keywords by people planning to attend event > A and who have been to place B or C. Thanks to Qwerly, Twitter didn't just > get a giant new developer search and filter feature - it got integration > with a whole lot of other social services.

Thank God I use different pseudonyms on the internet. That just gives me the willies.

\---

On another note, this may come at a very opportune time when the Obama 2012 is kicking into gear.

<comment><USER6> I don't see a problem with it. If you use twitter, then you clearly don't have a problem living your life in the open and the price of being an attention whore is getting attention. For those of us who don't need to play-by-play our every thought and action in 140 character narration throughout the day to our imagined hoards of clingy worshippers, it won't be a problem. We likely don't use twitter or we don't use the same identity as we do elsewhere. I mean, as much as I'd kind of like to have all of my identities tied together, I don't like the potential problems that may draw. Therefore, I use a different identity at HN than I do at Slashdot which is different than on LinkedIn which is different than on Amazon, which is different than on Steam, which is different than on XBOX Live.

Yeah, people like you and I are still open to having any anonymity data-mined out of us through aggregate manipulation -- but at least it's a simple layer of abstraction.

In the meantime, if someone can get rich using the wealth of public information that every vapid college girl posting a thousand twitpics a night from her cell at the club puts up online, then more power to them.

</comment></comment><comment><USER7> Despite being "dirt cheap" for potential clients such as companies, I find the 30 cent barrier an awkward price point since people who will want the data are probably willing to pay more, and it definitely bars the casual developer from access to social data.

<comment><USER8> There is always the garden hose sample feed from Twitter. Or, use this service for short test intervals.

Seems like data hackers are being taken care of.

Also, if you want a lot of social media cheaply, check out the sample web app for Google Buzz that runs on AppEngine. I ran it last summer with some of my own filters. I could run it about 5 hours a day before I hit the limit of a free AppEngine account - so it would not cost too much to pay to keep a derivative of this example program running 24x7.

</comment></comment><comment><USER9> This is a vastly better business model than the dickbar.

<comment><USER10> Assuming other companies can figure out how to monetize the tweets and be able to keep paying to access the firehouse.

<comment><USER11> I've always said the ultimate Twitter business model is to know what everyone in the world is thinking right now, and to sell that information.

</comment></comment></comment><comment><USER12> Nice. Another service that I'm not going to give free data to anymore.

<comment><USER13> I sort of have this feeling too, but Twitter _is_ a business after all, and hosting god-knows-how-many twits can't be cheap.

Sure, they could do something else instead of selling data, but until end users pony up for the service, then they're the product.

<comment><USER14> Actually, God has told us how many tweets they're hosting, and tweets are mercifully short. As of about a month ago, Twitter says it gets about 140 million tweets per day. Assuming the maximum length of 140 characters, this means they're storing about 18 GB per day. At Amazon S3 rates (which are considerably higher than what Twitter pays if they have a working brain anywhere in their corporate structure), that means that their storage costs increase by about $1/day. After five years of storage at that rate, their monthly storage costs (again, at S3 rates) would be around $2000. If they're making less than $2000 per month with that wealth of data, nickel-and-diming developers is a drastically misguided underreaction.

I'm not pretending this is all it takes to run Twitter, but I'd be surprised if storing a few TB a year is a major cost center. (Serving up so many concurrent users seems like a much bigger and more expensive problem — that's an average of 1600 tweets per second, to say nothing of readers, and I suspect tweet rates are very lumpy.)

<comment><USER2> this is what is known as a straw-man argument, and it does nothing to move the conversation forward.

Clearly storing just the 'tweet' contents alone would be unhelpful because what about the username or any of the other 40+ metadata point a tweet carries.

What about keeping the mechanisms needed to store, sort, search, send those tweets, etc etc. I could go on.

Also, what were you expecting - Twitter to their business at-cost?

<comment><USER14> As far as I can tell, the comment I was replying to was about Twitter's storage costs. So the fact that my response focused on storage and how much it costs does not make a strawman, which would involve arguing against something other than what I was replying to. Moreover, the point of my comment was that Twitter's storage costs are _not the interesting part of their operation_ , so your question "Were you expecting … Twitter to run their business at-cost?" actually is a straw man. You're just repeating what I said, except with slightly less hard data.

</comment></comment></comment></comment></comment><comment><USER15> Among other factors, this will probably increase the degree to which your twitter username represents your identity online. I'm not saying that's good or bad, just predicting it'll happen.

</comment><comment><USER16> Why don't they just charge businesses to use the site? And charge relative to their number of employees or revenue. All on the honor system - you get a badge by your company name when you pay.

</comment><comment><USER17> Can someone explain why I can't register the 10k most popular keywords and then resell arbitrary subsets of that stream to interested entities at pennies on the... nickel and a quarter?

<comment><USER18> Well, it's all in the TOS. You can't resell, and you can't display the tweets.

</comment></comment><comment><USER19> Surprised it took Twitter this long to monetize this.

</comment><comment><USER20> Twitter Announces Fire Hose Marketplace: Up to 10k Keyword Filters for 1/2 cent per minute</comment>


Search for and Contribute to Open-Source Government Projects - <AUTHOR> http://govcode.org

<comment><USER2> It might be worth mentioning somewhere that the site/project is focused on the United States. Correct me if I'm wrong, but that's the impression I get when I look through the projects and filtering options.

It'd be nice to associate each agency with a country, so that you could filter on a particular country as well.

<comment><USER3> I have been working on adding other countries as well but right now it is US only, the new importer would let us bring code from many more orgs...

</comment></comment><comment><USER4> Near the bottom, project bluebird description: "Guided twitter messages for surveillance prototype.. " Not a reassuring description. Or a very positive one.

The project is, afaik, a flu survey that tweets an encoded set of answers. Like, both piggybacks off twitter's infrastructure and benefits from its wider userbase: [http://www.phiresearchlab.org/?p=2781](http://www.phiresearchlab.org/?p=2781)

Considering what the project actually does, might be a good thing to rewrite the description entirely - if of course, it wasn't retired since it was a prototype and the test ended successfully.

</comment><comment><USER5> Here is a government entity I fully support!

[https://github.com/cfpb](https://github.com/cfpb)

</comment><comment><USER6> Particularly useful site for me... but that is one jacked up mobile site.</comment>


Recursive Type Signatures in Scala - <AUTHOR> http://blog.originate.com/blog/2014/02/27/types-inside-types-in-scala/

<comment><USER2> This is known as F-bounded polymorphism if you are interested in knowing more about this approach.

</comment><comment><USER3> Awesome article, simple and to the point!</comment>


Opinion: Getting rich has more to do with luck than talent - <AUTHOR> http://www.bbc.com/capital/story/20170525-opinion-getting-rich-has-more-to-do-with-luck-than-talent <comment><USER3> Hoo boy the tone in this article is toxic.

The truth is that, yes, there is luck involved but a lot of hard work, too. It is a rich spectrum.

Lucky is buying a winning lotto ticket. Luckier still is finding one on the street.

The same is true for life.

Someone works hard in school, gets very good at mathematics and finance, and takes the money from his father's successful life in the restaurant business to start a HFT firm just at the right time to take advantage of inefficiencies in the market while also having access to previously unavailable technological tools to put his ideas to work. He makes a billion dollars.

There's a lot of luck there. But there's also a lot of hard work. Someone sitting at home watching Simpsons reruns is not going to get there.

We shouldn't be so hostile to success but we should also recognize the places where luck can be mitigated and work to do that.

In the example above, the person had a fairly wealthy father. That helped but how can we make that less necessary for the next talented hard working individual that comes along? He got a good education partly because his family could pay for it. Again, how can we make the luck of birth less of a factor? He was able to develop his talent because his mother was able to keep an eye on him and make sure he had a stimulating childhood. Same issue.

We can work to level the playing field and make the rules fairer without tearing down the champions operating under legacy rules. We can acknowledge luck and work to put people in a position to be more lucky. People can learn from the success of others without waving away their hard earned accomplishments by chalking it up to mere luck. Luck is a description of circumstance not the circumstance itself.

<comment><USER4> The article's tone and the general attitude in society echoed seems partially caused by a false dichotomy:

Lots of people think first that success and wealth are zero sum games. And second, that we have to decide between extensive social safety nets and a dynamic economy.

Just the opposite, of course.

About the second: the trick to get over the apparent trade-off between welfare state and dynamic economy is to realize that we don't have to tax labour nor capital to provide. Taxing land values works just fine.

More discussion on Georgism here: [https://news.ycombinator.com/item?id=10442929](https://news.ycombinator.com/item?id=10442929)

Also [https://news.ycombinator.com/item?id=11158398](https://news.ycombinator.com/item?id=11158398)

<comment><USER5> About the second: You don't need to think about taxing land, or anything like it. You just have to realize that, if we don't have a dynamic economy, we can't afford the safety nets (at least not for long).

<comment><USER4> Yes, that point is true as well.

In general, most real world policies are far from the kind of efficiency frontier where trade-offs between growth and safety net have to be made. Eg the US could replace the subsidies for employer provided health care (with eg a budget neutral tax credit for everyone or more radically a budget neutral slim UB) and improve both efficiency and safety nets.

But, of course, policy is seldom driven by efficiency frontiers amongst its ostensible goals. Politics is harder and more complicated.

Though know I do wonder if a general programme to simplify taxes and close loopholes would have a better chance of passing step-by-step if every such bill came with a budget neutral per-head tax credit. The sound-bite "Senator Smith proposes to give YOU an extra $100 a year by closing obscure loophole exploited by a few corporations!" would certainly do well with the general public?

</comment></comment></comment></comment><comment><USER6> I'm much less concerned about the role of luck in getting rich (since luck, after all, is presumably uniformly distributed), than I am about the pervasive economic feedback loops that reward the rich simply for being rich. It seems to me that the latter are a much better justification for more sharply progressive taxation than the luck argument.

<comment><USER7> I'm more concerned about the feedback loops that punish poor people for being poor. Stuff like monthly fees on bank accounts, being thrown in jail for failure to pay child support (in the process losing one's job), etc. It's expensive to be poor in the United States.

<comment><USER4> British retail banking is by all accounts even worse. (It's certainly bad enough in an absolute sense. But I never had first-hand experience with the US system to compare.)

</comment></comment></comment><comment><USER8> While I agree with the general sentiment of this article, there is something that articles with this theme always seem to miss. For those not born into wealth, hard work and applied intelligence in an endeavor where it is at least _possible_ to make tremendous amounts of money is generally a _prerequisite_ for financial success. Luck may well be the deciding factor in the the level of success that those _in this group_ attain, but the vast majority of the population never does what is necessary to be a part of this group. Accordingly, most people have almost no chance of obtaining vast wealth, regardless of whatever "luck" they may or may not have, while the small percentage that are in this group have a very high (relative to the general population) chance.

<comment><USER6> _For those not born into wealth, hard work and applied intelligence in an endeavor where it is at least possible to make tremendous amounts of money is generally a prerequisite for financial success._

Let me play devil's advocate for a moment. Why do we believe this? I suggest that it's because we see examples of people who worked hard and got rich. But what if the truth is that most people work relatively hard---ie, keep busy at something. What if the pool of truly lazy people is actually rather small, which would make the pool of lazy, lucky people even smaller? Then it's entirely possible that success is all down to luck, but it would seem to us that most of the lucky people were also hard-working. But we would be wrong in associating the hard work with the success.

I firmly believe that hard work and applied intelligence is a key factor in making a decent living---doing a bit better than your neighbor, say. But getting rich, to the point where you're making 10x, 100x, 1000x the average, requires some extraordinary intervention somewhere along the line. I'm not so sure there is a strong correlation between the two phenomena.

<comment><USER8> What I meant was that, for example, your pool cleaner probably works hard, but is not engaged in an endeavor where it is possible for him to make millions of dollars. His earnings are directly proportional to the number of hours that he works. Therefore he has essentially taken himself out of the competition to be highly successful. Looked at in this light, only a small percentage of the population has any chance at tremendous success.

<comment><USER6> _[Y]our pool cleaner probably works hard, but is not engaged in an endeavor where it is possible for him to make millions of dollars_

My pool cleaner, like most of us, has a finite lifespan. While it's certainly theoretically possible that he _could_ have been a hedge fund trader for Goldman-Sachs, I think we can both agree that to accomplish that would've required a fair amount of luck somewhere along the way. He might, by grit and discipline and sacrifice, have managed to become a lawyer, and make more than he is making as a pool cleaner. But to become independently wealthy by age 30, he also would've had to be very, very lucky.

 _Looked at in this light, only a small percentage of the population has any chance at tremendous success._

Not only that, but only a small percentage of the population has ready _access_ to the positions that offer the financial leverage necessary for that kind of success.

</comment></comment></comment></comment><comment><USER9> Luck and hard work / talent are both required. Which is more important is a silly question -- they're both required, they're both essential. If you have one, then the other is more important. If you have neither, acquiring just one won't help.

Luck is what happens when preparation meets opportunity - <USER2>

</comment><comment><USER10> Key word: opinion.

Getting rich most likely has to do more with grit than with either luck or talent.

<comment><USER6> Why do you believe it is "most likely"? And how do you define grit in this context?

</comment></comment><comment><USER11> Being Jewish helps.

<comment><USER12> We ban accounts that post this way to HN and have banned this one.

<comment><USER4> Jerkass might have a point though. Compare the ever enlightening Slate Star Codex ([https://slatestarcodex.com/2017/05/26/the-atomic-bomb- consid...](https://slatestarcodex.com/2017/05/26/the-atomic-bomb-considered- as-hungarian-high-school-science-fair-project/)).

It's hard for us goyim to keep up with those genetic advantages in brain power. (Though sega01 was probably suggesting something like a secret cabal keeping down everyone else..)</comment></comment></comment>


Scaling Synchronization in Multicore Programs - <AUTHOR> http://queue.acm.org/detail.cfm?id=2991130 <comment><USER2> Right off the bat it presents three possible queue arrangements none of which is the one I prefer.

A single MPSC queue per core works very well and this is what VoltDB does. For most sane task sizes this works fine and it scales because you can only process so many tasks at a single core so queue contention isn't an issue.

If you want to crank things to 11 you can do an SPSC queue between every core, and this is what ScyllaDB does and is suggested in the article. This does relieve CAS failure, but outside of microbenchmarks I am not sure it is a real world issue.

I had never considered the advantages of atomic instructions that never fail. In the context of things like Disrupor or Agrona those choices start to make more sense.

<comment><USER3> Have you benchmarked MPSC-per-core and SPSC-per-pair-of-cores against any of the three concurrent queues in the article?

<comment><USER2> I haven't benchmarked them comparatively, but when I benchmarked and profiled VoltDB there was very little time spent on CAS failures according to flight recorder. It could be the profiler is a wrong so maybe instrumenting makes more sense just to fully prove the point.

A single queue is only being asked to handle low hundreds of thousands of events per second which isn't that much. This occurs because there is substantial (for some definition of substantial) work associated with each task. If you are looking up a value in a hash map then yes it matters, but if you are doing something more substantial it doesn't matter. The research is definitely interesting and it solves real world problems, but I don't think you necessarily need to move to groups of input task queues directly.

If you look at the graph it's at five million operations/second and 1 CAS with two threads. If you aren't going to push more than 1 million ops through a single queue/core I think you will be fine.

</comment></comment></comment><comment><USER4> It's very interesting to see a lot of the traditional assumptions behind system design reserach change with hardware. In-memory databases, choosing O(n) cache-sensitive algorithms over O(log n) random access ones, and single nodes that are starting to look more like shared-nothing distributed systems were all major themes at VLDB this year.</comment>