Software Folklore – A collection of weird bug stories - <AUTHOR> http://beza1e1.tuxen.de/lore/index.html <comment><USER2> I've posted this story before, but it fits here rather nicely.

I had a function that looked like this:

      void f() {
        bool flag = true;
        while (flag) {
          g();
        }
      }

This function would sometimes exit. But that's really all there was to the function. Somehow flag was becoming false, even though nothing ever wrote to it.

So you might think about g() smashing the stack, when a variable is mysteriously changing, but you'd expect the return address to also get written, and it wasn't - the function returned from g() to f(), found flag to be false, exited the loop, and returned from f().

Eventually I got desperate enough to look at the assembly code produced by the compiler, and I became enlightened. (This was g++ on an ARM, by the way.) flag was being stored in R11, not in memory. (Might have been R12 - it's been a while.) When g() was called, f() just pushed the return address. Then g() pushed R11, because it was going to have its own variable to stash there, and then created space for its stack variables. And one of those variables was smashing the stack by 4 bytes, over-writing the saved flag value from f().

Worse, the way the stack was getting smashed was on a call to mesgrecv(). This takes a pointer to a structure and a size, but the relationship between the two isn't what you'd expect. The size isn't the size of the structure, but rather the size of a substructure within that structure. A contractor had gotten that detail wrong when they used that mechanism for IPC between two chips. (They'd gotten it wrong on the sending side, too, so the data stayed in sync.)

The net result was that the flag got cleared when four next-door-but-unrelated bytes _on another CPU_ were all zero. It took me a month, off and on, to figure that out.

<comment><USER3> Crazy thing to go with that... if your compiled with different (more aggressive) optimization flags, it might have gone away!

<comment><USER2> It already went away when I tried to print out the address of the variable, so that I could watch it in the debugger (because, in order to take the address of it, it had to become a stack variable).

<comment><USER4> In the end, do you remember what tools you used to confirmed that R11 was overwritten? The tools and the path to the root cause are also quite interesting.

<comment><USER2> I first looked at R11 because of the assembly output. There are flags that you can give g++ to produce the assembly output when it compiles. That showed me that the variable was in R11, and where it wound up on the stack in the g() function body.

From there, it was a question of how g() was smashing the stack. (I hadn't looked at that before, because I assumed that it had to be _f()_ smashing the stack in order to change the variable.) Well, the next thing on the stack was the structure for mesgrecv. If too much got read into it, it would overwrite the stored copy of R11. That led me to look _very_ carefully at the mesgrecv call. Checking the parameters against the man page showed up the unexpected (at least to me) requirements for the size parameter.

I never "verified" that the stored copy of R11 was being overwritten, except by changing the size parameter and noting that the loop in f() never terminated any longer.

</comment></comment></comment></comment></comment><comment><USER5> I love these toughies, especially the full ssh one! A true debugging wizard.

"Dumb" problems can happen to anyone too. I once walked by the desk of$well_known_open_source_developer who was struggling with a mysterious bug. He'd narrowed it down to the specific function and was groveling in the function setup code (what the compiler generates before your code is called) He asked me to take a look and within seconds I saw an uninitialized variable being read.

This is not because he was a bozo! He had decades of experience. It's simply that sometimes we get slightly wedged and can't see the thing that is "staring us in the face". He was embarrassed (so not mentioning his name) but he should not have been. If anything it simply proves that it can happen to anyone.

Related to this: at one organization my debugging skills were (spoiler: undeservedly) legendary...literally word got around until some new hire asked me about it months later.

Why? I came in one morning to find some folks trying to get some new model of terminals to work with the mainframe. Back then you needed the right combo of byte length, stop bits etc. they asked me if I could fix it and I said sure. As one does I poked at the setting switches and walked off to get my coffee So I could come back and think clearly. By the time I came back all the terminals were in use so I just went on with my day.

Apparently I had randomly toggled the necessary bit. But the way the story was told: I had walked in, agreed to help, rubbed my chin then simply pushed the right button and walked off without another word. Which in some sense is true, But gave me completely undeserved credit.

<comment><USER6> When I was a kid (teenager) I worked at an indoords shooting range, mind you, not real guns but just BB guns. I was supervising a bunch of school kids do some practice shooting at biathlon targets (just 10m however) and one of them had an issue with the gun with the pellet getting stuck somehow. I had a look at the gun, sorted the pellet out and fired the gun off the hip and hit a bullseye without aiming. Pure luck of course but the kids were like "woaaah" and of course I never told them that it was just luck and not my mad leet shooting skills XD

</comment></comment><comment><USER7> Here's the craziest one that actually happened to me.

The company I worked for had installed what's best described as a mini- supercomputer (though we avoided the term) at a site in Boulder. We started getting reports of failures on the internal communication links between the compute nodes ... only at high load, late in the day. Since I was responsible for the software that managed those links, I got sent out. Two days in a row, after trying everything we could to reproduce or debug the problem, I got paged minutes after I'd left (and couldn't get back in) to tell me that it had failed again.

Our original theory was that it had to do with cosmic rays causing bit-flips. This was a well known problem with installations in that area, having caused multi-month delays for some of the larger supercomputer installations in the area. But we'd already corrected for that. It wasn't the problem.

What it ultimately turned out to be was airflow and cooling. The air's thinner up there, so it carries less heat. But it wasn't the processors or links that were getting too hot. It was the power supply. When a power supply gets warmer it gets less efficient. Earlier in the day or with shorter runs as we tried different things this wasn't enough to cause a problem. With it being warmer later in the day, continuous load for longer periods was enough to cause slight brown-outs, and _those_ were making our links flaky. And of course it would always restart just fine because it had cooled down a bit.

The fix ended up being one line in a fan-controller config.

<comment><USER8> > Our original theory was that it had to do with cosmic rays causing bit- > flips. This was a well known problem with installations in that area, having > caused multi-month delays for some of the larger supercomputer installations > in the area. But we'd already corrected for that.

Wow, I sense a more interesting story in here. Care to reveal how it was first found out and how common it actually is?

<comment><USER7> In a nutshell, cosmic rays causing bit-flips really is a thing, and it's more of a thing at higher altitude because of less atmosphere. It's rarely a problem at sea level. At higher altitude you really need to use ECC memory, and do some sort of scrubbing (in Linux it's called Error Detection And Correction or EDAC) to correct single-bit errors before they accumulate and some word somewhere becomes uncorrectable.

The incident that brought this home to a lot of people was at either NCAR or UCAR, both near Boulder. Whichever it was, they were installing a new system - tens of thousands of nodes - and had _not_ been careful about the EDAC settings. Therefore, EDAC wasn't running often enough, and wasn't catching those single-bit errors. Therefore^2, uncorrectable errors were bringing down nodes constantly. According to rumor, this caused a huge delay and almost torched the entire project. It's easy to say _in retrospect_ that they should have checked the EDAC settings first, but as it happened they probably only got to that after multiple rounds of blaming the vendor for flaky hardware (which would generally be the more likely cause especially when you're on the bleeding edge).

<comment> yjftsjthsd-h > It's easy to say in retrospect that they should have checked the EDAC > settings first, but as it happened they probably only got to that after > multiple rounds of blaming the vendor for flaky hardware (which would > generally be the more likely cause especially when you're on the bleeding > edge).

Yeah, part of the nightmare of cosmic-ray bitflips (or any random bitflips, I suppose) is precisely that they don't look like anything. A server randomly locks up. A packet has a bad checksum (and is silently resent). A process gets into an unexpected state. That buggy batch job fails 1% more frequently than it used to. Nothing ever points to memory errors, except that there is no pattern.

</comment></comment></comment></comment><comment><USER9> "Fail on certain moon phases" reminds me of a C++ bug I encountered while trying to set up the demo for PSIP (Digital TV Guide) destined for NAB in Las Vegas. We had programming schedules resembling excel spreadsheets and my job was just to create a good one for the demo. I would spend all night making one and sent it to my boss and each morning would get in trouble for sending in blank schedules and had no idea why. On one occasion I happened to be editing at 3am and noticed all of my edits rolling back one by one. It was actually viewable on the screen as if someone took control of excel and was rolling back each field. My immediate thought was I really need to get some sleep but later we found the auto-save feature inverted itself after 3am exactly and would go through each delta one by one rolling itself back as it had been edited. The bug was found in the calculation of the vernal equinox which moves from 3am to 9pm to 3pm. Since it was triggering the leap year code 6 hours of time would get rolled back edits and all! This was of course 2008 year of the digital transition from analog cable which happened to also be a leap year.

</comment><comment><USER10> I can't scan documents when my daughter is asleep. When she is awake, all is fine, but the minute she goes to sleep, and I'd like to use my free time to scan documents and suchlikes, forget it. I could still print documents on the same device though. Here's what I found:

The printer-scanner was connected to wi-fi. The wi-fi router was in my daughter's room, as that is where the cable socket was, tucked just behind a bench in her room. It was also near that bench that her baby monitor camera was standing. It wasn't wi-fi connected, but for whatever reason it interfered with the wi-fi signal. Same with the receiver, if I put it near my laptop, the wi-fi connection would die.

The monitor was off most of the time, and on precisely when my daughter was asleep.

As for why I could still print, just not scan: presumably that's something to do with the bandwidth, I'm guessing it took more wi-fi bandwidth to send a scanned image than to print a document (I never printed pictures on this printer).

<comment><USER11> You really want to think about moving the Wi-fi out of the baby's room. Get a GQ-390 meter and you will see the torrent of dangerous radiation flooding her room, above recommended levels.

I was able to get my Internet provider to relocate the device to my basement.

<comment><USER12> I just exposed myself to radiation from the largest fusion reactor in the solar system! Should I panic?

....errr....

Wait.

I have just been slipped a note that the first sentence I wrote means "I just went for a walk in sunshine."

<comment><USER13> I have a fun story about the sun.

I used to work for an IPTV provider (who also happens to make a search engine). We received TV feeds from TV stations via satellite; we had an antenna farm, received their signals there, compressed/encoded the signal, and then sent them to customers over the network. Because we only had one antenna farm, we would have TV outages throughout the year -- sometimes the satellite happens to be directly in front of the sun, and the sun is a huge RF emitter that would overwhelm our receivers. (I asked if we can just move the satellites, but was told that we didn't have the delta-V budget. We eventually built another antenna farm. Another question I asked is why can't the TV stations just send us video files over the network. Apparently it simply isn't done; they have used satellites for decades, so why switch?)

</comment></comment></comment></comment><comment><USER14> I got a tiny one of these.

One time I was writing some code in C. I found a bug, the solution seemed pretty obvious, so I fixed it, recompiled the code, and ran it again. The bug was still there.

I took a look at the rest of the code in case that I missed something. I couldn't find anything, so I added a few print statements and recompiled. I ran the code and nothing came up.

Interesting, apparently the code is not executing the branches it should. I verified the input data and code. It didn't make sense, there had to be some serious bug there that I didn't consider. I added a bunch more prints.

Recompile and execute. Still nothing. Wait a minute, THAT doesn't look good. I added a print statement right at the entry point of the program. Nothing.

At this point the root problem became apparent; my changes just weren't getting compiled. Phew, problem solved! I cleaned all the cached files and recompiled the source code. Those print statements still weren't coming up.

At the end I had to move my source code to another machine and compile it there to get it working. I suspect some global variables or path trickery to be involved, but up to this day I still haven't got a clue what was wrong, or have I seen it happen again.

<comment><USER15> .o file stamp newer than .c timestamp. Make figures out file can be skipped. Run recursive touch on these problematic files and headers.

<comment><USER16> The most fun is when the build then restarts, and do a build cycle until the the clock passes some magical timestamp making the build succeed.

<comment><USER17> More magical, your object files reside on a network shared drive that has a clock slightly different than that of the compiler machine.

</comment></comment></comment></comment><comment><USER18> Display intermittently blanking, flickering or losing video signal:

[https://support.displaylink.com/knowledgebase/articles/73861...](https://support.displaylink.com/knowledgebase/articles/738618-display- intermittently-blanking-flickering-or-los)

"Surprisingly, we have also seen this issue connected to gas lift office chairs. When people stand or sit on gas lift chairs, they can generate an EMI spike which is picked up on the video cables, causing a loss of sync. If you have users complaining about displays randomly flickering it could actually be connected to people sitting on gas lift chairs. Again swapping video cables, especially for ones with magnetic ferrite ring on the cable, can eliminate this problem. There is even a white paper about this issue."

<comment><USER19> I've seen this one on twitter [https://twitter.com/royvanrijn/status/1214162400666103808?la...](https://twitter.com/royvanrijn/status/1214162400666103808?lang=en) There's probably some civilizational complexity limit where the unexpected interactions between seemingly isolated pieces of tech become so bad that we cannot introduce anything new without introducing legion of weird bugs.

<comment> no-s >There's probably some civilizational complexity limit where the unexpected interactions between seemingly isolated pieces of tech become so bad that we cannot introduce anything new without introducing legion of weird bugs.

Come to think of it, I believe that meme was wondering about on UseNet from before the early days. I think Vernor Vinge alluded to it in one of his novels, (paraphrasing here) some interplanetary civilization crashing because in-transit space traffic was so dense no new launches could occur in a useful time frame due to safety-lock outs, and they didn't want to accept the risk in changing the safety margins...

</comment></comment></comment><comment><USER20> My favourite crazy bug was during a university course on autonomous robotics. One of the other groups was using a a metal castor at the back of the robot along with 2 driven wheels. After a little while their robot would completely crash and stop responding.

I'd previously encountered a similar issue which was due to the library code we'd been given which opened a new /dev/i2c file for each motor command, eventually exceeding the max file handles and killing the program. So I assumed it was something sensible like that.

Some time later they got all excited and called us over to explain the real reason it was crashing. Their robot would initially work fine for a reasonable period of time. Then when the robot drove over the metallic tape on the floor of the arena it would die. The robot must have been building up a static charge while moving around which would eventually be dissipated when the metal touched the tape.

I wouldn't have believed it had they not setup two tests, one outside the normal arena and one inside. Changing the metal castor for a bit of lego fixed the problem.

</comment><comment><USER21> Obvious in retrospect, but very surprising to my inexperienced past self:

I'd been working on some C code for an hour or two. It wasn't behaving how I expected it to (and at the time I knew nothing of debuggers), so I added a print statement and recompiled. I got a compilation error: something like "Syntax error on line 123: #incl5de <stdio.h>". Shocked, I scrolled to that line in my text editor to fix the typo, but it wasn't there. I compiled the same code again and there were no errors.

Turns out there wasn't a bug in my code! I immediately shut down my computer because my RAM was going bad. To this day, what surprises me most is that my computer was able to successfully boot and behave normally for an hour or two, even though random bits were apparently being flipped.

<comment><USER22> Reminds me of a rouge NIC, flipping a single bit every now and then, that took down S3: [https://youtube.com/watch?v=swQbA4zub20&t=46m02s](https://youtube.com/watch?v=swQbA4zub20&t=46m02s)

Ref: [https://news.ycombinator.com/item?id=13859733](https://news.ycombinator.com/item?id=13859733)

<comment><USER23> I recall a talk about someone registering domains similar to google.com and due to occasional bit flips people landing to these domains.

<comment><USER24> An article on something like this was linked on HN a few years ago: [https://news.ycombinator.com/item?id=2944445](https://news.ycombinator.com/item?id=2944445)

</comment></comment></comment></comment><comment><USER25> I remember reading one years ago where someone had a problem installing new software on some embedded device - whatever they did it came up "checksum is bad".

After much testing they eventually realised that the checksum literally was the hex "bad".

</comment><comment><USER26> I have two favourite bugs, one weird and one dumb.

Weirdest one was an IDE where the colorizer gave up on source lines longer than 998 chars. Instead it rendered the whole line as background, i.e. invisible. I once wasted two hours debugging a program with an invisible line of code!

The dumbest was a postage billing system for a bank using a third party Print- and-Mail company. Somehow the billing system went live adding the previous day's total postage costs to itself, then adding the new day's postage. These expontentially growing totals were then paid automatically by the accounting system each night... So it goes live, ... and a week later Finance gets an alert the account is overdrawn... They actually paid out nearly $1b in postage costs before hitting their internal credit limit with the bank's treasury.

<comment><USER27> Which IDE was it?

<comment><USER28> _That_ is what you want to know? :D

<comment><USER26> A highly customised Visual Studio I think, with lots of in-house extensions. To be fair, this was long, long ago. Nowadays you'd write an extension and leverage the Language Server Protocol.

<comment><USER28> Cool, good to know... Now, did they get that $1B back? :)

</comment></comment></comment></comment></comment><comment><USER29> My favorite was when I was working at SGI after it had taken over Cray Research. I was one of the lowly Cray guys in Wisconsin working with the wonderkind in California. I was to run the regression tests on the chip being design in California using some software that they had provided. I would run the tests, but some days they would crash in the middle of the night. Then the California guys would be angry that they got no tests results. I started debugging the code and got to a program called lswalk that would dole out jobs to the dozens of servers to be 'run'. The code was written by a hot shot young MIT graduate, but I was sure that the problem was with this code. I got the source code and started looking for problems and one thing I found was that if one of the servers resplonding with error the code would print out an error message. One problem though... The error string printed had an uninitialized string, so that when the printf routine would search for an end of string that was never there, probably overwriting buffers and crashing code all over the place. So one lesson is that even the best and brightest make mistakes. Sometimes I wonder how we accomplished anything in those days with software that had so may trap doors beneath it.

</comment><comment><USER30> I love this article.

My most recent bizarre bug: a coworker came to me with a bug where no matter what he tried, he could not get an if some_var is null to be true. The debugger would show the value was null. The logger showed the value was null, but the if statement would not work. After a morning of trying to fix it, he asked if I would take a look. I told him to put the null in quotes in the if. It worked. Turned out a JavaScript library had a bug where it would use the string "null" instead of null.

<comment><USER31> There's a popular progressive API I've been forced to work with that uses "true" and "false" instead of their respective booleans. The most egregious that I've ever seen in this class of errors was an API (from Google!) that returned " false".

</comment></comment><comment><USER32> The very first time my company farmed me out to work onsite with a client. Day 1, Job 1: download the client's website code and get it running on my laptop.

... It just wouldn't work. Everything I tried - failed. Everything else on my laptop was working fine, except this code. Everyone else who had ever downloaded the code had managed to get it working on their machines within a couple of hours. Colleagues working onsite with me tried to help, but everything they tried - failed. Finally the decision was taken to reset my laptop to factory defaults and reinstall everything. That took up half of Day 2. Tried to get the client's site running - failed! Things were beginning to get really embarrassing - all this was happening in full view of the clients. In desperation, my company called me back to their offices and issued me with a new laptop. Back onsite, the code downloaded ... and worked first time!

Turned out that the issue was that my hard drive filesystem had been setup (not by me!) as case-sensitive, and the client code included a file with an all-caps filename, which the code called using a lowercase string. Almost lost my job over that one.

</comment><comment><USER33> My weirdest that I can recall right now was a PDF file that would not print. Since the printer was typically unhelpful with the error message, as was support (this is a room-sized commercial printer but we didn't get the help I'd really expect), I had to dive into it myself.

Long story short, whatever had produced the PDF had also embedded a TrueType font where one character was named //something. This is fine. The character just has a weird name, but it works. It's technically up to spec AFAIK, and I got it out of the PDF with ttfdump to have a look at it.

Well the printer's internal RIP, unknown to us, converted the PDF to Postscript when rasterising. And //something is called an "immediately evaluated name" which I forget the details for, but basically this font character, interpreted as postscript, was causing a lookup for a named variable which did not exist. Hence the crash.

I had a similar one where Adobe InDesign had been used to make a PDF where someone had selected the words to change the font, but not the spaces between (or perhaps they did, and it was a bug). This meant that the PDF included a subset font that only included the space character. Since the space character is not drawn, this resulted in a 0 byte long glyf table. Based on my reading of the TrueType font spec at the time, this isn't really proper.

Printer didn't like that one bit and died as it does to anything that smells slightly wrong. Adobe said it was fine and up to spec though, apparently 'TrueType' has a different meaning inside a PDF :)

</comment><comment><USER8> My favorite out of these is the 500 miles email limitation one. I work mostly on big bulky manufacturing equipment but my job is to abstract out the computing part. This story reminds me that every time I want to do something I am still limited by physics. I am reminded of this story whenever the hardware people ask me to insert an artificial delay in computation.

<comment><USER34> Yes the old limits of the time style bugs are great. SOme arbitrary limit or variable to hold a value deemed way more than enough at the time, for years/decades later to jump out and catch you out. Y2K was one of those well known ones, but been many of those types of bugs.

</comment></comment><comment><USER35> I love programmer stories like this. My favourite personal experience was on my first Ruby on Rails project after first moving to London. I was pretty green at the time, having had only a few years of PHP experience under my belt and little else.

We had to build a Rails app around a poker game. We didn't own the source to the poker game or its API, but we had to embed it nonetheless. We had this really strange issue where some people, under a certain circumstance, couldn't get into the game. It would just boot them out. Me and my team mate must have poured through the Ruby code dozens and dozens of times and found no evidence of this bug, no ability to reproduce it; bearing in mind I was still learning the ropes and jumping head first into an unfamiliar codebase is quite daunting.

Eventually I decide to get my hands dirty and I start poking into this game engine. We embedded it as s flash widget, but the server doing most of the work was written in a mix of C++ and Python. I didn't fully comprehend what I was looking at but, even though things looked suspicious, I couldn't put my finger on an actual problem until I looked at the API written in Flask and noticed that one line of code didn't look like any other.

        some_value = params['some_key']

If the request didn't contain the parameter `some_key` then this would raise a KeyError.

After maybe three solid weeks of trying to debug this thing, I submitted a one line patch:

        some_value = params.get('some_key')

It's not quite as weird or as fun as most examples but for me personally it was such a great lesson in debugging and being curious about unfamiliar stuff, rather than closed off or afraid.

</comment><comment><USER36> See also:

"COMPUTER-RELATED HORROR STORIES, FOLKLORE, AND ANECDOTES"

[https://www.cs.earlham.edu/~skylar/humor/Unix/computer.folkl...](https://www.cs.earlham.edu/~skylar/humor/Unix/computer.folklore.from.net.rumors.html)

"Computer Stupidities"

[http://www.rinkworks.com/stupid/](http://www.rinkworks.com/stupid/)

</comment><comment><USER37> There's a GitHub repository for such stories [1]. I've even contributed one of my stories: "Script crashes before 10 a.m." [2]

[1]: [https://github.com/danluu/debugging- stories](https://github.com/danluu/debugging-stories)

[2]: [https://darekkay.com/blog/script-crashes- before-10/](https://darekkay.com/blog/script-crashes-before-10/)

</comment><comment><USER38> This scratches a "thedailywtf" itch I had forgotten I had :)

</comment><comment><USER39> We're actually working on a collection of such stories internal to our division. We've found that these tales are a great way of helping people understand the complexities and quirks of our nearly 3 decade old code base.

<comment><USER40> I think story telling is an underrated technique in our profession.

In all projects there are coding rules (like "make destructors noexcept"). A rule sticks much better if you also tell a story about some debugging caused by _not_ following the rule.

<comment><USER41> I worked at a place one that had a style guide for the main front-end language used with links to terrible things that had happened as a result of breaking the style guides rules.

It was surprisingly effective, so I completely agree with you on the story- telling.

</comment></comment></comment><comment><USER16> One favorite in the category is always the good old "Can't print on tuesdays" ubuntu bug that has been submitted here a bunch of times: [https://bugs.launchpad.net/ubuntu/+source/cupsys/+bug/255161...](https://bugs.launchpad.net/ubuntu/+source/cupsys/+bug/255161/comments/28)

</comment><comment><USER42> I had this one.

We were using mSQL in the 90s for web projects. A very important customer wanted a "real" database so we bought DB2. Because we didn't have an IBM plattform or Solaris we went with Windows NT.

Everyhing went fine, until one day we recognized the website being slow. Investigating brought the database as the culprit. So I went there and logged into the NT box in the data center and checked the DB2. Everything was fast. Back to my desk and the database was slow again after some time. Back to the NT server and the same thing happend.

After quite a long time I found the real culprit. The NT pipes GL software render screen blanker. After some time without interaction the screen blanker started up and took all the CPU. So the database and the website went slow. Someone had set the screenblanker to the nice GL pipes renderer.

[Searching the web, IBM introduced DB2 for Windows NT 31.10.1995 and I went to Cebit that year to check it out]

<comment><USER43> This reminds me a nice day we spent at customer's premises trying to figure out why DB2 won't install or start properly on a win2k box. Weird error messages etc. Problem was that it didn't like that the box was named 'DB2'...

</comment></comment><comment><USER44> Was looking for the 500-mile email - it's there.

</comment><comment><USER45> First month or so at my new employer, big consultancy firm for a financial institution. Had a fairly complex distributed monolithic application integrated with Tibco EMS, Oracle DB and distributed XE transactions.

Regularly, but randomly, in production, after receiving a good amount of messages in the input queues, (which then got rerouted to other event queues for parallel processing) some DB transactions simply were getting stuck. Not rolled back, but stuck in limbo -- after a while the DB simply refused new transactions because so many were stuck. Nobody got a clue on why that was happening, it meant regular manual restart of the services and re-feeding of the failing messages. Users started to get fed up and the project threatened to fail.

Got into it, after couple of weeks of investigations and trial and errors with all possible weird flags, turned out that the version of Tibco EMS had a _wierd_ behavior with distributed transaction when the queues got full of messages (queues had 50MB size limit).

Instead of rolling back gracefully the JMS+JDBC XE transaction, it...kinda exited with an IO error.

Turned out that newer versions of Tibco EMS fixed that issue, but no way to ask ops to install that new version. Since upgrading was out of the question, the actual fix was to enable message compression to limit the size of the messages coming into the queues, turned out that the XML we sent there were up to 1.5MB (!)

After discovering that, became basically a war hero and respected by the client as the "savior of the project". Good times.

<comment><USER46> Your compression workaround reminded me of an issue I ran into a while back.

My team at work uses a reporting tool for vulnerability assessments and pen- tests; basically you can import a bunch of data files, review it in the web app, and generate a report.

I would run into cases where I couldn't upload one of my data files. The web app is JS-heavy, lots of things going on in the background without much visible feedback. It turns out that the programmers had implemented the upload as this async task with a hard-coded timeout for completion, and they likely wrote it while they had great network speed.

I'm on DSL, and generally, it gets the job done. However, upload speed is only 1Mbit/s, so with a big file, my upload would time out. It's hard-coded remember, so it didn't matter that it was still functioning when it got clobbered.

It occurred to me that some file formats, like WAR or Office documents, are basically Zip archives under the hood, so I put my large XML file into one, and tried that.... and it worked! Something on the back-end quietly unzipped my upload and imported the file it contained.

Funnier is that when I mentioned it to the devs, this behaviour was not something they expected. Probably built into a library they use.

</comment></comment><comment><USER47> I had a function which only failed at 8 or 9 minutes past the hour.

It parsed a string containing the timestamp and "08" or "09" was interpreted as an invalid octal number. Argh.

<comment><USER48> The beauty of loose dynamic typing

</comment></comment><comment><USER49> This one happened last night. A student contacted me because her Anaconda Jupyter notebook (installed on her laptop) just wouldn't connect to the Python kernel. (The notebook itself would load, though, meaning that the server was running fine. It's just that the kernel and its websocket was failing.) I should point out that, because of COVID-19, this troubleshooting was over Zoom, which complicated the diagnosis a bit.

She had not been using Jupyter for several months, as we have been writing stand-alone programs in class using Spyder (the editor that comes with Anaconda), and the command line, and Jupyter had worked the last time that she tried it.

We restarted everything, and still the problem was there. I helped her to update everything, but that didn't solve the problem.

Finally, I looked at the error messages in the console where the Jupyter server is running. It had a huge list of errors, all relating to the pickle library.

We had done an exercise with pickle in the class, but nobody had reported a similar problem. When we looked in her classwork directory, though, we saw that she had created a "pickle.py" file when she was testing something with pickle. But, at that point in the class, we were working in the command line, and everything (including Spyder) still worked just fine.

Evidently, this was the cause of Jupyter's problem. When trying to start the Python kernel in Jupyter, it imported pickle, and evidently it imported her test file rather than the actual library. The fix was simple: we renamed her test file, and everything worked perfectly.

</comment><comment><USER50> Had a flaky unit test that would randomly fail with some random Chinese character in the output.

The test was running a log parsing tool against a temporary file that had a pseudo-SQL syntax where you could "select ... from c:\Users\\...\temp\abcd1234.xyz\testdata.dat". The temporary directory was a randomly generated name so that the folder was guaranteed to be empty before every execution of the test.

The test failed on the rare occasion that the randomly generated temp dir consisted of the letter 'u' plus four characters that were valid hex digits. When this happened the randomly generated dir name interacted with the backslash before it and become a Unicode escape sequence. It was easy to fix but that test was flaky for months before anyone worked out why.

</comment><comment><USER51> A bank I did some contracting for had a problem where their Token-Ring network would crash at random intervals during the day in one of their branches. It would also crash at night, but the times when it would happen were more predicable.

And that was the clue they needed to solve the problem - it turned out that the wiring installers had run the cable up the elevator shaft. When the elevator stopped at a certain floor the door motor was sometimes interfering with the signal. The more-regular nightly disruptions were because of the security guard making his rounds.

It turned out that run was pretty close to the length limit for 16mbps Token- Ring, so they added a repeater in the middle to boost the signal strength.

</comment><comment><USER52> I eventually gave up without finding the issue, but somewhere deep inside one version of the Sphinx full text search software was a bug that would sometimes switch query got what result set. It only happened sometimes when queries were within a few seconds of each other, but it wouldn't happen with only one front end process even in multithreaded mode and would disappear if requests were _too_ close together. If I'd found a way to reproduce it I'd have submitted it to the Sphinx team, but after a few days of potentially private info leaking I gave up and moved to PostgreSQL's FTS.

</comment><comment><USER53> I just had a weird bug in a programming competition.

You basically had to sort the English letters that occur in a text according to their frequency descending. Except the one letter that occurs the least, needs to be sorted as if it occurs the most.

The expected output of the sample case was TPFOXLUSHB

I ran my program on the sample, the output looked correctly; then I submitted it, and the judge said it failed the sample case. In fact, it was printing ͲPFOXLUSHB

That nearly looks like the correct output.

I had confused two variables and it was printing the frequency count as codepoint rather than the letter. But such a coincidence that it looks the same

</comment><comment><USER54> Glad to see the More Magic story in the list.

</comment><comment><USER55> I used to work as a senior technical support for BEA Weblogic and had all sorts of crazy situations to debug remotely. Including one time when I had to get a person to edit a config file in Vim (which they never used before), on Unix (which they never used before), with me guiding them by phone (no visuals).

This is the one I recorded that seems to fit into the current theme: [https://www.outerthoughts.com/2004/10/perfect-multicast- stor...](https://www.outerthoughts.com/2004/10/perfect-multicast-storm/) (tldr: multicasting on 220.127.116.11 is bad).

And if somebody really understands network and multicast, I would love to know whether I actually nailed the problem or just made it go away accidentally. I have no problems with being wrong, especially this much later :-)

</comment><comment><USER56> Similar to the train being stopped by a toilet flush, in the 90's, I worked with devices based on Microsoft's Pocket PC OS. These were equipped with wireless radios. The transmission of a packet caused some interference that the device interpreted as a click on the screen. The cursor was over the [X] to close the application window, so the application would just quit, looking like it crashed.

</comment><comment><USER43> IBM Java 1.1.8 that was embedded into Lotus Notes 5 (if I remember correctly) didn't have 29th of April if it happened to be on Tuesday.

When you constructed a Date object of 29th of April with such a year that it was a Tuesday, you get the 30th of April when you read back the value. Took a while to figure out why date calculations were sometimes off. The flux of expletives was impressive when we finally did...

<comment><USER57> Any ideas on why it happened? I'm impressed you found that bug though.

<comment><USER43> No idea, it was years ago. JVM-s were extremely buggy. We were quite n00bs at that time also so we didn't dig into the depths of it, we had to fix production issue that we had already spent a lot of time on. We found the bug by just narrowing down a broken case further and further. Finally we wrote a code that just tried a range of dates one by one over several years to find out the pattern. Unfortunately it wasn't possible upgrade the JVM as it was embedded into Lotus Notes, so we wrote our own date implementation (yay!) that satisfied our needs. It was fixed on later Notes version, but our sh*tty date implementation lived much much longer...

</comment></comment></comment><comment><USER58> Not software, but along the lines of cool engineering stories one of my favorites is this one about fixing 230 kV, many-hundred-amp, 10 mile long coax cable in Southern California.

[https://www.jwz.org/blog/2002/11/engineering- pornography/](https://www.jwz.org/blog/2002/11/engineering-pornography/)

</comment><comment><USER59> Thanks, this is hilarious! "Okay! I'm braking now", definitly my new going to the toilet catch phrase.

</comment><comment><USER36> I'm very disappointed that the very first entry in the list is a bogus story _confirmed_ to be false.

I mean, if you simply want general computing legends of unconfirmed veracity, read "The Devouring Fungus" by Karla Jennings.

<comment><USER40> The list desperately needs another story where the title starts with A or B.

</comment></comment><comment><USER60> The SSH one is brilliant

I find that most of these "hard problems" are something small, so something that's almost unnoticeable to not break immediately but that makes it kinda work.

Now finding exactly what is the trick

</comment><comment><USER61> for me it was 21 day bug for a broadcast video encoder, some internal frame counter was coded with int instead of int64. would reset every 21 days. fun to debug it was not!

</comment><comment><USER62> It is good to pore over these legendary tales. They come in handy when we need to break out of the moment and try something outrageous to solve the problem.

</comment><comment><USER63> Crash cows: "There were often significant food shortages in the Soviet Union, and the government plan was to mix the meat from Chernobyl-area cattle with the uncontaminated meat from the rest of the country. This would lower the average radiation levels of the meat without wasting valuable resources."

There is some sense to that: low levels of radiation are not a cancer risk last I read - everything we eat is slightly radioactive. That said, I can't think how significantly radioactive cows could be "diluted" enough.

<comment><USER64> It's worth knowing that all the cows you eat are radioactive, just like bananas. If the cow didn't die, you've got a good chance too.

</comment></comment><comment><USER65> The Crash Bandicoot story is my favourite.

<comment><USER66> This was my first thought too - "Wonder if tht story with the PS1 controller is in there" \- man that must have sucked to debug!

<comment><USER67> It did.

</comment></comment></comment><comment><USER68> Always impressed by Mel's story.

<comment><USER69> That describes a real machine, the LGP-30, which really had absolutely no business being as powerful for the price and era as it was. It used an oscilloscope for its (tiny) debug display: the operator'd read the voltage waveform directly as binary.

[https://en.wikipedia.org/wiki/LGP-30](https://en.wikipedia.org/wiki/LGP-30)

</comment></comment><comment><USER70> So at my previous employer we still had an old frontend running an atrociously old version of EmberJS. I run the build but it suddenly fails. We were on a three weekly release schedule, so I figure it must have happened somewhere in the past three weeks, about 400 commits. So I start Git Bisecting, teaching it to a handful of my more junior colleagues as I go along. It took us way to long to figure out that the original build also failed however.

So my teammates wish me good luck, and I go off on a journey debugging what the issue actually is. As it turns out, the horribly old Ember.JS CLI package version we're using is a version called '0.2.0-beta'. That did not bode well. This frontend of course did not use the nice yarn dependency pinning, just a regular old package.json file, so I go tracing the error into the dependencies.

Eventually I trace the thing do a dependency nested three layers deep or so. A library added a deprecation warning when being used. That in itself is not so bad, but it did that using an injected logging framework from the package using that library. Except that wasn't introduced until a way later version. Ofcourse this tiny little addition could never cause any breakage, so this was released as a semver bugfix release.

The commit time shows it was 23:00 local time ([https://github.com/goldenice/ember-cli- babel/commit/c4c95d6f...](https://github.com/goldenice/ember-cli- babel/commit/c4c95d6f1637bfb8988f68dcacbaa436c6eb94bb)) when I figured the problem out and committed a fix. So I submit a PR to the library, figuring that if the author happens to be awake I won't have to figure out a way to pin the dependency to an earlier version (which would have been easy in a regular dependency, but this was a global dependency, where it's not as trivial as switching from npm 2.x or 3.x to yarn)

The author thankfully responds almost immediately, asking me to provide a fallback to console.warn instead of skipping the deprecation entirely. Makes sense, so I update my PR, submit it within a few minutes, and I see that the author immediately publishes a new version. Finally something works out for me. Or so I thought.

As it turns out the author made a tiny stylistic fix in my code. Except that tiny stylistic fix butchers my carefully crafted if statement, and now the code is broken again. It took me a while to figure out that the new version of the dependency WAS being used, but was also broken.

So I contact the author again, explain the situation. They changed the code immediately, pushing out another update. In the meantime I figured out how to do dependency pinning and all was well with the world again.

And that kids, is the story of how I came to appreciate transitive dependency pinning as a really useful feature.

(It's still amazing to me by the way that I can contact somebody that wrote some random code that our code happens to rely on and get a response within half an hour.)</comment>


Show HN: Golang httpcache. Get a working HTTP-cache with only 3 line of code - <AUTHOR> https://github.com/bxcodec/httpcache <comment><AUTHOR> Hi there, I'm the maintainer of this library. I made a simple library in Golang to ease the engineer if want to use http-cache on each call.

Current version: v1.0.0-beta with support to

\- Custom Cache Storage, in case people want to use their own custom storage

\- Default in-memory cache. For default storage, currently, it's supported only for the in-memory cache.

\- Comply with RFC 7234. Yes, I try to comply with RFC 7234, TIL: Building something with complies with the RFCs is a new challenge for me.

</comment><comment><USER2> Cool!</comment>


Totally useless websites - <AUTHOR> https://www.totallyuselesswebsites.com/ <comment><USER2> I wasted far too much time looking through this list.

</comment><comment><USER3> I disagree with the "totally" part.</comment>


Modern Java – A Guide to Java 8 - <AUTHOR> https://github.com/winterbe/java8-tutorial <comment><USER2> My Java code has improve dramatically since I started (ab)using Optionals.

      String foo = Optional.ofNullable(paramater)
                           .filter(...)
                           .map(a -> ...)
                           .map(b -> ...)
                           .orElseGet(() -> ...)
                           .orElse(defaultValue)

Between this kind of thing, and similar playing with Streams, some of the ugliest code I work with is suddenly quite clear, coherent, understandable.

<comment><USER3> I'm not so up on Java, could someone explain why they chose "->" for lambdas instead of the "=>" that C# and ES6 use, as those languages seem to me like they would be the first place to look for inspiration.

<comment><USER4> In addition to the other answers, I bet it's easier for a CFG to parse knowing that "->" can't also be "negate and greater than" (since negation is only a prefix op) whereas "=>" can easily be a fat arrow or greater-than-or-equal.

<comment><USER5> Why would that be any harder? The tokens are different.

<comment><USER4> True, Java does not support => as an alternative for >= so the ambiguity does not exist. It was just a guess, but since it was probably not right, I decided to dig. This mail post[1] explains that the extra equals sign becomes a bit annoying w/ other equal signs in the same expression.

1 - [http://mail.openjdk.java.net/pipermail/lambda- dev/2011-Septe...](http://mail.openjdk.java.net/pipermail/lambda- dev/2011-September/004021.html)

</comment></comment></comment></comment></comment><comment><USER6> I've always had some sort of mistrust of Java (neo-Cobol ohnoes!) but I'm liking it more and more. Mostly I've come around to the fact that static typing isn't as bad as my young self thought (and can in fact be quite cool). The verbosity and "enterprisy" and committee oriented feel are still a bit off putting but I've made a commitment to try some Java on a couple of weekends. Did a bit of research and there's actually nice looking lean web-stuff, too (Spark Framework, Ratpack). Played with the J8 features back when it was released and they are pretty cool.

Cliffnotes: I think I have discriminated against Java mostly on a "religious" basis. I think it's a great time for people like me to give it another shot (especially if you have grown accustomed to functional ideas form other languages in the meantime).

The repo is an excellent resource, bookmarked.

<comment><USER7> Good luck with your experiments.

Many of us in the 90's jumped into Java, in spite of its issues, because it made it more pleasing to write cross platform code, than using C or C++ with CORBA/COM across OSes with compilers that were still playing catchup with the standards.

<comment><USER8> I never understood the following : Eiffel had a better syntax, better support for core issues, Void type, design by contract, multiple-inheritance and an intermediate virtual machine for portability although the most common target was C code. To name a few. I was using Eiffel in a small side project just about when Java released their 1.1.4 (a version before the object serialization library, iirc). It was a sticking point for me. Add to that, to program in java before IBM Visual Age was released was to using vi/notepad(++?) editors. Coming from Smalltalk env that was a bit of a shock. Eiffel commercial IDE from ISE was way better and its clever use of pick event on the mouse's right click to drop to compile/run targets was actually easier on my fingers, which is a side show, but good quality products demonstrate quality at unexpected places.

I am still programming in java in my day job, knowing fully well that Java was not my top five language choices. Eiffel's type system was a bit more advanced than Java even way back in those days, so not only was I stuck without an ide till VA for Java showed up or VisuallJ++ I also had to accept weak inheritance and design by contract models. Performance of course was not even close to an Eiffel compiled C code, which was probably addressed in later versions of java.

My point being, that just as Eiffel was something that was better and set aside by a few leading development shops, there must have been other languages that could/should have received a fair share in language evaluation by programmers, I dont necessarily mean limiting to Ruby, Python for example, despite them being excellent tools, they may fall short in an enterprise ecosystem. What happened at least I seem to think instead is a sort of groupthink to start coding in Java because of rubbernecking. All in all, I think that things could have been better if people paused to understand the Java language model and fixed in java 1.2/3, or some earlier version so we didnt have to wait till Java 1.8 to get these features. /rant.

<comment><USER9> I developed a great interest in Eiffel after I read Bertrand Meyer's book "Object Oriented Software Construction" (which I still think is one of the all time great books on object-oriented programming), but I never found anyone else that was using it and the IDE was way too expensive for me at the time. It just seemed to me like one of those technologies you'd probably get to have fun with on some niche project at a defense contractor, but that you'd probably never see out in the more general software development world.

<comment><USER8> I was one of those folks who actually bought the personal license. Over years, SmallEiffel came along and improved the ecosystem. Of course, now Eiffel has a community/enterprise model. In any event, I do think that they were pricing them out of the market. Defense contractors, seem to like ada more. The feature that I miss from Eiffel is the Expanded (allocated on stack feature). Even haskell has some #untyped to deal with boxing primitives. I thought that Expanded was a stroke of genius: bringing specification to implementation without the middleman(aka heap).

</comment></comment></comment></comment></comment><comment><USER10> This is a really nice overview of the new Java 8 features. I like how this tutorial communicates mostly with code snippets and has "read more" links for most topics.

If you are using Java and have not tried Lambda Expressions, method references and default methods, do so now! After that, you'll think "How was I able to live without that?".

I also think that the way Java integrates lambda expressions (via functional interfaces) is really nice. Sure, you have to write some boilerplate code in some scenarios. But since they re-used interfaces and did not event some "function" type (or similar), it fits really nicely into the existing language.

If you don't know Java, this tutorial will probably not be enough - But that probably wasn't the intentian anyway. It's a great intro to what has changed with Java 8 (and all those changes are real improvements for the Java language, IMHO).

<comment><USER11> _If you are using Java and have not tried Lambda Expressions, method references and default methods, do so now! After that, you 'll think "How was I able to live without that?"_

I think instead of "How was I able to live without that?", people are more likely to exclaim "about time!"

<comment><USER10> Well... depends on who you ask. I, personally, was more in the "about time!" camp. But I have met several programmers who didn't think they'd need those "newfangled features" at first. Especially in the "early days" of Java 8... And sometimes I still meet people like that. So that scentence was targeting those developers that are still stuck with Java 5 or 6 or 7 and have never tried Java 8.

Also, I was once co-teaching a C# workshop where we showed people how to work with LINQ and extension methods (long, long time ago ;) ). Some participants asked us if there was a way to configure the compiler to _forbid_ those features for their team.

Not everybody likes "modern" features, I guess ;)

<comment><USER12> Well technically they didn't need them, no - I've been working with for and for-in loops for years before learning other languages (like Scala, JS) that had a .map function that could do the same thing but much more compact and expressive. So their argument isn't wholly invalid, and TBF I'd rather see a regular for-in loop than my colleague's atrocious 200-character oneliners.

</comment></comment></comment></comment><comment><USER13> > maps don't support streams

This is very misleading. There is not a #stream method on Map because Map supports three different streams:

        map.keySet().stream()
        map.values().stream()
        map.entrySet().stream()

<comment><USER14> But those are streams on sets.

<comment><USER13> Those are streams on views on the map. My complaint is that a reader seeing "maps don't support streams" with no further explanation is unlikely to get the correct idea, and may infer that something like

      map.entrySet().stream().filter(e -> e.getKey().equals(e.getValue())).findAny()

is not expressible.

</comment></comment></comment><comment><USER15> This is a really nice write up!

Java 8 is such a huge improvement over previous versions. One thing it still lacks is a better literal format for defining maps, etc.

I just self published a book that uses Java 8 ([https://leanpub.com/powerjava](https://leanpub.com/powerjava) if you will pardon a plug) and to be honest I got some pushback from a few people who like my books as to why I didn't use Clojure, Haskell, etc. To be honest I do prefer Clojure and Haskell, but the Java ecosystem is so huge, with so many good libraries that sometimes it is the more practical language to use.

<comment><USER16> Mark, I am sure you know this, but I am posting this for the benefit of others who might not.

For literal maps, I tend to use double brace initialization style. It is more verbose than languages with direct literal support for maps and creates an extra anonymous class, but it keeps the map in a single syntactic construct. For lists, I use Arrays.asList(a,b,c) style; if you static import the method it reads nicely as-list-a-b-c.

<comment><USER7> Which is a no-go in Android as it can prevent GC in some cases, because as anonymous inner class it holds a reference to its parent and you end up leaking Activities.

It is usually presented as anti-pattern in talks about Android performance.

<comment><USER16> Ah, I have never programed for Android devices. Thanks for pointing this out.

</comment></comment></comment></comment><comment><USER17> I personally jumped right into Clojure on the jvm without much Java experience. I've found it to be awesome. Is there a non-political reason that someone should choose Java over Clojure (or possibly Scala)? Where does Java shine?

<comment><USER18> All IME, and as a Scala fan:

Clojure has a weakly integrated type system (an inherent disadvantage of optional type systems). At the simplest level this makes silly errors much easier and means you have to write more tests to maintain the same defect rate.

The lack of types mean you require extensive use of macros for advanced functionality. IME macros have major maintainability issues in a multi-person codebase.

Both these things are major disadvantages for automatic comprehensibility of code. Autocompletion can be more-or-less usable but will never be as good as in Java or Scala. Automated refactoring is inherently unsafe in the presence of macros (Scala's fancier typed constructs (typeclasses, for/yield with custom types) mean you need macros much less often; Java tends to force you to expand these things out by hand (or else use annotations which act as de facto macros), which has its own maintainability issues but does at least mean automated refactoring will work correctly).

Some of the language culture pushes people towards less principled abstractions. From this side of the fence that looks like anti- intellectualism; no doubt from their side it's pretension on typed programmers' part. But either way I think they're setting themselves up for long-term maintainability issues (e.g. the semantics of clojure transducers in the presence of errors are infuriatingly not-quite-right, which will either remain a painful gotcha forever, or necessitate a painful migration in the future).

As a minority language Clojure may not be as well supported in the surrounding ecosystem - partly things like IDEs but also code coverage tools, profilers, monitoring.... Remember the JVM ecosystem is wider than just the languages themselves.

There are good things about Clojure, but it's by no means clear-cut.

<comment><USER19> > IME macros have major maintainability issues in a multi-person codebase.

I have heard this a bunch, and I believe it, but I've never personally had an opportunity to use a language that supports macros for a multi-person codebase. I'd be interested to learn what are some of the pitfalls you've seen, if you don't mind sharing.

<comment><USER18> It's not like I have a lot of experience (macros are much rarer in my primary language, Scala). But it's usually just someone doing a refactor that "makes sense" according to the logic of the regular language but not within the macro. A lot of it is simple, "silly" things like trying to extract a repeated piece of logic into a method call, only it doesn't work because the logic uses something the macro hooks into, or just is transformed in a surprising way by the macro that makes the reasoning incorrect. To take a completely trivial real-world example, renaming a field is usually a safe operation that you can do automatically (with an IDE) without even testing, but if that field's used in a macro then maybe it represents a key in a JSON object and you've just broken your web API. You notice it pretty quickly, and it's a two-minute fix - but these kind of "thousand cuts" scenarios really slow development. Outright production failures are less common but I've seen those when test coverage was poor or other failures interacted.

</comment></comment></comment></comment><comment><USER20> It wasn't touched on in this document, but I think more people should hear about Nashorn. It is the new JavaScript Engine that, I believe, supports ES 5.1. It allows you to do some pretty neat things with Java/JavaScript interacting in the same VM. Java 9 is planning on supporting ES6 <source needed>.

You can see an introduction on this Oracle documentation: [https://docs.oracle.com/javase/8/docs/technotes/guides/scrip...](https://docs.oracle.com/javase/8/docs/technotes/guides/scripting/nashorn/api.html)

</comment><comment><USER21> I am the only one that has been thinking for months now that "modern" is the most over used, pointless, and almost negatively connotative word in this industry? Thought exercise: Imagine the ugliest flat and incomprehensible UI widget you have seen lately, the most bloated website load that doesn't crash the browser, and some mindless office dialog about "rising cloud costs". Now pick one word. Survey says - you guessed it. So most certainly, if you need to be more "modern", I'm sure java 8 is a vehicle to take you there.

</comment><comment><USER22> My company switched from 7 to 8 recently, and being able to use streams has been a colossal productivity boost.

I'm actually in the process of putting together a presentation on how to work magic with streams (because I'm one of the few people here who actually knows how to use them), and this will probably get a mention in a "further reading" slide, because it's really well written.

It's also inspired me to make a few slides focusing on the changes to Map and Comparator, which I wasn't aware of until now.

</comment><comment><USER23> A really nice tutorial.

I have been writing Java code since 6 years. Although I can't say that I have programmed in many other languages (I have coded in C/C++ and Javascript), Java has been always very comfortable to use. It's easy to think of a solution in the Object Oriented paradigm, and I think refactoring (when a requirement changes) has not been much of a problem for me. It's only bad designs that make the code more verbose than it should be, and I think verbosity is actually a good way to articulate your thoughts, and see them evolve in front of you. Recently when I started learning Haskell a bit, I found that there are much faster and concise ways of writing code, but I still prefer Java because of the clarity of the way it allows me to express my logic. People would say that writing a line of code in Haskell would do what 10 lines of code in Java would do, but in the long run, I have found that expressing programming paraphernalia like interfaces and classes actually helps the coder in his job (not necessarily results in wastage of time).

</comment><comment><USER24> For people getting into Java 8, jOOL is a great, small library with a "Seq" class that fills in the gaps missing in j.u.stream.Stream:

[https://github.com/jOOQ/jOOL](https://github.com/jOOQ/jOOL)

E.g. if you're coming from groovy/Scala and wonder "why doesn't Stream have this method?", Seq probably either already has it, or the maintainer will be open to adding it for you.

So, use jOOL. It's great.

That said, I still cringe that we have to do "someCollection.stream()..." (or use fill-ins like jOOL) at all, because these methods aren't on j.u.List/etc. itself...

My naive impression is that the JDK designers fixated on making streams support parallelization (because fancy!), and so made some compromises on the API, when in reality 98% of collections are small/not parallel, and I assert a non-parallel, more complete API (e.g. more default methods directly on j.u.List/j.u.Iterable themselves) would have been a net-win to most programmers.

And the parallelized version could be a separate library/jar/something.

<comment><USER20> That library has some fun/interesting generics declarations like this one ([https://github.com/jOOQ/jOOL/blob/master/src/main/java/org/j...](https://github.com/jOOQ/jOOL/blob/master/src/main/java/org/jooq/lambda/Seq.java#L4301-L4303))

> static <T1, T2, T3, T4, T5, T6, T7, T8, T9, T10, T11, T12, T13, T14, T15, > T16> Seq<Tuple16<T1, T2, T3, T4, T5, T6, T7, T8, T9, T10, T11, T12, T13, > T14, T15, T16>> crossJoin(...

<comment><USER25> Pretty much all languages with generics will have that kind of declarations, the only question is how many type parameters they'll go up to. In C# Action goes up to 16[0] while Rust's tuple go to 12[1] and GHC goes to 62[2] (intending 100, but a comment notes declaration of a 63-wide tuple constructor segfaults)

[0] [https://msdn.microsoft.com/en- us/library/dd402872(v=vs.110)....](https://msdn.microsoft.com/en- us/library/dd402872\(v=vs.110\).aspx)

[1] [https://github.com/rust- lang/rust/blob/master/src/libcore/tu...](https://github.com/rust- lang/rust/blob/master/src/libcore/tuple.rs#L238)

[2] [https://downloads.haskell.org/~ghc/7.2.2/docs/html/libraries...](https://downloads.haskell.org/~ghc/7.2.2/docs/html/libraries/ghc- prim-0.2.0.0/src/GHC-Tuple.html)

<comment><USER7> Yes, the only way to avoid this is to do as C++ and D, accept variable number of types in the generics.

<comment><USER25> That's an option, but it requires a significantly more complex generics runtime.

</comment></comment></comment></comment></comment><comment><USER26> interfaces with default implementations? Doesn't this look like... multiple inheritance?

<comment><USER27> You can still not have member variables, so the default implementations can only be implemented in terms of other methods. So it should probably be classified as 'limited multiple inheritance'. It does have the diamond problem though.

<comment><USER28> That was my immediate thought - this is closer to multiple inheritance, as it introduces the diamond problem: [https://en.wikipedia.org/wiki/Multiple_inheritance#The_diamo...](https://en.wikipedia.org/wiki/Multiple_inheritance#The_diamond_problem) According to that page, Java forces programmers to resolve the ambiguity by making it a compiler error. A reasonable approach, I think.

I've done more OO development in C++, and I've found this technique useful - putting a default implementation in a class which was mostly an interface because only a small number of subclasses needed specialized behavior.

</comment></comment></comment><comment><USER29> so... when is this coming to android?

<comment><USER30> Support for java 8 will come with jack & jill. It might also need a new API level (24 ? ), which would seriously hamper its adoption. This is why many people see kotlin as a great thing for the future of Android (even though kotlin on android could also benefit from java 8 as a target)

</comment></comment><comment><USER31> Really like quite a few Lambda stuff in Java8, sadly once you need to touch any of the intrinsic types it gets ugly real fast. Wanted to do some byte manipulation with Lambda and I'd be creating a new Byte object for each byte I touched.

</comment><comment><USER32> Where is aysnc await?

<comment><USER33> CompleteableFuture[1] is the closest we got for now in the JDK.

[1] [https://docs.oracle.com/javase/8/docs/api/java/util/concurre...](https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html)

<comment><USER34> I just started using CompletableFuture last week. The way each method ties your lambdas together without a bunch of type boilerplate is really nice. I still don't have a great handle on all of its methods, though; e.g., what's the difference between get() and join()?

<comment><USER35> CompletableFuture implements Future, so the get method is an implementation of Future.get [1].

Anyway, the major difference is that Future.get throws 2 checked exceptions, InterruptedException and ExecutionException, while CompletableFuture.join does not throw any checked exception. Instead, it wraps any exceptions in CompletionException.

[1] [https://docs.oracle.com/javase/7/docs/api/java/util/concurre...](https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Future.html#get\(\))

</comment></comment></comment></comment><comment><USER36> Gosh... java is getting uglier by these weird syntax.

</comment><comment><USER37> > Default methods cannot be accessed from within lambda expressions.

Why is that?

<comment><USER38> If I had to guess, it's because the lambda will be expanded at runtime to an anonymous interface implementation. Then either the runtime part is the problem or maybe they use a different, but compatible functional interface for the ad-hoc implementation, which then has no access to default methods on the one you're assigning to.

<comment><USER7> That is not how lambdas work actually work in the reference JVM.

They make use of invokedynamic, there is no expansion taking place as many think.

[http://www.infoq.com/articles/Java-8-Lambdas-A-Peek-Under- th...](http://www.infoq.com/articles/Java-8-Lambdas-A-Peek-Under-the-Hood)

</comment></comment></comment><comment><USER39> The changes are breathtaking. It doesn't feel like Java.

</comment><comment><USER40> Sorry, I absolutely cannot stand this "modern" meme.

      (define-syntax stream-cons
          (syntax-rules ()
              ((_ x xs)
                  (cons x (delay xs)))))

      (define (stream-cdr xs)
          (force (cdr xs)))

      (define (stream-map f xs)
          (stream-cons (f (car xs)) (stream-map f (stream-cdr xs))))

      (define (stream-filter f xs)
          (if (f (car xs))
              (stream-cons (car xs) (stream-filter f (stream-cdr xs)))
              (stream-filter f (stream-cdr xs))))

      (define (stream-take n xs)
          (if (<= n 0)
              '()
              (cons (car xs) (stream-take (- n 1) (stream-cdr xs)))))

      (define (repeat x)
          (stream-cons x (repeat x)))

      (define (iterate f x)
          (stream-cons x (iterate f (f x))))

      (define (replicate n x)
          (stream-take n (repeat x)))

      (stream-take 5 (stream-filter odd?
          (stream-map (lambda (x) (* x x)) (iterate (lambda (x) (+ x 1)) 1))))

Not mentioning all these different flavors of Common Lisp streams packages.

As for other features, Scala is around for almost a decade.

<comment><USER41> It may not be modern computer science, but it is modern Java. This post is about modern Java and explains the new (in v8) features very well. What's your problem?

<comment><USER40> OK, what then is Java < 8?)

The point was that if one has a _proper_ old-school computer science (to realize the crucial importance of first class procedures and the power of uniformity 20 years ago) all these modern features are coming for free, to which the code above is an illustration.

But who cares. There is a whole industry which praised Java for almost two decades even without these "modern features".

Any PL student of a decent school, if he is not a hypocrite, would tell you that Java is the worst thing that happened to CS since MS DOS, but who cares about PL theory or even CS? Availability bias and Cargo Cult is enough.

<comment><USER42> Yes, and now that Java finally has these modern features: people are migrating to GO who lack many :(

But then LISPs have always been shunned for bad reasons. While crazy bad languages like C,Java and PHP are picked up by the masses.

<comment><USER43> Your JVM runs on all things written in C and the concurrency primitives are far more elegant in Go than in Java.

<comment><USER42> Agree and disagree. The thing I was going for is that compscience advantages don't seem to matter for language adoption. i.e. popularity does not correlate to compsci purity.

<comment><USER40> Yes, there are different forces in a play.

Let's name it as The Law of a Decent Runtime, and The Law of Attention to Details, and The Law of a Bazar of Ignorant.))

 _The Law Of a Decent Runtime_ is very simple - evolving a decent runtime is very costly and time-consuming. It is also related to the second law and to the inverse of the third - which is a _the Law of Dictature of The Most Competent_. A decent runtime cannot be produced without talent, time, financing, competence and attention to details.

The examples are what came out of Xerox Parc, Bell labs, Ericsson and the best parts of academia (MIT Scheme culture, Scala, Standard ML, LLVM and Haskell, monads aside).

There are also many in-house (Tensorflow) and sponsored open source porjects (LLVM, Golang, Julia, Torch, to name a few). Basically, it is about resources spent on a talent.

What is important distinction - a decent runtime cannot be produced by _the Bazaar of Ignorant_ (PHP, amateur Java code, SAP, and other "fractals of a bad design").

The Linux kernel is very special example, because it combines the law of big numbers, and all these three - it is a product of a whole planet of competent volunteers and paid developers - unlike PHP or Java ecosystem there is very high barrier to entry, thanks to _The Dictature of The Most Competent_ .)

 _The law of Attention To Details_ \- is quite obvious, and related to the first. This is why most of successful projects had a passionate and competent leader who sets the standards, be it Linux kernel, Erlang, Nginx, Gambit Scheme, Python, OpenBSD, Redis, Scala, SBCL, PostgreSQL, you name it.

Popularity has nothing to do with it. It is based on the principle of instant gratification (PHP, MongoDB) or Availability Bias boosted by paid content brainwashing (Java, SAP, MS) without understanding and preferably any thinking. Popularity doesn't mean quality at all, be it junkfood or PHP.

<comment><USER44> Your _Law of a Decent Runtime_ reminds me exactly of the "Lisp Curse":

[http://www.winestockwebdesign.com/Essays/Lisp_Curse.html](http://www.winestockwebdesign.com/Essays/Lisp_Curse.html)

<comment><USER40> There is another good reading.)

[http://the-programmers-stone.com/the-original-talks/day-1-th...](http://the- programmers-stone.com/the-original-talks/day-1-thinking-about-thinking/)

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER45> Enjoy these features while you can.

It's a matter of time until your team lead takes a glance at the code-base, deems them unreadable and "too clever" (as in "where am I going to find cheap developers now?") and makes you revert all the way back to Java 6.</comment>


Ask HN: Do you pay for a commercial server OS (Solaris, RedHat..)? Why? Why not - <AUTHOR>

<comment><USER2> It's like paying home insurance. If something breaks down we can point to these support company and say I'm waiting on their reply. Most of the time the support is useless as they say it's your custom code blah we don't support it.

Lot of times Google search is faster than getting an answer from support. Most of the support is outsourced and they're not developers. They take product manual and will instruct things like did you reboot it or did you kill the process and reboot. We pay for these kind of nonsense trouble shooting.

</comment><comment><USER3> Non-commercial (CentOS) for all my development environments. Don't need to worry about licensing fees -- simply spin-up another VMWare instance.

Commercial (Red Hat) for production. Corporate clients want commercial support.</comment>


Whatever Happened to BeOS? - <AUTHOR> I love to tinker with operating systems -- perhaps too much. I was just thinking about BeOS and the non-Linux replacement, Haiku, which is still in Alpha 4.
What went wrong with BeOS? Was it hardware, software? No one cared because of Linux and BSD?
BeOS was original, fast, and elegant. In my opinion, it should have been the next OS X instead of the remnants of Next. <comment><USER2> When BeOS was becoming interesting, it was competing against MacOS 8 and Linux (assuming you had already ruled out Windows and you couldn't/wouldn't pay for a UNIX workstation).

MacOS had commercial apps and a good reputation in certain fields. The hardware was expensive by PC standards but high quality.

Linux was growing like a weed: adding just a little more RAM to the kind of desktop a college student could afford let you do everything that the Sun workstations in the computer lab could do, and you never had to share it.

If BeOS had been free, compatible with less-blessed hardware, and not offended Apple so much, it might have had a chance.

<comment><USER3> For Apple, BeOS would have been a disastrous choice and NeXT knew this. BeOS and its public APIs were built on C++ which is terrible for binary (ABI) compatibility, which is a critical feature to convince 3rd party commercial (proprietary) developers to develop for your platform.

You can see that BeOS and later Haiku were stuck on an ancient version of gcc 2 because they could never upgrade without breaking all existing binary compatibility.

After NeXT did their not-so-secret reverse takeover of Apple, you can see they solved the binary compatibility problem for the most part by leveraging both C and Objective-C which both had a predictable/stable ABI, even with all the tinkering they did to extend the languages and eventually replace the compilers (Codewarrior->gcc->clang).

<comment><USER4> I've heard this story relayed in person in nearly identical language before. Do I know you? Either that or this was more in the zeitgeist than I think.

<comment><USER3> I think this information is common among NeXT old-timers and passed onto by them to early Cocoa developers where I learned it. It used to be a small community.

I disagree with the original post that implies BeOS was better than NeXT. There are a lot of really useful, cool things in NeXT that are not appreciated by the masses and forgotten with time. All the new iOS developers don't know how much history/heritage their stuff is built on.

For example, fat binaries are so incredibly useful and elegant. While Mac easily transitioned from PowerPC to Intel, then 32-bit to 64-bit, and now all all the different ARM architectures on iOS by using fat binaries, all the other platforms are a mess with just their 32-bit to 64-bit transition partly because the need to differentiate between two different architectures on the file system breaks most tool chains. (Not to mention that it confuses the hell out of users about which version they need to download.)

Linux had their chance with Icculus's FatELF patch and they rejected it. Only years later after great suffering is some of the Linux community starting to think maybe FatELF was a good idea after all. Too bad our 64-bit migration is almost over. But Android still really could benefit from this because that system is a mess that looks like it will need to deal with multiple architectures indefinitely.

<comment><USER4> This was definitely the story I got from a NeXT old-timer.

I also agree on Be. While I used BeOS (even got to play with it on a BeBox) and was impressed by it, NeXT was the better system. Oh my, networking was a mess in BeOS, and I'm not sure they would have ever gotten full multi-user support.

As a youngster, I pined for NeXT after playing with one at a university lab. I still marvel that we now carry NeXT workstations around I our pockets.

I really wish the Linux / BSD communities had rallied around GNUStep. Would have been nice to see how two implementations coevolved.

<comment><USER3> I remember when Gnome and KDE were first emerging. They were both too slow and wonky for me. Somebody suggested WindowMaker and I really liked it. At this time, I had no knowledge of NeXT or GNUStep. It was some years later after OS X emerged and I started learning Cocoa that I learned of all this history and discovered that WindowMaker was also connected to this universe and it really sunk in just how advanced they were for the time.

I'm still saddened that GNUStep never went anywhere. I still find GTK/Gnome/Qt/KDE wonky for a lot of basic things like copy-and-paste and drag- and-drop.

My wildest dreams hope to see a re-emergence of something like GNUStep, but this time be driven by all the new interest in Swift.

<comment><AUTHOR> Funny you should mention GNUStep/WindowMaker... absolutely my favourite WM. I agree most of the rest are bloated and wonky. Gnome has removed almost all of its menus and bells and whistles and it's largely unusable now. KDE is the kitchen sink inside the kitchen sink. It's overly complicated for a DE. Sadly, there is no real focus on WM now, although last year, Window Maker Live, a Debian-based live ISO was released. It hasn't gained a real foothold among aficionados.

There was an attempt to get Window Maker back in the limelight last year. Not sure where this has gone...

<comment><USER4> Yeah, I ran a poor-man's NeXT for a long time (WindowMaker on NetBSD in my case). The Etoile project seemed like they were going to try to make a full GNUStep-based desktop environment, but I fear they are suffering from too few coder hours: [http://etoileos.com](http://etoileos.com)

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER5> Partly they just didn't have enough money to build an OS, so every time they pivoted it put them further behind. And they pivoted a lot of times.

I think they made a lot of design decisions that were overfitted for 1995, so if they had survived they would have quite a bit of legacy cruft by now. And the brittleness of C++ may have made the cruft worse.

In some cases their idealism seems to have held back practical adoption. Their "pervasive multithreading" made it very difficult to port Java and Mozilla. (The idea of "only native apps, no ports" made perfect sense in the siloed PC market of 1993. By 1997, not so much.) Treating all developers equally meant that professional developers may have gotten shortchanged.

</comment><comment><USER6> I was a heavy BeOS user during its last days, and from what I remember, Microsoft put some serious pressure on certain OEMs to not sell BeOS on their computers. Compaq was one of those; they were told in no uncertain terms that if Compaq sold even one computer with BeOS on it, they'd never sell another computer with Windows, period.

That's certainly not the only reason the company went under, but it was a major factor.

</comment><comment><USER7> Like most companies of that era, they built their own hardware (BeBox) which was very expensive (compared to the PCs of the time). Later they moved to Apple hardware and Apple did try to buy them but they held out for more money.

Linux took hold because you could pull any old PC out of the junk pile and load it up and go. BSD, at the time, was fracturing into multiple versions and having legal troubles.

Haiku alpha 4 is awesome. But there are too many other projects out there so there aren't enough resources to make faster progress.

The wikipedia page [http://en.wikipedia.org/wiki/BeOS](http://en.wikipedia.org/wiki/BeOS) has a lot of more good information.

</comment><comment><USER8> I think the problem was the same as that faced by languages like D competing with C++11:

It wasn't "better enough" than the alternatives.

It was better than NeXT (what would become OSX), but not enough to be revolutionary. It was better than Windows, Mac Classic, and Linux, but also not enough to be revolutionary.

The existing closed incumbents won out by market share and inertia, and Linux won out by being free, open, and by basically going viral.

If BeOS had been open and free I think it would have given Linux a run for its money, but it wasn't.

</comment><comment><USER9> My bet: a lack of a killer app, plus no compatibility with killer apps on other platforms.

OS/2 was slightly more compatible with existing apps and therefore had slightly more success for a while.

<comment><USER6> That was a thorn in the side of many a BeOS user; while GoBe Productive was an amazing office suite, Microsoft Office had a stranglehold and if you didn't have Word and Excel, you didn't have a computer in many people's minds.

The world has changed in the past 13 years; we've had OpenOffice and LibreOffice for many years, and Google Docs/Drive has forced Microsoft to focus on cloud-based productivity apps. Back when BeOS was alive, it was just as common to have a dial up modem as it was DSL or cable, and sometimes broadband simply wasn't available in certain areas. Cloud computing as we know it now was still over the horizon.

In today's cloud-centric Internet, Haiku could easily thrive if it got enough momentum. Unfortunately the only people who seem to even know about it are old BeOS hats like myself.</comment></comment>


Ask HN: What MySQL IDE do you use? - <AUTHOR> MySQLWorkbench is nice, but crashes a lot and isn't very customizable (for example, there's no way to do light-font-on-dark-screen).
Sequel Pro is nicer in that regard, but it doesn't support multi-tab within the same connection and further "doesn't regard itself as a query editor."
There's no save-as selection on the menu. If you use the keyboard shortcut, it'll try to save over whatever you last saved, regardless of tab.<p>I'm happy to pay for something good, especially if it's cross-platform; is there something good that people use? <comment><USER2> Navicat [https://www.navicat.com/products/navicat-for- mysql](https://www.navicat.com/products/navicat-for-mysql) while expensive allows for saving of queries and forward and reverse model engineering

</comment><comment><USER3> I use the PhpStorm IDE which comes with it's own database tool which is more or less the same as the stand-alone DataGrip product/tool. It is customisable and have tabs +a lot of nifty features. Only thing I miss is 'migrate database from vendor to another vendor', but I guess that is too specialised for such a tool.

[https://www.jetbrains.com/datagrip/](https://www.jetbrains.com/datagrip/)

Cheers!

<comment><AUTHOR> Thanks! I'll definitely give it a shot.

</comment></comment><comment><USER4> HeidiSQL is actually pretty simple and easy to use. I used it quite a bit before switching to Postgres.

<comment><USER5> Supports postgres too, these days.</comment></comment>


OCaml Multicore Update: April 2020, with a preprint paper - <AUTHOR> https://discuss.ocaml.org/t/multicore-update-april-2020-with-a-preprint-paper/5630 <comment><AUTHOR> As with previous months, happy to answer any questions people might have.</comment>


Just posted SmartMenu 1.1: use to create keyword driven meta-information portals - <AUTHOR> http://braindance.com/ <comment><AUTHOR> We are a software startup based in Seattle striving to make your actionable information conveniently available no matter where it is located. Would love feedback on our latest version. Will send you a free license key to the Standard version $10 value to say thanks.</comment>


So You Want To Join A Startup? - <AUTHOR> http://blog.thirdyearmba.com/so-you-want-to-join-a-startup

<comment><USER2> For anyone thinking about joining a startup : be very, very careful before you accept a job offer. The main thing to consider is that you'll be working very closely with just a few people (from 2 to a dozen) for the foreseeable future. You'll have to really like these people as you'll be spending an awful lot of time with them.

When I was interviewing last, I was a lot more concerned with what the teams I met were producing than their culture, or how they interacted with each other. Long story short, I ended up working for a company with 10 employees, but I had never felt more alone in my entire life. Small things I took for granted at other jobs like eating lunch with people, sharing jokes, or any type of bonding felt near impossible.

I don't want to discourage anyone from working at a startup, but I feel it is very important that you consider the reality of working in that environment for a considerable amount of time.

<comment><USER3> I don't think this only applies to Startups, but in most jobs. There will always be niche groups and little clicks and you may not fall into them, or you might. Just with a larger company there is a larger pool of potential work-buddies.

</comment></comment><comment><USER4> Currently, I work for a company with 25k+ employees, but I know that down the road I would like to work for a startup. The author raised some good points that got me thinking about exactly what size startup I would like to be a part of.

The author's description of a startup that has "taken a decent-sized round" with about 15-30 employees (including senior engineers) who "will be less focus on getting things done NOW, and more on getting them done correctly" sounds great.

Is it typical for a startup of that size to offer competitive salaries? I would love to get my student loans paid off before I join any startup where I have to live in a closet with 3 other people and eat spaghetti every day.

<comment><USER2> I worked at a smaller startup recently and it was fairly competitive as far as jobs at other businesses in the area. I wasn't making as much as my friends at Apple, VMWare, or Google, but I made enough to live comfortably and had stock options that would have started me towards "F-U money" had they gone public.

However, that might have something to do with their business model performing very well, and the business being profitable earlier than expected. It all depends.

</comment></comment><comment><USER5> A really good read. I am currently a senior in college and working for a startup has been something I feel like I want to do after I graduate. This article doesn't really do much to scare me away from that, because I am sort of expecting that sort of thing out of working for a startup.

And I agree with conorgil145, the description that the author of this post gave about a company with 15-30 employees sounds like it would be a great work environment.

<comment><USER2> If you're a senior in college right now, the best advice I can give you is to take an idea you have (even if it's not your best) and try to execute it. Even if you don't ever launch, you'll learn a lot about starting a business and getting from the initial idea phase, to design, to launching, to (hopefully) making money. That sort of knowledge is invaluable.

I'm several years out of university myself and just now getting a few of my side projects to that point. It's exciting, but I can't help but worry that I'm behind the curve. Tinkering and design comes fairly naturally to me, but everything else that goes with entrepreneurship ... not so much.

<comment><USER6> > I'm several years out of university myself and just now getting a few of my > side projects to that point. It's exciting, but I can't help but worry that > I'm behind the curve.

There is no curve. Comparing yourself to others based on age and level of startup success isn't a good comparison. I used to do this to myself a lot, and it only causes pointless anxiety.

The only curve that matters is your curve.

</comment></comment></comment><comment><USER7> I think the leading edge of startups is finally paying real equity to the first 5-10 (i.e. pre series A, taking a very low salary) people.

It used to be you'd get 0.1-0.5% as an individual contributor engineer; it is going up to 1-5%+. Much more on par with what founders get. Obviously this varies a lot with the size of any seed raise, where the company is, who is being hired, etc., but it's a whole lot better than it was in the past.</comment>