Where are all the alpha geeks and what are they doing? - <AUTHOR> http://benjaminste.in/post/1223476561/hey-guys-whatcha-doing <comment><USER3> Being a geek / hacker at heart is not about using the latest whizbang gadget. It is also not about using the newest medium of communication or hanging out in the places tech journalists write about in exquisite detail. It is about mastering one field and using the experience gained from it to jumpstart into the next one. The problem is that it is often impossible to tell what will be the new thing of today that we will fondly look back on tomorrow.

Some may say today it is Twitter streams or Facebook walls or they may say it is the startup culture like we have here on HN but nobody knows. It could be something as benign as upvoting only thoughtful comments here on HN instead of funny ones like on reddit/fark/digg that could bring about a long-lasting effect on the future of open discussion on the Internet. We could be the guys that beat Eternal September while sharing our code and ideas with others. We don't have to phreak or use IRC to be considered true geeks. We could use the same tools as our moms (I'm using an iPad right now, I got one for my mom too) and still be doing very different things. She plays Farmville while I'm trying to create apps that help the disabled.

The fact that our tools have matured to the level of flexibility and stability that both hackers and their moms can use them is something to be proud of, no despair. After all, I use the same faucets as my mom and the same washing machines. 60 years ago the geeks made their own electronic devices. Oh and we still do that too. Check out the Arduino forums.

<comment><USER4> So Alpha geeks are failing fast and failing cheap (or failing expensive on somebody else's dime) whilst making stepping stones across the pond for others to follow (or see)?

<comment><USER5> I don't think the Alpha geeks are failing. I do think that computer literacy has increased as younger generations have grown up with technology and also, because a lot of smart alpha geeks have improved the usability of the technology (two forces meeting so to speak).

It is a little bit like the motorists of old - they were first really weird, then being good was largely regarded as being ultra cool as others battled with the new cars, then they were sponsored in races and now they still exist. They exist as kitset car builders, engineers, racers, and oh dear... boy racers. The alpha geeks may not use the latest technology (some Linux users manage to keep their machines going with their skills) but hasn't it always been the skills and knowledge of the geek? If that's the case, I think we are doing very well. The improvement in usability of the internet and easy availability of knowledge means the barrier to entry is lower which means more geeks. On the other side of the spectrum it also means more non-geeks know how to fix their computers etc. Rather like cars. However, being able to replace a battery doesn't make you a car enthusiast, nor does being a savvy Facebook user mean you could build it.

</comment></comment></comment><comment><USER6> We are all geeks now.

This is sort of like the disappearance of the "jet set" as air travel became a commodity. People no longer dressed up to go on a flight, jobs involving flying all over were no longer so glamorous, and socializing on a plane or in an airport wasn't as fascinating.

But the huge benefit is what happens when so many have access to the power only a few had before.

So now that "everyone" is a geek, being a geek is not that distinctive anymore. But as the technology is a lot closer to people's lives, the impact of the alpha geek has become much larger.

<comment><USER7> If internet access is enough to declare yourself a 'geek' then we're not there yet, only ~29% of the world now has internet access:

<http://www.internetworldstats.com/stats.htm>

If just having a cell phone counts then we're there at least for the majority of the population, apparently we now have 5 billion cell phone subscriptions.

<http://cellphones.org/blog/interesting-stats-on-cell-phones/>

<comment><USER8> Thats pretty useless - you have to remove the couple of billions of people from the pool who are too poor to feed themself, stuck in refugee camps or otherwise unable to live their life. It sounds heartless, but they don't really matter in any sense since they have to live life by the rules others make.

</comment></comment></comment><comment><USER9> The alpha geeks are into transforming old-world technologies into geek- friendly forms. The field I'm personally familiar with is desktop manufacturing. Robotics used to be expensive and arcane. Machinists used to be either old graybeards or undereducated trade school graduates. Custom plastic parts were something you got made in China at great delay and expense. Now, there is a new wave of machinists who read Make and have 3d printers and buy lathes for their garage. There's people with a software background putting robots together out of hardware store parts and arduinos. The whole field is transforming from high-capital professional only to geek and hobby friendly. So I think that geekdom has moved out of computing and software to some degree, into the rest of the world because frankly, computing for the sake of computing is a solved problem, a boring field, now.

<comment><USER10> Exactly.

> the rest of the world

This is what fascinates me. I want to know how to get my programs to reach out into the world and do something "real." That's been the most interesting thing to me for a while.

<comment><USER9> Grab an Arduino. Make it blink things. Then make it move things. Then make it move itself. Then make it interact with the things around it. Then your creativity's the limit.

</comment></comment></comment><comment><USER11> How can one say "where are the alpha geeks" and then say "please don't mention Linux on the desktop" in the same damn posting? Unix is where they all headed. That OS X has a GUI that you can _completely_ ignore while you're in the perfectly usable terminal with bash is a sexy proposition. Then again, there's always Fedora with GNOME.

The geeks that I knew first started with .mod files. Remember those? Those were terrible-sounding representations of music. The beat tracks were mixed with "instrument packs" that approximately recreated the melodies of songs. They were passed around usenet and on places like Undernet IRC. Then we all moved to .mp3 and the Napster beta. Then AudioGalaxy. Then eDonkey. Then gnutella, then bittorrent. All the while, alot of the geeks stuck with usenet.

While "everyone else" played Playstation, we were making quake mods and "total conversions". While the rest were chatting on AOL, we were IMing with ICQ. When the public zigged with Windows, we zagged with Linux. Now, there seems to be a strange convergence toward OS X. Maybe _that's_ what Benjamin is talking about? The convergence?

Here's what happened. Geek culture merged into mainstream culture. The geeks that used to be shut-ins and social outcasts are suddenly normal in the suburbs. Put them in some big scary city with a questionable populace that has a perhaps less-than-honorable stance on high intellect and literacy and they'll be the same outcasts they were before the Social Network was #1 at the box office. But for now, geeks are the EF Hutton of modern culture: They talk, the masses listen. And retweet. And Wall. And submit to reddit/HN/etc. And, when an alpha geek suddenly gets attention, I believe they stop being a geek.

Maybe the bible had it wrong: it's the _g_eek that shall inherit the earth. Perhaps we're at the tail end of the "hey, let's all be/act stupid as hell like we're rich heiresses or supreme-court appointed drunken presidents, derp" phase. We're now entering our "meritocracy or bust", Libertarian-ish hyper- intellectual Vulcan mind meld phase. The geeks stopped living in the basement and started living in the now. They know the future of business, hell _America's_ future, is in technology. The geeks he's talking about are now Mark Zuckerberg, are Paul Graham, are Elon Musk and Steve Chen and Notch. They're building. They're launching. And, they're monetizing. It's a great time to be a geek. So much open source software and open APIs to hook into. So many frameworks and languages to evangelize and adopt and exploit and forget. The world is gigantic breadboard from Radio Shack, and we're all Captain Crunches looking for the right frequency for our Blue Box.

Perhaps the alpha geek is post-"user". Now, they're (we're?) trying to be Creators.

<comment><USER12> I read the Linux disclaimer to mean that he doesn't want to hear

"Now + 1 will be the Year Of The Linux Desktop for the mainstream".

<comment><USER13> We've moved on anyway, this is the year of Linux on the Smartphone.

</comment></comment></comment><comment><USER14> The OP is asking what the alpha geeks are doing?

Hardware:

<http://thecrucible.org/kinetics-electronics> <http://www.kk.org/cooltools/archives/004214.php> <http://www.arduino.cc/> [http://shopping.netsuite.com/s.nl?c=ACCT88394&sc=2&c...](http://shopping.netsuite.com/s.nl?c=ACCT88394&sc=2&category=1&search=el%20wire)

Software:

Clojure, Haskell, Scala, Any of the NoSQL's, Startups of any variety

Go to the Maker's Faire. You'll see a swarm of R2D2 robots, people making scale miniature battleships that shoot 1/4" ball bearings at each other, robot battles, physical pong games that play like the original video game, etc...

I strongly disagree with the premise that alpha geeks are doing anything other than what alpha geeks have always done. We're trying to build stuff that makes our friends say, "That's awesome!".

<comment><USER15> Yea, I see the Maker movement as in the alpha geek phase, but it's certainly becoming mainstream. Now MS, Intel, TI, etc. are all embracing it with their own products. I guess we'll see where it goes.

</comment></comment><comment><USER16> _But then something strange happened. Lots of people switched to Apple laptops. But the nerds didn't move on. My nerdiest programmer friends use the same computer as my wife._

No? He circulates in a different universe to mine where it feels more like a 50/40/10 Mac/Lin/Win mix among the "nerdy programmer" group.

<comment><USER17> Pretty similar here, and rapidly edging closer to your ratios. My parents are Win, but my sister and her husband are Linux, my brother, my wife, and almost all of my geekier friends are Mac, and nearly every other geeky person I encounter with Windows is planning for their imminent escape.

</comment></comment><comment><USER18> It's important to note that during each "nerd phase" that the author cites there were also other technologies being developed that simply never exploded. That doesn't mean that nerds weren't toying with them.

That same concept applies today. There are tons and tons of startups out there with plenty of alpha geeks playing with each. Some will inevitably take off while others won't. Regardless, the alpha geeks will not serve as an indicator of what will go mainstream as the alpha geeks will play with everything.

 _> In those days, it was really easy to see where consumer technology was going. You could just look at the nerds and know that's what you'd probably look like in 2-5 years._

I heavily disagree with that statement. No one knew that the internet (or any other example the author cites) was going to be THAT big. Let's look to more modern times... No one in 2007 was watching the "nerds" play on Twitter and thinking to themselves: "that's what I'm going to be doing 2-5 years from now."

Ultimately, this post is indirectly asking what's going to be the next big thing. That's a question that only time - and not the nerds or alpha geeks - can answer.

<comment><USER19> I know geeks who use IRC and Usenet way more than the average person. So I'll probably be re-explaining this stuff to my brother-in-law all over again in a couple of years.

</comment></comment><comment><USER20> We've hit a bit of a plateau when it comes to gadgets. We have laptops, which fit in a backpack and are more than fast enough for most purposes. We have smartphones, which fit in a pocket and are a decent tradeoff of portability vs speed/features. And we have desktops, which can have more storage than laptops, but for most stuff they're not much faster.

Now which one of those three things can you build yourself?

That's why nerds/hackers are using the same things as everyone else. They simply don't have the resources to build or customize tightly-integrated devices like laptops and phones. So they buy what works best for them. Lately, a lot of the time macs work best for them.

I'm probably being overconfident, but I think there's a new frontier for nerds and hackers. Advances in microdisplays and embedded computing have made wearable computers a somewhat practical idea. And individuals can build them. And the costs are decreasing. A few people have already hacked their own together from video goggles (like the Myvu Crystal) and single-board computers like the BeagleBoard. The biggest problems with wearables are fashion and software. The fashion issues will probably be resolved as technology gets better and people become more used to them. Software for wearables has to be really really responsive, since the idea is to completely integrate the computer with your life. Smartphone software has a similar constraint, but not to the same degree.

I think as wearable UIs and software improve, people will see the advantages of them. It would be incredibly useful to have a local cache of Wikipedia, e-mail, personal notes, and other data sources and software. You could become the smartest person in the room, and that's just the tip of the iceberg. Lifelogging (don't you wish you had video of the first time you met your significant other or best friend?), augmented reality, etcetera etcetera.

</comment><comment><USER21> Why is it logical to presume all the geeks would/should switch to OSX/Apple?

1) it doesn't fit with classic geek- you pay a lot of extra money for shiny hardware. Classically, a geek did it on the cheap and 'hacked' something together.

2) I consider myself pretty geeky, and yet I run Windows exclusively now. (and my servers run Linux, and I agree with OSX for media computing) Why is this? Well, to start all the tools I care about run on Windows. CAD programs, Matlab (USED to work on OSX. Key word USED), development tools, and to some extent games.

<comment><USER21> I should mention I'm into hardware more than software. Unlike software dev, with hardware dev tools, Windows is the common denominator (and often the only supported platform period)

<comment><USER22> Yeah, I'll agree to that. For HW Dev, It's still easier to talk myself into buying a laptop with a docking station and a bunch of different ports than to buy a beautiful, clean, yet limited mac.

Thank god Virtual Machines feel fast enough and do usb i/o these days, I'd go nuts if I still had to dual-boot Windows and Linux.

</comment></comment></comment><comment><USER23> "So my question is, why haven't the nerds moved on?" + "Please don't mention Linux on the desktop. Seriously." ?

Let me paraphrase the post: "I have a random thought that is edgy but not well thought enough for me to permit reasonable counterexamples."

</comment><comment><USER24> The people that I think of as 'alpha-geeks', that is the ones for whom technology is their creative outlet have moved from doing things with the internet to doing things with the physical world. Whether building 2-d tables for homebrew lasercutters, enthusiastically joining the cult of arduino, or more mundane things like building houses they want to live in or learning to fly, the common theme seems to be getting physical. That's just my take.

</comment><comment><USER25> My "nerd" friends (I hate that term) are fiddling around with robots and hardware. Stuff like Arduino and mbed. I'm trying to spec out a DYI book scanner robot that can automatically scan all of my old books.

But frankly, I don't think that's very nerdy. My blue collar friends that welders and electricians build stuff like that all of the time.

</comment><comment><USER26> You are looking for something computer related, because it's the last change. The world is probably not going to evolve twice in the same way. That would be ungeek ;). Alpha geeks seems to be in space rocketship and biology today. May be this is where we will see the big changes in the future ?

<comment><USER27> My bet is on 3D printing, personally. Can you imagine how the world will change when _that_ hits the mainstream?

</comment></comment><comment><USER28> I think the issue is that things move faster these days which means the gap (in terms of time) between early adopters and mass market has shrunk.

Part of this is because of the internet, but it's also down to the fact that the changes are largely software rather than hardware and can be adopted faster and cheaper.

This means that where in the past alpha geeks would have adopted a behaviour and had it as their own for several years before the man or woman on the street, that period is now several months and the gap between something being identifiable as a geek trend and going mass market is so small that it's effectively disappeared.

This isn't true for everything but I think it accounts for why there are less solid non-mass market geek trends than there might have been.

</comment><comment><USER29> Sure the mac is pretty mainstream but I'm sure the hacker is using it completely differently to the mainstream he talks about.

</comment><comment><USER12> Perhaps the nerds will in fact take root and be the only ones still running a full "desktop" OS in the coming future.

I like iOS but I do not want my primary computer to go that simplified route.

<comment><USER3> Your main computer will not run iOS till you can build iOS from within iOS. When that is possible, iOS can be considered a full desktop. Till then, you will need an OS that can build iOS.

<comment><USER30> Most people's main computer won't be a computer, it'll be a games console and a Tivo-niche filler (maybe the same thing, it's Sony vs Microsoft vs Apple vs the cable and satellite companies, and a pox on all their houses) paired with a smartphone.

Why do most people need a computer? I can't remember the last time I _had_ to send a personal letter - all my bills are electronic and if someone really needs my signature they send a form. And my entire extended family does email.

<comment><USER31> _Most people's main computer won't be a computer, it'll be a games console and a Tivo-niche filler_

How long before that's just running on the TV, with the TV acting as a thin client?

For what most people do, why doesn't a HD TV, a smartphone, and a bluetooth keyboard suffice hardware-wise? Just put the PC in the TV and make it capable of nice 3D graphics.

<comment><USER32> The idea of computer-as-integration-point is really a Western one. In Japan you have your smartphone, your console, your camera, etc, and they all overlap more than Western counterparts on what would otherwise be done by a PC.

That's why Sony make great hardware but lousy PC software. Their core market in Japan doesn't need to plug their phones into their PCs to manage their music, they do it all on the device...

<comment><USER31> _In Japan you have your smartphone, your console, your camera, etc, and they all overlap more than Western counterparts on what would otherwise be done by a PC._

So, for reasons of user-habit, there's no single point of integration in Japan? I'd like to be able to use my smartphone to tell my TV-attached device to record show X or download its bittorrent, and have it all just work.

<comment><USER32> Just that the assumptions are different. Actually, the West is becoming more and more like Japan in this regard. In the past people started with a computer and built out from there. Now I know people with a smartphone and a games console and a Sky+ box and no actual "computer".

</comment></comment></comment></comment></comment></comment></comment><comment><USER33> To examine this I think we need to think about the stereotypes in modern terms and apply them to future trends.

Here are my top 3:

1\. Startups - Nerds these days are worshipped more-so for their mainstream accomplishments than their underworrld ones. It's cool and sexy to own your own startup in the nerd world and it's still for the elite. The best data point for this is the movie The Social Network. All around the world people are envious of Mark, who is portrayed as the ultimate alpha geek. Moving forward, startups are going to be further romanticized and every fanboy is going to want to try their own (more-so than the common trend already).

2\. Anime. It seems to me that a lot of Alpha Geeks like Anime and Manga. Over the summer Naruto was the #1 show on Hulu. This trend will definitely continue to grow. As a general trend, I'd bet on more mainstream adoption of Asian culture as a whole.

3\. Social Shopping. Sites like thinkgeek.com, woot.com, sparkfun.com are filled with Alpha Geeks tweaking with the latest gadget (Arudino, which is still primarily an Alpha Geek toy). This will slowly adapt to other more mainstream consumer companies as consumers start coming online.

</comment><comment><USER34> How does old money separates it from new money? I can't think of anything less nerdy/geeky than an Iphone or Iwhatever - everyone have them now. New money.

Being a consumer of new tech things and download fart apps does not make you a nerd.

Building a computer within MineCraft or doing something inherently difficult which seems like a waste of time to normal new-shiny-money-iPhone-people- without-creative-thinking. That's nerdy.

</comment><comment><USER35> 3D Printing, for example? Hm, I need to get my hands on one of those MakerBots...

</comment><comment><USER36> Transhumanism.

<comment><USER36> Also, nootropics / smart drugs / neurohacking.

<comment><USER32> For that 90s retro vibe.

<comment><USER36> That is, in fact, what the alpha nerds I know are doing that other people are not doing.

<comment><USER37> Hell yeah.

</comment></comment></comment></comment></comment><comment><USER38> Largely, this is what I think has happened; the interests of big business and corporations have aligned with the interests of nerds and geeks.

Geeks like cool technology because it's interesting .. whereas the other interested (commercial) parties like cool technology because it's enables them to increase their reach, power and profit.

Society has become more enamoured with technology because a shift was necessary for commercial enterprise to flourish. In my opinion, the fact that our families all use the technologies we were championing 15 years ago, is a result of commercial might.

Therefore, my bet is that the majority of 'alpha geeks' are all working happily for 'the man'. At the end of the day, most people who have a love for learning and technology just want interesting projects to work on - if they can be handsomely rewarded for doing just that, the majority probably wont say no.

</comment><comment><USER39> How do you define the alpha geek identity? Is it the guy who writes down a beautiful piece of algorithm in notepad or on a piece of paper, or is it the guy who knows everything about the latest cool things, but maybe never programmed? Is it a 'long-beard' or a tech journalist? Is it Donald Knuth or Michael Arrington?

As far as I am concerned I was always a geek in the first meaning of the word. When I was young we were competing with my friends to write the fastest polygon filler routine in ASM. I was never really a geek in the second meaning. I was mostly late with the newest gadgets, I don't use yet Facebook and I am not yet on Twitter. I did not care too much about what OS I use in case it was not really important.

</comment><comment><USER40> He notes at the end to ignore maker culture but why not consider that as possible to reach the masses? With things like the Arduino and 3D printers making hacking easier and easier, maybe non-nerds will start (return to) building things.

<comment><USER5> One thing to consider is that the usability and maturing of the mainstream tech has helped it's easy adoption by people who otherwise wouldn't be bothered. I'd say the same thing will happen with 3d printers. Possibly we will see an easy abstraction over the top of things like the arduino - <USER2> type gadgets which are customizable?

</comment></comment><comment><USER41> The biggest change is the improvement of devices, platforms, development tools and distribution. The bar for playing with new tech has lowered significantly, in effect shifting the line of "alpha geek" further and further into developer territory. It used to take a pretty determined geek mindset just to get things like mp3s working, whereas now it doesn't take much at all to bring some new idea to the consumer market in a reasonably usable form. In other words tech is no longer geeky in general.

</comment><comment><USER42> I'm a geek and I've always belonged to the late majority when it comes to new technology. Within my area of expertise (programming) I'm at the frontier, but outside of that area I see technology as something that should be useful.

Bought my first smart phone (an Android) two months ago, but have been using Linux as my sole OS for five years now.

And if I look at my nerd friends it seems to be the same thing. The nerds have the newest and coolest in a thin niche, and outside that niche they don't care.

</comment><comment><USER43> The computer revolution is over. Today's alpha geeks are hacking genes and cells, not silicon.

</comment><comment><USER44> Well, we grey-beard geeks love and use Mac OS X for many reasons, but the most important is that a very pleasant Emacs subset is hard-coded into the Cocoa text subsystem. ;-)</comment>


How are the 30 inch monitors working out for you? - <AUTHOR>

I'm wondering about getting a 30" monitor (or two even) for my coding setup. The additional horizontal space sounds great in theory, but I'm a bit afraid that the screen will be just too big and might for example require constant horizontal neck movements to view different parts of the screen. So, I'd like to hear from people who have it already - how's it working out for you? <comment><USER2> I have three monitors. One [1680x1050] 22" from 7 years ago, a cheap [1980x1080] 23" and a [2560x1440] 27".

The productivity you get from a second display is phenomenal. Programming is a lot faster and especially if you work with graphics and have fireworks next to Sublime Text and a Chrome window open at the same time. The workflow is quicker and if you buy a calibrated screen to work with you'll see beautiful colors and be able to see more pixels.

Get a screen that can tilt, rotate and swivel and you'll be able to turn it to portrait mode so you get more vertical screen estate which may be too big for a 30" screen to even look at.

If I'm not using the big screen for work it's amazing to watch movies in or to use it for gaming, you'd need a high-end graphics card to go with that too if you want to play it at native resolution. If you have three screens you can even benefit from eyefinity[1] and paired with a surround sound system you'll get a gaming experience that is parallel to none.

Really, I'd recommend two screens if you can spare the money, but one is already a big boost in productivity, and the second boost is much smaller.

[1]: [http://www.amd.com/us/products/technologies/amd-eyefinity- te...](http://www.amd.com/us/products/technologies/amd-eyefinity- technology/for-consumers/Pages/experience-eyefinity.aspx)

<comment><USER3> tl;dr: Your mileage may vary, but I find I work best on a small screen, light weight hardware, in a layered environment with semi-transparency.

I used to be a big proponent of "more screens = more efficiency." There was a time when I was developing/running/debugging on a rather large network of in situ oceanographic sensors when I had four monitors arranged in a 2x2 grid.

Since then I was forced into working on some small, low-res displays in some cramped/uncomfortable environments. There's nothing like being afraid of heights and debugging code at the top of a swaying hydraulic scaffolding to make you want lightweight, productive efficiency.

I migrated to using yakuake, and now iTerm2 in a quake-like setup, and I do most of my coding in vim. I set the terminal and vim to the solarized-dark color scheme, and set the background at 80% opacity. I find this allows me to focus on what I'm doing but still "peer" through to see reference material behind the terminal. Further, because the terminal rolls down from the top with the press of a hot-key, I find that transitions between browsing/etc and writing code are very fluid and the burden of a mental context switch is minimized. This setup allows me to be incredibly productive on a 13" rMBP.

I recently had the opportunity to switch back to a multimonitor setup. I found that I spent too much time worrying about how things were tiled to make appropriate use of the real-estate, and began to wonder why I ever craved four monitors. In the end I just went back to my normal quake-like terminal/vim usage pattern as I feel this helps me focus better, and I really like the feeling of not being tethered to my desk/chair.

</comment></comment><comment><USER4> I have 2 27" monitors (an iMac and an Apple display) next to each other, and it's fantastic for programming. One display has Xcode using up almost all of the screen; the other display has Safari showing documentation and other reference documents. What would make it ideal is a usable keyboard with a usable trackpad built into it, like the laptops do; moving my hand to use the separate trackpad is just as irritating as using a mouse.

I use a standing desk, and find that I am far enough away from the monitors that head movement is not an issue; moving the pointer can be but you can increase the tracking speed setting to make that a non-issue.

What I am considering switching to is using the display attached to my laptop. Xcode would be on the laptop screen and the display would be used for showing reference material.

<comment><USER5> "What would make it ideal is a usable keyboard with a usable trackpad built into it"

[http://www.amazon.com/MeshWe-Bluefin-Seamlessly-metallic- sil...](http://www.amazon.com/MeshWe-Bluefin-Seamlessly-metallic- silver/dp/B007QKLODO)

</comment></comment><comment><USER6> I've been using 27" monitors for years now. I find that I put my work in the middle and use the periphery as window storage for ancillary context windows. (Like tailing server logs and performance tickers that I peek at from time to time while developing.)

I also use a different virtual desktop for each set of tasks. (email/hn, development, research, …)

I do add a second monitor when debugging full screen applications, but find I don't use it otherwise.

An important variable here: My eyes have reached an age where they are no longer variable focus devices. My computer glasses are set to a fixed distance, so the center of the screen is in focus, but by the time I pivot over to the edge of my monitor, the focus is not sharp anymore.

</comment><comment><USER7> I use a 2560x1440 27" display. I very rarely use the entire width of the display, but find the size very useful, as I can effectively divide the display into four laptop-sized 1280x720 windows, or two tall 1280x1440 windows, or some combination of the two.

This provides a multitude of useful workspace configurations, especially if you use a read a lot of documentation on screen. A 1280x1440 window is sufficiently large for a full A4/letter page without needing to zoom or scroll.

I would be extremely reluctant to go back to a smaller display. 1920x1080 is neither fish nor fowl in my opinion, being too large for a single window in most applications but too small to usefully split.

</comment><comment><USER8> I recently got myself a 27" 2560 x 1440 monitor at home primarily for programming. At work, I have 2 24" 1920 x 1080 screens with a really nice monitor mount and a laptop stand, so I have 3 screens at the same time.

I have to say, that I right now, I'm preferring my setup at home. Not only is the screen beautiful, but the increased resolution allows me to have 2 vertical buffers of code in Emacs with a browser next to it. I sit far enough back that I don't have to use constant neck movements, but then again, I only have 1 monitor.

At work, I use a standing desk, and have the monitors mounted in a half- hexagon around me, so it's easier to just do slight neck movements.

</comment><comment><USER9> I have been using a single 30" monitor as my main display since 2005. I love it. Only problem is having to adjust whenever working on another system (also been using it with a standing desk since 2008).

I split the screen space up into conveniently sized tiles most of the time. Most often using only half the monitor width at most (like two 1280x1600 monitors side by side, but no bezel line in the middle). But occasionally using the full screen real estate for one program is really nice.

</comment><comment><USER10> What matters most in making use of such monitors is the depth of your workspace coupled with your eyesight. You want to place large monitors as far back from you as you can comfortably work with given your eyesight. If the monitors are placed far back enough, most of the monitor should be within your field of vision and require mostly eyescanning and not having to crane your neck.

<comment><AUTHOR> I think you might have nailed it. Unfortunately, my eyesight is less than perfect - for example currently my screens are closer than arms's length from my eyes (granted, I have pretty long arms). So, in my case, the bigger monitors would force me to do neck movements, while for people with really sharp eyes they could be great.

</comment></comment><comment><USER11> Careful it doesn't hurt your eyes. I have an LED IPS monitor and I have to use sunglasses when using it, even with brightness turned right down. I only use it for video editing, the rest of the time I'm using the laptop screen. It's a shame.

</comment><comment><USER12> I have a 30" next to a 24" but I find myself using the 24 more often. I adhere to the 80 columns per line rule so I don't really have problems with horizontal neck movements. Even at 24" there's plenty of room for side by side windows.

</comment><comment><USER13> I use a 27" monitor (iMac) and love it. I rarely move my head, but I do sit a bit far back and utilize zooming from time to time.</comment>


Heroku - Bamboo Routing Performance - <AUTHOR> https://blog.heroku.com/archives/2013/2/15/bamboo_routing_performance/ <comment><USER5> I don't understand why people think this is a great response. They know how their routing works, just say so. It can't be that hard to give a basic overview of it before they release a more comprehensive post.

As for the comment "Improving our documentation and website to accurately reflect our product". That is a very round about way of saying "our website indicates our service does things that it does not" which is a VERY bad thing. People are paying for this service based on what Heroku claims it does.

If the website has been inaccurate for years, that is false advertising and really a bigger problem than they are giving credit to.

If anything, I am more disappointed now that I have read this response, it has not appeased anything.

<comment><USER6> Documentation discrepancies happen. I've seen them with pretty much every platform I've worked on.

Just yesterday, I found a critical discrepancy between the ActionScript documentation and the actual behaviour of the ActionScript compiler, costing my team a day of work. (I tried to report the issue to Adobe, but the Adobe Bug Reporting System was down. Perhaps they need a Bug Reporting System for the Bug Reporting System.)

I think it's pretty heroic (yeah, pun) for Heroku to own their mistake, make the changes they've proposed, and accept the fire we've been pouring on them. They could have easily tried to weasel their way out of this, or attack the claims (Tesla/NYT comes to mind). Instead, they've accepted their own wrongdoing, and have pledged to make it right.

Who cares if the explanation comes today or tomorrow? Give them a few more hours to make sure their new round of technical claims are accurate, since such accuracy is exactly what's at issue.

<comment><USER5> In answer to "who cares if the explanation comes today or tomorrow", I care if the explanation comes today or tomorrow. I use Heroku and have hit scaling issues in the last few weeks very similar to this. More information on what is going on behind the scenes will help me immediately.

As for discrepancy in documentation, this is one of the most major parts of their infrastructure and directly relates to how well applications scale. To claim they have intelligent routing and then not having so, that is completely misleading and not just a minor documentation discrepancy. This isn't a tech document that got out of date, this is straight from their main "how it works" page... <http://www.heroku.com/how/scale>. Read the bit on routing.

<comment><USER7> Well, for marketing purposes, random doesn't sound as impressive as intelligent. It's a discrepancy but it does appear to be disclosed.

On the page you link to, it says: "Incoming web traffic is automatically routed to web dynos, with intelligent distribution of load instantly as you scale."

When you click on "Read more about routing..." it says: "Request distribution - The routing mesh uses a random selection algorithm for HTTP request load balancing across web processes." <https://devcenter.heroku.com/articles/http- routing>

<comment><USER8> It may sound more impressive, but it's simply wrong. It's not even ambiguous.

If they had said some thing like "with advanced algorithmic distribution of load instantly as you scale", its wishy-washy enough that its technically correct and those that need to know exactly how it does it will need to go and look at the docs.

As it is, intelligent distribution tells those that need to know that the distribution of load is based on intelligence gathered from the system, so they may not look farther. And it's simply not true.

</comment></comment></comment></comment></comment><comment><USER9> That's actually a pretty impressive response as far as it goes. Obviously there's no details at this point, but he absolutely takes responsibility, doesn't try to deflect or sugar coat it, and manages to find a tone that is both professional/serious, yet also down-to-earth and earnest. I guess the real impact will be how they go about "making it right" but in terms of a first response to the situation the tone is near perfect.

<comment><USER10> No this is not impressive. This is them fucking up and misleading customers for 3 years, enjoying a great reputation and now FINALLY getting called out for their BS. They're about to lose that great reputation that they've spent the past years building up, so of course they're in major crisis mode and doing everything they can to fix this.

<comment><USER9> Your response doesn't really seem to be directed at my comment. In your first sentence you're using "this" as if you're talking about the blog post which is what I praise in my comment(the tone of it), but then in your second sentence "this" refers to the routing issue raised by rapgenius, I wasn't commenting on that at all - I'm not a Heroku customer I was merely commenting on a GM's response to negative press.

<comment><USER10> Yeah, it's still not impressive. The only thing that I'd say is impressive in response to the situation would be:

"Hi, CEO of Heroku here.

Sorry. We've been misleading customers and only telling the truth when pressed-hard for years. We've created this financial model for all of our customers who have been overpaying on dynos because of our shitty routing and will be reimbursing them based on that.

We've also rolled out a second dyno-tier, called "dyno with non-shitty routing". It's 10x as expensive, but at least we're being honest about it. All current customers will enjoy our "dyno with non-shitty routing" for the price they're currently paying for the next 2 years. Enough time for them to migrate away, like any reasonable person would expect them to after this."

<comment><USER11> This is spot-on.

 _Of course_ they're going to post _something_ , and of course they'll make it sound as good as possible. But it's baffling how so many people applaud such meaningless damage control drivel time after time.

<comment><USER9> Considering how many people/companies fail so spectacularly at it, why would it be baffling? I also think you underestimate the difficulty in making things "sound as good as possible" that one quality is the basis of the entire marketing and political industries - and a large component of many others. You're basically saying "It's obvious - just be perfect!" - it is not that easy.

All that being said - I'm really not "applauding" Heroku or their actions (which are what matter) I'm waiting to hear what they'll say. In the mean time I thought their messaging (which matters much less) was good.

<comment><USER10> No worries antoko. We're not really banging up on you, just pissed off at heroku :) Their wording was good, better than usual when companies fuck up. But they're deeply embedded in the startup community so good PR with us is expected, betraying our trust like this, however is not.

</comment></comment></comment></comment></comment></comment></comment><comment><USER12> It's a good response in that they _are_ taking responsibility, but it is pretty obvious that they are reluctant to say anything about a fix. In my mind, "it's hard" isn't a valid excuse in this case, especially when there are relatively straightforward solutions that will solve this at a practical level. For example, you could imagine a naive form of intelligent routing that would work simply by keeping a counter per dyno:

\- request comes in and gets routed to the dyno with the lowest count. Inc the count.

\- response goes out. Dec the counter.

Since they control the flow both in and out, this requires at most a sorted collection of counters and would solve the problem at a "practical" level. Is it possible to still end up with one request that backs up another one or two? Sure. Is it likely? No. While this isn't as ideal as true intelligent routing, I think it's likely the best solution in a scenario where they have incomplete information about what a random process on a dyno can reliably handle (which is the case on the cedar stack).

Alternatively, they could just add some configuration that allows you to set the request density and then you _could_ bring intelligent routing back. The couple of milliseconds that lookup/comparison would take is far better than the scenario they're in now.

EDIT: I realized my comment could be read as though I'm suggesting this naive solution is "easy". At scale it certainly isn't, but I do believe it's possible and as _this_ is their business, that's not a valid reason to do what they are.

<comment><USER13> What if their inbound routing is hundreds of machines, each of which may get a request for any of their thousands of apps, spread across tens of thousands of web dynos?

Do you have a distributed sufficiently-consistent counter strategy that won't itself become a source of latency or bottlenecks or miscounts under traffic surges?

<comment><USER12> Atomic counters are pretty fast. Redis, for example, should be able to handle it without breaking a sweat: <http://redis.io/topics/benchmarks>

<comment><USER13> I doubt they want every inbound request to require:

• query remote redis for lowest-connection-count dyno(s) (from among potentially hundreds): 1 network roundtrip

• increment count at remote redis for chosen dyno: 1 network roundtrip (maybe can be coalesced with above?)

• when connection ends, decrement count at remote redis for chosen dyno: 1 network roundtrip

That's 2-3 extra roundtrips each inbound request, and new potential failure modes and bottlenecks around the redis instance(s). And the redis instance(s) might need retuning as operations scale and more state is needed.

Random routing lets a single loosely-consistent (perhaps distributed) table of 'up' dynos, with no other counter state, drive an arbitrarily large plant of simple, low-state routers.

<comment><USER14> This has all been solved previously. In Google Appengine the scheduler is aware of, for each instance:

* the type of instance it is

* the amount of memory currently being used

* the amount of CPU currently being used

* the last request time handled by that instance

It also tracks the profile of your application, and applies a scheduling algorithm based on what it has learned. For eg. the url /import may take 170MB and 800ms to run, on average, so it would schedule it with an instance that has more resources available.

It does all this _prior_ to the requests running.

You can find more docs on it here:

[https://developers.google.com/appengine/docs/adminconsole/in...](https://developers.google.com/appengine/docs/adminconsole/instances)

For eg.

> Each instance has its own queue for incoming requests. App Engine monitors > the number of requests waiting in each instance's queue. If App Engine > detects that queues for an application are getting too long due to increased > load, it automatically creates a new instance of the application to handle > that load

This is what it looks like from a user point of view:

<http://i.imgur.com/QFMXeT1.png>

Heroku essentially need to build all of that. The way it is solved is that the network roundtrips to poll the instances run in parallel to the scheduler. You don't do:

* accept request

* poll scheduler

* poll instance/dyno

* serve request

* update scheduler

* update instance/dyno

This all happens asynchronously. At most your data is 10ms out of date. It would also use a very lightweight UDP based protocol and would broadcast (and not round-trip, since you send the data frequently enough with a checksum that a single failure doesn't really matter, at worst it delays a request or two).

<comment><USER15> > It also tracks the profile of your application, and applies a scheduling > algorithm based on what it has learned. For eg. the url /import may take > 170MB and 800ms to run, on average, so it would schedule it with an instance > that has more resources available.

That is very awesome technology, but it something like that available for non- google people?

<comment><USER14> Expensive commercial appliances like the popular f5 big ip's can, and that is what a lot of large-scale websites use:

<http://www.f5.com/glossary/load-balancer/>

In terms of open source, HAProxy has layer 7 algorithms but they are much simpler:

[http://cbonte.github.com/haproxy- dconv/configuration-1.5.htm...](http://cbonte.github.com/haproxy- dconv/configuration-1.5.html#4-balance)

If you were inclined, you could write an algorithm to implement something similar in one of the open source routers.

</comment></comment></comment></comment></comment></comment></comment><comment><USER14> There is a perverse conflict with platform service providers - the worse your scheduler performs the more profitable your service will be.

You replace intelligent request scheduling with more hardware and instances, which you charge the user for.

How much investment is there in platform service providers towards developing better schedulers that would reduce the number of instances required to serve an application? That answer, in this case, is "not a lot"

The incentives between provider and user are not aligned, which is why I am more inclined to buy and manage at a layer lower with virtual machines.

Edit: AppEngine went through a similar issue. Here is an interesting response from an engineer on their team:

[https://groups.google.com/forum/#!msg/google- appengine/y-LnZ...](https://groups.google.com/forum/#!msg/google- appengine/y-LnZ2WYJ5Q/j_w13F4oSSkJ)

<comment><USER16> > There is a perverse conflict with platform service providers - the worse > your scheduler performs the more profitable your service will be.

I think the practical significance of this kind of incentives is overrated. The company I work for does outsourcing work, paid by hour. Do they have incentives to make me produce less so that their customers pay for more hours? Theoretically. Do they act on it? Hell, no - there is competition and customer satisfaction matters.

<comment><USER17> The business of government contracting shows these conflicts of interest are real and lead to Billions of $ of waste annually.

There is plenty of competition for government work, but there are many ways to game the system even in the rare truly open/fair competitive bids.

<comment><USER18> That's because competition for government work is not really based on customer satisfaction in the same way it is in other industries AFAIK.

</comment></comment></comment></comment><comment><USER19> Wow, I feel like Heroku is really dropping the ball here. Like, they are acting punch drunk or something. Basically all this says is "we hear you and we are sorry". They could have posted that a day ago. This still says nothing about what is wrong and what they are doing to fix it.

Also, I'm not sure at what point this is, but at some point around say $3-5k a month, (100+ dynos) you really should rethink using Heroku. At that point and higher, you really ought to know about your infrastructure enough to optimize for scale. The "just add more dynos" approach is stupid because adding more web fronts is often the lazy/expensive approach. Add a few queues or some smarter caching and you'll need fewer web servers. Throw in something like Varnish where you can and you need even fewer servers. Point being, at some point scaling is no longer "free", it takes work and Heroku isn't magic.

<comment><USER20> At $3-5k a month Heroku may as well start offering a consultancy service rather than hosting. Wanting unlimited scaling without needing local talent is a reasonable thing to want, but its unrealistic to expect if from one single platform.

<comment><USER21> A lot of their success stories at <http://success.heroku.com/> are sites you'd expect to be spending the $3-5k/month.

If their platform can't handle higher amounts of load, they really should indicate as such.

</comment></comment></comment><comment><USER22> This is a horribly inadequate response. Prices for hardware have dropped 30% over the last 3 years and heroku is admitting their performance has degraded by many orders of magnitude. It's completely unacceptable to simply say, "yeah there's a problem, we'll give you some metrics to understand it better."

Sure, it's great they responded. The response should be "you're right, we are fixing it and issue credits" for revenue gained from fraudulent claims about the performance of their product and a credibility straining bait-and-switch.

</comment><comment><USER23> Most people are going to come here and mention how they are not planning on fixing the problem.

Put it into context. Heroku made this change 3 years ago, and also has had no issues admitting the change to users. Their documentation has lagged far behind and I believe they will be more transparent in the future. This is an engineering decision they made a long time ago that happened to get a lot of PR in the past 24 hours. Until there is a business reason (losing customers), I don't see them "fixing" the problem.

<comment><USER24> I think this PR has already hurt Heroku & caused them to lose customers.

<comment><USER25> _The only thing worse than being talked about is not being talked about - <USER2>

You are almost investing in heroku by using their stack and tool chain, it isn't easy for well established customers to just up and move. This is probably a PR win for them, rather than a loss. Truth be told, it will be how they handle this in the coming months that will make them win/lose customers.

<comment><USER26> If you are locked in the trunk with any cloud solution , then you are a bad programmer/syst admin/whatever taking bad decisions, period.

You should be able to move your project infrastructure quickly from a service to another ,if you cant to that, well too bad when your infrastructure fails...

</comment></comment></comment></comment><comment><USER27> What the hell? It's good he owned up...I guess. But the response basically sounds like "yeah, we've been charging the same prices over the last few years for increasingly degraded performance and we would have continued to do so, but someone finally caught on so I guess we have to now do something about this, right?"

</comment><comment><USER28> I think this is really a fine response considering the pretty terrible way the original post was written and the community responded. The simulation was a bit of a stretch because the supposed number of servers you need to achieve "equivalent" performance is highly dependent on how slow your worst case performance is, and if your worst case isn't that bad the numbers look a lot better. Don't remember the precise math, but back when I studied random processes we studied this problem and the conclusion was that randomly routing requests is generally not that much worse than doing the intelligent thing, and doing the intelligent thing is nowhere near as trivial as Rapgenius and random HN posters would have you believe. Given generally well behaved requests he random solution should be maybe 2-3x worse but nothing near 50x worse.

And besides, I really don't see why someone who needs that many dynos is still on Heroku.

<comment><USER29> Rap Genius cofounder:

> The simulation was a bit of a stretch because the supposed number of servers > you need to achieve "equivalent" performance is highly dependent on how slow > your worst case performance is, and if your worst case isn't that bad the > numbers look a lot better

It's still pretty bad. Here's a graph of the relative performances of the different routing strategies when your response times are much better (50%: 50ms, 99%: 537ms, 99.9%: 898ms)

[http://s3.amazonaws.com/rapgenius/1360871196_routerstyles_fa...](http://s3.amazonaws.com/rapgenius/1360871196_routerstyles_fast.png)

See <http://rapgenius.com/1504222> for more

<comment><USER30> Disclaimer: I've never used heroku :)

if I understand the chart correctly, using unicorn with two workers gets you pretty close to intelligent routing with no intelligence. I imagine adding up to three or four would make things even better... I don't know about puma/thin etc where you can perhaps crank it even further without too much memory tax(?)

To me this seems like the easiest approach for everybody concerned. Heroku can keep using random routing without adding complexity, and most users will not get affected if they are able to split the workload on the dyno-level.

On a slight tangent: On the rails app I'm working on I'm trying to religiously offload anything that might block or take too long to a resque task. It's not always feasible, but I think it's a good common-sense approach to try to avoid bottlenecks.

</comment></comment></comment><comment><USER31> What I find incredibly irritating about this blog response by Heroku is that it took a very visible post on Hackernews for them to act and reconsider their way of doing business.

They saw the potential loss in customers, and then acted. What this means is that they never had in mind to provide the best support and product they could for their customers before this news broke out.

Sad.

</comment><comment><USER32> Credit for owning the scope of the problem (allowing serious discrepancies for 3 years), which is sure to cost them trust from the community. But the skeptic in me reminds me that it's likely there was no way out of admitting it.

What disheartens me is that the documentation discrepancy caused real, extremely substantial aggregate monetary impact on customers, yet there is no mention of refunds. Perhaps that will come, but in my opinion, anything short of that is just damage control.

This is a time excessively demonstrate integrity, for them to go above and beyond. It's in their interest not to just paper over the whole thing.

</comment><comment><USER33> It's so refreshing to see this kind of communication. I don't use Heroku, and don't know much about this specific issue, but they're responses to downtime and complaints have been so direct and BS-free that I'll definitely consider them when I need a PaaS.

</comment><comment><USER34> I feel like there is an answer for this, but why are two companies in the "YC family" at odds so publicly? If RapGenius is "starting beef" like is done in the music industry, I find it odd that it would happen with someone on their own "label".

Perhaps this is ignorance on my behalf of how companies who have already been sold (Heroku) fit into the picture, but some explanation would be appreciated.

<comment><USER35> Heroku has been owned by Salesforce.com since dec. 2010

</comment></comment><comment><USER36> Well, let's put it like this. Those of us who know our programming shit and aren't afraid of a little math know exactly what has being going on here and that this answer is pretty much BS (what else is he supposed to say? basically he makes minimal concessions given the facts).

</comment><comment><USER37> _Working closely with our customers to develop long-term solutions_

Of the five action items they listed, it seems that only the last of them is about actually solving the problem. I hope they are committed to it - <USER3> visibility of the problem can help, but I'd rather not have the problem in the first place.

<comment><USER38> Actually, that's the only one that smells like BS to me. The others have clear meaning and goals.

<comment><USER39> The other ones are things that are obvious and immediate responses to the problem on description that don't take any deep analysis of alternatives.

Long-term fixes actually do require deep analysis of alternatives (and even what the appropriate parameters are for a solution that will deal with customers problems while maintaining Heroku's scalability), and aren't something you can do much more than make vague references to off the cuff.

The key question on that point will be follow-through.

</comment></comment></comment><comment><USER40> Interesting -- he seems to be saying that they'll explain all about the problem, but not do anything about it.

<comment><USER13> That's not fair. What do you think "tools to understand and improve the performance of your apps" and "develop long-term solutions" from his bullets mean?

But, I'm surprised they didn't wait until the "in-depth technical review" was available to apologize. And the idea that they were informed of a problem "yesterday" doesn't quite match the impression RapGenius gave, that they'd been discussing this with Heroku support for a while.

<comment><USER41> I think RaphiePS's comment is fair.

"tools to understand and improve the performance of your apps" only commits them to updating their docs and tools to reflect how their system really works. It doesn't indicate any intention to fix the actual problem (the fact that requests can be routed to busy dynos), nor that they will make any kind of reimbursement to people who made business decisions based on incorrect docs.

"develop long-term solutions" doesn't really mean anything.

<comment><USER40> Yeah, that phrase is a bit suspect. It could easily mean "teaching people how to effectively deal with our dumb routing."

</comment></comment></comment></comment><comment><USER42> Can someone explain, to people who know nothing about scaling infrastructure, why routing to idle dynos is a hard problem?

<comment><USER13> It requires statefulness and decisionmaking at the routing layer, and that's another thing that adds overhead and can go wrong at scale. (For example, there may be no one place with knowledge of all in-process requests. Traffic surges may lead to an arbitrary growth of state in the routing layer, rather than at the dynos.)

There are probably some simple techniques whereby dynos can themselves approximate the throughput of routing-to-idle, while Heroku's load-balancers continue to route randomly. For example, if a 'busy' dyno could shed a request, simply throwing it back to get another random assignment, most jam- ups could be alleviated until most dynos are busy. (And even then, the load could be spread more evenly, even in the case of some long requests and unlucky randomization.) Heroku may just need to coach their customers in that direction.

<comment><USER43> I must be stupid, because surely it can't be _that_ hard to partition the routing groups?

For example, use a hashing algorithm that switches to 1 of N intelligent routers based on domain name.

If you pick the right algo you can pretty much add routers whenever you like.

(It would be nice to know what Heroku have tried so far, at the very least to drive off know-it-all blowhards like me.)

<comment><USER13> They could partition the routing, and maybe they do. But then (a) there's one extra hop mapping to the specialist routing group; and (b) it's still nice to have super-thin minimal-state routers, for example with just a list of up dynos updated once every few seconds, as opposed to live dyno load state updated thousands of times per second.

I too hope their full response givss more insight into their architecture... I have a couople of small projects at Herooku already and may use them. for several larger ones in the future.

<comment><USER43> > _there's one extra hop mapping_

I thought about mentioning this. Because 1 small hop is still less than random blowouts in response time.

You can even cheat by pushing the router IP into DNS. Hop eliminated.

> _it's still nice to have super-thin minimal-state routers_

I imagine Heroku's customers are not interested in what is nice for Heroku, they want Heroku to do the icky difficult stuff _for them_. That was the whole pitch.

Anyway, we're arguing about Star Wars vs Star Trek here because we have no earthly idea what they've tried.

<comment><USER13> _...pushing the router IP into DNS..._

Maybe, but they don't currently give each app its own IP, and might not want the complications of volatile IP reassignments, DNS TTLs, and so on. (Though, their current "CNAME-to-yourapp.herokuapp.com" recommendation would allow for this.)

 _...want Heroku to do the icky difficult stuff..._

Yes, but to a point. Customers also want Heroku to provide a simple model that allows scaling as easy as twisting a knob to deploy more dynos, or move to a higher-resourced plan. Customers accept some limitations to fit that model.

Maybe Heroku has a good reason for thin, fast, stateless routing -- and that works well for most customers, perhaps with some app adjustments. Then, coaxing customers to fit that model, rather than rely on any sort of 'smart' routing that would be overkill for most, is the right path.

We'll know a lot more when they post their "in-depth technical review" Friday.

</comment></comment></comment></comment></comment></comment><comment><USER44> (Wonders what this response would look like if Elon Musk was running Heroku.)

<comment><USER45> RapGenius wasn't deliberately making shit up to get page views.

</comment></comment><comment><USER35> So the issue only affects Bamboo? that's what it seems to be saying

<comment><USER46> It does _seem_ to be what they are saying but, unfortunately, no :(

Random request routing is also present on Cedar [1]. The difference is that, on Cedar, you can easily run multi-threaded or even multi-process apps (the latter being harder due to a 512mb memory limit) which can mitigate the problem, but does not solve it. Modifying your app so all of your requests are handled extremely quickly also mitigates the problem, but does not solve it.

Seems to me the obvious solution is to do these things (multi-threaded app server, serve only/mostly short requests) _and_ use at least a somewhat intelligent routing algorithm (perhaps 'least connections' would make sense).

[1] - [https://devcenter.heroku.com/articles/http- routing#request-d...](https://devcenter.heroku.com/articles/http- routing#request-distribution)

</comment></comment><comment><USER47> Maybe I'm missing something here, this response speaks specifically about Bamboo - do all new services now not run on Cedar?

</comment><comment><USER48> This is a great response, and I'll look forward to the follow-ups in the days to come. Kudos to the Heroku team. Bravo.

</comment><comment><USER49> Is it me not understanding disqus, or did Heroku's moderator just deleted my comment?

<comment><USER50> I think I saw your comment and that they must have deleted it. Apparently their idea of keeping it civil means keeping out links to the blog post that it was a response to.

</comment></comment><comment><USER51> It seems strange for me to read in Heroku's response how forthcoming they are to accept blame and responsibility for the "a degradation in performance over the past 3 years".

Yet they state their action plan to "fix" this issue is to update their DOCUMENTATION and no mention of fixing the DEGRADATION issues itself.

Just bizarre.

<comment><USER39> > Yet they state their action plan to "fix" this issue is to update their > DOCUMENTATION and no mention of fixing the DEGRADATION issues itself.

This is flat out untrue. The third bullet point in their action plan is to update their documentation, and the fifth is "Working closely with our customers to develop long-term solutions".

Updating the documentation to accurately reflect what the platform does is obviously critical to allow people to make decisions and manage applications on the platform as it is, so is an important and immediate part of the action plan.

Long-term fixes to the problem are also important, and are explicitly part of the action plan. Its clear that they haven't identified what those solutions are, but its not at all true that they haven't mentioned them as part of the action plan.

</comment></comment><comment><USER52> I'm very curious to see what the technical review turns up tomorrow.

This feels like something that would have been connected to the Salesforce acquisition 3 years ago, and then making the service less efficient in order to increase profits or revenue targets on paid accounts. Not to mention saving money on the free ones.

It would be a little bit like Tesla not only selling you the Model S, but also selling you the electricity you charge the vehicle with. At some point, they make the car less efficient, forcing you to charge more often, and then claiming they didn't document this very well. Frankly, there are only so many people who will be a capable enough electrical engineer (or in Heroku's case, a sysadmin) to catch the difference and measure it.

The apology should be, "we misled you, and betrayed your trust. Here's how we're planning on resolving that, and working to rebuild our relationship with our customers over the next year. [Insert specific, sweeping measures...]

</comment><comment><USER53> Seems to me like a classy response to a real problem from Heroku.

We all need to remember that there are no magic bullets. The fact that Heroku can get a startup to, say, 5M uniques per day by dragging some sliders on a web panel and running up a bill on a corporate AMEX is pretty impressive.

At some point scaling a web business becomes a core competency and one needs to deal with it. I'm guessing by the time scaling an app on Heroku becomes an issue, if better understanding your scaling needs and handling them directly isn't going to save you a TON of money, your business model is probably broken.

</comment><comment><USER29> Rap Genius cofounder:

Our response: [http://rapgenius.com/Oren-teich-bamboo-routing- performance-l...](http://rapgenius.com/Oren-teich-bamboo-routing-performance- lyrics)

</comment><comment><USER54> So do the issues in the RapGenius post only affect those on the Bamboo stack? I'm procrastinating migrating to Cedar now but this could be a very good reason.

Also, I really love seeing a company take responsibility like this. I know the situations (and the stakes) are not comparable but this is a lot better than what Musk did when Tesla got a bad review. As a company just take the blame and say you can and will fix it, that's good enough for most people.

</comment><comment><USER24> Honest question, why would Rapgenuis still be on Heroku if the y needed 100 dynos? Why not go directly to AWS at that scale? The cost savings would be pretty significant. Am I missing something?

<comment><USER12> Ops guys cost a lot more than just using Heroku, not to mention the cost of simply having the responsibility of servers (even if they are virtual). Never underestimate the value of just not having to think about something, especially when you're small group of people.

<comment><USER55> I think the amount of time and energy they've invested in studying Heroku's routing and queueing strategy counts as having to think about something.

</comment></comment></comment><comment><USER56> Did they just say that they have no plans to return to intelligent routing, just making naive routing more visible to you?

</comment><comment><USER57> Bamboo routing? Is Cedar not affected?

<comment><USER58> I think Cedar is just not affected _as badly_ because it will route > 1 request at a time to a dyno, which helps if you're using something like Unicorn.

</comment></comment><comment><USER59> At this point they haven't really done anything. I'm really curious to see what they come up with.

</comment><comment><USER60> Wait, so those guys were on Bamboo, and complaining? Fuck, that is so not cool.

We've been on cedar ever since it launched, and been running puma threads or unicorn workers. The idea of one dyno per request is bullshit, and I wasn't sure if they were on cedar or not. A dyno is an allocated resource (512mb, not counting db, k/v store etc)

How ballsy of them to complain when they are doing it wrong.

<comment><USER61> > We've been on cedar ever since it launched, and been running puma threads or > unicorn workers. The idea of one dyno per request is bullshit, and I wasn't > sure if they were on cedar or not. A dyno is an allocated resource (512mb, > not counting db, k/v store etc)

It doesn't matter if you think one dyno per request is "bullshit" or not, Rails isn't multithreaded, so what do you propose they do? Using unicorn_rails on Cedar lets you fork off a few subprocesses to handle more requests on the dyno queue which gets you a constant factor bite at the dyno queue lengths, a few weeks or months of scale at best - it's not a real solution.

Heroku knows that Rails on Cedar is _just_ as affected by their inability to route requests and they're only not copping to it in this blog post because they don't have a customer running Cedar complaining so loudly. Which is cowardly.

> How ballsy of them to complain when they are doing it wrong.

If you mean that deploying a rails app to Heroku is doing it wrong - <USER4> sentiment many are agreeing with right now - then yes, you're correct!

<comment><USER62> If you pay attention to queueing theory, you'd know that even a modest amount of parallelism per worker will let you run much closer to capacity while still having very few bad request pileups.

Another way to put that is that using Cedar lets you get acceptable end user performance with far fewer dynos.</comment></comment></comment>


Quitting your day job to make video games. - <AUTHOR>

I recently quit my day job working for a major search engine company, in order to pursue my passion for making video games full-time (http://monkeywrenchgames.com).
We're targeting the iPhone, and hoping that we didn't miss the gold rush.
Has anyone else taken a similar plunge? What have your experiences been? I'd also be happy to answer ANY questions you might have about getting started on the iPhone. <comment><AUTHOR> Thanks all!

Yeah, we're learning that the App Store alone is not enough marketing -- I'm amazed at how many games are released every day.

We've reached out to app review sites (avoiding sites that charge $$ for reviews), which seem to be the lowest hanging fruit (if your game gets good reviews!). Facebook and Twitter are all the rage these days, and we're trying these as well.

However, by far, the most important thing that we did was release a free "Lite" version. Our best single-day sales had been about 75 copies, until our demo version was released yesterday and had over 1300 downloads in a single day, and drove up our sales. I highly recommend this approach.

Funny story about trial versions -- don't call your free version a "DEMO" -- we were rejected by the Apple store last week ("DEMO" phrase is bad, "Lite/Free" is good), and had to wait 7 days for our renamed "Lite" version to be approved and put up for sale.

</comment><comment><USER2> Good luck, games are fun, but it's a very tough business since it's so hit- based. I'd recommend working on your marketing strategy, particularly if you're working on iPhone games.

Repeat after me: The App store is NOT a marketing strategy.

Create a webpage, do advertisements, etc. Good luck, and keep us informed!

</comment><comment><USER3> I presume you're asking about quitting your day job in order to become an indie game developer. As it is, making videogames _is_ my day job, as it is with a number of other people who post here, but in my case, I'm currently not working for a small independent company.

Having not done the indie game entrepreneurship thing myself, I can't speak with any certainty, but I know enough about the business to know that some of what works for web startups won't work for game companies, and yet this industry has a lot to learn from the cutting edge of how software gets made in other sectors. I'd be interested to hear if you've got any thoughts about all this that might be interesting to the HN crowd.

I'm happy to say I've heard of Kids vs. Zombies, though I don't remember in what context (I don't own an iPhone, so I don't keep up much with games there). Given that indie games seem to be a growing segment of the industry (or at least an increasingly coherent cultural force and identifiable subsection within the industry), I hope you're doing your best to get indie and casual game press on your side. A lot of free publicity can be gained that way, and can really help word-of-mouth advertising.

Best of luck!

</comment><comment><USER4> I'm also interested in doing that, but with windows/mac/linux & possibly browser games.

The problem is that people seem to be somewhat averse to paying for a browser based game (like a Java applet or webstart app), even when they'd have no problem paying for the same game as a standalone program (literally simply using a JPanel instead of a JApplet, or providing an installer to a local version instead of a webstart link).

Of course browser games still get a lot more hits than locally installed ones.

<comment><USER5> I run a moderately popular browser game. We have over 175k registered users and get millions of requests a day.

Our conversion rates are low, but we've managed to get enough people playing to sustain itself, one full time employee (me!) and a few contract employees.

It's been a lot of fun. If you are passionate about games I highly recommend it since it can definitely be sustainable. Having said that, I'm not sure it's necessary a lucrative path for most people.

<comment><USER4> Cool! Care to provide a link?

<comment><USER5> Oh I thought the HN standard way was to provide a link in your profile instead of spamming up comments.

The site is <http://www.forumwarz.com>

<comment><USER6> When I saw that Mafia Wars was one of the most popular games on Facebook, I went to check it out, and before long thought "Wow, this is just like forumwarz except with mafia." Apparently html rpgs are now a viable genre.

Have you ever considered making more games that are essentially gameplay clones of forumwarz but with different themes? That seems to be a lot of what Zynga has done with their facebook games - Mafia Wars, Vampire Wars, Zombie Wars, etc. Also have you considered social network integration yourself. I hate to suggest such un-innovative strategies but it might work.

<comment><USER7> Zynga has one trick, that they don't always do well. I would suggest NOT emulating them.

</comment></comment></comment></comment></comment></comment><comment><USER8> I left my day job at a major search engine company a few months back. I was wondering if we are colleagues? :) Drop me a mail?

</comment><comment><USER6> Good luck!

</comment><comment><USER9> Just downloaded your lite version. Well done.</comment>


How many of you don't use Google CS Curriculum search? - <AUTHOR> http://code.google.com/edu/curriculumsearch/ I was surprised when I met several CS grads  who were not using this, because they hadn't heard about it! <comment><USER2> Interesting.

The first 4 results on the standard SERP for "trie" are Wikipedia, a pretty decent BU article on Tries with Sedgewick-style source code, an intro-to-tries article, and an article about using Tries for spellchecking.

The first 4 results on the CS SERP are a Princeton CS PPT on tries, Java source code for a trie, a paper on inductive logic programming using tries, and a tech report on "Nye's Trie" which looks like an interesting distributed systems solution modeled on tries.

The results for the two searches for "red-black tree" are approximately the same; all tutorial material.

The results for "bloom filter" are like those for "trie": the standard SERP is all tutorial material, and the CS SERP is mostly research.

CS search looks like an interesting middle ground between Scholar (which I use all the time) and standard Google. Thanks!

</comment><comment><USER3> Bit of a link-bait title, this along with various other of the labs search projects are interesting but hard to find.

I wonder if they build separate data sets for these lab searches or if it is just a special tag that could also be passed to your main Google search box like labfilter:csci or something?

<comment><AUTHOR> I agree it does look like link-baiting-after all this is a link to Google's own site.

But since there are two places to search- Code, and curriculum,PERHAPS Curriculum search is somewhat overlooked. I didnt know about filter:csci before.. I often dont type in 'filter' in my searches. Also, if this was very widely used, I would have expected to see some mashups,competitors- which I didnt. Anyways, if there is even one CS grad who got to know about CS Curriculum search through this post, I think it will increase my real life karma tremendously :-) :-)

<comment><USER4> If you agree that the title is link-baity and it's your title why don't you change it? (Or for that matter why would you submit it with a link-bait title to begin with?)

A better title would be something along the lines of Google CS Curriculum Search.

<comment><AUTHOR> What is really interesting is that, Curriculum search is not new- I have been using it for around 2 years. I just happened to discover from course of conversations, with some of my friends, that they haven't heard about google curriculum search - and I was really surprised. It is a great way to look up tutorials, course materials. If you notice the poll results for Curriculum search (cant draw a definitive conculusion from this sample,still), it seems to be terribly skewed towards people who haven't heard at all about Curriculum search- for a CS-centric community like Hacker news, that's very surprising.

Another thing to think about is that- proper vertical search for course contents of top universities around the world, would go about a long way in providing quality "self-education" for students all around the world.

Look at khanacademy- when one guy making educational videos on algebra and arithmatic,is making such a huge impact - here we have a good search facility of top cs curriculum /course material of the best universities all around the world, lying idle?

I dont want to sound patronising- but just think- what an incredible feature is this in the hands a CS curious hacker in Africa, or for that matter any 'third-world place' -all he/she needs is internet connection! That's it!

I really feel sad that something which ought to be very,very popular, probably is gathering dust at Google.

<comment><USER4> I completely agree with you that Curriculum is an interesting, useful, and underutilized feature. However that is completely unrelated to what I said.

Again, why do you make a submission with a title that you openly admit is link-baity, unless you are trying to artificially increase your karma? I don't know how long you have read HN but your account is less than a month old and intentionally submitting link-bait (especially by a relativity new user) reeks of karma whoring and really pisses me (and I assume/hope a number of other users) off.

</comment></comment></comment></comment></comment><comment><USER5> Based on the poll over here : <http://news.ycombinator.com/item?id=1756193> : the answer is "Almost Everyone."

</comment><comment><USER6> The comments on this page seem surprisingly negative. I for one just had a great experience using that service. I've been looking on and off for months for a course on Abstract Algebra (with lectures).

So, I type in "abstract algebra" into the search and here's what I get: [http://code.google.com/edu/curriculumsearch/results.html?cx=...](http://code.google.com/edu/curriculumsearch/results.html?cx=009384408222384877262:p27tltxalv0&q=abstract+algebra&sa=Search&client=google- coop-np&cof=FORID:9;CX:Curriculum) Perfect.

So this search engine passed my initial test with flying colors.

</comment><comment><USER7> The stuff on Algorithms is interesting, but not very deep for anyone who knows the basics and is trying to expand their understanding. But I think these are meant to be general lessons rather than specific.

<comment><AUTHOR> I'm guessing you saw only the course listed on the left- not the curriculum search itself. If you use the search, and see lecture notes, assignments,ppts on various topics, you might come across things you didnt know before.

</comment></comment><comment><USER8> Well, I _didn't_. But now that I know about it...

</comment><comment><USER9> You should really ask that question on Digg, Ask Yahoo, and maybe the Bing support forum.

</comment><comment><USER10> Having recently gone through Google's interview process, I think this site is useful getting up to speed for that, since Google is looking more for Computer Scientists than Computer Programmers.

</comment><comment><USER11> Didn't know about this until NOW. The traditional Google knows what I want better than the Curriculum search. Also, a lot of students have enough to study from their lecture material -- what they would want are snippets to specific topics which they find difficult to understand (Again, Google does it pretty well too). In short, even Google can't beat Google.

</comment><comment><USER5> I've started a poll on this: <http://news.ycombinator.com/item?id=1756193>

Only 4 votes so far, but 75% have never heard of it.

ADDED IN EDIT: And now over 87% have never heard of it.

</comment><comment><USER12> I don't.

I might if I ever need any info on algorithms in the future, but I'd wager that there are quite a few professional programmers on here who, not being in college anymore, don't really need to read academic papers.

</comment><comment><USER13> I was a little startled when results didn't appear as soon as I started typing. I'm actually used to instant search now. When did _that_ happen?</comment>


Android phones from OnePlus, Samsung, others reportedly reboot when dialing 911 - <AUTHOR> https://www.reddit.com/r/Android/comments/6nz3fx/psa_dialling_911_may_reboot_your_oneplus_5_xpost/ <comment><AUTHOR> As far as I've seen, users in comments report seeing similar issues on OnePlus 5, Nexus 6P, Samsung Galaxy S7 Edge, slightly different behavior on MotoG5+.

There's a second thread specific to the MotoG5+ report [[https://www.reddit.com/r/Android/comments/660jsl/crosspost_m...](https://www.reddit.com/r/Android/comments/660jsl/crosspost_motog5_cant_call_911_call_ended/)]

EDIT: apparently this is related to their non-stock dialers or something. People in /r/oneplus are saying to install and use the stock android dialer</comment>


Show HN:  Leanthingy.com - Rate Startup Ideas - <AUTHOR>

Lean Thingy - Helps you get fast feedback about your startup idea.
I hope this tool can help you determine whether to pursue or kill a startup idea quickly.
I started building it after a few startup idea posts on HN with 0 replies.  I think of it as a hot or not for startup ideas.
I started on this two days ago - it's not 100% complete, but I wanted to start getting feedback as soon as possible. <comment><USER2> I Love the idea (it does sound familiar..), just a couple nitpicks:

1\. Please don't make me sign in with Twitter. Not everyone has a twitter account and not everyone wants to give you access to theirs.

2\. I'd like to be able to rate ideas right from the leaderboard. If i'm just browsing and i see an idea i think is excellent i'd want to be able to tell them that.

3\. The design struck me as a generic blog design. I'm not sure exactly what it was, but it looked like a generic wordpress blog at first glance.

Other than that, i like it. Good luck!

<comment><AUTHOR> Thanks for the comments.

Reason for Twitter: Wanted to get this up fast - Faster/Easier to confirm twitter account than build confirmation emails.

Leaderboard - Great idea - I'll add that in tonight.

Design - yes - in an effort to get this up quickly - I used a free html/css template I found online. It looks like crap.

This is a test to see if the idea is viable. If it is, I'll put more effort into making it look nice.

</comment></comment><comment><USER3> Cool. Inspired by the same reasons as you, I too was thinking of creating something like this but you've just beaten me to it. :-)

Now, my 0.02:

(There will be some duplication [read reconfirmation] of suggestions others have already made here and some new suggestions.)

1\. Yes, don't force Twitter as the only sign-in option. (If you want, we can contribute a plug-n-play type e-mail authentication sub-system to save you the time and trouble - let me know.)

2\. Create a separate (exclusive) page for each idea and allow people to post comments/replies.

3\. Allow posters to (optionally) upload images/sketches to explain the idea better, faster. (Show such material on the exclusive page.)

4\. Many idea posters would also be having a thread on the idea running at HN and/or other forums. Allow posters to (optionally) provide link/s to such URLs (as a separate field). This way, everybody can get an expanded view of all opinions about the idea.

-HTH

</comment><comment><USER4> Great idea, few things as well.

1\. Ability to vote from leader board

2\. Only allow people to vote once (cookies/username)

3\. Browse though a full list of ideas

4\. Show comments for ideas.

</comment><comment><AUTHOR> Added voting on the leaderboard - thanks for the suggestion awfulcoder14 and<USER4>
</comment><comment><USER5> I just posted our concept on your site: <http://leanthingy.com/idea- spotbankscom>

</comment><comment><USER2> clickable: <http://leanthingy.com/>

</comment><comment><USER4> I think the biggest issue is duplicate voting.</comment>