Ask HN: Advice for finding housing in SF/the Valley? - <AUTHOR>

To those who've done this recently - where did you start looking?
What helped you a great deal, and what didn't?
Also, where did you end up and what do you pay? <comment><USER2> Procedure was straightforward.

First, decide on the minimum accommodation you'll be able to live with (shared bedroom, own room, own room with own bathroom, studio, 1br, 2br, pets, no pets, etc.).

Second, determine the hourly value of your time. If you are salaried, divide the number by 2000 to get it.

Rank all accommodations you can find (Craigslist has a nice map feature) by monthly rent plus monthly travel time times your hourly rate. If you're earning minimum wage, maybe commuting in from Antioch isn't such a bad deal. If you're earning six figures, two towns over might still be too far.

I worked in Redwood City and lived in Redwood City. 1br's now go for $1800 and up out that way. I could've saved a few hundred a month by living across the bay, but the time lost couldn't come close to justifying it.

Getting "a deal" is nearly impossible if you're new to the area; even people with networks of contacts end up paying far more than their neighbors who moved in five years ago.

Now if you get to telecommute at least partially, this advice may not apply.

</comment><comment><USER3> When I lived in Van Nuys/ Sherman Oaks I sort of wished I lived in Burbank or NoHo

but ask these guys. Its how I found my job when I first got there

[http://www.reddit.com/r/losangeles](http://www.reddit.com/r/losangeles)

<comment><USER4> Wrong area of California</comment></comment>


New Macbook Air with SSD Available Now  - <AUTHOR> http://www.engadget.com/2010/10/20/apples-new-macbook-air/

<comment><USER2> $999 for a netbook?</comment>


Classified Cable Proves US Ok'd Saddam's Kuwait Invasion - <AUTHOR> http://www.veteranstoday.com/2011/01/31/cong-ron-paul-classified-cable-proves-us-okd-saddams-kuwait-invasion/ <comment><USER2> I read the whole cable. It doesn't support Ron Paul's interpretation or the submission headline.

The "NO POSITION" comment is about the particulars of a specific border dispute; it doesn't mean no position on war/invasion. (The U.S. probably has no position on the Canada/Denmark dispute over Hans Island. It doesn't mean the U.S. would greenlight either country resolving the dispute by force.)

Also, the summary at the top of the cable states: "AMBASSADOR MADE CLEAR THAT WE CAN NEVER EXCUSE SETTLEMENT OF DISPUTES BY OTHER THAN PEACEFUL MEANS."

Also, this "TOOK NO POSITION ON THESE ARAB AFFAIRS" section has been leaked, and discussed, years ago, and the whole memo was declassified more than 2 years ago. See for example: [http://www.washingtonpost.com/wp- srv/politics/documents/glas...](http://www.washingtonpost.com/wp- srv/politics/documents/glaspie1-13.pdf)

And finally: not hacker news. Flagged.

</comment><comment><USER3> The relevant section seems to be

NOTE: ON THE BORDER QUESTION, SADDAM REFERRED TO THE 1961 AGREEMENT AND A "LINE OF PATROL" IT HAD ESTABLISHED. THE KUWAITIS, HE SAID, HAD TOLD MUBARAK IRAQ WAS 20 KILOMETERS "IN FRONT" OF THIS LINE. THE AMBASSADOR SAID THAT SHE HAD SERVED IN KUWAIT 20 YEARS BEFORE; THEN, AS NOW, WE TOOK NO POSITION ON THESE ARAB AFFAIRS.

Which I suppose can be read as the US taking no position on Iraq-Kuwait affairs at all, but given the context it seems to me to refer only to the statement by the Kuwaitis to Mubarak that Iraqi troops were too close to the border. If Saddam is asserting this is a lie, then the US response may only refer to the US not taking a position on these statements.

Or, to conclude, I would really like to hear confirmation or refutation by someone more familiar with the history or with experience in diplomatic matters.</comment>


How Women Got Crowded Out of the Computing Revolution - <AUTHOR> https://www.bloomberg.com/view/articles/2017-08-19/how-women-got-crowded-out-of-the-computing-revolution <comment><AUTHOR> Even if the author's premise that women were "crowded out" of programming 40 years ago is true, that does not imply that "innate dispositional differences" between sexes don't influence one's tendency towards pursuing certain fields.

The author suggests that the current stereotypical portrait of a geeky socially awkward neckbearded male engineer was socially engineered, causing a self-fulfilling prophecy. That's idiotic conspiracy nonsense. Geeky reclusive men tend to pursue computer programming because dealing with a machine doesn't involve dealing with a human, not due to society's marketing of computer engineers as geeks.

The author is clearly not a computer programmer (professor of history apparently), nor someone capable of basic logic. If you're going to cash in on the diversity scandal for views, at least make a coherent argument.

</comment><comment><USER2> This kind of article is so bad that it actually makes the opposing opinion sound better.

It's clear that whoever wrote this doesn't understand logical deduction. There were many unconnected thoughts. The logic was like "it can only have one simple answer (marketing or biology).

"Women dominated programming at one time, but got pushed aside once men discovered the field's importance"

If it were marketing alone (and it's impossible to know that)...Why spread the idea that women could easily be pushed aside?

Just "rebrand" and boom gender diversity solved :)</comment>


Seeing an elephant is easier than describing one - <AUTHOR> We're looking for the best Elephant quotes for our startup EnergyElephant.com.<p>Please let us know your best ones. <comment><USER2> An elephant is a big animal with thick grey skin, the trunk is his nose/arm/tool. He has 4 cylindrical legs, one tail and big ears. The brain is well developed, if you are cruel to an elephant he will remeber.

</comment><comment><USER3> An elephant never forgets.</comment>


Three States and a Plan: The A.I. of F.E.A.R (2006) [pdf] - <AUTHOR> http://alumni.media.mit.edu/~jorkin/gdc2006_orkin_jeff_fear.pdf <comment><USER2> Ahh GOAP This is a very interesting approach to AI. This topic was a huge part of my FYP. I got this to work on a GPU with cuda.

Anyways, [http://alumni.media.mit.edu/~jorkin/goap.html](http://alumni.media.mit.edu/~jorkin/goap.html) (more from the same author : Jeff Orkin)

has a lot more information about this approach.

[https://github.com/stolk/GPGOAP](https://github.com/stolk/GPGOAP) (not mine) for anyone who wants to look at this in action (c/c++ code).

</comment><comment><USER3> The AI of Quake3 is also a great read!

[http://fd.fabiensanglard.net/quake3/The-Quake-III-Arena- Bot....](http://fd.fabiensanglard.net/quake3/The-Quake-III-Arena-Bot.pdf)

</comment><comment><USER4> I would be interested in knowing how does constraining state complexity in AI get handled when designing games that must meet certain framerate and performance reqs- especially in FPS games? Are there instances where decisions are forced because of the need for an agent to undertake an action? And what heuristics are used? This is definitely part of chess/ board game playing AIs with time restrictions.

Has any work been done in trying to model AI as a deep learning problem, getting testers to play against one another and running feature extractors on their behaviors (a la deep mind) and training NN models on this data?

<comment><USER5> Google's Go playing AI which recently beat some world champions? Sounds just like what you describe.

<comment><USER4> As I described in my question- board games ai aka go this has been done for years. I'm interested in fps games

</comment></comment></comment><comment><USER6> I've read about half the article so far, but isnt this basically prolog?

Is there reference to efficency in the document? Constraint solvers like prolog are pretty slow.

<comment><USER7> They added a cost per action and use A* for graph traversal instead of Prolog's depth-first search.

</comment></comment><comment><USER8> I am not a gamer, but I did know of F.E.A.R.

The paper is from 2006? Are these still valid techniques, or have they been supplanted with evolving A.I.?

<comment><USER9> > have they been supplanted with evolving A.I.?

Not to my knowledge. In fact, the AI of F.E.A.R. is still more or less looked at as a benchmark of good FPS AI. I'm really struggling to think of any game that has surpassed it. The FPS genre has been dominated by titles focused on multiplayer for the last 6-8 years, so AI has been fairly stagnant.

<comment><USER10> In games, AI is not my area. But, what I've seen is that games try to avoid training-based AI because it is difficult to get predictable, controllable results. Training an neural/genetic/whatever system is great until a designer asks you to code in a few exceptional situations and a few extra guarantees. However, there is apparently some movement into that area going on.

Here's a recent vid about game AI. [https://www.youtube.com/watch?v=jse_ZleruJU&t=11m44s](https://www.youtube.com/watch?v=jse_ZleruJU&t=11m44s) It talks about the progression from FSMs, to Behavior Trees, to Utility AI, into Neural Nets and eventually Neural Evolution.

</comment></comment></comment><comment><USER11> F.E.A.R. is one of my favourite games, and the AI was really well done.

<comment><USER12> Agreed. Great story too, for an FPS. Shame the sequels were rather lackluster.

</comment></comment><comment><USER13> HE'S TOO FAST

<comment><USER12> SQUAD, CHECK IN!</comment></comment>


New Razor Blade 15 release - <AUTHOR> https://www.razer.com/gaming-laptops/razer-blade <comment><USER2> Interesting how they're pushing the high refresh rate. Is this a standard move of gaming oriented laptops?</comment>


Concurrency in Ruby almost as good as Node.js - <AUTHOR> http://openhood.com/ruby/node/heroku/sinatra/mongo_mapper/unicorn/express/mongoose/cluster/2011/06/14/benchmark-ruby-versus-node-js/ <comment><USER2> I just can't resist, Serbs have saying 'almost is what makes girl unhappy' :).

On more serious note, ruby is definitely better when it comes to richness of available libraries which is what node is missing. I think in real life node can handle more, I heard that issues with memory leaking are solved, probably uses less memory as well. I agree with conclusion that both are more then capable or handling async problems, I would use event machine on ruby end, this whole example seems flawed a bit. Anyhow, thanks for posting your results.

</comment><comment><USER3> I would be interested to see how the unicorn variant, rainbows, would benchmark in this test. Rainbows seems much better suited for high concurrency applications because it utilizes fibers for concurrency as well as worker processes. <http://rainbows.rubyforge.org/> Also after looking at the sinatra app's code, I believe all requests to MongoDB are going to be blocking. For the comparison to be more fair, the ruby app should be written to take advantage of Goliath, em-synchrony, event machine, or another non-blocking ruby fiber based option. Until the ruby app is non-blocking, this comparison is apples to oranges.

</comment><comment><USER4> This is great work but...please redo the charts to start at Zero and end at 50,000 to keep things consistent and prevent Edward Tufte from killing a kitten if he ever sees your charts.

<comment><USER5> I thought Tufte was OK with non-zero baselines:

"The urge to contextualize the data is a good one, but context does not come from empty vertical space reaching down to zero, a number which does not even occur in a good many data sets. Instead, for context, show more data horizontally! "

\-- Edward Tufte, October 18, 2001

[http://www.edwardtufte.com/bboard/q-and-a-fetch- msg?msg_id=0...](http://www.edwardtufte.com/bboard/q-and-a-fetch- msg?msg_id=00003q)

<comment><USER4> A non-zero baseline is fine in some cases (line chart of the temperature of a person with a fever for example), but the issues here were multiple charts at varying Y scale starting points, and bar size comparisons of a property that we do want to go to zero (zero failures).

</comment></comment></comment><comment><USER6> But node.js doesn't have any concurrency. It's single-process, single-threaded and event driven, so by definition it's not concurrent. It's just not blocking.

Also, was someone really trying to run performance benchmarks on a shared machine running on a VM and expecting to get anything meaningful out? Really? Like really? I'm embarrassed for you.

<comment><USER7> [http://stackoverflow.com/questions/1050222/concurrency-vs- pa...](http://stackoverflow.com/questions/1050222/concurrency-vs-parallelism- what-is-the-difference)

</comment></comment><comment><USER8> Hey, look, requests/s on a synthetic benchmark.

Let's not look at things like thread safety, resource consumption, average/best case/worst case latency, quality of available libraries and their support for concurrency, or code quality/maintainability issues.

Meh.

At least he provided the source code and plenty of data. That's cool.

<comment><USER7> I await your proper and full analysis, or a link to one.

<comment><USER9> I don't think we are required to provide a better analysis in order to point out the problems with a flawed one.

Whatever the author is measuring here, it's most certainly not "node" vs "ruby". This should be obvious when you look at the test-apps that he's using (which involve mongodb and serialization).

<comment><USER7> It's very easy to complain, cast stones, etc. I don't think I've seen a benchmark on HN or reddit where many of the comments were not smug complaints. Yet, I've never seen any of the complainers produce a benchmark with the exception of zedshaw and one other person whose name I've forgotten.

<comment><USER10> Why should they?

Just because someone posts a benchmark that is obviously flawed, doesn't mean that someone else should be obligated to do their own benchmark as a counter.

These guys didn't ask for node.js to be used in any benchmark.

The onus is on the person who chooses to do a benchmark, and publish the results, to get it right. If they don't get it right, then they are in the wrong and deserve the criticism that they get as a result.

<comment><USER11> Because it's like complaining about the weather. It doesn't do anything to make things better. I think there's too much negativity on the Internet already.

<comment><USER10> There is too much negativity on the Internet, but there's also too much of just about everything else. The ease of publishing on the Internet means that people don't think too critically about their work prior to publishing it.

For example, if these guys were publishing this benchmark as part of a study or argument in a quarterly magazine -- and this was there only opportunity to publish it for the next year or so -- then they would have been far more critical of their own work prior to publishing it. Instead they've quickly thrown together something and pushed it onto the Internet without too much further thought.

So long as it is civil, criticism is fine.

<comment><USER7> I don't think the comment I originally responded to was particularly civil. It was quite smug.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> This is not a Ruby vs Node.js benchmark because Node.js is still a layer above the V8 Javascript engine. Moreover Node.js was optimize for things like concurrency and this is similar to what eventmachine is optimized for. The difference is node.js makes the event model a language construct. I think it would have been better to use eventmachine but you did not because:

> _At first I tried to compare bare node.js against eventmachine_httpserver. > It did quickly became obvious that this kind of micro-benchmark wasn't going > to be very helpful in deciding which one to choose._

Your first observation was the correct one. The best thing to do is just pick a language and go with the best tools available.

</comment><comment><USER13> Yes the nodejs concurrency model takes some time getting used to, but working with threads is much harder imo. Raw speed benchmarks don't do enough to give a full picture.

<comment><USER14> This is comparing nodejs with Ruby's EventMachine, so where do threads enter the picture? It's been a while since I looked at EM, but if I remember correctly, it's pretty much in the same area as Python's Twisted or node.js, as opposed to something like plain ol' Java threads or Erlang.

<comment><USER15> I don't actually even see eventmachine in there, aside from a remark about having tried it in the opening introduction. Seems it is just comparing node with sinatra+mongomapper running under thin or unicorn.

Based on the title I assumed it was something like goliath or sinatra/async with rainbows!/thin. Looking at the git repo linked in the article though, I don't believe that is the case.

</comment></comment></comment><comment><USER16> Even if you are right, your title should be "Concurrency in Ruby almost as good as Node.js ON Heroku"

</comment><comment><USER17> Requests/sec is not a good benchmark, especially considering how node.js handles streaming.

For example, if he tested a 1 meg stream of chunked data being served on request, the node.js numbers would be very different.</comment>


Things to Know When Designing for iOS - <AUTHOR> http://www.teehanlax.com/blog/5-things-to-know-when-designing-for-ios/

<comment><USER2> If you're an aspiring iOS dev/designer and any of this sounds new, you should read Apple's human interface guidelines: [http://developer.apple.com/library/ios/ipad/#documentation/U...](http://developer.apple.com/library/ios/ipad/#documentation/UserExperience/Conceptual/MobileHIG/Introduction/Introduction.html)

</comment><comment><USER3> 1) Learn to code your own God Damn Design™

2) Get an illustrator (or learn how to draw, but this is much much more harder than learning how to sling junk in Photoshop/CSS/UIAppearanceProxy )

For iOS, a developer that can do the design work is worth much more than two entities who does not know the other side.

<comment><USER4> I've seen you write a variation of the same idea on so many Hacker News stories in the past few weeks. Maybe you should write a blog post explaining in detail why you think that this is true, and perhaps we can have a civilized discussion about it without resorting to name calling (UX monkey, slinging junk).

<https://news.ycombinator.com/item?id=5393816>

<https://news.ycombinator.com/item?id=5382167>

<https://news.ycombinator.com/item?id=5393816>

<https://news.ycombinator.com/item?id=5306823>

<https://news.ycombinator.com/item?id=5306817>

<https://news.ycombinator.com/item?id=5182105>

<https://news.ycombinator.com/item?id=5138904>

<https://news.ycombinator.com/item?id=5128165>

<comment><USER3> I'm on it :)

</comment></comment></comment><comment><USER5> "Error establishing a database connection"

<comment><USER6> Google Cache link: [http://webcache.googleusercontent.com/search?q=cache:www.tee...](http://webcache.googleusercontent.com/search?q=cache:www.teehanlax.com/blog/5-things- to-know-when-designing-for-ios/)</comment></comment>


Metal – Apple's new graphics API - <AUTHOR> https://developer.apple.com/library/prerelease/ios/documentation/Miscellaneous/Conceptual/MTLProgGuide/Introduction/Introduction.html <comment><USER2> Anything that provides access to GPU command queues is welcome. It's been clear for a while that OpenGL and D3D are ill-suited to modern ways about thinking about GPUs. This also explicitly supports multithreaded clients, each with their own command queues.

[https://developer.apple.com/library/prerelease/ios/documenta...](https://developer.apple.com/library/prerelease/ios/documentation/Miscellaneous/Conceptual/MTLProgGuide/Cmd- Submiss/Cmd-Submiss.html#//apple_ref/doc/uid/TP40014221-CH3-SW1)

The shading language appears to be precompiled (which is sorely missing from OpenGL) and based somewhat on C++11.

[https://developer.apple.com/library/prerelease/ios/documenta...](https://developer.apple.com/library/prerelease/ios/documentation/Metal/Reference/MetalShadingLanguageGuide/Introduction/Introduction.html#//apple_ref/doc/uid/TP40014364)

My concern is drivers: drivers for video cards on OS X haven't been as good as Windows drivers. That, and of course the specter of another platform-specific API. This invites a comparison with Mantle. I don't think either Metal or Mantle will "win", but they're good prototypes for the next generation of GPU APIs.

<comment><USER3> As far as I know, Metal has only been discussed for iOS. Did I miss an OS X announcement somewhere? Otherwise, the driver point is moot.

<comment><USER4> I'm daring to post from a position of ignorance, trusting intuition to be correct. But this is a framework for shipping (graphical and other) workloads over to the GPU. _Surely_ it isn't mobile-only?

Just for instance, Apple sell an extremely expensive workstation with two very powerful graphics cards. Providing better ways to use a Mac Pro's GPUs is a good way to sell more Mac Pros, and a way to unlock more computational power as the power of CPU cores has levelled off.

<comment><USER5> What we know: the stated goal of Metal is exactly the same as the goal of Mantle; reducing CPU overhead.

Hypothesis 1: Metal aims to replicate that for iOS, while Mantle can be used in the Mac Pro (which uses ATI cards)

Hypothesis 2: Metal could wrap around Mantle on OSX and some other similar interface on iOS where Mantle is not available, for a unified Apple interface without having to write their own ATI drivers

<comment><USER6> Don't Apple already write their own graphics drivers on OS X?

<comment><USER5> Sorry, I was unintentionally vague, I meant without having to write their own version of Mantle. The other upside being if Metal wraps around Mantle, they can still expose Mantle to software that targets Mantle.

<comment><USER6> Ah, I see. That would make sense, though if they're trying to reduce overhead, it might not be the best idea.

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> It's somewhat scary to see new graphics APIs being introduced.

The fragmentation of OpenGL is enough of a headache, but at least it offers some semblance of "write once, run anywhere." The introduction of Mantle and Metal, plus the longstanding existence of Direct3d, makes me worry that OpenGL will get no love. And then we'll have to write our graphics code, three, four, or goodness knows how many times.

I know: It's not realistic to expect "write once, run anywhere" for any app that pushes the capabilities of the GPU. But what about devs like me (of whom there are many) that aren't trying to achieve AAA graphics, but just want the very basics? For us, "write once, run anywhere" is very attractive and _should_ be possible. I can do everything I want with GL 2.1, I don't need to push a massive number of polys, I don't need a huge textures, and I don't need advanced lighting.

<comment><USER8> OpenGL 5 _will_ happen in the next two years (and GLES 4) and it _will_ take the wind out of all these alt-language sails.

I can't see Khronos making the same fuck up twice, as they did with Longs Peak.

Yeah, it means depreciating a bunch of 3.x shit, and probably making another profile, but this time they better do it right.

<comment><USER9> I think it's reasonable to assume if an entity had fucked up once, they are at least capable of doing so again.

<comment><USER10> Even if they don't fuck it up, the half-assed implementations of it will.

</comment></comment></comment></comment><comment><USER11> You can thank AMD for this one. This is exactly why I supported Mantle initially - not necessarily because I thought Mantle will replace DirectX and OpenGL, but because it would push all the others to adopt similar changes to their APIs.

And this is exactly what happened, first OpenGL (through AMD's extension for now at least), then Microsoft with DirectX 12 [1], and now Apple, too.

Before you get _too_ excited, though, remember Mantle "only" improved the overall performance of Battlefield 4 by about 50%. It can probably get better than that, but don't expect "10x" improvement or anything close to it.

[1] - [http://semiaccurate.com/2014/03/18/microsoft-adopts- mantle-c...](http://semiaccurate.com/2014/03/18/microsoft-adopts-mantle-calls- dx12/)

[http://hothardware.com/News/New-Reports-Claim-Microsofts- Dir...](http://hothardware.com/News/New-Reports-Claim-Microsofts-DirectX-Rips- Off-Mantle-Wont-Help-Xbox-One--But-Is-It-True-/)

<comment><USER12> From what I understand, it's more accurate to say that it reduces CPU use by about 50% (rather than improving performance). If CPU is not your bottleneck, you won't see a performance improvement. If it is (which is common these days), then you'll have twice as many free CPU cycles on your main thread, which will translate into however much improvement your GPU can muster.

<comment><USER5> Even if CPU isn't the bottleneck, you ought to see some power savings.

Either way, it will make it easier to bolt a monster GPU onto a smaller, more efficient CPU.

(What we do about GPU power consumption is a different problem...)

</comment></comment></comment><comment><USER13> This to me is more surprising than Swift. But it will make for difficult platform decisions. But since there are 4 game platforms already working on it (Unreal hasn't committed) maybe it's not a bad idea at all.

<comment><USER14> I thought Unreal was 1st in line? Why would the be on stage as the poster child for Metal yet not commit?

<comment><USER15> first was Unity, Unreal wasn't mentioned but Epic Games were.

<comment><USER16> Epic Games makes the Unreal Engine.

<comment><USER14> good news: [http://i.imgur.com/ke0czgU.png](http://i.imgur.com/ke0czgU.png)

</comment></comment></comment></comment></comment><comment><USER17> So OpenGL seems to loose support on the platform that helped to make it relevant again.

<comment><USER18> You can still use OpenGL directly. I believe Metal is aimed more at engines, so they aren't providing an abstraction of an abstraction of a system call.

</comment></comment><comment><USER19> This seems to be Apple's answer to Google's RenderScript. It is too bad big companies (Google, Apple) are developing their own GPU software stack instead of building upon and furthering existing frameworks such as OpenCL. OpenCL desperately needs a kick in order to catch up with CUDA. Instead they are focusing on things like SyCL, hoping to catch up with already superior projects such as C++AMP. OpenCl should rather fix their poor specification and get implementers on the same page about it. The mobile community could have been a driving force. Instead, frustrated with what OpenCL is, mobile decided to roll their own. As always.

<comment><USER17> Metal is for graphics programming, not GPGPU.

But yes, OpenCL with its basic C API is way behind what CUDA offers in terms of language support.

Maybe SPIR will fix it, but it remains to be seen if anyone on HPC will care.

<comment><USER20> > But yes, OpenCL with its basic C API is way behind what CUDA offers in terms > of language support.

strange, language support is really the only thing that CUDA doesn't have over OpenCL. there are C++ (and python, Java, various others) bindings for host code, at least. if you're looking to use device intrinsics in your kernel code (at the cost of portability) then blame nvidia for not exposing it (and for their lack of support for OpenCL in general).

> Maybe SPIR will fix it, but it remains to be seen if anyone on HPC will > care.

yes, there are people doing HPC that care.

<comment><USER19> I guess "language support" was referring to the restrictions within the kernel language. And CUDA is better there - they support templates, you can typecheck a CUDA kernel call. With OpenCL you have to jump through hoops to get that. While some projects have managed to do it, it could be better still.

What bugs me about OpenCL is the intentional vagueness of the specification that gives every implementer the freedom to do whatever they want with the result that performance portability is often difficult to achieve.

<comment><USER20> templates are not something that (outside of simple uses) you'd want to use in your kernels. regardless, nvidia should be pushing their improvements through to OpenCL by exposing extensions. they might get adopted into the core profile.

> What bugs me about OpenCL is the intentional vagueness of the specification > that gives every implementer the freedom to do whatever they want with the > result that performance portability is often difficult to achieve.

well, that flexibility is required for OpenCL to be meaningful. that's where the variation in the hardware platforms exists. it's what differentiates compute devices. if that vagueness wasn't there, then we couldn't have things like OpenCL on FPGAs (altera, xilinx)

as for your statement on performance portability, perhaps that is an issue (but that's entirely dependent on the type of problem you're trying to compute). but something i don't understand is this;

you could have picked a proprietary API to do your compute. but say you choose CL. you optimize for your hardware, then what do you know - it's not really that fast on other hardware. but you're entirely overlooking the biggest boon here - your code ran on the other hardware _in the first place_. getting performant code is now only a matter of optimizing for that piece of hardware.

you could argue that's entirely too complicated, but that's what we have been doing already with our regular C/C++ programs (SSE/AVX/SMP...)

<comment><USER19> Templates are an essential tool to write type-independent algorithms. They enable meta-programming, an invaluable tool to provide flexible yet efficient active libraries to users. They allow automated kernel-space exploration. So templates are exactly what you want.

I understand the need for a standard that supports various different architectures, even architectures that might not exist yet. I guess I just dislike the way the did it. Compared to other standards (that also leave various things to the implementer), I think they did a poor job. They should have defined the semantics and the types better. The entire buffer mapping for example is a huge mess. Nvidia went ahead and fitted pinned memory in there somewhere. Others didn't, with the result that the meaning of the code changes completely depending on which library you link against.

I'm not arguing against OpenCL here, I'm saying they could do even better. It should not be too much effort too. And if companies like Apple and Google would have chimed in, we would have pretty awesome OpenCL standard and implementations today.

As for your argument about hand-optimization: C++ library implementers [0,1] (and compiler vendors probably too) found abstractions, tricks and tools that give performance portability today. They are of course domain-specific but it is possible.

[0] [https://github.com/MetaScale/nt2](https://github.com/MetaScale/nt2) [1] [http://eigen.tuxfamily.org/](http://eigen.tuxfamily.org/)

<comment><USER20> > Templates are an essential tool to write type-independent algorithms. They > enable meta-programming, an invaluable tool to provide flexible yet > efficient active libraries to users. They allow automated kernel-space > exploration. So templates are exactly what you want.

but OpenCL C only has primitive types. templates become more useful when you have classes, but bringing classes to the GPU is.. well, less than optimal.

> Compared to other standards (that also leave various things to the > implementer), I think they did a poor job

i don't know what your complaints are exactly, but i don't share your opinions - i think OpenCL is almost as flexible as it needs to be.

> The entire buffer mapping for example is a huge mess

i disagree. clCreateBuffer creates a buffer, clEnqueue(Read|Write)Buffer reads or writes to it. you can do more advanced transfers with the *rect variants, but you kind of probably know what you're doing at that point.

you want pinned memory? call clCreateBuffer with CL_MEM_ALLOC_HOST_POINTER. and instead of Enqueue(Read|Write) use Enqueue(Map|Unmap). wether or not you get pinned memory is up to the runtime (and nvidia's runtime does not guarantee it - it's an impossible one to make).

> Others didn't, with the result that the meaning of the code changes > completely depending on which library you link against

as mentioned, use map/unmap. it works on all the runtimes, and at least isn't any slower than read/write. as for what library you link to, that's also a moot point - we have ICDs now, you link to a shim layer that dynamically links the appropriate run time during context creation (you can have several OpenCL platforms on one machine).

> As for your argument about hand-optimization: C++ library implementers [0,1] > (and compiler vendors probably too) found abstractions, tricks and tools > that give performance portability today. They are of course domain-specific > but it is possible.

i haven't looked into either of your links in detail, but with the various BLAS/LAPACK libraries that exist, which are also far more mature (and more widely used), would almost certainly be a better choice. lots of these already work on GPUs and are optimized to death by beings who think in assembly.. most of them are in fortran, as well (although they have front ends for several languages).

</comment></comment></comment></comment></comment></comment></comment><comment><USER21> Any mention of hardware requirements? I'm betting this is 5s/Air-only. But, I'd love to be wrong.

<comment><USER22> A7 and above, if the keynote is correct.

<comment><USER23> Isn't this part of the new iOS8?

If so, I think I recall seeing the screen which models are eligible and I think the 4S was the first on the screen. Going up all the way to the 5S.

So, I think everything before and including the iPhone 4 will be left out?

</comment></comment></comment><comment><USER24> This one is quite surprising. I would have guessed that given the pervasive use of opengl(es) in their os, they would have advanced a azdo approach ( as described in [http://www.slideshare.net/CassEveritt/approaching-zero- drive...](http://www.slideshare.net/CassEveritt/approaching-zero-driver- overhead) ).

<comment><USER25> The 'fill command buffers with commands' approach they adopted is basically an alternate way of achieving the same goals as AZDO. You can view AZDO GL as just filling a command buffer for you behind the scenes (with a few other quirks)

</comment></comment><comment><USER26> This has to be a move towards the development of games on an Apple TV.

<comment><USER27> Indeed, it also explains why they put the A7 in all their devices (instead of recycling the A6 as they did previous years). An Apple TV with A7 and Metal will be competitive with Xbox360 / PS3 / Wii U spec-wise. And without a fan!

<comment><USER28> Currently Apple TV has an A5. The A8 is due in the fall. So an updated Apple TV with an A8 could be a powerful gaming machine.

</comment></comment></comment><comment><USER29> Judging by what we have it looks usable not only for graphics. Finally we got OpenCL on iOS. Kind of

</comment><comment><USER30> Does this mean now that xplatform graphics code will be a thing of the past?

<comment><USER31> Open GL ES is not going away, just a more performant option is available. Also, Metal is supported by at least two of the most popular engines (Unity and UE4), so they should handle most of the cross-platform work, provided you are using one of those.

</comment></comment><comment><USER32> Another framework, another needlessly generic word to play havoc with search.

</comment><comment><USER33> Key questions:

1\. Is it open and cross platform?

2\. Is it going to be supported across all GPU vendors?

If either of those is no, than it's a failure from the start and just another walled garden thing.

<comment><USER34> Being "another walled garden thing" will hardly be its death knell. Proprietary single-vendor APIs are already common on games consoles, and it hardly seems to have caused a problem. Look at DirectX, even; not quite single-GPU-vendor, but hardly renowned for its portability, and popular nonetheless. These things will succeed if the platform is popular, and other factors are pretty irrelevant.

I suspect iOS is popular enough to make this work.

<comment><USER33> I don't mean it won't be used. You are right, for example PS4 uses its own PSSL. But it's still a failure, because it's bad for developers who have to support multiple APIs and bad for users who likely won't get some titles for their platform because developers have no resources to support multiple APIs.

<comment><USER35> > But it's still a failure, because it's bad for developers who have to > support multiple APIs and bad for users who likely won't get some titles for > their platform because developers have no resources to support multiple > APIs.

That doesn't make it a failure, that makes it something you don't like.

This could easily be very big. If it helps developers make faster/better looking games on iOS they'll do it, especially if it's supported by middlware (since most devs probably don't make their own engine).

It that happens to make it harder to port games to Android (or at least get them to look as good), so much the better for Apple.

<comment><USER33> I don't like it for a reason that it multiplies exclusive titles. I consider it a failure. Normally authors should aim to reach the most audience, not to exclude users because they use a different OS. This API will proliferate the later.

<comment><USER36> A failure is something which doesn't succeed in achieving its goals. We often approximate this by looking at how widely used something is. For an API, this measure seems reasonable.

It has nothing to do with your views on vendor lock-in.

Direct3D also causes lock-in. That doesn't make it a failure.

<comment><USER33> Failure applies here to cross platform availability. Vendor lock is is a failure by default.

 _> Direct3D also causes lock-in._

It does indeed. That's why it also fails to be the proper portable graphics API.

<comment><USER36> [https://news.ycombinator.com/item?id=7838196](https://news.ycombinator.com/item?id=7838196)

I already spelled out for you what the word 'failure' refers to, but fine, I'll try again.

"Failure" does not mean "failure to please shmerl". That simply isn't the way the term is commonly used.

Direct3D fails to make me a coffee, too. So what? Portability to non-MS platforms was never a goal. Neither was my coffee.

It is clear that you are being deliberate obtuse to avoid conceding the point.

<comment><USER33> Failure means failure to be a portable API. Whether it's pleasing me or not is irrelevant. I explained the context of what I was discussing above.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER37> Yet another special Apple-only lockin mechanism.

<comment><USER38> I really don't see this being the case. Any competently designed graphics engine has an abstraction layer between the app's graphics routines end the platform's graphics API. While it's not trivial, another API shouldn't be something that prevents an app from being ported to other platforms.

<comment><USER39> Did you completely miss just how much DirectX locked games into the Windows platform?

<comment><USER17> You mean the same that were equally available on Wii, PS 3 and XBox 360?

</comment></comment></comment></comment><comment><USER40> I hate pages that deliberately break the functionality of the Space bar.

<comment><USER40> Downvotes!? Why? The Space bar is supposed to let me _scroll_ , and on the linked page it doesn't do anything.

</comment></comment><comment><USER41> Saw lots of HN news on Apple, I own absolutely no Apple devices, as I think while them are well made, the software ecosystem coming with it is similar to whatever Microsoft had before, i.e. essentially closed, and that is important to me, it made me to own zero Apple devices.

Am I alone here? I'm running Linux everywhere from home to my office for years, and my tablet/cellphone is Android-based.

<comment><USER42> You are not alone but you are probably in the minority.

(quick edit: You saw a lot of Apple news because the WWDC keynote was today)

<comment><USER41> for what reason that my comment as above deserves down-votes? could those down-voters explain why this should be down-voted? you have down-vote rights does not mean you can abuse it, shame on you HN

<comment><USER43> You're getting downvoted because your comments add nothing to the discussion and have nothing to do with the article that has been posted.</comment></comment></comment></comment>


Machine Learning for People Who Don't Care About ML - <AUTHOR> https://glamp.github.io/blog/posts/machine-learning-for-people-who-dont-care-about-machine-learning <comment><USER2> Machine learning and artificial intelligence are _not_ the "same thing". To say so does not "oversimplify things just a little bit": It is grossly inaccurate. Expert systems, pattern recognition, robotics and fuzzy logic- all part of A.I.- do not, per se, involve learning (though they may).

</comment><comment><USER3> Good read, broken down simply. Thanks for sharing :)</comment>


The map-reduce cargo cult - <AUTHOR> http://blog.locut.us/2012/04/16/the-map-reduce-cargo-cult/

<comment><USER2> Uh, no. The reason you copy your data to EC2 is so you can perform arbitrary transformations on it in a scalable way. I can dump 10GB of data to EC2 and then use 1000 machines to perform some processing, during which there are many terabytes of intermediate data, and an output which is less than a few kilobytes. (For example, computing the 10 most popular search queries and their click through rates over the last year.)

A tip for the author: if you notice a number of your peers doing something, perhaps you should give them the benefit of the doubt and inquire further.

<comment><AUTHOR> If the best way you can think of to determine the 10 most popular search queries and their click-through rates involves terabytes of intermediate data for only 10GB of input data, then you're doing something very wrong.

<comment><USER2> The point is that small inputs can have large intermediate scratch datasets and even smaller outputs.

Edit: Even more to the point is that being able to do scalable transformations like computing top N CTR on a lot of data with little regard to available computing/network/disk resources is the reason why you would copy your input data to EC2 for processing. If the author has a point to make he failed to do so beyond making himself look like someone who enjoys labeling things he doesn't understand as a "cargo cult."

<comment><AUTHOR> > The point is that small inputs can have large intermediate scratch datasets > and even smaller outputs.

Perhaps in some rare circumstances (although not the one you cited), however most people use map reduce for aggregation of one form or another, which doesn't require vast amounts of intermediate data unless you are being deliberately inefficient.

> Even more to the point is that being able to do scalable transformations > like computing top N CTR on a lot of data with little regard to available > computing/network/disk resources is the reason why you would copy your input > data to EC2 for processing.

Actually you'd copy it to S3 for processing, and then it would need to be downloaded into EC2 (unless you want to leave your EC2 instances running, which you won't unless you have a large number of shares in Amazon). It's hard to imagine situations where it is faster to move the data across Amazon's LAN, than to simply process it on the machine it's already on.

> If the author has a point to make he failed to do so beyond making himself > look like someone who enjoys labeling things he doesn't understand as a > "cargo cult."

The author looks like someone pointing out that the original purpose of map- reduce is that you do your computations where your data is, and that moving your data so that you can do map-reduce on it misses the point. The author is correct.

You might have a stronger argument if you could show some common non-contrived situations where there would be a relatively small amount of input data but vast amounts of intermediate data. You haven't yet.

<comment><USER2> Collaborative filtering.

I use S3 and EC2 interchangably when it comes to EMR, which is what I presume is what the author is referring to. Most EMR jobs consume and write their data to S3 and use a temporary HDFS cluster for scratch. By and large scratch data ends up being much, much larger than the original inputs, if for no other reason than that needed during the shuffle/sort stage. (I am assuming we are talking about non trivial map reduce jobs here, not word counters, where you have many reduce steps.) it goes without saying there are many applications where user-created functions will generate more data than they consume (combinatorics, etc)

Data locality is but one reason to use map reduce. In practice EMR allows you to draw upon elastic computing resources to allow you to process data however you like. It provides developer and cluster isolation and linearly scalable I/O from S3 as well. The author sounds like someone who may have read the academic papers and a few books but hasn't used these tools in practice.

<comment><AUTHOR> > Collaborative filtering.

What collaborative filtering algorithm are you using that requires terabytes of intermediate storage for gigabytes of input data?

I'm familiar with most approaches to CF (SVD, gradient descent, etc) and I can't think of any that require large amounts of intermediate storage.

> By and large scratch data ends up being much, much larger than the original > inputs, if for no other reason than that needed during the shuffle/sort > stage

I can't think of a single practical situation where you couldn't do your sorting online as you progress through the data. Again, the overhead of moving the data to-and-from S3 would be greater than processing the data locally (unless Amazon's LAN is faster than a SATA bus, which is unlikely).

> The author sounds like someone who may have read the academic papers and a > few books but hasn't used these tools in practice.

You keep attacking the author in various ad hominem ways, yet you haven't yet provided a single uncontrived example of the small input data, large intermediate data scenario that your argument relies upon.

<comment><USER2> My argument does _not_ rely upon it, it was an example of one of _several_ reasons running map reduce jobs on the AWS cloud have nothing to do with the amount input data you are moving around. I am not going to go off into even more detail about specific jobs I run daily that generate a large amount of itermediate data because unless I paste the source code in this thread and write a paper on it I assume you won't believe me that there is in fact in the space of "all map reduce jobs" jobs that can generate more data than they input.

If you write a trivial map reduce job using cascading that has 10 reducers and each reduce step shuffles the data on a different grouping key you will find that Hadoop alone is generating more data than you input. But again, this isn't the point. The point is the author is calling anyone using AWS for map reduce a "cargo cult" based upon an academic argument that the sole purpose of map reduce is to move computation to your data, hence if you copy your data you are missing the point. In practice, the cost of uploading your data to s3 is a footnote compared to the computational flexibility and use cases that become possible once you are able to run arbitrary tranformations on that data via EMR. You keep ignoring my main point and are focused on my simplistic examples, reading way more into them than was intended.

<comment><AUTHOR> > it was an example of one of several reasons running map reduce jobs on the > AWS cloud have nothing to do with the amount input data you are moving > around

It would be an example if you had backed up your assertion that collaborative filtering required large amounts of intermediate data, but apparently you are unwilling or, more likely, unable to do this.

> I am not going to go off into even more detail about specific jobs I run > daily that generate a large amount of itermediate data because unless I > paste the source code in this thread and write a paper on it I assume you > won't believe me that there is in fact in the space of "all map reduce jobs" > jobs that can generate more data than they input.

Even more detail? You haven't given me any detail! You've yet to give me a single example of a practical situation where a task involves much larger amounts of intermediate data than it's input data. I'm asking you to back up your argument, I'm not asking for access to your source code.

> If you write a trivial map reduce job using cascading that has 10 reducers > and each reduce step shuffles the data on a different grouping key you will > find that Hadoop alone is generating more data than you input

If it's so trivial, why can't you give me a single practical use-case?

> In practice, the cost of uploading your data to s3 is a footnote compared to > the computational flexibility and use cases that become possible once you > are able to run arbitrary tranformations on that data via EMR

Yes, apparently so many use-cases that you can't provide a single example of one!

<comment><USER2> Apparently I am either horrible at explaining myself or you are being deliberately obtuse. The argument about intermediate data size is a sufficient but unnecessary argument to prove the author has no point to make.

First, let's show that I can write a job that can produce more data that it inputs. I have a map of user to score, and want to compute pair wise summed scores for every user pair. This is clearly O(n^2) outputs. Computing pair wise scores is a common algorithm for recommender systems (I realize in practice you generally do not compute the entire space because it will be too slow. However your output will be closer to n^2 than n, ie, it will be much larger than your input.)

Now, this is a complicated example. A simpler example is "I want to compute aggregations on all the fields my log file." If you have N fields, hadoop is going to sort the data N times. Ie, you will be producing lots of intermediate data, almost certainly more than the input size, just by using map reduce (the code doing this merge sort is not your code, it's hadoop.)

But again, the point you keep missing and conveniently ignore when you quote my posts is that the point of map reduce on Aws is not about data locality (obviously) but about downstream flexibility. I can run 100 jobs, in parallel, on 10k machines, and output much more data than I input, without running a cluster of my own and I get to pay by the hour. I am isolated from other devs and spin the machines down when finished. If you buy the argument that this is a useful feature (as any EMR customer would attest too) than this too is a separate more pragmatic reason why the author has no idea what he is taking about when he says all EMR customers are a cargo cult.

<comment><AUTHOR> > First, let's show that I can write a job that can produce more data that it > inputs. I have a map of user to score, and want to compute pair wise summed > scores for every user pair. This is clearly O(n^2) outputs. Computing pair > wise scores is a common algorithm for recommender systems (I realize in > practice you generally do not compute the entire space because it will be > too slow. However your output will be closer to n^2 than n, ie, it will be > much larger than your input.)

But that's exactly it - you admit yourself that this is a contrived example! For all but trivial numbers of users N^2 storage requirements would be impractical for any system, no programmer in their right mind would take such an approach.

And this is the point that you seem to be missing: the question is not whether you can come up with some contrived scenario where there is vastly more intermediate data than input data, but when you can come up with a realistic practical example - and apparently you cannot.

> A simpler example is "I want to compute aggregations on all the fields my > log file."

In this case the amount of intermediate data is no greater than the amount of output data.

> But again, the point you keep missing and conveniently ignore when you quote > my posts is that the point of map reduce on Aws is not about data locality > (obviously) but about downstream flexibility. I can run 100 jobs, in > parallel, on 10k machines, and output much more data than I input, without > running a cluster of my own and I get to pay by the hour. I am isolated from > other devs and spin the machines down when finished.

These are benefits of EC2, not of map-reduce. The critique was specific to map-reduce, not of the entire principle of on-demand computing.

> If you buy the argument that this is a useful feature (as any EMR customer > would attest too)

Strawman. You're attempting to conflate the general benefits of on-demand computing with the specific benefits of EMR. The author was criticizing EMR in particular, not the entire principle of on-demand computing.

> than this too is a separate more pragmatic reason why the author has no idea > what he is taking about when he says all EMR customers are a cargo cult.

And yet you still haven't been able to cite any realistic, non-contrived examples of where EMR is the appropriate tool for the job.

<comment><USER2> Ok, it's clear now you have horrible reading comprehension and/or have never used Hadoop in practice. If you perform an aggregation on 10 fields in a log file, a common pattern, you are going to be generating more intermediate data than your log files during the shuffle. (Hadoop is, in this case.) If you generate a dataset that has pairwise scores for users, even if you are only sampling 1% of them than you are generating _way_ more data than your input. (You are generating this data in this example, not Hadoop, which is why I included it.) Say N=1M. You want to compute the top 1% pairwise scores (pruned via some locality sensitive hashing.) Guess what, you are computing N^2 * 0.01 = 10B records. Perfectly tractable for a big data system. You seem too busy yelling at me to even read what I am writing, since I stated this clearly: "(I realize in practice you generally do not compute the entire space because it will be too slow. However your output will be closer to n^2 than n, ie, it will be much larger than your input.)"

You say: "These are benefits of EC2, not of map-reduce. The critique was specific to map-reduce, not of the entire principle of on-demand computing." This is patently false. The criticism of the author is that people using EMR are a cargo cult and do not understand the purpose of map reduce. Ie, they should not be using EMR, and he is so clever for realizing the folly of their ways. My claim is the author does not understand the purpose of EMR. By definition, EMR brings with it the benefits of EC2. I don't see how this is a strawman.

At least I am sure now that it's not my communication skills, I illustrated two perfectly fine examples and you glossed right over them without understanding. I've already spent enough time on this and argued in good faith and at this point I can at least be confident that you have no interest in understanding what I am saying and instead want to childishly prove me wrong on the internet.</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment>


Ruby & WebSockets: TCP for the Browser - <AUTHOR> http://www.igvita.com/2009/12/22/ruby-websockets-tcp-for-the-browser/

<comment><USER2> That was a good article! I would not worry about how fast WebSockets catch on with the general public: even if in 6 months only a small percentage of web users have a browser that supports WS by default, that is still a lot of potential users. Really, a great looking technology.

<comment><USER3> It's also possible to fully "backport" WebSockets through a flashembed that acts as an intermediate layer. Flash has about 98% penetration so this should make the technology usable right now. Especially with an eye to a future where the flashembed will be loaded less and less when browsers start to implement it natively.

</comment></comment><comment><USER4> Very clear write up on WebSockets in general and Ruby/EventMachine in particular. Loved the examples!</comment>


Ask HN: How do you listen to your users? - <AUTHOR>

I'm just wondering how you've gone about getting feedback while developing a product. I work for a fairly large company and can't directly communicate with our users. Sharing any experiences you've had and the methods you've used (either direct, or indirect, communication) is appreciated! <comment><USER2> The best form of communication is action. Do what your users request (assuming it's sensible of course) and they'll notice and appreciate it.</comment>


Instagram CEO responds to questions about why it cut off access to private API - <AUTHOR> http://www.quora.com/Instagram-told-3rd-Party-developers-today-to-stop-using-their-site-data-shutting-down-Followgram-and-possibly-others-Was-this-the-right-move-to-make-for-users/answer/Kevin-Systrom

<comment><USER2> I think Kevin's response is perfectly reasonable. If his account of Followgram's behavior is accurate (2 sides, stories, and all) I have to agree with Instagram's actions.</comment>


Local Startup Community Sucks, Are All Like This? - <AUTHOR> http://technow.info/2013/08/why-my-local-startup-community-sucks/

<comment><USER2> I haven't observed anything like that here in the RTP (NC) area, but I don't do that much socializing with other "startup people" these days. I stay too busy actually working on our startup to spend a lot of time hanging out at "startup events" and what-not. But even when I did spend a lot of time in that scene, I don't remember hearing about anything remotely like the stuff described here. shrug</comment>


Rate our startup: Jobzle.com - <AUTHOR> http://www.jobzle.com Currently a soft launch for employers to pre-populate jobs with the student side coming next week (to coincide with press release / marketing). <comment><AUTHOR> Would appreciate any and all advice. We're a young startup out of Rhode Island, just got out of an accelerator program and are launching to Rhode Island colleges.

Feedback, bug spots, feature suggests, criticism would all be great!

I'd especially like to hear from employers. The idea is that, as we grow, you'll be able to reach out across the country targeting specific colleges (best art schools, best engineering schools etc. etc.) or locations (certain cities).

<comment><AUTHOR> Also: The current launch is a "soft" launch for employers with the homepage with the student c2a going up on the 13th.</comment></comment>


APM, metrics, and logging- what's the difference? - <AUTHOR> What are the main differences between APM, metrics, and logging? <comment><USER2> APM is about monitoring of application performance, e.g. response time, latency, etc. Metrics typically point to host resource usage, e.g. CPU, memory, network, disk usage. They both give you an overview of how things are going in general in your environment, so they're both important. But to find the root cause of anomalies, logs are the single common thread. Developers still need to go back to logs to find errors and exceptions when things go wrong. So short answer is that you need both logs and metrics, ideally in a single interface. Loggly is a good option that combines both.</comment>


Even bad estimates are valuable if you use them right - <AUTHOR> https://ntietz.com/2018/08/31/estimates.html <comment><USER2> > you must explain to your stakeholders beforehand that these estimates are > only for course correction during the development process, and they're > separate from estimates you will give of when features will be done overall.

...

> if you do put in the time to do estimates, you will spend less time coding - > but because your team will be able to respond to things immediately, you > will still reach your objective more quickly.

This sums up the best argument I've read about the advantages of best practices agile software development.

It's a shame some companies cargo cult agile as a way to bully engineers into being "more productive" in a brute force manner. That is, unrealistically cramming tickets into a sprint to gamify higher "velocity." Really agile is just about a more efficient development process. Agile provides a framework for discovery and course correction via planning/imagination/whiteboarding, and that is significantly faster than discovery via writing code.

<comment><AUTHOR> Yes, exactly! It's such a shame that people think "more efficient process" means "we'll ship more lines of code" when really it is all about getting to the _correct solution_ more quickly.

<comment><USER3> Presuming you know what the correct solution is. This is the key point of agile processes, that you get to quickly discover what is required, optional and completely unnecessary. Generally by having a real customer (not a silly proxy) in the room or by sending out probe releases often and collecting feedback.

</comment></comment></comment><comment> awinter-py > If you never use the estimates to adjust what you are working on, then > putting in the time to do estimates is a pure waste

yes! using plans to create context for the larger project is the critical missing piece.

Business stakeholders just want delivery dates but the team doing the work benefits when the estimates fit into an organic, flexible master plan that is changed when things are late or hard.

</comment><comment><USER4> The book The Influential Mind does a chapter on (essentially) tips & tricks for estimates (of all kinds). Long to short, be careful of group think. It's best to collect the estimates independently, then aggregate. Else the group members influence each other and the advantage of "crowd sourcing" goes out the window.

The book is worth a go. It reads quick. A bit light- weight but still useful.

[https://upliftconnect.com/tali-sharot-influential- mind/](https://upliftconnect.com/tali-sharot-influential-mind/)

</comment><comment><USER5> The biggest value of bad estimates imo are that they teach you to get better at estimating. You should record your estimates and the actual time it took. That way the next time you need to make an estimate you have some more data to plug into your formula for making a guess: the previous slippage. So you improve over time and your post-mortem of _why_ something slipped will help you to get your initial guesses closer to where they should have been in the first place.

Between those two anybody can learn how to estimate with some reasonable degree of fidelity.

</comment><comment><USER6> Just a few days ago, one of the restaurants I own was getting a new air conditioner installed as a condition of lease renewal. Of course, the landlord did not tell me the AC people were coming to do the work and just showed up at 7AM with the expected high temperature to be in the mid-90sF. It was already 78F when they started.

I asked my manager to ask the AC guy how long it would take. "He doesn't know", she said. Such an answer drives me insane. You mean he has no clue? Will it take three months?! 30 minutes?! How can we plan to work around no AC in a restaurant when the installers have no idea how long it will take?

But of course they had a rough estimate. Why didn't they give me one? It would relieve a lot of anxiety if nothing else even if it was wrong.

At the three hour mark, I was told it would be about an hour. After 5 1/2 hours, I was told half an hour. It took 6 1/2 hours to complete.

Here are the issues with all that:

1) If I had an estimate, I wouldn't have needed to drive to the restaurant and talk to the installers myself.

2) If I didn't get an estimate, I would have had to raise hell with the landlord who I have a good relationship with.

3) Once I had an estimate, I didn't feel the anxiety or need to constantly check on the progress which would have irritated the installers. And believe me, I would have irritated them!

<comment><USER7> Unfortunately, a lot of people treat "estimate" as "a binding commitment written in blood" \- so I sometimes understand when people are reluctant to give such a thing.

<comment><USER8> So much this. Whenever I'm trying to pull an estimate out of someone that is reluctant I always try to reassure them that I know its an estimate, I know it might be wrong, I know conditions can change but its still helpful to have a guess.

</comment></comment></comment><comment><USER9> There are just too many unknowns to do estimates well in software. And getting things wrong in estimates isn't a matter of being off by a couple percentage points but being off in by orders of magnitude.

It isn't unusual for what appears to be a simple unit of work that seems like it should take a few hours to explode into days or weeks of work. And it does so in a way that is completely opaque to stakeholders.

Partly this is because of how we build software as layers of abstraction on top of other layers of abstraction. When you are at the top layer things are relatively quick and easy to estimate, but once you have to work at a lower level than normal all bets are off. And it is really hard to know beforehand if you will need to dig into the lower levels.

For example at my job I am updating an app that queries an api to filter its requests by some parameter. When the api supports the desired filter that is great. When the api doesn't, I have to dig into unfamiliar code for the api to support it. But turns out the api is just forwarding its request to another api, so I have to dig into yet another api to build the filter. And so on. The rabbit hole can get deep. And of course there isn't any documentation to help me know about this beforehand.

The article is right that the key is frequent updates and communication as soon as "blockers" like this are discovered. But it can be very difficult to communicate this to stakeholders who are uninterested in the technical details and are inflexible in scope. Trust is key, so that they believe you when you say that this task which normally takes an hour or so will now take several days.

</comment><comment><USER3> A team cannot reduce scope without involving the customer. And if you have the misfortune that you're dealing with someone who thinks difficulty is negotiable, you're in a real pickle.

You're missing the critical point - measure the estimation error and you'll see how hopeless almost all estimates are. In practice, they do not improve.

</comment><comment><USER10> Akin's 10th law of spacecraft design: "When in doubt, estimate. In an emergency, guess. But be sure to go back and clean up the mess when the real numbers come along."

</comment><comment><USER11> One caveat: the relative level of definition of work has a huge effect on the work. For example, well trodden engineering implementations can have rational top and bottom end estimates. A UX design or Eng implementation of a "never been done before" or strategically-shifting problem is far far harder to estimate.

</comment><comment><USER12> This was one of the very few valuable things I learned working towards a bachelors degree.

One of my professors popped a quiz on the class with the question "how many piano tuners live in the state of New York". After collecting the answers, he spent the rest of the session lecturing (especially those like me who left it blank) on how to make estimates without having any firsthand knowledge in the problem space, and how valuable that skill could be.

<comment><USER3> And none of the methods work in software development.</comment></comment>


Who really owns those patents that I thought I owned? - <AUTHOR> http://www.masshightech.com/stories/2011/02/21/daily51-Who-Really-Owns-Those-Patents-that-I-Thought-I-Own--.html

<comment><USER2> Seems like your wholly-owned subsidiary that is commercializing your patents is returning its profits to you as a royalty, and therefore its marginal profit on sales should be used as a guide to calculate the "reasonable royalty" that is your acknowledged due.

</comment><comment><USER3> Gosh, US patent law is more messed up than I thought. Not only do The Lawyers get a huge cut of the money on any "infringement" cases, but you have to pay them along the way to make sure that you get your rightful infringement vengence.

</comment><comment><USER4> Nathan Myhrvold</comment>


Is google appengine Down? - <AUTHOR>

Do you guys also face the problem in appengine today? <comment><USER2> nope</comment>