Ask HN: How to learn to web dev for real ? - <AUTHOR> Hello everyone
After reading all I could find on Python/Django/Ajax/JS/etc, I find myself able to write simple sites (blogs/etc). But I still don't know how to do a real project.
- Methods and tools to updating/patching a working server (with thousands of users). - Creating scalable sites - Setting up a testing environment And much more.
Can any one recommend articles/books on programming that go beyond the "blog app" ? <comment><USER2> <http://highscalability.com/> is a great resource for building scalable sites - it has consolidated architectural details (culled from presentations, blog posts, interviews, and more) for many large-scale sites (YouTube, Facebook, Amazon, etc).

As for testing environments, take a look into the principles of continuous integration (many resources available) and continuous deployment (especially Eric Ries's blog: <http://www.startuplessonslearned.com/>). Some tools commonly used to this effect: Git/SVN for source control; Hudson/Cruise Control/Team City for builds; Selenium for testing.

Finally, applying patches and updates to a server falls into the domain of systems administration, so I'd suggest looking for articles on patching/updating high-availability sites. Note, however, that cloud computing is a huge game-changer here. Whereas at many enterprises, applying patches is an arduous (and tedious) quarterly effort culminating in an late-night maintenance window with many hands on deck (all crossing their fingers), clouds afford their adopters the ability to simply fire up new, updated servers alongside the old ones, allowing them to be thoroughly tested in place before any traffic is redirected.

-Mike (@mikebabineau)

<comment><AUTHOR> Thanks, exactly the sort of resources I was looking for.

Perhaps I failed to introduce my questions correctly. I have been a system admin for years on military server farms (100+ servers) and have been an embedded developer for a few years more.

Currently I am expending my area to web. After writing a few basic web sites that have all the vanilla stuff (Users/Messaging/Pretty JS&AJAX&JQuery stuff). I am trying to learn how to build large scale sites.

* High scalability [more then one server. both web&db]. * Deployment [how to upgrade/patch software on a working site with thousands of existing users with minimal disruption] * Testing [both automatic and non-automatic, including different web browsers and OSes] * Any other thing in this category

Thanks for your info !

<comment><USER3> It seems you know how to program and how the web works, but want to know how it all fits together (testing, deployment, scaling etc). I think you will find Jacob Kaplan-Moss's presentation of all the pieces of building a web application. It is geared towards Django, but it should give a good overview of what goes into a webapp. And if you are using Django or Python, all the better!

<http://www.slideshare.net/jacobian/django-in-the-real-world>

</comment></comment></comment><comment><USER4> Hmmm... I'd say that you better start coding and stop reading. It will be a mess at first but you have to gain some experience.

I don't want to discourage you but you probably won't have thousands of users on your apps at the same time at the beginning so don't bother with creating scalable sites yet. One day, you'll have some slowness on your app and you'll learn it at this moment.

</comment><comment><USER5> Having recently started as a "professional" developer, coming from the realm of the hobbyist, I can say that the best thing you can do is get a job.

Start with an internship if you are not yet confident in your skills. The best way to hone your abilities is to set up a situation where you get unexpected problems from code you did not write or an environment you aren't entirely familiar with, and fixing it with the mindset that failure is not an option.

It also helps to get paid while you're doing it. Really. It shows that someone has put an incredible trust and investment into you, and they value your intellectual output.

Start a personal website to host your projects, make it available to the public. Take a look at the tools you use. Try submitting code contributions to the open source projects. Start a blog and document your hardships and discoveries for other budding hackers.

Hopefully this advice helps you some, best of luck!

</comment><comment><USER6> I would suggest starting with Google App Engine (the Python version). For the following reasons:

1\. Python is a good language to learn. You can use it for other things beside building a web app. GAE is similar to Django.

2\. The GAE datastore is easier to fathom than a traditional database. For some reason, I always found databases hard to understand, and it seems that designing schemas for non trivial projects is a separate and distinct skill set from software engineering.

3\. GAE solves the updating/patching of working code by providing a sane environment and tools to update and version your app. You can easily switch back to a previous version. Patching datastore schemas is slightly harder.

4\. You get scalability out of the box. You don't have to worry about hardware, or even configuring "virtual" instances. None of that. But you probably won't have to worry about scalability for a while.

5\. You can easily deploy code to multiple environments. You just need to signup for more apps, and update your app.yaml file to point to a new app name. You can do this in a simple build script.

I hope this helps.

Cheers.

<comment><USER7> Because of GAE's odd architecture, a lot of things that you could get away with elsewhere (especially at non-mega-scale) will give you obvious latency hits.

The upside is that the issues are easy to find, and your code doesn't get any slower if you jack up the load!

It leads you to think about scaling in a most pleasant way.

</comment></comment><comment><USER8> As the author of a web framework book, I have to say that the example applications are not actually much less complicated than real applications in any significant way. The real applications are often bigger, but it's just more of the same stuff. The bulk of your application is going to be a web- independent model (read any software engineering book for advice in developing this). Then you add some thin glue to make that model look nice over HTTP. (This includes things like REST APIs or AJAX or whatever.)

That's it. The glue to the book's Address Book example (or whatever) is not going to be any different than the glue of a complex app. All the good stuff in your program happens outside the domain of the web.

</comment><comment><USER9> It's been hinted at in dan_sim's post, but if you have a "real project" in mind just go do it. Learn what you're missing when you need to. You probably know more than you think.

<comment><USER10> Also, in my experience, I haven't been able to learn anything "for real" without using it on a real project.

<comment><AUTHOR> I have already created a few small projects (the biggest being 2 weeks of work).

But there is nothing to show me if I correctly handle my prod-server updates (Now I upload, move, restart)

My testing is very basic.

And there is a general feeling of lack of infrastructure.

I would love to learn how to do a few-man-year project (from infrastructure's point of view)

<comment><USER11> A real project is an excellent way to pick how to do web development.

Have you got any hobbies or interests? Why not create a website dedicated to your hobby/interest? That way it'll be something that you'll be able to run over the course of a few years.

Start with a small website, and slowly build it up. Add more features, then tweak them. Find out why it's loading slowly. Read a few books on website that scale too.

</comment></comment></comment></comment><comment><USER12> Well there is nothing stopping you from getting a small host somewhere and put up a site of some sort. This can then be used to practice the skills you will require to manage a host. Make sure that the site is actually being used for something so that you will need to maintain it and do upgrades.

Then you only have to learn how to...

1) Install and configure a machine 2) Configure a web server 3) Set up ssh access securely 4) Set up your database securely 5) Do backups (and as a test nuke the box and rebuild it from backups) 6) Automate the deployment of your application 7) Keep the machine up to date with security patches 8) Keep the hackers at bay 9) Monitor the general health of the machine and services

Basically you will start to learn the art of the sysadmin. It is a useful skill if you end up at a small company without a dedicated sysadmin and even if you have one it will have a better idea as to what is required to develop and host a site. Empathy with the sysadmin is always good thing.

You will learn a lot of new skills, most of which do not involve coding. I have a little hobby site that hosts fan art for a web comic, I have learnt much about automatic deployment by keeping the site up with minimal disruption. I have learnt much more about the running of databases (as opposed to just writing sql) and loads about unix security (users, groups, permissions, jails etc) that programmers just tend to ignore.

All for around $9 a month. Money well spent if you ask me.

<comment><USER4> I would suggest <http://prgmr.com/xen/> for 6$ a month.

<comment><AUTHOR> I have a few sites on Webfaction already.

While I figured out how to do most things you mentioned. I feel like I am re- inventing the wheel with most of them.

Surely there are advanced, open-source deployment / backup / version /etc systems in use in big projects.

Thus I am looking for guidance on where to start learning that part of the web-development process

<comment><USER13> You will be reinventing the wheel; that's part of working with open source on big sites. There just isn't one open-source deployment/backup/versioning/etc system out there. Nearly every large deployment/backup/versioning/etc system i've seen has been custom. Most of these details will be specific to the group you're working with and the project's requirements.

Don't worry so much about getting the process right or finding a pre-packaged solution. The most common solutions will be the same old tools re-worked and customized to get the job done on that site. I'd say the best thing you can do is put your fingers into every single open-source pie you can and get a feel for it all. The rest is a natural evolution of developing for your target platform.

</comment></comment></comment></comment><comment><USER14> "just do." I know that sounds rather cliche, but it's something that I've found a lot of beginners are missing. They spend a lot of time trying to find the "best" way to do something before they start doing it. Really, just do something. You'll learn from the experience either way. You'll probably learn more than if you _had_ found the maximal solution first.

I probably learned more about programming by hacking together a simple role playing game in Javascript+DOM in 2001 than my next 3 years of college and 5 years of work. That 8 year old code still worked up to the point GeoCities died, and then it was gone permanently. Not a big loss, but it was nice to see that it survived a myriad of browser updates. I learned tons about what made good, readable code just from having to live with it. I learned a lot of the art of optimization because of how much Javascript sucked at the time. But more importantly, I learned that it was more important to get-it-done than to be working in what was popular (you would not believe the flack I caught for not using C for it, even though I was also learning C as well and just wanted to do something in Javascript).

<comment><USER15> OT: Did you write your own cross-browser DHTML library or use one?

I did the same thing (considerable browser-based app in the pre jQuery dark ages) and it was painful.

Have you checked the Geocities archive <http://reocities.com> to see if it's there?

<comment><USER14> I don't think such things existed at the time. At any rate, the code was 100% new and performed no browser checking, but still worked exactly the same on all browsers. It took me a month to do what I could do in a day now, but it was a great learning experience.

I've started a new project recently, maybe about 10 hours of work into it, that is a chess game, 100% new Javascript, so far 100% cross browser compatible (with no features planned that would break that), with no browser checking. It's actually not that hard if you're smart about your debugging (don't use alerts, use try/catch and write out to a div).

</comment></comment></comment><comment><USER16> I'm not a web developer, but I posted a similar question on Ask.Metafilter last summer ([http://ask.metafilter.com/124165/Web-development-big- picture...](http://ask.metafilter.com/124165/Web-development-big-picture-for- a-nonweb-programmer)) and got some helpful suggestions.

For me, the most useful response in the long run has been delmoi's suggestion to just scrutinize the HTTP 1.1 spec and write a webserver from the ground up. (I'll release the webserver, an embedded coroutining/select-multiplexing server written in Lua, when I release the project driving its development.) My interest was specifically in using HTTP 1.1 / REST as a generic interface for servers, rather than creating dynamic web sites per se, though.

Either way, it definitely helps to have a real project in mind.

</comment><comment><USER17> For frontend stuff:

High Performance Web Sites Essential Knowledge for Front-End Engineers (By Steve Souders) <http://oreilly.com/catalog/9780596529307>

And for an intro to scalability:

Building Scalable Web Sites: Building, Scaling, and Optimizing the Next Generation of Web Applications (By Cal Henderson) [http://www.amazon.com/Building-Scalable-Web-Sites- Applicatio...](http://www.amazon.com/Building-Scalable-Web-Sites- Applications/dp/0596102356)

</comment><comment><USER18> Updating and patching a server running a web app without down time is an art. And, there's not a ton of stuff out there about django deployment best practices. Here's a few thoughts from a fellow developer:

Python has a decent way to isolate deployed apps using virtualenv. If you google virtualenv django, you should find some interesting reading.

For automated python deployment on unix boxen, fabric seems to be the state of the art. You might also want to take a look at capistrano on thr ruby on rails side. You'll probably find a lot more interesting reading about deployment tips and tricks by reading stuff from ruby/rails land. (Don't flame me) The Rails guys have historically had more problems with speed and scaling rails, so there's a lot more information out there. They've also built some really cool tools to deal with those problems.

Django has some really cool caching features. I was just playing around with caching and load testing <http://newsley.com> by throwing ghetto localmem caching on top of my main view, I got an order of magnitude more pages served per second. I didn't even try memcached as a backed yet, but I feel a lot better knowing the basics of how to do that if the need arises. Regardless, you'll want to spend a bit of time learning the caching tools anfd reading about memcached if you're interested about scaling django.

A lot of scaling web apps is about scaling a database. It's routinely the slowest part of your web app. You shouls be able to find a lot of reading about that. The highavailability blog is a great resource for that stuff.

Cal henderson (founder of flikr) wrote a great book called "building scalable websites" if I'm not mistaken.

<comment><AUTHOR> Excellent advice, thank you !

<comment><USER18> Any time. Django has been improving their test suite recently, and their documentation on testing has been improving a lot in the last couple of months. There's also projects like Twill and Selenium for building automated tests that I like a lot.

Another thought on practicing deployment. If you really want to learn how to deploy to a multi-server environment, I'd move off of webfaction hosting and onto something like EC2. EC2 is great for being able to spin up and then spin down server instances at will. You could set up a DNS round robin pointing at 3 or 4 django servers which in turn point to a separate db server. Write the build scripts to automate set up and take down of all the separate servers. You could also practice setting up database replication on separate EC2 nodes if you wanted to. And, you're only paying a a couple of bucks a day to practice setting up multiple servers.

You might also be interested in puppet: <http://reductivelabs.com/trac/puppet/wiki/DocumentationStart>

</comment></comment></comment><comment><USER19> If you are young and just starting, try to find a position with a web dev company who uses the technologies you want to learn. Pick an agile company who doesn't need a load of sign-offs to get something done. Don't worry about your $/hr. Worry about what you can learn.

Basically I'm trying to get you to find a good mentor. You can learn more in 1 week from a good mentor than you can by reading PDFs and blog posts for two years.

<comment> daniel-cussen Basically I'm trying to get you to find a good mentor. You can learn more in 1 week from a good mentor than you can by reading PDFs and blog posts for two years.

So very true. I got stuck with the latter, and there was a brief period in there where I learned loads from a tutor. World of a difference.

</comment></comment><comment><USER20> Create an idea that requires you to do several things for which you would like to learn. This idea should ideally include offline processing of work, so that you are forced to read and understand many issues with executing jobs outside of the client/server request cycle (maybe this is image creation, video/audio transcoding, log file analysis etc etc). Then make sure your project can leverage some sort of key/value storage system like redis or memcached, or maybe a document store like mongodb or couchdb. Finally, make it something that people like so you'll get enough traffic to need zero downtime deployments.

If you make a project like this, you will learn most of the high value systems/concepts being explored and demanded right now. Additionally, if you satisfy the last constraint, you may have a cool successful startup at the end of it.

</comment><comment><USER21> (warning: going a little off topic here, sorry) Might I ask how you got started in the first place? I've got a fairly reasonable background regarding the theory of it [programming], I know the principles of OO and basic algorithms, etc., but in school we don't cover the actual writing of code. (And even if we did, it's probably safe to assume it'd go nowhere near as far as I'd like it to.) Anyway, I picked up some Ruby lately, worked through a bunch of tutorials, and it's been going decently - at first much too easy for someone like me, then challenging, but then there's stuff I just don't know how to do. On the one hand, I've repeatedly had big problems with blocks in Ruby - I can't seem to grasp why to use such a weird format when you could just use regular loops instead. On the other hand, and much more importantly, I don't really know where to go from there. I can't write any real programs, and I'd like to get into Rails eventually (as a gateway to Web developing as a whole).

Edit: Out of the tutorials that I did, this one[1] was the one I liked most, as it had a lot of cool tasks that you could just try and solve for yourself, it really helped me get the basics down. However, none of the tutorials have gone any deeper than that one, and as I'm sure you'll agree, I'm not exactly a programmer yet after that tutorial :) Additionally, I've started to read this[2] book, but it appears to follow a really strange direction and is generally not very pleasurable to read (IMO). And, again, the moment it tries to explain blocks to me I just stand there puzzled.. dropped it after I hit that point, as I did with all the other materials I've tried out so far.

Again sorry for hijacking the thread and apologies for being unable to offer any advice on your situation. Regards

[1] <http://pine.fm/LearnToProgram/>

[2] <http://www.ruby-doc.org/docs/ProgrammingRuby/>

Edit2: Alright, just saw you've been a developer for longer than I've even used a computer :) So I suppose you can't answer this question either, bah. Wish there were more people that didn't get into programming either 1950 or at age 5. Really, where does someone start nowadays when they're 20 and have no clue.

<comment><AUTHOR> Since I am coming from a long background of embedded programming, I guess my starting point is different.

But, since my last company loved hiring bright college kids with 0 experience (and I was usually in charge of training them), I have to say. Work with someone experienced.

You can re-invent the wheel a thousand times, but you will get much better if you learn from someone else.

And until you do that, I have found a few blogs/videos that show how professional developers write a simple site, start to finish. Since they annotate the process, you really get to understand not just the how but also the why.

So IMHO best:internships second:the blogs above third: programming yourself (and if you ever decide to switch to embedded programming and move to Israel, I can hook you up ;)

<comment><AUTHOR> Try to get into a very large software company. Many of them are willing to take newbies and pay the teaching price.

Many people I know started by working a year or two at Checkpoint. Go in green, go out a super-star :)

</comment></comment></comment><comment><USER22> I wouldn't get to hung up on being "scabable" if I were you. Just do it and worry about that later if it becomes an issue (which it probably won't, plenty of sites are successful at what they're doing without having to many troubles in this area).

Oh, and learn to use source control (Git or Mercurial or similar) if you haven't already. When you break something on your site and don't notice for a bit it's pretty important to be able to look back through the code history to find out where it all went wrong :)

</comment><comment><USER23> The best way for you to learn is to do so as you go. Plan an idea, a "big project" without worrying about your skills. Make what you can, learn what you cannot. When you come up with a question Google cannot answer, go ahead and ask it at StackOverflow. They are very nice and do not mind "noob" questions.

That way, you will learn by doing, which is much more pragmatic. However, that does not mean that you should not learn programming theory! Embrace what you can.

</comment><comment><USER24> the best way to learn is to try. all these things that only apply to "real" sites can be tested in smaller doses. try setup a "scalable" blog site.

setup an artificial system that can't stretch very far by limiting the memory / instances of your web server e.g. then hit it with more than you know it can handle. deal with that how you think it should be done. if it works for 1000 -> 10000 requests the same might work for 10000 -> 100000 as long as your scale technique doesn't depend on small numbers.

same goes for all the other things you mentioned, they can be tested and coded for without having more than a "blog site" as a test platform. just do it :) code away!

the same applied to a "testing environment" would be to make yourself a 2nd copy of your blog site. push to the copy, make sure things work, then push to the main site. possibly even automate that 2 tier push by writing tests that can be tested without user intervention :)

</comment><comment><USER25> Scalability solutions are best found when you have a scalability problem.

I don't know that I have heard of a project failing because it wasn't scalable enough. The types of things that create scalability problems also tend to bring the resources (and motivation) to solve them.

</comment><comment><USER26> Software Engineering for Internet Applications by Philip Greenspun

<http://philip.greenspun.com/teaching/one-term-web>

</comment><comment><USER27> Spend some time at a company that does this stuff daily.

(Yes, I'm hiring.)</comment>


Why Gamification Can't Be Stopped - <AUTHOR> http://mashable.com/2012/04/07/gamification-bunchball/

<comment><USER2> Every new website/application these days needs some level of gamification to attract and retain users and grow fast. Best if it is inherently built into the actual product. Anybody disagree?</comment>


3 Things You Need to Do When Setting Up Your First AWS Account - <AUTHOR> https://dev.to/kylegalbraith/3-things-you-need-to-do-when-setting-up-your-first-aws-account-9j4 <comment><USER2> The article says "You" so I'll put aside for a moment the situation where there are three admins and they should all have their own account.

        Great! Now Stop Using Your Root Account

If you can make an unprivileged account for general use, great! Separation of privs is an awesome thing. But if you're going to make an IAM account with AdministratorAccess and BillingAccess as described, how is that any safer to use than the root account? I see this a lot and I'm quite interested in the reasoning.

I've also found that certain forms - like the Penetration Test permission form, must be done as root. So you need to use the account occasionally even if you don't want to.

<comment><USER3> The root account has no restrictions whatsoever, but you as an admin IAM user can block yourself from e.g. deleting DynamoDB tables to make accidental deletion harder: [https://aws.amazon.com/blogs/database/preventing- accidental-...](https://aws.amazon.com/blogs/database/preventing-accidental- table-deletion-in-dynamodb/)

</comment></comment><comment><USER4> _" Configure your billing alarm to send you an email when you cross a threshold that is more than you can afford."_

Does AWS offer you the option to just have all your services shut down when spending reaches a threshold?

I'd hate to get billed for something I can't afford when it could be prevented.

<comment><USER5> > Does AWS offer you the option to just have all your services shut down when > spending reaches a threshold?

Not really. "Shut everything down" is a VERY dangerous switch to have; you're almost always better off having a billing alarm that prompts you to react in a more considered manner. Obviously, from this perspective, AWS is targeting "business-critical production infrastructure" a lot more than "hobbyists" as the intended customer of AWS; the risk of unintentionally spending lots and lots of money on AWS is also much greater for organizations that can probably page someone to address the issue in a more fine-grained manner than just shutting everything off.

There's also the question of how that would even work. Do you delete all your S3 buckets? Or do you just hold the data hostage until the customer increases the budget enough to access it?</comment></comment>


It's not about this 5c - <AUTHOR> https://marco.org/2016/02/17/not-about-this-5c <comment><USER2> I think people are also missing the point when they say "newer iPhones wouldn't be affected by this anyway".

I think what's more likely to happen is that in a future case, the FBI will tell the judge that Apple has already complied with such a request before, and now "suddenly" it doesn't want to anymore. Then the judge will probably agree with the FBI that Apple is in the wrong here, and tell Apple "hey, do that thing you did with the previous iPhones, too".

And what _that_ will mean is that Apple will have to make it so the Secure Enclave _can_ be tampered with. That would affect _all_ devices, because Apple would have to cripple every single device like that if it wants to be able to comply with all future orders, and you can also say bye-bye to secure storage of fingerprint data and credit card data on iPhones.

It's all about the precedent here and it's an _extremely dangerous one_. As Apple said, too, if the FBI succeeds in getting Apple to push _malicious updates_ to their users, who knows what kind of other similar requests will come from the FBI - not just to decrypt local storage, but also to break end- to-end encryption for communications and so on.

FBI's fight is literally a fight against good security.

</comment><comment><USER3> I don't think Apple will lose.

First, there is an obvious pre-technology version of this problem: I have some information in my brain, and only in my brain, that I refuse to tell you. You are unable to pursuade me to cough it up. Therefore, you never get the information.

The FBI should be forced to treat an encrypted phone as being equally opaque and inaccessible. You can see the phone, you suspect it contains something you want, you can't get it. You can see my head, you suspect it contains something you want, you can't get it.

Besides, Apple has the money to fight forever in legal proceedings (shouldn't be hard...seems the FBI had to drag out an 18th-century clause to even make the request; it should only become more comical from here on out).

Also, phones are sold everywhere in the world. You don't even need a phone to be a "phone" because anything with WiFi access can effectively communicate; it doesn't matter if the device was purchased domestically or even has the ability to connect to cellular networks. So yeah, given these options, terrorists will obviously decide to only purchase a Certified Backdoored U.S. Cellphone™ for all their illegal activities, right!?

<comment><USER4> Just playing Devil's Advocate

Why would should your phone be treated as 'equally opaque and inaccessible' be any different to the belongings in your house? Your office? Your car? Digital representations are not 'thoughts', they are an encrypted form of a physical thing (by encrypted I don't mean technological encryption, but we can't read the 0s and 1s without a app to decipher). How would an undeveloped roll of film be treated in a similar case? Would the FBI just go 'oops, not developed yet, guess we can't see what is in there'. No, they'd develop the film, plain and simple.

Just to be clear, I'm NOT suggesting we create back doors, but I don't see any physical object containing data to be the equivelant of what is in our minds.

More troubling, what happens when we create mind-readers? That's where things get really edgy.

<comment><USER3> Undeveloped film doesn't meet the definition of inaccessible if it's sitting on a desk in your house. It would have to be a roll of film that was buried under a million tons of concrete — you may know exactly where it is and suspect that its contents could be useful, yet you also know that it's effectively _gone_ and there's no way it's ever going to be retrieved. Encrypted devices have to be treated like they're buried forever.</comment></comment></comment>


The Math Problem That A/B Tests Should Be Trying To Solve - <AUTHOR> http://en.wikipedia.org/wiki/Multi-armed_bandit

<comment><AUTHOR> Just to be clear: I'm not necessarily suggesting that full multivariate A/B testing is the best thing to shoot for, I realize very well that especially in site design, different splits cannot be composed together.

However, related to a discussion this past week where it became clear that periodically checking confidence intervals until they "hit" was a terrible way to test because of the high likelihood of picking the wrong result, I _am_ suggesting that the two-armed bandit problem should be considered mandatory knowledge for anyone working on A/B testing frameworks. That problem is _literally_ what A/B testing tries to solve (try to make choices between two options to maximize payoff when you don't know the payoffs of either choice beforehand, taking into account the fact that trials spent building up certainty by testing the payoffs are sometimes wasted on the least valuable option).

I'd have linked to the two-armed bandit problem if there was a page on that, but it just redirects...</comment>


ZEI° time-tracking totem makes hourly billing a fun IoT activity - <AUTHOR> http://www.theverge.com/2016/4/19/11459504/zei-time-tracking-totem-iot <comment><USER2> I know this - it was featured on PH a couple of days ago. Great tool!

</comment><comment><USER3> Definitely getting this for my team!

</comment><comment><USER4> This is such an awesome idea!

</comment><comment><USER5> wow! this will be great for freelancers!

</comment><comment><USER6> brilliant</comment>


LA is dumping millions of plastic balls into its reservoir to tackle the drought - chris-at http://www.sciencealert.com/la-is-dumping-millions-of-small-plastic-balls-into-its-reservoir-to-tackle-the-drought?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+sciencealert-latestnews+%28ScienceAlert-Latest%29 <comment><AUTHOR> They're not dumping plastic balls in order to tackle the drought, they're being dumped because the EPA mandated that all reservoirs be covered. Direct sunlight will cause bromide and chlorine in the water to mix into bromate, a known carcinogen, and the EPA wants to prevent buildup of this compound in the drinking water supply. Covering the reservoir with a tarp is too expensive (costing over $300 million) and plastic balls are a much cheaper option. The evaporation reduction is a secondary benefit, not the primary one, and the article title should reflect this.

There's much better coverage of this story here [1] and here [2].

[1] [http://www.npr.org/sections/thetwo- way/2015/08/11/431670483/...](http://www.npr.org/sections/thetwo- way/2015/08/11/431670483/la-rolls-out-water-saving-shade-balls)

[2] [http://news.nationalgeographic.com/2015/08/150812-shade- ball...](http://news.nationalgeographic.com/2015/08/150812-shade-balls-los- angeles-California-drought-water-environment/)

<comment><USER2> > _Covering the reservoir with a tarp is too expensive (costing over $300 > million) and plastic balls are a much cheaper option._

This is what I don't get in this story. How on Earth are plastic balls cheaper than just covering the thing? One would think that a tarp would use less material and be easier to manufacture than the equivalent number of plastic balls.

EDIT: Thanks for all your answers. I guess the heatwaves we have right now in Poland are negatively affecting my imagination :(.

<comment><USER3> Will tarp be a single piece or multiple pieces? What's the weight of the tarp? how do you make sure it's on the surface? Or do you need a support structure to keep it above the surface (say a meter etc.)? Will it have holes to drain? Will holes be sturdy enough in the long run? etc...

<comment><USER4> I was thinking why they use ball form instead of something sheet-like. E.g. thin pancake - it will have same properties as the balls but will cover surface more efficiently, if you count material.

</comment></comment></comment></comment><comment> david-given This may seem like a silly question, but...

Why on earth are you chlorinating the water _in the reservoir_? I've never seen any other country that does that. Isn't it a really, really terrible idea? Environmentally ghastly because you kill everything in the reservoir, expensive due to chlorine loss, subject to side effects (like this one), and ideally suited to breeding chlorine-resistant bacteria... not to mention giving you nigh-undrinkable water!

I'm from the UK, and we chlorinate our water pretty heavily, but even we don't do this.

(I now live in Switzerland. Water in Zürich is purified via activated carbon, live biofilm and ozonation. The water which comes out of the tap is unchlorinated and is better and more drinkable than most bottled mineral water.)

</comment><comment><USER5> I think the story sounds a bit ironic:

So you are essentially dumping objects made out of chemicals, you had to drill wholes in the ocean ground to get, into a lake, because you found out that the chemicals you dumped into it earlier kill you when the sun shines on them. You also think that this might help you with the water shortage caused by unsustainable use of resources.

To me this all sounds like we arrived at a local maximum here. This simply cannot be the way to do this. Maybe we should start over.

<comment><USER6> But we protected ourselves from diseases with the chlorine. That's a massive win. Then we discovered there's a new risk of disease with the bromate, so we protect ourselves from that too. Isn't this basically how most technology develops? We keep getting better and better off because of it.

<comment><USER7> There are countries that don't use chlorine: Switzerland, Austria, Germany, Sweden, ... and they're regularly touted as having the best tap water quality in the world.

<comment><USER8> Ozone purification is cheap but could allow for future contamination if there is backflow or possibly a leak in the water mains. Chlorination prevents anything from growing in the water from the filtration plant to your home.

One problem that major coastal cities have is that they collect water from streams that have already been polluted and are on their way out to the ocean. The US has a lot of these cities, 40% of the population occupy the coasts which is only 10% of the country. The cities in the US that typically are regarded as having the best tap water are midwestern cities, ones that do not have a coastline and tend to get cleaner water from natural sources. Switzerland, Austria, and Germany are exactly the same. They are very central countries with nearby access to Alpine runoff water, the water they get is pretty unmolested when compared to what the Potomac River supplies to Washington DC.

</comment></comment></comment></comment><comment><USER9> I wouldn't be surprised if the balls increased the evaporation rate, for the simple reason that it will increase the surface area of the water (the balls will rotate in the water, bringing wet surfaces up). Black balls will also heat up more, causing more evaporation.

<comment><USER10> > Black balls will also heat up more, causing more evaporation.

Question. Overall, is the heat absorption of the black balls greater than open water? I imagine it's about the same, but perhaps the issue is that the balls are concentrating the heat at the surface, and on the surface of the rotating balls. So it does seem like the evaporation rate could well be higher.

I suppose it comes down to the mean rotation rate of the balls.

<comment><USER11> Smart thinking. The balls could be slightly weighted to avoid this issue, albeit this could mean the balls collect a rim of sludge more easily.

<comment><USER9> The weighting won't prevent them from sloshing around due to wave action and getting the upper surfaces wet anyway.

<comment><USER12> They'd be fine. For example, the tops of these ducks are dry:

[https://i.imgur.com/RYyBMrf.jpg](https://i.imgur.com/RYyBMrf.jpg)

In a reservoir you're not going to get waves that manage to spin the weighted side of the ball up any considerable amount, even if it's only slightly weighted.

<comment><USER12> Actually maybe I'm talking out of my behind -- what if there was wind? Add more weight then. Obviously a weighted oblate spheroid, like the ducks, would definitely work.

</comment></comment></comment></comment></comment></comment><comment><USER13> What I don't understand is that they dumped black balls in the reservoir. If you want to keep the water cool to lessen evaporation wouldn't it make sense to use white or reflecting balls so the water stays slightly cooler?

I would be glad to know why black balls are chosen.

<comment><USER14> black is best suitable color to deflect UV rays.

<comment><USER15> Now you just said the opposite of what intuition suggests without any evidence, or even argument. Please elaborate and don't just heat up the discussion!

<comment><USER16> UV + Chlorine = Bromide

<comment><USER15> Does that look like an explanation to you? I don't see anything about black. I don't see anything about white. I don't see anything about reflecting materials. I see something from chemistry, maybe?

Customer: "Why should I buy the product?" Developer: "Here we had a problem with the inheritance tree. We worked around it with a factory method."

</comment></comment></comment></comment></comment><comment><USER4> Looks like 3d simulation example came to life :)

But seriously, why balls are used? They say it's cheap but 36 cents doesn't look too cheap on the hundreds of millions scale. Wouldn't it be better to use sheet-like form? It will be covering space more efficiently.

</comment><comment><USER17> I wonder if floating solar PV was considered as an alternative/complementary solution that might pay for itself over time:

[http://www.eastgreenenergy.co.uk/for- business/products/float...](http://www.eastgreenenergy.co.uk/for- business/products/floating-solar-pv/)

</comment><comment><USER15> Do they really need to invent that much? It's true that the current situation is maybe the worst they ever faced. But there are a lot of places where water is even more valuable as a resource. Are they also looking into what people in the desert do to keep their water sources alive, etc?

<comment> martin-adams I do wonder if it's a different set of challenges. The demand for water in California is probably much greater, rather than adapting everyone's use to the climate. So I suspect the innovation is to sustain the level of water usage that isn't seen in the desert.

<comment><USER15> Hm, okay. I guess that's the American Way to approach such a problem.

</comment></comment></comment><comment><USER18> This chlorine everywhere in the US astonishes me. When visiting I'm not able to drink (non bottled) Coke b/c of the bad and intense chlorine taste.

</comment><comment><USER19> "in which a total of 96 million balls have been poured"

"they're cheap (costing just 36 cents per unit)"

Is that really cheap, or is it just because it's in LA?

<comment><USER20> As sibling poster [1] mentioned, the balls are cheaper (and probably last longer) than the tarp alternative.

[1] [https://news.ycombinator.com/item?id=10052777](https://news.ycombinator.com/item?id=10052777)

</comment></comment><comment><USER11> The article title is misleading. If we assume water price of $1000/acre-foot, then they spending ~$30m today to save 10x$2m over the next ten years. Water Davis are clearly a by-product benefit (as the article states) and the balls are not being used to tackle drought.

</comment><comment><USER21> 10 Year lifetime? Just the lifespan of these plastic balls being beaten with sunlight?

</comment><comment><USER22> [http://imgur.com/JagJndf](http://imgur.com/JagJndf)</comment>


Denser 3D Point Clouds in OpenSfM - <AUTHOR> http://blog.mapillary.com/update/2016/10/31/denser-3d-point-clouds-in-opensfm.html <comment><USER2> COLMAP is an open source structure-from-motion and multi-view stereo software and is hopefully useful for people here as well: [https://colmap.github.io/](https://colmap.github.io/), [http://people.inf.ethz.ch/jschoenb/colmap/](http://people.inf.ethz.ch/jschoenb/colmap/)

</comment><comment><USER3> I discovered VisualSfM this weekend after trying Adobe ReCap 360. The photo reconstruction functionality of ReCap is cloud based, which was a blocker for me, because I have a beefy graphics rig and want to script it into a pipeline.

I am curious how OpenSfM compares to VisualSfM and if they are related in any way.

Edit: The VisualSfM license is only open for non-commercial uses. OpenSfM is BSD licensed, so I can safely build my business on top of it.

<comment><AUTHOR> Both VisualSfM and OpenSfM use similar methodology (incremental structure from motion) to reconstruct 3D scene points and camera positions from photos.

And yes, you can build your own tools (even commercially) with OpenSfM.

<comment><USER3> I am curious how the quality compares and if they have common library dependencies.

</comment></comment></comment><comment><USER4> For those who have not tried it, OpenSfM is an amazing tool. Can't wait to play with dense reconstructions. Congrats mapillary team, I know this has been a long time coming!

</comment><comment><USER5> I wonder if you wouldn't get cleaner results if you eliminated points that don't have an adjacent point nearer than a certain distance?

<comment><AUTHOR> Yes. We can apply different heuristics to remove outlier points. One interesting way to do it will be to train a neural network to refine the dense point cloud.

</comment></comment><comment><USER6> Do they use the same NeuralNets that are used to improve videos on these pictures to reconstruct the details?

</comment><comment><USER7> How well would this work in a situation with artificial (moving) lighting? I have seen reconstructions for wrecks underwater, and we are interested in doing something similar.

<comment><USER8> It is possible to correct for all sorts of different lighting conditions. See here: [http://grail.cs.washington.edu/projects/timelapse/](http://grail.cs.washington.edu/projects/timelapse/)

</comment></comment><comment><USER9> What techniques are available to transform point clouds like these into closed polygonal meshes (like the ones you see in Google Earth where 3D imagery is available)?

</comment><comment><USER10> Is it possible to use images from say Google Street View to reconstruct 3D city model with this tool? Maybe it could fall into fair use for small part of city...

<comment><USER11> You'll want to use the OpenStreetView dataset for that, which has a much more liberal license for collected images than Mapillary. (OSV is a direct competitor to Mapillary, with OSV being sponsored by Navteq)

[http://openstreetview.com/terms/](http://openstreetview.com/terms/)

You might also consider mining Flickr by geotag, constraining to CC licenses that allow for use.

<comment><USER12> Or even just directly capture geotagged photos. That's going to be the best way to ensure good coverage of the target area.

<comment><USER11> What would the capture source be if not an existing online repository?

<comment><USER12> A camera.

That's where Mapillary and OSV get their imagery from, people that decide to go out and take pictures.

An additional advantage is that you then have full rights to the imagery (so for example, you can upload them to both services).

<comment><USER11> That'd work of course! Bit more of a schlep to build out your processing pipeline to ingest those images, store them, and then render them with OpenSfM, but could definitely be done.

<comment><USER12> Oh yeah, it'd be quite some work, especially if small isn't small. But I think at the moment, neither OpenStreetView nor Mapillary can be relied on for coverage, so the shortcut of using images gathered by others isn't widely available.

</comment></comment></comment></comment></comment></comment></comment><comment><USER13> very cool. that's quite dense.

So the segmentation convnet is applied on the 2d image pixels before SfM, rather than on the point cloud data? I've been looking for neural net semantic segmentation for 3D point cloud data

<comment><AUTHOR> Exactly, the convnet is applied on 2d pixels instead of on 3D point clouds. If one have labelled 3D points (e.g. if a point is part of a car or not), one can train a network for recognition directly on 3D points.

<comment><USER13> Nice. I wonder if this could be used for robot navigation. The segmented point clouds especially are useful.

Been looking forward to a DIY Roomba project using point clouds and neural nets for navigation

</comment></comment></comment><comment><USER14> Never heard of OpenSfM before but have been looking for something like it.

<comment><USER15> I have personal and professional projects that involve mundane data feeding and service-building around this core capability. This totally fills a niche that's been on my tech radar for a while.

</comment></comment><comment><USER16> So why can't this be used instead of LIDAR for Self-Driving cars?

Experts?

</comment><comment><USER17> Pretty amazing stuff!!

</comment><comment><USER18> great</comment>


How I Came to Work at the Wendy's - <AUTHOR> http://imgur.com/a/mTWpc

<comment><AUTHOR> artists site (not single page format): <http://www.nickstjohn.net/w1.html></comment>


Another way of thinking about the European economic collapse - <AUTHOR> http://marginalrevolution.com/marginalrevolution/2013/04/another-way-of-thinking-about-the-european-economic-collapse.html <comment><USER2> There is no "European economic collapse". Out of all the ways of generalizing about Europe, this is one of the silliest.

It's not exactly boom time, but most Western European economies are far from collapsing, and should the worst case happen and the euro fall apart, it is still far from certain that it will drag the stronger economies down with it.

In the mean time most people in Western Europe suffer far less from the economic crisis than most Americans and there's actual growth in Eastern Europe. Greece is an outlier, there may be more artificially propped up economies to follow and it may have wider repercussions because of the euro, but until then the great "European economic collapse" is a fiction.

For most of Western Europe, life is still good, money a little tighter and they may have to skip that yearly 2 week vacation in the Turkish sun. And this is ordinary working class we're talking about, not the elite. Collapse my ass.

<comment><USER3> Defining how far or close economies are from "collapsing" will boil down to subjective semantics, but there is at least one metric by which things still seem to be trending worse in the EU; US unemployment is off record highs and job growth is continuing, albeit slowly, while the eurozone is still continuing to reach record unemployment rates[1]

[1][http://www.nytimes.com/2013/04/03/business/global/unemployme...](http://www.nytimes.com/2013/04/03/business/global/unemployment- in-euro-zone-reaches-a-record-high-of-12-percent.html?pagewanted=all&_r=0)

<comment><USER4> Keep in mind though that since Europe has virtually no hiring-at-will the unemployment rate is actually a lagging indicator.

<comment><USER5> Employment tends to be a lagging indicator even in the US.

</comment></comment></comment></comment><comment><USER6> The article falls apart with all of the unspoken assumptions in this sentence:

"If the relatively flexible U.S. lost 8% of median income, perhaps Italy and Spain and Greece have to lose 15%"

Why did the USA lose 8%? Did God say that the USA must lose 8%? Could different policies have lead to a different outcome? Perhaps with different policies, the USA would have lost 20%, or only 2%.

There are some interesting thought experiments you can do with economics where you start with the premise "All other things being equal, let us compare these 2 variables..." but that kind of thought experiment does not work in this situation, because if the first condition is highly variable and subject to circumstance, then there is no basis on which to make a comparison. It's like saying "Given a lamp, compare an octopus". There is no relevant point from which you can make a comparison.

There is the old joke that 2 economists, left alone starving on a desert island, but with many canned goods, could survive fairly well. The one economist simply says to the other "Assume a can opener."

What if, in real life, different fiscal or monetary policy would have cushioned the lost of income in the USA? Then we can assume the same might have been true in other nations.

The point is, there is no reason to assume that inflexibility is the main variable, unless you artificially constrain the scenario by first saying "Assume all losses were pre-destined." But in real life, they were not pre- destined.

Real life is not so easy on assumptions.

<comment><USER7> I believe that 8% is not merely an assumption, but comes from the paper he references here:

[http://marginalrevolution.com/marginalrevolution/2013/04/tra...](http://marginalrevolution.com/marginalrevolution/2013/04/trade- vs-technology-in-terms-of-their-labor-market-effects.html)

</comment></comment><comment><USER8> So, his argument is that the downturn is caused by sectoral imbalance, rather than AD. That's a falsifiable claim - but he doesn't give any numbers.

</comment><comment><USER9> Here's an in depth analysis by a more resourceful writer.

<https://news.ycombinator.com/item?id=5525422>

This was posted yesterday, but didn't get enough votes.

<comment><USER5> Soros is not only providing more insight, he has a concrete solution.

</comment></comment><comment><USER10> This article ignores a key difference between England and the rest of europe: the euro. The euro countries are revenue constrained in the same way as U.S. states. They owe Euros, but cannot issue Euros, so they must go begging to the market for Euros else default.

England (BOE) as monopoly issuer of the Pound, can dictate interest rates on Pound denominated bonds, therefore they do not have the same constraints on government spending.

</comment><comment><USER11> What Krugman said here applies to the EU too (infact he's basing it on IMF research using global data):

<http://krugman.blogs.nytimes.com/2010/08/12/the-price-> stability-trap/

"in the modern world, rapid deflation doesn't happen, and in fact slight positive inflation often persists in the face of an obviously depressed economy"

</comment><comment><USER12> [meta] I'm always pleasantly surprised and enjoy reading the discussion when Marginal Revolution articles gain traction on HN.

[not-meta] Can someone explain (to a 5 year old) the AD-AS model that's mentioned in the article?

<http://en.wikipedia.org/wiki/AD-AS_model></comment>


Keystone pipeline spills more than 350k gallons of oil in North Dakota - <AUTHOR> https://thehill.com/policy/energy-environment/468353-keystone-pipeline-spills-over-350000-gallons-of-oil-in-north-dakota <comment><USER2> Of course, it did. It was only a matter of time. My surprise is that it happened so soon.</comment>


A CS Education That's Free Until You Get a Job – Austen Allred of Lambda School - <AUTHOR> https://blog.ycombinator.com/a-cs-education-thats-free-until-you-get-a-job-austen-allred-of-lambda-school/ <comment><USER2> On the one hand, I like that incentives are aligned between educator and student. On the other hand, if you are smart and self-motivated, paying the max fee of $30k over 2 years is like taking out a 45% interest loan over the $20k prepayment option.

</comment><comment><USER3> Better: A CS Education That's Free And Gets You A Job

</comment><comment><USER4> Similar company in India. [https://pesto.tech/](https://pesto.tech/)

</comment><comment><USER5> Isn't this Revatures model?</comment>


Competition & Class - <AUTHOR> http://restrictionisexpression.com/post/2745382434/competition-class

<comment><USER2> He comes across as very whiny. The tactics he mentions seem pretty valid to me (and if he's mad, they must be working to some degree).

He should be happy Twitter haven't decided to roll those features into the platform (yet) and put him out of business altogether.

<comment><AUTHOR> (I think I'm the he?) Happy to debate the merit of the tactics (though metrics & past experience suggest to me they are ineffective). I took issue with "small guy vs small guy". I have no large pie worth stealing, and neither will a competitor if they focus their efforts on my piece.

Also re: Twitter. I'd be thrilled if/when they roll it into the platform. I built Proxlet to solve a very real problem, not to build an empire.

</comment></comment><comment><USER3> I generally agree. Ads targeting competition are fine, depending on the copy.

The only thing that has calmed me in similar situations is knowing that competition that does things without class doesn't do well. If your marketing is around a very small competitor instead of the usual alternative (nothing), then you're incompetent. If your product design is based on copying the competition, then you're incompetent.

</comment><comment><USER4> All's fair in love and war, and there's no point getting bent out of shape about it.

That said, spending time & money on these sorts of tactics is almost always less efficient than spending time & money improving your product. Every hour you spend improving Proxlet while your competitor's fussing with AdWords and downvoting on Quora is an hour you're winning.</comment>


Building Grove – interactive trees that come alive to your breath at Burning Man - <AUTHOR> http://www.ofbrooklyn.com/2017/03/31/building-grove-burning-man-art-installation-2016/ <comment><AUTHOR> Hey HN - a couple years ago we built Pulse & Bloom and that got some good discussion going on HN: [https://news.ycombinator.com/item?id=8285228](https://news.ycombinator.com/item?id=8285228)

I wanted to share my experience building a large scale art installation, showing what went right and what went wrong. It's always helpful to have a nice bog document written by somebody who's done it before.</comment>


Ask HN: Is there a website with advanced CSS puzzles? - <AUTHOR> I am looking for question like :
1) How would you transform this div from middle of page to end of page etc 2) Why is this div only 20px height even though the image inside is 100px height
3) Given this moderately complex CSS, how will you add a extra text div of dimensions 20px * 30px and how will you style it with minnimum classes.
Something like this: https://www.w3schools.com/css/exercise.asp?filename=exercise_positioning5 <comment><USER2> Highly recommend Flexbox Froggy ([https://flexboxfroggy.com/](https://flexboxfroggy.com/)) for learning about Flexbox. There's also Grid Garden for learning CSS Grid ([https://codepip.com/games/grid-garden/](https://codepip.com/games/grid- garden/)).

</comment><comment><USER3> I'm not sure if this is what you're looking for, but if you want advanced, you can go to Codepen and recreate some of the things there, then look at the code for answers.

<comment><AUTHOR> This is an oft repeated suggestion. But the problem is that, there is very little step by step commentary on these codepens. Its mostly only the final step that is presented. I tried recreating some and then got stuck with finer points of positions, margin-auto etc. Thats when I realised I need to solve some advanced CSS puzzles. Thanks anyways!</comment></comment>


Taxi ride in Flanders? Legally mandated fifteen minute wait - <AUTHOR> https://translate.google.com/translate?sl=auto&tl=en&u=https%3A%2F%2Fwww.tijd.be%2Fpolitiek-economie%2Fbelgie%2Falgemeen%2Fuber-rit-in-vlaanderen-eerst-een-kwartier-wachten%2F10203567.html <comment><AUTHOR> TLDR: If you book a taxi by phone or online, the driver is legally obliged to only start 15 minutes after that.</comment>


The Homeless in Silicon Valley - <AUTHOR> https://newrepublic.com/article/124476/dispossessed-land-dreams <comment><USER3> I live in a van in SF and I'm a software engineer. I park in the Sunset and ride a bike to work. I have a gym membership with towel service at a major gym chain. I have two comped meals a day at the office. I make more than $150K, but less than $200K. I go out drinking every night and wind up at a bar in the Sunset with clean bathrooms. On weekends I drive out of town and camp somewhere scenic. I'm saving 80% of my take home pay. I have been doing this for six months. I date regularly. On the first date I state that my place is a no-go. Camping is a good second or third date. If they are special, I explain that I live in a van.

My van was towed once when I was out of town and construction notices were placed after a few days. That stung. Otherwise, it has been fine. If everything in the van was stolen I'd be out $1,000.

<comment><USER4> I live in the sunset district- there are a bunch of vandwellers who do this in our hood and it's frankly disgusting. They urinate out the side of their van in the gutter and leave their feces in the gutter. Perhaps YOU are clean but there are plenty of people who do it as well who are not as conscientious. It's making the neighborhood pretty disgusting.

You're telling me in the middle of the night if you wake up and have to pee, you're really going out to a bar and finding a restroom? I think not. At worst you're peeing onto the sidewalk or into a bush and at best you're peeing into a bottle in your van and then dumping it out on the street.

I report anyone I can tell living in a van on our streets to the non-emergency PD line, and they come out and they make them move.

<comment><USER5> I think this is much more than a local Silicon Valley thing.

I have noticed similar things in Europe and Australia.

As I see it this is a symptom of housing becoming too expensive. By a perverse logic the only thing the powers that be do is to make sure that housing becomes even more expensive. In that way people buying too expensive houses are saved by house price appreciation.

How this will end is no longer obvious too me, however people in top paying jobs living out of hostels or vans for me is a sign of erroding real wealth in our society.

<comment><USER6> I think we just have to build much more housing. The housing prices are too high and simply must go down. It will suck _a lot_ for all those people who bought houses in the past 20 years, but it's the only option that is good for society.

The political will is a big problem. Homeowners are much more likely to vote than people like me. In San Francisco, the latest new regulation/roadblock is that any increases in waterfront height limits have to be approved by voter proposition. That regulation was passed by simple majority, 8.5% of San Francisco residents voting in favor. This is pathetic. We need to be involved in reforming our neighborhoods.

</comment></comment></comment></comment><comment><USER7> > That homelessness persists in Silicon Valley has puzzled me.

The reason is (as the paragraphs just below this line indicate) in fact the same reason for a great deal of issues in the Bay Area: lack of regional planning.

Planning in the Bay Area is far too localized. Cities constantly pawn off problems on each other. There are cities that approve thousands of new units of office space but zero housing. Transit systems don't connect to each other. Sports stadiums are constructed with no thought as to how people will arrive. Freeways have lanes that appear and disappear at various county boundaries, causing dangerous congestion. On and on. It's also why so many cities don't have enough shelters and just seem to expect (as the article states) that other nearby cities will pick up the slack.

What is needed is for the disparate cities of the Bay Area to stop passing the buck and come together as a region and plan together. Continuing to hope someone else does the hard work of solving problems will continue to result in them not getting solved.

<comment><USER8> The real question is how do we push for that to happen. Most of the tech people I know move around- they rent one place for a few years, then end up in another part of the bay shortly after that. There's no real incentive for the politicians on the local level to give up power, and the state doesn't seem to care.

Regional governance seems to be the answer, but making that happen seems like an impossible task.

<comment><USER9> you could de-regulate the planning all together and the the market decide the best way to allocate resources...

<comment><USER10> The market decides human beings are not worth shelter, and you're suggesting it needs _more_ power? What the _fuck_?

<comment><USER11> The market certainly decided that human beings ARE worth shelter and more shelter should be constructed. More supply will result in lower prices. However, cities are barely approving residential zoning and this artificial distortion is amplifying the homeless problem.

As we have seen here, regulation like rent control is much more harmful than helpful. Rather than needing to cough up an extra $50 per month for example, you wait until the market rent is double what you're paying now - <USER2> naturally, get kicked out for one reason or another.

If zoning was dramatically curtained and / or abolished, the housing problem would be solved almost overnight. The high property prices ARE a market signal saying, build more houses. Regulation is preventing efficient markets.

<comment><USER10> Certainly building more housing will bring down the price, and that will make life easier for tech workers who will still be well-paid. But that won't do anything for the homeless who are unemployed or making minimum wage, who are still going to be vastly outbid compared to tech salaries.

Given the geographical constraints and the velocity of tech growth, it's really unlikely that a free market would build _so many units_ that all the high-paid tech workers who could possibly come to the Bay Area were situated, and landlords were forced to start catering to the working poor. We have to force them to, because market incentives won't.

<comment><USER12> Companies could build offices in other towns. That's an option too.

<comment><USER13> Oddly, one way to encourage that is to make the permitting process easier. Tech companies want to all be in the location, because its much more desirable to tech workers, and the belief that the best talent is worth the price means that money is made available for that.

Without the restrictions on development (which are designed to keep the area desirable, and succeed at that, as shown by market prices) desirability would drop -- because you'd have more people, and less money for services per person -- along with prices, and tech companies would be more likely to locate elsewhere.

<comment><USER12> You'll get no argument from me on that front. I still believe tech companies should go with other locations now because there is already incentive and opportunity to do so.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER14> Palo Alto, Mountain View, and other peninsula communities don't spend money on homeless shelters for two big reasons:

1) San Francisco spends a ton of money on the homeless. Over $40k per homeless person per year. "If big SF spends so much on the homeless, why should little ol' us?"

2) The weather in the Bay Area is not particularly harsh. Yeah, it might get cold 1-3 days a year, but it's not really bad. You're not going to die from exposure to the elements like you would in Cleveland, Detroit, or even Dallas. "No one will die sleeping outside here" is a common thought.

I'm not agreeing or disagreeing with these thoughts, but this is what peninsula planners think.

<comment><USER15> Citation needed for "$40k per homeless person per year". I used to work directly with homeless people and that doesn't sound even remotely true. And even if you have the top line figure correct I suspect there's some missing context.

<comment><USER14> $189M in the city's budget goes directly to homeless services. That's roughly $25k per homeless person in direct spending. The city estimates it spends another $20k per homeless person in indirect city services (health, police, fire). The "frequent fliers" cost the city more than $100k per year.

In many ways, $40k per person is a low-end figure.

<comment><USER16> Mayor just committed to $250MM/yr ([http://www.sfmayor.org/index.aspx?recordid=1017&page=846](http://www.sfmayor.org/index.aspx?recordid=1017&page=846))

</comment></comment></comment></comment><comment><USER17> I'm mixed on the issue. I've got roots going back multiple generations in the bay area. The only immediate family member besides myself still living in the "bay area" (monterey to Santa Rosa) is one of my Aunts. Both parents born/raised in the Bay Area, my sister and I as well. Choices made, people moved. There is a golden rule - "if you sell your house and move out of California, it is hard to move back". My parents found this out in spades. They live in WA now and I'm supporting them a bit.

Old friends of my family still own a house in Campbell. Retired now, even with ups and downs, their mentality is "we aren't splitting up, because we can't afford to go anywhere". The build out of the Bay Area, lack of planning, etc. is taking a toll and making it hard on current and retired blue collar workers.

That said, I live in Santa Cruz and have very little pity on the "I should be allowed to live where I want". Mostly because over here, our "3000+ homeless" are 2/3 (according to a UCSC/Civinomics survey) didn't reside here before becoming homeless. We have way too many services, few rules, and a lot of "compassion" \-- to the point of people being publicly called out if you publicly ask about "compassion".

The Bay Area is expensive, yes one would like to stay where their friends are, but one should also be open to where they can afford. For instance, locally, we keep being lectured about needing "safe places for RVs to park", yet these RVs aren't local, and a simple search of CL, a trailer park in Modesto has spaces (including utilities) for $15/day.

</comment><comment><USER18> I feel somewhat terrible for thinking this… but, to me this story seems like a sad example of why responsible financial planning is important. It seems like they had plenty of opportunity to build equity into their house over time. Taking out multiple mortgages and refinancing seems like dangerous behavior.

Also, why is living in a van preferable to moving somewhere where they can afford the housing?

<comment><USER14> Unexpected, high medical bills and crappy insurance coverage would probably bankrupt most Americans. A really unfortunate hospital stay can ruin many people's lives.

<comment><USER19> Even when your insurance is 'top of the line', if it's cheaper to hire lawyers than to treat you, then they might just hire lawyers. Insurance has a big incentive to not provide it's service when the cost is high.

And this can happen to anyone with the right kind of car accident.

</comment></comment></comment><comment><USER20> There are 35k homeless people in the Bay Area [1] 7-15k of them in SF. There are ~7M people living here.

While those numbers sound at first high. It's not such a crazy high amount of people. (from what i know cities like Vienna or London also have at least a forth of that number (couldnt find stats))

The "homelessness situation" is next to cost of living the #1 thing i hear from people who don't want to relocate here. And tbh it's also the reason why i dont plan to stay in SF for longer term.

I know there is no easy answer to the problem of homelessness in the bay area - but it doesnt sound unsolvable.

[1]: [http://www.myphilanthropedia.org/top-nonprofits/bay- area/hom...](http://www.myphilanthropedia.org/top-nonprofits/bay- area/homelessness/2009)

<comment><USER15> The "homeless situation" is roughly the same in San Francisco as in every major city, where a near-constant 1% of the population are homeless. There a variety of reasons why homeless people are more visible in San Francisco than in other places, but there aren't more of them.

<comment><USER21> If the 35k figure is correct for SF, that means it's over 4% of the population.

<comment><USER20> 35k whole bay area

</comment></comment></comment></comment><comment><USER22> Two things irritate me. 1\. A food bank should never ask for a person's home address. If you need to use a food bank, chances are you're homeless.

2\. Homeowners should not worry so much about their property value. It's all going to be destroyed by an earthquake. I wager that the homeless are better prepared, and more likely to survive the Big One.

</comment><comment><USER23> OT to the article but following up on some other comments about van living. I don't think it will be long before one of the big tech companies opens a dorm. There's no way it hasn't been discussed somewhere already. There was an article last month on "dorms for grownups" [http://www.theatlantic.com/business/archive/2015/11/coliving...](http://www.theatlantic.com/business/archive/2015/11/coliving..). and the big tech companies already offer so many perks would access to a company dorm actually hurt at this point?

Of course this is common place in China already for the factory workforce...

<comment><USER24> I paid $100/mo for my corporate dorm in Japan. It was pretty dingy, but there were better, well designed studio and 1BR arrangements for $150 that would open up after a year or so that you could move into. I'd roughly estimate that this was 1/5 the cost of market housing.

I had to move out because the particular one I was in was too close to industrial smog, but I have friends who still live in a $150/mo corporate dorm after 6 years working there. Not a bad arrangement at all since many of us went home exclusively to sleep anyways.

</comment></comment><comment><USER11> It's time to index everything to an area's cost of living. $1 in SV is not the same as $1 in {somewhere else). It's time to stop pretending that you can set a federal poverty line, or set a federal social security pension.

</comment><comment><USER12> This is a pretty easy optimization problem and I think it's hilarious that SV can't seem to solve it. If this was a network you would just push the resources further out closer to the edge to alleviate the load in one place.

What you have is people from all over the world swarming SV because you can't get that job in Cork or Boise.

Everything doesn't need to be in silicon valley. If someone is willing to live in their car for a job, with all the social baggage, sneaking around and unhygienic side effects that entails, surely they would be willing to move to a less crowded place like Portland to do the same job.

If the nexus for every single tech company must remain in SV then these companies could set up satellite offices around the country or allow people to telecommute with robust teleconferencing setups that turn a monitor into a window. Then just hang it up on the wall in your office and turn it off when you're not working.

Seriously people. You don't have to live like animals. I have a decent software job where I live. I pay $700 rent and look what I get to do every weekend

[http://imgur.com/a/d5BkF](http://imgur.com/a/d5BkF)

[http://imgur.com/a/iFN9Z](http://imgur.com/a/iFN9Z)

[http://imgur.com/a/ow47m](http://imgur.com/a/ow47m)

<comment><USER25> "Praise The Sun!" Nice!

<comment><USER12> It was funny because I had to explain it to my buddy who took the picture afterward and I'm pretty sure he didn't get it.

We literally climbed in a shadow for 6 hours before getting into the sun. It was ridiculously invigorating.

</comment></comment></comment><comment><USER26> For those without a car or van, life can be even worse on Hotel 22 -- the only bus in Santa Clara county which runs 24 hours a day. People will ride it back and forth all night.

[http://www.nytimes.com/2015/01/28/opinion/hotel-22.html?_r=0](http://www.nytimes.com/2015/01/28/opinion/hotel-22.html?_r=0)

</comment><comment><USER27> This is really sad. Not to change the subject, but I'm wondering how difficult it would be to live in a Van in SV and work as a Software Dev?

The temperature seems right, how hard would it be doing a stealth van situation and showering at the gym?

<comment><USER28> I lived in my Honda Civic in Palo Alto for a while. It was a little stressful, but my total burn was ~$300/month.

Shower: YMCA on Ross Road Food: non-perishable stuff Work: Hacker Dojo.

I slept in a Mormon church parking lot most nights

<comment><USER29> Hacker Dojo and TechShop had to crack down on people sleeping there.

<comment><USER28> I never slept in the Hacker Dojo, I slept in my car</comment></comment></comment></comment>


Keeping the Keck Telescopes Running - <AUTHOR> http://darkerview.com/wordpress/?p=25813 <comment><USER2> I always wondered how old machines get fixed when replacement parts are no longer in production. This was very educational.

<comment><USER3> A lot of these types of installations are bespoke, at least in large part.

I used to have a friend that worked at a particle accelerator lab, they had a complete machine shop on site so that they could make repair parts, or new parts as needed for various experiments. These are not things you will ever buy at Amazon or even a place such as McMaster-Carr.

</comment></comment><comment><USER4> The act of unboxing a prototype PCB for the first time, soldering all the components up by hand at the hot air station and then switching it on for the first time (and cringing in anticipation of magic smoke or expensive bangs) is one of my favorite things, speaking as a software engineer who dabbles with hardware design in smaller projects for work.

It encourages really deep thought and full exploration of your design before "compilation" (spinning a PCB), something that I don't get as much with the quick turnaround run-error-fix-run cycle of the interpreted languages I develop in day to day.

It reminds me more of my really early days playing with Gentoo on a Pentium II with 128MB of RAM, where running _make_ had a real cost on a large project. It really encouraged you to think changes through instead of just slapping some changes in and running it to see if they were correct or not.

</comment><comment><USER5> That's a great web site. Worth the time reading through his posts.

I spent the last two weeks laying out a PCB with iPhone-like density; now I hate laying out PCBs.

It's therapeutic until you spend too long on it, then it's drudgery.

<comment><USER6> High desity PCBs are a challenge. Most of the time as hobbyists or for engineering prototyping that density is not necessary. Giving yourself 10% or 15% extra space is hugely beneficial to your sanity.

Production boards are a different story, but that isn't likely to be the first iteration.

Article:

> The details in a PCB layout seem endless, you have to check everything > thrice. Even after doing so you just have to accept that there will be some > small error. Hopefully something you can live with.

At the robot club I give these encouraging words to people trying PCB design for the first time: "PCB design is like golf. Low scores are better, but holes-in-one are rare."

A couple of wires tack-soldered on can salvage most boards.

</comment></comment><comment><USER7> Thank you for your service to science.</comment>


New Favourite Writing Font - <AUTHOR> http://toomuchlatte.com/my-new-favourite-writing-font/

<comment><USER2> Interesting write-up. I also recently began using this font in my code editor. Loving it so far!</comment>


For true panoramic images, toss this camera in the air - <AUTHOR> http://news.cnet.com/8301-11386_3-57611859-76/for-true-panoramic-images-toss-this-camera-in-the-air-seriously/

<comment><USER2> It's a camera you throw in the air and it takes a panoramic picture of the surrounding area.

The actual Indiegogo site: [http://www.indiegogo.com/projects/panono- panoramic-ball-came...](http://www.indiegogo.com/projects/panono-panoramic- ball-camera)

<comment><USER3> I don't think it's 3D, it's just a panoramic image you can scroll around.

<comment><USER4> Why not 3D, in fact, if it had accelerometers and gyroscopes? Just throw it at around 45 degrees of elevation, capture photos during flight at a few fps, and post-process the results for a 3D scene reconstruction. (You'd have to design it much more durable for that mode of operation, of course.)

<comment><USER5> This is a nice idea -- the biggest practical problem would be motion blur while the cameras are in flight. If thrown straight up, there's a moment of zero motion at the top that mitigates that problem, but it's still an issue the rest of the time -- fast shutter speed helps but rolling shutter complicates it. Note that you could still get plenty of parallax for 3D from throwing straight up, just not right above and below the camera path, so that minimizes the durability problem from landing on the ground. It also provides a nice prior estimate (and additional constraints) on camera poses for structure from motion as the ball has to obey gravity.

Edit: On the indiegogo page they specify throwing is mostly for outdoor use: "While throwing works great outdoors during daylight, you can also use the camera indoors or in the dark when mounted on a stick or handheld." This would be due to the motion-blur/shutter-speed/sufficient-light problem.

</comment></comment></comment></comment><comment><USER6> This isn't really a new idea - the military already have versions of these that they can throw into a room or inclosed space to gain optics on it upon before entering it to clear it. They also have versions that can be launched over a longer distance via grenade launcher.

eg [http://www.epicos.com/epicos/extended/switzerland/macroswiss...](http://www.epicos.com/epicos/extended/switzerland/macroswiss/products_mq.htm)

see also from 2008: [http://www.gizmag.com/the-i-ball-short-range-throwing- camera...](http://www.gizmag.com/the-i-ball-short-range-throwing- camera/10399/)

</comment><comment><USER7> I dream of a weighted version of this which I can use while skydiving.

IE: like a spaceball [https://www.youtube.com/watch?v=sRUoe1nPxOE](https://www.youtube.com/watch?v=sRUoe1nPxOE)

</comment><comment><USER8> Original website from 2011: [http://jonaspfeil.de/ballcamera](http://jonaspfeil.de/ballcamera)

<comment><USER9> I thought I had seen it before. So it was patent pending in 2011, guess it takes a while in Germany too.

One of the downsides of the new Kickstarter/Indiegogo etc world is I keep seeing article on new "products" which don't exist yet. Interesting that the time to print time has shortened and now come inside the 'time to produce' margin. I think we need some backup words for these things, like "Possibly a new camera that can take panoramas"

<comment><USER10> Well to be fair, the camera exists, you just can't buy it yet. But perhaps it would be good to have some way of differentiating headlines about buyable products from prototypes/research projects etc.

</comment></comment></comment><comment><USER11> A similar project, that also supports live video [http://bublcam.com](http://bublcam.com)

</comment><comment><USER12> It's a very cool product but panoramas still look awkward and awful to me.

<comment><USER13> Is it the really curvy bits at the bottom that bother you? You can fix it by using a different projection when rendering the spherical image to a flat surface.

<comment><USER12> Yeah in general the warping and disconnect from our normal perceptions; I'd rather just see a wide-shot landscape than something distorted and alien.

But I take your point, I have seen some that have been well-corrected and look pretty normal so I believe that it's doable.

<comment><USER13> The trouble is that the parts toward the bottom of the photo should really be beneath you! So you need an infinitely tall photo, or a screen that wraps around below your feet :)</comment></comment></comment></comment>


Getting into Google for average software engineer - <AUTHOR> Case Study : Average software engineer. Current : A software engineer 13 years of experience in average companies. Currently, on a tech lead role. Technology Stack: Java, Spring, ORMs, kafka, golang. Currently self studying machine learning concepts and data science Objective : To get employed in companies like Amazon, Google etc How : ???.. spent time in brushing DS and Algols ?
Someone , please guide me how to achieve my objective. <comment><USER2> I suppose this can be helpful to you [https://github.com/jwasham/coding- interview-university](https://github.com/jwasham/coding-interview-university)

</comment><comment><USER3> You could start by practicing your written English skills. This is generally a must-have to get into Google.

<comment><AUTHOR> Not amused..</comment></comment>


Show HN: Aiomas – An asyncio based library for multi-agent systems in Python - <AUTHOR> https://bitbucket.org/ssc/aiomas

<comment><USER2> Interesting work!

At first glance it would seem that Erlang or Akka provide a proven infrastructure for building such a multi-agent system+. Yet, a key construct is the notion of time and its passage.

(+Which is subtly different from an agent-based model, for simplicity let's assume they're both the same thing.)

There is a profound observation by Rich Hickey in his talk [1] 'Are we there yet? - A deconstruction of object-oriented time' (the baseball field slide): "Perception is massively parallel and requires no co-ordination - This is not message passing!"

For example if the said ballpark suffered an earthquake, would the Matrix need to pass messages to each agent? Would a spectator continue walking toward the hot-dog stand just because the message hasn't shown up yet?

From this perspective, the notion of 'container clock' presented here can be useful. Question is whether we 'get to stop the world when we want to look around' (Hickey), or not.

[1] [http://wiki.jvmlangsummit.com/images/a/ab/HickeyJVMSummit200...](http://wiki.jvmlangsummit.com/images/a/ab/HickeyJVMSummit2009.pdf)</comment>