AresDB: Uber's GPU-Powered Open Source, Real-Time Analytics Engine - <AUTHOR> https://ubere.ng/2HzMPVK <comment><USER2> When I worked at Uber, this project was openly mocked. One of the CTO's biggest failures was implementing a promotion scheme where you needed to create a new service in order to be considered "innovative". This promotion scheme marked what I consider the end of Uber's engineering excellence and the start of what made Uber turn into a bureaucratic mess.

One of the VP's of engineering called it "toil vs talent". People who "toiled" at work, meaning doing good maintenance work, would be rewarded with good bonuses but those with "talent" would be rewarded with promotions. Of course this drove people to come up with fake new services so that they could demonstrate "talent". This also lead to an explosion of new services that overlapped or did nothing useful. Instead of working together, groups would make new services instead of working with existing service-owners because they needed to justify writing a new service. It was sickeningly transparent.

This project was one of those projects. It has no real use case because why the fuck would we want to use GPUs except to look cool on your resume. The sad thing is that the projects is overstating how well it's being used internally. Internally people use Pinot instead of this.

For all you future CTOs, consider your incentive schemes carefully and don't be so far removed from the action that you can't see when your org is rotting. This is what the CTO did, and like I said, it was one of his biggest failures because it gutted the engineering org. Instead of working together, every team was looking at get promotions at the expense of the company and it showed.

<comment><USER3> I wish I had understood this earlier. My past company made a push to hire from top companies like Uber for some key positions. Some of them were great people who were relieved to be out of the FAANG rat race. Others were single-mindedly focused on rewriting everything they touched with cleverly-branded project names, regardless of whether or not it made business sense.

Early I on, I made a harmless comment in Slack about how one person's pet project wasn't a good fit for our needs so our team would be using the older, more proven solution. Later that evening the person pulled me aside, almost in tears, begging me to never say anything critical about his project in a Slack channel again. He explained that at his previous role, success or failure depends entirely on the perception of one's personal projects and that seemingly innocent comments could tank someone's promotion chances for years.

I felt bad for him because he had clearly come out of a toxic situation. However, one of his teammates later warned me that he was keeping a journal of potentially incriminating things that I had said in Slack and a detailed log of every issue that he could find with our team's project in case he "had to use it against me later".

I could never tell if this was a unique experience or the norm at some companies like Uber.

<comment><USER2> What you describe was definitely not a common occurrence. Engineering wasn't toxic for many years until the last 9-12 months or so I would say. Pre-Susan Fowler memo, it was the best company I had ever worked at. From 2017-2019 we sort of stalled because of the internal drama and it didn't get really bad until the last 9 months, where attrition of our best engineers and vile political maneuvering from the dregs made it too much for me to stick around.

The engineer you describe sounds like they have mental health issues. There may be some teams with terrible managers but all companies have this, and I've seen similar or worse situations at Amazon.

Most engineers I worked with were great but there were many engineers that "played the game" in order to get a promotion and more money. It was sickening but if that's the way the CTO sets the incentive scheme, who can blame an engineer for following it? It's more on the CTO for setting the terrible culture than the engineers.

<comment><USER4> Being blind to politics rarely ever means politics isn't going on.

Pretty much every company out there has a concept of _' promotion packet'_, its basically building a case for one's promotion. Of course in a company the budgets are fixed, and so are promotion cycles(yearly in most places). You miss out a turn, you could lose an year, or even risk losing two. In that case its fairly common for managers to build a list of accomplishments(file/packet), and rival managers to build a anti-case/defence for the same. Stack ranking eventually is all about a combination of merit+advocacy+lobbying+counter-lobbying at so many levels that I'd say the engineer who cried wasn't wrong at all.

This is the case in nearly every company. We just wish to delude ourselves that politics is absent at some places.

This sort of power play comes with the territory in a large people structure.

</comment></comment></comment></comment><comment><USER5> > Like Pinot, Elasticsearch is a JVM-based database, and as such, does not > support joins

Uh. What does the JVM have to do with the data model's ability to do handle joins?

<comment><USER6> Yeah that's a very odd statement, I mean, PrestoDB, Impala, KSQL...

</comment></comment><comment><USER7> GPUs and analytic DBMSes / query engines are actually my own field of research ([https://eyalroz.github.io);](https://eyalroz.github.io\);) and it's obviously beyond what a comment would encompass, but:

1\. There are very few analytic DBMSes which are actually fast (and compare against reasonable baselines). Most claims of speed are bogus. Or rather, might be better than what's otherwise available to use, but are still slow.

2\. Designing an analytic DBMS to properly utilize a massively-parallel processing device is a monumental task, and I would claim that it has not yet been undertaken. Existing research and production systems graft such use onto a system whose fundamental design dates back to the 1980s in many ways.

3\. CPU-utilizing anallytic DBMSes are typically faster than GPU-based ones, to a great extent due to the above - but also since we've had decades of work on optimizing them.

4\. GPUs are artificially handicapped on Intel-architecture systems, because they are placed "far" from main memory relative to the CPU. More literally - the bandwidth you get t between your GPU and main memory is typically 0.25x the bandwidth a CPU socket has with main memory. This is critical for analytic query processing (as opposed to neural network simulation which is more computation-heavy and can tolerate this handicap much better).

\--

PS - Always glad to discuss this further with whoever is interested.

<comment><USER8> How much would the trade offs change if GPUs shared the same main memory as CPU?

<comment><USER7> Not sure I understand exactly which trade-off you're referring to, but on systems without the GPU-handicapping (e.g. IBM Power; and also when you put link up many GPUs together with NVLink) - there is still a significant design and implementation challenge to produce a full-fledged analytic DBMS, competitive vis-a-vis the state-of-the-art CPU-based systems.

There are also other considerations such as: The desire to combine analytics and transactions; performance-per-Watt rather than per-processor; performance- per-cubic-meter; existing deployed cluster hardware; vendor lock-in risk; etc.

</comment></comment></comment><comment><USER9> Curious - what's the use case for an organization like Uber needing real time analytics at high frame rates? I noticed the emphasis on dashboards but was curious what a real-time dashboard at this scale actually ends up being used for.

Maybe my question is more around, what business decision would be impacted by not having real-time instantly reserved dashboards.

Honest question here not trolling.

<comment><USER10> All sorts of departments at Uber use real-time queries (operations, marketplace, eats, new mobility, their data science and growth group, finance, communications, legal.) Marketplace in particular has a demand for real-time prediction, matching and dispatching, and pricing queries.

<comment><USER11> I understand why any marketplace-based system would need real time data but why would any of "growth group, finance, communications, legal" require real- time data to do their jobs?

</comment></comment></comment><comment><USER12> It must have been fun to be at Uber 2017-2019. They seemed to have an unlimited appetite (and funding) for "invent it here", and a lot of those projects made it into open source.

<comment><USER13> I'm sure it was fun, but it does seem to point at a lack of focus. I suppose that's hard to resist when there's an endless pipeline of money.

</comment></comment><comment><USER14> I worked at Uber before. The team and the project is pretty much gutted after last couple layoffs. Check the commit history/contributors and go figure.

It was some amazing tech, but it falls into the category of "when all you have a hammer, everything looks like nail". sometimes you really need a company culture to reward people for creating values instead of deliverable for promotion

</comment><comment><USER15> Does anybody with Clickhouse experience at scale know if AresDB is better on some use cases?

<comment><USER16> It's hard to say, though I think the UPSERT capability looks useful because it simplifies handling duplicates. On the other hand it does not appear that Ares offers clustering, which is critical for large datasets.

(I work on ClickHouse and enjoyed this article when it came out.)

<comment><USER7> It should be mentioned, to ClickHouse' credit, that they made an effort to publish relatively detailed benchmark results for a some data sets and queries, when they first came out. They even got in contact with my research group at the time (the MonetDB group at CWI) to make an effort to present the MonetDB results in a fair manner.

</comment></comment></comment><comment><USER17> Past comments: [https://news.ycombinator.com/item?id=19028860](https://news.ycombinator.com/item?id=19028860)

<comment><USER18> > GPU databases are brilliant for cases where the working set can live > entirely within the GPU's memory. For most applications with much larger (or > more dynamic) working sets, the PCIe bus becomes a significant performance > bottleneck. This is their traditional niche.

> That said, I've heard anecdotes from people I trust that heavily optimized > use of CPU vector instructions is competitive with GPUs for database use > cases.

This comment is important imo. Also related to applied ML inference in applications... the memory needs can grow quite a bit and this data transfer cost, including the memory size limitations vs RAM, becomes very real very fast.

Not sure I understand the scale of the use case or where it's mentioned as well as in comparison to the big data tools mentioned.

<comment><USER19> Next gen Nvidia 30x0 series, can have direct access to SSD without hitting the CPU. In that case would they be any worse than cpus on any workloads? I guess you could still have larger ram amounts on the cpu, albeit slower ram usually

<comment><USER20> Wow nvidia is selling SBCs now?

<comment><USER21> No, there's a new thing about giving GPUs some kind of DMA to storage. And it's pointless on HDDs, so it's only discussed in terms of SSDs.

Microsoft is bringing the DirectStorage API from XBox to Windows, Nvidia calls theirs RTX IO. I think they're the same class of idea, like Vulken vs. Metal.

They do have SBCs, I think, but other than being the basis for the Nintendo Switch I haven't heard much about them.

</comment></comment></comment></comment></comment><comment><USER22> > Like Pinot, Elasticsearch is a JVM-based database, and as such, does not > support joins and its query execution runs at a higher memory cost.

What does the JVM have to do with joins?

</comment><comment><USER23> > In the past, we have utilized many third-party database solutions for real- > time analytics, but none were able to simultaneously address all of our > functional, scalability, performance, cost, and operational requirements.

Completely original excuse for an over-staffed engineering organization to justify doing some crazy stuff.

</comment><comment><USER24> Does anyone know how this compares to the RapidsAI project called BlazingSql?

<comment><USER25> Howdy, full disclosure I'm the CEO at BlazingSQL (BSQL).

I'm not incredibly familiar with Ares save the linked article, but we aren't a DBMS or manage data in any way.

BlazingSQL is a SQL engine, it's easier to think of it similar to SparkSQL, Presto, Drill, etc.

We're core contributors to RAPIDS cuDF (CUDA DataFrame), which is a Pyhton and C++ library for Apache Arrow in-GPU memory. The Python library follows a pandas-like API, and the compute kernels are in C/C++.

BSQL binds to the same C++ as the pandas-like cuDF. What this enables users to do is interact with a DataFrame with either SQL or pandas depending on their needs or preferences. This interoperability means that the rest of the RAPIDS stack can be applied to a variety of different use cases (data viz, ML, Graph, Signal Processing, DL, etc), with the same DataFrame.

The DataFrame also has performant libraries for IO, Joins, Aggregations, Math operations, and more.

Here is an example of running a query on ~1TB on a single GPU in under 9 minutes. The data was stored on AWS S3 in Apache Parquet. [https://twitter.com/blazingsql/status/1303370102348361729](https://twitter.com/blazingsql/status/1303370102348361729)

Here is an example of scaling that same query up to 32 GPUs and running it in 16 seconds. [https://twitter.com/blazingsql/status/1304450203030880257](https://twitter.com/blazingsql/status/1304450203030880257)

Again, think of BSQL as a query engine, that runs queries on data wherever and however you have it. Here is a BSQL user running 1-2 minute queries on 1.5TB of CSV files using 2 GPUs. [https://twitter.com/tomekdrabas/status/1303824164273270789](https://twitter.com/tomekdrabas/status/1303824164273270789)

Let me know if that helps at all (or not).

</comment></comment><comment><USER26> This should have the 2019 tag in the title</comment>


Analyse Asia Podcast 4: JFDI, the Y Combinator of Southeast Asia - <AUTHOR> http://analyse.asia/2014/09/28/episode-4-jfdi-asia-ycombinator-southeast-asia/ <comment><AUTHOR> Synopsis for Episode 4: In this episode, Hugh Mason, co-founder and CEO of Joyful Frog Digital Incubator (JFDI) joined us on an interesting discussion on the role of incubators and accelerators in Asia @ Blk 71 (where most startups in Singapore are located with a total valuation of US$1.5B). We traced the story of JFDI, from how they started from the Hackerspace Singapore to the present state where they have incubated and delivered three batches of 100+ startups worth about US$36M all over Asia. In the same podcast, Hugh also shared his perspectives on the business models and business trends of incubators and accelerators across Asia and stories on what works and fails with entrepreneurs within the JFDI network. Hugh also shared his thoughts on how corporations can work with local incubators to jump-start new projects in coping with disruption and also providing a glimpse of what the future entails for JFDI in the next 3-5 years.</comment>


Show HN: Build your own Feedly-like app, for your readers - <AUTHOR> https://myownreadingapp.herokuapp.com/

<comment><AUTHOR> This was a project I built over the last weekend for fun. Appreciate your feedback, thank you :)</comment>


Finding past hacker news articles and posts - <AUTHOR>

So, I've realized that finding past hacker news stories can be somewhat of a pain sometimes. Are there any tricks you guys have to help me keep track of these if I close them or at the time don't find them important but later decide I wished I'd saved it only to find it impossible to find?<p>I recently read a comic on hackernews about questions, it was a simple web comic with two characters one little guy with one eye and I can't seem to find it. Anyone with a link to this awesome comic perhaps? <comment><AUTHOR> I did find it here: [http://kiriakakis.net/comics/mused/a-day-at-the- park](http://kiriakakis.net/comics/mused/a-day-at-the-park)</comment>


Bank of America gets Twitter to delete journalist joke - <AUTHOR> http://arstechnica.com/tech-policy/2015/12/bank-of-america-gets-twitter-to-delete-journalists-joke-says-he-violated-copyright/ <comment><USER2> The joke was something about risks clouding Volkswagen's view. It's probably fair use but he's not really reporting on anything, just making some dumb commentary with little to no journalistic weight.

<comment><USER3> 3rd parties should not get the chance to weigh in on journalist's reporting quality. Reporters should be able to succeed or fail on their own. What are you, working for or on behalf of B. of A.?

I happen to have a bank of america/ML account because I am a software engineer who got stock options that vested. Like probably a lot of readers of ycombinator. So I am going to write them a letter complaining about this action. I'm about to close my account there. This is one way to attempt to make an impact. Also I recommend complaining about the third party company that apparently sent the dmca. Those dmca takedowns seems to experience little or no negative effects when they make mistakes. This is hard to change through legal means, but it can be changed through public pressure on those companies.</comment></comment>


Bay Area tech boom not cause of region's problems - <AUTHOR> http://www.sfgate.com/opinion/openforum/article/Bay-Area-tech-boom-not-cause-of-region-s-problems-5080195.php <comment><USER2> Although I generally agree with the sentiment of the article, I strongly disagree that we need specifically "affordable" housing. State-subsidized "below market rate" housing that enables people with low salaries to occupy highly-demanded space is incredibly inefficient and also likely doesn't serve the interests of the people it is supposed to help.

I don't see why the ability to live in San Francisco is any different than the ability to afford a BMW or a Tesla. Should the government subsidize "below market rate" luxury cars too? What would be the incentive to get an education or work hard? Personally I feel somewhat offended when there are dozens of below market rate units available and I'm actually excluded from attempting to buy them because I earn too much money. Meanwhile there are hardly any homes available for people who can afford the houses without subsidies and we bid up the houses to obscene prices due to lack of supply.

Also, if the people who work in SF or other high income areas cannot afford to live there at current wages, there should be pressure on them to leave and work elsewhere. This would increase demand for their labor and drive prices up, all else being equal. (I know, there are other factors here, but those should be dealt with separately, not by adding more layers of complexity to the problem.)

Further, if those workers cannot afford to rent or buy, lowering the cost of renting or buying also reduces the pressure on employers to raise their wages.

<comment><USER3> As a recent nyc -> sf transplant, i think the best short term solution to making living in the bay area more affordable is to run one track of bart back and forth from 12am - 4am. SF needs a brooklyn, a real brooklyn, that you can get to when your shift at the restaurant/bar/other-service-job ends at 1am.

Public infrastructure is a public good, and service industry workers can't be expected to cab it to oakland when their shifts end. Heck, the bikelane on the new bay bridge segment ends before treasure island. I could NEVER have lived in brooklyn if the non-auto infrastructure was the way it is between sf and oakland.

<comment><USER2> I also wouldn't mind if they would add some extra cars so it isn't standing room only during commute hours. How hard is it to add an extra car to an existing train? Raise my fare, I'll pay not to have to be packed in like cattle.

<comment><USER4> I'd imagine the length of the train is limited to the shortest platform it has to support. Not from SF, so not sure if there really is room for expansion.

Frequency might be another answer - run the trains closer together, but again there are limits there too.

<comment><USER5> I've never seen a BART station shorter than ten cars, though many rush hour trains only use eight or nine.

I seem to remember that BART has an outdated and overwhelmed computer system which requires greater spacing between trains (and sometimes fails all the way down to manual control). At least new cars will have three doors instead of two, which should get a train out of a station faster.

</comment></comment></comment></comment></comment><comment><AUTHOR> The most striking figure in here is this one:

"The hard truth is that the Bay Area's housing and transit systems are bursting at the seams. According to the San Francisco Apartment Association, last year the city added 68,000 new jobs and just 120 new housing units. Silicon Valley's rental vacancy rates are even lower than the city's."

That's math that just undeniable.

<comment><USER6> This one really jumped out at me too. I haven't read this anywhere before:

"research from the Bay Area Council Economic Institute showed that each of these tech jobs had an astounding effect of creating an additional 4.3 jobs in the local economy."

<comment><AUTHOR> Yeah I thought that was striking as well. I'm generally skeptical of stats like this but I'd love to see the paper and see the structure of those gigs (eg which industries, where geographically are they concentrated etc)

<comment><USER6> I found it. It's mentioned on page 5 in the executive summary. I don't have enough time to read through the whole report right now.

[http://www.bayareaeconomy.org/media/files/pdf/TechReport.pdf](http://www.bayareaeconomy.org/media/files/pdf/TechReport.pdf)

</comment></comment></comment></comment><comment><USER7> So, first off, it needs to be acknowledged that this was written by the "Bay Area Council", which is a PR machine for a dues-paying group of bay area corporations, including Facebook and Google [1]. Consider the source.

Second, while I'm the first to acknowledge that SF needs to build up, it's clearly _absurd_ to say that the tech boom is "not the cause" of the city's housing problems -- that's like saying that skiing isn't the cause of Aspen's housing prices. Nonsense. It's plain that tech is the _proximal_ cause, even if you believe that the boom has brought both positive and negative consequences to the city (as I do).

Fair debate on this topic has to begin with the acknowledgement that the tech companies' policies are _indirectly driving_ a displacement of non-tech workers from the city. It's dishonest to claim otherwise.

[1] [http://www.bayareacouncil.org/about-us/member- companies/](http://www.bayareacouncil.org/about-us/member-companies/)

<comment><USER8> > It's plain that tech is the proximal cause

It's half of the proximal cause. Housing prices have increased because of the interaction of supply and demand. Blaming the tech boom focuses only on the demand side, but the supply constraints (height limits, zoning density limits, etc.) are just as important.

<comment><USER7> The supply didn't decrease. The demand increased. That's the proximal cause.

<comment><USER8> If you like. The policy levers are all on the supply side, though (unless you want to start kicking out major employers or something). So it's pointless to complain about demand side changes.

<comment><USER7> _" The policy levers are all on the supply side, though"_

Are you kidding? Remember the big white buses that everyone is mad about? They play a role here.

The valley tech companies have mitigated the natural barrier that would ordinarily limit the number of tech employees living in the city -- the pain of commuting. So rather than staying close to work and building communities where they live, they use the city as a glorified suburb, but with cushy private transportation. There are a variety of things wrong with this, not the least of which is the fact that they reduce housing supply for the natives.

Econ 101 theories about supply and demand don't apply when large external forces are subsidizing the demand side of the equation. One can quite reasonably argue that instead of "building up" (which is expensive and disruptive) SF should just ban the private buses -- it would be cheaper, more natural, and would solve the same problem.

<comment><USER8> Ban private buses? You mean, like, prevent people from operating private buses entirely? No greyhound? You can't mean that, because it's ridiculous, but I'm not sure exactly what you do mean.

A huge number of people employed on the peninsula would absolutely live close to work and build communities there, if those peninsula cities weren't so hell-bent on preventing dense development. There just isn't the supply there.

And what the f do you mean by "natives", anyway? Native Americans? Everyone else is an immigrant. If you moved here a year before I did, do you have some more legitimate claim to live here? I don't think so.

Of course supply and demand applies to the situation. Your suggestion of banning private buses, while ridiculous, is specifically an attempt to influence demand. Your own suggested solution assumes that the "Econ 101" theory applies.

</comment></comment></comment></comment></comment></comment><comment><USER9> "tech's shuttles are a godsend, eliminating 327,000 cars"

That doesn't seem accurate. Over a third of San Francisco's population leaves town on tech buses?

<comment><USER10> "tech's shuttles are a godsend, eliminating 327,000 cars and 10,000 tons of carbon every year at no cost to the taxpayer"

I presume this means 327,000 car trips per year.

<comment><USER11> That seems much more reasonable - assuming 260 work days a year and 2 trips per day, we'd be looking at just under 450 cars off the roads due to shuttles.

</comment></comment></comment><comment><USER12> This just in: young PR guy for for business lobby group thinks business is awesome.</comment>


A New Editor for OpenStreetMap: iD - <AUTHOR> http://mapbox.com/blog/announcing-id/ Funded by a grant from the Knight Foundation, today they are announcing the launch of ideditor.com, home of the web-based OpenStreetMap editor iD currently in Alpha phase. This launch marks a stepped up involvement in the development of this editor. iD is designed to help create an even better, more current OpenStreetMap by lowering the threshold of entry to mapping with a straightforward, in-browser editing experience. <comment><USER2> MapBox is an amazing company. This new editor is very early alpha but already excellent and usable. Good editing tools should be a huge boost for OpenStreetMap. Funny that OSM is getting a modern editing tool before Wikipedia.

The Javascript libraries they're using are interesting. D3 is a big part of it, but equally notable is that there's no conventional slippy map library like OpenLayers or Leaflet. Instead they're doing something simple with d3.geo.tile, a relatively new component.

Particular attention is due to MapBox's OAuth implementation ohauth, purely client-side (no server proxy): <http://mapbox.com/osmdev/2013/01/15/oauth-in- javascript/> They also use RTree (for spatial indexing), Lo-Dash (an Underscore alternative), and a few extra bits like JXON for XML, the async library Queue, and Bootstrap tooltips.

<comment><USER3> FWIW, Wikipedia's Visual Editor has been in public beta since December, and it's arguably a much harder job because of the baroque complexity of MediaWiki's markup.

<http://en.wikipedia.org/wiki/Wikipedia:VisualEditor>

Not that this takes anything away from MapBox's achievement, and OSM were foresighted in choosing a sanely structured data model that allows things like this.

</comment></comment><comment><USER4> This looks amazing, props to MapBox for putting out yet another beautiful, approchable and useful tool.

From the product page[<http://ideditor.com/>]:

"We're building iD in pure Javascript, using d3.js and SVG for map display."

This has got to be the coolest non-graphing use of d3.js I have seen so far.

Anyone who hasn't should view the demo here: <http://geowiki.com/iD/>

</comment><comment><USER5> Wow, this is amazing! I always wanted to contribute to the OSM project, specially because my area (Sacramento) isn't that well mapped, but I found the tools to be cumbersome and difficult to use. This changes the game.

</comment><comment><USER6> I said they should rename it because iD is taken by the makers of a popular game called DooM. I have no opinion of the quality of the service but it looks good. I apologize for my snarky comment.

<comment><USER7> Point taken :), the name is quirky, hard to google, and surely used for other things. The eventual goal, though, is for it to be just 'the openstreetmap editor' with iD being a codename - so the problem will 'go away'.

</comment></comment><comment><USER8> This is great! A good way for beginners to get their feet wet with OSM.

I'm sure they'll add a lot of improvements eventually, but if you're looking for a more feature complete, more polished OSM editor in the interim, take a look at the ArcGIS OSM editor: <http://www.esri.com/software/arcgis/extensions/openstreetmap>

<comment><USER9> JOSM is right now the most popular editor in OSM. It's desktop software and very powerful. JOSM is very much a power user tool where iD is aiming for a great experience for beginners and casual users.

<http://josm.openstreetmap.de/>

</comment></comment><comment><USER10> It runs on Node.js and is open source, too: <https://github.com/systemed/iD/>

<comment><USER11> I think you're seeing package.json and leaping to Node.js. I think they use the Node package management to help during build.

<comment><USER10> My bad. It also mentions node as a test harness: <https://github.com/systemed/iD/blob/master/CONTRIBUTING.md>

</comment></comment></comment><comment><USER12> Fantastic work. Love the tagging autocomplete - I find JOSM's icon menu completely counterintuitive, you have to guess what classification the tag is under before you can use it (and of course some things aren't there)

</comment><comment><AUTHOR> I'm glad to see that the grant from the Knight Foundation to Mapbox, helps OSM is bringing something great. OSM really needed a new editor.</comment>


Show HN: I made explorable +38M flights in a 3D visualization - <AUTHOR> Hi HN,
Since I was a kid, I have always been fascinated by the flight display screens at the airport wondering where I can fly direct if I had an open ticket
There are more than 38 million commercial flights per year departing from +3,200 airports and served by +580 airlines! So much data that can be summarized and made accessible to everyone in an easy and convenient interface.
Today, I'm releasing Direct flights, a tool to explore all non-stop flights from any airport: https://direct-flights.com/
# How it's made?
The frontend is built on React - using a server-side rendering on NodeJS for SEO purposes. The backend is built in Python. The map is rendered with HTML5 Canvas (TopoJSON map with d3-geo) while interactions with the zoom are managed with SVG (with d3-geo-zoom).
# Geeky and unique features
• Dynamic social preview card: When you share a link on social media, it automatically generates a thumbnail by taking a screenshot (using headless chrome) of the visualization with live data.
• Showing flight routes on a 3D globe: Representing the earth as a 3D globe made more sense to visualize realistic flight routes instead of a distorted flat map (where you wonder why the pilot is taking a "long route" over the North Pole to cross the Atlantic...).
• Select the closest airport to your cursor: It's very hard to nail the right airport on a map when there's a cluster of items (especially for multi-airports areas like SF Bay Area). Direct Flights computes a Voronoi diagram to have larger clickable areas.
# Business model
Direct Flights can be integrated as a B2B white label solution for airlines and airports so that passengers can discover new destinations in a more visual manner (Would love to discuss with anyone connected in this ecosystem). Moreover, there's a B2C aspect, where we get an affiliate commission for ticket sales through kiwi.com
I look forward to reading your comments and questions! <comment><USER2> Posts without URLs get penalized, so you'd be better off posting this with a link, then adding your text as a first comment in the new thread. If you do, email us at email@example.com so we can make sure that the repost doesn't get flagged. Good luck!</comment>


Ask HN: What is good business advice for independent mobile app developers? - <AUTHOR> <comment><USER2> This is probably not what you're looking for, because it's very specific practical advice, but it's particularly valuable all the same: check out SKStoreReviewController on iOS. We utilized it and our reviews grew from ~250 in the first ~5 months to over 27,000 two months later. Of course there has been growth with our app, as well, but the scale at which we saw reviews come in after implementing the UI was incredible: over 1k new reviews within hours.

I say all this because ratings have a huge impact on your app store listing position, and any small developer can benefit from such a boost immensely.

If you're building on Android too, this functionality unfortunately doesn't exist, however we have also seen a considerable boost in reviews after implementing our own UI which asks users to review (and allows them to press a button to go to the store.)

<comment><USER3> I didn't realise it makes such a difference. I'm going to implement it in our apps. Do you have any suggestions on when to show it? I would imagine it makes sense to show it only after some reasonable time. Let's say if user opened the app 5 times or something like that.

<comment><USER4> Some of the best practices regarding it that I've heard (and found to work very well in the projects I used them):

\- You show the user a internal 5 star rating UI. If they rate anything below 5 you don't forward them to the App store for adding a review. Works well with 3 buttons: "Rate", "Late", "Don't show again"

\- Best time to show it to the user for the first time is when they completed the first "major action" in the app. For e.g. a shopping app that would be completing their first order, or for Airbnb booking your first room

\- After that you show it to the user every ~5th time they open the app until they rated it or click "Don't show again"

<comment><USER5> Extremely common pattern on Android, but it's so transparent in its dishonesty it really annoys me. But I guess it works for the masses anyway..

<comment><USER6> In what way is this dishonest?

It's really just saying 'if you love this app, tell others - if you don't love it, tell us'. This is an honest request.

<comment><USER7> For me, this bit:

> _If they rate anything below 5 you don 't forward them to the App store for > adding a review._

This method attempts to passively hide negative reviews from the public while actively prompting users to leave positive reviews.

Storefront ratings don't matter unless the user leaves the app and enters one via the official app store. Without forwarding the user, anyone with a negative review will have to specifically leave the app to leave the review. The dev will usher the positive reviewers forward to leave a public-facing review, while anything neutral or negative is sent to /dev/null or (hopefully in the least) used internally for improvements.

Dishonest may not be the correct term. Perhaps "slightly misleading" is a better way to put it.

<comment><USER8> If the app stores want honest reviews, they have to be the ones to request them, on a fixed schedule after someone downloads an app. Allowing the apps themselves to forward users to the app store review interface is an invitation to game the system in exactly the manner mentioned.

Whether grossly dishonest or slightly misleading, the mere fact that we can be arguing over how scummy the pattern may be is a symptom of the failure of the app stores to manage their own review systems. When one seller games the system to get more 5-star reviews, that's them being dishonest. When everyone has a 4.5-star rating on every product, that's the ratings system being lazy and useless. (Looking at you, in particular, Amazon.)

An honest internal review would publish the results on the developer's website, and leave all the app stores out of it, no matter what the user rating might be.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> An approach I did sometime back was to create a generic template kind of an app (in code not published) for B2B that had integrations with ERPs where the bones were finished but the UI was pretty bare so it could be customized for each client and be deployed as its own app each time. I had built up this toolkit overtime and it got more profitable and easy as I went. I could easily tailor it to clients so it was very branded and unique but yet the bugs and time to complete was very low in the finished product. I still get paid for this monthly even though I'm doing a startup and have moved on. It still pays the bills. Everyone's experience is different. I had an advantage of niche ERP integrations that pretty much no one else did and could sell it way lower than the competition. This is just a hindsight thing and people tend to fool themselves that their decisions were correct but in reality I got lucky in my approach and got lucky I had flow of people that gave me referrals. Find a pattern that works for you but gives you some clear advantage.

<comment><USER10> Great idea

</comment></comment><comment><USER11> As in selling your own apps? Seriously my advice would be give it up. You can make a good living writing apps for other people but the app store is a very very unfriendly place for indie app developers. Your odds of making enough to keep food on the table are much better in other niches.

<comment><USER12> Yes, I also agree with this. App stores are not made for indie unless you "know someone"

<comment><USER13> What can you recommend for indie?

<comment><USER14> Make good VR games.

<comment><USER15> Everybody I know who bought any kind of VR system has it sitting somewhere only to be taken out if someone comes over who "hasn't done this already". Then they go back to playing regular video games.

<comment><USER16> Then make good games.

<comment><USER17> Just my opinion, but I don't think VR - especially mobile VR - is mature enough yet to support "good games".

Maybe in a couple of years it will be. If you believe that, then yeah, now is the time to be building your skills and user base. I don't think you're going to hit it big just yet though.

<comment><USER16> right, i meant "good regular games". imho VR may not catch up at all ever, or we may have retinal implants earlier than that.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER18> Work towards making yourself into a 'brand'. It can be giving talks, blog posts which focus on not just tech bits but a mix of tech and business. You should look credible just by looking at your LinkedIn and/or webpage. Having things like conf talks, a moderate traffic blog will give an edge over other similar devs with just a portfolio.

I'm not sure how far along you are in this so this advice may seem basic:

1\. Factor in costs of your insurance, office space, tools, books, commute etc when working on your pricing model. You can plainly do x usd/hr and get the amount for 200 days and feel confident compared to a salary but there are more hidden costs when running a business and you may not have billable work for the full year.

2\. When you underprice your offering to get customers, they undervalue your service. This is where things like blogs, talks, portfolio help in justifying your value.

</comment><comment><USER19> The best advice I've heard recently was said by a playwright. It was something to the effect of "People don't go to the theatre to see a movie"

Focus on the things that distinguish a mobile device from other platforms. The two things that immediately come to my mind are the camera and messaging. It does not surprise me that those types of apps dominate the charts.

</comment><comment><USER20> Once you have a good app idea, flesh it out in at least 5 different implementations and send them all to the store. See which implementation takes off, and focus on that one and develop further. This is true both for games and utility apps. Use the A/B testing thinking and try several different "look and feels" for your app idea.

<comment><USER6> Is this advice tried-and-tested? Because it sounds a lot of work. I would suggest that putting all of your energy into 1 implementation (at first) would probably work better.

<comment><USER21> Parent is basically describing A/B testing, so seems solid enough. It does seem to presume a fairly simple functionality, though, it probably won't work well if each version takes you five months to build.

<comment><USER22> A/B testing should generally be done within an app, though. If your app already calls home to a server, you can collect usage stats and send feature flags for A/B testing. If it doesn't, you probably have to run your tests on in-person groups.

The parent here proposes releasing multiple versions of an app and seeing which ones get downloads, which seems much harder and riskier - you end up splitting downloads across multiple apps, and then have to deprecate or maintain the ones that don't work out well.

</comment></comment></comment></comment><comment><USER23> If you work with clients:

\- Negotiate for a half upfront payment and start only when you have received it. It will help you avoid clients that don't have the necessary budget and will save you a lot of pain.

\- Document decisions done over the phone or in f2f meetings (e.g. send an email saying something like. As we discussed, we decided to ...). It will help you with unforeseen change requests that the client sees within the scope of the work.

</comment><comment><USER24> Look for what's next in tech trends and focus there. Mobile apps were something special 5-6 years ago, now it's just a commodity. You'll do much better living closer to the edge than app dev unless you have something special to offer.

<comment><USER25> I'm curious what's new in tech trends and not oversaturated? Alexa, Google type voice apps, VR?

<comment><USER26> Mobile app with AR capability and/or AI capability.

<comment><USER15> I feel like AI just re-solves a lot of things that are already solved. In the tech bubble it sounds way better than to the actual consumer. If the consumer has a to-do list app he is happy with, how will you convince him/her that s/he needs AI in there?

</comment></comment></comment></comment><comment><USER27> What I found personally tricky is to advertise my professional hedge as a mobile dev. I have received lots of resume of freelancer to help me on projects, and it is always quite difficult to know how professional and experienced they really are. Any blog or contributions to open communities (stackoverflow, github etc.) are very useful.

<comment><USER28> > my professional hedge

Sorry, but what does this mean exactly?

<comment><USER27> I mean: what makes me a good mobile developer, or in other words, why is hiring me a good idea.

</comment></comment></comment><comment><USER14> Pre-sell your ideas. Otherwise it's a massive waste of time.

<comment><USER29> Before doing that you need to build the audience first otherwise you'll have a hard time marketing your idea.

</comment></comment><comment><USER30> Create a brand for yourself and for your company - logo, website, portfolio, LinkedIn company page with logo, etc. That makes you appear legitimate to anyone who finds you.

I see too many freelancers or independent 'studios' not spending any time to create any kind of branding for their companies, and people assume that it's not a legitimate operation.

There is a fairly large difference psychologically between seeing a LinkedIn profile that says "Freelance iOS Developer, Self-employed" with no logo and someone who says "Principal, $COMPANY" with a shiny logo. First impressions matter.

<comment><USER22> I think it was an old BOFH story that said "when the model and the company have the same name, I know I"m in for a bad time". There's a lot to be said for having your app's developer be a business, rather than your name or worse, your app's name.

edit, found it: _One of my personal warning signs on gear is when I get a brand name the same as the device – ie, with this UPS brand UPS from UPS Ltd, I 'm fairly sure I'm in trouble_

</comment></comment><comment><USER31> I've recently listened to a podcast where the guy behind Drafts (Agile Tortoise) discussed how he's finally been able to make a viable income and work on it full time because he switched to a subscription model instead of a one-off purchase model.

Drafts has been out for several years and has been relatively successful and he's only now able to work on it full time.

Podcast in question: Do By Friday, Episode 79.

</comment><comment><USER32> 1) Figure out how to reliably and predictably get people into your app. Know where to advertise, how much it costs and how to optimise your ads and storefront. Don't have a "build it and they will come" attitude. Solving this problem at scale will be roughly half your total effort.

2) Once users get into your app, figure out how to keep them around. Identify how often you want users to use your app (once a day? Weekly? Etc) and design the entire app experience around promoting that form of behaviour.

3) Really understand what is unique about your app (compared to all others) and emphasise it. Ensure new users understand it and experience it. Most users drop out minutes after first launching your app.

</comment><comment><USER33> Contracting or making your own apps?

</comment><comment><USER3> I have an open source project which consists of back-end, web and mobile apps. I frequently get requests for customisation and branding for the apps. It's a good balance between having control of the core functionality in your own hands and making money by developing apps for someone else. It's also much easier to make adjustments to existing codebase than developing a completely new app from scratch.

<comment><USER34> That makes sense. I thought about doing something similar for a while, but I'm not sure about licensing. Which open source license did you choose for your base code and customizations?

</comment></comment><comment><USER35> Apologies if this is a bit off cuff, but I'd start around reading some things from the release notes community: [https://releasenotes.tv/](https://releasenotes.tv/)

Also..."startups for the rest of us" (which isn't mobile focused, but arguably more focused on bootstrapping), amy hoy's 30x500, etc.

</comment><comment><USER36> It´s not easy to distribute on the bigger App Stores but the biggest challenge would be to become known. You need to get covered in Magazines and blogs, reviews are also great and maybe, if you have enough money, you could pay to get featured on the main page of the app stores. You need good pitches and a lot of work to get in contact with authors.

</comment><comment><USER37> Make desktop apps instead: 10-30x higher prices, lower reseller fees, B2B and professional customers, less OS fragmentation.

</comment><comment><USER38> I use to make my own apps when it was profitable to do so. At any point, we had 50-100 apps across many categories, different companies.

We were making good revenue, it was still early in the mobile world. These days, it's not as profitable as it used to be outside of few VC funded categories which are heavily subsidized, good luck making a big profit there.

Most apps today are complementary to running at loss. So, need a SaaS app (with enough profit margin) before you make your mobile app.

Today, we develop apps for small businesses which is much easier and doesn't require any special skills. 80% apps are just CRUD apps. If we need something advance like machine learning etc..., we delegate it to Google and Amazon like API providers.

There are some of our competitors in app business who are hiring wizards yet not able to compete with this formula ^

</comment><comment><USER39> Always be looking for leads.</comment>


Show HN: Depstack – Find and vote for popular libs - <AUTHOR> http://depstack.io/

<comment> Dorian-Marie I think stars in Github are the same, kind of votes for librairies.

I really like the responsiveness of the interface, it makes it really fast and easy to search and vote.

</comment><comment><USER2> Is there a list of all of the popular libraries?

<comment><USER3> Nop there isn't; will work on it later today. Thank you for the feedback

</comment></comment><comment><USER4> Search is great. It would be cool to have Bower in it and command lines / snippets for easy copy-paste.

</comment><comment><USER5> shameless plug for my project: [http://addonlist.com](http://addonlist.com)

:)

<comment><USER6> Just a quick suggestion but the header green bar on the homepage is an eye- burning shade of green.

<comment><USER5> Thank you. I am not much of a designer, I have been playing with a few different colors.

</comment></comment></comment><comment><USER7> This is awesome. Happy to see a centralized place where I can quickly gauge library quality across many languages.

<comment><USER8> Quality or popularity?

<comment><USER9> Wait wait wait...you mean there's a difference?

D:</comment></comment></comment>


In december, you'll need a GitHub account to contribute to MDN - <AUTHOR> https://developer.mozilla.org/en-US/docs/MDN/Contribute/Howto/Link_a_GitHub_account <comment><USER2> See [https://news.ycombinator.com/item?id=12312825](https://news.ycombinator.com/item?id=12312825)</comment>


Show HN: Rb – Turns Ruby into a command line utility - <AUTHOR> https://github.com/thisredone/rb <comment><USER2> Interesting, so it's reading text from STDIN into an Array, then executing its arguments as Ruby code in the context of that Array (or if you pass -l, in the context of each line individually). So all the standard methods on the [Array][1] (or [String][2]) class become accessible as top-level methods.

A few examples:

Capitalizing a line:

       echo hello world | rb -l capitalize

Printing only unique lines:

        printf 'hello\nworld\nhello\n' | rb uniq

Computing the sum of numbers in a table:

       printf '1\n2\n3\n' | rb 'map(&:to_i).sum'

Personally I think it's a really cool idea.

[1]: [https://ruby-doc.org/core/Array.html](https://ruby- doc.org/core/Array.html)

[2]: [https://ruby-doc.org/core/String.html](https://ruby- doc.org/core/String.html)

<comment> star-techate

        echo hello world | ruby -pe '$_.capitalize!'

        printf 'hello\nworld\nhello\n' | ruby -le 'puts STDIN.to_a.uniq

        ps | ruby -lane 'BEGIN { b = 0 }; b += $F[0].to_i; END { print "sum of PIDs: #{b}" }

select whatever column you want, to sum up.

Really what Ruby needs is a flag that adds some capabilities to NilClass, so that these BEGIN blocks aren't needed.

<comment><USER3> If this was an attempt at showing this tool is unnecessary, you just did the opposite.

Each of your examples use different flags, inputs and print methods. The last is particularly good at proving the point!

<comment><USER4> Each of those examples is doing different things. This rb script locks you into two of the options.

</comment></comment></comment></comment><comment> star-techate Command line tools in _10 lines of ruby_ (using this script):

        docker ps | rb drop 1 | rb -l split[1]
        docker ps -a | rb grep /Exited/ | rb -l 'split.last.ljust(20) + " => " + split(/ {2,}/)[-2]'
        df -h | rb 'drop(1).sort_by { |l| l.split[-2].to_f }'

Command line tools in _zero lines of ruby_ (using ruby):

        docker ps | ruby -lane 'next if $. == 1; print $F[1]
        docker ps -a | ruby -lne 'print $1 if /(Exited .*? ago)/'
        df | ruby -lane 'BEGIN { lines = [] }; lines.push [$F[4].to_i, $_] if $. > 0; END { lines.sort { |a,b|b[0] <=> a[0] }.each{|k| print k[1] } }

Bit of a pain as ++ is lacking, as is autovivication. a /bin/sort at the end usually beats `<=>` for terseness.

<comment><USER5> In case it's helpful, as someone who doesn't know ruby, it's hard to understand. Not too readable. Split.last.ljust I just barely understood (left- justify the last match), but I have no idea what the next part is even trying to accomplish.

It's a cool script though.

<comment> star-techate That's the author's oneliner. I don't know what that code is for either, which is part of I rewrote it in the second set of examples.

It's normal for a oneliner to be somewhat inscrutable though, since they're often extremely dependent on their inputs. Like if it's important to fetch the 3rd, 5th, and 7th column--why? You can't tell without the input.

</comment></comment></comment><comment><USER6> The command should process stdin in streaming fashion rather than slurping it all at once:

      code = ARGV[0]
      STDIN.each_line do |l|
        puts l.instance_eval(code)
        STDOUT.flush
      end

<comment><USER7> OP apparently wants to process an array of lines, not just each line separately.

<comment><USER8> We need to be using lazy coolections.

<comment><USER9> So:

      execute(STDIN.each_line, code)

which gives you a lazy enumerator, in place of:

      execute(STDIN.readlines, code)

which gives you an Array.

</comment></comment></comment></comment><comment><USER10> It makes me a little uncomfortable that they're using curl|bash for something as simple as "put this 10-line script somewhere in your $PATH," especially when the script involves sudo (to move into /usr/local/bin). Sure, it's easy to inspect the script and see that it's not doing anything malicious, but it makes install processes like this, where it'd be incredibly easy to, seem normal.

<comment><USER2> Yeah, and it's not even getting the code from an official source; it's coming from a random bit.ly link [created by a contributor][1].

I've [submitted a PR][2] to inline the install script in the README.

[1]: [https://github.com/thisredone/rb/pull/5](https://github.com/thisredone/rb/pull/5)

[2]: [https://github.com/thisredone/rb/pull/8](https://github.com/thisredone/rb/pull/8)

<comment><AUTHOR> Thanks, merged

</comment></comment></comment><comment><USER11> The second I start wanting a bash pipeline, probably around the third pipe, I scrap it immediately and move to using a text editor to write a script. Because if I'm wanting a pipeline, I'm also going to want to store it, and transmit it over the network, and manage it, and put it down and come back to it later.

All things perfectly manageable inside a PORO. Bundler even has an inline mode, so you can put everything in one file, close to your data, Bundler just handles dep management under the hood like a boss. Check out bundler-inline.

Sure, you _can_ do all that with bash. But you can also make system() calls with Ruby, with full string interpolation power tools. If you're a Rubyist, and you want to do data analysis, this is the workflow you want.

<comment><USER12> xargs -P and other ad-hoc parallelism tools usually stop me from leaving the command line too quickly (e.g. <(), fork / join).

The other thing that keeps me back is scale. If everything fits in memory, cool, but sometimes I need to sort or uniq something that won't.

<comment><USER13> sort(1) is a merge sort (specifically, an "external R-way merge")—and, as a merge sort, it actually isn't constrained by memory. It spills its sorted intermediates to disk as temp files! (And, in fact, it even _compresses_ each intermediate, for the same reason a column store like Cassandra does.) It's pretty optimal!

If you're wondering how this is at-all fast (and it is), in Linux at least /tmp is a tmpfs, meaning that these tempfiles are actually ending up as memory after all. They get "spilled to disk" in the sense that the memory associated with the files spills to swap; but, importantly, tmpfs pages know that they're from files, so they have much better swapping semantics than regular memory for the case where you're writing the file as a stream, and then reading it as a stream. (Basically, juggling streams under a tmpfs is equivalent in overhead to managing memory when you've got it, and managing files when you don't, but without any of the dev-time overhead of having to write code for both cases.)

</comment></comment></comment><comment><USER14>
      $ docker ps | rb drop 1 | rb -l split[1]
      $ docker ps | perl -anE 'say $F[1] if $.>1'

perl solved this problem a long time ago, people

<comment><USER15> Yeah, just like eating dirt solved the problem of world hunger.

Look again at your code. I'm not 100% sure what the Ruby version does, but once I do figure out the exact semantics of drop and split, it's going to be waaaaaaaaaaaaaaaaay easier to remember, understand and modify the Ruby version than the Perl one.

Your post comes off as something from reddit's /r/nottheonion, you're just making the case against Perl for Perl-haters :)

<comment><USER16> The perl here is not really that crazy to remember.

-a autosplits each line by whitespace and puts each element into the array F (this was inspired by AWK).

-n loops through each line of the file, and -E executes the perl.

$. (NR in AWK) is the line number.

As others have noted, you can write the same thing in ruby on the command line already with `ruby -ane 'puts $F[1] if $.>1'`

(Notice the similarities?)

If you write one liners in AWK, Perl, or Ruby often the "odd" variables look more like useful shortcuts.

*edit You could also write the perl without any of the special variables, but it would be much more verbose, hence the special characters and flags.

</comment></comment></comment><comment><USER17> What does this have over the default ruby?

cat your-file.txt | ruby -ne '$_.your_ruby_stuff'

Just syntactic sugar? (it does look cleaner!)

<comment><USER18> I've always found it quite painful to do string escaping when using `ruby -e`

</comment></comment><comment><USER19> "pyped" does the same for python.

I though it was great, knowing python better than bash, plus it was portable to windows.

But eventually I always end up using built in GNU tools.

Don't know why. It just rolls better.

</comment><comment><USER20> "With 10 lines of Ruby replace most of the command line tools that you use to process text inside of the terminal."

Well, it's these 10 lines, plus an unlimited amount of Ruby that you have to compose on the fly for each operation.

</comment><comment><USER21> I have no idea what this is doing, but it sure looks interesting.

<comment><USER22> It's using Ruby to replace Awk, the way Perl once tried to.

<comment><USER23> Why the hell would anybody want to replace AWK? It's super fast, extremely light on resources, doesn't have any dependencies, does auto memory management and type inference, and is extremely powerful for large dataset processing, not to mention easy to learn and program.

<comment><USER24> > easy to learn and program

I think that's relative and it certainly hasn't been my experience. If I program in Ruby, Node, Python, etc for 8 hours of every day, it makes sense that I would reach for that over a command line tool with a syntax that looks a bit arcane. The best tool for the job sometimes is the tool you know best.

<comment><USER23> It's arcane only if you don't know C. If you're on a UNIX-like or a true UNIX system, not knowing C will come to haunt you with a vengeance sooner or later.

</comment></comment></comment></comment></comment><comment><USER25> That's similiar to the ruby-each-line gem I created [https://github.com/Dorian/ruby-each-line](https://github.com/Dorian/ruby- each-line)

</comment><comment><USER26> First I thought: This is stupid. But then I watched the examples on github and here in the comments, and I changed my mind: It's pretty neat :) I like it.

</comment><comment><USER27> A bashtardization:

        rb () {
          [[ $1 == -l ]] && shift
          case $? in
            0 ) ruby -e "STDIN.each_line { |l| puts l.chomp.instance_eval(&eval('Proc.new { $* }')) }";;
            * ) ruby -e "puts STDIN.each_line.instance_eval(&eval('Proc.new { $* }'))";;
          esac
        }

</comment><comment><USER28> I'd love this for python as I'm not a big fan of Ruby. Does such a thing exist?

<comment><USER29> Out of interest, why are you not a fan of Ruby?

<comment><USER30> Right tool for the job; don't be a fan boy.

<comment><USER31> As someone who prefers Python to Ruby, Ruby is better for this job because there are so many more methods on Array/Enumerable than on Python list (or Python's global functions.) And them all being methods makes life simpler too.

</comment></comment></comment></comment><comment><USER32> I found the last example hard to read so I made this fork : [https://github.com/kimat/rb](https://github.com/kimat/rb)

</comment><comment><USER33> Cool.

</comment><comment><USER33> Nice

</comment><comment><USER34> its a pretty nifty idea

</comment><comment><USER35> \- Reading all the lines into an array is a big error.

\- It's a bit unintiuitive because you don't really know what strings to escape on the bash command line. In the example, some are escaped and others aren't.

\- In the end I think this could be achieved with a simple bash function:

        $ rb() { ruby -lane "print \$_.instance_eval(\"$@\")"; }
        $ echo hello | rb capitalize
        $ Hello</comment>


Show HN: Airfi, a bot to find wifi passwords at Airports - <AUTHOR> https://m.me/1194578137263075 <comment><AUTHOR> Also all the code is available at [https://github.com/GanadiniAkshay/airfi](https://github.com/GanadiniAkshay/airfi)</comment>


OkCupid,2.8M profiles out of 3.2M tested, open to all by default - <AUTHOR> http://yakamo.org/?p=112 <comment><USER2> By sheer coincidence a research dataset containing OKCupid data was released today: [https://osf.io/p9ixw/](https://osf.io/p9ixw/)

I looked through it and it's very robust. I should have a few data visualizations up using it soon.

<comment><AUTHOR> give me an email i would love to see some more visualizations, what do you use?

<comment><USER2> I use R and ggplot2 for my visualizations (coincidentally those are also the tools used for the linked paper). Most of my visualizations are on my blog. (check HN profile)

<comment><AUTHOR> Love the Taxi article!!

</comment></comment></comment></comment><comment><USER3> To the author: scientific notation (with only 2-3 digits of precision) would make most of this much more readable. Also avg and std deviation are usually more informative than avg + max. Or better yet just make a histogram (probably with log axes). Cool article!

</comment><comment><USER4> Totally not important to the content of the article, but to the author: your date line is broken. It's saying it was published on 2016-0-0, currently.

<comment><AUTHOR> cheers fixing it now

</comment></comment><comment><USER5> Are you saying 42% of profiles are empty? That would seem to indicate that OkCupid is really puffing up their MAU numbers.

<comment><AUTHOR> yeah i noticed it a lot while checking profiles that there was just nothing written up at all. never really thought about it being a fake profile.

<comment><USER6> Not Okcupid, but another dating site I went to had one profile message me. In her hobbies list it said "I really enjoy: NULL"

<comment><USER7> I've seen a few women* list a programing language in addition to other languages on online dating profiles.

* I'm sure there are some men that do it too, but I don't browse those profiles.

</comment></comment></comment></comment><comment><USER8> > My personal opinion is that profiles should be set as private by default as > people don't often investigate the options they have, even i didnt realise > my profile could be viewed publicly by the rest of the world.

It's a dating site.

<comment><USER2> To clarify, the default state is that the profile is available to _logged-out users_.

<comment><USER9> Is that worse than it being available to logged-in users?

Sure, you can't control what randoms on the Internet might look at your public profile, but you can't control who signs up for OKC accounts and looks at your private one, either.</comment></comment></comment>


How Norway has avoided the 'curse of oil' - <AUTHOR> http://www.bbc.com/news/business-28882312 <comment><USER2> Statoil and how Norway essentially protected its future is a lot like how Australia should have dealt with the so-called mining boom.

Named and outlined better, our "mining tax" could have been this. It could have protected against capital flight and essentially built national strength for all at the expense of those (especially foreign interests) looking to dig up serious swathes of our ground.

Instead, we had a predictable response from the mining magnates and Coalition, an easily duped and panicked public, and a flailing government at the time who named the concept terribly and defended it poorly. And when challenged on the whole "it's barely made any money" front, caved instead of noting that it'd been potentially hampered for political reasons.

What's an easier sell to the public? "Mining tax" or "Future Fund; funded by giant, mostly foreign mining companies."

Look at the Coalition's "$20b" medical research fund. So many people think that's a current $20b fund, rather than a far smaller fund to be built up to $20b over a number of years, and then to fund research only from the earnings of the fund and not from the fund's base value. Labor should have framed the MRRT much more like this and it would've made for a better sell. Avoid the word tax, outline the goal with a specific value range and target date, and name the beneficiaries of the earnings - technology research, medical research, etc.

I can't see how this wasn't a big missed opportunity for Australia.

<comment><USER3> Australia is basically like many African nations, where foreigners run the resources sector and the locals hope to get a piece of the pie through employment/wages.

However, Norway's resources are completely different to Australia's. Norway _knows_ where the oil is, they just have to dig it up. So, they can just have a state-owned company pump it out of the sea and keep the profits for themselves.

OTOH Australia's minerals are hidden across a barren landscape the size of the continental USA. Projects are very speculative and require huge investments. New discoveries are being made every year all across the continent.

Hancock Prospecting (company now owned heiress Gina Reinhart, one of the richest women in the world) was created when its founder crashed a personal plane somewhere in the outback and discovered resources in the dirt (or something like that).

Even when you do know where the stuff is, tens of billions often need to be invested before anything is produced, like the new LNG wave.

There is no way the 23 million Australian citizens could invest this kind of capital into speculative projects. One failed project would bankrupt the nation. So, foreign investment is required.

But, even with minimal resource taxation/royalties, Australia is becoming uncompetitive for many resource investments, with the AUD and commodity prices where they are. Iron ore at its current price is now unprofitable, basically. As well all know, corporations require a certain ROE to do anything. Nobody is going to risk $10 billion for a possible 1% return. Massive LNG projects are getting shelved left and right.

There's oil hidden all across Australia and her waters, too, but there is very little infrastructure in place such as pipelines to transport the product to terminals/refineries, like there is in the US. If Australia was to nationalize its oil industry, would its people be willing to shell out tens of billions of their own tax dollars to build this infrastructure? All speculatively, knowing that their type of oil production is not profitable at all if oil once again goes below $70 a barrel, which could happen overnight (see Sept-Nov 2008)?

While I could see some sectors of the resource industry being nationalized, broadly it's just not sustainable. With costs where they are, there isn't a ton of room for more royalties.

Best thing would be for policymakers to just move on from relying on mining and start focusing on educating the population and competing where it counts on the global stage.

<comment><USER4> How can prospecting and extraction be both simultaneously unprofitable and also creating incredibly wealthy heiresses? If resource extraction and profit- sharing in Australia is most closely compared to that of African nations, it doesn't seem like the public is benefiting much from allowing their natural resources to be appropriated and sold by multinationals.

<comment><USER5> > How can prospecting and extraction be both simultaneously unprofitable and > also creating incredibly wealthy heiresses?

Not agreeing with OP, but playing the lottery can be both unprofitable, and create a few rich people.

</comment></comment></comment></comment><comment><USER6> The steps of building infrastructure and investing outside the country, are pretty much exactly the same as was suggested for Scotland in the McCrone report in 1974.

Unfortunately, that report was classified as "secret" at the time, as it was felt the conclusions would boost support for Scottish independence. It wasn't released until 2005.

Link: [http://www.oilofscotland.org/mccronereport.pdf](http://www.oilofscotland.org/mccronereport.pdf)

</comment><comment><USER7> Probably one of the best examples of how natural resource wealth has been preserved and invested for the whole nation, instead of being lost to the pockets of few through privatisation, corruption or mismanagement of funds.

I think the magic ingredients (aside from being lucky with natural resources in the first place) are egalitarian society, high level of trust, very low level of corruption, a functioning democratic government and highly skilled fund managers.

<comment><USER8> Another Norwegian here. I think you are right. The high level of education also helps. But it was actually an Iraqian immigrant named Farouk al-Kasim that proposed the laws to protect Norways natural oil and gas resources.

Further reading: [http://www.ft.com/cms/s/0/99680a04-92a0-11de-b63b-00144feabd...](http://www.ft.com/cms/s/0/99680a04-92a0-11de-b63b-00144feabdc0.html#axzz3BUIU9zB6)

<comment><USER7> Finnish, not Norwegian - I think you guys have done great.

Read the story of Farouk al-Kasim a few months ago, fascinating. It does seem like a stroke of luck that the right people were there in the beginning. His nationality is both ironic and sad considering where Iraq is today

<comment><USER9> Could it be the same one: [http://www.psmag.com/navigation/nature-and- technology/iraqi-...](http://www.psmag.com/navigation/nature-and- technology/iraqi-vikings-farouk-al-kasim-norway-oil-72715/) ?

<comment><USER7> Yep, same guy

<comment><USER10> Planet Money did a story on Norway and the oil curse: [http://www.npr.org/blogs/money/2011/09/06/140110346/how- to-a...](http://www.npr.org/blogs/money/2011/09/06/140110346/how-to-avoid-the- oil-curse)

</comment></comment></comment></comment></comment></comment><comment><USER11> Since we're talking about how awesome Norway is, this [0] should be mentioned.

[0] [https://en.wikipedia.org/wiki/Freedom_to_roam#Norway](https://en.wikipedia.org/wiki/Freedom_to_roam#Norway)

<comment><USER12> It's an awesome right.

As a Finn I always find it really weird when visiting places outside Nordic Countries that for them most of the country is unaccessible.

<comment><USER13> In Germany we don't got this right, but seldom I have been prohibited from "roaming" the landside. But we aren't allowed to put a tent anywhere and sleep in it, which is rather sad.

A friend of mine went on a hiking holiday in Scotland and he told me, he just walked till he was tired, put his tent up on the next green spot and slept there.

<comment><USER14> _we aren 't allowed to put a tent anywhere and sleep in it, which is rather sad_

Probably connected to the risk that squatter's rights poses to the land owner... not that pitching a tent for a night is a serious risk, but squatter's rights really encourages that kind of culture.

<comment><USER15> Not knowing anything about squatter's rights (or law in general), shouldn't it be possible to just create a law putting whatever limits we want on camping WITHOUT the need for it to be completely forbidden?

I am just now in a place where free camping is not allowed, but the local authorities interpret the law as: "no tents between dawn and dusk". I wish that was put into a written law.

<comment><USER14> The key item of squatter's rights is that if a squatter occupies land with the land-owner's knowledge for a number of years, the land now legally belongs to the squatter.

One night of camping is not years, but squatter's rights certainly encourages land-owners to be ever-strict about visitors to their land.

</comment></comment></comment></comment></comment></comment><comment><USER16> The story of how Norway's oil industry started is pretty interesting [http://www.ft.com/intl/cms/s/0/99680a04-92a0-11de-b63b-00144...](http://www.ft.com/intl/cms/s/0/99680a04-92a0-11de-b63b-00144feabdc0.html#axzz3BTZjZWgz)

<comment><USER17> Non paywalled link: [https://www.google.co.in/url?sa=t&rct=j&q=&esrc=s&source=web...](https://www.google.co.in/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0CBwQFjAA&url=http%3A%2F%2Fwww.ft.com%2Fcms%2Fs%2F0%2F99680a04-92a0-11de-b63b-00144feabdc0.html&ei=ZzD8U5-2PNKhugTgkIKABw&usg=AFQjCNGZKE8V3f8HTdRGPOV72rHCjwFMvg&sig2=QGH8Wwx3weGMwCVXMAXu5w)

</comment></comment><comment><USER18> I am jealous of Norway. Social relations in most other countries, even the "first world" ones, seem uncivilized by comparison. I always wonder how they arrived at this point - this looks like they have some magic ingredient that made their Prisoner's Dilemma converge on 'cooperate'. It must be the fjords.

<comment><USER19> Simple. I grew up in the US, but am in Norway now. I have Norwegian ancestry that comes from my grandparents. Doesn't matter whether here or there. Mix Lutheran-piety (even if Norway is quite secular) and a bit of boredom does help in the cooperation department. I am not applying this to everyone, but that is how it is.

None of this is a culture shock for me. Probably someone else has something deeper to say about it, but I knew what I was coming to when moving to Norway, in this regard.

<comment><USER20> I have Norwegian ancestry as well. I would love to visit, but I don't think I could live there. Also, The language looks absolutely terrifying.

</comment></comment></comment><comment><USER21> I've often said the Nordic countries in general are as close to a Star Trek style civilisation as we've ever gotten.

As someone who lives in America Lite (the UK) your policies on just about everything strike me as insanely reasonable.

I pretty much feel shame whenever I think about our last two governments.

<comment><USER3> Is there freedom of movement for UK citizens into Nordic countries? If not, is the migration process hard or easy?

If it's simple, why not move there yourself?

<comment><USER22> Yee, the proces is very simple. You can simply arrive and you have six months to look for work. Once you get a job you just need to register with the police (you make an appt a couple months in advance and then he visit takes 5 min) and get your tax card (same visit, takes another minute). AFAIK all EU citizens can do this, no special work visa required.

</comment></comment></comment><comment><USER23> Kinda ironic having the picture of the current finance minister, Siv Jensen, there, as she is so opposed to us saving this money. She is mainly elected on promises on lower immigration, cheaper alcohol, cheaper fuel _and_ lower taxes. Planning to achieve that by using the fund's money.

<comment><USER24> Were she around in 60s/70s, she'd likely have banned al-Kasim, the geologist who discovered Norwegian oil, from moving in.

</comment></comment><comment><USER25> There is also the related concept of "Dutch Disease" ([http://en.wikipedia.org/wiki/Dutch_disease](http://en.wikipedia.org/wiki/Dutch_disease)) which is what I expected this would be about. The idea is that once a country discovers it has reserves of a valuable natural resource, it tends to exploit and build its economy around that resource, at the expense of its more complex sectors like manufacturing. Norway seems to have handled that decently well so far, and part of the lack of spending probably comes from the fact that they realize their oil's a short term freebie boost not an economic strategy.

<comment><USER26> I know a lot of people talk up Norway, but I'm not aware of anything that Norway does economically that matters outside of energy production.

What great technology, companies, etc. have they produced in the last 40 years? What big innovations come out of Norway? Do they have the top universities in the world?

They've only won two nobel prizes since oil became the center of their economy, and they became rich. They won four in the 1920s, and four between 1968 and 1973. The oil wealth sure doesn't seem to have spurred great knowledge acquisition.

<comment> pi-rat The Cheese Slicer and Black Metal ofc! :)

On a more serious note, we're only like 5 mill people, and while we don't rock the nobel prices I think the oil has pushed us more towards engineering than theoretical research (for better or worse..). Agree our Universities should be better than they are - it's apparently hard to get funding for something that isn't directly applicable to oil, subsea or salmon.

But you probably use lots of stuff designed/invented in Norway without knowing. Example: Do you use a GSM phone? GSM was invented at a Norwegian University.

Lots of research for subsea-technology, oh, and we're pretty good at ship design stuff - dating back from the viking age until now (x-bow). Techy stuff: Energy Micro, Nordic Semiconductors, some parts of Atmel, Opera, FAST, etc.

Most of the oil welfare fund is invested abroad - all over the world (reduces risk and avoids inflation), these days heavily geared towards green technologies / energy.

<comment><USER26> Definitely fair points.

It's interesting to read about the debate on who deserves credit for inventing GSM, with Switzerland, Finland, and France all claiming various credit.

And of course Norway re:

[http://www.ntnu.no/gemini/2005-01e/gsm.htm](http://www.ntnu.no/gemini/2005-01e/gsm.htm)

<comment> pi-rat Didn't know GSM was controversial, that's very interesting, wonder what happened there.

But still, I bet you interact with gadgets powered by tech from Norway on a daily basis, even if we remove GSM from the equation: the Atmel AVR micros, radios and micros from Nordic Semiconductor, Energy Micro and Chipcon, etc. Maybe you also touch a UI built on QT (Trolltech), even though the QT phones never became a success. :)

</comment></comment></comment></comment></comment><comment><USER27> The economics of mining are complicated, but there are some similarities to the startup/VC complex. A big part of developing mines is exploration. Finding minerals can be hard, especially in the north sea. The oceans are big. The difficulty is why there are still new deposits being discovered.

Exploration usually happens when the company doing it can hope to make money off of a successful discovery. They might even pay for the privilege. Imagine a scenario where a company finds a motherload of oil after a low probability exploration. Their exploration contracts (being signed before the deposits were known) guarantees them a huge profit. Unseemly, even. They are earning that because they took a risk. Now they get their 100X. Try explaining that in an election year.

New technologies are constantly being invented that improve exploration, surveying & mining. This means that there are new possibilities every year impacting which mine/well is profitable (minerals can be extracted at a profit. These all change underlying economic realities. A $1bn per year mine can only make a slim profit in years where commodity prices are high. It employs many people. The next year commodity prices change or some new mining or processing technique (fracking is a huge gamechanger) mean that some complicated contract is now worth a whole lot of money. The $1bn goes from a 2% margin ($20m) to a 30% margin ($300m) and everyone want a piece of it.

Meanwhile government departments, unions, armies, etc are salivating over the prospect of this wealth. The National University's long impoverished Oceanography faculty has their eye on a fleet of research vessels. Academics are starting companies offering to do (mandated) environmental impact studies. There is a lot of pressure for money now, from industry, politicians, constituents.

Politicians definitely don't want to be investing during their term for the benefit of politicians 10 years from now. The world can be cynical, but not always.

Norway has done well. I'm not sure if any one thing can be learned from them. Have smart people running things. Meanwhile Norway have their own political traditions, values, and probably pathologies.

<comment><USER28> > Politicians definitely don't want to be investing during their term for the > benefit of politicians 10 years from now.

you've hit the nail - short term gains vs long term losses. What you need is some mechanism to make it so that the long term benefits align with the benefits of the politician - for example, the project(s) you start stays with you, even after you're term is over.

<comment><USER27> One nail perhaps.

There are a lot of forces at play and real-politic cynicism is one of them, not always one of the biggest.

</comment></comment></comment><comment><USER29> Bergen is not the center of Norway's oil and gas industries. Stavanger is.

[http://en.wikipedia.org/wiki/Stavanger](http://en.wikipedia.org/wiki/Stavanger)

<comment><USER30> That's what I thought too... But I'm biased, because I've only been to Stavanger and not Bergen.

The reason I assumed for not having super cars around there is because it gets too damn cold. Now the number of high end 4x4's I saw there... that's another story :-)

</comment></comment><comment><USER31> The rise of these sovereign wealth funds has been pretty extraordinary- if corruption is kept under control those resources are politically "out of bounds" for spending.

This list ranks SWFs by size- notice there are several US _states_ on the list: www.swfinstitute.org/fund-rankings/

</comment><comment><USER32> I'm from Latin America and many countries here have more-than-decent resources to make things happen, but the difference between us and Norwegians is trust, and that changes it all. Take a look at Brazil, for example. There were over 20 megaconstructions planned for the World Cup and one one of them was completed, the Maracana stadium. That's just an insane amount of corruption. So, what happens? Well, since everyone's stealing, I might as well try to get my piece of the cake, and that's when people start to become selfish A-holes for lack generation after generation. We need a new mindset, better governments and trust. Lots of trust.

<comment><USER26> Venezuela is the great example in South America.

They're, in theory, sitting on the world's largest oil reserves. Meanwhile their production has been falling for 15 years because the government, under Chavez and now, has squandered the opportunity.

They have somewhere between $10 trillion and $30 trillion in proven oil reserves, depending on who you believe (either way it's immense). They should be one of the wealthiest countries in history, right now. That's despite the challenges around extraction and the quality of their heavy oil.

<comment><USER32> That's a great example, as well. Although it gets even worse when you realize what kind of government they're under. I mean, it's almost a dictatorship. It's all just a big pitty over another.

</comment></comment></comment><comment><USER33> It seems to me that the high levels of trust necessary for this to work also imply a high level of social cohesion (possibly homogeneity). In a society with less internal agreement, I don't see how you could get this much alignment over what is the right management strategy, let alone trust that it's being followed.

<comment><USER34> Don't forget SIZE. Norway is a tiny country, and so the distance between people and government is small.

Norway's success is a strong case for localism.

<comment><USER35> Reading this, I realize that almost everything is a case for localism, since we now have nuclear weapons and international instantaneous communications that mean trying to invade a small country is much less reasonable (unless you are Russia).

I mean, that is (in part) why the Greek model failed. Small countries are vulnerable to conquest. But today that is not much of a problem, so I hope the natural social pressure to minimize can overcome the entrenched imperialistic powers that hold a lot of the larger nations together. It just makes more sense to me to, say, have 50 countries and a EU in the US instead of one massively overbearing federal that has authority over a land area competing with continental Europe.

</comment></comment></comment><comment><USER36> relevant:

[http://en.wikipedia.org/wiki/Resource_curse](http://en.wikipedia.org/wiki/Resource_curse)

</comment><comment><USER37> Norway could have let its own citizens decide how to spend their share of the money. That would probably result in some spending more than others, resulting in inequality and an overall worse quality of life, even for the wealthier.

Instead they decided on a mandatory saving of the money. This might have been good for them, but it creates a giant State which wouldn't fly in other cultures.

<comment><USER38> Not really. It's a bunch of traders sitting in an office. They don't really make the state any bigger.

</comment></comment><comment><USER39> C'mon man. Norway's the #1 importer of Teslas [1]

Otherwise, man I wish Canada would manage our oil money like Norway. But we burned that bridge a long time ago.

[1] look it up

<comment><USER40> The Tesla isn't priced as a luxury car in Norway. Electric vehicles are exempt from taxes and fees, which is mostly where the money goes when you buy a more traditional vehicle. They also don't pay anything when passing toll booths, IIRC. Where I grew up, people now pay ~$10 just to pass toll booths to and from work every day.

There are other benefits as well, like free parking, Tesla's free charging station--and most importantly for many--you get to drive in the lane reserved for buses and taxis instead of dealing with traffic to and from work.

<comment><USER41> All of this is indeed correct. People here don't buy Teslas or other electronic cars because of the environment. Most do it for the economical reasons listed above [1]. However there has been a lot of discussion recently about whether these economical advantages are here to stay. Currently the electric cars are clogging up the bus/taxi lanes and the government is losing money as a result of the electric car boom we've seen here recently [2]. It will certainly be interesting to see what removing the advantages will do to the electric car market in Norway.

[1]: [https://translate.google.com/translate?sl=no&tl=en&js=y&prev...](https://translate.google.com/translate?sl=no&tl=en&js=y&prev=_t&hl=no&ie=UTF-8&u=http%3A%2F%2Fwww.aftenposten.no%2Fnyheter%2Firiks%2FFolk- flest-kjorer-ikke-elbil-for-a-spare-miljoet-7522953.html&edit-text=)

[2]: [https://translate.google.com/translate?hl=no&sl=no&tl=en&u=h...](https://translate.google.com/translate?hl=no&sl=no&tl=en&u=http%3A%2F%2Fwww.aftenposten.no%2Fnyheter%2Firiks%2Fpolitikk%2FApner- for-a-kaste-ut-elbilene-av-kollektivfelt-7666860.html)

<comment><USER23> The clogging of the bus lanes because of electric cars is mostly wrong. It's been a lot in the news, but no one has proven it. They all just point to the correlation of more electric cars in the lanes. Not the correlation with more traffic in general.

The clogging happens because every car wanting to get on or leave the highway has to cross the bus lane. It's easy to see for yourself.

<comment><USER41> I don't live in Oslo so I can't comment on the situation there (or anywhere else for that matter), but in Kristiansand the EVs are a very real issue in the bus lanes. Granted, the bus lanes weave in and out of the general lanes so congestion in the general lanes is the root of the problem for sure.

</comment></comment></comment></comment></comment><comment><USER42> I think the key is that Norway was organized, coherent and educated _before_ they discovered oil.

</comment><comment><USER43> Common resources should be treated as public assets. There's multiple ways you could accomplish that, besides just nationalizing oil fields.

Land value taxation and severance taxes can help recapture the value of our Commons, and allow us to reduce taxes on labor or investment.

</comment><comment><USER44> Fascinating contrast to this Backstory Podcast episode about America's relationship with oil: [http://backstoryradio.org/shows/black- gold-2/](http://backstoryradio.org/shows/black-gold-2/)

</comment><comment><USER26> The oil curse is a silly notion. The US and Canada have both avoided any 'curse.' China is the world's 5th largest oil producer, it too is certainly not cursed by oil.

Norway however has a huge problem coming, soon. Their oil production is running out rapidly: [http://i.imgur.com/UlR1B0a.jpg](http://i.imgur.com/UlR1B0a.jpg)

It has fallen about 40% in a decade +/-.

Even with their large sovereign fund, they will exhaust that quickly if the oil production doesn't stop falling soon. At the rate it's falling, in another decade Norway is going to be in the midst of a crisis economically.

I know some people in Norway are paying attention to this and trying to think ahead. Being conservative about the sovereign fund certainly seems like the very wise thing to do.

<comment><USER45> _The US and Canada have both avoided any 'curse'._

Erm, consider the massive amount of environmental and policy issues that we've had due specifically to oil. Had we not had so much of it in the early 20th century, we probably wouldn't have had the automobile boom of today and the long-term sabotage of public transportation.

There's a curse, all right.

<comment><USER26> Perhaps, but then again America would not have participated very fully in the industrial revolution if not for oil, and would not have fielded a strong military during WW2 without oil.

It's likely America would have not have been able to afford a vast, high quality public transportation system had the economic foundations not been built up in the first place. Especially considering the spread out population base, and geographic size of the country.

</comment></comment></comment><comment><USER46> TL;DR "Fairly homogenous country of 5 Million people total" coupled with some good decision making.

For reference NY alone has a population of about 8 Million.

</comment><comment><USER47> So are they saving because they're afraid of overheating the economy or that they will run out of oil?

</comment><comment><USER48> And [https://en.m.wikipedia.org/wiki/There_Will_Be_Blood](https://en.m.wikipedia.org/wiki/There_Will_Be_Blood)

</comment><comment><USER49> i want to go there so bad :(

<comment><USER50> Well, make it happen. University is one option:

[http://www.studyinnorway.no/Tuition- Scholarships](http://www.studyinnorway.no/Tuition-Scholarships)

I say this not to be glib, but as someone who yearned to move abroad and found out that it wasn't as hard as I thought.

<comment><USER49> "However, you should take into consideration that living expenses in Norway are higher than in many other countries."

I don't have a job right now, no savings, 29 years old, and paperwork bore me. There are dozens of those "programs", and I don't want to spend one entire week sorting it out to discover I can't afford to just live over there.

I want to go there no question asked, to work, not to study.

<comment><USER51> Try Stockholm, it has a huge tech sector and you can go there to work as an EU citizen, no questions asked. You can't afford to live in stockholm either without a job of course, but programmers get jobs quite easily if they speak English, and salaries are decent.

<comment><USER38> Actually, I think it's just as easy to get into Norway as an EU citizen because Norway is in the EEC. It's confusing, I know, but the freedom of movement thing is applicable here as well.

<comment><USER51> I'd think so too, but not sure about exact terms and amounts of paperwork. Oslo has higher rents and prices but also higher wages and lower taxes. I think he size of the tech/programming job market is probably quite a bit larger in Stockholm though. So if the advice is for developers only, I'd recommend Stockholm over Oslo. For many other jobs it would be the other way around.</comment></comment></comment></comment></comment></comment>