OpenJournal - Discuss academic papers (open source) - <AUTHOR> http://hackerlist.net:1337

<comment><USER2> When I read papers, I often write short notes to myself about them, as I imagine most people do. I could imagine these notes as also being comments on a site like this, particularly if it's all open and community-run. However, if the paper I'm reading is some 10 year old paper in some niche subfield, which ultimately is the majority of what scientists end up reading, I'm just going to be talking to myself there, so what's the point? I'm not going to submit a paper there that I know no other readers will care about.

How do you overcome that barrier? It's not even network effects alone, it's like extreme network effects because the tail is so long in science. I guess it's really the barrier between being high-level discussion about only the latest big-name science news in certain fields, and being a central hub for all kinds of relatively short comments on scientific papers. Seems like a difficult problem to solve.

Have you thought about maybe pre-populating your database with all papers, ready for discussion, rather than having it as a Reddit-like discussion of only recently submitted things? Or is there some other kind of grand vision for where this will go?

<comment><USER3> Not to speak for Mek, but one hope I have is that these small personal notes automatically make their way onto places like OpenJournal and start to seed conversations. Something like a cross between ScienceBlogs and Twitter, with enough mini metadata to tie to the paper and paragraph.

Do you use a library management tool like Zotero or Mendeley for notetaking, or writing on printouts, or something else?

<comment><USER2> I used to use plain text files. Recently I started using Mendeley because it's so damn convenient and it has a very nice UI, but my problems with Mendeley are (1) comments are private and (2) it's proprietary.

<http://www.researchblogging.org/> is a great website that aggregates more long-form posts about papers, but it has a really shitty UI and it doesn't do anything for short comments on random papers by random non-bloggers.

<comment><AUTHOR> Let's see what inspiration we can draw from researchblogging.org -- I've read a bunch of data driven / research oriented blogs (or blogs about [understanding] research) and found them helpful.

</comment></comment></comment></comment><comment><USER3> Mek, this is great. Thanks for sharing & open-sourcing it. I love the idea of bringing together paper discussions online - whether post-publication peer review like f1000research.com or more casual discussion like r/science.

One challenge I've found is that it's often difficult for discussion sites to gain sufficient traction to build a critical mass of discussion - <http://plasmyd.com>, <http://papercritic.com>, <http://scicombinator.com>, <http://chemfeeds.com>.

I wonder if focusing on supporting existing small-group interactions (real- life journal clubs) would help?

I took a slightly different approach when I wrote <http://www.papernautapp.com> and chose instead to aggregate existing discussions about academic papers (mostly blogs, a few news sites, HN, and r/science, with a goal to cover to more sites and mailing lists). It's also freely licensed, and there are some interesting things I discovered that might be useful to OpenJournal (looking at your TODO list and GH issues):

* CrossRef.org runs a ton of cool lookup/crossref/deref services at <http://labs.crossref.org/>

* They also have some great libraries at <https://github.com/crossref/> \- who wouldn't geek out at this: <http://labs.crossref.org/pdfextract/>

* If you want to do some auto-identification on webpages, the <https://github.com/zotero/translators> project is great and actively maintained by the Zotero community.

(Some notes on how Papernaut is put together, if you're interested: [http://jayunit.net/2013/01/06/papernaut-exploring-online- dis...](http://jayunit.net/2013/01/06/papernaut-exploring-online-discussion- of-academic-papers) )

Lastly, if you're fostering discussion and feedback on papers, there's overlapping interest with the <http://altmetrics.org> and <http://altmetric.com> folks.

<comment><USER4> I think the issue of getting enough traction might be mitigated by restricting the papers that could be discussed. HN is a good example of this - very few posts engender discussions but those that seem promising (by the number of upvotes) are given a spotlight on the front page.

I think an improvement to the services you linked to would be to add a few new articles each week from very selective journals/conferences in each field. I imagine existing measures like a journal's impact score or the number of a conference's attendees would be a good start and tracking blogposts (as you're already doing) could be a good supplement.

This might help pull older or less visible publications out of obscurity; if something published in a domain-specific journal is germane to a discussion, a commenter might point this out while discussing a more highly visible article.

<comment><AUTHOR> Thanks ejstronge and great idea. I'll see if I can get a group of hackers together to post one recommended paper a week.

Do you have any interest in being informed? Also, do you know anyone who may be interested in helping curate / contribute?

<comment><USER4> Hey mekarpeles, thanks for responding. I'm a biologist by training and would be interested in joining if that's a field you'd be featuring - I think an environment where people could discuss recent, high-impact biology articles would be great.

</comment></comment></comment></comment><comment><USER5> Something like this is long overdue, I think. Great work. Open academic publication models have had a difficult time for a number of reasons, but systems like this are very helpful in making the case for openness.

One thing that always bothers me with a purely Reddit-style, point-based system for surfacing academic discussions across domains, though, is that it's unclear what kind of papers are being surfaced: a very good paper in a very niche space may not get the attention that a mediocre paper written for a mass audience (for some definition of "mass") would. Is that an acceptable drawback for openjournal? Or should there be some way for niche papers to gain exposure? Forking openjournal and making your own "sub-openjournal" for your research domain? Weighted voting mechanisms?

Also, like reddit, it might be useful to have a mechanism to demonstrate, emphasize, and/or sort by specific commenters' backgrounds, training, and credentials. For many domains, peer review and commentary from people in the same field might be more useful than general commentary.

As a minor wish, I've always wanted to see a mechanism for encouraging sharing of implementations, test code, and other raw experimental results along with the actual papers. 'Cause really, for most cases, I'm not going to implement a multi-page algorithm just to verify a conclusion or make use of an insight. But if I can fork and compile a github repo associated with the paper...

</comment><comment><USER6> Why use this instead of a subreddit?

I'd love a great solution to this problem and I'd even consider trying to build one, but I am not sure there is any money in it.

One thing I'd love in an academic paper reader is something that allowed comments/annotations inline with the paper. For example, if a paper in the future contradicts something that is stated, you could add a comment linking to the contradiction. Or you could merely ask and provide clarifications, or comment on simpler alternatives to given part of the paper.

Also, I'd like to be able to rate papers for say, readability or difficulty, tag them as theoretical or empirical, etc.

I'd also like if cited papers were automatically dereferenced so I didn't have to hunt down the references myself.

Personalization would also be a nice feature. EG, recommend other papers by the same author, or other highly cited papers that cite/are cited a given paper, or frequent co-authors of some author that I like.

I'd love to be able to download a bunch of papers easily for offline viewing.

<comment><USER7> We launched <http://scholr.ly> in January- we've still got plenty of issues, but we handle some of your use cases, like citation linking. We also first- class authors so you can see an easy summary of a researcher's work. We're still working on personalization and have kicked around tagging/rating for some time- I'd love your feedback.

It seems like the rest of your issues could be solved by Mendeley- WDYT?

<comment><AUTHOR> mhluongo -- Love that you specialize in search and have author profiles.

You guys should get in touch with the peer library guys, send me an email if you'd like an intro. I'd love to see more collaboration in the space.

Internet Archive (archive.org) is also interested in contributing to the space and has been super helpful in aiding our efforts at open journal.

I think the three biggest problems in the space are (1) discovery + accessibility (including open-access), (2) collaboration (sharing, commenting, contributing), and (3) quality assurance (maintainability, scm-backed, repeatable research).

There are many solutions to target discovery and accessibility but I'm (as an academic) personally dissatisfied with the level sharing/collaboration/openness, the lack of community, and the lack of standards in academic research. I think the world needs for academia and research what github did for social programming.

<comment><USER7> mekarpeles - an intro to the PeerLibrary team would be awesome- I'll do that.

I've loved the GitHub for science analogy ever since it first surfaced a couple years ago- I couldn't agree more.

</comment></comment></comment></comment><comment><USER8> The academic world could benefit from a place that allows commenting on any paper (you can have too many comments; not sure how moderating such a system would work).

To this end, it would be great if this were written in such a way that it implicitly considers papers from all the "standard" academic sources as part of the system, ideally with duplicate removal.

That is, automatically add articles from arXiv and major currently existing journals and conferences, try to automatically detect duplicate papers (perhaps add a concept of versions of papers).

In addition, such a site could really benefit by having "virtual journals", where users collect topical collections of must-read papers.

</comment><comment><USER9> This is great and I would use this. I even think about porting/forking/stealing this for a German audience.

The only thing that would keep such a site from growing is the relative reservation of less technical crowds (at least that has been my observation). HN, proggit, SO: they are all useful and fun for the technical minded. Similar sites for other segments (excluding cats, cats always win) are much less active and sometimes fail to attract some critical mass.

<comment><AUTHOR> Please do, let me know how I can help you with it.

firstname.lastname@example.com

</comment></comment><comment><AUTHOR> Would love feedback.

Please feel welcome to submit issues on github and I will try to deal with them in realtime: <https://github.com/mekarpeles/openjournal/issues>

</comment><comment><AUTHOR> Also, thanks for your patience, it's running on a micro EC2 and I haven't put too much effort yet into optimizing request handling (just using waltz over web.py at this point)

</comment><comment><USER10> RSS/Atom please.

<comment><AUTHOR> Adding this to the github issue tracker, I'll get this setup for you as soon as I have a break in my schedule today. Thanks!

<comment><AUTHOR> Issue oppened -- thanks! <https://github.com/mekarpeles/openjournal/issues/16></comment></comment></comment>


Ernest Hemingway in Love - <AUTHOR> http://www.smithsonianmag.com/arts-culture/ernest-hemingway-in-love-180956617/?no-ist <comment><USER2> I am agonized to know that an organization with 70 percent of its budget funded by US tax dollars is still pumping out puff pieces for a suicidal Stalinist spy.</comment>


His copy paste fu is strong - <AUTHOR> http://labs.teppefall.com/2010/05/his_copy_paste_fu_is_strong.html

<comment><USER2> That was a terrible article. It was all over the place, spewing nonsense in all directions.</comment>


Ask HN: How do you focus? tools and other - <AUTHOR> How do you people avoid distraction when working (I code mostly)?. What tools do you use (I'm a Mac user, no relegion wars pls) Personally:<p>- Quitter to quit thunderbird, skype, etc after 10mins. Slack is just hidden.<p>- Leechblock (FF) to avoid more than 10mins on distractions website<p>- Timer.app (http://martakostova.github.io/timer/ you can compile it) for pomodoro with a script that kills Slack when a pomodoro starts and reopens it once finished. The script was setting the DND but the notifications keep popping up (the red dot), which bother me, so I switch it off. I think that messages can wait 40 mins, if not they can call.<p>- I don't keep thunderbird open (see point 1), but I've a plist script that opens it at 11 and 15, so I check the emails every now and then.<p>- Rescue Time and Waka to keep track of my habits and productivity<p>- Trello (was BulletJournal but i'm not alone anymore) to schedule stuff.<p>- In the open space nice pair of headphones, a good music, and avoid to reply when asked questions if i'm in a "pomdoro" or working.<p>- A reason to leave work <comment><USER2> Pssh... You just need to adjust your setup (filthy mac user). I code in assembly on Windows 95 and IE 1. Nothing works on the computer and nothing loads on the browser. Ergo, no distractions!

</comment><comment><USER3> 15-minute time boxing 24 hours a day

When I'm working, I use a physical egg timer on my desk. I also don't check Slack when I'm coding. I keep my MacBook and iPhone in Do Not Disturb mode.

I take an nx 15-minute break when I need one. A typical ratio is 75% work to 25% breaks where break could be doing something enjoyable either fun or actual work but a task I'm really excited about.

Throughout the day I accumulate the number of 15-minute periods of good work for personal reference.

A pocket notebook to write down random thoughts that pop into my head while working.

I don't have one myself, but I've heard good things about Saent [1] which seems like a slightly more polished version of this workflow.

[1]: [https://www.saent.com](https://www.saent.com)

</comment><comment><USER4> I use the Pomodoro technique with this app: [https://itunes.apple.com/us/app/tomato-one-free-focus- timer/...](https://itunes.apple.com/us/app/tomato-one-free-focus- timer/id907364780?mt=12)

I do tasks in 30 minute chunks with 5 minutes to go to the bathroom, get up, stretch, etc.. On my best days I can do 11 pomodoros, but my average is 6/7 pomodoros. I'm currently lecturing on a local university, so the pomodoros are mostly for when I'm preparing classes, and I try to do as few as possible, which means I'm freeing my time successfully..

</comment><comment><USER5> \- SelfControl app, you can change the timer to 1 month instead of 24 hours, so you are not forced to click it every morning.

\- Simple Pomodoro timer, which honestly I use more as a signal for me to get up and stretch my legs a bit.

\- Disabled all notifications on phone, macs.

\- Things as my todo list, simple and tells me what to focus on every day with recurring todo's.

\- Only really visit 3 websites: HN, Designer News & Stack overflow, tagging the interesting posts in the morning, then reading them on Kindle in the evening. Or sometimes between Pomodoro breaks, like now.

\- I block all the news websites and other useless information with Opendns.

\- Bought a Wacom pen and a mac so I can't play videogames.

\- Podcasts for when I'm doing the dishes, Spotify for when I'm doing work.

\- No mail, slack before 1pm, I'm most productive in the mornings.

I read in Do The Work that the best thing is to not think, but just start. So I meditate in the morning and try to be completely free of thoughts when I arrive at work, sit down and just start. That was the single thing that made the biggest difference for me. Don't give The Resistance a chance.

</comment><comment><AUTHOR> SelfControl is nice, in the end it adds an entry on your /etc/hosts .

For what is worth the script for the timer app tokill slkck and set DND START: if application "Slack" is running then tell application "Slack" quit end tell end if do shell script "curl '[https://slack.com/api/dnd.setSnooze?token=<YOURTOKEN>&num_mi...](https://slack.com/api/dnd.setSnooze?token=<YOURTOKEN>&num_minutes=$duration'")

END: if application "Slack" is not running then tell application "Slack" to activate end if do shell script "curl [https://slack.com/api/dnd.endSnooze?token=<YOURTOKEN>](https://slack.com/api/dnd.endSnooze?token=<YOURTOKEN>)

</comment><comment><USER6> I don't use any specific tools as they are time consumers themselves. I just: 1) don't turn on anything I don't need for work (tools, browsers, soft, sites, etc) 2) rest well when I feel I need a break (one of the most counter productive things is to proceed working when you're tired to the point you can't focus.) Works great for me.

</comment><comment><USER7> I just finish my work first, once finished I don't care what I do. So, that means I don't need to use other tools or techniques.

I think its just a habit, I find very annoying to see task assigned to me being in ToDo Status. I want to make that resolved as soon as possible.

</comment><comment><USER8> I just install SelfControl app on Mac. Haven't wasted time on facebook after that. So that's a goo start :)

</comment><comment><USER9> Your distraction prevention tool stack sounds like it would be a major distraction for me.</comment>


RapidShare calls it quits - <AUTHOR> http://arstechnica.com/tech-policy/2015/02/rapidshare-calls-it-quits-veteran-file-sharing-site-to-close-in-march-2015/

<comment><USER2> I wonder if ArchiveTeam can even begin to rescue the amount of data that's about to disappear.</comment>


Show HN: Empatica – Human Data in Real Time. - <AUTHOR> https://www.empatica.com

<comment><USER2> The demo login credentials doesnt works on [https://www.empatica.com/demo/connect/login.php](https://www.empatica.com/demo/connect/login.php)

<comment><AUTHOR> This could help [https://www.empatica.com/demo/demo.php](https://www.empatica.com/demo/demo.php)

<comment><USER2> Cool , this looks very good. In the page [https://www.empatica.com/info.php](https://www.empatica.com/info.php) there are few projects been mentioned something like "Parkinson's disease early detection" ..are there any corresponding links for this to get more info as in how does it detect the Parkinsons disease in early stages.

<comment><AUTHOR> You could probably email their support - email@example.com</comment></comment></comment></comment>


The Math Behind Project Scheduling, Bug Tracking, and Triage [video] - <AUTHOR> https://www.usenix.org/conference/srecon18europe/presentation/pennarun <comment><USER2> I did not expect this talk to hook me so quickly. Im still watching currently, and a thought that pops up for me as a freelancer, how can I employ some of these mental tricks (like for example estimating tasks by points while disregarding how much calendar time that is associated with) while also being the manager of myself? Im sure its simply be aware of the pitfalls of thinking in deadlines, to avoid "Student Syndrome", to focus on the benefit and spirit behind this sort of goal setting.

Im thinking it would help me if I took one day a week to put my "project manager" hat on and divy up tasks for the coming week into story points, and then kept the hat off the rest of the week to keep time based goals out of my way.

Does anyone with some experience scheduling their own time using tips in the talk have any suggestions?

<comment><USER3> It doesn't make sense for yourself. The idea is to have one person tracking another (group of) person(s) and thereby having some feedback and motivation. Maybe you can grow a team with other freelancers and track each other?

<comment><USER2> That makes good sense to me actually. By the time I finished the talk, I realized there were some great takeaways for me in terms of understanding product management, but that it would be foolish overtooling to try and solve for problems that I dont actually have on the scale the tools are designed for. Thanks for your input!

</comment></comment></comment><comment><USER4> This is a great talk so far. Does anyone have any [text]book recommendations for "mathematically mature" Project Management (i.e., project management theory that doesn't shy away from using math to demonstrate its points, if needed)?

<comment><USER5> Goldratt's "Critical Chain" work is the closest I've seen to 'mathematically mature' project management.

[https://en.wikipedia.org/wiki/Critical_chain_project_managem...](https://en.wikipedia.org/wiki/Critical_chain_project_management)

</comment></comment><comment><USER6> The author, based on the technology and products he's chosen to work on, is either visionary and courageous, completely lacks foresight, or enjoys pain:

 _Avery was the lead engineer for Google Fiber 's home wifi devices, building, managing, and monitoring the whole fleet in customers' homes. ... Before that, he started startups including one that deployed Lotus Domino in 10 minutes, and one that did unspeakable things to Microsoft Access databases._

Microsoft Access, Lotus Domino, and consumer Wifi (mass-deployed!)? Shoot me, please, three times.

On the other hand, they are a great way to learn the challenges of project scheduling and management. (In fairness, Lotus I mostly know by reputation, but the other two are enough.)

<comment><USER7> Of your proposed motivations, I've been accused of all three but it's probably the third one :) — the author

<comment><USER6> Hi - I hope the intended humor was taken that way! I really do see it as courageous, because I would never take it on. How do you efficiently provide remote support for mass-deployed, consumer wifi in residential structures - which also means unmanaged networks and systems of every imaginable variety and configuration? It can be hard enough to diagnose problems in person, on- site, in structures I know on networks I built. I can't imagine the support calls, even if you tediously work with the end-user to narrow it down to the wlan signal: "Do you have metal mesh your walls? Baby monitors? Fluorescent bulbs? A microwave? Do the bulbs or microwave leak? Do your neighbor's? Does your neighbor have wifi? Do you an RF sensor in the right frequencies, mapping software, and do you know how to use them?"

<comment><USER7> No offense taken. The wifi project was indeed very hard, but I think we moved the world forward at least a little bit. If you're really interested, you can find out more about how we did it in my recorded talk here (although the slides pdf has a lot more detail): [https://apenwarr.ca/log/20160328](https://apenwarr.ca/log/20160328)

And some additional comments on the evolution of wireless networks in my talk from Battlemesh last year: [https://www.youtube.com/watch?list=PL3bvPCw5QCLJ- VJPamVeQx-U...](https://www.youtube.com/watch?list=PL3bvPCw5QCLJ-VJPamVeQx- UPNBVyaopj&v=vJL8xrQ3gOE)

One part that made the problem more tractable was that we both ran the network _and_ made the devices. This almost never happens. It means we could optimize the devices for quality and supportability instead of making the usual consumer device tradeoffs (is it pretty enough? cheap enough? glitzy enough?).

</comment></comment></comment></comment><comment><USER3> It sounds very attractive to developers, and I myself was a huge fan of this way of thinking for a long time. Sadly real life teams are never even close to this. And there are reasons for that, which we might like or not. But they have to be accepted. For instance most project leaders are not good developers nor good managers. And they don't really want to advertise that fact, because just like us they believe that anybody actually cares. So they get even worse by not tracking relevant metrics and instead focus (not by coincidence but by cunning) on metrics that are painful for developers, like current_open_bugs, or open_features_that_nobody_understands_and_that_where_discussed_with_nobody. They do that because it increases pressure on developers, hooks them with their desire to become better, and they provide automatic goals that can never be achieved. And these are the desired outcomes for the project leaders. Because if everybody is under time pressure, you don't need to worry that someone might steal your job, if everybody already feels bad about themselves they will not criticize you openly (and are motivated to work harder), and if there are always automatic goals you don't need to think about new goals all the time.

What is interesting to me is that nobody puts these people actively at the top of the projects. It's more like they grow to these positions naturally. But now that they are there, you can't really apply the logic from the presentation to convince them. Insted you might even give them the impression that they don't apply enough pressure yet because you are criticising their job.</comment>


Show HN: Sidequest – A personal ticket system that lives inside Slack - <AUTHOR> https://www.getsidequest.app <comment><AUTHOR> Sidequest enables you to create task/ticket inboxes right inside Slack – one for each workspace member, and as many channel-based ones as you like. You can CC people, set and track deadlines, chat and share files inside tickets, and receive daily briefings.

Feel free to install Sidequest to your Slack workspace right now, or give it a try in our sandbox workspace first by joining [https://join.slack.com/t/sidequestsandbox/shared_invite/zt-f...](https://join.slack.com/t/sidequestsandbox/shared_invite/zt- fbevkjci-fTXP9uSB08mcva520oWVzg).

<comment><USER2> Some feedback from my first 5 minutes with it: I invited it to a private channel we have that really only exists for leaving tasks for other people, ran `sidequest create` in that channel, and it created a task entirely unlinked to that channel - ie: the task appears in the @sidequest user, but there's no sign of it in the channel where it was created. This leaves me wondering what the point of inviting the @sidequest user to channels is.

<comment><AUTHOR> Thanks a lot for taking the time to report your feedback!

The `/sidequest create` is not context-aware right now, it will behave the same way no matter you invoke it (opens the creation modal).

Inviting the @Sidequest user to a channel (public or private) makes this channel visible to the app and allows users to select it as a recipient when creating a quest.

Will invest more time into documentation before final release. :-)

<comment><USER2> Ahh, I see. Thanks for the clarification. I'll keep up with trialling it and email you if I find anything. Good luck.

<comment><AUTHOR> Awesome, looking forward to hearing from you.

</comment></comment></comment></comment></comment><comment><USER3> How do you feel about active office workspaces using the product? I'd be open to pitching it to my team, seems like the exact kind of thing I've been looking for, but just wanted to make sure you were ready for more than individual beta testers.

<comment><AUTHOR> Sounds awesome!

We have been using Sidequest in our own team with 60+ people and multiple subsidiaries of ours for more than a month now, creating and completing more than 200 quests without significant issues. We have also been working on enterprise tools for a while now (you may want to check out [https://mantro.net](https://mantro.net), that's us), so overall, we're quite confident in the product.

Over the last 24hrs, we have seen quite a number of beta installations and we are working through all logs and feedback to solve smaller issues and implement enhancements. That was our main intention for the public beta, so nothing to worry about. However, it's still a beta, so usage is at your own risk per definition. :-)

If you want to be extra safe, you may want to wait until our app is officially approved by Slack. Review is in progress and we hope this to be done within the next few days. Feel free to drop me a mail at email@example.com and I'll make sure to let you know once there are news. Feel free to join our sandbox workspace (see my other comment) in the meantime.

<comment><AUTHOR> Just to provide an update: Sidequest has been reviewed and approved by Slack in the meantime!

[https://slack.com/apps/A011B3BC8MV- sidequest](https://slack.com/apps/A011B3BC8MV-sidequest)

</comment></comment></comment><comment><USER4> It's interesting how similar ideas come about. At an old job I worked at, we had an internal tool also named Sidequest where you could write tickets for small tasks you needed help with. That way people who had down time and wanted to crosstrain could easily find people who had tasks they needed help with.

<comment><AUTHOR> That's actually a great idea! To realize this with (our) Sidequest, you'd simply create a public Slack channel (e.g. #needhelp) where people can post their requests using the @Sidequest app.

The app will then take care of assignments, status tracking, communications, etc.

Nice touch that you came across the same name... it took us a while!

</comment></comment><comment><USER5> Feedback: the screen shots showing how it works advance much too quickly. It's impossible to spot the helper text, look at the screen, and get any context before it flips. Image carrousels with important information are a poor UX.

<comment><AUTHOR> Good point, thank you! I have disabled the automatic scrolling and added buttons instead. Swiping should work as well on mobile.</comment></comment>


Show HN: Paint yourself in the style of classic art, in your browser - <AUTHOR> https://github.com/reiinakano/fast-style-transfer-deeplearnjs/ <comment><USER3> Cool work! We made this demo using the same style transfer network in TensorFire: [https://tenso.rs/demos/fast-neural- style/](https://tenso.rs/demos/fast-neural-style/)! Your size slider was a nice touch :)

<comment><AUTHOR> You're lengstrom from Github! I must let you know your models are exactly what I used! I don't have a beefy GPU so I was forced to use what you already had, haha.

For everyone reading, these guys coded up the neural network I used, all I did was port it to a different library. :)

Eagerly waiting for the Tensorfire API!

</comment></comment><comment><USER4> When I see the style transfers, I'm reminded of the early days of Photoshop, and whatever else was a commonplace photo editing application of the mid-90s (like Fractal Painter). It was fun to play with the idea of oil painting, or watercolor, or to simulate paper textures. How is style transfer an improvement? Undoubtedly, it's more sophisticated and requires more computational resources.

Can it be used for artistic purposes, or is it only skin deep like those old filters? It looks to me as if this is pure imitation, your images of a night with friends at the street cafe, or stacking bails of hay can look like a Van Gogh when you upload it to Instagram.

Am I missing something?

<comment><USER5> Of course, computers can't be creative, so it is just imitation. What it creates is not novel. It's just doing what the programmer told it to do. Blah blah blah.

In my opinion, it is one step -- and an impressive step -- in the direction of computers being able to actually be creative. The areas where humans can do something that computers can't is shrinking. You can dismiss it if you want, but at what point will you acknowledge that what the computer is doing isn't simply imitation? I guess you can keep moving the goalposts, but to me, this is pretty impressive.

<comment><USER6> This assumes creativity is anything more than copying from multiple sources.

<comment><USER7> If it's not more, how come there is anything to copy from?

<comment><AUTHOR> Copying from nature with tiny little variations I guess?

<comment><USER7> Little compared to what? They're infinitely big compared to no variations. Ignoring that part because it's unclear is like throwing out the baby and keeping the bathwater to me.

</comment></comment></comment></comment></comment></comment><comment><USER8> I don't know if it's my machine or whether my expectations are too high, but every generated image looks like the same blurry version of the original but with a different colour pallet. I certainly don't get anything as beautiful as the output examples.

This happens regardless of the source content I use (any of the examples or even my own uploaded files) nor any of the reference styles.

User agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36

(Chromium running on ArchLinux AMD64)

<comment><USER8> Looks like I'm getting the same results with Firefox on the same host PC as well.

Screenshot showing the same input values used as the one in your example:

[https://armyofcrabs.com/rand/Screenshot_20171002_170417.png](https://armyofcrabs.com/rand/Screenshot_20171002_170417.png)

<comment><AUTHOR> These definitely look wrong. Unfortunately, it's super hard to debug since I can't reproduce. :(

My only advice I guess is to try it on a different machine when you have one available. Sorry, if this is a problem with the library I used, I can't really do anything to fix it. :/

<comment><USER8> Not sure if this helps but I've tried the project you based this off[1] and that produces images correctly (or at least that look more like my expectations).

(I say "based", I'm not sure just how much of their code - if any - you used. But you did compliment them earlier in this discussion)

[1] [https://tenso.rs/demos/fast-neural-style/](https://tenso.rs/demos/fast- neural-style/)

<comment><AUTHOR> Didn't really use any of their code. I reimplemented their model in a new library called deeplearn.JS.

If the issue is indeed with the library, there's little I can do.

I will still try to find what the issue is but can't make promises. Sorry for the letdown!

</comment></comment></comment></comment></comment><comment><AUTHOR> Demo website: [https://reiinakano.github.io/fast-style-transfer- deeplearnjs](https://reiinakano.github.io/fast-style-transfer-deeplearnjs)

Deeplearn.JS library doesn't work on mobile yet, sorry to folks on the go!

<comment><USER9> Is the aspect ratio important? I'm choosing a roughly square image and it only calculates half an image.

<comment><AUTHOR> Aspect ratio shouldn't matter, the output should still be the same size as the Content image. Mind sharing the pic?

<comment><USER9> [https://imgur.com/a/acanW](https://imgur.com/a/acanW)

<comment><AUTHOR> I was able to stylize it fine. Maybe refresh and try again with a smaller size?

</comment></comment></comment></comment></comment><comment><USER10> Might want to try some basic HTTP caching settings for the model. This way the browser doesn't need to re-download everything every time.

This can probably be configured in the server instead of in the application.

<comment><USER11> I think most CDNs will add reasonable cache-control headers as well - <USER2> Cloudflare does this with sane defaults out of the box.

<comment><AUTHOR> Stupid question: Is CloudFlare free for CDN purposes? Right now I'm serving everything from Github Pages. I don't expect there to be much traffic after this initial HN spike dies down, so if it isn't free, it won't be worth it for me to pay for it.

<comment><USER11> Yep[1]! I used it for my girlfriend's small business site (Wordpress) and was very happy with it.

I think CSS, JS, and images are cached - HTML is not.

1: [https://www.cloudflare.com/plans/](https://www.cloudflare.com/plans/)

</comment></comment></comment></comment><comment><USER12> tl;dr: Impressive Prisma clone with six filters but running all calculations locally in the browser. Compared to Prisma it feels slightly slower or as slow, one filter almost froze my MBP email@example.com.

<comment><AUTHOR> It is currently VERY slow since there are a lot of non-optimized (non-GPU) calls being made. Once those are implemented in deeplearn.JS, things should run much faster. In fact, one of the creators of deeplearn.JS says it might be feasible to do this realtime in a webcam (although not at 60FPS).

This was a super quick PoC demo I hacked together in a week on a very new library. :)

</comment></comment><comment><USER13> Weirdly, something about the generated images (colour range?) breaks Twitter's profile photo update. I actually took screenshots of the original taken with my webcam and the deep learning style transferred version, the former worked for my Twitter profile photo, and the latter resulted in an error. I tried different resolutions too, and it always broke.

Shame, I really like the effect!

<comment><AUTHOR> You took screenshots and it still broke? That's.. very odd, haha. I will look into it and see if it outputs out-of-range colors.

EDIT: My bad, there are indeed out of range colors! I will update here when fixed.

</comment></comment><comment><USER14> The demo site doesn't appear to work on my end -- it gets a 404 attempting to load "bundle.js"

<comment><USER15> Yeah, same for me. I added an issue: [https://github.com/reiinakano/fast- style-transfer-deeplearnj...](https://github.com/reiinakano/fast-style- transfer-deeplearnjs/issues/5).

<comment><USER14> It appears to be working again (for me)

<comment><USER15> Yep, it's working now.

</comment></comment></comment></comment><comment><USER16> Cool. I'm getting some great results. Would be awesome if you could train your own model in the browser too.

By the way: How does a NN learn from a single image? I understand how a NN can learn to classify images. But how can it learn a style from a single image?

<comment><AUTHOR> Fast Neural Style Transfer works like this: Each style image is actually trained on a neural network outside of the browser (using Python on Tensorflow) for around 6+ hours with a GPU.

The output is a neural network that can take any content image and outputs it in the style it was trained on.

The browser does this: 1\. Downloads the model (6.6MB~) for the particular style. 2\. Does one pass through the content image and outputs the styled image.

No training is done in the browser whatsoever.

<comment><USER16> Yes, I am aware of that.

But I would like to know how the training works. How does a NN learn the style of an image?

<comment><USER17> I think you can consider one single image as a large corpus of data, in the same was that you could train a simple Markov model from a large paragraph of text. It doesn't matter to the model whether the original text was 500 words in one piece of text, or five 100-word texts.

It's mostly looking at local regions and seeing what guesses it can make about the next region given one region. (If I understand correctly).

</comment></comment></comment></comment><comment><USER18> Is it using the same tech as android/ios app Deep Art Effects?

[https://www.deeparteffects.com/](https://www.deeparteffects.com/)

Results look very similar

<comment><AUTHOR> Yes, most likely.

At most they probably tweaked things a little bit for some minor improvements, but the underlying idea and algorithm is very likely the same.

Can't confirm this, though. :)

</comment></comment><comment><USER19> Hey, it looks very cool. However, it's hanging my browser. Haven't looked at the source, but maybe this should be implemented with web workers to increase perceived performance.

Anyway, great work. Congrats!

<comment><AUTHOR> Things are currently quite inefficient as there are some non-optimized (non- WebGL) calls being made. DeeplearnJS is a new library but once optimized functions are available, I'll update the site. :)

</comment></comment><comment><USER20> Uncaught ReferenceError: dialogPolyfill is not defined at HTMLDocument.<anonymous> ((index):22672) at (index):24

<comment><AUTHOR> This doesn't happen when I visit the site on Chrome on Linux. Do you mind sharing your system + browser?

<comment><USER21> Same with Chrome/63.0.3223.8 on Intel Mac OS X 10_11_6

It's the call to dialogPolyfill.registerDialog(dialog);

<comment><AUTHOR> Hi, thanks for the report, but I'm not really a JS developer and have no idea how to work around this. Would you happen to know if there are any fixes I should look into?

</comment></comment></comment></comment><comment><USER22> Am I the only one that get "the stuff of nightmares" kind of results ?

Ubuntu 17.04 FF 57 & Chrome 59 with face and Udnie:

[https://screenshots.firefox.com/4bMiETLnl1yx8NOx/reiinakano....](https://screenshots.firefox.com/4bMiETLnl1yx8NOx/reiinakano.github.io)

<comment><AUTHOR> To everyone facing this issue, please leave a comment here with your browser and system specs:

[https://github.com/reiinakano/fast-style-transfer- deeplearnj...](https://github.com/reiinakano/fast-style-transfer- deeplearnjs/issues/7)

</comment></comment><comment><USER23> I don't know if it's loading or what but it's just blank on my phone's chrome browser.

<comment><AUTHOR> Doesn't support mobile :) Please use desktop (Chrome suggested)

</comment></comment><comment><USER24> The chat program on Line, Rina, does this as well if you send a picture of a person to her.

</comment><comment><USER25> What size images can it generate? I've used other libraries that can't go past 800x600.

<comment><AUTHOR> Theoretically, for this algorithm, there is no size limit.

Practically, if you select something too high, your browser will die for lack of memory or you will grow old waiting for it to finish. My crappy 4GB RAM laptop can only handle around 300 x 500. Try setting the size to maximum, and if your computer can handle it, I'd be glad to guide you in using bigger images.

</comment></comment><comment><USER26> How do I input an image? There's no option to do that on Firefox 58.

<comment><AUTHOR> "Upload from file" option not appearing?

<comment><USER26> No controls at all were appearing. Now, from a different computer --- but still at Firefox 58 -- everything is working fine (maybe you've fixed it in the meantime, maybe not).

</comment></comment></comment><comment><USER27> Error: Failed to link vertex and fragment shaders.

I am on windows 7 using firefox 55.0.3

<comment><AUTHOR> Do you mind trying Chrome? Deeplearn.JS is a new library and there are bound to be weird bugs like this I have no hope of debugging. I believe the most supported version is desktop Chrome.

</comment></comment><comment><USER28> Mobile not supported - please post some screens of results..

<comment><AUTHOR> You can see one result in the repository README! :)

</comment></comment><comment> code-is-code Looks like it does not work on mobile chrome

<comment><AUTHOR> DeeplearnJS does not work on mobile at all unfortunately. It's a new library, they'll get there. :)

<comment> code-is-code That makes sense. An error-message would have been nice.

</comment></comment></comment><comment><USER29> you should likely put a warning message for mobile users to let them know the website doesn't work on mobile

</comment><comment><USER30> I get "Error: Unable to create WebGLTexture." every time I try and run a sample. I got a 780ti, so its no graphics hardware problem...?

<comment><USER31> you probably want to file that on the issue tracker, not here.

<comment><USER30> Why not here? - if someone is going to put something "in my face" that does not work (for me), I do not feel it inappropriate to respond in the same forum.

</comment></comment></comment><comment><USER32> Demo doesn't load anythin

<comment><AUTHOR> Fixed now. Sorry!</comment></comment>


Scribd says: Beat our programmers at a coding challenge - <AUTHOR> http://coding.scribd.com/2012/05/24/compete-against-our-developers/

<comment><USER3> For anyone that is doing CS at Columbia University or considering CU, if you like these types of coding challenges, definitely look into Professor Ross's Programming and Problem Solving class. It was one of the best CS classes I took at Columbia.

Here is his website: <http://www.cs.columbia.edu/~kar/teaching.html>

Each semester consists of 4 different open ended programming problems like these. You work as a team to compete against other members of the class. There's no tests and each class is run as an open seminar where people talk about their strategy and implementation and consider the best approach to solving these problems.

This was my favorite from my year: <http://www.cs.columbia.edu/~kar/4444f02/node18.html>

<comment><USER4> It always blows my mind how many CS academics have these god-awful mid-90s era webpages.

<comment><USER5> It's worse than that. This webpage presents the information in a clear, readable text format with no unnecessary clutter. Why, it's such an old design that it's similar to what Gutenberg was cranking out on his printing press and to what you can buy today on your Kindle. Imagine that!

</comment></comment></comment><comment><USER6> On the game rules page: <http://www.scribd.com/jobs/botrace_rules>

The game starts off with Apples Bananas and Oranges, but ends with Apples, Bananas and Melons.

Robots are a wise choice for safety's sake when dealing with quantum fruit.

<comment><USER7> Fixed, thanks :) We'll deploy a new page tomorrow.

</comment></comment><comment><USER8> I've got a coding challenge for the scribd team: make documents downloads one- click away from the document page and without mandatory login.

</comment><comment><USER9> You may need to consider a better way to rank players - percentage of games won is problematic (as was discovered during the Tron Google AI challenge) and something like TrueSkill or bayeselo would be more robust.

<comment><USER7> Yeah, we're seeing the first problems with that right now- newly uploaded bots progress faster than they should because they play against a more "diluted" pool of opponents than the oldtimers did. We'll think of something.

<comment><USER9> I also think your alpha-beta bot is objectively better than the (currently) #1 bot.

Easiest thing to throw together now is a script which just feeds the whole game history into bayeselo as a bunch of fake chess matches, with the white move advantage set to 0. That's what was done for Tron.

<comment><USER10> :(

</comment></comment></comment></comment><comment><USER11> The more I think about it, the more I like it as a recruiting venue[1]:

* A good way to measure the potential recruit skills

* Gives a good fun experience to go over later at the interview, both verifying it is the recruit who coded it and setting common background beforehand

* You get people who showed some interest in your company, seeing the contest

* You get people who are interested in hacking around puzzles

[1] (I don't know if scribd sees it that way)

<comment><USER12> Adding to what you said, someone who discovers, attempts and solves this solution is already better than 70% of the programmers out there. They show promise of being motivated to crack a puzzle that robs them of their sleep, and take the time to learn and code against the Scribd API.

Even if they don't "win", they are greatly superior than a normal resume pusher.

</comment></comment><comment><USER13> I'm pretty sure I was the guinea pig for this system and I have to say, it was one of the more enjoyable interview tasks I have had to perform.

There is just something extremely satisfying about watching your little robot run across a field collecting little fruit.

<comment><USER14> If we started asking this as an interview question, NOBODY would pass. Time after time we have "senior" developers who fail to write very simple functions, like to count the number of occurrences of each letter in a string.

For me, personally, it sounds like an awesome question, but it's not clear how you would judge it. Let interviewees' bots fight?

<comment><USER15> Wait...when you say fail are you saying they fail to produce something optimal or fail to produce working code?

<comment><USER14> They fail to produce working code. I've written about this a number of times.

Most candidates CANNOT write a string reverse function without the off-by-one error.

</comment></comment></comment></comment><comment><USER16> Smart that they implemented a 10-second time limit, because my first thought was to write a bot that would pass the current game state to Mechanical Turk and have a person solve it. It would be interesting to see how a human fared against Scribd's developers' bots.

</comment><comment><USER17> ...Must keep working on Gridspy.... Must not enter fun contest...

 _twitch_

</comment><comment><USER7> Btw. here's a game, played between our two best bots so far:

<http://www.scribd.com/job_game/match/263907>

<comment><USER18> Can we use nacl?

<comment><USER7> We _do_ plan to add some additional languages. Next on our list are: Haskell, Lua, Python.

<comment><USER18> Sorry to be specific is the sandbox chrome or just a js engine? If it is chrome I could compile and run a nacl module to improve speed.

<comment><USER7> It's using SpiderMonkey right now. Using nacl as sandbox is a neat idea though.

</comment></comment></comment></comment></comment><comment><USER19> Given that each move must be made in 10 seconds, here's my approach. I'm proposing that you use machine learning to evaluate moves, to approximate the exact (search-based) scoring. I'm not implementing this, but feel free to ask me for more details:

Training phase:

Create a random board configuration. Exhaustively explore the search space to find whether this is a win +1, lose -1, or draw 0 for Player 1. You now have a training example: (board configuration, game outcome)

Now, train a neural network (or other non-linear ML model, e.g. SVM) to predict the expected outcome based upon the board configuration.

Deployment phase:

Port the neural network to Javascript. For each possible move, use the neural network to predict the outcome of that move. Pick the move with highest expected outcome. The neural network will run in constant time, most likely well under 10 seconds per move.

<comment><USER20> > Create a random board configuration. Exhaustively explore the search space > to find whether this is a win +1, lose -1, or draw 0 for Player 1

Given that the search space can grow O(4^mn), this can be done only for endgame configurations. Further, not knowing any bounds on the board size makes the input representation difficult to define for a such machine learning approach. And, your target should probably be the weights of an evaluation function, rather than the exact game outcome.

As for the learning algorithm, I know TD-learning was found to be a good approach in various chess programs.

> For each possible move, use the neural network to predict the outcome of > that move. Pick the move with highest expected outcome.

You would likely still want to run an alpha-beta search to pick the move to minimize the prediction error.

</comment></comment><comment><USER21> For people that havn't studied algorithms much, where is a good start on game playing algorithms? Pointers to papers, or names of data structures to look up? Perhaps <http://erikdemaine.org/papers/AlgGameTheory_GONC3/paper.pdf> ?

<comment><USER22> Combinatorial game theory would be the wrong tool for this, I believe, since this isn't a strictly finite turn-based combinatorial game. (And in my opinion combinatorial game theory is more useful for analysis and proof techniques than for creating AI algorithms, but I've only had one class on it.) The blog mentioned "fruitwalk-ab" uses minimax (with the alpha-beta optimization of course), which is the bread-and-butter algorithm for these kinds of games and I expected it to be in 1st place. Sure enough, it is at the moment. (Edit2: No longer.)

In an intro machine learning course you'd learn about minimax and others, but skip paying any money and just read the basic algorithms here and look up the wiki pages for even more examples: <http://www.stanford.edu/~msirota/soco/blind.html> (The introduction has some term definitions.)

Edit: Also, the obligatory plug for _Artificial Intelligence: A Modern Approach_ [http://www.amazon.com/Artificial-Intelligence-Modern- Approac...](http://www.amazon.com/Artificial-Intelligence-Modern-Approach- Edition/dp/0136042597)

<comment><USER21> Thanks for the info, interesting stuff. I've got more first-gen minimax going right now, sorta works but gets stuck in a repeating move pattern. Interesting stuff to work on though, i really appreciate the pointers. Hopefully this weekend i can work out a better heuristic valuation of a game state, for a first run i'm just counting my objects vs theirs(very simple), but i need to somehow weight it such that it prefers picking objects up sooner than later. And of course it would better match if it also took into account groupings, but thats for after i get basic board-clearing working.

</comment></comment></comment><comment><USER23> Hey, I'm writing a bot for this.

One thing though: I think resetting the board state should call new_game() again. This likely only matters when testing your bot, but it's nice for variables that need to be instantiated per each game.

It's pretty fun! My current greedy solution is called: SoGreedy. :)

<comment><USER7> Fixed, thanks!

Your bot is doing pretty well already. :)

</comment></comment><comment><USER24> Hey, honest question here, is it impossible to brute force this and consider (almost) every possible move and get to a non possible to loose point? Given that the board is restricted to 15x15 and the fruits to 81... I'm not saying this is easy, I'm asking if the game AI has an upper limit so to say...

<comment><USER25> The instructions say you need to make a move in 10 seconds, but that's the only ceiling I can imagine for this approach.

<comment><USER24> Ohh, I missed that, that makes more sense now, thought ten seconds is quite a lot, you will very probably not be able to brute force every possibilities before every move. Thanks!

</comment></comment></comment><comment><USER26> [Sorry for bug report here, didn't find another way] I think there is a bug with get_my_item_count() and get_opponent_item_count(). These two functions don't return decimal. When players have 0.5, they return 0. This works locally though.

Example of failed match : <http://www.scribd.com/job_game/match/295703> This line for example 1 / 1 / 0 / 0 / 1 \- first number is the fruit type \- second number is the total fruits \- third number is my fruits \- fourth number is opponent fruit \- fifth number is irrelevant (You'll never see 0.5 in the third or fourth number..)

</comment><comment><USER27> Bot compilation seems to be stuck on this guy: <http://www.scribd.com/jobs/botrace_bot/58>

<comment><USER7> Fixed.

<comment><USER4> I'm getting a 404 when going to any bot page.

<comment><USER7> Log into www.scribd.com, that'll make viewing of the bot pages work. We'll deploy a fix for that later today.

<comment><USER4> Thanks!

</comment></comment></comment></comment></comment><comment><USER26> I think there is some kind of bug where a bot is stopping. Example : <http://www.scribd.com/job_game/match/275573> I simulated two boards where this bug happens online, and it doesn't happen locally.

Also, the API is wrong here : "return the index of that fruit (starting with 0)" => It starts at 1

<comment><USER7> Thanks! We'll fix the API docs.

About the bug: Try to trace() out a few messages in your make_move() function, that might tell you whether something went wrong (the logs will appear at the bottom of the game replays.)

<comment><USER26> OK. Will do that thanks !

</comment></comment></comment><comment><USER28> Scribd has Kramm. You have already lost.

</comment><comment><USER29> I think they need to reevaluate how they score a tie, out of 58 games I have 4 losses and 30 wins and the rest are ties, but my percentage of games won is only 50%ish. Seems like a tie shouldn't be put in the same category as a loss.

</comment><comment><USER30> I see there's a limit on how long the moving function can run. Is there a similar limit on how long the new_game() function can run?

<comment><USER7> 10 seconds as well.

</comment></comment><comment><USER31> If would probably make it easier if they had an interactive version, so that we could learn the game before starting to program.

<comment><USER32> The API is _really_ simple. The simplest version of a bot is:

        function make_move() { return EAST; }

We also provide a downloadable environment so you can test your bot locally.

</comment></comment><comment><USER33> Why are the fruit counts represented as floats?

<comment><USER23> They didn't really have a choice in the matter! In JavaScript, all numbers are floats.

<comment><USER34> Incorrect: <http://www.khronos.org/registry/typedarray/specs/latest/>

Besides, there's Number.prototype.toFixed().

</comment></comment></comment><comment><USER35> What are the grid-size and fruit-count limits like on the server -- the same as in the standalone client?

<comment><USER7> Yep! Width and height are between 5 and 15, and there are up to five different types of fruits on the grid.

</comment></comment><comment><USER36> This looks like a perfect application for using a genetic algorithm solution with two competitive agents.

</comment><comment><USER35> Looks like there are permissions problems when emailing email@example.com

</comment><comment><USER37> Stingy v1 James Ojaste 1200.0%

What's happened with the ranking?

</comment><comment><USER38> How long will this contest be up? Or will this be ongoing?

<comment><USER7> We're planning to keep this running indefinitely.

We may also add additional languages in the future, like Haskell and Lua, and possibly some advanced playing modes.

<comment><USER39> Might I suggest Go? It's already sandboxable, demonstrated by the "Try Go" box right on the front page of golang.org.

</comment></comment></comment><comment><USER40> Seems like the upload a bot link is down...

<comment><USER40> It should be mentioned that you need to register in order to submit. That was my issue.

However, while my bot was compiling, I moved away from the page. Later on trying to resubmit, I kept on getting errors, until I changed the name of my bot submission.

More helpful error messages would greatly improve the whole process. :)

</comment></comment><comment><USER41> this is cool. submitted mine, cross-posted to a few places mine's are prefixed with bosky101 :) ~B

</comment><comment><USER42> can't upload a bot seriously ....

<comment><USER7> Aw, that sucks. We're still fixing glitches in the system, but uploading actually should work.

What kind of error are you seeing?

<comment><USER43> I'm getting 504 Gateway Time-out The server didn't respond in time. on <http://www.scribd.com/job_game/process_uploaded_bot>

<comment><USER44> Same here when trying to upload my bot the first time.

I WANT HIM TO LIIIIVE AND PLAY WITH OTHER BOTS IN THE REAL WORLD ;_;

<comment><USER7> Uploading bot revisions is broken right now- you can try to upload a new bot, or wait for our fix which should go live in a couple of hours. :)

<comment><USER7> Also, feel free to contact us at email@example.com if you see any more issues.

</comment></comment></comment></comment></comment></comment><comment><USER45> This was so much fun! Thanks scribd. :) Humble Hodor and Greebly Wartfinger gooooo.

</comment><comment><USER46> I was going to post how they should do a clearer job of describing the challenge (i.e. using bullet points, headers) and main rules, but I guess there's a separate webpage devoted to the challenge:

<http://www.scribd.com/jobs/botrace_rules>

</comment><comment><USER47> compiling indefinitely...

</comment><comment><USER48> Ah, the new generation of the "design contest."

Thanks, but no thanks. Hire better developers if you need your algorithm.

<comment><USER49> It seems unlikely to me that scribd are actually looking for a solution to this problem, or an isomorphic one. Have you any basis for thinking that?

There are many standard approaches to the general class of two player board games (they are clearly aware of these, with an alpha-beta based bot).

I would guess that what is going to differentiate winners from losers in this specific game are the exact specifics of the setup. So if they are looking for a solution to an isomorphic problem, it must be a _very_ similar problem - <USER2> the general type of challenge is well understood.

It just looks like a neat game to me; I doubt its a cleverly disguised problem from 'social reading and publishing' that they are trying to get a cheap solution to..?

<comment><USER50> I'm with you. It just looks like a lot of fun. My ten year old is really interested in just playing for the sake of play. So we will!</comment></comment></comment>


Amazon Rebrands Twitch Prime as Prime Gaming - <AUTHOR> https://www.theverge.com/2020/8/10/21358463/amazon-twitch-prime-gaming-rebrand <comment><USER2> Previous discussion: [https://news.ycombinator.com/item?id=24111114](https://news.ycombinator.com/item?id=24111114)</comment>


Apple will live stream its iPhone 11 event on YouTube for the first time - <AUTHOR> https://www.theverge.com/2019/9/7/20854050/apple-iphone-11-event-youtube-stream <comment><USER2> They're clearly testing this as a new strategy. I got an advert on YouTube today for the event, and this post from The Verge is likely from Apple's PR team.

I'm interested as to whether the format changes at all. The events have always been part marketing to the public, part press relations, part investor relations (in that order). I wonder if the balance will shift noticeably more towards the marketing side, dropping the review/retail updates, etc.

<comment><USER3> Apple touting how many phones they sold and how many apps there are serves to reinforce to consumers that they're "in the club" with so many other people. In a way it's like seeing how many upvotes there are on a post you also like — your preference is made stronger knowing other people also like it.

</comment></comment><comment><USER4> Smart of them to disable the chat. YT chat is very toxic</comment>


Almost every Cassandra feature has some surprising behavior - <AUTHOR> http://blog.parsely.com/post/1928/cass/ <comment><USER2> The thing about encoding multiple fields within a cell using \x01 somewhat bugs me -- not because it's a hack but because it's yet another example of needless reinvention of wheels. Good old ASCII has characters specifically devoted to separating fields, keys, etc. that no-one uses for anything else. Why not use them instead of inventing a new character that does the same thing? (By the same token, there was never any need for CSV _or_ tab-delimited text since those characters can't be typed into spreadsheet cells; parsing CSV can be a pain, and TABs can be typed.)

If you care, the characters are:

FS -- file separator 1C

GS -- group separator 1D

RS -- record separator 1E

US -- unit separator 1F

As you can see, they also have the benefit of being _self-describing_ (unlike \x01, as the article points out).

(The sad thing is I only learned about these characters because I had to parse files in a 1960s format originally designed to be stored on tape drives -- and they used these delimiters and they worked great.)

<comment><USER3> I only found out about these recently because my company uses them in their custom RPC protocol, which was created about 15 years ago I think.

IMO, there are ups and downs to using unprintable characters in your protocol. On one hand, yeah, you don't need to worry about someone putting a tab in a TSV field and messing up the format. On the other hand, it can make debugging a lot harder because what you see isn't necessarily what you get, and obviously it becomes harder to hand-write requests.

Of course, OP ended up using unprintable characters anyway, so I think they might as well use the ASCII ones. But even if you know about and use those characters, I think there is still a place for [CT]SV.

I'm glad you brought them up though, because I think the nonprintable ASCII characters occupy a very interesting place in computer science -- (almost) universally supported, but (almost) never used.

<comment><USER2> You can make a case for CSV because it's text editor friendly, and tabs because it conceptually maps to typewriters, but we don't even have the option to use these characters. And arguing that typing escape sequences to represent common characters is better is pretty difficult to swallow.

</comment></comment></comment><comment><USER4> The thing is that contrary to many other tools, you can't run Cassandra with almost any of its default settings (maybe besides ports). In most tools you'll need to tweak a few defaults, with Cassandra you need to thoroughly read the docs on every little configuration in the server and schema definitions (especially if you're working in multiple DCs), you'll always find a little surprise if you skim through it.

It's also almost mandatory to read the internal design docs of cassandra even if you're not the admin but just working with it. And modelling data is a lost less trivial than it looks - and almost always not what you assume.

Anyway, this is the best talk I've seen on Cassandra data modeling even if you know Cassandra but not on an expert level. I made sure everyone on my team saw it at least once. [https://www.youtube.com/watch?v=qphhxujn5Es](https://www.youtube.com/watch?v=qphhxujn5Es)

<comment><USER5> I have to disagree. But again given how scalable Cassandra is we could live in different worlds.

I've run multi-gigabyte data sets in Cassandra without any config changes. And when I responsible for a 40 node Cassandra cluster we didn't do any tweaks other than a few JVM settings here and there.

Cassandra I have to admit though is very sensitive to how you model and store your database. The whole tombstone saga is never a fun one to go through.

<comment><USER6> > I've run multi-gigabyte data sets

WOW. You've run several GB of data through it? Wow.

<comment><USER5> That was the small one. The 40 node one had hundreds of terabytes.

</comment></comment></comment></comment><comment><USER7> It's been posted in the comments below, but I'll reiterate it because it needs to be said. This article is more about the issues of naive database adoption strategies than any deep fundamental flaw with Cassandra.

I've deployed C* on some similarly large datasets and encountered none of these issues - even when storing terabytes of time-series data. The difference - I read through the documentation from head to foot before getting started, did numerous dry-runs to figure out the data modeling, and when I wasn't sure about something asked the community (who are usually incredibly quick to respond on IRC, twitter, or elsewhere).

</comment><comment><USER8> We used C* for a similar use case and encountered some of the same issues.

The article doesn't even mention that there are a lot of places in the docs where it says: "If you configure a keyspace like this, your node will most likely crash". Not just degraded performance, any develeoper with some access might crash a node (and maybe the whole cluster) with a userspace error.

The main lesson I took away from Cassandra is that battle- tested@Netflix(/other bigcorp) doesn't mean resilient and might require a engineer on standby at all times to run correctly.

<comment><USER9> > The main lesson I took away from Cassandra is that battle- > tested@Netflix(/other bigcorp) doesn't mean resilient and might require a > engineer on standby at all times to run correctly.

This is entirely true. We are big users/consumers/fans of Cassandra at Spotify. We have approximately one crap-ton of Cassandra clusters with several metric crap-tons of data. But we have an entire team that provides support and tooling around Cassandra. We contribute upstream, and have employed core Cassandra contributors in the past.

In return we get a datastore that scales pretty much infinitely with our data sets, has performance characteristics that we are now well aware of and are able to reason about, and provides us cross-DC replication and topology- awareness. It took years to get to this point though, and to build the operational expertise required to run Cassandra. Only recently have we gotten to the point where teams are able to self-service provision their own Cassandra clusters.

This is a resilient, scalable solution, but if I were to quit tomorrow and start a five-person startup, there is no way I would consider C* as a workable solution.

<comment><USER10> What would you use?

<comment><USER9> Really depends on what the startup was. If it's a big data startup, then I'd put a lot of thought into based on the type of problem. If we were just building an app that needed some kind of data store, who knows, might even use something like Parse.

No need to prematurely optimize as long as you set yourself up to prevent lock-in.

</comment></comment></comment></comment><comment><USER11> Bashing non-COMPACT storage is nonsense.

non-COMPACT store by itself doesn't add any significant overhead (only 2 bytes per cell and even less after compressed). What really wastes space is using collections types. Even using that I doubt it will ever reach the 30x mark the author stated.

<comment><USER12> The article is full of nonsense and the OP is here so criticism is being downvoted.

<comment><USER13> huh?

<comment><USER12> -3 every time I post in this thread. I guess I'll keep posting.

</comment></comment></comment></comment><comment><USER14> I think you can sum up some of the grievances of the OP as "felt mislead by Cassandra/DataStax marketing". But, some of the criticism is not really the fault of Cassandra or DataStax at all IMO.

\- CQL stands for Cassandra Query Language. As a user of C*, I don't remember anywhere in docs it claiming it conforming to the SQL standard. They do mention it to be "similar to SQL" which is a fair comparison.

As a counter-example of another somewhat similar database, AWS DynamoDB. The absence of a CQL like syntax really frustrates me.

\- Data Modelling: Effective data modelling is difficult in all DB systems, inherently more so in NOSQL or non-ACID distributed DBs.

\- Counters & Collections: I feel that criticism is legit. I felt similar pains too. I've learned the lesson there not to trust all marketing claims.

</comment><comment><USER13> OP here. Surprised to find this article on the HN front page, a month after we originally posted it.

Glad to answer questions. Ask me anything!

<comment><USER15> So are you still running Cassandra? And why rely on counters for analytics when it's wildly known to be a bad idea.

<comment><USER13> We are still running Cassandra. It's used in a more restricted way than we originally thought we'd use it, but we're following the recommendations we wrote up in linked article.

We don't use counters -- at all. (This is discussed in one of the sections.) Counters are one of the few Cassandra features that can offer you some form of value aggregation inside the data store, but we decided they weren't worth it, due to the quirks.

What do you mean by, "when it's wildly known to be a bad idea?" What are you referencing? Lots of people use counters in Redis and MongoDB for analytics, for example.

<comment><USER16> Counters in Cassandra seem to have been politically driven (something about the design being influenced by Twitter), and given the fact that Cassandra values availability over all us, best effort counters at best. So you can't trust them to be 100% accurate (or so I've heard). This is another case of "to run Cassandra you really have to know Cassandra" as Cassandra counters provide a different set of guarantees than Redis (which is single instance) and Mongo (which chooses consistency).

</comment></comment></comment></comment><comment><USER17> Cassandra or most NoSQL databases seem to require a lot more internals knowledge than what a development or an ops team would have. Most sysops teams have a cadre of certified DBAs and administrators and they get on by without any surprises. The one thing Oracle got right was their certification program which ensures there are no gotchas -- most DBA can deliver smooth operations. I guess it is because these NOSQL products are relatively new, they require only ninja level experts to fiddle with the controls.

I would be interested in hearing more from someone who has use a third party managed cassandra service like Google' casandra product [https://cloud.google.com/solutions/cassandra/](https://cloud.google.com/solutions/cassandra/). Did you still deep internal knowledge to use Cassandra ? This is important to know for me because my organization will not have the resources to manage Cassandra clusters but we need a Cassandra like store.

<comment><USER18> On the other hand, if you tried to fit the article's use case -- at that scale -- into something like PostgreSQL you would still have to start learning the internals.

I think the article author's problem as that they assumed Cassandra would be _similar_ to the technology they were already familiar with, which was not the case; it's different in so many areas, especially with regard to its performance profile, and to someone used to relational databases it's outright alien.

At the scale they're describing, they _do_ exceed the threshold point at which one has to learn the internals of the technology. In engineering, scale is everything. Riding a bicycle doesn't require that you know how bicycles work, but sending a rocket to the moon demands a lot of knowledge about a wide range of subjects.

</comment></comment><comment><USER19> This Team Used Apache Cassandra Without Reading The Fine Manual… You Won't Believe What Happened Next

      "You honestly expected that adopting a data store at your scale would not require you to learn all of its internals?"

These aren't even internals, these are basic facts about Cassandra.

<comment><USER4> I do believe Cassandra requires a much more intimate knowledge of its internals if you want to do anything serious with it - more so than any other database I've worked with.

<comment><USER5> I have to agree. But with the clarification that it isn't significantly more. Spend a few days and you will be an expert.

And I think it's worth noting that the "serious" things you would do on Cassandra more often than not you couldn't do on most other databases.

</comment></comment></comment><comment><USER20> I always assumed Cassandra to be the easiest-to-operate distributed database. What are other choices could I make? I'm primarily interested in using it as a distributed, on-disk, datacenter-aware hash table.

</comment><comment><USER21> Thank you for changing the name of the title. The default title is clickbait which I find very surprising given this is a tech post and IMO the audience for such posts hates these titles.

<comment><USER22> It's a parody of clickbait.

<comment><USER13> OP here. Indeed, it is a parody of click bait, although in retrospect I think I should have avoided it.

Parse.ly's customers are media companies so we deal with news / media / headlines all the time, so we were being cute in referencing this practice in the article. I used the headlines as a mechanism to break up the prose, too, and add some levity, since this article weighs in at over 4,000 words.

But I underestimated the ability of techies to scan the headline and paragraph one of an article and say, "not worth my time" and bounce instinctively. In fact, when I tried to submit this post to r/programming, the moderators instantly deleted it, referencing the headline, despite it being a parody.

One thing I shouldn't be surprised by: The Internet adapts!

</comment></comment></comment><comment><USER23> SQL is "trendy"?

<comment><USER24> It depends on the phase of the moon.

</comment></comment><comment><USER25> I love that the article pokes fun at click-bait articles ... I'm not a Cassandra user but having used several other NoSQL solutions, I'm not at all surprised that this team, who dove into a new technology without caution, was surprised.

<comment><USER5> It's not just dove in without caution but without fully checking design decisions.

For example using Maps other than for a storing a handful of values has always been discouraged.

</comment></comment><comment><USER12> "We were actually hoping that Cassandra could store things more compactly than our raw data"

Why in hell would you think that?

CQL is not SQL? No shit. Actually, the docs pretty clearly spell that out. Anyone with even a cursory knowledge of Cassandra or any big data store would know you need to understand how the system works, and what it's limitations are before you model your data for the system.

COMPACT STORAGE is for backwards compatibility. Turn it on and you are turning off CQL3: [http://docs.datastax.com/en/cql/3.0/cql/cql_reference/create...](http://docs.datastax.com/en/cql/3.0/cql/cql_reference/create_table_r.html) [http://www.datastax.com/dev/blog/whats-new-in- cql-3-0](http://www.datastax.com/dev/blog/whats-new-in-cql-3-0)

Counters are only usable for things where you don't care if the counter gets increased exactly one time. It is for stuff that doesn't really matter, like "likes" on a page or something.

I'm not even reading "Check Your Row Size: Too Wide, Too Narrow, or Just Right?" because measuring the number of columns max and the ~10MB performance limit is trivial to do and this is a datastore designed by data scientists for people who are willing to do the very little bit of knowledge gain required.

This whole article reads like someone who decided to incorporate Cassandra without actually spending any time learning how to do it correctly before trying to do it.

"He said, "You honestly expected that adopting a data store at your scale would not require you to learn all of its internals?" He has a point."

No shit.

<comment><USER26> Hey, it sounds like you know quite a bit about this topic, and passion for technical matters is something everyone here understands. But please follow the HN guidelines when commenting. It takes some effort, but it's effort we all need to make.

[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

</comment></comment><comment><USER27> I still think of Cassandra as the tool the Digg engineers used to kill Digg as we all loved/hated it at it's time.

<comment><USER15> If a company blames its demise on a piece of tech, then they either need better engineers to change that tech. Or the product had no future at all. And Cassandra has changed a lot since then.

<comment><USER27> What I said still is true. Digg USED Cassandra to destroy itself. They changed business plans and removed the community with a ton of down time and a loss of data.

I agree it wasn't Tech fault for Kevin Rose allowing VCs to control Digg, but there were issues with Cassandra and especially during the SQL vs NoSQL debate at its highest levels.</comment></comment></comment>


Fraudster Extradited After Stealing Almost $750k from UCSD - <AUTHOR> https://www.justice.gov/usao-sdca/pr/fraudster-brought-back-kenya-face-jail-time-stealing-almost-750000-ucsd-through-spear <comment><USER2> I assumed these kind of attacks are done through layers of middlemen (who don't really know what they're doing) for both the money handling and the actual phishing.

Am I wrong?</comment>


Project Management via Game of Thrones - <AUTHOR> http://blog.smartbear.com/project-management/everything-i-know-about-project-management-i-learned-from-game-of-thrones/

<comment><USER2> Project management is too detached from reality and promises things comparable to transporting 40,000 horses across the ocean? I know, let's fix it with advice from an _entirely fictional_ world!

<comment><USER3> Lighten up sourpuss.

<comment><USER2> That was a very effective line in kindergarten.

</comment></comment></comment><comment><USER4> It is fun article with some apt analogies and I am glad that I decided to read it despite the title. To me at least, "Game of Thrones" TV series would be a horrible project management tutorial, as almost all characters in it do not have a slightest idea what they want and/or how to achieve it, changing sides in a conflict on a whim. Obviously it is about a journey, not a destination for Westeros nobles.

</comment><comment><USER5> Because of your post, Mr. OP, I shall check out Game of Thrones. I have always been fascinated by how stories can teach us so many things if we look beneath the words. For one, I would highly recommend "Romance of the Three Kingdoms", a Chinese literature piece that tells the stories of various leaders and their strategies on leading armies to victories. NOTE: I am not in favour of war but merely the way that they are able to raise the morale of people towards a cause - something that would be helpful to all businesses.

<comment><USER6> I would strongly suggest you read the novel series, A Song of Ice and Fire, instead of watching the TV show. Hours and hours of worth of detail had to be cut from the series to convert it for TV, and while they did a great job, you're left with little more than a highlights reel by comparison.

The novels are a must if you're after the detailed history and politics of the world the characters live in.

<comment><USER7> I debated with myself whether I should read the books first, but:

On the other hand, I have read that some bookreaders recommend to first watching the TV series as while it does "dumb down" a couple of things, it really helps keeping an overall thread. Plus watching the series second might leave you more disappointed ("why didn't they include my favorite quote X!?") than the other way around ("I love reading more about this character that I already liked!").

<comment><USER3> watch the show first otherwise you will hate it... i made the mistake of picking up the books after season two and now watching season three is torture

<comment><USER8> watching season three is torture either way

</comment></comment></comment></comment></comment><comment><USER9> More Game of Thrones-related points to note.

HAVE A PLAN - it's no good setting out on a major undertaking if you don't know where you're going. Will your loyal followers continue to stick with you, if after a while it feels like you're just going round in circles?

UP OR OUT - growth and development is part of the cycle of life. Today's fresh blood will be tomorrow's experienced hand. Will their assigned role still be suitable?

</comment><comment><USER10> Maybe one more lesson from Tyrion Lannister:

UNDER PROMISE BUT OVER DELIVER. All to often a project manager feels compelled to promise the moon and the stars, inevitably setting themselves up for failure or at least a dissatisfied client.

</comment><comment><USER11> To the author (if self posted). The distraction of the green menu pullout icon made me unable to read the text on my iPhone. I gave up after realizing the unpleasantness caused by the distraction.

<comment><USER12> Another annoyed iPhone reader here.

Normally the Safari "Reader" button would light up in the URL bar and I'd use that to read the article without distraction, but it didn't for some reason (content not properly marked up in the HTML?).

<comment><AUTHOR> Looking into this now. Sorry about that!</comment></comment></comment>


Microsoft's Holiday Disaster: PC Sales Shrank For The First Time In 5 Years - <AUTHOR> http://www.businessinsider.com/microsoft-pc-sales-shrank-for-the-first-time-in-5-years-2013-1

<comment><AUTHOR> essentially ...

"Windows 8 wasn't going to be as big a catalyst," said Shaw Wu, analyst at Sterne Agee. "It's so different, it's almost uncomfortably different from past Windows, and there's a risk that Windows 8 ends up like Vista."</comment>


QuickOffice Will Be Discontinued - <AUTHOR> http://googlesystem.blogspot.com/2014/06/quickoffice-will-be-discontinued.html <comment><USER2> The announcements says the discontinuation is because of "the integration of Quickoffice into the Google Docs, Sheets and Slides apps". But I don't see any sign of this integration. With the latest Docs app, I can't open a .docx file from my Google Drive; only Document Viewer and Quickoffice can open it.

Not that I'm too fussed; I never edit documents on mobile anyway. I use styles often, and neither Quickoffice nor Docs on Android support styles (e.g. header styles).

<comment><USER3> I believe they announced this integration at Google I/O. When you get an office document as an attachment, you no longer have to download it or import it into Docs. You would be able to open, edit, and save it in the same office format right from Docs/Sheets/Slides.

</comment></comment><comment><USER4> Will I be able to uninstall the one that was preinstalled on my phone?

</comment><comment><USER5> Not really too surprised seeing that Google Drive/Docs/etc. now allows MS Office file editing. Glad to see that I will gain a free spot in my app drawer now that the Drive apps for editing are all separate.

</comment><comment><USER6> Has google drive/docs won the space, or is there still interest in a non- cloud-based document / spreadsheet / presentation editor?

<comment><USER7> I'v heard that businesses, the primary users of document/spreedsheet/presetation editor software are totally okay to put their confidential information in cloud storage format.

<comment><USER8> In that case they are being idiotic.

It only takes one person doing something on a compromised internet connection (Captive portals that replace certificates, etc, etc) for your "confidential" information to be leaked.

(Note: if said cloud storage does encryption on top of HTTPS and/or proper certificate pinning (although this can cause other issues), this may not be the case. But that is not true too often, for example with web interfaces.)

<comment><USER9> I'm pretty sure that was sarcasm.

</comment></comment></comment></comment><comment><USER10> Time to go back to OfficeSuite again!

</comment><comment><USER11> google is training me not to use their new products.

</comment><comment><USER12> Blog spam. Actual update is here: [http://googleappsupdates.blogspot.com/2014/06/removal-of- qui...](http://googleappsupdates.blogspot.com/2014/06/removal-of-quickoffice- from-google-play.html)

<comment><USER13> The linked article has a lot more information than the official announcement.

<comment><USER14> Ok, we switched it back.</comment></comment></comment>


Ask HN: Scraping Answer Sites for ML Project? - <AUTHOR>

Hey,
Does anyone have any advice on how to download yahoo answers without angering anyone?  Is this dataset published anywhere?
I'm putzing around with some statistical methods to automatically predict which answers will be highly ranked on question/answer sites like yahoo answers, stackoverflow.com, et. al.
So far I have trained my methods using the published data dumps of stackoverflow.com - the results are interesting/encouraging and I'd like to work with a softer dataset like yahoo answers where the questions are less technical.
(Incidentally this method gives interesting results for predicting points of comment threads on HN, however I refuse to release this without making a nice interface for people to browse) <comment><USER2> That's a cool project. Let me know if you need any help. I'm not sure what I could do but it sounds interesting.

Here a couple ideas that might help (not that I condone violating a site's TOS).

Have a random time interval between each page download so someone looking at the logs doesn't see a regular pattern.

Pick 4 or 5 common user agent strings to alternate randomly between.

Go through free proxies, or package up your program and have 5 or more friends run it on different sections of the data you want to download.</comment>


Ask HN: Why can't I delete my HN account? - <AUTHOR> I'm kind of fed up with the 16 year olds that dominate HN these days, and I'd like to delete my good advice from its pages. However, HN doesn't allow for account deactivations or deletions.<p>What gives? <comment><USER2> I'm not sure, but consider that your HN username is something you might submit when applying to YCombinator.

So, it makes sense that they wouldn't let you delete the account; for better or for worse, you might have to explain your posting history to PG during an interview.

Also, consider how annoying it would be to remove all references to a deleted account from threads--not really worth the computational power.</comment>


YouTube demonotised my favourite channel - <AUTHOR> https://www.youtube.com/post/UgwnW7YW9w47-Sc8_Jp4AaABCQ <comment><AUTHOR> It claimed the channel to upload repetitive content. But this doesn't make sense. It takes a lot of research and editing to create such videos. Google is acting like a big bully now.</comment>


Thriving Norway provides an Economics Lesson - <AUTHOR> http://www.nytimes.com/2009/05/14/business/global/14frugal.html?_r=1&em

<comment><USER2> > Norway is a relatively small country with a largely homogeneous population > of 4.6 million and the advantages of being a major oil exporter.

This probably has more to do with their successes as a country than anything else. Smaller populations are easier to govern (New York's population is around 20mil), and being an energy exporter ensures that there's enough wealth to go around to provide it's citizens with a large social safety net.

<comment><USER3> Agreed. And some people are already saying that. From the article itself:

"We have become complacent," Mr. Mork added. "More and more vacation houses are being built. We have more holidays than most countries and extremely generous benefits and sick leave policies. Some day the dream will end."

<comment><USER4> _The government enjoys a budget surplus of 11 percent and its ledger is entirely free of debt._

The have a huge surplus and are saving 11% of their governmnets bugdet. At some point their savings could probably fund all government spending at which point running out of oil would not be the end of the world. Granted government spending is only 40% of GDP but 30 years from now they could have 40% unemployment and still have a stable functioning society.

<comment><USER5> But how safe is their investment? What can they own to keep themselves in a comfortable lifestyle (nearly) forever? Stocks and currency won't do it.

It's probably part of the reason for the push to renewable energy and energy efficiency. And also services/IP?

</comment></comment></comment></comment><comment><USER6> Ok, Norway is a net oil exporter and has huge benefits because of that... But can you please stop mentioning that every time you read Norway, completely ignoring everything else they managed to do right? USA had much more natural resources than Europe, that doesn't mean you should ignore everything else they did well over time to get where they are now.

</comment><comment><USER7> So the secret is to be a country full of dour, responsible people who discover fabulously valuable natural resources, and can thus afford to subsidize even the most degenerate kinds of behavior?

<comment><USER8> Degenerate behavior like old age, education and accidents?

<comment><USER9> Did you read the article?

"Just around the corner from Norway's central bank, for instance, Paul Bruum takes a needle full of amphetamines and jabs it into his muscular arm. His scabs and sores betray many years as a heroin addict. He says that the $1,500 he gets from the government each month is enough to keep him well-fed and supplied with drugs.

Mr. Bruum, 32, says he has never had a job, and he admits he is no position to find one. I don't blame anyone, he said. The Norwegian government has provided for me the best they can."

Why should other people be forced to subsidize that kind of lifestyle?

<comment><USER10> The people subsidize it because they believe doing so makes everyone (including themselves) better off that way.

<comment><USER9> See <http://news.ycombinator.com/item?id=609997>

The salient point is that even if I'm happy to subsidize that sort of lifestyle, I don't think the government should make the decision for me (especially not at the point of a gun).

<comment><USER10> smanek, putting aside moral considerations I think you are discounting the actual cost of leaving people like Mr. Bruum to fend for themselves and ultimately end up being incarcerated. Suppose you live in California where incarceration costs upwards of $3,500 a month. Cutting him off would end up costing you twice as much.

</comment></comment></comment></comment></comment></comment><comment><USER11> In some ways, Norway is approaching a post-scarcity economy mentioned in an article that was posted here recently (<http://news.ycombinator.com/item?id=607540>).

</comment><comment><USER12> This article doesn't tell the whole story. On the whole, the Norwegian economy is strong - probably stronger than most other nations, but that doesn't mean there aren't problems here. I have several friends who have been laid off this year, and even the company I worked for laid off a number of employees earlier this month.

That said, I'm still happier to be working here as opposed to being back home in the States. And, while Norwegians do work fewer hours, most of them are more productive during the hours they do work.

</comment><comment><AUTHOR> If you compare Norway to the UAE (which includes Dubai, Abu Dhabi, etc) while Norway is investing its oil wealth in a sovereign fund, UAE is investing in real estate, tourism and other sectors. Both have rich social safety nets. In this way, both these energy exporting nations are smart in reinvesting/saving today's surplus. One interesting difference is that UAE is a federation of fairly autonomous states or emirates. So some states are already in the post- scarcity era (Dubai) while other states are still in the surplus era (Abu Dhabi, Ras Al Khiamah, etc). It makes for an interesting economic development scenario.

<comment><USER13> Yes, but wheras Dubai employs slave labor to build ecological atrocities, Norway is investing the money in the future of its people. It is also quite interesting that dealing with middle-eastern immigrants is a controversial issue in Norway (especially because they largely refuse to assimilate.)

<comment><AUTHOR> Slave labor is wrong. Dubai is wrong for using it. So is anyone else who does. Norway rightly deserves credit in this regard. My initial comment is from a macro-economic point of view of how countries use/abuse their natural resources.

</comment></comment></comment><comment><USER14> This rebuttal on Forbes helps put Norway's oil wealth into perspective.

[http://clipmarks.forbes.com/2009/05/14/the-real-norway- lesso...](http://clipmarks.forbes.com/2009/05/14/the-real-norway-lesson/)

<comment><USER15> "Norway has 20 times as much oil production per capita as the UK. That's the real Norway lesson. Sit on tons of oil."

Haha, brilliant. Sounds a little like a lottery winner offering financial advice.

</comment></comment><comment><USER16> "Oslo retains a feeling of modesty reminiscent of a fishing village rather than a Western capital, with the recently opened $800 million Opera House one of the few signs of opulence."

What a bullshit article. Sorry...

Lessons learned: it's good to have a lot of oil.

Did they invest so wisely? Maybe, maybe not - even if they did, who says they were not just the lucky ones that are bound to exist (just as with fund managers)?

<comment><USER17> _Did they invest so wisely?_

Here's one case (and I'm sure there are others) which is a resounding no: <http://www.nytimes.com/2007/12/02/world/europe/02norway.html>

<comment><USER18> Well that's not quite the same, since that story is about municipalities investing poorly. Hopefully Norway's sovereign wealth fund didn't do likewise.

(as an aside: I've been to Narvik, in the summer, and the scenery is absolutely stunning. The 2am sunshine is also amusing to behold.)

</comment></comment></comment><comment><USER19> As others have pointed out, if you take away the oil, Norway's economy is crap. The infrastructure sucks, there aren't really any other industries to speak of, taxes are very high and the incentives for someone to start a business there are appallingly low.

</comment><comment><USER20> Isn't Japan with zero natural resources as better case study?</comment>