Salsify – A New Architecture for Real-time Internet Video - <AUTHOR> https://snr.stanford.edu <comment><USER2> To me, the key innovation here is the tight integration between network conditions and codec frame size. Standard codecs are created with specific bandwidth requirements and they provide encoded frames that 'average' around that size. You _could_ just re-initialize a codec at a lower bandwidth on the fly, but you would have to send an I frame (large full frame) to kickoff the new series of frames (as video most video frames are just updates of a previous frame). Having a codec accept a bandwidth target per frame is a really good idea.

<comment><USER3> Codecs used by real time video systems are able to adjust the bitrate on the fly. There's not a keyframe request every time that happens unless resolution is changed. How quickly they adjust might vary, software implementations generally does it for the next encoded frame. The frame still will be somewhat larger or smaller than the target size since the codecs can't accurately predict the encoded size for given quality parameters.

The Salsify implementation in the paper has slightly more accurate way of producing one single frame as it encodes two frames with different quality targets and takes the largest one below the frame size target.

<comment><USER4> For a resolution change, couldn't you just scale the old last frame to the new resolution and use that as the basis for more P frames? (originally replied to the wrong comment)

<comment><USER5> Short answer: Yes.

Longer answer: The codec needs to support it. Codecs actually allow prediction from multiple reference frames, and maintain a buffer of them (2 to 16, depending on the codec, profile, and level). An individual frame may refer to several (potentially all) of those. So re-scaling up to 16 frames for every frame you decode will get quite expensive, not to mention the generational losses of doing this repeatedly for every resolution change. In practice what happens is you scale individual blocks when they get referenced by the current frame. But that has to be integrated into the motion compensation routines of the codec.

Both VP9 and AV1 support this, for example.

</comment></comment></comment></comment><comment><USER6> Aside from the fact that the tech is obviously cool, I think the FAQ section is really well-written. Props to the team.

<comment><USER7> I agree, refreshingly top-notch FAQs!

<comment><USER8> It reminds me of the Def Con and Coachella FAQs

</comment></comment></comment><comment><USER9> I'm fairly sure that Ben Orenstein and friend are forming a company to commercialise this as a Screenhero replacement. Discussed on this podcast: [http://artofproductpodcast.com/episode-39](http://artofproductpodcast.com/episode-39)

Very interested to see what they cook up (and kinda envious I didn't have the idea / don't have the space in my life to have a crack at it myself---it sounds very interesting).

</comment><comment><USER10> I've been taking Financial Markets course by Robert Shiller and he continuously makes the point when talking about inventions and new ideas that "it's crazy to me that this didn't exist before". It's usually the sign of a really good invention when you have that thought. And that's the thought I'm having looking at this combining the codec and transport protocol together: "Why hasn't this been done before?" == "This is awesome!"

<comment><USER11> That's easy to say in hindsight, but it's easy to come up with all sorts of crazy ideas, ask yourself _" why hasn't this been done before?"_, and then find out that the answer is _" it has, it turned out to be a terrible idea, and that's why I've never heard of it"_.

</comment></comment><comment><USER12> A bigger frustration I experience is that some streaming seems to just "give up"; stalling and never resuming. I know the connection and server are okay because I can usually force it to resume manually, e.g. doing a page refresh, so is it just bad server architecture or a codec problem?

<comment><USER13> Youtube fixed this problem many years ago, but it recently became un-fixed. New codec?

<comment><USER14> Maybe, it happens on FF fairly frequently but never (that I can recall) on Chrome (same distro Fedora 27).

</comment></comment></comment><comment><USER15> Um ... from the paper ...

"6.1 Limitations of Salsify

No audio. Salsify does not encode or transmit audio."

Claiming that you beat a bunch of codecs that have synchronized audio (even though they disable it) is kind of misleading ...

<comment><USER16> Co-author here. Totally reasonable reaction, and we've heard this when the paper was posted elsewhere (e.g. on Reddit), but have not heard it from specialists, and honestly we suspect it's probably a red herring. Salsify's gains on the "delay" metric are mostly coming from two things: (1) the way that it restrains its compressed video to avoid building up in-network queues (which audio must also transit) and provoking packet loss, and (2) the way that it recovers more quickly from network glitches (check out the video).

If you wanted to add audio to Salsify, you would want to control a receiver- side video and audio buffer to reduce audio gaps and keep a/v in sync during periods of happy network, but this is unlikely to affect the system's ability to recover more quickly from glitches or to avoid building up in-network queues that delay audio and video alike. If you watch the video (or see Figure 6(f), Figure 7, and Figure 8), I don't think there's much reason to think audio can justify what the Chrome/webrtc.org codebase is doing -- WebRTC's frame delays are distributed over a broad range (so it's not like they're synchronized to some fixed timebase either) and are very high, especially in the seconds after a network glitch.

More to the point for our academic work, it would have been trivial to add shitty audio that made no difference to the metrics. The hard-but-necessary part is in designing an evaluation metric to assess (1) the qualify of the reconstructed audio (including how many gaps/rebuffering delays were there when the de-jitter buffer went dry), (2) the delay of the reconstructed audio, keeping in mind this is not constant over time, (3) the quality of the audio/video synchronization, which also will not be constant over time. Then measuring that in a fair way across Skype/Facetime/Hangouts/WebRTC/Salsify, and then trying to decide which compromise on those three axes is desirable. Somebody should do all that work at some point, but it's a major piece of work to bite off and pretty far from anything we've done so far.

<comment><USER17> Opus, with its low delay and solid rate controls would seem to be the natural pair here. But I agree audio is likely not the real problem in this space.

Any reason you didn't choose to start from VP9? Is the encoder still too slow overall?

</comment></comment></comment><comment><USER18> Is this at all related to the company Salsify? [https://www.salsify.com](https://www.salsify.com)

<comment><USER19> Seems highly unlikely, at a quick glance there's no overlap.

From the FAQ:

> Why the name "Salsify"?

> It's not a very interesting reason. Salsify comes from an older project > called "ALFalfa," for the use of Application Layer Framing in video. Alfalfa > gave way to Sprout, a congestion-control scheme intended for real-time > applications, and now Salsify, a new design where congestion-control (in the > transport protocol) and rate control (in the video codec) are jointly > controlled. Alfalfa, Sprout, and Salsify are all plantish foods.

The company you linked seems to be meant as "sales-ify".

<comment><USER20> Salsify the PXM company comes from salsify the root vegetable and is pronounced the same way. Our logo is a stylized salsify flower.

</comment></comment></comment><comment><USER21> Slighty tangential...

 _Salsify is led by Sadjad Fouladi, a doctoral student in computer science at Stanford University, along with fellow Stanford students John Emmons, Emre Orbay, and Riad S. Wahby, as well as Catherine Wu, a junior at Saratoga High School in Saratoga, California. The project is advised by Keith Winstein, an assistant professor of computer science.

Salsify was funded the National Science Foundation and the Defense Advanced Research Projects Agency (DARPA). Salsify has also received support from Google, Huawei, VMware, Dropbox, Facebook, and the Stanford Platform Lab._

Financially supported by the government, tech juggernauts, and executed by top tier doctoral students + a high school student + a top tier university professor.

Assuming this could be game-changing innovation to further advance worldwide communication, it's refreshing to see the positive externalities of a combination of capitalistic (F500 tech co's) and socialistic (university, government) systems executed by a seemingly diverse set of actors.

<comment><USER22> How is university and government funding socialistic? Neither involves the workers' ownership of the means of production (and indeed, they exist in a state that is anything but socialist).

<comment><USER23> Socialism is not Communism.

<comment><USER22> That's correct, but it does not invalidate my point. Socialism involves the workers' ownership of the means of production.

</comment></comment></comment></comment><comment><USER24> Even Richard Hendricks didn't combine the codec and the transport. Genius.

<comment><USER25> Richard Hendricks is an idiot who is currently attempting a private takeover of the Internet (presumably) on behalf of the NSA.

<comment><USER26> I don't get why they say pied piper is decentralized and a "new open internet" when pied piper is building it and is closed source

<comment><USER27> Because the majority of people watching the show either don't know or don't care.

</comment></comment></comment></comment><comment><USER28> (Disclaimer: This comment is my personal opinion, not that of my employer.)

Really exciting work.

Encoding multiple versions of a video and picking a smaller one in response to congestion already happens for video-on-demand (think YouTube and Netflix videos) in DASH. That said, with VOD you can encode the video slower than real-time.

I can't imagine this ever making it into Skype/FaceTime/Hangouts/Duo. The big corps will probably continue to focus on "more internet" (fiber optic, zero rating, wi-fi hotspots, and internet traffic management practices).

<comment> a-dub DASH uses big chunks of operator selectable size though as it is codec agnostic. I wonder if coupling transport and codec could have benefits for the massively scalable VOD case. (ie pre-render a bunch of stuff up front and run a "lite" codec that is coupled to the transport layer and aware of network conditions)

</comment></comment><comment><USER29> The cost here seems to be bandwidth, is this because you're using VP8? Could this be adapted for other codecs like AV1?

<comment><USER30> The cost is encoding frames that won't be sent, and non constant frame rate. Bandwidth is more optimally utilized.

<comment><USER29> Let me rephrase the question. Assuming that in the video the red background is the network capacity and the lines are the video bitrate used, salsify seems to be using ~6000kbps and WebRTC ~2500kbps. Is this higher bitrate because of you're using VP8, or it a limitation of the protocol? If it's because of VP8 how hard would it be to adapt to modern codecs like HEVC and AV1?

<comment><USER31> I don't see anything in what they did that won't work with HEVC or AV1. They just hacked VP8 to be able to save and restore codec state per frame so they can generate multiple versions of the next frame, choosing smaller when network condition is bad. Their innovation is in preventing congestion rather than reacting to the aftermath of congestion.

<comment><USER2> Right, they may have used VP8 because VP9 and HEVC are more CPU intensive and they could technically encode more frames per second than the input 60 FPS.

<comment><USER32> They probably used it because source code was most available.

</comment></comment></comment></comment></comment></comment><comment><USER33> It's 2018 and I still have many dropped calls and other weird stuff when I talk with people on my mobile. FaceTime Audio is often a good alternative but still not perfect. So, I really hope the audio version of this will be commercialized soon.

</comment><comment><USER34> Unfortunately this would only apply to one-on-one low latency video chats. For streaming to an audience, which generally uses a distribution network between the user and the video source to help handle load and geographical distribution, the CDN itself has no influence on video encoding. The CDN would need to jump in and do this back-and-forth negotiation and delivery of lower quality frames, which it is not currently suited for. I'd love to see it come about, but it's not just the codecs we need to look at for adoption beyond point-to-point video calls.

<comment><USER5> The other major limitation is that forking the encoder state significantly inflates the number of reference buffers you need to keep, which greatly increases memory requirements. That's not much of an issue for software, but it can be a significant problem for hardware (a lot of real-time interactive encoding is still done purely in software, however).

</comment></comment><comment><USER35> There's a vegetable named "salsify", very yummy. [https://duckduckgo.com/?q=salsify+vegetable&t=ffab&ia=recipe...](https://duckduckgo.com/?q=salsify+vegetable&t=ffab&ia=recipes)

</comment><comment><USER36> Barely related to this, but looking at the results (section 5.2) I'm amazed at how much worse T-Mobile is for latency. AT&T and Verizon both give about 2 s of delay for Hangouts, while T-Mobile gives 7 s of delay.

<comment><AUTHOR> The reason T-Mobile looks so bad is because the T-Mobile trace was from a 3G network with very poor conditions, while the others (AT&T and Verizon) were from LTE networks under relatively good conditions. You shouldn't compare the quality of the carriers from our results.

<comment><USER36> Darn, I forgot UMTS was 3G. I think I was confusing it with HSPA+ (which I guess isn't really 4G either). Great work on the project.

</comment></comment></comment><comment><USER37> > What would you say to tomorrow's codec implementers?

> Standardize an interface to export and import the encoder's and decoder's > internal state between frames!

Can't this be achieved using sandboxing/emulation/VM techniques?

<comment><USER38> Not very efficiently, which is kind of the point here.

</comment></comment><comment><USER39> Another recent discussion was [https://news.ycombinator.com/item?id=16802079](https://news.ycombinator.com/item?id=16802079).

</comment><comment><USER32> Kudos for making things accessible. However, joint source-channel coding is not news, especially at the level of scalable video coding (probably 20-year- old research by this point). In academia this isn't as exciting as it sounds to industry.

<comment><USER28> Do you know if scalable video coding actually ended up being implemented in industry (YouTube/Netflix/Hulu/Amazon)?

<comment><USER40> In a completely different way (i.e. DASH). SVC came out with bad timing, as the move towards HTTP video was gaining momentum. Also, around 2008(?) or so when the spec was finalized, there was no HW encoding support, so it was pretty unusable in practice.

The idea of using layers, though, is much older (I remember reading papers about this already back in 2001 or so)

</comment></comment></comment><comment><USER41> this is not new. If you google video telephony adaptive rate adaptation techniques based on network conditions, you will find many(even from 1980s).

</comment><comment><USER42> How does this compare to Pied Pipers algorithm?

</comment><comment><USER43> "Is this a startup company?

No.

Are you sure? Your website looks like a startup company's.

It's just the HTML template! They all look like this. [...]"

Brilliant

<comment><USER44> It is a surprisingly nice looking website for an academic project.

<comment><USER32> Or a trial balloon for commercialization.

</comment></comment></comment><comment><USER45> I saw this on Reddit before. Most not impressed since it doesn't support audio.

</comment><comment><USER46> > Salsify is led by [...] Catherine Wu, a junior at Saratoga High School in > Saratoga, California.

Oh

<comment><USER47> Shows you the difference in opportunities for a smart kid in Bumblefuck vs. a smart kid living in a $5 million house in Silicon Valley and having transportation to Stanford.

<comment><USER48> I always laugh (while secretly crying) about the stories from high school people write on here. Seems like everybody is going to top-tier _tech high schools_, meanwhile the most computer science I got was being taught Turing by a gym teacher.</comment></comment></comment>


Robots will pay as much attention to us as we do to ants - <AUTHOR> https://www.theguardian.com/technology/2017/apr/18/robot-man-artificial-intelligence-computer-milky-way <comment><USER2> Somehow the so called over optimistic intellectuals of our world (specially the AI ones) have this fundamental approach to their thought process that goes something like - "Once you solved some small scale problem, all you need to do is to scale the solution up to solve the bigger complex problem". This has been the case with first era of AI researchers and the same has been with the current set of researchers. Sorry to say but that's not how reality works.</comment>


Razer unveils $2800 'Blade' laptop (With Touchscreen Keys) - <AUTHOR> http://www.joystiq.com/2011/08/26/razer-unveils-2800-blade-laptop/

<comment><AUTHOR> [http://www.blogcdn.com/www.joystiq.com/media/2011/08/rzrblad...](http://www.blogcdn.com/www.joystiq.com/media/2011/08/rzrbladev05cmykbbg.jpg)

Couldn't you just sell the touchscreen and fancy keys as a USB device instead?

Also, how would you really play a game with those keys? Most of the time my hands are on WSAD and the mouse.</comment>


Ask HN: What can Google do with me or to me if all I have with them is Gmail? - <AUTHOR> With all the different kinds of controversy that Google is choosing to get itself into lately, I've stopped using Google Chrome, deleted all of its saved data, and deleted data for a bunch of other Google products (YouTube, Google Pay, everything but Gmail). My web history, location history, and bunch of other things are set to "paused" (everything that Google allows to be paused is paused). I deleted third-party access to my Google account. My ad settings are set to not be personalized.
I use uBlock Origin, HTTPs Everywhere, EFF's Privacy Badger, and Canvas Blocker on Firefox, but I also have Firefox set to clear all my history, cookies, and other browsing data when I close it. I also use a router-based ad-blocking via Diversion (formerly AB-Solution) and pixelserv-tls as well as dnscrypt via dnscrypt-proxy.
Obviously, Gmail is the last holdout.
My question is: with just my Gmail account, what can Google know about me or do to me? My only concerns with leaving Gmail for something like FastMail or MailFence are the potential reliability issues (both in sending email reliably from my own domain and accessing it as often as I do now). It'd be nice to have both end-to-end encryption, and encryption, but given how powerful a $5 wrench is, it's probably wishful thinking.
(For completeness: I stopped having a Facebook account. I log into my LinkedIn account once a year. I log into my Twitter under five times a year. I don't have the mobile apps (first-party or third-party versions) with any of them. I don't have a tinfoil hat, but I have bought https://www.amazon.com/dp/B01I497JAM/ for someone before.) <comment><USER2> Well, along with knowledge of your Gmail account is the potential for knowledge of... every single other account you have connected to it. People rarely seem to realize their email account is often the most important and valuable property they have, as every other bit of digital content they have is tied to it, and often dependent on it to get access to it.

You should ask yourself: Can you get into your Steam library or Amazon account if you can't get into your email? Do you have a workaround if Gmail deletes your account? (If you have your own domain you can redirect, you do. If you use @gmail, you don't.)

It's exceedingly unlikely Google would do this, but bear in mind, the steward of your email account could reset the passwords and/or grant themselves access to any data you have on almost any digital account... and in many cases, hide the evidence they did so.

With regards to reliability, FastMail does occasionally have very short outages for technical difficulties which you can see on their Twitter account, but as someone who lives out of their email account so much that I browse social media largely by mass subscribing email notifications to a folder, I've only once tried to get to my email and been briefly unable to in the time I've had FastMail (over two years, I believe).

<comment><USER2> Oh! Another cool one you should know about: If you're worried about Google knowing what you buy, Google actively scrapes your email for receipts for purchases you make, you can view it here: [https://myaccount.google.com/purchases](https://myaccount.google.com/purchases)

There is no way to opt out of this, the only way to remove this data from this form is to delete the email. While Google claims they no longer use your Gmail data for ad tailoring, they arguably could use the purchase history data they store about you... much of which comes from your Gmail data.

<comment><USER3> Had no idea Google was doing that. Not surprising of course, but despicable. And of course they make it a chore to delete the records. Thanks for pointing it out.

<comment><USER2> It's more than a chore: If you're like me (or anyone who bought into the whole "archive all your mail instead of deleting it" thing that Gmail pioneered), the only solution to the purchases tab is to leave Gmail. If I want to keep my receipts in my email as a record, I have to keep my email somewhere else.

</comment></comment></comment></comment><comment><USER4> [https://wiki.fuckoffgoogle.de/index.php?title=GoogleAndMassS...](https://wiki.fuckoffgoogle.de/index.php?title=GoogleAndMassSurveillance)

email metadata analysis [https://labs.rs/en/metadata/](https://labs.rs/en/metadata/)</comment>


Top VC Firm Rankings - <AUTHOR> http://www.privco.com/top-20-venture-capital-firms-with-the-most-private-tech-company-exits-in-2012

<comment><USER2> Good to see some of these firms get some good PR, but the metric they are judging by is _number_ of companies in their portfolio that were acquired (IPOs aren't included). That's going to heavily favor funds that do lots of deals, and doesn't give a good indicator of whether the exits were profitable ones.</comment>


Why Recruiters Are Bad For Your Career - <AUTHOR> http://www.brandonsavage.net/why-recruiters-are-bad-for-your-career/

<comment><USER2> Great post Brandon, I hate individual recruiters/hh and recruiting agencies and always mark them SPAM in my gmail and LinkedIn</comment>


Extending Legal Protection to Social Robots - <AUTHOR> http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/extending-legal-protection-to-social-robots?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+IeeeSpectrumFullText+%28IEEE+Spectrum+Full+Text%29 <comment><USER2> Shouldn't we first grant legal protection, and perhaps citizenship, to non- humans we know for a fact are conscious and intelligent? I speak primarily of dolpins (and maybe other toothed whales), gorillas, chimps and elephants.</comment>


Toronto MaRS statement by BOD - <AUTHOR> http://www.marsdd.com/media-centre/statement-mars-board-directors/

<comment><USER2> There was a great discussion on Startup North's facebook page about this. Seems like more needs to substantiated?

[https://www.facebook.com/groups/startupnorth/permalink/10152...](https://www.facebook.com/groups/startupnorth/permalink/10152469066797840/)

It doesn't help that $1b has been poured in. It's too monstrous imo. The opposite of lean. It doesn't help things that their science building project failed/is failing and backed by tax payer $$.

[http://www.thestar.com/business/real_estate/2014/05/30/secon...](http://www.thestar.com/business/real_estate/2014/05/30/second_phase_of_mars_tower_nearly_empty.html)

</comment><comment><USER3> This statement is a response to articles like this [1] that have been floating around lately.

[1] - [http://www.thestar.com/news/queenspark/2014/05/31/mars_build...](http://www.thestar.com/news/queenspark/2014/05/31/mars_building_deal_projects_36_million_loss_over_10_years_secret_documents.html)

<comment><USER4> This is what is so frustrating about Canada... In this case, the government gets involved [1] in investing in early-stage companies, a ton of money goes instead to building a shiny building in a hyper-expensive real-estate market, and the government steps in with taxpayer money to bail itself out when things don't go well.

[1] Partners in MaRS include the Province of Ontario, the Government of Canada and the City of Toronto: [http://www.marsdd.com/about/our- partners/](http://www.marsdd.com/about/our-partners/)

<comment><USER5> Because what early-stage startups desperately need is luxury office space available for lease terms of _only_ 4 years.

</comment></comment></comment><comment><AUTHOR> Have never heard of any Canadian startups raising $1 billion through the MaRS Discovery District. I'm sure that value is an amalgamation of a few startups, nevertheless I'm interested in learning more about this.

<comment><USER6> They do pretty traditional gov-funded stimulation of science/tech industry. I heard on the grapevine most of the money goes to high-paid former-industry executives who are consulting to help accelerate existing companies. That plus support mostly patent-based research.

I've been there a bunch of times and it definitely doesn't have the culture of small nimble SF-style tech startups you'd expect. Feels much more corporate. Mostly a place where suits get to talk excitedly about 'innovation' and 'entrepreneurship', with nary an entrepreneur around.

<comment><USER7> Agreed, MaRS definitely doesn't have a small SF feel of tech startups at all. Waterloo is where it's at really if you're looking for a place that supports really early stage stuff.

<comment><USER8> Are you plugged into the W'loo community? Any intros or tips on how to get started? Sadly, I'm a UofT alum - when I looked at the programs at Waterloo, they seemed focused on their students.

<comment><USER9> I'm not plugged into the startup community in Waterloo, but Communitech seems to make the most noise: [http://www.communitech.ca/](http://www.communitech.ca/)

</comment></comment></comment></comment></comment><comment><USER10> I'm working with a startup that regularly meets at MaRS. We're in the cafeteria; office space is a little too expensive to be realistic at this point.

For me, the "buzz" is a big part of it. The fact that investors just happen to be in the neighbourhood. The fact that it's a good, central location right on the subway. There is a lot happening, including events that bring other entrepreneurs and investors to the same building.

I'm not aware of having raised any money through MaRS, but I know MaRS nevertheless plays a big role in the startup scene in Toronto, and it's been a big part of my personal experience. This is a great investment for Canada, and it would be depressingly foolish to interfere with the good thing they have going.

</comment><comment><USER11> "Young companies receiving advice and support from MaRS raised more than $1 billion in financing in the last 3 years, and generated more than $500 million in revenue."

Can someone help me understand this properly? Because from my perspective right now, this isn't bragging material. Turning 1B$ in 500M$ revenue, not even profits ... seems like poor investment returns.

</comment><comment><USER12> Perhaps their companies aren't going to IPO, they are getting acquired instead. I think HOVR.IT was bought by Slyce last year. Intel bought a MaRS company 4 years ago as well.

<comment><USER13> Are you talking about Intel's purchase of RapidMind?

<comment><USER14> RapidMind was in Waterloo, not Toronto, and purchased several years ago now. aperco perhaps is referring to this one [http://www.newswire.ca/en/story/722543/intel-corporation- acq...](http://www.newswire.ca/en/story/722543/intel-corporation-acquires- toronto-start-up-cognovision)

</comment></comment></comment><comment><USER15> The Ontario government should be informed that the big, expensive skyscrapers come after the innovation, not before.</comment>


Quantum Weirdness: Two times zero doesn't always equal zero - <AUTHOR> http://www.spectrum.ieee.org/aug08/6609

<comment><USER2> You don't need quantum mechanics to produce this kind of a result. One-time- pad encryption does essentially the same thing. Both the "key" and the "ciphertext" (I put them in scare quotes because they are mathematically indistinguishable) by themselves are just random bit strings containing zero information. But together they contain all the information of the original cleartext.

<comment><AUTHOR> I don't think that's a correct analogy. This announcement is about signal detection in the presence of noise not information decryption.

My interpretation of the article is that two noisy quantum channels (where each individual channel is too noisy to recover any information) can have their noisy results combined to recover clean information.

There is no classical analogy to this case. If two classical channels are too noisy to recover information from them individually, then there is no way to combine the noisy results to recover any information.

</comment></comment><comment><USER3> Umm, isn't this what Bell Labs found under Gell-Mann in like 40 years ago? You overcome transmission problems with redundancy. Or am I missing something?

<comment><USER4> Actually no... They were not trying to overcome noise. They were actually testing what can come out of combining "entangled" noisy channels. The fact that they retrieve information out of nothing (noise is the equivalent of "nothing" information-wise) is the beauty of their work.

</comment></comment><comment><USER5> Isn't this the same as spread spectrum broadcasting?</comment>


Towards a Common Ruby definition - <AUTHOR> http://elmcitycraftworks.org/post/37849197797/towards-a-common-ruby-definition

<comment><USER2> Ruby Newbie here.

Is this pretty common in programming languages? It seems like there's been a lot of Ruby-specific drama in the last couple months (refinements come to mind) - does this happen in Python, JS, etc. as well? It seems like most BDFLs create the _canonical_ version of the language without much debate, but maybe I'm just coming into it fairly late.

<comment><AUTHOR> I'm the author of the article, but it's worth noting that I'm _not_ involved in any implementation efforts. That said...

The overall way that Ruby is developed causes these issues to happen, because major language experimentation happens very close to feature freeze dates. Because MRI is the defacto-standard (and the ISO standard defines a Ruby which simply does not exist in reality), contention happens because of the lack of coordination between the core Ruby team and the alternative implementers.

So for example, something like refinements are controversial because they are very challenging to implement and create all sorts of additional complexity. Under the current way of doing things, alternative implementers would be forced to either implement this feature or to not be able to fully claim they are "compatible" with whatever Ruby version they're released in.

Matz is generally a good BDFL, and the alternative Ruby implementers are generally very reasonable as well. But the tight time cycles, informal arrangements, and the communication barriers between a Japanese-speaking and Japanese-cultured core team and a more globalized language community combined lead to a lot of heated debates and frequent turbulence.

But all that having been said, compromises are almost always made sooner or later and the community moves forward together. They just often come at a high cost, and I wish that cost were lower. After all, the list of things we agree on is miles longer than that which we disagree about.

</comment></comment><comment><USER3> How has Python avoided this? Is it because GVR has been accepted as the dictator because there are no language barriers? I can see how the language issue can cause a major disturbance.</comment>


How does our language shape the way we think? (2009) - c-oreills http://www.edge.org/conversation/how-does-our-language-shape-the-way-we-think <comment><AUTHOR> As a speaker of 3 different languages (English, German and Portuguese/Spanish) I believe that although correct this is a wrong discussion. Yes, languages does influence the way we think, but overall culture does influence a million times more.

One example is the democratic debate in English speaking countries compared to the same debate in Portuguese/Spanish countries. I hear a lot of words that don't have a direct translation to a single word: entitlement, pork, accountability, delusion, bullying, patronage, solipsism, ... Because words similar to these don't exist in Spanish/Portuguese I feel that the democratic debate is sometimes so truncated and clunky.

Another example might be how any discussion in English is so strongly focused on providing empirical evidence for claims; even today Anglo-Saxon remains the culture of Sir Francis Bacon and the inductive method.

<comment><USER2> I was an English <> Spanish translator in another life.

While I agree with your general idea that there are some untranslatable words, most of the words you cite do have equivalents in Spanish, unless you are referring to false friends. In that case, it'd be great if you could elaborate. But generally speaking, the translations are regionalisms. Also, I'd like to add that language is very much embedded in culture and vice versa. They work off each other and influence each other.

I think words are concepts. Some of the discourse of translatability in the Spanish-speaking world, especially when the source language is English, usually deals with the illogical sense of English words. Bullying is a good example, actually. Bullying comes from the Dutch word "brother" and its definition has drifted far away from "fine fellow". In Spanish, the word "bullying" in the 21st century English-speaking sense would be related to mean "acosador", which comes from the Latin cursus/currere which can probably best be defined as "to proceed" in English. So "bullying", as you suggest, might not have a one-word translation, but it could be translated as "acosador escolar". However, because Spanish-speakers understand the importance of world integration, "bullying" is also used, though you get purists and word-rebels in some regions who, incidentally, are also anti-American, anti-consumerism and sceptical of the English-speaking world in general.

A similar phenomenon occurs in the UK. You see certain people use an indigenous equivalent instead of adopting a standard "London English" word. It certainly happens in South America. So yes, culture and language are very much tied together.

Personally, I think Portuguese, though a romance language, really shouldn't be grouped together with Spanish, even though I can understand most of it, though I've never taken a single Portuguese course in my entire life. My understanding it has to do with studying Latin and Italian. I think the average, _educated_ Spanish speaker would be able to read maybe 50% and understand spoken Brazilian Portuguese even less. Some words and pronunciations have seeped into Argentina, Paraguay and Bolivia, however.

<comment><USER3> > I think words are concepts.

My notion of this is that words are _pointers_ to something in a concept space. The words themselves don't and can't contain everything that the concept is; through cultural immersion, you gain a more complete notion of the concept and can more reliably refer to it by using an appropriate word.

This is why a lot of words are used to mean things completely unrelated (apparently) to their definition. The address they have in concept space simply changed.

And then there's the detail that concepts easily and frequently overlap.

I'm sure this has been covered by an Actual Linguist more credibly, though I've never seen it.

</comment></comment></comment><comment><USER4> Edge.org presents things as 'conversations' but they are generally just essays, with no possibility for other edge members to leave comments. Thats not a conversation. There's no dialogue, nothing anyone says is challenged. None of these 'great minds' at Edge are actually presented talking to each other.

Consequently although Edge.org says its all about conversation and advancing knowledge, I suspect its really just about selling pop-science books.

It would be a much better site if it was a proper forum (for the 'great minds' that are members) and we could all watch the discussion, disagreements, flameouts and ragequits. Seriously.

<comment><USER5> Well I'm looking at the front page story, "A Conversation with Lee Smolin." It leads with a monologue from Smolin followed by a response from Arnold Trehub, followed by a response from Sean Carroll, followed by an entry from Lee Smolin responding to Carroll's response. Bruce Sterling enters the conversation to express sympathy with ideas Smolin as raised then Trehub comes back with another response and finally Amanda Gefter gives a reply to Smolin.

This event is put under the site's conversations section, and a quick perusal of this section reveals tons of similar conversations with multiple contributors all responding to each other, and they've been doing this for years.

<comment><USER4> Hmm, I looked before and didn't find much, but you're right. Still, on page 1 of the list of 'conversations' there only seem to be 9 that are genuine dialogues, the other 41 are just singular essays or people all answering an open question. But yeah, thats a bit more like it. Still some way to go before it becomes a bubbling forum of discourse though.

</comment></comment></comment><comment><USER6> And just when I'd finally flushed Sapir-Whorf out of my system...

<comment><USER7> From what I understand, the original Sapir-Whorf thesis (now the strong version) was (correctly) discredited because it was about language _limiting_ and determining thought. The weak version, that it _influences_ the way we think, is now widely accepted.

<comment><USER8> Widely accepted is stretching it. There are very few linguists who believe there's enough evidence to support the weak hypothesis.

</comment></comment></comment><comment><USER9> Wittgenstein said it best: "The limits of my language are the limits of my world."

<comment><USER10> Die Grenzen meiner Sprache bedeuten die Grenzen meiner Welt.

<comment><USER11> And the quote attributed to Einstein ("If you can't explain it to your grandmother, then you don't understand it yourself") also expresses this idea.

</comment></comment></comment><comment><USER12> "For a long time, the idea that language might shape thought was considered at best untestable and more often simply wrong."

I dislike sensationalism. The idea that this can be tested is as old as the modern controversy. There were studies on words for colors in the 1950s.

[http://en.wikipedia.org/wiki/Linguistic_relativity_and_the_c...](http://en.wikipedia.org/wiki/Linguistic_relativity_and_the_color_naming_debate)

<comment><USER13> All of which brings back memories of Loglan[1] which was specifically designed as a tool for testing Sapir-Whorf, although in the end and after many years of painstaking development it was never used for that. Loglan has a successor Lojban[2] which is still a hobby project for a small community.

[1] <http://en.wikipedia.org/wiki/Loglan> [2] <http://www.lojban.org/tiki/Lojban>

<comment><USER14> There are a few children who are being raised bilingually with Lojban as a mother tongue so the original purpose may end up being fulfilled. It'll be interesting to see how they turn out.

xu baupli la .lojban. .i mi nintadni

</comment></comment></comment><comment><USER15> Check out her talk as well - <http://www.youtube.com/watch?v=cPGpZp1pfQQ>

<comment><USER7> I just saw that last week. Fascinating!

</comment></comment><comment><USER16> This is extremely subjective but as a speaker of two languages (English and French), I find it interesting how certain things come to me in one language and other things in the other. I have friends more or less in the same boat as I am and it's funny to see how we switch languages when expressing different ideas without even thinking about it. Funnily enough, I happen to talk in my sleep on occasions, and I apparently speak a mash-up of both languages, but in a fairly consistent manner (i.e. words will be one language or the other). The brain's a wonderfully intricate thing :)

</comment><comment> c-oreills Adds weight to the idea of the Blub Paradox: <http://www.paulgraham.com/avg.html>

<comment><USER17> Yep. And I think this linguistic influence means _way more for programming languages than for natural languages_ \- when thinking or communication to a human you can always "escape the language traps" by thinking visually or drawing/sketching or just showing things, but _when you "communicate" with a computer your minds is basically "trapped" in the language_ (the only escape is being able to think the "domain language" instead, like thinking an algorithm in mathematical language or a business problem in business terms).

<comment><USER18> I think you can "escape" a programming language by sketching in quite the same way you can do it for natural languages, if not more easily. After all, we'll been using Flowcharts and the like since the beginning of computing.

</comment></comment></comment><comment><USER19> Number system of some Asian languages (e.g. Chinese) is base 10, so the children can learn Maths easier.

<comment><USER20> What do you mean? The arabic number system that most of the world uses is base 10. Also, I don't see why the base of the number system would make math any easier or harder.

<comment><USER21> The arabic numeral system is base-10, but English numerals are only sort of base 10: consider 2, 10, 12 and 22. In English, these are spoken as "two", "ten", "twelve" and "twenty-two". The words "twelve" and "twenty" are both derived from "two", it's true, but in two different ways. "Teen" and "ten" are similar, but there's nothing about "twenty" that indicates a connection to "ten".

In Japanese (IIRC), 2, 10, 12, 22 is spoken "ni, ju, ju-ni, ni-ju-ni": "two, ten, two-tens, two-tens-and-two".

It's not that one base is better than another: the idea is that a single consistent base is better than inconsistency.

<comment><USER22> Thank you. I had flirted with the thoughts before, but never really explicitly considered how odd the English words "eleven", "twelve" and then the "teens" are. Not only is "teen" not quite "ten", we also put it _after_ the ones digit. That is, 19 is "nine- _teen_ ", but 91 is " _ninety_ -one".

<comment><USER23> This is true of Germanic languages more generally, where they have special cases for teens too but also read higher numbers as "five-and-twenty" etc. And then French has their vigesimal thing - 92 is quatre-vingt-douze, ie. "four twentys and twelve". English gets off relatively lightly with teens only...

</comment></comment></comment></comment></comment><comment><USER24> Very simple - the way you cursing affects how you think. Some cultures are exceptional in using "bad words". Russians, for example, using all the dirty details of a sexual intercourse, while, say, Italians using some blasphemy. Here lies the answer.)</comment>


Ask HN:"Engineering" a sequence a projects to become a more confident generalist - <AUTHOR>

Is there a particular path that can truly accelerate one's learning to become a true generalist (ability to stitch layers together with confidence). I've been lurking around HN for a while now, and when questions of this vein are asked I've often noticed that people usually say something to the effect of "dude. just build stuff" which frankly is really a good way to get started (definitely beats analysis-paralysis). But I'm trying to understand if there's a particularly clever way to sequence the "building" of said projects in a particular way that leads to incremental insights that in turn converge to a less frustrating and more fulfilling that could give me a big-picture understanding of how things are connected end to end.
I'm looking for something along the lines of:
Step1 => Build a ridiculously simple MVC app like a todo list that lets people add things to an online list (like http://todomvc.com/)
Step2 => Back the app with some RESTful backend service that includes persistence (ex: mySQL/redis)
Step3 => Add simplified monitoring to the app to get a sense of how your app is performing (ex: How many 4xx errors yesterday)
Step4 => Hook your monitoring to some simple analytics processing (ex: How many ppl from SF have added things to the list)
Step5 => Deploy it onto AWS and configure your machines to figure out 2 flavors of the page (A/B testing + DevOps)
Step6 => Make your analytics processing "offline" by queueing events from the RESTful service into an MQ
Step7 => Make your analytics interesting by storing this data (warehousing) by using things like elastic search to answer more interesting questions (ex: What are the 'general' things ppl from SF tend to add) etc.
The idea here is to tackle things that progress naturally and minimize any new dev frustration by incrementally adding more bells and whistles. Does the HN community have any opinion on such examples that they've found worked for them and things they'd recommend to relatively new developers (1-2 years of experience)? <comment><USER2> I like angersock's list since it shows a lot of areas that I would expect a generalist to know. Obviously not exhaustive but it's a pretty good list.

That said you are going about it wrong if you are looking for a linear path to become a generalist. It is more like a giant graph of areas and you start from one you like and branch. The "right" node to start at is what you enjoy the most.

</comment><comment><USER3> That is a very tame definition of "generalist" you have there.

Here, let's propose some things I'd expect a generalist to be able to do:

      IT
      </comment><comment>=====

      - Provision wired networks for small businesses/homes/labs
      - Do basic troubleshooting of printers and networked printing (and know when to throw in towel)
      - Setup and debug towers and rack servers from parts
      - Provision wireless networks using access points for small businesses/homes/labs
      - Setup and debug mail clients
      - Setup and debug file servers
      - Setup and provision virtual machines with network access
      - Setup and debug scripts for common administration tasks
      - Setup and secure user accounts on remote systems with centralized management
      - Setup disaster recovery systems and policies for above
      - Write project plans and deliverables for doing above
      - Write cost analyses and bills of materials for doing above
      - Explain existing service providers and when they'd be preferable to doing things in-house

      Web
      </comment><comment></comment><comment>

      - Write front-end JS, without frameworks, to do basic tasks
      - Write single-page app with proper history integration
      - Write front-end JS using AJAX endpoints
      - Write functional JS where appropriate
      - Write CSS
      - Write CSS with less pain using LESS/SASS/Stylus/whatever
      - Write CSS with rough idea of compatibility concerns
      - Write HTML from scratch
      - Write responsive web pages (and know when this is a fool's errand)
      - Explain how HTTP works, what it gets wrong, and what alternatives exist
      - Write JS using websockets
      - Write JS using webworkers
      - Write JS using canvas (2D or 3D)
      - Provision a basic website using turnkey hosting ( Heroku, etc. )
      - Provision a basic website using virtual hosting ( Linode, Digital Ocean, etc. )

      Software engineering
      </comment><comment></comment><comment></comment><comment>=====

      - Write useful and succinct comments in code
      - Write useful and succinct documentation for projects
      - Provide useful root-cause analysis for production failures
      - Provide reasonable estimates for project completion
      - Provision and use a centralized version control system (CVS, SVN, etc.) and explain its uses
      - Provision and use a decentralized version control system (git, mercurial, etc.) and explain its uses
      - Write useful and approriate tests for code (TDD, BDD, library unit tests, whatever)
      - Provision continuous integration services for an application

      Systems
      </comment><comment></comment><comment>==

      - Explain how different data types can be represented in memory
      - Explain how filesystems work
      - Explain functions of a kernel
      - Explain at least 2 approaches to operating systems design
      - Explain basic architecture for at least 1 processor family, and be able to provide design considerations (MIPS, x86, x64, Itanium, ARM, SPARC, etc.)
      - Write organized project in a systems language (assembly, C, C++, D, Rust, Go)
      - Create build environment for native code projects
      - Create makefiles (or platform's equivalent)
      - Write in systems language: hash table, linked list, resizable array, binary tree
      - Write networking code in a systems language supporting IPv4, IPv6, and which can support lookups by hostname
      - Write UDP code with reliability mechanisms
      - Write TCP client code, explain common mistakes
      - Write multithreaded code, explain concurrency approaches
      - Write asynchronous code in a systems language
      - Write cross-language binding for systems language library ( Ruby over C, Python over C, etc.)
      - Create and document binary format for storing structure data to a file
      - Create and document binary format for streaming and message-based communication over a network
      - Explain why we prefer JSON for above purposes
      - Explain why JSON is slow and dumb, and we'd rather use <CapnProto, protocol buffers, STOMP, whatever > instead
      - Write code handling file IO using normal methods, using asynchronous methods, using memory mapping in systems language
      - Write code performing IPC in systems language

      Databases
      </comment><comment></comment><comment>===

      - Provision an RDBMS (MariaDB, Postgres, whatever)
      - Write basic queries for fetching data
      - Write useful and succient data model documentation for database
      - Explain performance considerations for hardware running that RDBMS
      - Perform maintenance (backups, benchmarking) on RDBMS
      - Provision at least one NoSQL solution (neo4j, MongoDB, Redis, Cassandra, etc.)
      - Perform maintenance (backups, benchmarking) on NoSQL databases)
      - Explain when to use that NoSQL solution in preference of an RDBMS, and why.
      - Perform ETL using above systems
      - Perform map-reduce using above systems
      - Collect real-time time-series data using above systems
      - Connect to above systems using web framework of choice, using manual and secure queries
      - Connect to above systems using web framework of choice, using an ORM
      - Scale database of choice across multiple machines
      - Perform failover of database of choice across multiple machines

      Maths
      </comment><comment></comment><comment>==

      - Understand basic linear algebra and rigid transforms (i.e., 2D graphics)
      - Understand basic statistcal properties
      - Understand basic themes of intergration and differentation
      - Understand IEEE 754 floating-point and basic numerical issues in computation
      - Understand trigonometric functions and their relations
      - Estimate computational complexity of implementing above

This is not an exhaustive list, and I don't expect everyone to be 100% on it. That said, if you call yourself a "generalist" and _can 't_ do like at least half of everything in each of those categories, you're either a novice generalist, or you're full of shit.

EDIT:

I left out so things applying to programming languages, applying to machine learning, applying to graphics, applying to electronics, and several other things. Being a generalist is _hard_.

This is also why a good one is _expensive_.

<comment><AUTHOR> That's a massive compendium! I wish I had been more crisp in my question; the thought over here was to figure whether there's a simpler step by step process to minimize frustration and maximize your confidence on your path to becoming an enlightened generalist (which would naturally takes years to accomplish). I'm mostly interested in the "sequencing" of these steps which is why I provided a rather contrived example of the web app that grows in features with each iteration. For junior devs it's usually pretty hard to figure this out on their own so they end up either doing a deep dive in a very limited number of technologies (ex: Building a webservice in Spring + Hibernate ecosystem comes to mind) without or just float up top at very superficial layers.

You've done an incredible job at detailing the expectations of a generalist but might you have some recommendations on forging that path for people interested?</comment></comment>


Show HN: Export HN Favorites to a CSV File - <AUTHOR> I wrote a short JavaScript snippet to export HN Favorites to a CSV file.
It runs in your browser like a browser extension. It scrapes the HTML and navigates from page to page.
Setup and usage instructions are in the file.
Check out https://gabrielsroka.github.io/getHNFavorites.js or to view the source code, see getHNFavorites.js on https://github.com/gabrielsroka/gabrielsroka.github.io <comment><USER2> I made an API for favorites a while back.

[https://github.com/reactual/hacker-news-favorites- api](https://github.com/reactual/hacker-news-favorites-api)

Here's an example query:

[https://reactual.api.stdlib.com/hnfavs/?id=sbr464&limit=1](https://reactual.api.stdlib.com/hnfavs/?id=sbr464&limit=1)

<comment><AUTHOR> Ok, now that's really smart!

Would a client using your API paginate to fetch, say, 50 pages? When I tried it using ?limit=50, I got a 504 error.

Thanks!

(Edit, never mind, I see you explain it in the readme.)

<comment><USER2> I had made it pretty quickly, the limit would be too large, thats 50*30, you may need to provide an offset and make a few requests.

</comment></comment></comment><comment><USER3> This is cool, I love HN metadata, too :)

Plug for a related golang tool I wrote and use which exports favorites upvotes as structured JSON:

[https://github.com/jaytaylor/hn-utils](https://github.com/jaytaylor/hn-utils)

Just

        go get github.com/jaytaylor/hn-utils/...

</comment><comment><USER4> It's a shame favorites aren't exposed in the official HN API: [https://github.com/HackerNews/API](https://github.com/HackerNews/API) \- this is a smart workaround.

<comment><USER5> Our plan is for the next version of HN's API to simply serve a JSON version of every page. I'm hoping to get to that this year.

<comment><USER6> Are there plans for an export tool, e.g. a user downloading all their comments and upvoted submissions? I tend to use the submission upvote button more than the favorite one, and an export tool wouldn't require a user API key for non- private info.

<comment><USER4> My tool here can export all of a user's comments: [https://github.com/dogsheep/hacker-news-to- sqlite](https://github.com/dogsheep/hacker-news-to-sqlite)

Not upvoted yet as they aren't in the API I am using.

</comment></comment></comment></comment><comment><USER7> This is smart. I'm adding this to my HN favorites.

<comment><AUTHOR> Thanks dvfjsdhgfv. If there's sufficient interest, I can easily turn it into a Chrome extension.

(Edit: haha, I see what you just did there. A little recursive humor.)

</comment></comment><comment><USER8> This is great! My biggest problem was I couldn't search through my upvoted items to find the article I liked again. I used google custom search and cleaned the data as flat urls.

[https://www.heyraviteja.com/post/projects/deep-search- hn/](https://www.heyraviteja.com/post/projects/deep-search-hn/)

<comment><USER8> oops - I didn't realize that favorites != upvoted.

</comment></comment><comment><USER9> Can it will be done with "Scrape Similar" Chrome plugin?

<comment><AUTHOR> Thanks for the tip. I gave the "Scraper" extension a try, and 1) I got an error, 2) it only seems to scrape 1 page -- it doesn't paginate (or, did I miss something?).

I used the jQuery selector `a.storylink`.

</comment></comment><comment><USER10> Is there any way to find most Favorited items on HN ?

</comment><comment><USER11> Could someone convert it to _Python_ -script?

<comment><AUTHOR> Part of the advantage of running JavaScript in your browser is that you might already be authenticated and it can use your session. But, fetching your HN favorites doesn't require authentication.

      #!/usr/bin/env python3
      import requests
      from bs4 import BeautifulSoup

      for p in range(1, 17):
          r = requests.get(f'https://news.ycombinator.com/favorites?id=app4soft&p={p}')
          s = BeautifulSoup(r.text, 'html.parser')
          print([{'title': a.text, 'url': a['href']} for a in s.select('a.storylink')])

<comment><USER11> Thanks!

One more question: what is the best way stop it when it will reaches last page?

> _for p in range(1, 17):_

Actually _p=17_ [0] is empty (as _p=16_ is maximum as for now).

Maybe, script should scrap pages from `1` to `infinity` UNTIL it detect next message on page[0]:

> _app4soft hasn 't added any favorite submissions yet._

[0] [https://news.ycombinator.com/favorites?id=app4soft&p=17](https://news.ycombinator.com/favorites?id=app4soft&p=17)

<comment><AUTHOR> In Python, `range(1, 17)` produces the numbers 1-16. I hard-coded it just for your favorites.

A better way to solve it is to look at the `len()` of the results, and stop when it gets to 0:

      p = 1
      while True:
          r = requests.get(f'https://news.ycombinator.com/favorites?id=app4soft&p={p}')
          s = BeautifulSoup(r.text, 'html.parser')
          faves = [{'title': a.text, 'url': a['href']} for a in s.select('a.storylink')]
          print(faves)
          p += 1
          if len(faves) == 0:
              break

<comment><USER11> Great!

<comment><AUTHOR> I think this one is a little cleaner. I used some of the ideas in sbr464's code.

        path = 'favorites?id=app4soft'
        while path:
            r = requests.get('https://news.ycombinator.com/' + path)
            s = BeautifulSoup(r.text, 'html.parser')
            print([{'title': a.text, 'url': a['href']} for a in s.select('a.storylink')])
            more = s.select_one('a.morelink')
            path = more['href'] if more else None

<comment><USER11> Go on! ;)

</comment></comment></comment></comment></comment></comment></comment><comment><USER12> What is HN's GDPR compliant way of requesting a copy of all stored data? Email dang?

<comment><USER13> Considering that there is no mention of GDPR in the HN FAQ or on the "legal" page, my guess is that their position is that GDPR does not apply.

According to Article 3 of the GDPR, it applies to:

1\. Processing that takes place in the context of processors and controllers that are in the Union, regardless of whether or not the processing itself takes place in the Union.

2\. Processing the data of subjects who are in the Union by controllers or processors who are not in the Union if the processing is related to offering goods or services to such subjects in the Union or the processing is related to monitoring the behavior of such subjects that takes place in the Union.

I don't know how HN is structured, but I've not seen any indication that they are in the Union, so #1 probably does not apply.

#2 applies if they are doing processing related to "offering goods or services to such subjects in the Union" or "monitoring the behavior of such subjects that takes place in the Union".

One of the recitals elaborates on the first branch of that:

> In order to determine whether such a controller or processor is offering > goods or services to data subjects who are in the Union, it should be > ascertained whether it is apparent that the controller or processor > envisages offering services to data subjects in one or more Member States in > the Union. Whereas the mere accessibility of the controller's, processor's > or an intermediary's website in the Union, of an email address or of other > contact details, or the use of a language generally used in the third > country where the controller is established, is insufficient to ascertain > such intention, factors such as the use of a language or a currency > generally used in one or more Member States with the possibility of ordering > goods and services in that other language, or the mentioning of customers or > users who are in the Union, may make it apparent that the controller > envisages offering goods or services to data subjects in the Union.

Does HN "envisage" offering services to people in the Union? Or are they a site that is merely accessible from the Union without envisaging offering services there?

There's a recital that elaborates on the second branch, too:

> In order to determine whether a processing activity can be considered to > monitor the behaviour of data subjects, it should be ascertained whether > natural persons are tracked on the internet including potential subsequent > use of personal data processing techniques which consist of profiling a > natural person, particularly in order to take decisions concerning her or > him or for analysing or predicting her or his personal preferences, > behaviours and attitudes.

Does the data HN stores about its users satisfy this? And if it does, is the behavior being monitored taking place in the Union?</comment></comment>


60,000% growth in 7 months using Clojure and AWS - <AUTHOR> http://www.colinsteele.org/post/27929539434/60-000-growth-in-7-months-using-clojure-and-aws

<comment><AUTHOR> To me, this was the big surprise, and the big revelation:

"This led to Decision Two. Because the data set is small, we can "bake in" the entire content database into a version of our software. Yep, you read that right. We build our software with an embedded instance of Solr and we take the normalized, cleansed, non-relational database of hotel inventory, and jam that in as well, when we package up the application for deployment. Egads, Colin! That's wrong! Data is data and code is code! We earn several benefits from this unorthodox choice. First, we eliminate a significant point of failure - <USER2> mismatch between code and data. Any version of software is absolutely, positively known to work, even fetched off of disk years later, regardless of what godawful changes have been made to our content database in the meantime. Deployment and configuration management for differing environments becomes trivial. Second, we achieve horizontal shared-nothing scalabilty in our user- facing layer. That's kinda huge. Really huge."

<comment><USER3> I work for a food startup (Nutrivise) and we do something similar'ish. Our dish database is baked in and deployed as a fixture. We _should_ consider distributing it to the frontend web servers instead though.

There are some other fun hacks, like local redis instances for caching search space stuff.</comment></comment>


How to name your startup (and land the perfect domain) - <AUTHOR> http://calacanis.com/2010/12/03/how-to-name-your-startup-and-land-the-perfect domain/

<comment><USER2> Original Source: [http://launch.is/blog/2010/12/3/how-to-name-your-startup- and...](http://launch.is/blog/2010/12/3/how-to-name-your-startup-and-land-the- perfect-domain.html)

</comment><comment><USER3> It's Calacanis recursive blogspam. Incredible.</comment>


Mbed is a tool for Rapid Prototyping with microcontrollers - <AUTHOR> http://mbed.org/handbook/Tour

<comment><AUTHOR> Came across this today, thought it was neat because they provide an ARM compiler / IDE as a webpage. No need to set up a cross-compiler locally, which can be a royal pain in the ass.</comment>


Intel Reveals New Spectre-Like Attack, Advises Disabling Hyper-Threading - <AUTHOR> https://www.tomshardware.com/news/intel-disable-hyper-threading-spectre-attack,39333.html <comment><USER2> Apparently disabling HT won't save you from this one in all cases.

</comment><comment><USER3> here is the primary source: [https://news.ycombinator.com/item?id=19911715](https://news.ycombinator.com/item?id=19911715)</comment>


PHP on Google App Engine (using Quercus) - <AUTHOR> http://weirdhenge.appspot.com/test.php <comment><USER2> Is there a rationale for doing this, other than "because it's there"?

AFAIK the problem with running existing PHP apps on App Engine isn't the PHP part. It's the rest of their architecture. I've only ever seen two kinds of PHP apps: The ones that are tightly coupled to the LAMP stack or a very close equivalent, and the ones that should have been written in some other language.

But maybe someone else can enlighten me.

<comment><USER3> I'm not sure about GAE, but as to Quercus in general: You get to write in Not Java while your employer gets to deploy on the JVM.

This is huge.

Take Jython for example: Put jython.jar in WEB-INF/lib/, add a small stanza to WEB-INF/web.xml, write a WSGI-compliant app in Python. When you're done, you can zip it up and deploy it on Tomcat. None of the existing infrastructure needs to change, and you get to write in a more expressive language. That's how we'll sneak better things into enterprise.

<comment><USER2> Okay, that's a good answer: I was focused on the App Engine part, whereas the real exciting news is the Quercus part!

</comment></comment></comment><comment><USER4> <http://github.com/moriyoshi/quercus-gae/tree/master> Seems to be the patched code. There is also a HOWTO document.

<comment><USER5> So this is basically one very big hack?

"Content cache and session management don't work." That's a pretty big drawback for me. Are there any performance issues?

</comment></comment><comment><USER6> Hi All, See details on how to Run PHP on Google App Engine using Quercus [http://www.webdigi.co.uk/blog/2009/run-php-on-the-google- app...](http://www.webdigi.co.uk/blog/2009/run-php-on-the-google-app-engine/) Just download the file in the link and you can run PHP on app engine in minutes. Also see <http://phpwithjava.appspot.com/webdigi.php> Regards

</comment><comment><USER7> Wow this would be great, I have more than 7 months wondering when they'll enable it as more than 1,000 comments were posted on this issue.

<comment><USER8> To be fair, a good proportion of those comments were complaining about "+1" comments left on the issue. To an extent where there were more complaints than +1's.

</comment></comment><comment><USER9> PHP modules implemented in Java for Quercus: <http://tusul.net/I2KmMk>

Caucho says that the real gain is taking advantage of built-in advanced features of Resin App Server like load balancing. Without porting MySql, PHP on GAE means little.

</comment><comment><USER10> People talk about Vendor Lockin with Microsoft, but there are lots of links to AppEngine tools and services, so I'm wondering what is the difference?

Why is Vendor Lock-in okay with Google's proprietary platform, but not with Microsoft's?

<comment><USER11> Perhaps because they rely on a lot of open source components and open up some of their technologies whereas Microsoft has been slow at coming around to the OS way of thinking.

If you're worried about lock-in with App Engine, check this out: <http://code.google.com/p/appscale/> As you can see, Google is one of the supporters of this project, although it's not clear exactly what they have contributed.

</comment></comment><comment><USER12> Anyone here has experience with Quercus? How compatible is it with PHP? Do apps like Wordpress or Drupal run out of the box? Any noticeable performance difference?

<comment><USER13> <http://www.caucho.com/resin-3.0/quercus/#PHP-Users>

Quercus claims to be able to run "MediaWiki, Drupal, Wordpress, Gallery2 and DocuWiki" on their site, and equivalent performance to a PHP deployment with APC. Which is pretty impressive, if true.

<comment><USER12> Thanks!

</comment></comment></comment><comment><USER14> Hmmm, I just looked on my Google App Engine account, and I can't see how to use / enable PHP. I see the link to quercus, ... can someone add some details to how to use it?</comment>


AsK YC/PG: Resource Section? - <AUTHOR>

Some questions such as these (http://news.ycombinator.com/item?id=198502) seem to come up time and again and I was wondering if it would be a good idea to have a "Resources" section for these commonly asked questions (another one is on valuations and advisors). These resources could just be a collection of links at YC and elsewhere on the web... <comment><USER2> Ask HN: [http://www.ycombinator.search.xirium.com/cgi- bin/search.cgi?...](http://www.ycombinator.search.xirium.com/cgi- bin/search.cgi?p=0&q=%22ask%20hn%22)

Ask PG: [http://www.ycombinator.search.xirium.com/cgi- bin/search.cgi?...](http://www.ycombinator.search.xirium.com/cgi- bin/search.cgi?p=0&q=%22ask%20pg%22)

Ask YC: [http://www.ycombinator.search.xirium.com/cgi- bin/search.cgi?...](http://www.ycombinator.search.xirium.com/cgi- bin/search.cgi?p=0&q=%22ask%20yc%22)

</comment><comment><USER3> ironically this question gets asked all the time

<http://ask.searchyc.com/>

<http://www.gabrielweinberg.com/startupswiki/Ask_YC_Archive>

<comment><USER4> that's not ironic. it's just coincidental.

<comment><USER5> Maybe he meant that the people ask it ironically.

</comment></comment></comment><comment><USER6> What about a wiki?

<comment><USER7> Because the two resources mentioned already cover everything w/o creating any data duplication or extensive manual labor?</comment></comment>