Image GPT - <AUTHOR> https://openai.com/blog/image-gpt/ <comment><USER2> The model is open sourced on GitHub: [https://github.com/openai/image- gpt](https://github.com/openai/image-gpt)

Oddly, it still uses TensorFlow like the original GPT-2 release despite OpenAI's declared switch to PyTorch, and it has dependency hell so it's not easy to create a wrapper tool for it.

Since it's still the GPT-2 architecture, it might be possible to port the weights to Huggingface Transformers (for the RGB generation), and then write a wrapper to extend it for the image rendering. (filed an issue here: [https://github.com/huggingface/transformers/issues/5088](https://github.com/huggingface/transformers/issues/5088) )

<comment><USER3> I'm still learning from deep learning papers and videos before dipping my toes in myself, is there a summary of why PyTorch vs TensorFlow? Does it matter for me?

<comment><USER2> _Nowadays_ , there isn't a huge practical difference in terms of performance/tooling aside from edge cases and deployment options. It mostly depends on your syntax preference. (although there are flame wars from both sides)

I noted the TensorFlow usage because the original GPT-2 release was TensorFlow 1.X, which led to issues when TensorFlow 2.0 was released soon after.

For model training, I strongly recommend using the higher-level Keras APIs for TensorFlow/pytorch-lightning API for PyTorch than the respective base tools.

<comment><USER4> There is a huge difference in developer experience still. When TF fails it typically offers little to no clue as to what you can do to rectify the situation. PyTorch is a lot more helpful most of the time. PyTorch is worth choosing on this basis alone, unless you intend to deploy to mobile hardware, where TFLite is the only real viable option.

</comment></comment></comment></comment><comment><USER5> Apparently this takes "2500-V100 days" which is an insane amount of resources for images of this resolution.

For context, this is equivalent to 100 $10,000 GPUs running for 25 days, 24/7.

[https://twitter.com/jm_alexia/status/1273349716915470340?s=2...](https://twitter.com/jm_alexia/status/1273349716915470340?s=20)

<comment><USER6> Cost to train the model (which can be used for many images, and images can be upscaled using AI):

$10000 * 2500days / 5yr = $15000 hardware cost

200W * 2500day * (0.10 USD / Whr) = $1.2M in electricity

<comment><USER7> If electricity cost $0.1/Whr, a cup of tea would cost ~$10 just to run the kettle. You've overestimated the cost of electricity by roughly a thousand- fold.

Residential electricity prices in California are more like 0.19 USD/kWh, which is 0.00019 USD/Wh.

<comment><USER8> Found the Brit.

</comment></comment></comment></comment><comment><USER9> In case anyone looking through the linked article is also wondering why the images look odd, it's because (buried in the middle):

> motivated by early color display palettes, we create our own 9-bit color > palette to represent pixels. Using this palette yields an input sequence > length 3 times shorter than the standard (R, G, B) palette, while still > encoding color faithfully.

Makes total sense now. In fact, those images remind me so much of how photos looked on the early internet, since many were palletized.

</comment><comment><USER10> I'd be surprised if this architecture scales to larger resolutions, but any move towards "general learning" is really the interesting next step to me, not scaling up an inefficient architecture. Can they train the same GPT model on both text and images tasks at once, and would either task benefit at all from training on the other task?

Even GPT-3 seems to have trouble with world-modeling, it writes convincing text that has all the signs and form of good prose, but the output repeatedly violates physics and common sense in funny ways.

I know just enough about machine learning to have dangerously unrealistic expectations, but I'd like if I could reasonnably hope to see signs of a shared representation or shared knowledge between say image labeling and language modeling. This looks like a very concrete data point to take if you care about generality. Maybe then we can seriously talk about world-modeling.

<comment><USER11> Of course it's not going to scale much past this, it's quadratic and already hitting painful compute levels.

However, if you were starting this research today, you'd use any of half-a- dozen different self-attention variants which are roughly linear, including OA's own Sparse Transformers (which they did use to generate images, just on a far smaller scale which wouldn't be adequate to show competitive performance with SimCLR etc). With those, it's perfectly possible to do self-attention over whole images at 256px or higher.

(As a matter of fact, Aydao has been working on a StyleGAN which just uses self-attention every layer instead of convolutions, using one of the new attentions; you can see some generated image samples from Flowers here: [https://github.com/tensorfork/tensorfork/issues/31](https://github.com/tensorfork/tensorfork/issues/31) GAN loss, not autoregressive pixel likelihood, but it makes the point.)

<comment><USER11> * lucidrains, not Aydao

</comment></comment></comment><comment><USER12> I made a video explaining this paper if interested! [https://youtu.be/7rFLnQdl22c](https://youtu.be/7rFLnQdl22c)

<comment><USER13> What an excellent video. I know very little about this field but I was able to make sense of the things you've explained. I need to learn some more fundamentals I guess, but I'll surely be revisiting your video after that.

</comment></comment><comment><USER14> This looks like a new iteration on the Pixel-RNN idea: [https://arxiv.org/pdf/1601.06759.pdf](https://arxiv.org/pdf/1601.06759.pdf)

GPT2 definitely looks better.

<comment><USER15> No, this is a new iteration on the AIAYN idea.

<comment><USER16> What's this ?

<comment><USER15> Attention is all you need is the title of the 2017 paper that introduced the Transformer architecture.

It reflects the evolution of NLP models: (RNN) —> (RNN + attention) —> (attention) and the idea that you don't need the recurrent component. Just look at all elements of the sequence at once, applying varying weights (attention).

</comment></comment></comment></comment><comment><USER17> "As further proof, features from the model achieve state-of-the-art performance on a number of classification datasets and near state-of-the-art unsupervised accuracy on ImageNet."

Impressive stuff! This performs well even without domain-specific architecture choices.

<comment><USER18> One more step for ML. It used to be that we needed hand designed image features. Now we can learn even the image priors (spatial locality and translation invariance) from data.

Transformers are basically learning relations between pairs of input tokens, moving the problem to a more abstract level than predicting directly on tokens. While CNNs excel at benefiting from those two forms of invariance, transformers have permutation invariance, they can predict on sets, graphs and non-euclidean spaces.

<comment><USER11> > Now we can learn even the image priors (spatial locality and translation > invariance) from data.

Right. The attention layers even learn attention patterns which look like convolution layer kernels! But better, presumably: [https://arxiv.org/abs/1911.03584](https://arxiv.org/abs/1911.03584)

</comment></comment></comment><comment><USER19> The post states, "When we train GPT-2 on images unrolled into long sequences of pixels, which we call iGPT, we find that the model appears to understand 2-D image characteristics such as object appearance and category." This, to me (former philosophy student) feels like a very low bar for "understanding." Could the model _explain_ 2D image characteristics, or can it only generate them? I'm sure this debate will rage on for a while, but when it comes to intelligence, I believe we ought to be more rigorous with our use of the word.

<comment><USER20> Author here. You're absolutely right that "understanding" is a fuzzy word. As you pointed out, part of reason we hold this belief is that the model can generate diverse samples and successfully complete out-of-distribution inputs. But the other part is that the model learns (without labels) useful features for classifying objects. Would be very interesting to test it on a broader set of datasets which measure other 2D image characteristics.

</comment></comment><comment><USER21> This may be my imagination, or maybe it's just because the images are so small, but these seem to have fewer of the slightly disturbing artifacts you see in CNN generated images.

<comment><USER6> The huge artifacts obscure any uncanny valley small artifacts.

</comment></comment><comment><USER22> There's some definite overfitting apparent in the completion of the bottom half of the cat (blue bkg) photo. Every completion has a funny index card covering the bottom including the "original", while autocomplete should be more robust than recreating the original photo.

<comment><USER23> I think that the edge of the index card is included in the top half of the image.

</comment></comment><comment><USER24> In the design field, there's an adage — constraints inspire creativity.

This work seems so unconstrained in its use of computation, that is almost screams to me that they must be going about it the wrong way.

<comment><USER25> You might take a look at The Bitter Lesson [1], it's referenced by the article and linked around on this thread.

> _One thing that should be learned from the bitter lesson is the great power > of general purpose methods, of methods that continue to scale with increased > computation even as the available computation becomes very great. The two > methods that seem to scale arbitrarily in this way are search and learning._

> _The second general point to be learned from the bitter lesson is that the > actual contents of minds are tremendously, irredeemably complex; we should > stop trying to find simple ways to think about the contents of minds, such > as simple ways to think about space, objects, multiple agents, or > symmetries._

[1]: [http://www.incompleteideas.net/IncIdeas/BitterLesson.html](http://www.incompleteideas.net/IncIdeas/BitterLesson.html)

<comment><USER24> I did take a look at the article before writing my comment, and I disagree with its premise as well as conclusion. The human mind has a variety of specialized functions. Our ocular cortex does a sort of convolution on a 2D field with three color and one alpha 'sensors' (cones and rods). If specialization were less powerful than generality, why isn't our brain one giant lobe with no diversity in neuron topography?

The idea that specialization is not as powerful as computation fails the most basic test of a proactive, rather than retroactive, theory. Can you make proactive claims about what works in any given domain? Is the solution to take the hungriest algorithm and apply it? What about feature engineering, cleaning, parameter tuning, analysis, etc.? Is the most power hungry solution still the most effective? In my opinion, part of the reason humans aren't just giant computation blobs is that we thrive on constraints (physical, sexual, emotional).

</comment></comment></comment><comment><USER26> As someone not up-to-date with literature, are one-pixel/few-pixels/small- delta attack issues resolved yet?

<comment><USER27> Well, there is a whole seperate line of research concerning the topic of these input perturbations ranging from PGD to just Gaussian noise. This model does not claim to defend against any of those.

<comment><USER20> Author here. I ran some early experiments a while ago, and it looked like adversarial examples for convnet classifiers didn't transfer to transformer classifiers and vice versa. Definitely worth looking more into!

<comment><USER28> Could you elaborate a bit more? What were the differences between transformer adversarial examples and cnn adversarial examples?

<comment><USER20> I didn't notice any obvious visual differences, but I'm also not an expert on adversarial examples. The transformer models were similarly susceptible to attacks, but while adversarial examples transferred well within a model class (~40%), they did not across model classes (~5%). These are rough numbers from memory, don't hold me accountable!

</comment></comment></comment></comment></comment><comment><USER29> What are some real world use cases for something like this?

<comment><USER18> We now understand better the capabilities of the transformer, which is the hottest thing in AI in the last 2 years. The transformer apparently can learn images even if nobody tells it about the properties of space. CNN's on the other part rely heavily on these properties (spatial locality and translation invariance).

</comment></comment><comment><USER30> i can imagine that in 15-25 years, consuming too much content esp. "AI" generated content will have guidelines around it. The possibilities with this stuff is just starting, it could 'overload' youtube and other hosting services once 'creators' begin to normalize its power (10 yrs from now thers an easy photoshp/adobe plugin to generate random video/image scenes with actors and generated voices/movement (animated rigs aka mixamo))

</comment><comment><USER31> So how would this model be used for a classification task?

<comment><USER32> By extracting the features for the image (that is, the encoding, as in transfer learning in computer vision or natural language processing with, for instance, VGG or Bert, respectively) and feeding this to the classifier.</comment></comment>


Introducing Nebula One private cloud system - <AUTHOR> https://www.nebula.com/blog/2013/04/02/introducing-nebula-one/ <comment><USER2> This looks like a great little system. Congrats to all the team. So many questions!

The most important one: Is that Patrick Stewart doing the voice over in your video? :-)

My only concern with the design is that it's not clear that clustering gives you true redundancy, as it appears the 48 port 10gbps switch is integrated to the same PSU as the controller, and could be a SPOF for a rack of servers?

I assume the switch is essentially running a completely separate control plane from the x86 hardware, and forwarding plane, that can be controlled by other appliances in the cluster?

<comment><USER3> I think the idea is to have multiple nebula controllers, one per rack or so? From 2nd page of The Register article:

"You can run a cloud with a single Nebula One controller, but the system was designed to have multiple controllers for high availability and resiliency, says Kemp. The Cosmos operating system can currently span as many as five controllers in a single OpenStack controller domain and automatically load- balances work across controllers and the five racks of servers attached to them. With those five racks, you can have on the order of 2,500 cores and 5PB of storage, depending on the servers you pick."

[http://www.theregister.co.uk/2013/04/02/nebula_one_openstack...](http://www.theregister.co.uk/2013/04/02/nebula_one_openstack_controller_appliance/page2.html)

<comment><USER2> Yes, I think we both read that article correctly.

I'm saying that a rack full of servers is all hooked up to a single controller. You then have multiple, interconnected controllers, each one connecting a rack of servers. This is meant to give you HA and resiliency.

However, that is only true if the switch control plane is separate from the x86 hardware running in the controller (because, here we're getting HA and redundancy against the risk of controller hardware failure). I'd hope or expect that to be the case, because Nebula has smart people, but I don't see any material to that effect, yet.

(Typically a DC server farm will have two switches per rack of servers, with a dual-homed server having a connection into each. Therefore if one switch dies, you still have a redundant data path (though, only half the bandwidth)).

<comment><USER4> This is correct. The switch will continue to work if the x86 hardware fails. In this case the other controllers will take over management of the servers that are plugged in to the controller with the failed x86 hardware.

</comment></comment></comment></comment><comment><USER5> I work at Nebula, I'm happy to answer any questions.

<comment><USER6> Hi, first off...the Nebula One looks great, really hope to try it out someday at work.

Regarding the underlying framework, I'm not very familiar with OpenStack (we use CloudStack right now), but is this a proprietary fork of OpenStack or are you guys moving the project forward upstream?

<comment><USER4> Hi, I'm a nebula employee and I have been a key contributor to OpenStack since its founding. We have not forked OpenStack and we are committed to making the upstream project a success. We have some proprietary pieces around the control plane, storage and UX, but we will continue to participate heavily in upstream development and plan on integrating more OpenStack pieces as they stabilize.

<comment><USER7> Hi Vish! When you say "proprietary pieces around the control plane", does that mean inside or outside of nova?

<comment><USER4> Outside of nova.

</comment></comment></comment></comment></comment><comment><USER8> I've spent some time deploying Openstack on Cisco UCS and this box would seem to replicate much of that functionality while adding versatility.

Pretty much exactly what I've wanted.

The benefit of converged infrastructure while leaving a choice in compute vendor and integrating the controller, interconnect and orchestrator in a single unit.

Very, very cool.

</comment><comment><USER9> Really Awesome. This sounds like an awesome market to get into. I just spent the last three days toying with various Data center visualization solutions and they all suck ( even OpenStack) in terms of usability and accessibility. It would be awesome to plug in a server and have all the power that OpenStack promises up and running, maybe Nebula can do it.

</comment><comment><USER10> I really love this. I have been waiting for somebody to do this to OpenStack and for it to be a team of original contributors makes me even more excited. I would love to "cloud-enable" our server infrastructure.

</comment><comment><USER2> Another random question: does this require any special software on the managed servers? For example, HP iLO Advanced, for additional server management features? (Don't think so, but worth asking).</comment>


A User's Guide to FiveThirtyEight's 2016 General Election Forecast - <AUTHOR> http://fivethirtyeight.com/features/a-users-guide-to-fivethirtyeights-2016-general-election-forecast/ <comment><USER2> This election has been a bit unusual (e.g. multiple Republican politicians announcing they will vote Democrat), I wonder how much of a problem that is when your model builds on historical data? Though it seems they mostly use that for longer term predictions, and trend towards just believing the polls later on.</comment>


Ask HN: How does Google Docs do print layouts? - <AUTHOR> I am trying to figure out if there is an HTML WYSIWYG editor available that displays documents in a print view mode the way Google Docs. If not, how does Google accomplish it? <comment><USER2> Not that I'm aware of. Most wysiwyg editor libraries rely on contenteditable and they all (?) use the browser's layout engine, which doesn't give you access to pagination.

Google just wrote their own layout engine. Open your browser's dev tools and watch what it's doing to the DOM as you edit text- it essentially positions each glyph as it sees fit.

I'm not aware of any easily available package that can do the same. I wish there was, I've actually started to write something but quickly got intimidated at the enormity of the task.

Even draftjs or quill which have a very cool input/editing story still use the browser's layout engine to render the text so you'll never have the level of control gdocs does.</comment>


New York Under Water - <AUTHOR> http://nautil.us/issue/49/the-absurd/new-york-under-water <comment><USER2> This is a science fiction story about a world in which the sea level has risen "more than 50 feet" (15m) in 2140. This is a reasonable conceit for a story, but, it's important to not update our beliefs about the real world based on fictional evidence.

Eyeballing a graph in the IPCC climate change report, the worst-case scenario for sea level rise by 2200 has a wide error bar, but it's from .6m-2m; the average-case scenario is from .3m-1.1m. Reduce those numbers further for a setting in 2140, and this is not enough to flood New York as described. [https://www.ipcc.ch/pdf/assessment- report/ar5/wg1/WG1AR5_Cha...](https://www.ipcc.ch/pdf/assessment- report/ar5/wg1/WG1AR5_Chapter13_FINAL.pdf) pg 1188.

<comment><USER3> I heard a radio programme once about nature-vs-nuture and they were using the fictional novel, We Need To Talk About Kevin, to sort of present evidence and discuss the subject, and all the way through I was thinking 'but this novel is just what someone sat and made up based on imagination...'

</comment></comment><comment><USER4> I have a suspicion that the approach that will be used is the Dutch approach and a Dam will be built across the Block Island Sound enabling reclamation of significant amounts of land. The value of NYC and the ability to expand the city should make it worth while.

I suspect dams will be used to protect Miami which already suffers from sea flooding.

<comment><USER5> I'm actually really intrigued, somewhat enamored even, by this idea. The newly-created land seems like it would be astronomically valuable, and so the value proposition is surprisingly interesting to me. Brings a lot of questions to mind --

Are there any comparable projects, of such massive scale (or even somewhat close), to compare to? Especially on cost?

Does the sudden influx of that much land not cause existing land value to plummet, or is there an established way to ration it into the market to help mitigate that? Is this something that could even happen politically, or would entrenched interests / existing land owners prefer to create canals (not sure if realistic, but curious) which would probably reduce the property inventory & in theory increase value?

<comment><USER6> Chicago expanded their shoreline via landfill from 1870-1940. Not sure about cost / land value.

[http://www.encyclopedia.chicagohistory.org/pages/3713.html](http://www.encyclopedia.chicagohistory.org/pages/3713.html)

[https://www.cityofchicago.org/content/dam/city/depts/cdot/Sh...](https://www.cityofchicago.org/content/dam/city/depts/cdot/ShorelineHistory.pdf)

A modern article says an additional 225 acres could add $62.5B.

[https://chicago.curbed.com/2015/1/21/10001048/we-could- keep-...](https://chicago.curbed.com/2015/1/21/10001048/we-could-keep-the- lakefront-sacred-by-adding-more-of-it)

Also, somewhat related, Chicago raised their city 6 ft in the 1850-1860's.

[https://en.wikipedia.org/wiki/Raising_of_Chicago](https://en.wikipedia.org/wiki/Raising_of_Chicago)

</comment></comment></comment><comment><USER7> The hardest thing to believe in NEW YORK 2140 is that the stats language R is still in active use 120 years in the future.

<comment><USER8> Right? All the core models are going to be FORTRAN2110 because later editions weren't widely adopted.

<comment><USER9> All the core models? Are you forgetting the critical ones baked in PyCOBOL 3.8 (circa 2084)?

</comment></comment></comment><comment><USER10> There's always the possibility that an array of dams and locks could be established, to preserve New York harbor, as it exists.

It'd be expensive, but New York is already expensive. Would such a project consume more concrete than other dams?

</comment><comment><USER11> When I was about 12, I was in a academic play in which I had to recite a portion of the quote on the statue of liberty by Emma Lazarus:

"Give me your tired, your poor, your huddled masses yearning to breathe free, The wretched refuse of your teeming shore. Send these, the homeless, tempest- tost to me, I lift my lamp beside the golden door!"

I was very impressed by this thought.</comment>


Alan Kay – The center of "Why"? (2004) [pdf] - <AUTHOR> http://www.vpri.org/pdf/m2004002_center.pdf <comment><USER2> There's an idea repeated through the lecture that I find very interesting:

> _" A deeper realization came further on: there might be limitations in > language itself and our ability to represent ideas and think about them that > could preclude us from actually understanding the foundations of our > situation._

> _Living organisms are shaped by evolution to survive, not necessarily to get > a clear picture of the universe._

> _There's a nice saying from the Talmud: We see things not as they are, but > as we are. That is, whenever we're looking out into the world, we're always > seeing ourselves, we're not really seeing what's out there._

For most of my life I'd assumed that human concepts were this limitless sort of thing capable of making fundamental connections to what's going on in the universe at some deep level. In more recent years I've come to consider them as almost like another sense: they are symbolic patterns consistently formed when we are exposed to certain stimuli (like our experience of smells consistently reappearing when exposed to similar molecules). Our conceptual faculty augments the pure pattern-correspondence of our more primitive senses in that the patterns can be associated with other internal patterns, and in that we can generate new patterns purely from existing patterns (using logical and analogical processes), which at some future time may be usefully associated with some never-encountered external stimulus. This is of course extremely _useful_ —but we really must be falling into what should be an obvious trap of anthropocentrism in ascribing to them much more than that.

As an example of where we're likely tripping up by considering concepts to be somehow transcendent: there's a common unjustified stance that mathematics is not only capable of _describing_ the universe, but that it's somehow behind its operation. Even further than that, I often run into smart people with the view that reality is somehow _made_ of mathematics. I wonder if that's what Kay's getting at with this:

> _Art is "all the stuff that people make", and this includes our beliefs > (which we like to call "reality")._

Almost in a similar manner to the way one might fail to recognize the absurdity of a misplaced pink elephant in a dream, we seem structured to unknowingly substitute our mental reference systems with the things they refer to.

<comment><USER3> > > "A deeper realization came further on: there might be limitations in > language itself and our ability to represent ideas and think about them that > could preclude us from actually understanding the foundations of our > situation.

I see this play out every day. As a result of the continued linguistic apartheid in my country, the languages are dead, and the people are stupid as dirt (on average), to be blunt. However, this is mostly Karma, the result of accumulations of centuries past, playing out as the complete decimation of histories; naturally, the education/media system is a propaganda op, with tons of shills at every corner. Those whom you'd wish to see break out of chains, appear to find comfort in them alone. New ideas can't touch them nor move them, because the "programming language" is so outdated, that such sentences will be treated as garbage.

I believe there are deep reasons why India could only think of independence through a class which was more British than Indian. India is but a grave for the civilization it once sustained, and we are but mere flies who wish for a phoenix.

Language does indeed governs our thoughts, and in that respect governs the world we create - it's what makes us human, far more than anything else. Take out the language, and what you get is a mere animal.

<comment><USER4> This is a little vague to me. Could you please go into more detail?

</comment></comment></comment><comment><USER5> It's clear that he was lucky to have a school teacher who inspired him from an early age and effected his world view significantly.

I guess he got lucky, but as a parent I worry about this. He points out that most educational establishments don't encourage individual exploration, and that was my experience too.

Makes me consider home schooling, but that seems like a less than ideal solution too. Do we just have to rely on luck?

<comment><USER6> Parents are more important than schools, because things the parents fuck up in their kids' education are nearly impossible for the school to reverse. The reverse isn't true, though. The disenchantment with education a child experiences through shitty curricula or teachers can fairly easily be counteracted by the parents by engaging the kid. Which takes a lot of time and love, which is probably why so few do it.

I'm not a fan of homeschooling because normal school is usually a great place for kids to learn society. So much _stuff_ happens at a school that can never be recreated by dragging your kids to clubs or events. All the fighting, the teasing, the friction, but also the playing. I've had a couple truly horrible teachers, but I've also had a couple truly great ones. The former taught me how to deal with authoritarian assholes while the latter have advanced me humanistically.

If you want to help your kids, stick'em in a stink normal school with stink normal people, and fascinate them privately.

<comment><USER7> _I 'm not a fan of homeschooling because normal school is usually a great place for kids to learn society._

It's a great place for them to learn a completely artificial society designed by 19th century Prussian educators to train proles to become good factory workers. Modern society? Or any real society? Not so much.

There is no natural society that consists solely of a group of peers all within 12 months of age, under the control of an arbitrary authority figure, where every activity is controlled by a bell. The only thing that's remotely similar is military boot camp (that may be why this model appealed to the Prussians so much). Even prison has more diversity of perspectives, ages, etc.

</comment></comment></comment><comment><USER8> Does anyone know the source for his Einstein quote on page 9: "You should be careful to distinguish what is true from what is real."?

<comment><USER9> I'm guessing that isn't a direct quote. It looks like the difference between truth and reality was a bit of a theme for Einstein. For example, his "I don't believe in Mathematics" quote sounds like it might be trying to say the same thing.

Another, "As far as the laws of mathematics refer to reality, they are not certain; and as far as they are certain, they do not refer to reality."</comment></comment>


Zoom E2E Encryption Explained Like You're 5 - <AUTHOR> https://github.com/azianmike/zoom-e2e-explained/blob/master/README.md <comment><USER2> Zoom video conferences are NOT E2E encrypted, but they claimed it is. That is the main problem.

Also, there are possibilitys to archive this while still beein usable. Apples FaceTime does support E2E encryption for video calls.

<comment><USER3> The discussion of Zoom caused me to look into what FaceTime does. AFAIKT, there is no protection against MITM attacks by Apple against FaceTime. So in the end we have to trust Apple much like we have to trust Zoom. Of course one of those parties might be a lot more trustworthy than the other but the point still stands. For the sort of protection that e2e can provide the user has to have the ability to verify who they are talking to. Both Apple and Zoom are misleading by omission, one more than the other.

</comment></comment><comment><USER4> In a better world, they would be legally accountable for false claims of being end-to-end encrypted.

</comment><comment><USER5> This article is rather badly written, it puts the most important part in the second to last sentence, almost as an afterthought.

Plus Caesar encryption is a bad example, since it's so bad that you can actually compress the ciphertext using standard lossless compression algorithms.

</comment><comment><USER6> I agree there are complications that e2e adds that makes it pretty much unfeasible for something like Zoom. But why is client-side compression not a solution to the problem statement here?

<comment><USER7> Assume you are on a call with three other people. Alice has a solid connection and you can deliver best quality video to her, Bob is on a low bandwidth connection from home and you will need a better compression ratio for him because he can't handle full bandwidth, while Carol is on a shit mobile connection that has high latency, low bandwidth, and drops packets. For each of these you would want a different encoding scheme and possibly very different packet sizes, but if you are sending it from your desktop at home you would basically be sending almost 3X the amount of data to handle the three different streams (plus the additional CPU for all of the different compression steps) -- Zoom sends one stream to their servers and then re- broadcasts at different compression levels and different packet schemes to give the best result for each user.

Justifiable, but in no way shape or form is this E2E encrypted and any company that makes this claim is committing fraud.

The tl;dr of "Zoom E2E Encryption Explained Like You're 5" is rather simple: "It isn't E2E encrypted"

<comment><USER8> Sounds like a good use case for Scalable Video Coding [https://en.wikipedia.org/wiki/Scalable_Video_Coding](https://en.wikipedia.org/wiki/Scalable_Video_Coding)

<comment><USER9> > A subset video bitstream is derived by dropping packets from the larger > video to reduce the bandwidth required for the subset bitstream.

How do you combine that with worthwhile encryption?

<comment><USER8> Do you mean that it will leak additional information due to compression?

</comment></comment></comment></comment></comment><comment><USER10> "Explained Like You're 5 and don't care about the way things actually work".

Read about Scalable Video Codecs (H.264SVC, HEVC, VP9, AV1), SFU vs MCU architectures and then try again. The real reason end-to-end encryption is hard with SFU-mediated multiparty is very different from what is being described.

</comment><comment><USER11> It may be a naive question: what about compressing and then encrypting?

</comment><comment><USER12> > Why can't we compress encrypted messages?

Not sure if eli5, but you can compress that message. Counter example: Huffman encoding that message.

<comment><USER5> You can't compress the ciphertext encrypted with typical encryption algorithms, because the output is indistinguishable from uniformly random bytes.

But you can losslessly compress data encrypted with Caesar encryption, because it's unbelievably weak.</comment></comment>


Search for Google's "New Approach" post, get locked out of baidu.com - <AUTHOR> http://yro.slashdot.org/comments.pl?sid=1508260&cid=30746302

<comment><USER2> Don't blame Baidu — it works on any server located in China. Just append "?google.blogspot.com" and you're blocked: <http://www.faas.cn/?google.blogspot.com>

<comment><AUTHOR> Awesome. It only locks out the site you searched on -- so you can do that with faas.cn, then go to baidu with no problems. Sneaky!

<comment><USER3> It can be used against arbitrary oversea ip and ip:port pair over TCP.

GFW can also act like a 10x amplifier for arbitrary UDP DoS

Best part is that, GFW is implemented on the backbone, no body can stop it :)

also the RST will have an exact 90 seconds countdown, not 5 or 10 minutes, unless you trigger another keyword.

<comment><USER4> GFW=Great Fire Wall?

<comment><USER3> Yup. That's why Microsoft Game for Windows can not do any business in China.

</comment></comment></comment></comment></comment><comment><USER5> Wow, it's the first time I am taking a look at baidu, and the UI seems to be so familiar ... very much related to google!

Maybe this is one of the reasons people are unwilling to shift to google from baidu, both seem similar to them, nothing new in google !

<comment><USER6> I actually like the animated links on Google.cn's page. How well would that type of search page fly in the US? I know Japanese media (news, weather, adverts) tend to use a lot of bright colors and cartoon like characters.

</comment></comment><comment><USER7> If Google were to put a link on there front page which kept using a different key as an argument to link to the post would they block the whole of Google to?

<comment><USER8> They'd probably just block blogspot. If Google really wanted to, they could cycle through practically endless different domain and IPs, but the firewall could probably just pull the current ones out of the pages as they were loaded, if necessary. Or, yeah, just block Google. If they did something like that, it doesn't seem unlikely.

</comment></comment><comment><USER9> Also: <http://news.ycombinator.com/item?id=1050795></comment>


Code Review Best Practices - <AUTHOR> http://kevinlondon.com/2015/05/05/code-review-best-practices.html <comment><USER2> > If the reviewer makes a suggestion, and I don't have a clear answer as to > why the suggestion should not be implemented, I'll usually make the change

This I feel is bad. Code reviews are usually between peers so you shouldn't be afraid to seek out clarification where possible. You shouldn't be making edits to code that goes in production without clearly understanding why.

The other thing that wasn't mentioned, that I think is important, is to not act as a blocker for code reviews unless its absolutely necessary. Lots of engineers take on the attitude that they're going to "gate" code they don't agree with by with holding their +1 and bogging down the review with questions and all sorts of runarounds till its what they want. this is a bad attitude to have, even when you're dealing with Junior engineers.

I'm generally going to +1 something unless I fundamentally disagree with it or think its going to break things in production. What I do, though, is leave lots of comments with questions/suggestions and mention it in the +1 with (see comments).

This builds trust on teams, and stops things getting personal, especially with people who aren't very good at dealing with criticism, even in something as banal as a CR. On a team that works well together, teammates will see those comments, think about them and make thoughtful responses, especially once they understand that you're not trying to get in their way. Giving the +1 gives them the freedom to consider your suggestions without being irritated that their PR is being blocked. They feel like they're in control not you.

In rare exceptions, someone will brush off my questions and merge ... which means that next time, I get to be tougher on the review and specifically ask for responses before the code can be merged, because they've degraded the implicit team trust. Usually repeat offenders are assholes, and assholes generally don't last on healthy teams.

<comment><USER3> I agree 100% with the "don't be a blocker." I personally look for about 80-90% yes and if it meets that criteria with no errors, I thumb it up. If I can't thumb it up, I try to put a "thumbs up with commented items fixed". I try to not block -- comment for bad issues, sometimes sigh but agree when it's a 10% disagreement and go forward. Life is a series of iterations. That 10% will be addressed in a future iteration.

And I never close another persons PR.

</comment></comment><comment><USER4> While I agree with all the points listed in the article, it highlights for me a major problem of a lot of code reviews.

Most code reviews seem to focus on:

1\. Examining what the change does 2\. Finding ways to make the change in a nicer way. E.g. Refactoring etc.

This leaves out the key step 0 - what is actually trying to be achieved, does it need to be done and is there a better (maybe completely different) way to do it.

This leads to a focus on relatively trivial matters such as naming conventions and method lengths.

I think that the underlying reason for this is laziness. Talking about clever refactoring is an easier/faster process than understanding the 'why'.

<comment><USER5> I'm guilty of getting stuck up on trivial formatting issues. When someone pushes a commit that has random whitespace (trailing or arbitrary newlines all over, or just inconsistent spacing), it feels sloppy. Same for many other simple things. If the code is unnecessarily superficially ugly, it sets up a block in my mind that makes it harder to focus on the real issues.

Is it wrong to kick this stuff back and tell devs to make it pretty first?

<comment><USER6> I believe that ugly code is usually buggy code. If someone didn't take the time to deal with trivial issues like formatting then it makes me wonder if they spent time thinking hard about the actual problem they were trying to solve.

I usually explicitly do code reviews in 2 passes. I look for low level problems first, and also try to identify ways that they can be tested using tools so that next time they are caught before the code is sent for review. My second pass looks at higher level problems with the overall design and checks the functionality to make sure it matches the objectives.

</comment></comment></comment><comment><USER7> setting thresholds on method / class sizes seems quite arbitrary and potentially harmful. Splitting a method into n different ones, none of which is called more than once is setting up the code for opportunistic reuse and obscures it's true function. It's especially wrong if the code that was split is mutating / non pure functional.

see [http://number- none.com/blow/john_carmack_on_inlined_code.htm...](http://number- none.com/blow/john_carmack_on_inlined_code.html)

<comment><USER8> I disagree. Breaking down a function into several one time functions that all sit at the same mental model of abstraction make code much easier to reason about.

<comment><USER9> I agree with you in principle but find that code editors let me down in that regard.

Say I split a function up into a few sub-functions because it's getting big. Now I have the problem that I'm jumping forwards and backwards through the code when I want to explore what that function does:

SomeMassiveFunction() { SubfunctionA(); SubfunctionB(); SubfunctionC(); SubfunctionD(); }

SubfunctionA() { } ....

In this case, SubfunctionC() might end up a few pages down in source code. So now it's a context switch to go there and then go back.

Now this can be somewhat avoided with good function names (so you don't HAVE to jump backwards and forwards) and keyboard shortcuts (to make the process quicker), but it's still a trade-off that I think needs to be kept in mind.

<comment><USER10> I think it's a trade off we could get away from if we improved our tools. We should make tools that reduce our cognitive over head. I am regularly diving through code at my new job trying to figure out how everything fits together. Reading code split between various functions and figuring out program flow is a necessary skill, even if you do keep monolithic functions.

I would love an IDE that showed inline in some sane manner function calls, letting me recourse arbitrarily deep - some thing ala code collapsing and expansion. Extra points if you can show a for chart to side of function calls, class interactions and general code flow, highlighting where your cursor is. More points for a pane that shows what variables have to be in order for a the branch of code you're working in to be reached. A changeable granularity setting would be great for all of these.

I think often times we discuss pros and cons without discussing what they could be if we put some effort into changing the current situation.

<comment><USER11> You have suggested so many wins here. Gallons of win.

</comment></comment></comment></comment></comment><comment><USER12> The one thing that is missing, which ALWAYS seems to fall by the wayside, is security. If people incorporated more iterative security testing (static AND dynamic, automated AND manual) and threat modeling into their SDLC reviews there would be a plummeting number of vulnerabilities.

But, because it doesn't fit in with the whole "Lean" approach to software (deliver features yesterday), all but the most established enterprises don't seem to care much unfortunately. Once more people experience a breach because of their desire to deliver first and remediate vulnerabilities later then perhaps more awareness will be raised. By then it's too late though.

</comment><comment><USER13> I have an interesting problem. A co-worker of mine appears extremely sensitive to his code being reviewed and I honestly don't know how to deal with it. He feels attacked (and becomes defensive during code reviews) because the reviewers focus on the "bad things and mistakes" of his code instead of the accomplishments.

Has anyone here dealt with similar issues during reviews?

<comment><USER14> +1 I'm currently dealing with this. The individual doesn't offer feedback on my code when under review, so it appears like a personal attack.

<comment><USER15> Go have coffee (or whatever) with your colleague and have a 1-to-1 conversation with them saying you _want_ their feedback on code. You want their expertise to make you a better software developer. Even if you are better at this than they are, you _can_ learn something from the sharing.

The team's lead should make sure that the individual in question knows that software development is a cooperative process and that code review helps build the team.

<comment><USER16> This. Although I'm less civil about it - I'll start giving people shit for rubber stamping my code reviews.

"Why are you letting me ship this terrible code I wrote? Do you want us to end up in a death spiral of technical debt and crunch? Don't be an asshole, critique my shit! We all need a second set of eyes - I'm no exception!"

...okay, I'm maybe a little more civil than that. But I'm not above leaving some mistakes unfixed to call them out on.

<comment><USER17> Ha! I'm like that with my QA people, begging them to come after me with a blunt weapon.

</comment></comment></comment></comment></comment><comment><USER18> Nice list. This is more or less what I look for. It's nice to see your rules of thumb.

Anyone have any suggestions for time-estimating code review? That's the biggest issue we've faced trying to implement code review into our workflow.

<comment><AUTHOR> SmartBear has their own set of best practices and they recommend about 300-400 loc per hour[1]. I think that's probably about right. It's something I think I've gotten faster at over time, partially because we're improving from previous reviews and partially because it's a skill that develops.

[1]: [http://smartbear.com/smartbear/media/pdfs/wp-cc-11-best- prac...](http://smartbear.com/smartbear/media/pdfs/wp-cc-11-best-practices-of- peer-code-review.pdf)

</comment></comment><comment><USER19> This is not a bad list, and can be useful as a checklist.

But note that it basically tries to define good programming practice in general. That's a very big topic with a lot of room for debate and disagreement.

</comment><comment><USER20> >Variable names: foo or bar are probably not useful method names for data structures. e is similarly not useful when compared to exception. Be as verbose as you need (depending on the language). Expressive variable names make it easier to understand code when we have to revisit it later.

I so agree with this!

Properly named variables is perhaps THE first line of defense against bad code. Too many developers think they are concise and having little code if they only abbreviate their variable names enough.

Honestly, "em", "erg", "fc" and "sc" may make perfect sense to use, but it's a form of obfuscation to future developers (including your future self).

Other pet peeve; adding things to variable names that don't add anything meaningful.

E.g.

"usersList"

Does your code really care that it's a list? Should the reader be pointed at this each and every time. I much prefer just using:

"users"

Clear, to the point, and readable.

<comment><USER5> It's not a form of obfuscation if there is clear and simple context. Catch(ex) is perfectly fine, and adding 7 letters does nothing but add noise. Similarly "var us = getUsers()" is clear - it's not one u, it's multiple, and makes sense to have a loop like "for u in us".

A better alternative if you find the code is still confusing is to add context by cutting a function up into inner functions. This is why I really hate working in languages that make it difficult to define little closures. In F#, I'll often end up with a couple of 1 or 2 line inner functions and it makes things much easier to read. This simply isn't practical in e.g. C#.

In exported function names and certain class names or modules, sure, a bit of verbosity might help. But inside a function it just make it visually harder to understand and I've rarely found it to be beneficial. There's going to be enough context to load into my brain inside a function accurate, and unfortunately I still subvocalize when reading code and all those extra syllables add up.

Additionally, removing letters means you need to split lines less based on length and focus more on when it makes sense.

</comment></comment><comment><USER21> Some of this is frustrating to read. Architectural and detailed design decisions should be made and approved almost entirely before the coding of those features is started. (Obviously this is the ideal and not always possible).

This means that the code reviews should involve little more that a checklist of those approved design decisions against their implementation, perhaps a code style is verified as well.

Coding without a design is just hacking, which I believe is the primary cause of burn-out and should be avoided as much as possible.

So, the question is, do you have a design document? I know it doesn't sound very agile, but traditional engineering procedures, when managed well, have a lot of merit in controlling product quality and cost.

</comment><comment><USER14> Another thing to check for that I didn't see addressed is debug statements. There's nothing quite like seeing console.log("shit"); during a code review :)

</comment><comment><USER22> Awesome. I'm joining Cisco for the summer, so I think this would help me get a head start since they do code review. Thank you!

<comment><USER23> Are you the person who took my place? :) Got to the last stage, twice, in London.

To me, the Code Review Best Practices seem awfully like very general knowledge, just gathered together. Not sure if it's HN-worthy per se.

However, the John Carmack link is quite enlightening.

<comment><USER22> I sure hope not -- I'm working at the headquarters in San Jose this summer.

</comment></comment></comment><comment><USER24> This list is very very basic, most of the things like style shouldn't have to be discussed and design should preferably be done before the code is written.

Just adding "error handling and potential bugs" as a generic bullet on the list just doesn't cut it, these should basically be the only items on the list but specified in much greater detail. A serious code review checklist should contain concrete scenarios of these, preferably tailored for your specific application.

Examples of this are: What happens during system startup, before all other modules are ready to answer to requests? What happens if the user enters malformed data (sql injections etc)? Does this function shut down gracefully (transaction safe)? How will the algorithms scale? Race conditions and deadlocks? What happens if the network to the database goes down? Is localization handled properly? Backwards compatibility?

</comment><comment><USER25> Style, complexity and coverage checks should be left to automated tooling. Code reviews should focus on whatever remains.

</comment><comment><USER26> While I agree with all the points in the blog, I wonder how many programmers do really follow them perfectly(even the author of the blog) because doing code review to such great detail requires plenty of time which is often not the case when you work for corporations.

<comment><USER27> One company I worked for relied heavily on code reviews after every feature. At least two co-workers (one of whom had to be a supervisor) read, ran, and gave feedback on every piece of code. Reading others' code and providing feedback allowed me to improve my sight-linting ability, and it felt like each day my group's code as a whole was improving.

Having some accountability for writing sloppy code is very sobering.

<comment><USER28> > supervisor ... ran ... code

This and this again. Engineering managers, and all that. But also, don't just review, 'sight-lint', and reason about code. Rather, run the tests! It's the shared accountability and knowledge. Does the baseline capability exist (tests pass)? Can you, the reviewer, spot fallacies that the tests don't capture (if not, criticisms feed back to you later)?

</comment></comment></comment><comment><USER29> > If we have to use "and" to finish describing what a method is capable of > doing, it might be at the wrong level of abstraction.

When I coee, somewhere a method needs to initiate this execution flow and will therefore contain "and". Even if all it does is call two other methods where this principle is followed. How do I avoid this? I mean, somewhere in the code it makes sense to execute the methods together.

</comment><comment><USER30> Having worked for businesses that use code reviews and those that don't, I personally favor not having code reviews. The reason is that they hinder development speed quite a lot, since you have to try to predict what other engineers will say on your reviews, which takes a lot of brainpower.

<comment><USER31> Also having experienced both, I wouldn't want to work in a place that didn't have code reviews.

I ask myself, "WTF was this guy thinking when he wrote this?" far less often when I'm working on code that's been code reviewed. Most of the time, the reviewer catches code like that, and requests that it be fixed and/or commented, before I end up debugging through it 6 months later.

IMO, open office plans, reading HN and nerf guns hinder development speed far more than code reviews.

<comment><USER16> I experienced both within the same company - they started without code reviews, and the later introduced them. The result was overwhelmingly positive, greatly improving code quality and helping spread knowledge of new systems and utilities.

I certainly believe it's possible to do code reviews sufficiently "wrong" that they're a net hindrance. Drape them in too much ceremony, put too much emphasis on style (Bob doesn't like your whitespace preferences, and he's the code owner, so your changelist is vetoed!) and too little emphasis on subsistence (these corner cases and possible bugs concern me).

.

We went with a relatively light touch approach. Nothing was technologically enforced - our lead dev simply told us he wanted everyone to start having everything reviewed by anyone - both for code quality and to spread knowledge - and that he'd be pissed if he found a bug in one of our changelists and found out it hadn't been reviewed, going forward.

Whitespace / naming stuff I tended to submit without review.

Quick and obvious fixes, small changes, etc., I was fairly willing to start a review request, submit, and then fix anything caught in the review in a followup changelist.

For larger changes where I had more reservations, I'd usually hold off on submitting until after a review. Maybe threaten to submit if they were dragging their feet too long :P</comment></comment></comment>


Hidden features of Python - <AUTHOR> http://stackoverflow.com/q/101268/375966 <comment><USER2> >>>import antigravity

try it!</comment>


GCC Moves from C++98 to C++11 - <AUTHOR> https://github.com/gcc-mirror/gcc/commit/5329b59a2e13dabbe2038af0fe2e3cf5fc7f98ed <comment><USER2> Good stuff. Saves me adding an extra flag in every makefile.</comment>


Ask HN: What are some good mobile SDKs (Android and iOS)? - <AUTHOR> I need design inspiration. <comment><USER2> In no particular order:

Stripe. React Native. Firebase.

</comment><comment><USER3> My English is poor , Sorry

In iOS platform , XCode is best. XCode is developed by Apple .so Apple optimise it several years.

In Android platform . There are various choices. Android Studio ,You can try it .

If you can use others languages fluently. E.g Node.js ,It have lots of frames can work on Android & iOS platforms .

In my opinion ,i don't think any SDK is best. I think the best SDK are what your use best.

By the way , design inspriation, I can't help you , But you can find them anywhere. Before you write your the first line of code, You should consider twice . 1) Who will use my App 2) Worth of my App 3) Features of my App

This is my first port.the typesetting is poor , Sorry</comment>


Unified Memory: The Final Piece of the GPU Programming Puzzle - <AUTHOR> https://www.nextplatform.com/2019/01/24/unified-memory-the-final-piece-of-the-gpu-programming-puzzle/ <comment><USER2> Lol. Perhaps you can help me ? The q before yours thanks</comment>


How I got better at debugging - <AUTHOR> http://jvns.ca/blog/2015/11/22/how-i-got-better-at-debugging/ <comment><USER2> While your article is interesting re building better debugging skills, I wonder how many of the tools mentioned require root access or more than just user access.

Is tcpdump or strace available to those who don't have control over the machine? Or is there a variant that would be available ?

<comment><USER3> You need root for tcpdump. I don't think you need it for strace. In these situations, you want to reproduce the issue in an environment, where you have root access.</comment></comment>


Stanford University acquires source code for Multi-User Dungeon - <AUTHOR> http://www.gamasutra.com/view/news/216421/Stanford_University_acquires_source_code_for_first_ever_online_virtual_world.php <comment><USER2> Multi-user graphical online virtual worlds, multi-user dungeons -- these things were already booming on the University of Illinois' PLATO system in 1978; they'd started popping up in 1973, maybe even '72\. You had pedit5, moria, oubliette, krozair, dnd, dungeon, then in 1979 avatar, just to mention but a few. Multi-user dnd games were one of the most popular activities on PLATO all during the mid to late 70s. And these games were all graphics-based, not limited to just text like the far more primitive MUD1.

Bartle has gotten a lot of mileage over the years with the MUD1 story, leading to a misinforming of the media and the public, both of whom, when it comes to technology and the history of technology, seem to be perfectly fine with being misinformed. It's a shame to see an institution as prestigious as Stanford fall for the same misinformation and ignore the real history. Five minutes of Wikipedia browsing would have set them straight.

I'm sure MUD1 was very cool during its time, and got a lot of people interested in the ideas of MUDs. Unfortunately, it was not first. Hard as it might be for people to grasp the concept, but more sophisticated multi-user dungeon games existed before MUD1 was even conceived. Deal with it.

<comment><USER3> I decided to do the "Five minutes of Wikipedia browsing" you suggested.

Under [http://en.wikipedia.org/wiki/MUD](http://en.wikipedia.org/wiki/MUD) I found this:

"In 1978 Roy Trubshaw, a student at Essex University in the UK, started working on a multi-user adventure game in the MACRO-10 assembly language for a DEC PDP-10. He named the game MUD (Multi-User Dungeon), in tribute to the Dungeon variant of Zork, which Trubshaw had greatly enjoyed playing.[18] Trubshaw converted MUD to BCPL (the predecessor of C), before handing over development to Richard Bartle, a fellow student at Essex University, in 1980.[19][20][21]

MUD, better known as Essex MUD and MUD1 in later years, ran on the Essex University network until late 1987,[22] becoming the first Internet multiplayer online role-playing game in 1980, when Essex University connected its internal network to ARPANet."

<comment><USER2> Sigh. You're making my point for me. But the irony escapes you I guess.

<comment><USER4> Honest question, does PLATO or any other intranet count for 'first online' anything in the terms of this discussion?

<comment><USER2> PLATO was a network of systems, not just one. By 1978 there were PLATO systems in Illinois, Minnesota, Delaware, Florida, Belgium, and other locales, all internetworked using what was called "the link", a high-bandwidth connection between these various CYBER mainframes. PLATO had collectively many more users than ARPANET during this time; ARPANET's population wouldn't exceed PLATO's until probably around 1981, though it might even be later.

<comment><USER5> Here is Wikipedia page on PLATO: [http://en.m.wikipedia.org/wiki/PLATO_(computer_system)](http://en.m.wikipedia.org/wiki/PLATO_\(computer_system\))

</comment></comment></comment></comment></comment></comment><comment><USER6> Trying to automate my mud play from aliases to scripting with triggers was how I got started programming.

My regex is still better than most ;)

<comment><USER7> Hacking tinyMUD and bots for tinyMUDs is how I really learned C programming.

</comment></comment><comment><USER8> Though a cool artifact, it seems like the essence of something like this comes from interaction with the others using the service.

There's a really interesting podcast segment[1] about the Preserving Digital Worlds[2] project which tries to answer some of the questions about how to best document / preserve something who's value comes from the experiences users share with each other

[1][https://soundcloud.com/roman-mars/99-invisible-13x-game- over](https://soundcloud.com/roman-mars/99-invisible-13x-game-over) [2][http://pvw.illinois.edu/pvw2/](http://pvw.illinois.edu/pvw2/)

<comment><USER9> 99% Invisible should turn up on HN more often. It's a very interesting and very well produced podcast that should interest a lot of people here. The episodes are short and it's worth digging through the archives.

</comment></comment><comment><USER10> I still play on [http://new.vampire-wars.com](http://new.vampire-wars.com) :)

<comment><USER11> Cool! I'm still playing Discworld MUD ([http://discworld.starturtle.net/](http://discworld.starturtle.net/)) since 2000. It was my saviour through my leukemia treatment.

</comment></comment><comment><USER12> Direct link to the announcement:

[http://www.stanford.edu/group/htgg/cgi- bin/drupal/?q=node/11...](http://www.stanford.edu/group/htgg/cgi- bin/drupal/?q=node/1191)

</comment><comment><USER13> I miss the good old days of: [http://en.wikipedia.org/wiki/GemStone_IV](http://en.wikipedia.org/wiki/GemStone_IV)

The best MUD I've ever played!

<comment><AUTHOR> If you played that in the late 1990s and early 2000s, you may have seen artwork of mine in there in low-level areas. =)

<comment><USER13> Awesome! I played while it was very popular on AOL and for a few years after they started charging for it.

</comment></comment></comment><comment><USER14> I will always love Medievia.

</comment><comment><USER15> star wars mud ftw swmud.org 6666</comment>


Welcome to 2030. I own nothing, have no privacy, and life has never been better - <AUTHOR> https://www.weforum.org/agenda/2016/11/shopping-i-can-t-really-remember-what-that-is/?utm_content=buffer5e0e4&utm_medium=social&utm_source=athnshah&utm_campaign=buffer <comment><USER2> so communism again?</comment>


Lamest bug we ever encountered - <AUTHOR> http://joostdevblog.blogspot.com/2011/12/lamest-bug-we-ever-encountered.html

<comment><USER2> Reminds me of the time I had written a physical simulation engine back in grad school and there was a "minus" sign error. Of course, the error was rare enough that we didn't notice it until after the code was used in a real production environment. Tracking down one minus sign in several hundred thousands of lines is a pain. Not to mention the uneasy feeling you get after you solve it, "How was everything ever working correctly before!? What else did we overlook?"

<comment><USER3> If I have to venture a guess, I guess you didn't have a comprehensive set of tests at the function/method level of the code? Having that would probably have caught the bug, because you would have written a test for correctly executing the code in that branch.

<comment><USER2> You're right. But it was after that pain-staking experience that I became fully engrossed in using unittests for all non-trivial functionality. Live and learn.

</comment></comment></comment><comment><USER4> I'm not completely satisfied by the explanation. I still have that uneasy feeling that you get when you solve a bug, but an unsolved mystery remains. "Also, I still don't know why not all consoles connected to that PC froze."

<comment><USER5> He didn't mention how the logging was done but if it was over a TCP connection then the send() call probably blocked until it timed out since the sleeping computer didn't close the socket nicely, then it had to re-establish the connection. Although reliability is nice, if I were writing a remote logger for a something like a game, I think I'd use UDP.

<comment><USER4> Are you trying to explain how it's possible for some of the consoles to freeze but others not while talking to the same sleeping computer? If so, I did not understand your explanation.

<comment><USER6> I believe socket writes don't block until you've filled the internal socket buffer, so it's likely that the unaffected machines simply hadn't done this yet.

<comment><USER4> ah, there's the missing piece of information. Now I got it, thanks.

</comment></comment></comment></comment></comment><comment><USER7> I'm reminded of this story of the folks who worked on LEO hunting down a similarly difficult-to-find bug that was eventually found to be caused by an unrelated external machine: the manager's elevator. [https://www.youtube.com/watch?v=Lrn24SdW64I&t=2m50s](https://www.youtube.com/watch?v=Lrn24SdW64I&t=2m50s)

</comment><comment><USER8> I once spent an afternoon tracking down a "bug" as to why sales tax wasn't being calculated on LedgerSMB only to find out I had set the tax rate to 0 in the tax interface.... Ok, it was working as intended. I felt pretty sheepish too.

<comment><USER9> The worst bugs are when things work as intended, but you still think it's a bug, such as your example.

<comment><USER10> It's worse when your users find these and are all mad because the computer did exactly what they told them to.

<comment><USER4> Nah, then it's a bug in your user interface.

<comment><USER8> While I am sympathetic to this argument, I would say that is not always the case. Some configuration issues are usually required and when something is set up for a specific case, and it behaves for that case, and the user simply forgot that this is what they did, then it's a bug only in the storage retrival routines of the user's own memory.

</comment></comment></comment></comment></comment><comment><USER11> They could have solved that bug with one developer in ten minutes by just telling the PS3 to generate a core dump and running addr2line.exe on the core dump report's callstacks.

And the report places the blame on the server instead of their code. Clearly it's their code's fault for doing blocking sockets calls in a main thread.

</comment><comment><USER12> This looks like an interesting bug. I wonder if there are more bugs like this from the website view such as analytic tools giving you false or misleading information? Or, even monitoring or performance tools?

</comment><comment><USER13> The lamest bug you will ever encounter deletes your whole /usr.

<comment><USER14> How is that lame?

<comment><USER15> I think he's talking about this [https://github.com/MrMEEE/bumblebee/commit/a047be85247755cdb...](https://github.com/MrMEEE/bumblebee/commit/a047be85247755cdbe0acce6f1dafc8beb84f2ac#diff-1) , where the deletion of /usr was not on purpose... the bug was a space in the middle of a file path in the install script.</comment></comment></comment>


Perverse incentive - <AUTHOR> https://en.wikipedia.org/wiki/Perverse_incentive

<comment><USER2> very interesting, there has been some great answers on this topic on quora as well. [http://www.quora.com/Economics/What-are-some-good- examples-o...](http://www.quora.com/Economics/What-are-some-good-examples-of- perverse-incentives) [http://www.quora.com/Law-of-Unintended- Consequences/What-are...](http://www.quora.com/Law-of-Unintended- Consequences/What-are-the-best-examples-of-the-law-of-unintended-consequences- in-action)</comment>


Ask HN: How effective is monthly Hire me? - <AUTHOR> I see Hire Me posted every month here, was curious to know what is the success rate of landing a job. If not getting an offer, how many companies approached you? <comment><USER2> Related

"Ask HN: Has anyone ever got contacted from the "Who Wants to be Hired?" posts?" [https://news.ycombinator.com/item?id=22468924](https://news.ycombinator.com/item?id=22468924)

"Ask HN: Has anyone ever been hired from "Who wants to be hired?" threads?" [https://news.ycombinator.com/item?id=20330818](https://news.ycombinator.com/item?id=20330818)

</comment><comment><USER3> I posted in today's thread and have received three emails from companies so far. All in my local area.

I also included my linkedin and have received a few connection requests, but no messages from any of them.

<comment><USER4> That's pretty cool, and also pretty lucky. I just found I get my email added to a bunch of spam-lists and received contacts from people saying things like "I see you're in Helsinki, would you like to relocate to Canada?"

<comment><USER5> Body shops always want me to relocate to the Midwest for six months to be a Lotus Notes admin because I developed for it … 20 years ago.

</comment></comment></comment><comment><USER6> It is very high visibility. I posted in the December job thread and received several responses.

Most of those responses occurred in January just weeks before Covid changes and job losses. I talked with several employers. Instead of accepting a new job I chose to instead investigate employment options with the military which likely proved to be a more stable consideration.

</comment><comment><USER7> I haven't posted there. I imagine it's similar to the who's hiring thread. I was excited to look through them, but seems the postings are usually for senior dev or higher positions. I'm not qualified for that (at least not in a relevant technology), so it's been a pretty disparaging experience for me.

</comment><comment><USER8> I look at hire me thread every month but really there aren't any interesting candidates. Pretty much everyone is nodejs/react type and I need to hire people that can build software to last. If I see anyone mention Arc in that thread I will instantly send a job offer

<comment> non-entity Is your company actually using Arc?

I wouldn't be shocked if most of the people well versed in non-mainstream tech don't bother advertising themselves as being such.

<comment><USER8> No, Arc isn't actually a good fit for what we are building.

<comment><USER9> Nice filter. Personally, for engineering positions I only hire a capella singers who play lacrosse :)

</comment></comment></comment></comment><comment><USER10> I have been contacted multiple times from HN Hire me threads. I would say the quality of job requirements coming through HN is very good. I ended up doing multiple contract projects with these monthly Hire HN threads.

</comment><comment><USER11> Saddest part is that if someone declines / folds Your application w/o interview, they are hesitant to provide constructive(good, bad) feedback :/

</comment><comment><USER12> I've hired off it and found good candidates from it.</comment>


Mars rover camera project manager explains 2MP camera choice - <AUTHOR> http://www.dpreview.com/news/2012/08/08/Curiosity-interview-with-Malin-Space-Science-Systems-Mike-Ravine <comment><USER2> It should be noted, as well, that the Hazcams and Navcams are "build-to-print" exact copies of the ones that flew on the MERs in 2003. So most of the images we've been seeing thus far are actually taken with tech much older than the Mastcams, probably designed around the turn of the millennium. And there's nothing wrong with that -- they are very reliable and do exactly what is needed.

Source: [http://www- robotics.jpl.nasa.gov/publications/Mark_Maimone/f...](http://www- robotics.jpl.nasa.gov/publications/Mark_Maimone/fulltext.pdf)

</comment><comment><USER3> The mast camera is actually two cameras with zoom of each camera fixed at two different values:

<http://msl-scicorner.jpl.nasa.gov/Instruments/Mastcam/>

James Cameron lobbied to make zoom lenes for the cameras so they could be set at the same zoom level to create stereo pairs for 3D images.

JPL had "problems designing the lens without using wet lubricants which would require battery-sapping heating"

Difficult trade-offs must have been made everywhere - who knows where that extra battery power went that the zoom lens gave up.

<comment><USER4> Given that the Plutonium power core is putting off 2000 watts of heat in order to generate 125 watts electricity, I find it surprising that they don't have enough heat to keep wet-lubricants heated sufficiently. But - that's an awfully big Rover, and it does get pretty cold on Mars...

2000 watts at 14 years in kilowatt hour - 245,000 kilowatt hours.

<comment><USER5> Remember, it's got a 14 year lifetime because the half-life of Pu-241 is 14.1 years. At the end of that time, half of its mass will be gone, and it'll be producing half as much heat. It won't produce a steady 2KW for 14 years and then _bing,_ go out like a lightbulb. Power produced will slowly fall over time.

<comment><USER6> You mean, half of the plutonium mass will be gone. It will be almost completely be replaced by other elements. Though I don't know how much energy they give off in radiation, or whether they are stable.

</comment></comment></comment></comment><comment><USER7> They should eventually release a series of regular-quality images taken sequentially as the camera pans left and right, and have a competition to see who can squeeze the most resolution out of them using image enhancement and fusion techniques. There are lots of interesting stochastic methods out there (e.g. "compressed sensing") that have application for this kind of thing. It would make a great academic competition.

<comment><USER8> During one of the press conferences Ravine mentioned they had hundreds of images of a single spot on the ground taken by MARDI, the descent camera, after it had landed and that a member of his team was working on using those to build a super-resolution image of the surface. I like the idea of a competition though.

</comment></comment><comment><USER9> I think the explanation is satisfactory, but how often, if at all, do other components of such projects change, up until lift off? I imagine engines and other propulsion parts won't be upgraded much after the original spec, even if such components even remotely followed the same curve of improvement as digital chips do. But what about other types of sensors used by the rover? What is the most recent piece of tech used by the rover?

In any case, it's kind of surprising that by 2004, NASA engineers wouldn't have proposed a solution that anticipated vast improvements in digital sensor technology, so that something, in 2009-2010, could be "dropped in" (relatively speaking, not literally) as a replacement.

Of course such a design feature is going to take way more planning and resources than it would for the holiday consumer camera lineup...but a) this is NASA, some of the best of the best engineers. And b) while panning-and- stitching is always a solution, doesn't that have additional operational risk of its own? Additional panning requires additional mechanical movement and attention to moving parts.

<comment><USER10> Last minute changes are made. Or at the very least I know of one such change in the case of Curiosity.

The dust covers over the haz cams (the navigation cameras on the belly of Curiosity) were added in the last minute. Here is the engineer who implemented them writing about the covers: [http://forum.nasaspaceflight.com/index.php?topic=29612.msg93...](http://forum.nasaspaceflight.com/index.php?topic=29612.msg938852#msg938852)

Basically, the Phoenix lander (landed on May 25th 2008) kicked up more dust than expected. They were worried and did a review for Curiosity, but found out that only the haz cams (because of their location on the belly) were in danger, so they decided to add dust covers to them (and also kept them transparent to see whether there really was a dust problem – as you could see from the first photos with covers still attached there very clearly was).

My guess is that stuff gets changed and updated a) if there is money and resources left or b) if the mission is in danger if you don't change something.

The 2MP sensor is very clearly good enough. Any update in resolution would give you diminishing returns – so something like that gets pushed back.

<comment><USER11> It's all about risk. It's not about "leftover money".

Cost, schedule, risk. They are the fundamental resources for a project like this. Cost and schedule are more familiar than risk.

<comment><USER10> So if it's about cost it's about leftover money. That's one and the same. (One just doesn't sound as professional as the other.)

<comment><USER11> You offered two possible explanations for the use of a 2MP camera, (a) money, (b) risk. I was saying that, in this case, it did not really have to do with money.

My second sentence above was trying to point out that many people underestimate how important risk is to a space mission.

<comment><USER10> Oh, no, I wasn't claiming that this change had anything to do with money. In this case it was clearly related to risk. I was just trying to make a general point.

<comment><USER11> Thanks, understood.

</comment></comment></comment></comment></comment></comment></comment><comment><USER12> They can generate low-res, closeup 3D images of the ground using the stereo hazcams and manual colorization. It's far from what stereo zooming mast cameras would have given them, but I still think it looks cool: [http://nitrogen.posterous.com/curiositys-view-of-mars-in- pse...](http://nitrogen.posterous.com/curiositys-view-of-mars-in- pseudocolor-3d)

And yes, I'll probably keep posting this image in nearly every thread that mentions Curiosity's cameras until NASA starts giving us more color imagery.

</comment><comment><USER13> 2004 technology aside, this is called the "Good Enough Factor". Obviously every super geek at NASA wants an 20 MP camera with full zoom,100 year lifetime, 3d, blah blah blah. But, they are working in very exact specifications and budget. So, you have to opt for the option that satisfies the Good Enough Factor.

What will get us closest to what we actually want, without totally breaking spec and screwing with the time and monetary budget.

<comment><USER14> Everyone is missing the point here. The problem isn't that these cameras are old tech and using outdated CCD censors. If these were the best images we've seen to date of Mars, you're right, there'd be a "Good Enough Factor" — the best images we've seen from a NASA mission, but not as good as modern DSLRs. Understandable.

The problem, that I think a lot of people are missing, is that Viking 1, from 1976, took higher quality pictures of Mars. [http://f.cl.ly/items/0k2w2d1C1O3w3e0t300f/NASAQualityDegreda...](http://f.cl.ly/items/0k2w2d1C1O3w3e0t300f/NASAQualityDegredation.png)

<comment><USER15> You are comparing the very first initial images from Curiosity with the final processed images from Viking. Also, the haz-cams on Curiosity are there only to help Curiosity see where it is going. Whatever design trade-offs they make, they need to make the priority be "don't break the 2.5 billion dollar rover that is supposed to last for over a decade."

If you use a low-res camera to take 1000 images of the same thing, you can use software to make a high-res image from those.

</comment></comment></comment><comment><USER16> I would like to see this result in a frustrated James Cameron building his own Mars probe for filmmaking

<comment><USER17> The lack of an on-board microphone could/should do that, too. :-) Pictures are a great, but a full-30fps video (with audio!) would greatly enhance our feeling of what it's like to be on the surface there.

<comment><USER18> Mars has a very thin atmosphere, so a microphone would capture very little data if any. _Star Wars_ aside, there isn't much sound in the universe.

<comment><USER19> Mars is also windy, and microphones can be made to be very sensitive. Plus, there might be some interest in hearing the dust from one of the common dust storms hitting the rover.

Not a high priority in terms of data, but potentially interesting nonetheless.

<comment><USER18> It appears that NASA already sent a microphone to Mars.

And you can listen to it:

[http://mars.jpl.nasa.gov/msp98/lidar/microphone/mic_found.ht...](http://mars.jpl.nasa.gov/msp98/lidar/microphone/mic_found.html)

[Edit] Never mind, _Mars Polar Lander_ crashed.

[http://en.wikipedia.org/wiki/Mars_Polar_Lander#Communication...](http://en.wikipedia.org/wiki/Mars_Polar_Lander#Communications_loss)

<comment><USER20> NASA sent microphones to Mars twice, but neither were used due to technical reasons (one because the vehicle crashed).

</comment></comment></comment></comment></comment></comment><comment><USER21> This is a serious lack of vision for such an iconic company. Good science and technical ability, doubtless. But someone skimped on a real public relations goldmine here.

</comment><comment><USER22> Because he's an actual scientist not an Internet rasterbator.

</comment><comment><USER23> I really liked the article - it was very coherent and easy to follow.

BTW, is there a full-res movie of the descent? I've only seen the 'thumbnails' stitched together of the heat-shield and the parachute.

<comment><USER24> Not yet. Only 20 or so of the full frames of the descent have been downloaded so far. Will probably be a day or more before they are all down.

You can see the frames that are down here: <http://mars.jpl.nasa.gov/msl/multimedia/raw/?s=0>

<comment><USER23> Thanks for the pointer. This really brought home to me the 'baud rate' issue of interplanetary communication.

Right now, I'm so used to streaming movies at home that I forget the challenges of communications across such vast distances .

Perhaps our children/grand-children will be using the faster interplanetary internet and will recall these days with the same whimsy as we do now our twisted-pair modem days.

</comment></comment></comment><comment><USER25> It would be neat if one day they send rovers in modules - one nuclear power supply to last dozens if not a hundred years, then a mini-robot sent to replace the modular camera with a 15MP one after they upgrade the bandwidth from 2mpbs to 10mbps and then one day 100mbps.

That 15-30 minute pingtime is a problem that cannot be overcome unless they find a faster than light wave that can be used for transmission - or a way to manipulate quantum entanglement for communications.

<comment><USER15> I think the problem is that, if you are going to make a second launch, it's nearly always better to land some place different. We will accumulate a bunch of data on Gale Crater over the next several years, but we have essentially _zero_ data on a whole bunch of very interesting parts of Mars.

<comment><USER26> With enough supplementary shipments, and enough patience, a rover could stroll all over the planet.

</comment></comment></comment><comment><USER27> Can anyone explain what the key limits on bandwidth are when communicating from Mars? Is it just a matter of higher bandwidth requiring higher energy consumption?

<comment><USER28> It's wireless transmission. Your bandwidth is a function of the frequency of your communication channel, and long-distance transmission generally uses low- frequency signals (and as a result, low speed).

The one way to get around this is to use a multi-channel link, that is to say you communicate over several frequencies simultaneously. This is more difficult, both technically and because you must find unused frequencies.

</comment></comment><comment><USER29> I'm a bit disappointed that they didn't take this as a chance to try out cobbling up a "radiation box" and put commercial equipment in that. Surely, that would be a good experiment data point that could be used in future endeavors.

<comment><USER30> It sounds like they already tried something like that on ISS, and the commercial cameras still had problems.

</comment></comment><comment><USER31> just curious, why they are releasing only black and white pictures

<comment><USER32> The sensor is black and white. This lets them collect the most accurate information about how much light hits each part of the sensor. To take a color image they put a color filter in front of the lens, then they can recolor the black and white image according to the color of the filter. If they take multiple pictures with different filters, they can reconstruct a color image. <http://areo.info/mer/>

<comment><USER5> Actually, the MastCams on Curiosity are bayer-pattern color sensors.

<comment><USER32> Wow, I had no idea. Thanks for the info. [http://www.nasa.gov/mission_pages/msl/multimedia/pia15109.ht...](http://www.nasa.gov/mission_pages/msl/multimedia/pia15109.html)

</comment></comment></comment></comment><comment><USER33> As for the 3D factor, given the way the Curiosity can move, can't it just snap one image, move slightly to the side and snap it again for stereo LR?

</comment><comment><USER34> Why not change the system so that new technology is able to be fitted?

<comment><USER35> The limiting factor is the testing and integration, not the new technology.

if it takes X years to do sufficient radiation and integration testing on new technology, you can't possibly include any technology "newer" than X years old. And trying to keep yourself open to include the latest-technology-you- possibly-can means you need to have manpower ready to do that testing at the last possible moment, which has scheduling implications on everything else.

So you need to prioritize which technologies you most want to be able to integrate in their latest/greatest form. And slightly fancier pictures are just not going to be that high on the list.

</comment></comment><comment><USER36> Please. The quality of images from the MastCam will be sufficient for the plebs.

</comment><comment><USER37> Now that I find out that the main reason for using a 2 MP camera is because that's what the specifications were in 2004, I'm a lot more disappointed than if it was just about the speed of transfer between Mars and Earth.

2 MP cameras were in high-end phones in 2004, but you'd expect _NASA_ to use something a little more advanced than what was available in phones in 2004. How much more would it have cost them to use a 5 MP one? $100 more if they chose one in 2004 and stuck with it, or $10 more if the camera was added just a year ago. So this makes me think that they just didn't think this would be such an important factor, compared to say making the robot work.

<comment><USER38> What do you think extra megapixels would give you? Judging about quality of the camera by pixels count is silly.

<comment><USER39> In a mission to gather data then clearly extra megapixels give you extra data. You cannot argue against that! We aren't talking amateur photography here where the quality of the shot is important and a decent lens beats higher megapixels, we are talking acquiring measurements of the amount of photons in a discrete spatial region. Clearly a higher resolution of sensor might allow scientists to see something not visible in lower resolution images. Although I do admit that in this case I understand the choice due to specification and testing, but to suggest that extra megapixels do not give more information is silly.

<comment><USER40> Actually, the opposite is true. Photon absorption/detection is a quantum event, and limited by probability. For a given sensor chip size (and technology generation), fewer, larger sensels are going to provide samples that are statistically closer to the Absolute Truth. (Averaging repeated samples will reduce the error further.)

Using a well-corrected lens of an appropriately longer focal length, and thus a narrower field of view, with or without panoramic stitching, will provide _at least_ [1] the same linear resolution of a given subject, but with less sampling error.

[1] _At least_ , since apochromatic correction is easier in longer focal length lenses provided that no super-wide-aperture bokeh heroics have gone into the design. Rectiliearity (the absence of barrel or pincushion distortion) is also easier to achieve. Flare can be reduced without inducing undue mechanical vignetting, increasing contrast.</comment></comment></comment></comment>