Artificial Intelligence Generates Christmas Song from Holiday Image - <AUTHOR> https://news.developer.nvidia.com/artificial-intelligence-generates-christmas-song-from-holiday-image/ <comment><USER2> The melody is all over the place, and the rhythm is hard to tap a foot to, but one thing is certain: it's absolutely convinced that Christmas trees get decorated with flowers. Lots and lots and lots of them.

<comment><USER3> I could have mistaken it for a track off The Shaggs'[0] lost Christmas album. I like to believe Frank Zappa would have liked this, too.

[0] [https://en.wikipedia.org/wiki/The_Shaggs](https://en.wikipedia.org/wiki/The_Shaggs)

<comment><USER2> Oh man, The Shaggs are something else.

And it just so happens that I live two towns from Fremont, NH, where they're from.

</comment></comment></comment><comment><USER4> One of the cofounders of the Echonest (acquired by Spotify) created this back in 2004:

        "A Singular Christmas" was composed and rendered in 2004. It is the automatic statistical distillation of hundreds of Christmas songs; the 16-song answer to the question asked of a bank of computers: "What is Christmas Music, really?"

[https://soundcloud.com/bwhitman/sets/a-singular- christmas](https://soundcloud.com/bwhitman/sets/a-singular-christmas)

<comment><USER5> Very interesting reference! Deep learning is statistical, so this is sorta one of the spiritual predecessors.

<comment><USER6> By my understanding, yes, in the sense that certain parts of each algorithm are looking to minimize something...the former model used principal components analysis, which is linear in the sense that you are using transforms which pick out for the least correlated pieces of a huge chunk of data, whereas neural networks, which uses a combination of linear and non-linear layers picked by the user to minimize, "errors."

What's interesting is that the former model sounds so much, "better." I wonder if anyone could chime in about how our ears and auditory nerves or perhaps auditory cognition works, and whether they are more, "principal component analysis-y" somehow than "error minimization-y" or something relating to the actual math, which may explain why this new neural network christmas song sounds like absolute crap to us, whereas the older version sounds pretty amazing. Also, whether my understanding of the underlying math is correct or not.

</comment></comment></comment><comment><USER7> Is it just me or it really sounds like a randomly generated chord progression that barely makes any musical sense?

<comment><USER8> "Musical sense" is a pretty subjective thing, which I suspect is one of the primary issues with artificial creativity.

That said, assuming training data comes from music with progressions that could be broadly classified as 'popular music,' you would expect to find some regression to the mean with more production and deeper training data. (to abuse a phrase)

One other issue that I think will come up is how insular and unevolving artificial creativity will be if it's based on present music for training data. What has historically moved creative trends is disruption; sometimes it's a slow burn and sometimes it's a few catalysts, but experimentation in artificial creativity will be hard to come by early on but quickly needed if it's to supplant human creativity.

<comment><USER9> The statistical approach is painfully naive and doesn't work - as is obvious from the example.

It's like feeding a net with the complete works of Shakespeare and expecting it to produce a genius-level original play. It's simply not going to happen.

<comment><USER8> The issue is not with the statistical but with the parameters around the output and the organization of training data.

<comment><USER10> I think that your assumption is that the genius of Shakespeare's plays can be statistically reproduced through sufficiently-clever organisation. That is not obviously true to me.

Some things are just art, capable of being truly understood only by a creature with a head and heart, arms & legs, love & hate, emotions, experiences — in short, a man.

<comment><USER8> Art is about perception not creation. If autonomously created art produces the same perception and evocative response, it sufficiently passes that test.

An autonomous agent doesn't need to understand the underlying emotion, it just needs to mimic it.

A autonomous car doesn't know why it shouldn't hit a child that jumps into its path, or why it is making any decisions at all despite those being some of the most important and fundamental to humans on earth. It just needs to reproduce the actions of a human that _does_ understand those things.

Yes, I believe that artificial creativity will produce art will be indistinguishable from that made by humans.

</comment></comment></comment></comment></comment></comment><comment><USER11> It kind of feels like a five-year-old trying to make a song. Which seems good - now they only need to improve the mechanisms and who know, maybe we'll get to the point when it's a ten-year-old?

<comment><USER12> Instead of five, don't you mean "two to three"? And under that comparison, isn't it scary how much it does sound that way? Like a kid who hears words together but don't know what they mean yet? (and doesn't really get the cultural things around it.) To me it sounds like a quite musical 2-3 year old stringing words together. Doesn't it strike anyone else that way?

These things are going to grow up very, very soon! We know that. It's scary.

You're watching a two-year old. It's interesting to think about what it will be like when it's ten, sure.

But what will really blow your mind is what it'll be like when it's 23. This is happening right now, before our eyes.

I cannot overemphasize that any large server farm at Google or Amazon is doing more, and much faster, processing than a human brain's neural net. Human brain is 86 billion neurons with an average of 7,000 synaptic connections each. That is a huge number. But they are firing at 15-300 herz (because they're at biological speed, instead of lightspeed like our CPU's) - which is 7 orders of magnitude slower than our silocon. Our brain is about 3 pounds (1,300-1,400 g) and uses some 20 watts.

It's not a question of "if" a server farm will have as powerful neural nets. It's a question of "when".

(Also although we won't be using it, the entire source code for the human brain has to be strictly less than 700 MB, because the fully sequenced human genome which obviously encodes the full human mind is less than 700 MB uncompressed.)

Guys, we are at an incredible pivot point in human history. We are coming up with computerized brains with in some ways comparable architecture to humans, and they are doing human activities.

Today, in 2016, there are thousands, perhaps tens of thousands, of server rooms all over the world that have more than enough computational power to do in real-time what a human adult brain does in real time - but we lack the software.

when we see advances like this in artificial intelligence, this is scary.

We're all but looking at the intellectual output of a two year-old in the field of music.

every single day AI results are astounding. this is it.

<comment><USER11> > the entire source code for the human brain has to be strictly less than 700 > MB, because the fully sequenced human genome which obviously encodes the > full human mind is less than 700 MB uncompressed

Actually those 700MB are compressed, in a way so sophisticated that we don't really know how to uncompress it yet - or whether it's even possible without the external resources our planet provides. And keep in mind that those 700MB only describe how to prepare the basic concept of the brain, whose memory is then packed with information we get from the culture.

<comment><USER12> What I mean by uncompressed is that you're reading the cytosine (C), guanine (G), adenine (A), or thymine (T) as being two bits per base pair (1 of 4 possibilities). There are 3 billion base pairs, which is 6 billion bits, divided by 8 to get bytes you get 750 MB.

The human genome is somewhat redundant and can be further compressed. That is just the string of "ones and zeros" (ACGT) could be run through whatever compression algorithm you wanted.

But don't take my word for it:

>"When the 4 bases are packed into one byte ( .2bit format) the size is 770M (hg18.2bit) , but you'll need an extra tool to decypher the data." [1]

2.

You raise an important point:

>And keep in mind that those 700MB only describe how to prepare the basic concept of the brain, whose memory is then packed with information we get from the culture.

Yes, absolutely. I simply called it an upper bound on how complex a brain's architecture could be. DNA obviously encodes the brain's architecture, since humans all have human brains. Beyond that, there is a very large variation in people's mental capabilities and brains, and the largest variation of all comes from culture.

But culture could be given to a virtualized brain (called training).

Bear in mind that when human brains receive culture, it takes them years of all-day training before they're even able to speak. So full 1x human brains take a long time to train.

When you see results out of neural nets that are similar to what very young toddlers can do, you should be awed. We have the computational power in server farms to do what full brains do -- if not now, then soon.

This isn't some sci-fi pipe dream. Go ahead and look at the facts.

[1] [https://www.biostars.org/p/5514/](https://www.biostars.org/p/5514/)

<comment><USER13> I think that your "700MB uncompressed" fails to take into account that the construction, development, and maturation of the brain relies heavily on cellular and molecular mechanisms. I think it is a little disingenuous to hide the enormous wealth of information necessary to create a brain, much less understand and utilize one, inside of your compiler.

<comment><USER12> I don't think you are correct from an information computation point of view. When looking at the computation done by neurons in the brain it is sufficient to abstract away the lower-level substrate in which it occurs.

You and other posters are all correct regarding the huge volume of information on which human minds are trained. it's hardly unsupervised learning either :)

\</comment></comment></comment></comment></comment></comment><comment>---

EDIT: In response to your comment, I've given it further reflection. DNA as source code may be misleading as an "upper bounds". After all, suppose we knew for a fact (assume there were a mathematical proof or anyway just assume axiomatically) that a one hundred megabyte source code file completely described (contained every physical law etc) a deterministic Universe, and that if you ran it on enough computation to fully describe a hundred billion galaxies with a hundred billion stars each, one of those stars would have a planet and that planet would contain humans and the humans at some point in the simulation would deduce the same one hundred megabyte source code file for their Universe. (This is a bit of a stretch as it's not possible to deduce the laws of the universe rigorously.)

Anyway under that assumption in a way you could argue that the "upper bounds" on the amount of entropy that it takes to produce human intelligence is "just" a hundred megabytes, since that source code can deterministically model the entire Universe. But practically that is useless, and the humans in that simulation would have to do something quite different from modeling their universe, if they wanted to come up with AI to help them with whatever computational tasks they wanted to do.

In the same way, perhaps DNA is a red herring, as there are a vast number of cells in the human body (as in, tens of trillions) doing an incredible amount of work. So starting out with DNA is the "wrong level" of emulation, just as starting out with a 100 MB source code file for the universe would be the "wrong level" of emulation, even if we posit axiomatically that it fully describes our entire Universe from the big bang through intelligent humans.

So I will concede that it is misleading.

All that said, I think that emulating or considering the computation on the level of neurons is sufficient - so it is sufficient to look at how many neurons are in the human brain and the kind of connections they have.

As for the efficacy of this approach - that's the very thing that is being shown in the story we're replying to and many places elsewhere. It works. We're getting extremely strong results, that in some cases beat humans.

I believe that emulating or comparing to humans at the neural level should probably be sufficient for extremely strong results. We do not need to emulate every atom or anything like that. I consider it out of the question that we would discover that human minds form a biochemical pathway into another ethereal soul-plane and connect with our souls in a way that you can't emulate by emulating neurons and the like, and that the souls are where intellect happens and brains are just like "radio antennas" for them. Instead, I think that the approaches we're seeing will achieve in many ways similar results to what humans brains produce computationally - a much higher level of abstraction is sufficient for the results that are sought.

<comment><USER13> I will confess to not being an expert, but I disagree: I don't think it's sufficient to abstract away the lower-level substrate when the OP was referring to DNA as source code, which absolutely depends on that level of detail to both construct the system (the brain) and to enable the continued development and maturation of that system (a physical, real brain).

I was not referring to the huge volume of information necessary, as I acknowledge that as being "outside the system" for purposes of this discussion, so my apologies for any confusion I might have caused.

It may be possible that (and it is my belief that) there is a higher-level abstraction for the computations taking place in the brain, even if it is on the neuron-level, but at that point I don't think you can claim that the source code for that is going to fit under 700MB by using DNA as a baseline.

<comment><USER12> you are right, I went too far. see my other comment:

[https://news.ycombinator.com/item?id=13081551](https://news.ycombinator.com/item?id=13081551)

however, that just concerns the DNA argument. We know roughly how many neurons there are in humans and their connectedness.

</comment></comment></comment><comment><USER14> I've seen more and more little A.I.-generated ditties like this recently and their reception tends to be the same: that they're interesting and funny but don't sound that great.

The output would probably be more compelling if A.I. were adopted more as an instrument by individual artists/composers to automate some of their more tedious tasks by learning their own particular styles rather than a magical music box that churns out top hits.

</comment><comment><USER15> "The best Christmas present in the world is a blessing."

This algorithm is throwing down some wisdom

<comment><USER16> I think generating fortune cookies is a really low hanging fruit for current AI. Someone could put it together in a week-end.

<comment><USER17> I once developed a hierarchical Markov chain, and I decided to use it to generate fortune cookie wisdom because I thought people would be more willing to overlook grammatical mistakes or be willing to interpret it as an expression of deep truth. [http://www.kylem.net/stuff/fortunes.html](http://www.kylem.net/stuff/fortunes.html) (You might need to increase the "sense" parameter.)

<comment><USER18> "The supreme happiness in life is simply to serve as a warning to others."

Oh my god, I'm in stitches. What a fun project, thanks for sharing!

</comment></comment></comment></comment><comment><USER19> When I think that this month I'm going to hear "Last Christmas" by Wham! in every store I set foot into, this new masterpiece doesn't sound so bad.

</comment><comment><USER20> I am pretty amazed at the effort that nVidia is putting into its corporate rebranding effort. I wonder if, in the not too distant future, they will be the AI company that also makes Graphics cards sometimes.

The other thing I find really amazing about it, coming from IBM, is that IBM has invested a ton of money in IBM Watson but they sold off their foundry business (could have made massively parallel AI machines) and their systems business is a fraction of what it was.

Looking at what can be done when you're leading versus when you are following is really sobering to me.

</comment><comment><USER21> I'm not sure what I should be impressed by. Maybe there's some real technical feat happening here, but I feel like a basic mad-libs style algorithm could produce something better.

<comment><USER5> I'm not very familiar with mad-libs so correct me if I was wrong. I think generating a lyrics passage (zero hard-coded rule on content or grammar or anything) from an image would not be something you can do with mad-libs.

</comment></comment><comment><USER22> One step closer to GLaDOS.

<comment><USER23> this was a triumph! im making a note here: huge success

<comment><USER22> It's hard to overstate my satisfaction.

</comment></comment></comment><comment><USER24> it is scary, sounds like GLAdOS singing "still alive"!

<comment><USER25> It sounds more like a corrupt core.

"Cave here. It's Christmas time, and you know what that means: Christmas bonuses have been suspended until further notice. We've gotta pay the judgement on that pesky class-action with _something_. But don't let that get you out of the Christmas spirit. The lab boys have come up with a way to stay festive by hooking up the Christmas Core to the lab's PA system. So enjoy free, continuous, computer-generated Christmas music from now until January 5!"

"Cave again. Apparently the Christmas music has been causing some employees severe emotional and psychological distress. We've had reports of people sticking their heads into active particle accelerators and drinking Repulsion Gel to get away from the sound. So until a full investigation has been conducted and the Christmas Core thoroughly debugged, we are discontinuing the Christmas music. We do _not_ need another class-action on our hands, folks."

</comment></comment><comment><USER26> Music for people who hate music.

</comment><comment><USER27> Auto-synthesis of music has been a topic of academic interest since the 1950s, when the first mainframe scribbled out code on paper tape to be translated into sheet music and performed. UToronto's work here is the latest expression of this desire.

The huge gap between our cultures' actual music and these synthetic projects can to an extent be described through "receptivity" or the phenomenology of music, in other words, _how_ it's experienced. The following fun, short talk does a great job of introducing the concept of through its analysis of "vaporwave."

[https://www.youtube.com/watch?v=QdVEez20X_s](https://www.youtube.com/watch?v=QdVEez20X_s)

<comment><USER28> That video is fantastic I watched it a week or so ago.

Your explanation also explains why computer performed music is so off. It still has that uncanny valley effect. So when Sony had a computer generate a "Beatles-esque pop song", they still had a human perform and produce it. But at the point there's so much creativity and human-added value on top of it that I don't think its fair to call it computer generated imho.

<comment><USER27> yes. I can tell you a little more about that, too, since I used to research this stuff and think about it a lot still.

One of my models of music is an external model of a regulated system that parallels and trains our own habits and responses. E.g. a song demonstrates tension and release similar to our own lives. The level of tension in a song before release occurs can inform us how much tension which should accept before performing some release activity.

Music's rhythms also inform the pace of our work. E.g. verse-chorus-verse represents switching between two different activities. Even the pitch of a single note acts as a reference for the amount of intensity of a sensation we should use in our own lives. E.g. thrash metal listeners enjoy sudden shifts into massive intensity and hold it there. Dub step listeners are training themselves for unusual, but rather intense aesthetics leading up to disproportionate release. Classical music tends to be for "long-chain thinkers" tumbling ideas over from various perspectives, e.g. writers and politicans, doctors, not factory workers.

With that as a background, consider that a live instrument is also a physical system with a human controlling it interactively. The live system is a bit different every time. Here's the critical part: _the human must listen and provide instantaneous feedback to a varying system in order to present the piece of music as a proper response model of a regulated system_. If the player fails to do this, the model communicated by the performance is different.

In open-loop systems, such as a sequencer, there is no (or limited) interaction between the player and the sound, so an incidental model emerges. That incidental model represents an _unintended and therefore most likely irrelevant model of how to interact with reality._ e.g. it relieves tension where no relief was needed. It lingers too long on an idea, long after a human novelty-seeking circuit has starved.

Some people, e.g. in discussions of unstable filters like the TB-303, chalk up the variations as being different at every performance because the instrument is random... However, they're missing the closed loop portion of the performance, in which _the performer reacts to the unpredictability of the instrument in order to maintain the model._ In other words, the score and notes are not the music, but the performer's response to the environment the score sets up is the music.

To revivify your uncanny valley observation, the "unstable filter creates variations" crowd has a parallel in Perlin noise used to subtly animate human models to make them not look so dead. However, it's incomplete because they don't use (short-term) feedback to determine when the movement suffices to be convincing. That feedback is the essence of performance.

In theory, computer scientists could implement these feedback models in performance to make the sounds more realistic. They could be used in synthesis, but the playback would still require observation of the listener! Which is possible. Personally, I just prefer playing electronic instruments live over using sequencers. It's only the sounds of electronic music I like, the zaps, peowms, zizzes, pews, and poonshes, etc. I don't care for electronics/computers to perform for me.

If you like this hypothesis, you can find more references on my wiki at: [http://www.diydsp.com/index.php?title=Computer_Music_Isolati...](http://www.diydsp.com/index.php?title=Computer_Music_Isolation)

</comment></comment></comment><comment><USER29> Did I just hear a new classic?

<comment><USER30> No.

</comment></comment><comment><USER31> This is definitely now my favorite Christmas song. While obviously not a masterpiece, it's incredible how far this tech has come. It's almost got a Dadaist feel to it. Can't wait to see where this ends up in ten years! I can foresee music labels buying a few of these AI's, getting some pretty people with decent voices and sending them on tour.

<comment><USER32> Is pop music not cheap and disposable enough for you already?

<comment><USER9> Pop is still recognisably human. This sounds like logic gates roasting by an open fire.

<comment><USER32> That was a brilliantly seasonal analogy.

</comment></comment></comment></comment><comment><USER33> While this is hilarious, it doesn't seem like a huge achievement to me.

The only thing (kind of) working well is feature/topic detection in the image (tree, christmas etc.), but that isn't really cutting edge.

The core part, learning and creating music, only produced melody and lyrics that seem not much different from accumulating random sentence and chordal fragments.

</comment><comment><USER32> I'm not sure that Christmas songs generally use the blues scale, I wonder what made them choose that for the melody?

<comment><USER34> I suspect it's an easy way to get something that won't sound completely dissonant, especially because you can use the same blues scale over multiple chords.

<comment><USER32> I can see the logic behind that, but without any sort of tension/release between the melody and chords, it still sounds just as dissonant to my ears.

<comment><USER34> Granted, it will never win any awards. Part of the choice behind it might have been the desire to "ship" it before Christmas.

</comment></comment></comment></comment><comment><USER35> It brings this recent story to mind: [https://news.ycombinator.com/item?id=13033299](https://news.ycombinator.com/item?id=13033299)

</comment><comment><USER36> Better than The Christmas Shoes.

</comment><comment><USER37> This is not even a proper song.

</comment><comment><USER38> We have a very long way to go it seems. That was almost nonsensical.

</comment><comment><USER39> This was a triumph!

I'm making a note here:

Huge success!

</comment><comment><USER40> Chilling...

</comment><comment><USER41> Show me the same algorithm generate songs in a different genre from different images and I'll be impressed.

<comment><USER5> I'm from the project team. This is a very interesting point.

While it is easy to crawl many songs from the internet, it is a little harder to gather the same amount but with proper genre/style/etc labels, although it is not impossible.

For now there's only one genre, which we call it "the genre of whatever is on the internet". So whatever music files on there, many of them quite "crappy", were used to train the model. Also there are many other problems on how to better structure and flavor the composition.

This is just a very early-stage attempt, as a CS student's fun side project. We are working with people with real musical talent now and hoping to make better songs in the next version.

<comment><USER41> I mean, where does the Christmas element comes from? The image alone, the music it was trained with, or is it somehow hardcoded in the algorithm?

<comment><USER5> The Christmas element comes from 1. the image, and 2. a 4800-dimensional RNN sentence encoding bias generated from ~30 Christmas songs.

Not sure how to hardcode this.

<comment><USER32> I'm interested that you used some Christmas songs as training (which wasn't obvious from what I read of the paper). Were they pop songs, traditional, or a mix?

Further to my comment up there[0] - and I don't wish to sound a grinch because this is a really cool project - but would I be right in thinking you spent more time on the image description than the music?

I saw that you specify a scale for the melody, would it be either possible to use a mode to generate the accompaniment around, so that the melody can move diatonically and risk too many clashes, or to allow the melody to follow the chord sequence somehow?

Again, sorry if I sound too critical. It's a really awesome thing you've done, and I'm just a guy that listens to the music instead of the lyrics.

[0] [https://news.ycombinator.com/item?id=13079355](https://news.ycombinator.com/item?id=13079355)

<comment><USER5> Thanks for the comments! Are you asking the lyrics or music generation?

For lyrics, we actually didn't train on Christmas songs. Training data was a large collection of romance novels. (See neural-storyteller by Jamie Kiros). The "Christmas trick" we did was applying a "style shifting" after image captioning and before lyrics generation, where the shifting vector was obtained from ~30 Christmas songs.

For the music generation. Although we are aware of some basic music performing rules, such as melody following chord etc, we actually didn't add this kind of rules.

For the blues scale here's the thing. I didn't really know much about music, so I spent several hours reading things like basicmusictheory.com. It happened to introduce blues so we just used it. But you're right on the relevance between blues and pop: only a very small percentage in our pop music collection is blues, after we ran the scale-checking code.

<comment><USER32> Thanks for the reply! I was concentrating on the music specifically. I thought the lyrics generation was really enjoyable.

I was asking more if you'd used any traditional carols, as they can have a more definitively "christmassy" sound than a pop song with sleighbells laid over the top.

Overall I meant that I think the music would be more convincing either following the chords in the melody, or sticking to a single mode for both melody and accompaniment.</comment></comment></comment></comment></comment></comment></comment>


GoDaddy is about to buy Nodejitsu - <AUTHOR> http://venturebeat.com/2015/02/09/godaddy-nodejitsu

<comment><USER2> The notion that this move would buy them any credibility with developers is completely laughable.

Why should I trust a company with my hosting who, last time I used them, waited less than a day after expiration to sell my almost completely unknown, personal domain to a squatter?

<comment><USER3> I remember some articles that showed connection of those squatters to GoDaddy and wasn't that surprising really.

</comment></comment><comment><USER4> I used nodejitsu (paid account) for 24 hours. They screwed up their DNS and was sending my client's domain to a random person's node app (and vice versa).

That's when I stopped trusting other people to handle my critical infrastructure.

</comment><comment><USER5> never felt the need to use Nodejitsu, definitely won't now.</comment>


4D toys - <AUTHOR> http://4dtoys.com/ <comment><USER2> Here's a thought I had watching the bit with 2D man:

Actually, the 3D view that 2D man does not understand but which we do understand is... still 2D. My screen is flat.

You can use a 2D viewport to render a 3D scene in a way that is natural and easy to understand for us humans: A human watching the 2D scene can very quickly surmise from a glance at the viewport: Which objects are in the scene, and where are they located, in _ALL_ 3 dimensions?

This raises the question:

Can you render a 4D scene onto a 3D viewport such that us humans are pretty good at understanding where every object in the scene is, in all 4 dimensions?

I assume the answer is 'yeah, you can do that'. I wonder what that would look like.

It's complicated of course; our eyeballs are involved and they kinda work in 2D and not 3D; where we humans can casually glance at a 2D viewport rendering a 3D scene for a second and know what's happening, we'd have to walk around the 3D screen rendering a 4D scene in order to even see everything.

<comment><USER3> The problem is that while everything you see on your (2D) monitor is effectively 2D our brain is good at taking 3D cues (shadows, perspective, parallax etc...) to reconstitute 3D "data" from it. For instance look at 3D rotating cube on your monitor: [https://www.youtube.com/watch?v=5E10zYKln3g](https://www.youtube.com/watch?v=5E10zYKln3g)

Even though there's no shading, a bad resolution and zero context anchoring it in a familiar setting you immediately see it as a 3D object when effectively that's just 3 2D quadrilaterals changing colors and shape. We really need very little to be able to extrapolate depth information from a very limited 2D-only display.

If you did that with 4D cues in 3D space (say in VR or something) I think it wouldn't work. It'd just look like a 3D object changing shape, not a projection of a higher dimension (the same way an hypothetical 2D being would see 3 quadrilaterals changing shape in the above video, not a solid 3D object that would be an abstract concept to them, not a familiar reality).

Could we teach our brain to "think in 4D" and interpret these cues differently? To a certain extent probably, maybe if you spent hours and hours and hours in a VR simulation with 4D objects you'd start getting a feel for it. I doubt you'd ever get as good as with 3D objects since we've been exposed to these literally every waking moment since birth, but maybe I underestimate the plasticity of our brain.

That actually leads me to an other question: is 3D somewhat hardcoded in our brains or is it purely learned? If we were mad scientists and took a baby brain into a 4D "plato's cave" style simulation, could it grow into being able to perceive 4D as intuitively and effectively as we do with 3D space? Also unrelated question: does anybody have a baby I could borrow?

<comment><USER4> > Could we teach our brain to "think in 4D" and interpret these cues > differently? To a certain extent probably, maybe if you spent hours and > hours and hours in a VR simulation with 4D objects you'd starting getting a > feel for it. I doubt you'd ever get as good as with 3D objects since we've > been exposed to these literally every waking moment since we're born.*

Good lord! Has anyone tried?

This reminds me of the backwards bicycle video on youtube ([https://www.youtube.com/watch?v=MFzDaBzBlL0](https://www.youtube.com/watch?v=MFzDaBzBlL0)). I... really want to try what you're suggesting.

<comment><USER5> In the YouTube video he says:

"So here's why I did. It was a personal challenge. I stayed out here in this driveway and I practiced about 5 minutes every day. ... after 8 month it happened ... in two weeks he (the son) did something that took me 8 months to do".

Well, if you were serious about learning a skill you wouldn't just do one 5 minute session per day for eight months.

A more appropriate training regime would be way more intensive than one 5 minute session per day. Are we to believe he limited his son to one 5 minute session a day?

Do children really learn languages quicker than adults? By age 3, children will probably have words for almost everything. Babies might even say mama and dada by 6 months of age.

Children have several language learning advantages over adults: complete immersion; it is imperative they learn; effectively unlimited time; no responsibilities.

But, it takes them _years_ to learn their native language to basic competency.

Compare: an English speaker can learn Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese, Romanian, Spanish, or Swedish to _General Professional Proficiency in Speaking and Reading_ in 600 hours carried out over 24 weeks (25hrs per week). Cantonese, Mandarin, Japanese, Korean, or Arabic will take 2200 hours over 88 weeks (25hrs per week).[1]

Is it _easier_ for a child? Yeah probably, they don't even have to look after themselves.

If I could live as a child in a foreign language house in a foreign language city, with only two tasks: learn to speak and read the language to basic competency, and learn to ride a backwards bicycle, I'm firmly of the believe I could out pace the 95th percentile of children at both tasks.

As an aside, he says his son is the closest person to him genetically, but aren't his parents both equally as close to him genetically as his son?

1\. [http://www.effectivelanguagelearning.com/language- guide/lang...](http://www.effectivelanguagelearning.com/language- guide/language-difficulty)

<comment><USER6> Yeap, [https://youtu.be/oI2aMKwXXnE](https://youtu.be/oI2aMKwXXnE) this guy took a more serious attempt at learning to ride the backwards and it seems it didn't take him that much time.

<comment><USER5> Thanks for finding that.

So it only took him about an hour, over three days, to get it worked out well enough to not immediately fall off, and an hour and a half over four days to get to 50 meters.

There's a comment in on the YT video from Destin saying _" the RC helicopter pilot was able to learn it in about an hour but I don't think his brain is normal"_ \- good point, the RC helicopter pilot has more experience understanding reversed input while the helicopter is flying toward him, also he had the opportunity to watch Mike learn first.

I reckon the learning process could be sped up even more by taking the pedals of, turning it in to a balance bike, learning how to ride it down slight inclines, get that sorted then putting the pedals back on.

</comment></comment></comment></comment></comment></comment><comment><USER7> This is an old video [1], but it remains the clearest visual explanation of multiple dimensions that I've ever seen. I really can't recommend it highly enough. I think anyone trying to visualize a 4D object will get something out of watching it.

I played with the 4D toys app after it showed up on /r/math a while ago. I like it and I think it's useful. My only complaint would be that it's a little too open ended. While it's nice to provide a simulated tactile experience of four dimensions, I think the app should provide a bit more visual intuition. That's one of the things I like about this video. ________________

1\. [https://www.youtube.com/watch?v=90olwwLdEYg](https://www.youtube.com/watch?v=90olwwLdEYg)

<comment><USER8> Reminds me of [https://youtu.be/K4JhruinbWc?t=110](https://youtu.be/K4JhruinbWc?t=110)

<comment><USER7> That was really great, thanks for sharing!

</comment></comment></comment><comment><USER9> Wow, cool to see this here. I've been working on reproducing some of it in numpy/matplotlib for the past 2 weeks! I liked their 3D-sliced 120-cell (equivalent of the dodecahedron), and it wasn't too difficult to reproduce.

Step 1: Use Schlafli generator from here [1]. Schlafli numbers are a compact description of regular polytopes, and there is a recursive algorithm to generate vertices, edges, faces, etc. from them. The base case of the recursion is dimension 1, so you make 4 calls to get to dimension 4.

Step 2: Intersect the edges of the polytope with a hyperplane (a 3D subset of 4D).

Step 3: You get a set of 3D points out of step 2. Draw the convex hull of them, which gives you triangles.

Step 4: Render the triangles somehow. I used matplotlib's 3d facilities (mplot3d), and we are working on raytracing them.

Step 5: Animate over different hyperplanes. Take the min and max in the w plane and that will give you non-empty slices. Now you can "see" the 4D polytope using time as the 4th dimension.

I sure he is doing something more advanced (4D collision detection), but this is all we needed to reproduce something that looks kinda cool.

[1] [https://github.com/aruth2/schlafli](https://github.com/aruth2/schlafli)

</comment><comment><USER10> Can someone help me out here?:

I can represent 3D quite comfortably on 2D monitors, can there be an intuitive mapping of 4D to a 3D VR view?

I know 3D mapped to 2D suffers from occlusions and heavily relies on clues like perspective, shadow etc. But given enough time even a less intuitive 4D view could become intuitive with time, too.

edit: found this: [https://youtu.be/S-yRYmdsnGs?t=252](https://youtu.be/S-yRYmdsnGs?t=252)

even better: [https://youtu.be/dy_MUfBuq2I](https://youtu.be/dy_MUfBuq2I) (turn on subtitles)

<comment><USER11> What would be the benefits of the 3D VR? There will always be a 2D bottleneck on your eyes' retina. The most important perspective clue I would say is temporal sampling. For dynamical systems, Takens theorem says that a high dimensional topology can be restituted by the sequential sampling of a single variable alone. It seems likely to me that our perception generally works by this principle. That being said, I think we are quite heavily hardwired to 3D perception. I suspect these circuits cannot be wholly overridden.

<comment><USER12> What would a 3D eye even be like? Is it something like having independent depth perception per eye?

<comment><USER11> Photons travel in 3D space and may only be occluded by 2D surfaces. There is no getting around that fundamental aspect of reality. In this universe.

<comment><USER13> This is why as an armchair physicist I'm excited about the new neutrino observations and research going on, mainly because neutrinos aren't affected by gravitational lensing and other effects other particles encounter as they 'roll around' on space-time. Would it be possible to understand conceptually how other dimensions extend out of space time? I'm imagining a Y-axis perpendicular to the space-time fabric....

<comment><USER14> Gravitational lensing is a distortion of spacetime. It affects all particles, including neutrinos.

</comment></comment></comment></comment></comment></comment><comment><USER15> There's also Miegakure, another beautiful game based on 4D puzzles.

[http://miegakure.com/](http://miegakure.com/)

<comment><USER16> I've been waiting for that game for more than 5 years now I think.. It'll probably stay in development for another 2 years or so.

<comment><USER17> It's getting _really_ close, though. Puzzles, mechanics, dialogue, effects, all finished. At this point, according to the dev, all that's really left for him to do is "small things like fix collision bugs, and I will keep placing props in levels and program the occasional cool 4D thing. We still need a bunch of 3D modelling done."

I'd give a 50% shot it lands on Steam before the end of 2019. (35% it doesn't make 2019 but lands before the end of 2020, remainder that it lands later or never releases.)

<comment> GuB-42 90% done. It means we are halfway there :p

My gut feeling is that we are not there yet, and that 4D toys is an attempt by the author to monetize his development tools in order to be able to complete the main project. I hope it turns out well, Miegakure is definitely in my watch list.

</comment></comment></comment></comment><comment><USER18> Kinda reminds me of "Flatland": [https://en.wikipedia.org/wiki/Flatland](https://en.wikipedia.org/wiki/Flatland)

<comment><USER19> This book is incredible, it's hard to believe it's 134 years old.

</comment></comment><comment><USER20> "To deal with hyper-planes in a 14-dimensional space, visualize a 3-D space and say 'fourteen' to yourself very loudly. Everyone does it." – Geoffrey Hinton, A geometrical view of perceptrons, [https://www.coursera.org/lecture/neural- networks/a-geometric...](https://www.coursera.org/lecture/neural- networks/a-geometrical-view-of-perceptrons-6-min-sPEhK)

* but remember that going from 13D to 14-D creates as much extra complexity as going from 2-D to 3-D

<comment><USER21> There is a joke that goes like this: A mathematician and an engineer attend a lecture on the 11-dimensional M-theory. At the end of the lecture the mathematician delighted says how much he enjoyed it. The engineer puzzled asks him how anything even makes sense. The mathematician replies "It's easy. Visualize an N-dimensional space, then let N go to 11."

</comment></comment><comment><USER22> This is amazing and looks really fun! I remember reading a short science fiction story once, "Mimsy Were the Borogoves"[1], where small children were playing with such 4D toys, which were found to be educative in unpredictable ways.

It really makes me want to try it out. I wonder if it's really the same without VR.

[1] [https://en.wikipedia.org/wiki/Mimsy_Were_the_Borogoves](https://en.wikipedia.org/wiki/Mimsy_Were_the_Borogoves) (watch out, there are spoilers here!)

<comment><USER23> the bottom of the website includes an illustrated frame and excerpt from that very story

<comment><USER22> Wow, you're right! Didn't see that, Thanks! :)

</comment></comment></comment><comment><USER24> Discussed last year: [https://news.ycombinator.com/item?id=14471931](https://news.ycombinator.com/item?id=14471931)

</comment><comment><USER25> If anyone wants to play with a 4D toy right now in their web browser, here is an interactive tesseract (4d cube) that I built. [http://transdimensional.xyz](http://transdimensional.xyz) It doesn't have physics but features a novel interface that I designed to help build an intuition about different rotations. (There are six axes of rotation in 4 dimensions)

</comment><comment> DC-3 Using the 4th Dimension, the two interlocking rings can be separated. However, what would be the equivalent puzzle for the 3rd dimension? What 2D system requires the 3rd dimension to be separable? Is there even an equivalent? Clearly a circle within a circle is one such system, but it corresponds to a ball wholly contained by another ball in 3D, not a pair of interlocking rings.

<comment><USER26> It would be a ring with a stick through it. In a 2D slice down the middle, it would look like a donut with a circle (the slice of the stick) in the hole. The circle appears to be stuck inside it, with no way out in 2D. In 3D, you just pull the stick out of the ring.

</comment></comment><comment><USER27> How curious. What would a haptic VR experience be like? Does a 4D object still exert mass in the other 3Ds? Could you _intuit_ the other 3Ds even if they were not visible?

<comment><USER28> The VR version has haptics. It's kinda interesting. I think there is a little bit of intuition to be gained.

</comment></comment><comment><USER29> I would have through the he would have done a projection from 4d space to 2d space similar to how 3d graphics does a projection from 3d to 2d. Similar to [http://christopheremoore.net/4d-renderer/](http://christopheremoore.net/4d-renderer/) It I guess in that case it would be difficult to interact with it in VR.

I guess this slicing technique works but it would be a bit weird.

</comment><comment><USER30> Some might remember this excellent treatment of higher dimensions [1] in which you first manipulate shapes to solve a puzzle in 2D, then 3D, and finally 4D. YYou develop a keen sense of 4D objects:

[1] [http://harmen.vanderwal.eu/hypercube/](http://harmen.vanderwal.eu/hypercube/)

</comment><comment><USER31> Agreeably playing with 4d objects is mindboggling enough, but still I would like to somehow visualize myself moving around in 4d space. I am not quite sure how a 4d room would look like and what kind of doors to other 4d rooms would be like, bit for sure navigating around a 4d house could be fun...

</comment><comment><USER32> The video ends saying that trying to fit a 4D cube in a 4D hole is like a child playing with toy blocks. Hypothetically, if a child was raised with regular VR simulations that allowed the child to manipulate 4D objects, would their brain "learn" it and in a way unlock the 4th dimension?

<comment><USER33> I was wondering this too as I watched the video. As an adult I know how physics works and I know that what I'm seeing is not normal based on decades of experience. If you showed this to a 1 year old and let them play with 4D virtual toys for a few years, would they more intuitively understand 4D objects? Or would it still remain foreign due to the fact that they can't see things moving around the fourth dimension?

<comment><USER34> As I played around with 4D toys, I did start to gain a sense of things that were "Behind" or "In Front" of me in 4D space, that is I could start to feel their presence after they had been pushed out of sight, if I knew in what direction they were moving in the 4th dimension. If I didn't know that I could only feel that they were near.

It helps that there is a little visualization that is just a line-per-object showing where you are, and where all the objects are intersecting the 4th dimension, that you also use to move "back and forth."

One thing I found myself doing was grabbing objects at one of their edges in the 4th dimension, by moving myself to near their boundary, and then using them like brooms. It's really easy to understand with the case of a hypersphere, since at its edge it's just a smaller sphere than at the middle. So you grab that small sphere at the edge, and push in the direction towards its middle in the 4th dimension, and it will act like a bowling ball. You won't see the stuff you are pushing around because the sphere is "ahead" of you, unless they roll around the sphere, then you'll pass them. Once you reach the edge of the 4th dimension, all the stuff you kept pushing will be there.

Predicting how 3D intersections change as objects rotate about in the 4th dimension still seems like chaos to me though, except in the case of hyperspheres, which basically don't change as they rotate, but I only played around for about an hour or so. The only way I found to rotate objects in the 4th dimension was to have them collide with each other, or the walls and floor, which makes it kind of hard to carefully experiment with their rotations.

</comment></comment></comment><comment><USER35> "Goundbreaking 4D+Time Physics Engine that uses new mathematics created for this project."

New mathematics no less!

</comment><comment><USER36> This passage from Death's End by Cixin Liu really gave me pause to stop and wonder about what the experience of seeing extra dimensions might be like (here translated to English by Ken Liu): \--

A person looking back upon the three-dimensional world from four-dimensional space for the first time realized this right away: He had never seen the world while he was in it. If the three-dimensional world were likened to a picture, all he had seen before was just a narrow view from the side: a line. Only from four-dimensional space could he see the picture as a whole. He would describe it this way: Nothing blocked whatever was placed behind it. Even the interiors of sealed spaces were laid open. This seemed a simple change, but when the world was displayed this way, the visual effect was utterly stunning. When all barriers and concealments were stripped away, and everything was exposed, the amount of information entering the viewer's eyes was hundreds of millions times greater than when he was in three-dimensional space. The brain could not even process so much information right away.

In Morovich and Guan's eyes, Blue Space was a magnificent, immense painting that had just been unrolled. They could see all the way to the stern, and all the way to the bow; they could see the inside of every cabin and every sealed container in the ship; they could see the liquid flowing through the maze of tubes, and the fiery ball of fusion in the reactor at the stern.... Of course, the rules of perspective remained in operation, and objects far away appeared indistinct, but everything was visible.

Given this description, those who had never experienced four-dimensional space might get the wrong impression that they were seeing everything "through" the hull. But no, they were not seeing "through" anything. Everything was laid out in the open, just like when we look at a circle drawn on a piece of paper, we can see the inside of the circle without looking "through" anything. This kind of openness extended to every level, and the hardest part was describing how it applied to solid objects. One could see the interior of solids, such as the bulkheads or a piece of metal or a rock—one could see all the cross sections at once! Morovich and Guan were drowning in a sea of information—all the details of the universe were gathered around them and fighting for their attention in vivid colors.

Morovich and Guan had to learn to deal with an entirely novel visual phenomenon: unlimited details. In three-dimensional space, the human visual system dealt with limited details. No matter how complicated the environment or the object, the visible elements were limited. Given enough time, it was always possible to take in most of the details one by one. But when one viewed the three-dimensional world from four-dimensional space, all concealed and hidden details were revealed simultaneously, since three-dimensional objects were laid open at every level. Take a sealed container as an example: One could see not only what was inside, but also the interiors of the objects inside. This boundless disclosure and exposure led to the unlimited details on display.

Everything in the ship lay exposed before Morovich and Guan, but even when observing some specific object, such as a cup or a pen, they saw infinite details, and the information received by their visual systems was incalculable. Even a lifetime would not be enough to take in the shape of any one of these objects in four-dimensional space. When an object was revealed at all levels in four-dimensional space, it created in the viewer a vertigo- inducing sensation of depth, like a set of Russian nesting dolls that went on without end. Bounded in a nutshell but counting oneself a king of infinite space was no longer merely a metaphor.

<comment><USER12> Wow, if this is how it's like to see three dimensions then where would one be able to hide information to the viewer? Probably another dimension. By moving an object from one location to another the information about the position in the previous states are lost. And information about the future states remains hidden. Alas, present states conceal everything hidden behind them, or ahead of them.

</comment></comment><comment><USER37> This is incredibly cool. I feel this would be super useful for teaching children (like my nephew who's 11-years-old) to think about the fourth dimension as a component of the fabric of space instead of time as it's usually understood.

</comment><comment><USER38> I'd love to know the math behind the physics engine, particularly angular momentum. Planar rotations are weird in 4d.

</comment><comment><USER39> What tools were used to build this I wonder?

<comment><USER17> Part of it is that the developer's spent the last decade building a game engine and toolchain for 4D content while developing his 4D puzzle game Miegakure ([http://miegakure.com/](http://miegakure.com/))

<comment><USER39> That looks incredible. Do you know if the developer has published any papers or code on the subject?

</comment></comment></comment><comment><USER40> This is the best visualization of 4 dimensions I have ever seen.

</comment><comment><USER41> Can't wait to see the 5D version. But seriously, what's the use of that? I don't get it.

<comment><USER15> It's meant to give an intuition about what a 4th space dimension might feel like, thru self experimentation.

</comment></comment><comment><USER42> Can this explain some of the quantum weirdness? Like some of the things quantum entanglement have is due to a forth dimension we are not aware of. Might have to do with general relativity and it is the 4th dimension in smaller scales.

Now where is my noble price?</comment>


"i am lonely will anyone speak to me" (2004) - <AUTHOR> http://lounge.moviecodec.com/on-topic/i-am-lonely-will-anyone-speak-to-me-2420/

<comment><AUTHOR> Besides the mere irony of lonely people finding each other by admitting their loneliness to an algorithm, I thought this was an interesting look at how loneliness persists (and is perhaps amplified, some would say) in our ever- more-connected society.

" _I feel so much better that I am not the only one that typed in "I am lonely" on google. How pathetic that I have nothing better to do. It is amazing that I can be so extremely successful at work and so lonely at home. I got married at 17, had four children, completed a masters degree, became the youngest professor at my work, opened my own practice, yet have found myself in a lot of debt with a husband that has worked out of town for almost a year. He visits every once in awhile. I am not sure what to do but I need to do something. I am not getting any younger._ "

This thread also has a Wikipedia article: [http://en.wikipedia.org/wiki/I_am_lonely_will_anyone_speak_t...](http://en.wikipedia.org/wiki/I_am_lonely_will_anyone_speak_to_me)

</comment><comment><USER2> the togetherness of search weighting.</comment>


Twitter Redesigns Signup Process to Entice New Users - <AUTHOR> http://blogs.wsj.com/digits/2014/08/29/twitter-redesigns-signup-process-to-entice-new-users/

<comment><USER2> Admittedly, haven't read the article, but ... really? Given the saturation of Twitter into the public consciousness (heck, my local news anchor/beat reporter incessantly pimps their Twitter handle), I find it hard to believe that a person has not determined whether or not they want to participate in the Twitter community long before they get the sign up page. Seems like a bit of turd polishing to me ...</comment>


Scaling Riak at Kiip - <AUTHOR> http://basho.com/blog/technical/2012/05/25/Scaling-Riak-At-Kiip/

<comment><USER2> It's been a riak-y day for me, so I watched this avidly.

You're no doubt wondering what the issues were; they come 20 minutes in, and were (paraphrased):

* At scale in production, adding a new node took days to complete all the handoffs; they recommend adding new nodes as soon as it's looking like you need them, rather than waiting until you're redlining.

* 2i is slow, especially in EC2; a straight KV "get" is milliseconds-denominated; 2i index queries were taking multiple seconds. Use 2i, they say, but in background processes.

* Javascript MapReduce is slow; this is well known. They confirm Erlang MR was adequate.

* As the LevelDB keyspace grows, there's a stepping function in latency; 5ms, then 15ms, then 25ms; the solution is to add nodes. (LevelDB is Google's KV store, a new option for Riak, required if you're using secondary indexes).

* Riak Control didn't work for them over low-latency connections.

* Once, a Chef misconfiguration left the whole cluster flapping on and off, which corrupted the cluster; they recovered with Basho support. Be careful about adding and removing nodes rapidly.

* Similarly, flapping a single node caused the cluster to get into a state where it wouldn't converge again; the cluster worked but no nodes could be added until they (presumably?) restarted it.

<comment><USER3> After watching the video, the takeaway for me was don't run $database on EC2 instances.

Last night I just finished replacing 6 maxed out medium instances with one $100 box from SoftLayer. :/

<comment><USER4> At that price point, it seems that you can get FAR better specs on Honelive. Kimsufi.ie is even cheaper, if you don't mind your server being hosted in the UK.

edit:

Softlayer: Intel 4x2.40GHz, 2 GB RAM ECC, $160/mo Honelive: Intel i7-2600 16, 16 GB RAM, $120/mo Kimsufi.ie: Intel i7 4x 2(HT)x 2.66+ GHz, 24 GB, $60/mo

<comment><USER5> Has anyone used Kimsufi? Are they any good? I'm at Hetzner right now, but I never found any use for that box, so I'm just paying for it for nothing...

<comment><USER4> I just signed up for kimsufi. They are the budget brand of OVH. Webhostingtalk gives them favorable reviews.

</comment></comment></comment></comment></comment><comment><USER6> Something they (understandably) didn't address in the video was fault tolerance on the Postgres side. What do people like these days for that?

<comment><USER7> I'll address that here: We use streaming replication with a hot standby for fault tolerance on the PostgreSQL side. For backups in addition to the streaming replication we do WAL shipping to S3 every 15 minutes or 16 MB, and we do a full base backup to S3 daily during our low-traffic time of the day, which uploads around 30 GB to S3 daily (but LZO-compressed, so actually down to around 3 GB).

<comment><USER6> Thanks, Mitchell. Is your hot standby in a different zone (e.g. one in the West, one in the East) so you can handle one of their big outages?

<comment><USER7> Yes

</comment></comment></comment></comment><comment><USER8> in the last couple of month reading this site, I see more negative MongoDB reviews than most other 'cool tech' stuff. On one hand MongoDB by carrot-or- stick was pushed into environments with high-write needs (OLTP kind of systems). On another hand having true secondary indexes and semistructured data makes Mongo a 'closest to RDBMS' choice. So things like global write lock, indexes must fit into memory, auto-sharding questions, single-thread map-reduce -- all are pretty significant limitations for an OLTP data store. I wish MongoDB does not get discouraged and instead steps back, reviews academic foundations of the system and pick a couple of use cases and optimise their builds for them (similar to datawarehouse vs oltp kind of offerings)

</comment><comment><USER9> The glossing-over cassandra (because of Digg history etc..) was the point in the talk when I started raising eyebrows. Engineers evaluating systems should let the systems speak for themselves.

We have a fairly small cassandra cluster in production, serving over 50x the volume they mentioned in their talk, with good latency (real-time bidding) and not-too-painful operational footprint.

</comment><comment><USER10> Great talk! Looking at Riak vs MongoDB right now for a production system in fact. The data isn't K/V though and we need rich queries so I'm not sure what our solution will be unfortunately.

<comment><USER11> Why not just use postgresql?

<comment><USER12> Because Postresql is not a NoSQL database despite the recent lipstick additions.

I love the database but it is not optimal if you have a fluid schema like many use cases have today.

<comment><USER13> I recently asked if anyone could point me to an (open source) app on github or wherever that has a 'fluid schema' on a NoSQL system and nobody was able to show me one. Do you have a concrete example I can look at please?

</comment></comment></comment></comment><comment><USER14> Anything concrete about why cassandra is bad?..

<comment><USER15> One key difference is that Riak will rebalance data across nodes as they are added or removed automatically, Cassandra will not. You have to manually adjust the partitioning of data, balancing it by hand.</comment></comment>


Ask HN: How Google APIs Work? - <AUTHOR> For curiosity I have decided to check how google APIs are done and understand them. I discovered some things but I'm not understanding much of them.
I have used the https://artsandculture.google.com/search/streetview?project=streetviews to check it. In the following words, I will write what catches my attention.
In their headers, they accept the following encoding: gzip, deflate, br. gzip and deflate, are some of the tipical old encodings of the web, but br is a format that uses the Brotli algorithm, a compression algorithm created by google with the initial release in 2013, that seems to have a higher compression rate.
They are also including a cookie for tracking.
But what really interest me, is the body of the data requested. This is the following body:
)]}'
[[["stella.pr","StreetViewAssetsQuery:6b2794d3-0346-4081-a42d-d7083ddb72a8",[["Trench \u0026 No-Man's Land Displays","Kansas City, Estados Unidos","//maps.googleapis.com/maps/api/streetview?size\u003d400x300\u0026pano\u003dqdAuuOptcfIAAAQrBkpCZQ\u0026heading\u003d-166.0\u0026pitch\u003d-5.0\u0026sensor\u003dfalse\u0026client\u003dgoogle-cultural\u0026signature\u003d2iuo0SS-I4fm1_Wl6jB5HklBME4","/streetview/trench-no-man-s-land-displays-national-wwi-museum-and-memorial/9AH8vCr07x69UQ",20,null,null,null,null,["9AH8vCr07x69UQ",1.3333334,null,null,null,null,null,[] ,null,"0,0,0,0,0,0,0,0,0,0",0] ,null,null,null,null,null,null,[] ,null,null,null,[1,"9AH8vCr07x69UQ"] ,null,null,null,null,[] ] ,
...
,null,3452,null,null,null,"CgUSA0NEQQ"] ,["e",2,null,null,13754] ]]
This body structure is compressed and hard to understand. The points are that there are no keys in the body, so there are less Kb of unnecessary information. Is there any process of mapping this information to a more understable information? What's the point of do this? Can make the developing process slower? <comment><USER2> > What's the point of do this? Can make the developing process slower?

It's not a public API and both server and client (their javascript library) are written by the same company. The data isn't meant to be consused by anybody else. And their terms of service probably doesn't allow it.

The public API is [https://developers.google.com/maps/documentation/streetview/...](https://developers.google.com/maps/documentation/streetview/metadata)</comment>


How Amazon's Ad Business Could Threaten Google and Facebook - <AUTHOR> https://www.wsj.com/articles/how-amazons-ad-business-could-threaten-google-and-facebook-1517157327 <comment> j-c-hewitt Since their ad program was released, I've been running ads on Amazon as well as ads on the other platform for both clients and myself.

Amazon's ad platform has always had fewer features, a crappier interface, worse third party software available, and has at times run by confusing rules that aren't fully documented.

None of that matters because it's 4-10x+ more cost effective and doesn't require a large budget or a sophisticated account structure to generate profitable sales in volume. In general the American people have demonstrated a preference and that preference is that more than half of them want to buy stuff on Amazon and not on other websites. Trying to get people to buy stuff on a non-Amazon website is rolling the stone uphill. It doesn't hurt that basically the entire middle class demographic in the US subscribes to Prime. If you sell physical things to customers online there is no better way to do it at this time.

There are many issues involved in selling stuff on Amazon (and being an Amazon customer) but those problems really pale in comparison to the business model of running your own store on non-Amazon fulfillment and paying GooFaceBannerAd to drive traffic to your website that would really prefer to be going to Amazon.

<comment><USER2> So, you are saying while the general theory is that people want to buy stuff on Amazon but using the GooFaceBannerAd you are able to drive traffic to your non-Amazon store and sell stuff to people?

Edit: As I am getting downvoted let me clarify to the query. I am a bit confused as the post seems to suggest getting people to buy stuff from non- Amazon site has been difficult.

In which case I wonder how effective it is to bypass Amazon ad system and getting people to your own site. Or am I misunderstanding the post and the ad connects back to amazon product page.

<comment> j-c-hewitt It is confusing. No I'm referring to the internal Amazon ad system that the article mostly refers to. If you search around Amazon with your ad blocker off you'll see results with "Sponsored" next to them.

<comment><USER2> Thanks for the clarification. You are using these banner ads to drive traffic to a non-Amazon site or re-direct people back to Amazon product page? Because selling stuff on former seems difficult.

</comment></comment></comment></comment><comment><USER3> We see (or are forced to see) ads on Google, Facebook et al. is because they are free to use services for consumers. Eventhough that bother some us so much that we use all sort of ad blocking plug-ins at our disposal to stop seeing those pesky ads. I pay for my Amazon Prime membership and for every product I purchase on Amazon.com. Last thing I want is getting bombarded by obnoxious ads. while shopping for my next two-day delivery of Topo Chico water bottles there. Unless Amazon decides to roll out a steeply discounted Prime membership program, I would be a very unhappy customer seeing ads on a paid platform.

<comment><USER4> Ads for Amazon is like Walmart choosing what product to be put on display and where. It is an established business.

I echo some of your sentiment. If Amazon is truly as customer obsessive as they branded themselves, they would avoid putting cheap ads out there. Since their click rate is going to be so high, those ads will be very expensive indeed, which I would assume to be quality ads as well.

<comment><USER3> Quality ads would be like "hey see that you are buying Topo Chico. Why don't you try these new flavors as well at a discounted rate?". Instead of "Buy Nestle water" ads. Or something very unrelated. Basically a win-win situation for both parties.

But to be honest I would be very much reluctant at seeing a lot of ads. They do this already with "Sponsored" products and most of the time it serves as distraction instead of convincing buyer into even at least clicking their ad.

<comment><USER5> This is what I never understood in online ads. You're looking to buy, say, a North Face jacket but the price is a bit higher than you're willing to pay, so you close the tab.

Now, you go elsewhere on the web and you see ads for that particular jacket, but there's no incentive for you to click on the ad because you already know the price is too high.

But if the ad says: _" hey, 10% off on this jacket in the next 30 minutes with coupon code xyz"_ then yeah you might click on the ad, buy the jacket and in the end, the customer, the retailer and the ad agency are all winning.

<comment><USER6> If you could manipulate coupon codes like that then why would anybody pay full price (if it wasn't urgent)?

<comment><USER7> Not everyone is that savvy or willing to put in the effort.

There are people who save ridiculous amounts with traditional coupons, but not many people are willing to put in the effort or make the sacrifices to to that, so they still work for the companies.

</comment></comment></comment></comment></comment></comment><comment><USER8> Facebook was more likely to threaten Google. But I think they failed, amazon won't be a threat.

Ps. A possible big competitor is not not considered a threat in this context. Just as Facebook is a big competitor to Google, but failed to be a threat

<comment><USER9> For me, Amazon will be because the nature of their advertising is intent- based, like Google's. What Facebook has always struggled with is the _context_ of the adverts (by their very nature), whereas Google and Amazon's advertisers already have the user's intent right there (searches for services/products).

<comment><USER10> What use is intent if you know every single intimate detail of a person's private life.

<comment><USER11> Judging by the ads I get on Facebook: a lot.

</comment></comment></comment></comment><comment><USER12> I use AMS for my publishing business. Amazon sales were up 73% the first year I started using it, although in the same year I also launched several new products that did well (and did not benefit much from AMS ads).

Yes, the UI blows, there are various unfathomable policies ("One or more of your products doesn't meet the financial threshold established by Amazon and is no longer eligible for advertising" \- this for one of our top-selling titles), and the reporting leaves a lot to be desired, but at the end of the day it works.

The same cannot be said for Facebook ads (waste of time and money for what I sell), Google AdWords (hurt by its own popularity and ridiculous policies such as this one: [http://www.digitalmediamachine.com/2017/10/google-kills- my-a...](http://www.digitalmediamachine.com/2017/10/google-kills-my-ad-for- revisionist.html)), and LinkedIn which is fleecing small-business clients with sky-high CPCs that don't perform.

I fear, however, that the popularity of AMS may cause the same fallout as AdWords - loads of buyers rushing in, driving up rates and driving organic results off the page.

</comment><comment><USER13> Hello!

I used to working on the AMS self-service advertising team as a developer.

 _Note all opinions are my own._

When I first joined the advertising team, I was unsure about a product that can have a seedy sort of reputation on the web. But I was very pleasantly surprised how much we care about the customer experience in regards to ads at Amazon.

The end goal is always to show the customer the best item for purchase, and Retail would always set a high bar for the Ads team. This has in-turn caused us to invest a lot into ensuring the relevancy of our ads is top notch, we do not want to display ads which are ineffective - this hurts both the retail customer and the advertiser.

Nothing would get approved without vigorous testing to ensure the ads had a neutral or positive impact on the retail customer.

<comment><USER14> While I don't doubt your personal integrity, I am sure a Google or Facebook employee would claim the same. After a while, organizational incentives start nudging you in a direction optimal for the company and not necessarily optimal for the user. It starts in slow imperceptible increments and before you know, your product is ad-infested sub-optimal UX.

<comment><USER15> Except ads aren't Amazon's primary business or revenue stream. They're incentivized to do exactly what the OP stated.

<comment><USER14> Or so one would think. But you can try for yourself and see how ad-infested Amazon has become.

To throw a few random examples, I just tried (from Mac / Chrome / Incognito / Signed-out mode), I searched for "sensodyne toothpase". Entire page is full of ads and I have to do a full scroll to see organic non-ads results. Same for "biodegradable trash bags". Almost the same situation for "qtips" though there were 2 partial images of organic search results. Its almost as if I am searching on Google (and no, thats not a compliment).

</comment></comment></comment></comment><comment><USER4> Ads is free money from Amazon. After all they have the best possible traffic one can imagine: clear shopping intent queries, with customer shopping history. They just need to put up the billboards, the money will follow. Maybe not Google scale, but free money, what not to like?

<comment><USER16> Several things... for example: as a seller advertising on Amazon, you are sharing all your profit metrics with a potential competitor who has a history of stepping in with their own proprietary branded products where they see significant profits to be made.

</comment></comment><comment><USER17> Maybe I'm biased, but personally, I never ever clicked on any ads showed up along my google search query, even if they are relevant. I just can't get rid of the nagging feeling that: "there gotta be better deals else where (e.g. on Amazon?)" or "The merchant paid Google a premium to show up along side my search results, the price can't possibly be the lowest". Am I the only one feeling this way?

</comment><comment><USER18> Why is there not a true rival to Google Adsense?

<comment><USER19> The winner-take-all dynamics of the internet.

</comment></comment><comment><USER20> We changed the URL from [http://archive.is/ZdCQL](http://archive.is/ZdCQL).

Please post the original link. It's fine to include a link to an archived version in the comments. Users frequently do that as a favor to other readers. But please don't submit such a link to begin with, because it obscures the source of the article and messes up the data.

<comment><USER21> What about domains that are instaflagged? For example, I tried posting a Breitbart article [0] about a conservative blogger at Google, because in this one case I felt like it was the best "primary source" to get the discussion started. It was auto-killed like every article on Breitbart. [1] So I resubmitted via archive.is and actually got 6 upvotes [2], showing there was at least some interest in the topic that would never had surfaced if I used the original link.

[0] [https://news.ycombinator.com/item?id=16103017](https://news.ycombinator.com/item?id=16103017)

[1] [https://news.ycombinator.com/from?site=breitbart.com](https://news.ycombinator.com/from?site=breitbart.com)

[2] [https://news.ycombinator.com/item?id=16246184](https://news.ycombinator.com/item?id=16246184)

<comment><USER22> Have we really arrived at a place where "conservative blogger" is an accurate description of a person who:

* "isn't exactly allergic to the stuff" (re: white nationalism)

* wrote "Not all humans are born the same, of course, and the innate character and intelligence of some is more suited to mastery than slavery. For others, it is more suited to slavery. "

* asked "What's so bad about the Nazis?"

* etc

?

I'm not sure if it's liberals or conservatives who should be offended by that. Hopefully both.

<comment><USER21> The interesting part of that article was that his views were sufficient to silently land him on a "do not visit" list at Google HQ, which implies that Google is tracking people who hold such opinions and adding them to such lists.

<comment><USER23> No great conspiracy here. Any mid to large org maintain such lists. If you fire someone who is wasting the orgs time, do you then want him to show up/get hired/be entertained by some group in the next building/next campus etc who have no idea what the story is?

<comment><USER21> The guy is not an ex-Google employee. This event was named in the Damore lawsuit as evidence of discrimination against political views because all he was guilty of was blogging things someone at Google had a problem with.

It doesn't make you uncomfortable to know that even if you've had no previous interactions with Google, you may be tracked and added to a list of undesirables? Who controls the list, who has power to add people to it, and what else are they doing with this list? What if they're looking at his search history too? Would that be okay?

<comment><USER24> Google are a private company with no requirement (barring legal strictures) to allow anyone onto their campus. If they want a list of undesirables, they can have one. Same as you've probably got one for your house.

</comment></comment></comment></comment></comment></comment></comment><comment><USER25> What would this look like compared to something like Kaiser Permanente?

<comment><USER26> Did you perhaps mean to ask this in the other Amazon thread right now?

[https://news.ycombinator.com/item?id=16264662](https://news.ycombinator.com/item?id=16264662)

<comment><USER25> Yup

</comment></comment></comment><comment><USER8> I call BS, it's a bubble ad network.

The chances are users will be happier with a product through natural selection ( eg. better/trustwurthy reviews), than products through ads.

Although, Amazon could earn more in short term, users are less satisfied and they lose money long term.

What happens if Chinese vendors buy all the ads and set a lower price ( which they can). Then most buyers won't be satisfied on Amazon ( terrible English) and they start using AliExpress ( cheaper and same shipping terms after eg. 1 year of Chinese vendors that dominate Amazon). This "free money" could be their downfall.

Already technical users here on HN don't want to use Amazon for the risk of receiving counterfeit products.

<comment><USER27> Amazon is in complete control of the ads they show. Quality of ads is a huge goal of anyone working on an ad network trying to compete with Facebook or Google.

<comment><USER28> > Quality of ads is a huge goal

How you do that is part of the lifecycle of an Ad platform. The ephemeral "quality of ad" is completely subjective and will forever be inadequately measured (to anyone's satisfaction for very long). Eventually all ad networks are subject to market pressures in pursuit of margins. Third party ads eventually arise from the existing technical implementation of ads (tons of untrusted code running in js and vpaid applications), even when "prohibited". There's nothing special about Amazon's platform. Lots of platforms have been the "new hot thing" with "total control".</comment></comment></comment>


Ask HN: What's th best way to guard against Ransomware? - <AUTHOR> <comment><USER2> When it comes to the "your data is encrypted" ransomware attacks, try this: make your backups and test them regularly. There is no excuse for data loss to affect anything prior to the current business day. Whatever happened to disaster recovery/ business continuity plans? 20 years ago I used to design them for a handful of Fortune 100 companies. Now it seems nobody cares.

I've always wondered why the more static part of web pages and resources don't typically get stored on a RO file system. It's easy enough to remount -o,rw to make changes for updates. It doesn't stop all the attacks, but it stops a whole lot of them.

This doesn't solve the stolen data type ransomware, but most companies seem to not even try to be the least bit resilient. The risks have been the same for decades. Why act surprised now?</comment>


Millennials' new retirement number? $1.8M (or more) - <AUTHOR> http://college.usatoday.com/2016/03/30/millennials-new-retirement-number-1-8-million-or-more/ <comment><USER2> >The youngest of the Boomers, those born in 1964, would need $1.3 million earmarked for retirement. A Gen Xer born in 1975 would need about $1.6 million. The figure for a Millennial: $1.8 million.

FFS. This is such non-news, the article even defuses itself.

Save for retirement? OMG! Inflation! Woah!

</comment><comment><USER3> Retirement?

That's when you start working for Uber or Starbucks, right?

:(</comment>


So Dan Lyons Called - <AUTHOR> http://daringfireball.net/2009/08/so_dan_lyons_called <comment><USER2> Gruber's reply was unbecoming and mean-spirited. Gruber routinely awards people the title "Jackass of the Week" for having opinions he disagrees with or for making erroneous predictions. When he himself whiffed hard on his Google Voice post he had to know that some ribbing and criticism was headed his way. He should have been prepared to handle it graciously.

Capitalizing on Gruber's mistake, John Lyons writes a post that primarily ridicules _Apple_ for its obsession with secrecy and for its contemptuous relationship with the media. Yes, he does make fun of Gruber, but it's from the point of view of Fake Steve. Lyons himself isn't saying Gruber is a "dipshit", he's saying that he bets that's how Steve Jobs thinks of him.

In return, Gruber seems to really scrape the barrel to find ammunition for a purely ad hominem attack, ridiculing Lyons' day job and implying that his writing is disingenuous. I takes a serious lack of perspective to call the creator of Fake Steve Jobs a tool of old media.

<comment><USER3> I think you're reading too much into it. John has posted to Fake Steve articles he enjoys in the past. Gruber receives and dishes out this kind of stuff all the time. I bet if you'd ask him, he just thought the article about him was funny, wanted to link to it and decided to play along a bit.

<comment><USER2> You could be right. Maybe Lyons get together every Thursday at a bar and rib each other over who had the harsher blog post while they sip on Deus Brut Des Flandres. Maybe they have pet names for each other's kids, and give them noogies at get-togethers.

Maybe it is all fun and games, but they don't write about that stuff on their blogs so it's impossible to tell. From my point of view Gruber's reply doesn't read as friendly or jocular. Lyons calls Gruber a egotistical, obnoxious fanboy and jokes that Apple hatched a wide-reaching conspiracy just to trick him into a journalistic mistake. Fair enough.

Gruber "jokes" that Lyons writes limp, insincere, watered-down articles for a failing magazine. Maybe Gruber misjudged his tone, but ouch. I'd be hurt if Gruber wrote that about me. I'd be hurt if a _friend_ wrote that about me.

<comment><USER4> The whole thing is written according to one of Dan's FSJ schticks. The form of this "So Foo Called" schtick obligates the recipient of the call to treat Foo with casual, but brutal condescension.

They don't have to be drinking buddies for this to be a sincere form of flattery.

</comment></comment></comment></comment><comment><USER5> Lighten up guys. Gruber's trying to be funny here and he mostly succeeds. I'm 100% certain that Lyons read this and thought it was hilarious. Journalists being hypercritical professional whiners, there's nothing Gruber's saying about Newsweek here that Lyons and his editors probably haven't said to eachother over lunch a hundred times already.

</comment><comment><USER6> That reads like it ought to be on a blog called Fake John Gruber.

</comment><comment><USER7> Oh, snap. Nothing burns like six days later.

</comment><comment><USER8> Why the drama?

Edit: oh -- [http://www.fakesteve.net/2009/08/dear-gruber-youve-been- pwne...](http://www.fakesteve.net/2009/08/dear-gruber-youve-been-pwned.html)

Not entirely undeserved.

<comment><USER9> I'm not sure if it's "drama" or "teasing fun". With those 2 it can be hard to tell.

</comment></comment><comment><USER10> Some blogger catfight over an Apple fanboy apologia?

How on earth did Apple manage to turn consumer electronics into a drama-rich soap opera? And more to the point, why?

<comment><USER11> It's a joke.

</comment></comment><comment><USER12> It's funny, but kind of hollow. I'll take Gruber over Lyons any day, but Newsweek is also the home of Steven Levy, not to mention Michael Isikoff.

</comment><comment><USER13> Gruber sure can take a joke. What a trooper</comment>