Von Neumann Universal Constructor - <AUTHOR> https://en.wikipedia.org/wiki/Von_Neumann_universal_constructor <comment><USER2> David Deutsch, the physicist who proved that quantum computers can be Turing complete, has recently created constructor theory:

[http://constructortheory.org/](http://constructortheory.org/)

"Constructor Theory is a new approach to formulating fundamental laws in physics. Instead of describing the world in terms of trajectories, initial conditions and dynamical laws, in constructor theory laws are about which physical transformations are possible and which are impossible, and why. This powerful switch has the potential to bring all sorts of interesting fields, currently regarded as inherently approximative, into fundamental physics. These include the theories of information, knowledge, thermodynamics, and life."

<comment><USER3> I love the intentional or unintentional reference to Trurl and Klapaucius, the constructors in Stanislaw Lem's novel, Cyberiad, who grappled with many philosophical issues like artificial intelligence, robotics, technology, thermodynamics, life, love, romance, poetry, self replication, and the origin of the universe.

[https://www.donhopkins.com/home/catalog/lem/Trurl.html](https://www.donhopkins.com/home/catalog/lem/Trurl.html)

[https://www.donhopkins.com/home/catalog/lem/Klapaucius.html](https://www.donhopkins.com/home/catalog/lem/Klapaucius.html)

[https://en.wikipedia.org/wiki/The_Cyberiad](https://en.wikipedia.org/wiki/The_Cyberiad)

>Trurl and Klapaucius

>Trurl and Klapaucius are brilliant (robotic) engineers, called "constructors" (because they can construct practically anything at will), capable of almost God-like exploits. For instance, on one occasion Trurl creates an entity capable of extracting accurate information from the random motion of gas particles, which he calls a "Demon of the Second Kind". He describes the "Demon of the First Kind" as a Maxwell's demon. On another, the two constructors re-arrange stars near their home planet in order to advertise.

>The duo are best friends and rivals. When they are not busy constructing revolutionary mechanisms at home, they travel the universe, aiding those in need. As the characters are firmly established as good and righteous, they take no shame in accepting handsome rewards for their services. If rewards were promised and not delivered, the constructors may even severely punish those who deceived them.

[...]

>On another occasion, Trurl and Klapaucius are captured by an interstellar "PHT" pirate. Trurl offers to build a machine capable of turning hydrogen into gold (something he can do manually, which he demonstrates by hand, mixing up protons and putting electrons around). However, the pirate turns out to have a PhD and cares not for the riches, but for knowledge (and in fact points out that gold becomes cheap if it is abundant). Trurl therefore makes a modified Maxwell's demon for him, an entity that looks at moving particles of gas and reads information that is, coincidentally, encoded in their random perturbations. This way, all the information in the universe becomes easily available. The demon prints out this information on a long paper tape, but before the pirate realizes most of the information is completely useless (although strictly factual) he is buried under the endless rolls of tape, ceasing to bother anyone.

<comment><USER4> See also Google's 2011 interactive doodle inspired by _The Cyberiad_ ,[1] in particular by the _machine that could create anything starting with n_.[2]

[1] [https://www.google.com/doodles/60th-anniversary-of- stanislaw...](https://www.google.com/doodles/60th-anniversary-of-stanislaw- lems-first-publication)

[2] [https://english.lem.pl/works/novels/the-cyberiad/146-how- the...](https://english.lem.pl/works/novels/the-cyberiad/146-how-the-world- was-saved)

</comment></comment></comment><comment><USER5> von neumann's original papers mention that his studies on CA were an abstraction due to the limitations of 1940's technology. His main concern was really automatic replication of real world physical objects.

In this respect, 3D printers offer a partial solution. Totally automated assembly is still being worked on.

<comment><USER3> Here's some stuff about that I posted in an earlier discussion, and transcribed from his book, "Theory of Self-Reproducing Automata".

His concept of self-reproducing mutating probabilistic quantum mechanical machine evolution is quite fascinating and terrifying at the same time (or outside of time), potentially much more powerful and dangerous than mere physical nanotechnology "gray goo" and universe-infesting self replicating von Neumann probes:

Can Programming Be Liberated from the von Neumann Style? (1977) [pdf] (thocp.net)

[https://news.ycombinator.com/item?id=21855249](https://news.ycombinator.com/item?id=21855249)

[https://news.ycombinator.com/item?id=21858465](https://news.ycombinator.com/item?id=21858465)

John von Neuman's 29 state cellular automata machine is (ironically) a classical decidedly "non von Neumann architecture".

[https://en.wikipedia.org/wiki/Von_Neumann_cellular_automaton](https://en.wikipedia.org/wiki/Von_Neumann_cellular_automaton)

He wrote the book on "Theory of Self-Reproducing Automata":

[https://archive.org/details/theoryofselfrepr00vonn_0](https://archive.org/details/theoryofselfrepr00vonn_0)

He designed a 29 state cellular automata architecture to implement a universal constructor that could reproduce itself (which he worked out on paper, amazingly):

[https://en.wikipedia.org/wiki/Von_Neumann_universal_construc...](https://en.wikipedia.org/wiki/Von_Neumann_universal_constructor)

He actually philosophized about three different kinds of universal constructors at different levels of reality:

First, the purely deterministic and relatively harmless mathematical kind referenced above, an idealized abstract 29 state cellular automata, which could reproduce itself with a Universal Constructor, but was quite brittle, synchronous, and intolerant of errors. These have been digitally implemented in the real world on modern computing machinery, and they make great virtual pets, kind of like digital tribbles, but not as cute and fuzzy.

[https://github.com/SimHacker/CAM6/blob/master/javascript/CAM...](https://github.com/SimHacker/CAM6/blob/master/javascript/CAM6.js#L4569)

Second, the physical mechanical and potentially dangerous kind, which is robust and error tolerant enough to work in the real world (given enough resources), and is now a popular theme in sci-fi: the self reproducing robot swarms called "Von Neumann Probes" on the astronomical scale, or "Gray Goo" on the nanotech scale.

[https://en.wikipedia.org/wiki/Self- replicating_spacecraft#Vo...](https://en.wikipedia.org/wiki/Self- replicating_spacecraft#Von_Neumann_probes)

[https://grey-goo.fandom.com/wiki/Von_Neumann_probe](https://grey- goo.fandom.com/wiki/Von_Neumann_probe)

>The von Neumann probe, nicknamed the Goo, was a self-replicating nanomass capable of traversing through keyholes, which are wormholes in space. The probe was named after Hungarian-American scientist John von Neumann, who popularized the idea of self-replicating machines.

Third, the probabilistic quantum mechanical kind, which could mutate and model evolutionary processes, and rip holes in the space-time continuum, which he unfortunately (or fortunately, the the sake of humanity) didn't have time to fully explore before his tragic death.

p. 99 of "Theory of Self-Reproducing Automata":

>Von Neumann had been interested in the applications of probability theory throughout his career; his work on the foundations of quantum mechanics and his theory of games are examples. When he became interested in automata, it was natural for him to apply probability theory here also. The Third Lecture of Part I of the present work is devoted to this subject. His "Probabilistic Logics and the Synthesis of Reliable Organisms from Unreliable Components" is the first work on probabilistic automata, that is, automata in which the transitions between states are probabilistic rather than deterministic. Whenever he discussed self-reproduction, he mentioned mutations, which are random changes of elements (cf. p. 86 above and Sec. 188.8.131.52 below). In Section 184.108.40.206 above and Section 1.8 below he posed the problems of modeling evolutionary processes in the framework of automata theory, of quantizing natural selection, and of explaining how highly efficient, complex, powerful automata can evolve from inefficient, simple, weak automata. A complete solution to these problems would give us a probabilistic model of self- reproduction and evolution. [9]

[9] For some related work, see J. H. Holland, "Outline for a Logical Theory of Adaptive Systems", and "Concerning Efficient Adaptive Systems".

[https://www.deepdyve.com/lp/association-for-computing- machin...](https://www.deepdyve.com/lp/association-for-computing- machinery/outline-for-a-logical-theory-of-adaptive-systems-efsWyqMa1l)

[https://deepblue.lib.umich.edu/bitstream/handle/2027.42/5578...](https://deepblue.lib.umich.edu/bitstream/handle/2027.42/5578/bac4296.0001.001.pdf?sequence=5)

[https://www.worldscientific.com/worldscibooks/10.1142/10841](https://www.worldscientific.com/worldscibooks/10.1142/10841)

Ericson2314 3 months ago [-]

> Although I refer to conventional languages as "von Neumann languages" to > take note of their origin and style, I do not, of course, blame the great > mathematician for their complexity. In fact, some might say that I bear some > responsibility for that problem.

From the paper. Whew.

</comment></comment><comment><USER6> Are there any good resources that build up from no experience in CA to a Universal Constructor?

<comment><USER7> I suspect there aren't. Please write the book while you learn about it, and we can all have it. Thanks!

(I'm a strong believer in writing the books yourself that you really would love, but don't exist.)

<comment><USER3> If you don't have the time to write whole books yourself, you can just write fictitious reviews and criticisms of books that don't actually exist, touching on all the best parts and ignoring the rest. And nobody will be able to contradict you!

[https://en.wikipedia.org/wiki/Stanis%C5%82aw_Lem%27s_fictiti...](https://en.wikipedia.org/wiki/Stanis%C5%82aw_Lem%27s_fictitious_criticism_of_nonexistent_books)

>Stanisław Lem's fictitious criticism of nonexistent books

>Stanisław Lem's fictitious criticism of nonexistent books may be found in his following works: in three collections of faux reviews of fictional books: A Perfect Vacuum (Doskonała próżnia, 1971), Provocation (Prowokacja, 1984), and Library of 21st Century (Biblioteka XXI wieku, 1986) translated as One Human Minute, and in Imaginary Magnitude (Wielkość Urojona, 1973), a collection of introductions to nonexistent books.

>While reviewing nonexistent books, a modern form of pseudepigraphy, Stanisław Lem attempted to create different fictional reviewers and authors for each of the books. In his own words: "I tried to imitate various styles – that of a book review, a lecture, a presentation, a speech (of a Nobel Prize laureate) and so on". Some of the reviews are lighthearted, concentrating mostly on the story; others, however, read more like serious, academic reviews. Some of the reviews are parodies, or the books being reviewed are parodies or complete impossibilities, others are quite serious and can be seen almost as drafts for novels that Lem never got around to write. Lem wrote: "With years passing a great impatience grew in me. It would be a hard work to convert ideas into narration, and that was one of the main reasons I went for such cruel abridgements of the books". Lem was not alone in passing through this kind of crisis: examples abound of works planned by literary celebrities, but never completed. Lem also remarked that he was eventually convinced that writing summaries and introductions enabled him to save time on producing things of importance, namely, his modeling experiments, compared to full-blown literary efforts, most of which would have constituted mundane craftsmanship.

[...]

>One Minute

>The reviewed faux book is alleged to be a collection of statistical tables, a compilation that includes everything that happens to human life on the planet within any given 60 second period. Reviewing it, Lem expresses his fascination with this project and points out its inherent flaw. He notes that these tables show "far more statistical evidence of human evil (murders, rapes, starving children) than of human decency". At the same time he remarks that it is impossible to measure "filial or maternal love", or to "gauge the heat of lovers' passions", or to register "those acts of kindness whose authors wished to remain anonymous."

<comment><USER7> Thank you! Gee, that's such a brilliant idea. I must try it.

If Chesterton's Mr McCabe (from _On Mr. McCabe and a Divine Frivolity_ [0] in _Heretics_ ) had never existed, it would make little difference. It's just an excuse to say what he wants to say.

...criticism of the highest kind...treats the work of art simply as a starting-point for a new creation. It does not confine itself...to discovering the real intention of the artist and accepting that as final. And in this it is right, for the meaning of any beautiful created thing is, at least, as much in the soul of him who looks at it, as it was in his soul who wrought it. Nay, it is rather the beholder who lends to the beautiful thing its myriad meanings, and makes it marvellous for us, and sets it in some new relation to the age, so that it becomes a vital portion of our lives, and a symbol of what we pray for, or perhaps of what, having prayed for, we fear that we may receive. – Wilde, _The Critic As Artist_

[0] [http://gutenberg.net.au/ebooks13/1301191h.html#ch16](http://gutenberg.net.au/ebooks13/1301191h.html#ch16)

Edit: I've 'borrowed' _A Perfect Vacuum_ from archive.org, thanks! Also, I've hardly read any Borges, but that Borges fake review of a novel which is identical with _Don Quixote_ but totally different is one of the funniest things I've read.

[http://www.coldbacon.com/writing/borges- quixote.html](http://www.coldbacon.com/writing/borges-quixote.html)

</comment></comment></comment></comment><comment><USER8> I would be interested in the simplest form of self-replication, that is, the simplest automata machine, and the simplest initial set of states...

Interestingly, Phi spirals -- seem to construct themselves in nature at many different scales, and without (apparently) running inside of an automata machine of any sort...

I nominate Phi spirals as universal (sans apparent automata machine) constructors...

If someone could explain what type of automata machine Phi spirals are running in, and what this automata machine's rules are, then I think that would go a long way to understanding the universe, but at this point in time, I cannot determine a universal automata machine, nor its rules...

I only know that Phi spirals are automatically formed by nature, at a variety of scales, and in a variety of mediums...

<comment><USER9> What are some examples of Phi spirals (assuming you mean golden spirals?) outside of biology? We see them all over the natural world but off the top of my head I can't think of any non-biological examples. A quick read of Wikipedia points to logarithmic spirals occurring occasionally (although not specifically golden spirals).

(Also if we're just accepting geometric shapes, I'd nominate ellipses as being far more common, since they show up all the time in orbital mechanics.)

<comment><USER10> Spiral galaxy's which occur due to decreasing orbital velocities (edit: in radians) as you move from the center. Whirlpools and hurricanes are shaped by similar rotational effects.

However, their close approximations not exactly the correct shape.

<comment><USER11> That orbital velocities _don't_ do that was one of the first mysteries which led to people thinking about dark matter:

[https://en.m.wikipedia.org/wiki/Galaxy_rotation_curve#/media...](https://en.m.wikipedia.org/wiki/Galaxy_rotation_curve#/media/File%3ARotation_curve_of_spiral_galaxy_Messier_33_\(Triangulum\).png)

<comment><USER10> That's not enough to offset the increased distances from longer orbits. So you still get a spiral as the velocity in in radians decreases. But, it looks like a Golden spiral because the velocities in m/s are almost constant due to dark matter.

PS: Should have said orbital period to be more clear.

</comment></comment></comment></comment></comment><comment><USER3> This is one of my favorite topics and wikipedia pages! Please forgive my wall of text, and inscrutable ascii graphics, and check out the Buckley paper for better illustrations.

Here's some JavaScript code that implements (and describes) John von Neuman's 29 state cellular automata machine. I based it on some older "jvn" C code by R. Nobili, U. Pesavento, and Umberto Pesavento I found on the net, but I have rewritten it to be more symbolic and self documenting, so I could understand it better.

What it really needs is a specialized set of rule-specific editing tools and templates to stamp down, since it's impossibly tedious to paint anything non- trivial with the CA painting tools that this version of CAM6 currently supports.

[https://github.com/SimHacker/CAM6/blob/master/javascript/CAM...](https://github.com/SimHacker/CAM6/blob/master/javascript/CAM6.js#L4569)

I've tried to document what everything means and describe how it works. With evocative function names and comments like "pointedToByExcitedOrdinaryOrSpecial" (Return 1 if pointed by an excited transmission state (ordinary or special), else returns 0) and "wellFlankedByExcitedNotNextExcitedConfluent" (Return 1 if well flanked by an excited (not next excited) confluent state, else returns 0)! ;)

For example, here are the bit sequences for constructing new cells. You can send these sequences of bits down a wire to an arrow pointing into an empty space, and it will construct the corresponding cell in that empty space -- the intermediate construction states are called "Sensitized", and they huffman- encode all the possible cells you can create. Note that you can only construct non-excited cells. (See Buckley, page 457, The mechanisms of construction, below.)

                    // Instructions to the jvn29 construction arm, whose tip
                    // is an arrow pointing into an unexcited state, that
                    // creates a sensitized state which evolves into other
                    // states over time, given the following excitement inputs.

                    constructionInstructions: {
                        OR:  '10000', // => S S0 S00 S000 OR
                        OU:  '10001', // => S S0 S00 S000 OU
                        OL:  '1001',  // => S S0 S00 OL
                        OD:  '1010',  // => S S0 S01 OD
                        SR:  '1011',  // => S S0 S01 SR
                        SU:  '1100',  // => S S1 S10 SU
                        SL:  '1101',  // => S S1 S10 SL
                        SD:  '1110',  // => S S1 S11 SD
                        C00: '1111'   // => S S1 S11 C00
                    },

Here is the table of all the possible cell values, symbols and names (which is useful for an editor's user interface):

                    // Array of dicts describing cell values for jvn29.

                    cellStates: [
                        { symbol: 'U',    value: 0x00, name: 'Unexcited'              },
                        { symbol: 'S',    value: 0x01, name: 'Sensitized'             },
                        { symbol: 'S0',   value: 0x02, name: 'Sensitized 0'           },
                        { symbol: 'S1',   value: 0x03, name: 'Sensitized 1'           },
                        { symbol: 'S00',  value: 0x04, name: 'Sensitized 00'          },
                        { symbol: 'S01',  value: 0x05, name: 'Sensitized 01'          },
                        { symbol: 'S10',  value: 0x06, name: 'Sensitized 10'          },
                        { symbol: 'S11',  value: 0x07, name: 'Sensitized 11'          },
                        { symbol: 'S000', value: 0x08, name: 'Sensitized 000'         },
                        { symbol: 'C00',  value: 0x10, name: 'Confluent 00'           },
                        { symbol: 'C10',  value: 0x11, name: 'Confluent 10'           },
                        { symbol: 'C01',  value: 0x90, name: 'Confluent 01'           },
                        { symbol: 'C11',  value: 0x91, name: 'Confluent 11'           },
                        { symbol: 'OR',   value: 0x20, name: 'Ordinary Right'         },
                        { symbol: 'OU',   value: 0x21, name: 'Ordinary Up'            },
                        { symbol: 'OL',   value: 0x22, name: 'Ordinary Left'          },
                        { symbol: 'OD',   value: 0x23, name: 'Ordinary Down'          },
                        { symbol: 'SR',   value: 0x40, name: 'Special Right'          },
                        { symbol: 'SU',   value: 0x41, name: 'Special Up'             },
                        { symbol: 'SL',   value: 0x42, name: 'Special Left'           },
                        { symbol: 'SD',   value: 0x43, name: 'Special Down'           },
                        { symbol: 'ORX',  value: 0xa0, name: 'Ordinary Right Excited' },
                        { symbol: 'OUX',  value: 0xa1, name: 'Ordinary Up Excited'    },
                        { symbol: 'OLX',  value: 0xa2, name: 'Ordinary Left Excited'  },
                        { symbol: 'ODX',  value: 0xa3, name: 'Ordinary Down Excited'  },
                        { symbol: 'SRX',  value: 0xc0, name: 'Special Right Excited'  },
                        { symbol: 'SUX',  value: 0xc1, name: 'Special Up Excited'     },
                        { symbol: 'SLX',  value: 0xc2, name: 'Special Left Excited'   },
                        { symbol: 'SDX',  value: 0xc3, name: 'Special Down Excited'   }
                    ],

I wrote an earlier version in OpenLaszlo that had some custom editing tools, but that requires Flash, and was on my old Drupal site that's not up any more. But here is the archive.org link and description (with a screen snapshot). I wasn't able to get the old Flash app to run though. The source code has some interesting initial condition configurations, that I'll try explain, and are described in more detail in a paper referenced on the Wikipedia page:

[https://web.archive.org/web/20110720235050/https://www.donho...](https://web.archive.org/web/20110720235050/https://www.donhopkins.com/drupal/node/41)

John von Neumann's 29 state Cellular Automata Implemented in OpenLaszlo

Submitted by dhopkins on Sun, 2005-09-18 04:12. Cellular Automata Live Demos

For fun, and to learn OpenLaszlo, I implemented the classic 29 state self reproducing cellular automata, invented by John von Neumann.

The JavaScript and XML code is written with no thought to efficiency, just conceptual clarity and convenience of implementation. It can't run a lot of cells at once, but at least it's slow enough to watch it compute. Don't worry: there's not space for it to reproduce!

I've configured it with several interesting initial conditions, including several different approaches to signal crossing, and an exclusive-or gate.

It uses pie menus for editing the grid of cells.

Laszlo von Neumann Cellular Automata Demo

[https://web.archive.org/web/20120319060434/http://www.donhop...](https://web.archive.org/web/20120319060434/http://www.donhopkins.com/openlaszlo/my- apps/vonNeumann/vonNeumann.lzx)

Laszlo von Neumann Cellular Automata Source Code in Laszlo

[https://web.archive.org/web/20120319060340/http://www.donhop...](https://web.archive.org/web/20120319060340/http://www.donhopkins.com/openlaszlo/my- apps/vonNeumann/vonNeumann.lzx?lzt=source)

Here are some of the most interesting initial configurations: all different ways to perform signal crossing. Signal crossing is difficult with this rule, which doesn't directly support it, so you have to "emulate it in software" with multi-celled machines or "organs". Here are a few different ways of dealing with signal crossing, each with their own problems and limitations.

Buckley's paper about these gadgets is called "Signal crossing solutions in von Neumann self-replicating cellular automata". The Wikipedia page discusses some of it and links to the paper, but the link is broken. So here is the paper about it on archive.org, on page 453:

[https://web.archive.org/web/20081209155223/https://uncomp.uw...](https://web.archive.org/web/20081209155223/https://uncomp.uwe.ac.uk/free- books/automata2008reducedsize.pdf)

or (faster download):

[https://donhopkins.com/home/documents/automata2008reducedsiz...](https://donhopkins.com/home/documents/automata2008reducedsize.pdf)

The Real Time Crossing (Buckley, p. 457, The real-time crossing organ) is like a road intersection that splits the two crossing lanes, then uses traffic lights to give cars in each pair of lanes alternating turns to cross, and then merges the lanes back together (since each intersection works at 50% throughput, you need to split, use two of them, and merge -- Factorio and Satisfactory players will get what I mean, in terms of conveyor belts, splitters and mergers, and conveyor belt throughput).

The description of the real time crossing, "This real time crossing is not easily constructible, but nonetheless here it is", is a reference to the fact that you can only construct non-excited cells (you'd get electrocuted if you tried to construct machines with the power on, so to speak -- it may be possible to kludge it with some simple machines, but in general it's extremely impractical if not impossible -- see Buckley p. 470).

And after you've constructed a machine in a powered-down state, then you have to excite just the right cells at just the right time, to get the machine to work. But that's impossible with this real time signal crossing machine! So it's like a beautiful tiny crystal alien artifact with perfectly blinking lights, that the laws of physics practically prohibit from ever being constructed, but there it is.

The Real Time Crossing depends on having a certain synchronized configuration of excited cells in it, acting as synchronized clocks or traffic lights that start and stop all the traffic at just the right time so the cars don't collide.

It's not actually possible for a universal constructor to construct a real time crossing, because it can't reach in through existing cells and excite internal cells from outside. It's essentially like a "Garden of Eden" configuration, that had to have been constructed by the Hand of God, and can't be copied or constructed, or used as part of a reproducible machine. Essentially it has a "spark of life" that is beyond the ability of creatures living in the world to ignite. Intelligent Design and DRM FTW! ;)

              <save
                name="Real Time Crossing"
                rows="16" cols="16"
                description="This real time crossing is not easily constructible, but nonetheless here it is."
              >
        sssssw-----sbhb-
        yggggggggg-wbyb-
        hhhhhhhhhy-wbyb-
        yggggggggg-wbyb-
        </comment><comment>sssw-wbyb-
        </comment><comment>w----wbyb-
        </comment><comment>w----wbyb-
        --ezshqaez-wbyb-
        --yty--wyt-wbyb-
        --shqshqsb-wbyb-
        ssq-wezw-z-wbybs
        --z-wyty-qswbybw
        --shqshqsy--bybw
        --ezw--wez--bybw
        --ytyqswyt--bybw
        -----w</comment><comment>hyhw
              </save>

The "Coded Channel Crossing" (Buckley, p. 460) is like two channels sharing the same wire by using a coding system, so you have to somehow make sure neither channel tries to send a message over that one wire at the same time (left as an exercise for the reader ;), otherwise there will be a collision. This is the only kind of signal crossing organ designed by von Neumann.

              <save
                name="Coded Channel Crossing"
                rows="8" cols="64"
                description="Coded channel crossings have interference problems, demonstrated here."
              >
        -zaaaaaaa-----qsssesqsq</comment><comment>-----qhshqsssqsqsqsq</comment><comment></comment><comment>---
        -z</comment><comment>w-----w---w-z-z</comment><comment>-----w</comment><comment>-w-z---z</comment><comment></comment><comment>---
        -sshhsssqsssssqsqsq-ssssssssssssqsusqhesqsq-ssssshshssssssssssq-
        </comment><comment></comment><comment></comment><comment></comment><comment>--w-----z</comment><comment></comment><comment></comment><comment></comment><comment></comment><comment>-
        zaagagaaa-thssqsssqsqsqsq-w-----b-tsssqsqsq</comment><comment></comment><comment></comment><comment>---
        z</comment><comment>-w-w</comment><comment>-w-z---b-w-----b-w---w-z-z</comment><comment></comment><comment></comment><comment>---
        ssssssssqsqsthqsqsq-ssssshw-----ssestsq-sshhsssssssssssssssssse-
        </comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment>----
              </save>

This is the fun one, and the subject of Buckley's paper, an "Autoinitializing Exclusive Or" organ (Buckley, p. 473 and on). You can use these to solve the signal crossing problem, without needing the intervention of a benevolent God to power up all your traffic lights with the right synchronization. It actually has a "boot up" sequence: you first send it a "reset" signal, and part of it is used to initialize all the clocks the first time it's run, then it fires a bunch of "explosive bolts" that cut off the auto-initialization circuitry, and start up the exclusive or gate, after all its clocks have been initialized. It's literally self modifying code, that pokes itself after it boots, to switch into run mode! That's why it's so big and messy, compared the elegant but impossible to construct real time crossing.

              <save
                name="Autoinitializing Exclusive Or"
                rows="28" cols="40"
                description="William R. Buckley's autoinitializing exclusive-or is initialized by 11111 at each input."
              >
        sstsb-sqqssssssssssssssssssssssssz--szsz
        y-z-b-wsw</comment><comment></comment><comment></comment><comment></comment><comment>z--wzwz
        w-b-b-w-sqqsqsssssssqfqssqsssssz-z--wzwz
        y-z-b-qawsw----ssz--wsw--wsz---z-z--wzwz
        w-b-b-qww-----sq-sqzqqz--wqaa--z-z--wzwz
        y-z-z-w-qsq---wsqr-swwassz--w--z-z--wzwz
        w-b-z-qaw-zcqsqsqsqssqsq-ssqw--z-z--wzwz
        y-z-z-qww-zfw-zsqc-szzasqr-szssz-z--wzwz
        w-b-zsw-w-z-w-sq-szwqqw--zqaqq-z-z--wzwz
        yaa-sqz-w-q-w--sswswzsz--zsw-w-z-z--wzwz
        </comment><comment>qsq-ssqsssssssqfqssqssqw-zsqz-wzwz
        zga-sqw-z-ssqsssssssqfqssqssqszsq-szwzwz
        b-y-wsz-z-wa---ssz--wsw--wsz--zwsqwzwzwz
        z-w-w-q-z--w--sq-sssqqz--wqaaazw-w-zwzwz
        b-y-w-z-z-qw--wsqr-wawassz---wzw-w-zwzwz
        z-w-w-q-qswcqsqsqsqssqsq-ssqswsw-w-zwzwz
        b-y-y-z-z--fw-zsqc-zazasqr-z---w-w-zwzwz
        z-w-y-z-z---w-sq-sqsqqw--zqa---w-w-zwzwz
        b-y-y-z-zsszw--ssw--zsz--zsw--qw-w-zwzwz
        z-w-y-z-sq-sqsssssssqfqssqssssww-w-zwzwz
        hstsy-z--sqsqsssssssqfqssqsssssw-w-zwzwz
        </comment><comment>z-----z--ssz--wsw--wsz-----w-zwzwz
        </comment><comment>z-----z-sq-sszqqz--wqaa----w-zwzwz
        </comment><comment>z----fz-wsqr-swwasqc--w----w-zwzwz
        </comment><comment>z----rqsqsqsqssqsq-sssw----w-zwzwz
        </comment><comment>z</comment><comment></comment><comment>----ssw</comment><comment>-w-zwzwz
        </comment><comment>sssssssssssssssssssssssssssw-zwzwz
        </comment><comment></comment><comment></comment><comment></comment><comment></comment><comment>-----swsww
              </save>

Buckley, p. 473 ("rtco" means "Real Time Crossing Organ"):

>Auto-initialisation provides much more capability than that used in the rtco, the examples of clock synchronisation and trivial reconstruction being the simplest applications. Mechanisms used are signal sampling, portal closure, clock synchronisation, and staged initialisation. Our implementation uses three separate start signals; a staged initialisation. Each clock of the rtco has a separate auto-initialisation circuit. Signal to a stage is intercepted from a suitable path of channel A, by an interposing confluent state, or portal, and directed to the pulser of the adjacent auto-initialisation circuit by an auxiliary path. Pulser output is directed to clock input; and to a special transmission path constructor, which is addressed shortly. Clock signal for the entire organ is suitably synchronised, as initialisation of the first clock pair occurs simultaneously upon input of the first start pulse. All subsequent initialisations occur given knowledge of timing for the first two clocks. In this way, synchronisation becomes a trivial concern. Figure 17 shows the auto-initialisable rtco.

>Fig. 17. The rtco, configured for auto-initialisation. The minimum start signal for this organ is 10^13 10^32 1, which is applied once to Ai. The organ begins crossing signal 30 clock ticks later. This organ can be started at the time of construction; there is no start signal propagation. All signal crossing paths impose equal delay. The organ has 18×18 dimensions. This configuration can be marginally reduced in area.</comment>


Show HN: Well curated and reliable API Marketplace for developers - <AUTHOR> https://promptapi.com/ <comment><USER2> Woah,

> DNS Lookup API is capable of programmatically perform lookups for A, NS, MX, > AAAA, TXT and SOA records of any domain. > 1,000 requests/day - 30,000 > requests/month -- $9.99/mo

What kind of crazy person would buy those, when practically every single programming language has a library for this?

On the other hand, there is "Redis API", which seems to have no storage limit (!), and free for <100 requests/day. There are no mentions of max request length there, so I wonder if I can use this for my backups.

</comment><comment><AUTHOR> Prompt API is a curated API marketplace with high scalability and security for developers. It differentiates from large and uncurated API marketplaces by its selection process and hosting options. Prompt API handpicks high-quality API applications from developers and hosts them in its servers for high availability and scalability (unlike many other API marketplaces). It also provides a single SDK to connect all APIs in the marketplace to solve vendor lock-in problem. Our customers can freely change vendors without the hastle of manuel development.

Moreover, Prompt API provides full featured "headless app backends" for building up new white-labeled products and all APIs have a rate limited free plan to try out.</comment>


IOS 7 to launch with FaceTime voice calls - <AUTHOR> http://www.theverge.com/2013/6/10/4413904/apple-ios-7-facetime-audio-only

<comment><USER2> Wait - someone used Skype in Cupertino and decided to make this a feature of FaceTime? What is going on here.

(Nice feature btw, but 5 years too late)</comment>


Cloudponics GroBox:automated internet-connected in-home plant grow system - <AUTHOR> http://www.cloudponics.com <comment><AUTHOR> The GroBox is a fully automated internet-connected in-home system for growing cannabis (and other plants) at home, it allows anyone to grow great quality plants at home without any hassle, controlling the whole process from your phone. Now on pre-order with 40% off

</comment><comment><USER2> Hey! I worked on something like this on my senior year project! At the time we didn't have easily available SoCs like the Raspberry Pi or a 'strong' smartphone app ecosystem we could use to make it as easily connected as things are today. We used an Arduino hooked up to a laptop with a simple demo app that changed humidity, temperature and lighting conditions for different plants.

I always wondered whether or not the idea was viable. Good luck with this, I'll be watching with interest.

You know I'd imagine you can do lots of cool stuff with this nowadays. Maybe have a camera in there to see how the plants are doing, etc.

<comment><AUTHOR> Thanks! we have a live camera in one of our test systems! [http://www.cannabislivestream.com](http://www.cannabislivestream.com)</comment></comment>


[Idea] DNS Fog, a tool to protect DNS privacy - <AUTHOR> DNS requests are logged either by your DNS provider or/and by your ISP. They log them to track each domain you visit or connect to. This cannot be fixed by using Google DNS because your ISP can (and will) also log requests to third party DNS servers. My proposal is the following: make this DNS logging plausible deniable and unusable by clogging it with random DNS requests, these requests would be generated by a software or daemon running on your PC or router. The software would fire DNS requests for random domains (all of them should be real, for example, domains in the Alexa Top 1 Million and weighted by rank (so popular domains get requested more)). This bogus DNS request would be fired randomly and, for example, at an average rate of 6 per minute, that way the cpu and bandwidth usage will be negligible. Also, the requests could be triggered just when you make real DNS requests. When checking your DNS logs, the attacker will have no way to find your real requests, its like looking for a needle in a haystack. (I am only suggesting this idea, I am not currently developing it). <comment><USER2> Can you cite an example of an mainstream ISP logging DNS queries sent by clients directly to third-party recursive/non-authoritative DNS servers (ie, 22.214.171.124 or OpenDNS)?

Obviously it's technically possible, but so is capturing and storing your cleartext HTTP traffic (not just URLs, the full request and response), and no mainstream ISP does that.

</comment><comment><USER3> Your ISP can STILL track when you do a TCP connection to any web page.</comment>


Blockchain could fix a key problem in China's food industry - <AUTHOR> https://qz.com/1031861/blockchain-could-fix-a-key-problem-in-chinas-food-industry-the-fear-of-food-made-in-china/ <comment><USER2> I can't see how this is a solution. The problem isn't whether corruption is possible, the problem is trust. It's a social problem, not a technical problem. You can have the most corruption-free system in the world, but normal people won't understand how it works, nor will they care to verify the system. They get their trust through their social circle, e.g. "my mother and my friends say X is reliable while Y is untrustworthy". The more people that _they_ trust they hear it from, repeatedly, the more they view X as reliable. The blockchain has no such reputation except possibly among certain techies, which is a tiny portion of the population.

The blockchain may allow a group of experts to verify truth, but this only impacts the population at large if somehow the opinions of these experts propagate to the larger population. _That_ step is a marketing and communication problem.

For example Nutricia, the company behind the Dutch Nutrilon milk powder that Chinese are buying en masse, opened factories in China and sold Nutrilon directly in China. They've been doing this for several years now, yet demand for Nutrilon produced in the Netherlands remains high. Why? Because people don't trust that the Nutrilon produced in China is not a scam. Only after mass marketing campaigns, e.g. hiring celebrities as spokespeople, are people beginning to trust it.

<comment><USER3> Hey there FBW, long time no see :)

I think you're mis-judging the intended userbase of an app like this. The word "blockchain" has unfortunately become more and more diluted over time as people explore use cases further and further away from technology. In some cases it ends up meaning little more than better living through technology.

Most developed countries have supply chain tracking systems that let regulators and industry investigate supply chain problems. Mobile apps to let you find the story of your personal steak are a bit of a gimmick. It's better to think of these systems as debug logs, intended to let professionals isolate problems in the pipeline.

These systems are of course only as good as the data that goes into them. They aren't silver bullets or whole solutions. But if you can get to a situation where most suppliers are honest, they can help guide investigations towards the suppliers that are dishonest. The UK horsemeat affair shows how investigators were able to track down the dishonest suppliers and, crucially, assemble enough evidence to do something about it.

The simple and obvious way to implement supply chain tracking is a big central web app with database. But it leads to common problems:

• Who builds this system?

• Who pays for it to be run?

• Can you trust the administrators?

• If an administrator is corrupted, can you recover?

The traditional answers are "the government via a small set of large public sector IT contractors", "industry via tax" or "industry via payments", "hopefully" and "no".

You can also get private sector solutions, but these tend to struggle to get wide adoption because industries fear being held in hock to a monopoly.

These outcomes are not ideal. They aren't standards based or open source, individual users can't really improve their experience of using the system, data is concentrated in a single place where it's ripe for hacking and manipulation, digital signatures are used rarely if at all, and in general it's just not ideal.

Modern blockchain-inspired systems like Corda, which I work on, are actually quite different to Bitcoin and Ethereum. Data isn't broadcast everywhere, there's no mining, etc. If you want you can think of them as document management systems. You have versioned digitally signed data structures with a PKI to root signatures in real world identities, a P2P network for businesses and organisations to run complex workflows over and so on. These are useful for managing large, distributed data management operations, for managing the evolution of data across many different parties and so on.

Ultimately, Chinese people will not learn to trust their food through an app. Instead China will develop the same sorts of largely invisible systems that create an ambient trust in the food system, and you won't need celebrities anymore. Software can play a part of that.

<comment><USER4> This is such a great response. Thinking of blockchains as a decentralized document management system for managing the evolution of data makes a lot of sense and really articulates the value proposition of Blockchain.

Thank you for this response.

</comment></comment></comment><comment><USER5> There is a robust meat certification process in the UK. Every piece of meat sold has to be able to be traced back to the cow it came from.

The system is centralized, and has many systems in place to stop tampering.

Yet..

[https://en.m.wikipedia.org/wiki/2013_horse_meat_scandal](https://en.m.wikipedia.org/wiki/2013_horse_meat_scandal)

This is the issue, the block chain stops tampering, but does nothing at all to make sure the data entered into it is valid in any way shape or form.

You are targeting the wrong problem.

</comment><comment><USER6> I don't get it.

Who are the participants in the blockchain and how do you ensure it cannot be tampered with i.e. could someone just go back to a previous block and rewrite history. Difficult admittedly. I don't understand what a consensus means in this implementation.

Also, it sounds like the trust is completely in the hands of one entity i.e. the person who adds details about the cow and cut of meat. How is that trustworthy ?

Either, the article doesn't explain how blockchain could fix a key problem in China's food industry, or it doesn't fix a key problem.

In my opinion food safety has to be regulated by a central agency like a Government. A decentral system like Blockchain isn't what solves the problem. Regulations and process are.

</comment><comment><USER7> As many mention, the system is not infallible because data entry is as safe as those who control the input of it; it does not increase trust directly. However, what it does add directly is accountability. If each process, is suppose, transmitted automatically by NFC bundled crates through the supply chain, you have a guarantee of the process through the checkpoints (even if it may deviate) with the information and status of each crate, all the way to the store. More so, with the variety of information, such as average supply times, it becomes very difficult to defraud the next check in the supply chain. In addition, with the transparency of the blockchain the store can better determine the integrity of the entire process, as the actors now begin to approach perfect information. Further, now to defraud, becomes much more expensive as an effect of the checkpoints and metadata; this should further discourage defrauding.

Even better, with all this accountability, the system does becomes more trustable; the integrity of each actor is known and can be investigated independently (i.e. stores and consumers can choose which supply chain they want to be apart of because of this transparent blockchain). Not to forget, that with accountability becomes increased shared liability that then aligns each actor in the supply chain further to constantly check the integrity of the actors involved––or at least press the risk of an integrity inquiry.

</comment><comment><USER8> Why blockchain doesn't solve the actual problem: you literally need human inspectors who know the scams. [https://twitter.com/a_ferron/status/896795477844389888](https://twitter.com/a_ferron/status/896795477844389888)

This is just a bad "but with blockchain!!" puff piece. You would probably believe how many of these I read ... It's "here's a problem!" (problem may or may not exist) "Blockchain will solve it!!" (it won't even if it does exist)

Note that nothing about the actual problem is automatable - it's not about tagging the items, it's about the _quality for humans_ of the tagged items. The blockchain approach relies on the assumption that the current crooks will suddenly tell the truth when they're entering data into the system, rather than _e.g._ lying, or bribing inspectors, like they do now.

The only thing that _actually works_ is to (a) give a hoot (an important prerequisite) (b) directly inspect your suppliers.

 _e.g._ Provenance Inc. is trying to sell this stuff in the UK, but is getting very little traction, because putting it on a blockchain doesn't solve any of the actual problem, and their prospective customers (the people in the middle and end of the supply chains) are going "what on earth, this doesn't solve any problem we have."

I wrote a book chapter on this ...

</comment><comment><USER9> I run a foodtech startup in China - [http://infinite- food.com/](http://infinite-food.com/) \- and previously designed a major cryptocurrency exchange.

What's the state of the market? If you read, for example, the last annual general report of _Yum! Brands_ (who run KFC, Pizza Hut, Taco Bell and some other chains in China + have EBITDA ~USD$0.5B/year+) they basically say "yeah, supply quality issues happen, but basically we can't prevent them even if we spend up big, so we just do periodic inspections, value win-win long term relationships with suppliers, and have reasonable levels of paranoia". That's the reality... and they have more money and lawyers to throw at the problem than the rest of us.

On the consumer side, they want to know there's low to no pesticide residue on their veggies and that the meat they're eating is free of hormones and is what it's supposed to be, but that's not something Blockchain necessarily has anything to do with as it actually needs a fairly constrictive government or private sector testing regimen which will cost money and (under user pays) will rapidly be ignored and sidestepped by the bulk of the market. The reality is that consumers _can_ buy organic vegetables and imported meat already: they just don't because it costs more.

IMHO then, in the Chinese market this sort of stuff realistically most probably falls in to the 'marketing gimmick to make my brand appear more trustworthy' category ... and is likely quite effective.

<comment><USER10> Exactly, no matter how reliable the proof in the blockchain is, it's no more reliable than the data that was encoded in.

</comment></comment><comment><USER11> Don't get it?

How is it different to a central body with one database where you lookup if this product has passed all the checks / tells you its lifetime info?

What problem can it help with?

<comment><USER12> The main advantage is that at the central body, somebody may take bribes to edit entries after the fact or to tamper the data. With a (proper) blockchain that's (mostly) impossible.

<comment><USER13> How can they attach an ID to a specific food and ensure that it hasn't been duplicated etc? With crypto, it's easy, but if they're just attaching a label to a box it seems ripe for fraud.

</comment></comment></comment><comment><USER14> I wonder if they've experimented with image analysis software to categorise the carcass at the slaughterhouse. It's done in Ireland to good effect[1] to grade the carcass to ensure compliance with the European beef grading system (beef and dairy farming is an important industry in Ireland). It doesn't guarantee fraud prevention, but it could increase consumer confidence in beef quality.

[1] [https://www.agriculture.gov.ie/farmingsectors/beef/eubeefcar...](https://www.agriculture.gov.ie/farmingsectors/beef/eubeefcarcaseclassificationscheme/)

</comment><comment><USER15> So...I'm not sure how this will stop the fraud.

All someone has to do is payoff a data entry clerk to generate the false backstory and data that is encrypted in the blockchain. In fact, once this illegitimate data is preserved in the blockchain it will be given _equal_ weight (or at least it should) as other legitimately generated data unless you have a way to dispute the data stored in the blockchain -- which then means you now have a __more __authoritative source so why do we need the blockchain

</comment><comment><USER16> This sound like just another case of misunderstanding the core technology. I can recommend watching: [https://www.youtube.com/watch?v=SMEOKDVXlUo](https://www.youtube.com/watch?v=SMEOKDVXlUo)

</comment><comment><USER17> probably should rtfa, but all china needs to do is bring back corporal punishment en masse for those that take lives or sicken others for profit, blockchains or hyper ledgers with group verification doesnt help.

</comment><comment><USER18> So... are there people mining CowCoin to make this all work?

</comment><comment><USER19> Blockchain could fix a key problem in X</comment>


64% rise in rental properties across Dublin in midst of Covid-19 crisis - <AUTHOR> https://twitter.com/RobCross247/status/1241325290565111808 <comment><USER2> Ha. Good luck getting paid when people are about to run out of money.</comment>


Mouseless workflow on the Mac - <AUTHOR> http://andrew-landry.com/mouseless-workflow-on-the-mac/ <comment><USER2> Divvy is the main tool that is missing from the list that I consider mandatory for avoiding the keyboard in OS X. It lets you assign keyboard short cuts to window positions and sizes.

<comment><USER3> There's a lot of different window managing tools on OS X. I personally use Optimal Layout, but there's a number of open source ones as well, such as Slate [1] (which unfortunately hasn't been updated in a while), Mjolnir (nee Hydra) [2], and Spectacle [3]. Both of the last two were updated a few months ago.

Spectacle is a bit easier to use than Mjolnir, but it looks like Mjolnir is more flexible. In comparing Mjolnir to things like Divvy, the Readme states:

"Mjolnir is intended for programmers who want to write programs that customize their environment. It's not intended to be a drag-n-drop solution; it's meant to allow you to write your own personalized productivity enhancement suite to keep and to use long-term."

[1] [https://github.com/jigish/slate](https://github.com/jigish/slate) [2] [https://github.com/sdegutis/mjolnir](https://github.com/sdegutis/mjolnir) [3] [http://spectacleapp.com](http://spectacleapp.com)

</comment></comment><comment><USER4> Really nice post. Shortcat is really great!</comment>


Shakespeare.txt.jpg: A JPEG compression experiment - <AUTHOR> http://www.tomscott.com/romeo/ <comment><USER3> As some comments are arriving below -- lossy compression does not work that way!

Interpreting one set of data as another is a clever trick for puzzles (thematic, for example, to: [http://web.mit.edu/puzzle/www/2013/coinheist.com/oceans_11/t...](http://web.mit.edu/puzzle/www/2013/coinheist.com/oceans_11/this_page_intentionally_left_blank/index.html) \-- but that's only a hint, the solution is still lots of fun!) but is kind of wrong in practice.

Compression is about conveying a similar 'message' while removing the stuff that just doesn't compress well. In signals such as pictures and audio, this is often seen/heard in high frequency detail.

To the layman, the simplest lossy compression of an image is to resize it by a half in each dimension and then stretch the output. (Not unlike retina-display artifacts you may have seen). Assuming you do it right (with an averaging filter) -- you've just removed the upper half of the representable frequencies, it just so happens that most of what the eye cares about isn't in those ranges -- and so the image still looks decent. A little blocky if you stretch it, but still pretty good. JPEG is just a more technically advanced version of similar tricks.

A better analogy of lossy compression on text is actually xkcd's Up Goer Five ([http://xkcd.com/1133/](http://xkcd.com/1133/)) -- using only the "ten hundred" most common words, you can still convey concepts (albeit simplistically) and stay under ten bits a word, or a little less than two ascii characters per. If you were to map the dictionary into "up-goer five" speak, you could compress Shakespeare pretty well. It would lose a lot in the translation -- but so does a heavily-compressed image. If you limit yourself to the first 64k words, you have a much larger vocabulary, still limited, and still fitting within two bytes per word. Though you may have to use more words, a word like "wherefore" still counts for four words worth of compressed space, when replaced by "why". Tradeoffs!

That'd be a curious hack -- sort the words in Romeo and Juliet by word frequency. Rewrite the least common words in terms of the other words. Compress.

<comment><USER4> _If you limit yourself to the first 64k words, you have a much larger vocabulary, still limited, and still fitting within two bytes per word._

Maybe I'm misunderstanding what you're saying but if you're compressing adaptively by building a dictionary of the target material, 64k words would give you lossless compression of Shakespeare - the unique words in the collected works (with some wiggle room for 'word forms') are in the 30-40k range.

<comment><USER3> True! You could do that too. Sending the codebook is a cromulent strategy. It'd be lossless too. That's (to a first order) how LZ works.

I was arguing you can use a standard codebook (some agreed-upon top-N list) instead. I imagine if you did you'd lose a couple words ("wherefore" springs to mind, as it's not in common English usage) but by and large you'd have just about everything.

Technically, you can also have a lossless JPEG (or at least, within epsilon), but that's not why it's done.

More importantly, the entropy of English text isn't really that high -- ~ 1 bit per letter -- which means lossless compression works pretty well on it. Images less so.

The lesson here being that compressing English text as suggested in the article is rather meaningless, and can be done much, much better. When the article asks at the end:

> We're sensitive to data loss in text form: we can only consume a few dozens > of bytes per second, and so any error is obvious.

> Conversely, we're almost blind to it in pictures and images: and so losing > quality doesn't bother us all that much.

> Should it?

My answer is "mu". I wanted to answer "no", but that would mean I even accept the question as valid.

EDIT: s/in this way/as suggested in the article/ for clarity.

<comment><USER4> I suppose my confusion arises from the fact that pretty much any practical compression scheme is essentially 'content adaptive' rather than 'standard codebooky'. In the lossy cases, add whatever magical filtering to toss out the perceptually irrelevant frequencies.

I agree with you that the article misunderstands compression and coding in general I just couldn't quite figure out what one of your counter-examples was about.

</comment></comment></comment></comment><comment><USER5> Hello there! I'm Tom. I made this. I didn't expect it to go this far. A few notes for you, which I've also added to the original site:

Yes, I'm aware this is pretty meaningless - my original tweet was "Not sure if it's net-art or just a load of guff." It seems to have sparked some discussion, though.

No, I'm not selling the books. I try not to create too many useless physical objects. If you're hell bent on it, or want to see the full text, I've uploaded all the source files and there's a link at the bottom of the page. I ask that you don't sell anything you make with it, although given I'm ripping off Shakespeare and the JPEG algorithm that's probably a bit rich.

If anyone wants to commission a professionally bound hardcover set for some net art exhibit, let me know. I'll happily pretend to be a serious artist.

</comment><comment><USER6> The whole reason this analogy breaks down is that letters are independent symbols while levels in an image represent a continuum. There's no mathematical correspondence among letters to exploit, only an accident of encoding. Apples to oranges.

That said here's a particular rant transformed as a whole signal with the DCT and quantized a bit [http://pastebin.com/B5zS8W8f](http://pastebin.com/B5zS8W8f)

</comment><comment><USER7> I'm not sure you should draw any meaning at all from this, whether technical or philosophical. JPEG is an image specific compression format, and works by removing visually insignificant detail. Applying the same system blindly to an unrelated byte-stream and then looking at the before and after isn't meaningful.

Those objections aside, it's a bit silly from a technical point of view - <USER2> text was read in as a RAW image format. As I understand it, RAW has to be converted into a bitmap image using a camera specific algorithm, so before he's even started compressing it the data has been munged, the data is then compressed and output as a completely different file format. I'm surprised he got anything recognisable at all.

Putting bitmap type file headers on the text data, then jpeg compressing it, and resaving it as a bitmap (and removing the headers), might produce something slightly more meaningful. I might try it later - curious to find out what Singular Value Decomposition looks like on text.

Actually, could just skip out Photoshop... _cracks knuckles and fires up Python_

<comment><USER8> Yea, it would be more interesting if he would have compressed the text with an algorithm that replaces similar sounding letters, using rules like in the SoundEx algorithm.

<comment><USER9> Or may a zip-style algorithmn, but as the "quality" setting gets lower, unique/low frequency sequences are replaced by the closest common sequence.

</comment></comment></comment><comment><USER10> That's an interesting idea, and it's cool that he took it as far as he did, but I'm surprised by the last paragraph:

 _We 're sensitive to data loss in text form: we can only consume a few dozens of bytes per second, and so any error is obvious. Conversely, we're almost blind to it in pictures and images: and so losing quality doesn't bother us all that much._

I'm not aware of any "lossy" encodings for text, but I assume they'd do pretty well on shakespeare and pretty terrible on images encoded as text. It's just that the techniques are tuned for the standard data, so most things you smuggle through are going to be "weird" and suffer more than usual.

<comment><USER11> You could probably quite effectively do 'lossy' compression on text given enough knowledge of the language. Compress words with similar meanings into each other, automatically select variants of words based on context (plurality, etc), automatically insert punctuation, and so on.

The result would probably look like it was written by someone with an incomplete knowledge of English (or by a machine) but might end up being quite readable.

Of course, on the other hand, you can just compress the text with LZMA :)

<comment><USER12> it's comomn kdownlege in inoitrmfaon thoery ciecrls taht you can reoerdr lttres in wrods and the relsut is pecfretly undeastnrdlabe as lnog as you keep the fsirt and lsat letrtes the smae.

[http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=4126...](http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=4126417)

<comment><USER13> That is more about built-in redundancy and error correction than lossy compression.

<comment><USER14> But such features could be exploited to make lossy compression. If humans can recognize words with some of the letters randomized, then there is no need to store those letters individually. Instead just insert random letters at decomperssion phase.

</comment></comment></comment></comment></comment><comment><USER15> I recently finished a similar project in computationally mutating a familiar text: [http://www.wellsortedversion.com](http://www.wellsortedversion.com)

</shamelessplug>

<comment><USER16> Your claim that the sorted version keeps all the information is false-- it is impossible to reverse the transformation.

I'd love to see this with the Burrows-Wheeler-Transform (suffix sorting), which doesn't actually lose any information-- it might even be possible for a human to (very) slowly read it.

</comment></comment><comment><USER17> "we can only consume a few dozens of bytes per second, and so any error is obvious."

That's not the point at all. The English language itself is already heavily compressed, by which I mean the space of all possible words is already densely packed. _This_ is why we can both understand misspelled words with no particularly close neighbors, but are nonetheless sensitive to misspellings in general.

The act of writing something down in language compresses it, significantly changing properties of the space such as comparability of neighbors

</comment><comment><USER18> So, this is a fun bit of tinkering and all, but I'd be more interested to see just how small HN could losslessly compress the Gutenberg plain text UTF8 file of Romeo and Juliet.

([http://www.gutenberg.org/cache/epub/1112/pg1112.txt](http://www.gutenberg.org/cache/epub/1112/pg1112.txt))

Especially if you're using interesting techniques to build the dictionary.

EDIT: I got it down from 178983 to 44830 bytes, which isn't as impressive as I was hoping for.

EDIT2: Currently at 43,666 bytes which is a bit better.

EDIT3: 40,125 bytes.

<comment><USER19> You'd want to count the size of the decoder, like [http://en.wikipedia.org/wiki/Hutter_Prize](http://en.wikipedia.org/wiki/Hutter_Prize)

<comment><USER18> Well, yes. Except the source text is tiny, so unfortunately all the software is pretty big in comparison.

I love the Hutter Prize, and the various compression benchmark sites.

<comment><USER19> It's just pretty unconstrained without that kind of rule. And a decompressor that doesn't have to be fast can be tiny.

</comment></comment></comment></comment><comment><USER20> If even max quality has that many single character shifts I'm worried that this is barely demonstrating compression at all, and is instead mostly demonstrating color space conversion.

</comment><comment><USER21> My brother did something similar a number of years ago.

[http://whaleycopter.blogspot.com/2007/09/project-jteg- compre...](http://whaleycopter.blogspot.com/2007/09/project-jteg-compression- format-for-ink.html)

It's interesting to see the same ideas pop-up over and over again. (Note the patent-pending message is a joke).

</comment><comment><USER22> ??? JPEG uses the redundant(or neglect-able) information in 8x8 blocks. I do not see how text is information of 8x8 blocks with neglect-able information within this block...

<comment><USER23> It's not. I think it would have been more insightful, if they outputted the text file to WAV instead, and then compared the audio quality of MP3 compressed version of the WAV file. Bring the Noise. Only true audiophiles can hear the difference.

Next up, the startling results of outputting a text file to random locations in RAM...Basically the author was just bored.

</comment></comment><comment><USER24> They say a picture is worth a thousand words. Thus, the more damage you do to a picture, you might get that thousand number lower, but it still retains a lot. A word is worth one, and so, even changing it slightly removes all meaning.

</comment><comment><USER25> Very neat idea. Would love to have those bound copies....

Reminds me of Nethack, where text in the dirt degrades every time you walk over it.

</comment><comment><USER26> It's too bad the full text isn't posted anywhere. I'd love to take a more in- depth look at the results.

<comment><USER27> He describes the process: treat text file as RAW image file, read into picture editor, save as .jpg with varying compression. Abuse your preferred example text accordingly.

</comment></comment><comment><USER28> > We're sensitive to data loss in text form: we can only consume a few dozens > of bytes per second, and so any error is obvious.

We've also optimized text to be pretty robust in its native form, such that all of those bits have a high chance of getting through successfully; for example, no writing system requires color to carry semantic weight. Therefore, text can be displayed successfully with monochrome technology, and is robust against color changes as long as contrast is preserved.

Also, every writing system has a fixed repertoire of atomic units (characters), even if some of them have a huge number of them. This makes writing robust against small changes as they're more likely to turn a well- formed character into a badly-formed version of the same character, as opposed to a different character entirely. This, of course, is where JPEG breaks the rules somewhat, because its idea of a 'small change', when applied to a text file, will _always_ produce an entirely different character, usually with no obvious relationship to the original. The properties are entirely different.

> Conversely, we're almost blind to it in pictures and images: and so losing > quality doesn't bother us all that much.

Images are captured and displayed largely irrespective of human perceptual limits. Lossy compression is, in fact, a way to recognize the limits of human perception and take them into account by throwing away information likely to be ignored anyway. For example, human hearing tops out at around 22 kHz, so a sampling rate somewhere north of 44 kHz will, mathematically, capture everything about the sound humans are interested in. Had we canine perception, our sampling rates would be higher, but we could use fewer bits to encode the colors in color photographs.

Text is inefficient in its uncompressed form, but human perception of that type has little to do with that. It's inefficient because our language is redundant, which is a boon when we're trying to talk in a crowded café and must accept some noise along with the signal.

A better test would be to take a picture of a sign, then edge-detect to bring out the letter forms at maximum contrast, pound it down to monochrome, and then compress that at maximum lossage. I guarantee it would still be readable.

</comment><comment><USER29> This "experiment" is garbage. No description of test methodology other than "then outputted the compressed file back to plain text."

Last time I had to use Omnipage to convert some highly compressed .JPEGs, it blew me away with it's accuracy. Not only did it get the text nearly 100%, it often got the tables right too. The only thing it struggled with was advanced mathematical symbols.

<comment><USER30> The methodology seems obvious - interpret every three bytes in the text as a pixel in an image, compress the image, then reinterpret each (post- compression) pixel as three ASCII characters.

This is not about OCR.

<comment><USER29> Thanks. Great explanation. Considering that the ASCII character map is arbitrary and there are so many great ways to test lossy compression algorithms( PSNR to A/B testing), this seems like a pointless exercise, but at least I see what the author was trying to do now.

If I wanted to demonstrate lossy compression, I'd use Image Subtraction plugin in GIMP, not ASCII.</comment></comment></comment>


Show HN: ProductMap – A change log as a service - <AUTHOR> http://productmap.co/beta <comment><USER2> Neat looking product. How much is this going to cost once it's out of beta?

</comment><comment><AUTHOR> We built ProductMap to have a place to tell users about new features, updates, and product announcements.

Down the road we'll be adding things like voting, user feedback, and automatically posting to social media.

Happy to hear feedback!

<comment><USER3> Cool, very useful.</comment></comment>


Show HN: PZ-100 – A small JavaScript puzzle game packaged as an NPM module - <AUTHOR> https://www.npmjs.com/package/pz-100 <comment><AUTHOR> I had this idea of building a puzzle game that you import as a module and play inside the Node REPL. I hope that you enjoy playing it as much as I enjoyed creating it!</comment>


Mozilla accidentally posts usernames and password hashes - <AUTHOR> http://www.tgdaily.com/security-features/53267-mozilla-accidentally-posts-usernames-and-password-hashes?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+tgdaily_all_sections+%28TG+Daily+-+All+News%29 <comment><USER2> _newly created passwords remain invulnerable to a similar disclosure, as they employ SHA-512 with per-user salts to store hashes._

I don't know how long it will take for people to figure this out. SHA-512 with a salt is _not_ the right way to store passwords. Yes, it's better by leaps and bounds than storing the plaintext. Yes, it's better than crypt(3) as gawker used. But come on, just use bcrypt already.

And the fact that they calls SHA-512 with salts "invulnerable" ... I don't even know what to say to that.

<comment><USER3> I don't think a developer working at Mozilla would be so careless that they would use "invulnerable" to describe their security. Seems like it was an interpretation by the tgdaily author.

Here's a link to the original blog post: [http://blog.mozilla.com/security/2010/12/27/addons- mozilla-o...](http://blog.mozilla.com/security/2010/12/27/addons-mozilla-org- disclosure/)

<comment><USER2> Yeah, my "they" was really unclear. I'm sure that no developer in his or her right mind would ever call anything "invulnerable." That's just asking for it.

I guess it makes sense that the media would call something like that invulnerable though, they don't know anything about it anyways.

</comment></comment></comment><comment><USER4> Does the problem with posting these hashes have _anything_ to do with "cryptographic weaknesses in MD5", or does Sophos' security expert simply have no idea what he's commenting to the trade press about?

<comment><USER5> It has a lot to do with the weakness of MD5. You can compute the md5 has of every 6 character lowercase string in seconds on a modern laptop. There are also vast rainbow tables available. However mozilla have disabled all the accounts that used MD5 anyway. Unfortunately this does not prevent the real risk, which is to people who reuse their password elsewhere.

Still, other hash functions such as the SHA512 they are currently using are also wildly inappropriate, as explained in the following excellent post: <http://codahale.com/how-to-safely-store-a-password/>

<comment><USER4> Speed isn't a hash function weakness. Collision susceptibility is, but that doesn't really help you crack passwords.</comment></comment></comment>


The need to protect the Internet from 'astroturfing' grows ever more urgent - <AUTHOR> http://www.guardian.co.uk/environment/georgemonbiot/2011/feb/23/need-to-protect-internet-from-astroturfing?CMP=twt_gu <comment><USER2> How do we detect the astroturfers and shills? It's pretty obvious that sites likeZDnet.com, techdirt.com and groklaw.net are targeted by some fairly well- heeled trolls, but what about the more subtle shills? How can we detect and banish them?</comment>