The SILE Typesetter: Simon's Improved Layout Engine - <AUTHOR> https://github.com/sile-typesetter/sile <comment><USER2> The manual seems to quickly disqualify the system. The numbers don't line up in the TOC in horrible ways. Then a little ways down there's a wrap figure that's got a horrible spacing problem. These indicate to me that the shiny new foundations are really quite wobbly.

<comment><USER3> Wow, I thought you were exaggerating but it really is _terrible_. Page 15 of the manual is inexplicably 90% blank too.

So it's clearly just a _super_ -rough prototype, which would be fine except that when it brands itself as "its job is to produce beautiful printed documents"... it gives the impression that it already does this, when the reality is clearly far, far away.

I think the project needs to be honest about where it's at.

[1] [https://sile-typesetter.org/manual/sile-0.10.2.pdf](https://sile- typesetter.org/manual/sile-0.10.2.pdf)

<comment><USER4> Well it doesn't even have a 1.0 release so that's enough to cut them slack.

For instance, the foundation of the house is most critical but when everyone looks at the foundation and asks, "where's the house? This house can't be much they brand themselves as a beautiful château but all it is is a big hole in the ground..."

A project like this is no minor task! :)

<comment><USER5> >Well it doesn't even have a 1.0 release so that's enough to cut them slack.

I mean, it's at 3000+ commits and 5+ years so it's not exactly a new project. There's something to be said for doing fewer things but doing them really well.

</comment></comment></comment></comment><comment><USER6> I admit I'm prejudiced by the author beginning the README by boasting about how "beautiful" their own creation is, but I'm dismayed at the thought of trying to do preservation and archival work on documents whose rendering changes every time Harfbuzz has a mandatory security update.

I was hoping to be able to soften this criticism by saying "at least it looks better than the output of the last layout engine I wrote", which would not be a hard bar to clear ([http://canonical.org/~kragen/dercuano.20191230.pdf](http://canonical.org/~kragen/dercuano.20191230.pdf) is full of egregious typographical sins), but [https://github.com/sile- typesetter/sile/commit/e72aafd58f855...](https://github.com/sile- typesetter/sile/commit/e72aafd58f8550815cb89c0bbd6002f16771a65d#diff-726110337c775dda423f7ea2d8fdfc53) (Liddell & Scott, I would have thought? Although it has no title page, table of contents, or even headwords, and [https://archive.org/details/greekenglishlex00lidduoft/page/x...](https://archive.org/details/greekenglishlex00lidduoft/page/xvi/mode/2up) contains about 20× as much text) seems to be missing most of the definitions and doesn't even look that good. The first entry, αδην, defined by Liddell & Scott as "to one's fill" with 12 lines of elaboration, reads simply, "αδην adv. .". (Except with the proper diacritics, of course.)

That's probably just a bug or something, but I am therefore unfortunately unable to soften my criticism with such a compliment.

<comment><USER7> The test example looks alright: [https://github.com/sile- typesetter/sile/blob/master/examples...](https://github.com/sile- typesetter/sile/blob/master/examples/test.png)

The kerning doesn't look great, but that's the fonts (DejaVu Sans + Libertine AFAICT).

<comment><USER6> The test example literally has a double hyphen "\--" in place of an em dash, which begins a line, provoking an unjustified momentary appearance of being an utterance by a character quoted in the French or Spanish style; and the author's name in body-text type right-aligned at the top of the page looks like a typographical error. Still, it's eminently readable, with reasonable hyphenation. My typographical sins in the PDF rendering of Dercuano were, in many cases, far worse (no hyphenation at all!), but I hacked together Dercuano's PDF renderer in 5 days as a last resort, and I don't claim that the result is "beautiful", although I wish it were.

Thank you helping me be slightly less of an arrogant jerk!

<comment><USER8> The faux em dash definitely caught my eye. The hyphenated "ory" on its own line at the end of a paragraph is another really bad look.

If these are the examples to pitch the system... it doesn't seem ready for primetime. That's OK. Every project has to start somewhere, but the README could stand to frame that a little better.

<comment><USER6> Yeah, I wouldn't want to influence the author to stop working on typesetting, but I found the arrogant boasting really annoying.

</comment></comment></comment></comment></comment><comment><USER9> I think TeX is terrible, so keep it up please, but the examples all look really bad. If you want to sell someone on "this system will help you make _beautiful_ documents", the examples need to wow.

<comment><USER10> Not sure if TeX is "terrible" or not, but the syntax here looks very similar.

On a practical note, it may be worth exploring other options: groff and TeXMacs (no affiliation with TeX or emacs).

<comment><USER11> Lout[1] might also be worth a look.

[1]: [https://en.wikipedia.org/wiki/Lout_(software)](https://en.wikipedia.org/wiki/Lout_\(software\))

</comment></comment></comment><comment><USER12> Is there a bug bounty program for SILE? Does the payout double for every bona fide bug reported?

I'm sympathetic to the motivations that would lead someone to write a new typesetting engine. The problem is that in terms of essentially bug-free behavior and being completely understood and documented, TeX sets a bar that, despite its age and certain kind of clunky, is very, very high.

</comment><comment><USER13> Related from 2014: [https://news.ycombinator.com/item?id=8392653](https://news.ycombinator.com/item?id=8392653)

A bit from 2016: [https://news.ycombinator.com/item?id=12411446](https://news.ycombinator.com/item?id=12411446)

</comment><comment><USER14> The first thing I want to know about a typesetting system is: does it do microtype? And are there any innovations at that level? I don't see any mention of this in the "SILE vs TeX" section of the manual, but to me it seems like the most important thing I'd need to know up front. After all, if I'm not going to put in the effort to make a document look _really_ good, why bother with a typesetting system at all?

Practically speaking, I think if they wanted users to adopt this, the best thing they could do would be to write an output format for Pandoc or similar. I could try this today if such a mode were available.

<comment><USER15> What is microtype?

<comment><USER16> I believe it's referring to small intercharacter typesetting details like tracking and kerning.

From the microtype package for latex:

"The microtype package provides a LaTeX interface to the micro-typographic extensions ... : most prominently, character protrusion and font expansion, furthermore the adjustment of interword spacing and additional kerning, as well as hyphenatable letterspacing (tracking) and the possibility to disable all or selected ligatures."

</comment></comment></comment><comment><USER17> Does it have a context-free syntax? Because this is basically the only problem with TeX: You cannot parse it.

<comment><USER18> What does context-free syntax mean? And what does "you cannot parse it" mean? Meaning you can't stream it, you need the whole file first?

<comment><USER19> Tex is like a programming language, you have to "run" the program to generate output, and that execution is tightly tied to producing output on paper pages. This means it is extremely difficult to produce good looking HTML, or accessible output.

</comment></comment></comment><comment><USER20> Check out python-typesetting for another alternative.

[https://github.com/brandon-rhodes/python- typesetting](https://github.com/brandon-rhodes/python-typesetting)

</comment><comment><USER21> I went to the the website and looked at [1]. Both examples there show complete lack of understanding of typography and typesetting. I don't even know where to start.

Tex is no fun to use but this system is years away from being an alternative.

[https://sile-typesetter.org/examples/](https://sile-typesetter.org/examples/)

</comment><comment><USER22> No math, sadly.

<comment><USER23> There are several tickets related to math handling. See [https://github.com/sile-typesetter/sile/pull/578](https://github.com/sile- typesetter/sile/pull/578) and other tickets referenced there.

</comment></comment><comment><USER24> See also a previous discussion here: [https://news.ycombinator.com/item?id=13674879](https://news.ycombinator.com/item?id=13674879) for Patoline, another LaTeX alternative (written in OCaml).

</comment><comment><USER25> Is this embeddable?</comment>


How HTTPS Handshake Happens - <AUTHOR> http://sudhakar.online/programming/2015/08/09/https.html <comment><USER2> As others have already pointed out, this explanation focuses on the RSA key exchange, which has been deprecated. It's not recommended for use with the current line of protocols (TLS 1.2 and earlier) and it's been completely removed from TLS 1.3 (work in progress, but close to being finished). The key weakness of the RSA key exchange is that session encryption keys are transported over the network encrypted with the server's public key. When the server's private key is compromised in some way, the attacker can passively decrypt all traffic, they can also "go back in time" to decrypt all previous conversations they might have recorded and kept.

For a more secure handshake approach, consider the Diffie-Hellman key exchange, which is a way for two parties to agree on a secret number (a very large number when used for encryption, of course) that third party can't discover from looking at the network traffic. It's a very simple and intuitive algorithm; there's a great video that explains it here: [https://www.youtube.com/watch?v=YEBfamv- _do](https://www.youtube.com/watch?v=YEBfamv-_do) Today, we generally use the ephemeral elliptic curve DH variant (ECDHE), which has better security characteristics and works much faster.

However, DH is not sufficient, you must also authenticate the other party. Otherwise you could be negotiating your key exchange with the attacker and not the real recipient. This is where public cryptography comes in. After the key exchange, both parties use their private keys to sign the transcripts of the entire conversation up until that point. When this is verified you know that (1) you're talking to the right person and (2) that you have a secret (session keys) that you can use to efficiently protect the conversation.

Disclosure: I am the author of Bulletproof SSL and TLS, already mentioned in another comment here. Feel free to ask me anything.

<comment><USER3> Ivan, I bought your book "Bulletproof SSL and TLS". It's excellent, and I recommend it to those who are charged with building, deploying, or managing a real world deployment that includes TLS.

<comment><USER4> I have yet to buy it, but I've recommended it to five or six people who said they wanted to understand TLS, because I've never heard of someone being disappointed by it!

I think I'll buy my own copy today. :-)

<comment><USER4> As an update, I've ordered a copy and look forward to reading it.

</comment></comment></comment></comment><comment><USER5> If you are interested in details of the TLS protocol, check out these two books:

\- Implementing SSL / TLS Using Cryptography and PKI [1]

\- Bulletproof SSL and TLS: Understanding and Deploying SSL/TLS and PKI to Secure Servers and Web Applications [2]

In the first one the author implements the protocol (RSA/DH) from scratch (without even using any crypto library). The second one is a classic and contains a lot of interesting scripts (the chapter on using OpenSSL and creating your own PKI is available for free: [https://www.feistyduck.com/books/openssl- cookbook/](https://www.feistyduck.com/books/openssl-cookbook/)).

I spent some time studying TLS and wrote two blog posts [3][4], in which I decrypt the network traces of the TLS sessions. Maybe someone will find them interesting too.

[1] [https://www.amazon.com/Implementing-SSL-TLS-Using- Cryptograp...](https://www.amazon.com/Implementing-SSL-TLS-Using- Cryptography/dp/0470920416)

[2] [https://www.amazon.com/gp/product/1907117040](https://www.amazon.com/gp/product/1907117040)

[3] [https://lowleveldesign.org/2016/03/09/manually-decrypting- ht...](https://lowleveldesign.org/2016/03/09/manually-decrypting-https- request/)

[4] [https://lowleveldesign.org/2016/05/10/tls-1-2-aes-gcm-and- ne...](https://lowleveldesign.org/2016/05/10/tls-1-2-aes-gcm-and-net-network- trace/)

<comment><USER6> I haven't read the books, and I'm sure that they are fascinating, but I am wary of any attempt to home-brew crypto. I'm specifically worried that some corner-cutters might use the Implementing SSL book's code or ideas in production.

<comment><USER7> What's the alternative, lock all books related to cryptography lest someone might do something in production? Educating is fine, and people reading relatively advanced technical books like this one should be considered as responsible, I think.

<comment><USER8> Not only reasonable, but it is a must. I think the "don't roll your own crypto" phrase is widely misunderstood. A better phrase would be - learn as much about crypto as you possibly can, to clearly understand why it is a bad idea to roll your own in production. This also means that after you learn it, you will have a much better idea which library, cypher or mode to pick for a particular task, and not just think "oh, crypto is complicated, i will just wrap everything in TLS and won't bother to check the certs"

<comment><USER9> People who are wanting to get into complex cryptography today have to start somewhere. Showing working models, with huge disclaimers, in blog posts and on github is a great way to lean, if you invite others to show you flaws and break your implementation.

<comment><USER10> If you want to build complex cryptography, you must either obtain formal training, or learn how to break complex cryptography.

What you can't do is read a book or two about cryptography, the way you would a new programming language, and then bootstrap your own expertise from that. Cryptography is _a lot_ trickier than most other topics in computer science.

<comment><USER11> > Cryptography is a lot trickier than most other topics in computer science.

Is that actually true? Or is it that most software development isn't really about topics in computer science. Like if you wanted to design your own distributed database wouldn't most of what you said above also apply?

Isn't this really a case where "hard topic is hard"?

<comment><USER10> I thought about that when I wrote that comment and my conclusion is, no: distributed systems protocols are notoriously hard, but they don't have _adversaries_. Crypto protocol flaws are marginally more complicated than distributed systems flaws, but also, a distributed system with a byzantine fault can be survivable, and a cryptosystem with a subtle flaw won't be.

<comment><USER11> I'd buy that having adversaries is what makes crypto different than other topics. Thats probably a better way to pitch your message as well.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> There are some really cool "tricks" for avoiding the round trip — round trips are why everyone should be using a global load balancer for SSL. Clients have to send 2 packets across the world and wait for a reply, which can add >100ms before any actual work happens.

http2 helps because you can multiplex a bunch of requests into a single connection, less waiting on new connections to be established.

TLS 1.2 with session resumption lets clients reuse existing sessions after the first connection.

TLS 1.3 has a 0rtt handshake, which is pretty baller. It's just not widely deployed.

<comment><USER13> >"There are some really cool "tricks" for avoiding the round trip — round trips are why everyone should be using a global load balancer for SSL. Clients have to send 2 packets across the world and wait for a reply, which can add >100ms before any actual work happens."

What is a "global load balancer"? A load balancer doesn't avoid any round trips. The "work" of TLS begins as soon as the client sends a ClientHello which is during the second round trip. On a new connection the total round trips is 4 if you include the GET request. It's 3 round trips if you only consider the TCP hand shake and the TLS handshake. This is true whether there is a load balancer or not.

<comment><USER14> I assume a load balancing / caching solution that is available on an anycast IP address. The TLS termination happens at the (ideally) closest point of presence (PoP). The idea is to reduce the RTT from client to its termination point.

Think CloudFlare CDN or the Google Cloud Load Balancer.

 _Edit_ Mistyped RTT as TTL.

<comment><USER13> Sure, you can reduce the RTT by moving the edge closer to the eyeballs but that's not the same as avoiding an RTT as the OP stated. That's what all I was commenting on. There are mechanisms however to do that such as sessions tickets/resumption but that's not something specific to load balancers.

<comment><USER15> The OP didn't claim using a "global" LB would eliminate round-trips, just that you should use one _because_ of the round trips.

<comment><USER13> The OP stated:

>"There are some really cool "tricks" for avoiding the round trip — round trips are why everyone should be using a global load balancer for SSL."

"Avoiding" means not incurring them, so yes they did claim a "global" LB would eliminate round trips.

<comment><USER12> That actually wasn't what I meant to claim, but it's too late to edit! It was an awkward sentence for sure.

The simpler sentence is: minimize round trip cost by getting close to users. Also avoid round trips if you can. :)

<comment><USER13> If you are terminating TLS on an edge network then you are likely caching the content there as well so as not to incur the latency in fetching it from origin. This is called a CDN not a load balancer.

<comment><USER12> There are a bunch of relatively pure global load balancer services available out there. Google Cloud's load balancer, for example. It works pretty well to terminate SSL close to visitors then maintain connections from load balancers back to app instances.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER16> Doesn't cover perfect forward secrecy (PFS).

Without PFS if the server's private key was stolen (e.g. by hacking the server) then all traffic sniffed in the past could be decrypted.

See: [https://en.wikipedia.org/wiki/Forward_secrecy](https://en.wikipedia.org/wiki/Forward_secrecy)

</comment><comment><USER17> Genuinely curious, as HTTPS is something I do not fully understand even with this simplification:

If the browser's symmetric key is encrypted with icicibank's public key, why can't a sniffer unlock it by also requesting icicibank's public key and decrypting the key sharing message?

<comment><USER18> The payload is encrypted with the bank's public key but it can only be decrypted with their private key. This is the basis of public key cryptography [1].

[1] [https://en.wikipedia.org/wiki/Public- key_cryptography](https://en.wikipedia.org/wiki/Public-key_cryptography)

<comment><USER19> When talking HTTPS it's important to note that the protocol uses both asymmetric and symmetric key encryption.

Asymmetric key exchange for secure session setup / authentication

Symmetric key for secure session data encryption

[https://tools.ietf.org/html/rfc4346#section-1](https://tools.ietf.org/html/rfc4346#section-1)

</comment></comment></comment><comment><USER20> Similar articles for reference:

1\. [https://www.ssl.com/article/ssl-tls-handshake- overview/](https://www.ssl.com/article/ssl-tls-handshake-overview/) 2\. [https://msdn.microsoft.com/en- us/library/windows/desktop/aa3...](https://msdn.microsoft.com/en- us/library/windows/desktop/aa380513\(v=vs.85\).aspx) 3\. [https://hpbn.co/transport-layer-security-tls/](https://hpbn.co/transport- layer-security-tls/)

Also note the delay created by each step in (3).

</comment><comment><USER21> I did not see it in the comments, so here is a great article on how the handshake happens : [http://www.moserware.com/2009/06/first-few-milliseconds- of-h...](http://www.moserware.com/2009/06/first-few-milliseconds-of- https.html)

</comment><comment><USER22> Even with TLS 1.3 the fastest handshake for an initial connection is one round trip. That's around 250ms for someone connecting to a server in New York from Sydney Australia. Is there any other way to reduce this handshake time for visitors located geographically far from the server with end to end encryption?

<comment><USER23> Under what circumstances would the _initial_ connection need to be faster than 250ms for arbitrary HTTPS access? I can think of maybe it mattering to people that work with financial data, but outside of that I can't really see it mattering.

<comment><USER24> Loading a bunch of assets from different servers in a serial or semi-serial (e.g. 6 at a time) gets really impacted by that.

<comment><USER25> Loading bunch of assets from different servers to enter a site is a very bad idea.

</comment></comment></comment></comment><comment><USER26> Can someone explain what "any of my trusted keys" in the graphic is referring to? (on the browser)

<comment><USER27> It's impossible to bootstrap a secure connection without some preexisting trusted relationship. Otherwise, you'd always be vulnerable to middle-person attacks.

Browsers solve this problem by bundling a number of trusted root certificates. (This is what they mean by "my trusted keys".)

When you connect to some web site, the server sends you their certificate along with a chain of signing certificates up to some root of trust. Assuming the root certificate is among those your browser trusts, you can verify the signature chain and establish a trusted connection.

</comment></comment><comment><USER28> Little ironic the blog isn't under a valid SSL cert

<comment><USER29> If you actually look at the details, that seems to be related to the blog being hosted on GitHub Pages. GitHub Pages does not support TLS for custom domains.

See also: [https://gist.github.com/coolaj86/e07d42f5961c68fc1fc8](https://gist.github.com/coolaj86/e07d42f5961c68fc1fc8) ("Please petition Github to support HTTPS on github pages") and comments

</comment></comment><comment><USER30> [http://sudhakar.online/visualization/2011/10/11/wedding- invi...](http://sudhakar.online/visualization/2011/10/11/wedding-invite.html)

Found this on this site, parallax.js wedding invitation. This is cool, how hard is it to learn how to do this?

<comment><USER31> Curious how you ended up asking on this thread.

<comment><USER32> If you read his past comments, he's either 15, Indian, or both.

<comment><USER30> So fucking what. I can't ask questions?</comment></comment></comment></comment>


The Coming of Age of Millennials - <AUTHOR> http://www.goldmansachs.com/our-thinking/outlook/millennials/index.html

<comment><USER2> I don't understand the "Beyond the Brand" section. They imply that Millennials find brand less important than others, and in the "The Power of Social" section in the same area of the presentation, they social is growing as the importance of brands shrinks.

Yet looking at the chart in "Beyond the Brand", which shows for each age group when asked about the statement "When I shop, I always try to buy branded products", Millennials had the most people who answered "strongly agree", the most who answered that plus "tend to agree", the least who "strongly disagree", and the least who answer that or "tend to disagree".

The differences between the groups aren't very much, so I question whether there is any actual difference in brand importance among the age groups, but if there is I don't see how they can conclude that it is least among the Millennials.

</comment><comment><USER3> Ugh, take a hike GS.</comment>


Why Love Generative Art? - <AUTHOR> https://www.artnome.com/news/2018/8/8/why-love-generative-art <comment><USER2> The problem with generative art is not that it isn't considered art by everyone. The main problems are:

\- It is very difficult to value. One can't see at a glance whether it took time or skill to create. And similar to other types of art that involve mechanical reproduction, e.g. photography, generative art can be produced on demand, it's not irreplaceable like a sculpture or painting is.

\- It usually lacks a good story. Getting attention in the art world may depend more on the artist's narrative than on the art itself, and many art consumers will similarly buy when the story is right, and avoid art they love when it has a bad or boring story. I think people legitimately and reasonably need to be able to have something interesting to say when their friends comment on the artwork in their house.

I say this as a generative artist and absolute generative art lover myself. I explored what it might take to make a living from generative art for a year, and I've met a lot of people who hate all things digital, as well as a lot of people who get what I do and love it. The thing is, the people who hate it or don't get it have a point.

> Myth Two: The artist has zero control and the autonomous machine is randomly > generating the designs. The computer is making the art and the human > deserves no credit, as it is not really art.

This one I've run into a lot. People who don't think I deserve full credit, or just don't know how to assign or think about credit.

> Myth One: The artist has complete control and the code is always executed > exactly as written. Therefore, generative art lacks the elements of chance, > accident, discovery, and spontaneity that often makes art great, if not at > least human and approachable.

This one I haven't seen much of, and I'd question whether it's a common myth. But, more importantly, I cringe at the idea that randomness leads to discovery or spontaneity or human approachableness. I would never compare randomness to surprises or creativity in traditional art, but _that_ idea is fairly pervasive among generative art initiates. I use randomness a lot, but randomness is very boring. It's mechanical variety without surprise, randomness rarely leads to surprise without a lot of generative creativity on the part of the artist.

<comment><USER3> I also love generative art ... mostly .... and I've made quite a bit but .......

The problem I have is generally when it appears to me it didn't require much effort. Making a program that draws random dots, random rectangles, random lines, is almost a rite of passage for new programmers. A lot of generative art looks like hardly many more steps beyond that. Draw 100 rectangle each slightly smaller and rotated from the previous and you get a beautiful spiral.

I know my own rarely take more than 30 minutes each. I just take some existing generator and start tweaking. If I get something mildly pleasing I save it. But I rarely promote it because I feel at some level it was farted out, it was a sketch, a doodle, not a "work" of art.

Of course I know effort is probably not a valid criteria but still, it's sometimes hard not to see what appears to be low-effort generative art as barely more than the random dot/line/rectangle exercise most programmers do early in their learning. Like instead of 100% random colors use HSL and only vary the values 10% from some initial value and suddenly it's "art". (pick light blue and it's snow on the ground, pick light brown and it's sand, pick green and it's grass, pick red/orange/yellow and it's lava). It makes it hard to know when it crossed the line from learning to code to "art".

Of course like I said I still appreciate generative art but for some reason my own block is that I need to perceive effort. It can even be simple but I have to look at it and think something like "That was simple but took a long time to find the right parameters to get it to look that pleasing".

Let me add, it's a pie in the sky dream of mine to have/run/start/manage a kinetic interactive art museum the size of the Exploritorium in SF with constantly new exhibits much of which would be generative art.

<comment><USER4> Don't forget that 99+% of the population can not program at all.

</comment></comment></comment><comment><USER5> Nice to see generative art discussed on the frontpage. :)

I think the next several years will be really interesting for generative art. Consumers are getting more interested in it, and some of our tools and collective knowledge has really grown since the days of Flash and before. In some cases, we are revisiting old techniques with new & easier-to-use hardware, like AxiDraw pen plotters. It's allowing artists to iterate more quickly and become better curators of their output.

Open source and online tools like Glitch are helping share knowledge and ideas, and together the community is pushing each other toward more and more interesting results (conceptually, algorithmically and aesthetically). The community (mostly Twitter & IG) is one of the things that makes generative art so fun to be a part of.

Totally shameless self-plug: you can follow my own generative art on IG below [1]. I hope to set up an online print shop in the coming months.

[1] - [https://www.instagram.com/mattdesl_art](https://www.instagram.com/mattdesl_art)

<comment><USER6> Absolutely beautiful work. My childhood love for sketching and doodling grew into a love for digital art (Macromedia Flash 5 anyone?) and later generative art. Anytime I'd try my hand at with generative I spent way too long trying to get a setup or a workflow together.

Eventually they get left in the island misfit quarter-done side projects well before I've gotten to playing with images.

Any advice? Stories on workflow? Is this all in processing?

<comment><USER5> Thank you! This is all in JavaScript with some open source tools I'm building[1]. I write about my process in a few places: often on Twitter, recently in a TinyLetter, and occasionally more in-depth on my blog[2].

I'd suggest Tim Holman's GenerativeArtistry[3] as a starting point to get back into it.

Good luck! Feel free to ping me any time on Twitter or IG if you have any Qs about generative art.

[1] - [https://github.com/mattdesl/canvas- sketch](https://github.com/mattdesl/canvas-sketch)

[2] - [https://mattdesl.svbtle.com/](https://mattdesl.svbtle.com/)

[3] - [https://generativeartistry.com/](https://generativeartistry.com/)

</comment></comment></comment><comment><USER7> Wew! Nice write up - I'm a huge fan of Tarbell's work. I'm curious if generative art lead him to found Etsy.

This field is still in its infancy - I'm excited for the future.

I wrote a generative art algorithm a bit ago - check it out :D - [https://anemy.github.io/concentric/#/](https://anemy.github.io/concentric/#/)

<comment><USER8> It's a really handsome piece of work. I love the papercraft aesthetic. Well done.

One of the randomizations I generated was uncannily like a winking eye. But sadly I only discovered the shareable link after I had lost it. Oh, the ephemeral nature of generative art!

</comment></comment><comment><USER9> Electric Sheep is generative crowdsourced evolutionary art that enables viewers to vote for their favorite "sheep" animations to send them off to the pasture to breed with other user's favorites with genetic cross-over and mutation.

So it just gets better and better, in response to what people actually enjoy!

[https://electricsheep.org/#/about](https://electricsheep.org/#/about)

[http://scottdraves.com/sheep.html](http://scottdraves.com/sheep.html)

[http://scottdraves.com/flame.html](http://scottdraves.com/flame.html)

[http://flam3.com/](http://flam3.com/)

[http://flam3.com/flame.pdf](http://flam3.com/flame.pdf)

Once I fell asleep while watching Star Trek Deep Space 9 on Netflix, and then half woke up after the Electric Sheep screen saver has kicked back in but I didn't realize it, so my brain struggled for several minutes trying to make sense of the never-ending wormhole sequence that I though was an out-of- control shuttlecraft lost in space!

</comment><comment><USER10> I do some generative art [1] and I think a lot of the concerns in this article have to do with it being accepted by the art world vs consumers.

In my experience consumers enjoy generative art, and are happy to buy generative work so long as its in a medium they are used to - such a as a print or plot - and are not interested in moving/digital screens.

Instead of picking a render and placing it for sale, show the buyers the software, show them the parameters and give them an option between a few renders to choose from. It makes it way more personal. People and popular culture are also much more tech savvy now and appreciative of elements of digital/internet culture.

This is just my opinion, but I believe many generative artists seem to put the algorithmic focus ahead of the aesthetic. From my experience if you treat the aesthetic as the primary objective and the technology as a tool it resonates better. Jared Tarbell and Manoloide both seem to do this, but have very different styles, and are by far my favorites in the space.

[1] [http://instagram.com/dmitricherniak](http://instagram.com/dmitricherniak)

</comment><comment><USER11> I got to live parallel to some of this. In the early 00's Agencies would send their designers/creatives/frontend devs to conferences that centered around Flash. Along with marketing talks and inspirational self help crap, the events gave a platform for some very smart/creative individuals. The enthusiasm drove a circle of innovation and competitiveness, which was good fun to watch. The scene has somewhat fallen off my radar since those days, but articles like this remind me of my soft spot for it.

<comment><USER12> I have fond memories of this time as well. It is the first time I have seen an article where I am seeing old names like Joshua Davis and Jared Tarbell. Manny Tan and Keith Peters were similarly cool.

I don't get a lot out of the art side, but I have always been fascinated by this particular topic. Where little pieces of code, depending on some inputs, can generate some really fun output.

</comment></comment><comment><USER13> Can anyone recommend tools for getting into generative art?

I have tried using a gamedev engine. It worked OK, but there have got to be better suited tools out there.

<comment><USER14> As mentioned in the article Processing is great! Super easy to start and there's some great sample libraries and code to to use for guidance.

[http://processingjs.org/](http://processingjs.org/)

<comment><USER15> After processing you can move onto openframeworks, which is written in c++ and more performant, but also comes with a bit of a learning curve if you dont have previous experience with c languages. [https://openframeworks.cc](https://openframeworks.cc)

</comment></comment></comment><comment><USER16> Good retrospective. Excellent playlist on youtube of early computer art, including some of Manfred Mohr's films that still have the power to hold the attention of modern, distracted minds ;)

[https://www.youtube.com/playlist?list=PLmmzCSWIggxABGOurYq2I...](https://www.youtube.com/playlist?list=PLmmzCSWIggxABGOurYq2IvB7Zr_OeNAAU)

<comment><USER17> They left out the demoscene :(

I mean, it's not generative art (with a few exceptions) but it definitely _is_ computer art. In the early 2000s I went to demoparties that had over 2000 attendees. These were combined showcase/gallery events with competitions and live art. Imagine a LAN-party where no one's gaming, but instead coding, drawing or producing music (and socialising--because _" real party is outside"_). And when I go to check, the scene is still active today. That's gotta count for something, right? :)

</comment></comment><comment><USER18> Great artist painting in "generative" fashion by hand ... one can see her murals in many North Carolina locales especially around Winston-Salem ...

Laura Lashley ... [https://m.facebook.com/laura.lashley.77/posts/picfp.15145307...](https://m.facebook.com/laura.lashley.77/posts/picfp.1514530781/?photo_id=10214569410858376&mds=%2Fphotos%2Fviewer%2F%3Fphotoset_token%3Dpicfp.1514530781%26photo%3D10214569410858376%26profileid%3D1514530781%26source%3D69%26refid%3D17%26cached_data%3Dfalse%26ftid&mdf=1)

</comment><comment><USER19> Saskia Freeke has some amazing work done via generative methods.

You can check out her profile here: [https://www.instagram.com/sasj_nl/](https://www.instagram.com/sasj_nl/)

Great example: [https://www.instagram.com/p/BltlYsgnVd9/?utm_source=ig_web_c...](https://www.instagram.com/p/BltlYsgnVd9/?utm_source=ig_web_copy_link)

</comment><comment><USER20> In light of this article, new genres like glitch/aesthetic/simpsonswave are just versions of generative art. All that stuff takes some simple code to augment images, then MS paint comes in to superimpose an 80's car, some green lines, and a statue of david, haha.

I really enjoyed reading about the newer AI methods though.

Here are some of my drawings :)

[https://www.instagram.com/p/BY_yHsuB8nF/?taken- by=kolmogorov...](https://www.instagram.com/p/BY_yHsuB8nF/?taken- by=kolmogorov_smooth_jazz)

[https://www.instagram.com/p/BklsIsBlmij/?taken- by=kolmogorov...](https://www.instagram.com/p/BklsIsBlmij/?taken- by=kolmogorov_smooth_jazz)

[https://www.instagram.com/kolmogorov_smooth_jazz/](https://www.instagram.com/kolmogorov_smooth_jazz/)

</comment><comment><USER21> Not exactly generative art, but still computer aided art - my home is filled with impressionist artworks I created from photographs using a product called Impresso Pro which does a nice job of adding brushstrokes to images in a realistic fashion. I use all the options to direct the choices and even manipulate the photo in Pixelmator first to influence the choices. Is the computer doing most of the work? Yes, but I still decide what it does and wind up with things I like. They are then printed to large canvases and I hang them up. It would be fun to make some completely generative art pieces if I had more time.

</comment><comment><USER22> Note that generative art is not necessarily digital. I used to make generative music using a pretty mundane electric guitar and amp, with a digital delay looper (which I suppose makes it kinda digital). I leaned the guitar against the amp to induce feedback, which went into the looper, which was on a long fade (probably 20+ repeats, well over a minute of music echoing around in there), with modulation to destabilize the pitch. Thanks to the configuration, the feedback was unstable, continuously generating an unpredictable melody of feedback.

Kinda loud, but kinda lovely.

</comment><comment><USER23> Generative art would be more compelling with clean, commented source attached. Even to non-programmers. I am always drawn to good explanations for inexplicable beauty, but what better explanation for the unknown than procedures to repeat it? (Especially if the source itself is formatted artfully.) I would love to see human readable DNA for example, one day.

Edit: Not that I believe DNA to ever be human readable, I'm referring to a level of abstraction that might be, and the expression of awe that art can evoke in relation to this and many other natural processes.

</comment><comment><USER24> In my view, Minecraft worlds are beautiful works of generative art and the appreciation and exploration of these worlds probably makes this the most popular and beloved form of generative art to date.

</comment><comment><USER25> Generative/algorithmic art:

[https://www.flickr.com/photos/abstractusinterruptus](https://www.flickr.com/photos/abstractusinterruptus)

</comment><comment><USER26> Check this shit out [https://www.youtube.com/watch?v=SqayDnQ2wmw](https://www.youtube.com/watch?v=SqayDnQ2wmw) (epilepsy warning)

<comment><USER27> As a demoscener, it's always awesome to see our craft leak out into other places. Aphex Twin getting in on the aesthetic is a real feather in the cap of the scene.

</comment></comment><comment><USER28> I was hoping it would mention Jared Tarbell, as I discovered his work back when he was making it on [http://complexification.net](http://complexification.net), thanks to StumbleUpon (may it rest in peace). Too bad the article didn't mention this site of his, but to be fair I didn't know about levitated, so I was glad to learn that.

</comment><comment><USER29> Are there any artists who work in kind of the intersection between data visualisation and generative art? I'm thinking of works where the main point is not necessarily to show the data as clearly as possible (i.e not a graph), but rather where the data is somehow interleaved and helps tell the story. Somewhat similar to the map of the US in the article based on the roads.

<comment><USER10> Yes, there are lots. I would say Mike Bostock of d3 fame is an example of someone who does that well, as so is Yi Shen ([https://twitter.com/pissang1](https://twitter.com/pissang1)).

</comment></comment><comment><USER30> I am currently writing master thesis about generative art, this article comes in handy :)

If you interested in generative art there is nice tool called Context Free Art: [https://www.contextfreeart.org/](https://www.contextfreeart.org/) It's not as well known as Processing or other tools and that's a pity.

</comment><comment><USER31> Vi Hart has a "Sol Lewitt Forgery" VR project. [http://elevr.com/portfolio/sol-lewitt- forgery/](http://elevr.com/portfolio/sol-lewitt-forgery/) I don't know how far it got but I liked the idea.

</comment><comment><USER32> @bkezy0 has some interesting work on Instagram. It's more of a mix between generative and non-generative than anything pure. Still worth checking out.

Profile: [https://www.instagram.com/bkezy0/](https://www.instagram.com/bkezy0/)

</comment><comment><USER33> I looked at the chart in the middle and was like, whoa, that one is pretty nice!

It reminded me of: [https://www.youtube.com/watch?v=OlLqYBU8qsY](https://www.youtube.com/watch?v=OlLqYBU8qsY)

</comment><comment><USER34> There is lot of equations and geometry that are hiding under the hood along with some clever programming. Personally one of my personal favorites is Fractal art which involves lot of recursive programming.

</comment><comment><USER35> Disclaimer: I come more from the contemporary art world even than from the tech world. (Much of my job is looking at thousands of art exhibitions from around the world each month. My little non-profit publishes the ones we think are most important to see at the moment so that the public can learn about them for free even if they aren't rich collectors.)

First, I want to say that I'm thinking about Generative Art in the context of "contemporary art," which despite sounding like it could apply to literally any art made in the current era actually denotes a specific social context and the set of people, institutions and works that constitute it. Some people call this "fine art" but I don't like the implied hierarchy and value judgment of that term. I think there's a separate case to be made that Generative Art is actually more akin to important historical craft and decorative arts traditions, which have great artistic importance and matter much more to many people than contemporary art. I mean "contemporary art" the way people usually mean it when they start institutions called "Museum of Contemporary Art" or "Contemporary Art Center."

Generative Art qua "contemporary art" seems to fit a pattern that you see a lot in startups: people who are into coding and technology enter a given field that they aren't so involved with and start applying their skills to it. As far as I know, the AirBnB people weren't running bed and breakfasts at the beginning, they were looking for problems to solve with technology. That worked great, for the most part: they identified concrete problems and found ways to solve them.

The odd thing about Generative Art, which seems to fit this pattern insofar as the energy and interest is coming much for from the tech side than the art side, is that art doesn't have any problems because it has no clear purpose. Any idea you may individually have about what art should do, or what makes art "good," can be easily defeated by famous counterexamples. Lots of important art declines to be beautiful, to start interesting conversations, or to care at all about the history of art, for example. In other words, no one ever sat on a bench in a contemporary art museum looking at paintings and, chin-on- fist, said "there has to be a better way!"

This article goes to great effort to insert Generative Art into the history of art, more or less finding math-y geometric art or artists who used algorithms. These artists were often doing so to resist earlier ideologies about the genius of the master's hand, or to pretend to escape their own subjectivity, rather than out of an enthusiasm for visual algorithms as a craft. I'd suggest those interested in contextualizing Generative Art historically also research "process art," which normally had a messier aesthetic but, like Generative Art, posited the activity of producing the artwork as the most expressive aspect of the artist's practice. The act of painting, for process art, would be more important than the painting itself, almost closer to performance art than any other movement. That certainly seems to fit the Generative Art ethos, which is often about the construction of a specific activity that leads to an uncertain result. That said, process art is associated with the 1970's.

We're about two decades into a deep rejection of art movements, which are inextricable from the problematic, exclusive teleology of modernism. A certain kind of person with a certain kind of power used to dictate what forms and ideas were important, and rewarded artists who fit the narrative and, well, were the same race and gender as all the other successful artists. These days, artists are generally expected to be individuals rather than to be swept up in some cultural wave. To be part of a group with such a narrow direction ("we make art using cutting-edge software!") subordinates the individuality of each artist and, I think, compromises their efforts to be successful in the field of contemporary art.

Therefor, I think that the project of including Generative Art as a movement into contemporary art discourse is probably quixotic. Instead, I'd suggest that we're likely to see more and more individual artists incorporate these tools, not just because they're neat tools but in service of their own unique aims. Simon Denny, for example, engages heavily with digital and start-up culture, but he does so (I would argue) from a place of critique and fascination about what the state of technology implies about both current society and human beings on some kind of fundamental level. I don't doubt that an individual artist could exclusively use Processing to make their work and have an art career. But they would need to make the case for themselves, rather than for Generative Art, and in my opinion the less they position themselves as representing a craft-driven community the more likely they are to be successful.

I did want to say that I really appreciate the effort in this article to highlight the contributions of women to Generative Art. This is very much in keeping with where the art world is at the moment. Just like tech, art has been the subject of extreme discrimination that is only just beginning to be improved, and there is broad consensus about that within the art world.

<comment><USER36> I found the post interesting and appreciated the copious references to 20th Century artists. In my opinion, the generative work is preoccupied with the method of creation and is nothing more than a self-pleasuring act, they provide no meaningful paths to insight or reflection for the viewer.

</comment></comment><comment><USER37> Not necessarily generative art, but I've always been a big fan of Team Lab's work.

[https://www.teamlab.art/](https://www.teamlab.art/)

</comment><comment><USER38> Warning to data plan users: The page is 11MB.

</comment><comment><USER39> Three more approaches not mentioned in the article:

      Pixel Sorting
      </comment><comment></comment><comment>-

A subtype of glitch art. Generally you do it horizontally or vertically. You choose contiguous intervals in a bitmap and sort pixels in it by color, hue, value etc. Here's a moose I made based on a Wikimedia Commons photo: [https://www.reddit.com/r/pixelsorting/comments/7oioko/moose/](https://www.reddit.com/r/pixelsorting/comments/7oioko/moose/)

      Glitch art
      </comment><comment>----

Also has its own subreddit. People over there get excited over printers breaking down in the middle of printing a page, etc. Sometimes results are truly intriguing.

      Primitive
      </comment><comment>---

Geometric primitives. The ingenious library written in Go. It transforms an image into a composition of 2D overlapping, translucent polygons. Output is available chiefly as an .svg [https://github.com/fogleman/primitive](https://github.com/fogleman/primitive) Using this and images from a search engine, I made a calendar with animals. I could post it if anyone's interested.

<comment><USER40> Re glitch art, I wrote a (moderately popular?) tool for it: [http://akx.github.io/glitch2/](http://akx.github.io/glitch2/)

</comment></comment><comment><USER41> Personally, I don't trust art from digital computers. It might contain subliminal messages from the feds. Gimme a few sticks and elastic bands any day. The Harmonograph - [https://www.youtube.com/watch?v=ck_JY-Z5kZI](https://www.youtube.com/watch?v=ck_JY-Z5kZI)</comment>


Why this tiny Italian restaurant gives a discount for bad Yelp reviews - <AUTHOR> http://arstechnica.com/business/2014/09/why-this-tiny-italian-restaurant-gives-a-discount-for-bad-yelp-reviews/

<comment><USER2> It's good to see someone standing up to Yelp's questionable ways in which it runs its site. It's ironic that Yelp tries to extort businesses into giving away free food and drink, as well as paying for advertisements and placement on the site, but a company offering discounts for one star Yelp reviews is in breach?

I am coming over to the states in a couple of weeks and will make it my mission to go to this place and give them my business because I respect what they are doing.

</comment><comment><USER3> I don't see how they can be in breach of yelp's "terms of service" seeing as how yelp is also taking the stance that restaurants will be listed whether they want to or not</comment>


A Tale of Two Users, or How Design is Tough - <AUTHOR> http://georgesaines.com/2010/09/14/a-tale-of-two-users-or-how-design-is-tough/

<comment><USER2> Are UX discussions now really so played out that this little anecdote is considered insightful?

To summarise for comment readers "some people have a different understanding of complexity and how something you see as trivial might be confusing to others".

<comment><USER3> I think so. For the non UX crowd or those starting out. You have to start understand the issues at hand somehow.

Anecdotes are stories. Stories are great ways to communicate complex issues in an easy to digest manner.

It may not be ground breaking insightful; but still relevant.

</comment></comment><comment><USER4> Reminds me of the elevator button example:

You're in the fourth floor, the elevator is on the fifth floor (indicated by a display above the door) and you want to go up.

Do you press the _up_ , or _down_ button?

At first this seems obvious, since you want to go up, but it's only obvious because you know how elevator interfaces work.

Another valid way to interpret the elevator interface is to press the down button, since you're in the fourth floor and want it to come down to you.

Yep, interface design is hard.

<comment><USER5> You press the single button on the panel, hypothetically labelled—hypothetically because a label is completely unnecessary when there's only one—"come to me." A lot of what makes interface design "hard" is that our minds are filled with cached solutions to no-longer-valid problems (like the problem of directing around a purely-hydraulic lift that cannot indicate its relative position to you.) The elevator now has a computer in it; it knows which direction it needs to go to come toward you, and you now know if it's coming from the "wrong" direction in order to pick an alternate lift out of a bank.

Some of these things are so codified that people would react badly if we took them away, though. For example, JRPGs no longer _need_ to display numeric damage tallies—but it persists because that's what makes a JRPG a JRPG.

<comment><USER6> But that's wouldn't solve the problem that elevator buttons are designed to solve.

You have to tell it "I'm going up" so that it doesn't stop downward-bound elevators on your floor and slow the system. Your single-button interface would make things considerably worse.

<comment><USER5> That assumes, though, that there is more than one elevator. There is already well-known and battle-tested solution for that case: put a destination-floor- number panel outside the elevator bank (not connected to any single elevator, but rather offset), with the current floor's button removed. To remove the possibility of thinking that the panel of buttons has anything to do with the numbers the elevators display, remove the external displays from all the elevators. When you press the button, the first elevator available (and going in the right direction) opens at your floor, already set to go to your destination floor. (If you really insist, you can pulse the backlight of the activated destination floor button as the elevator nears, in a sort of marco- polo way; I've seen a very upscale hotel do just this.)

Now, what I was referring to in my previous post was a _single-elevator_ scenario (which is most elevators outside of hotels and apartment buildings.) In the case of a single elevator, there's really no difference between the elevator either:

1\. passing you, turning around, arriving at your floor, picking you up, and heading to your destination floor; or

2\. opening at your floor, taking you along with whomever else is inside to their destination floors, then turning around and heading for your destination floor.

Either way, you spend the same amount of time waiting (arbitrarily either inside or outside of the elevator), and the elevator opens the same number of times. When there is one elevator, all that really matters is getting inside it.

<comment><USER7> Capacity is often a bottleneck in large buildings. The elevator might be full as it passes going the wrong way, so getting in isn't even an option. Also, more people might show up on your floor by the time the elevator passes going the right way, necessitating another stop.

<comment><USER5> Firstly, if capacity is a problem, there really shouldn't just be a single elevator serving this supposedly "large" building. But since a lot of places can't afford all the elevators they need... you know how a full bus will just ignore everyone at a bus stop, and only stop when someone wants off? Elevators know when they're full.

</comment></comment></comment></comment></comment></comment><comment><USER8> Case in point:

I had a friend's mum who asked me once (in reference to vertical scrolling on ie. browsers and Microsoft Word):

Why do you press down to make the page go up?

('down' is the down cursor button)

</comment><comment><USER9> Isn't this more an example of when it's best to let a customer go?

Is the cost associated with meeting this small percentage of customer's needs worth actually meeting them?

<comment><USER10> Depends. Not with mission critical processes like checkout IMHO. What this piece points to is a flaw in design processes. As designers and developers we see things through a logical lens of our own making.

What we don't realize is that the real world and other factors influence our target users. It's the mark of a good designer when he/she pushes user testing to understand what people will do with what we build.

In this specific case, _we_ all know you use the magnetic pin but not everyone does. It's hard to believe but the sometimes don't.

<comment><USER9> I agree that if your site metrics show large numbers of people set to buy but abandoning their purchase because the checkout process is confusing, then you should work to make it more user friendly.

But what I'm saying is every designer should be asking 1. How many people are having this unique difficulty and 2. Is it worth the time and COST to fix it? Moreover, will adding the information/steps to needed to clarify the process bore/put off the majority of customers who already know how to use this (for example, a tutorial that either adds an additional step or doesn't allow users to bypass it)?

A good designer should be able to plan for contingencies, but a good businessperson should be able to draw the line between pleasing a customer and watching their bottom line.</comment></comment></comment>


Heidegger and the Art of Motorcycle Maintenance - <AUTHOR> http://www.slate.com/id/2218650/

<comment> cunard-n See also: "Forces of Production" by David F Noble, for an account of how specialized skills and self-reliance were usurped my management in a slow but deliberate war of many losers between the early 20th century and the late 1970s. He takes particular aim at the role numeric control played in abstracting the skills of machinists into CAD-CAM. Just to say that I think management usurped self-reliance in many fields. And here we are.</comment>


Twitter / @Tumblr: The recovering database cluster is online... - <AUTHOR> http://twitter.com/#!/tumblr/status/11908476196360192 <comment><USER2> Anyone know what really happened?

[ Not inserting obvious comment about lack of communication during outage... ]

<comment><USER3> Best guess is the standard first huge failure, push an update and break everything.</comment></comment>


Ask HN: How lengthy should a technology book be? - <AUTHOR> I am writing a book in technology. Something like 'The Learn Startup.' I see usually books are around 300 pages. Do people get time to read in that much depth or books like "Bootstrapper's Bible" that are very short (103 pages) and to the point are read more? The one I am writing is around starting companies. <comment><USER2> Long enough to explain things well, but no longer than that. Especially if the goal is passing on information, be concise and do without the fluff and life stories.

So, better 100 than 300 pages.

<comment><USER3> I came here to say what you'd already said. I'd just add that we have to read enough _already_ to do our work.

If you're going to write, respect your reader's time as part of your writing.</comment></comment>


JVM Anatomy Park – 11: Moving GC and Locality - <AUTHOR> https://shipilev.net/jvm-anatomy-park/11-moving-gc-locality/ <comment><USER2> I think that the main reason why compacting GCs improve performance is that they obviate the need for a sweep phase, which always has some cost even if you optimize the hell out of it. I believe that from a scientific standpoint, it is still an open question exactly how much of the compacting GC speed-up is due to not sweeping versus improved locality. It's hard to construct a good experiment of this, because non-compacting GCs always have at least some kind of sweep phase. For example, if the non-compacting collector experiences fragmentation, then is its reduced performance due to the cache misses experienced by the sweeper (which have nothing to do with locality per se - they are just an inherent cost of sweeping) or because of cache misses experienced by accesses within the program due to the changed ordering of objects?

The experiments in this post are super interesting, but it's not a good idea to extrapolate from microbenchmarks. In my experience big programs always have some chaotic subtlety, which can sometimes even make its behavior be the opposite of what the microbenchmark tells you.

It's worth noting that GCs are tuned so that the typical program will typically allocate objects so as to maximize the locality of object accesses. Non-compacting GCs tend to allocate objects of the same type next to each other (so if you allocate O, P, Q where O and Q have type T and P bas type S, then Q will be right next to O but P will be elsewhere) while compacting GCs tend to allocate objects in sequence (so O and P will be adjacent, and then after P will be Q). There's no guarantee that any particular program will prefer one ordering over the other.

<comment><USER3> _It 's hard to construct a good experiment of this, because non-compacting GCs always have at least some kind of sweep phase. For example, if the non- compacting collector experiences fragmentation, then is its reduced performance due to the cache misses experienced by the sweeper (which have nothing to do with locality per se - they are just an inherent cost of sweeping) or because of cache misses experienced by accesses within the program due to the changed ordering of objects?_

What about adding a dummy sweep phase to the compacting collector? Then you could compare compacting + dummy sweep with non-compacting + real sweep.

<comment><USER2> There is no way to implement a dummy sweep that has the behavior of a real sweep if the collector is doing compaction.

Maybe it isn't obvious: compaction isn't a plug-in that you put into a collector. It's a different way of writing a collector. For example, compacting collectors often avoid having any mark bits. You need mark bits to sweep (among other things).

</comment></comment></comment><comment><USER4> _> Be ever so slightly wary when someone sells you the non-moving GC or no-GC solutions_

And you should also be wary whenever someone uses the JVM to make a general point about GC strategies.

The importance of a moving/compacting GC varies hugely depending on whether or not a language/runtime lets developers define the memory layout of objects.

In a language like Java (and other JVM based languages) that doesn't let you do that relying on the GC is the only way to fix locality. But that is not the case in languages that have structured value types (C#, Go, Swift, ...).

<comment><USER5> You might have checked but here is IanT's reply on "Why golang garbage- collector not implement Generational and Compact gc? There is also interesting discussion in thread.

[https://groups.google.com/d/msg/golang- nuts/KJiyv2mV2pU/wdBU...](https://groups.google.com/d/msg/golang- nuts/KJiyv2mV2pU/wdBUH1mHCAAJ)

"However, modern memory allocation algorithms, like the tcmalloc-based approach used by the Go runtime, have essentially no fragmentation issues. And while a bump allocator can be simple and efficient for a single-threaded program, in a multi-threaded program like Go it requires locks. In general it's likely to be more efficient to allocate memory using a set of per-thread caches, and at that point you've lost the advantages of a bump allocator. So I would assert that, in general, with many caveats, there is no real advantage .."

<comment><USER6> > And while a bump allocator can be simple and efficient for a single-threaded > program, in a multi-threaded program like Go it requires locks. In general > it's likely to be more efficient to allocate memory using a set of per- > thread caches, and at that point you've lost the advantages of a bump > allocator.

Ian is extremely confused here. Every production multithreaded generational GC uses thread-local allocation buffers (edit: thanks to Filip for correcting my terminology).

<comment><USER2> Very few production multithreaded generational GCs use per-thread nurseries.

But probably all of them use thread-local caches, or what you might call per- thread caches. I think that's what you might have meant.

Per-thread nurseries is a shit idea unless your language has extra-super-high levels of infant mortality. Remember that in a per-thread nursery genGC, the write barrier has to infect the pointed-to object with the global heap so that future stores to that object also trigger the barrier. In a normal genGC, the pointed-to object is unaffected; the stored-to object is merely shaded (or card-marked, or logged, or whatever) for revisiting.

Humorously for parent's post, to my knowledge the first instance of a thread- local cache was the idea that a multithreaded compacting GC can chop off a slice of memory from the global bump allocator and then use it to service a thread-local allocator. Not 100% sure though because maybe Boehm and friends beat everyone to it.

<comment><USER6> > Very few production multithreaded generational GCs use per-thread nurseries.

> Per-thread nurseries is a shit idea unless your language has extra-super- > high levels of infant mortality.

Isn't that what TLABs are? HotSpot uses TLABs by default, unless my info is out of date… For example, [1] shows the generated assembler for object allocation, which uses nonatomic pointer bumps.

I think we might be talking about different things by "per-thread nurseries". All I mean by a "per-thread nursery" is a TLAB.

[1]: [https://shipilev.net/jvm-anatomy-park/4-tlab- allocation/](https://shipilev.net/jvm-anatomy-park/4-tlab-allocation/)

<comment><USER2> Like I said, you're thinking thread-local caches. TLABS are thread-local caches, not per-thread nurseries.

There's a HUGE difference between a thread-local cache (which everyone uses and which you cite) and per-thread nurseries (which are an obscure idea that hardly anybody likes).

<comment><USER6> Yes. Apologies for using the wrong terminology :)

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> I ran the 3 visualizations at the end of the article and made imgur albums:

Images are in order.

JOLSample_22_Compaction: [http://imgur.com/a/agJaR](http://imgur.com/a/agJaR)

JOLSample_23_Defragmentation: [http://imgur.com/a/A74k9](http://imgur.com/a/A74k9)

JOLSample_24_Colocation: [http://imgur.com/a/RTrhL](http://imgur.com/a/RTrhL)

In JOLSample_22_Compaction, Aleksey writes: "It happens because many temporary objects are allocated while populating the list.". I'm having trouble knowing exactly what are the temporary objects. Is it the implicit c.toString()? Maybe also the "int minCapacity" in ensureCapacityInternal in ArrayList[1]? Is there more?

Also, in JOLSample_22_Compaction it is kind of nice to see some gaps spaced by growing powers of 2 in [http://imgur.com/4T5dBi8](http://imgur.com/4T5dBi8) ; it must be the actual pointer arrays of the ArrayList, replaced successively to grow in size. And the current one being in green color. Or am I mistaken?

Fascinating.

[1] [http://grepcode.com/file/repository.grepcode.com/java/root/j...](http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8u40-b25/java/util/ArrayList.java#222)

</comment><comment><USER8> Love these. Even after reading over the JVM spec, these writeups still show how much I don't know.

<comment><USER9> This might be because these are about (important) implementation details the spec quite deliberately does not specify.

</comment></comment><comment><USER10> OT, but I'm curious if there's any manual memory management analogue for moving GC. Generally in that setting if you want to allocate a bunch of objects with good locality you use a slab allocator. But what if you want to compact that slab after a certain period of time because of fragmentation, or you have a bunch of object already on the heap you want to move together onto a single slab? The complexity is you have to have a way to update all the references to those objects.

I'm curious if anyone knows of such work that has been done?

<comment><USER11> Doubtful, because compaction requires updating the pointers to the new locations. In manual memory management, the compactor would have to stop the world, know enough about each structures' internal layouts to know where the pointers are, and then update all of them.

That would be impossible when trying to interoperate among different languages.

Manual memory management has its place, but if you need compaction, you should use a garbage collector. Manual memory management, (and reference counting,) aren't always faster than GC. (Malloc has overhead too, and can run slower than GC.)</comment></comment>


Raspberry Pi Zero Power Consumption Comparison - <AUTHOR> http://www.midwesternmac.com/blogs/jeff-geerling/raspberry-pi-zero-power <comment><AUTHOR> Just posting the tl;dr at the top of the article:

 _The Raspberry Pi Zero uses about 1 /3 the power of the A+, or 1/6 the power of the model 2 B._

This is pretty awesome, as it's only about twice the power usage of a typical Arduino setup, so battery-powered embedded projects can start considering the Pi as an option. It helps, of course, that this is just $5!

I just got my mini-HDMI adapter in the mail today, so I'll be running a few more tests to see if there are any other major differences between the Zero, the model 2 B, and the A+/B+ (I currently use the A+ for most small projects right now, and the 2 B when I need a little more CPU).

<comment><USER2> The power consumption was the number 1 reason I couldn't use the PI in most of my projects so this is super exciting (the number 2 being cost).

<comment><USER3> if power and cost were 1 and 2 this is pretty good for you! Could I ask what the specific applications were? I'm very interested and wonder what other people have done. May have follow-up questions for you :)

</comment></comment></comment><comment> mike-cardwell I'd really like to find a USB battery pack for one of these which can be used to make the device portable. I know there are loads, but they usually have one or more of the following problems:

1.) Can't power the PI at the same time as being charged

2.) PI consumes power faster than the battery can charge up

3.) Neither of the above two problems, but when you plug the battery into the mains it momentarily disrupts the PI causing it to reboot.

Anyone know of a battery which will last with one of these for the best part of a day or more and which I can just plug into the mains when I'm near a socket to keep it topped up?

[edit] FWIW, my plan (if I can sort the power) is to migrate my email and xmpp services from the fixed in place Intel NUC under my TV at home, to a Pi Zero in my backpack.

<comment><USER4> I am not a battery or electronics expert but my understanding is that it is not ideal to simultaneously charge and discharge a LiPO battery, so most don't support passthrough.

I use a RAVPower Xtreme (26800mAh - though I'm not sure if I believe that rating) to power a dashcam and had similar requirements to you. However, I've implemented a DPDT relay and a 12V boost regulator to switch between the battery pack and car power (ACC), depending on the ignition status of the car. So when the car is on, the battery is being charged only and the dashcam is getting power from the car. When the car is off, the battery is only being used to power the dashcam.

I've not tested it with a Pi but there is not an interruption in power long enough to disturb the dashcam (the relay spec is 8ms operate time). The battery pack is charged using a 2A 5V charger. I can't properly assess the charge status/history of the battery as it only has 4 LED's and I don't want to take it apart but so far my commute and other driving is long enough to keep it ticking over.

<comment><USER5> Its definitely possible, think about the circuitry in your phone and how the batteries work with that.

As far as I'm aware, most modern phones can't turn on without a battery pack present in the phone, even when plugged in, which would suggest that the juice is actually coming from the phone battery at all times and not direct from the charger regardless of charging state.

</comment></comment></comment><comment><USER3> Very interesting. Still far from solar-powered territory for outdoor projects, for example. Unless there were a way to wake the pi very easily. However, the raspberry pi includes no power management whatsoever; it's either off or on[1]. Why not? Does the chip not support it?

\--> Note that the article specifically calls out disabling USB, disabling ethernet (not relevant here), and disabling video output if not using it. I wonder if the author could try disabling HDMI to see if power is reduced....

This can be mitigated if there were a way to boot quickly on demand. See this writeup on sleepypi, an expansion-board that does just that:[2] So it should be possible to wake on demand by powering up and going through a boot cycle. Since the hardware is SO very fixed (not going to change short of ripping parts of the board off) I wonder why the boot time couldn't be further reduced...what really takes 10 seconds, during which the SD card could have read 200 MB or so? It's nearly enough time to wake from hibernation, were that supported. (SD card bus speed is limited to reading 20 MB/second or so). I'm sure it could be improved.

This other write-up mentions using the previous generation for up to 24 hours for remote-location time-lapse photos [3] - also contains an interesting idea about underclocking. Not sure if this suggestion is relevant, possible, or works.

[1] No sleep mode on pi: [http://raspberrypi.stackexchange.com/questions/4773/raspberr...](http://raspberrypi.stackexchange.com/questions/4773/raspberry- pi-sleep-mode-how-to-avoid)

[2] * Pi and SleepyPi: [http://marktmarshall.com/2014/08/experimenting-with- raspberr...](http://marktmarshall.com/2014/08/experimenting-with-raspberry-pi- power-management/)

* An interesting FAQ on SleepyPi: [http://spellfoundry.com/sleepy-pi/sleepy-pi-faq/](http://spellfoundry.com/sleepy-pi/sleepy-pi-faq/)

[3] Powr saving on raspberry pi A in remote installations: [http://babaawesam.com/2014/01/24/power-saving-tips-for- raspb...](http://babaawesam.com/2014/01/24/power-saving-tips-for-raspberry- pi/)

<comment><USER6> _" I wonder if the author could try disabling HDMI to see if power is reduced...."_

from comments ~ [http://www.midwesternmac.com/comment/3216#comment-3216](http://www.midwesternmac.com/comment/3216#comment-3216) in this article, _" Controlling both the PWR and ACT LEDs on the Raspberry Pi 2"_ ~ [http://www.midwesternmac.com/blogs/jeff- geerling/raspberry-p...](http://www.midwesternmac.com/blogs/jeff- geerling/raspberry-pi-zero-power)

<comment><USER3> (could you clarify why you include the second link? Your second link is our article link.)

The comment that you mention mentions savings of "28ma" \- could it really be so high? This is hard to believe, or hard to believe that it would translate the raspberry pi zero, as our article link says the idle is 50-70 mA (third photo caption on page) - so if true and the same for this board, disabling HDMI would reduce current by up to 56%! However, on the other hand that might not be true at all. Perhaps the A has a dedicated chip for HDMI, whereas this chip drives it directly; or perhaps the hearsay on current savings from the HDMI is simply wrong. I've submitted a version of this comment as a comment to the author's blog, since he's all set up to investigate. (I think the post is pending moderation.) Probably there will be 0 power savings.

EDIT: Thanks, author!

<comment><AUTHOR> I just tested this three times, and it looks like it's right around 30 mA saved, so the rumors are true!

I'm going to tinker a little bit and see how low I can get the power consumption (disable HDMI, disable LED, and just have Pi running by itself). I'll post back here with more details.

<comment><USER3> Wow! That is an astounding difference. Thanks for your quick action. If your numbers are correct looks like you just reduced idle usage by close to 50% - 0.2 watts to 0.1 watts or something - if a solar panel has to get through fog and night-time on an attached battery, that is a _huge_ difference!

<comment><AUTHOR> Running barebones I can get down to 40 mA at idle. Almost Arduino territory!

<comment><USER3> going to sleep so you won't be getting more comments from me, but thanks for the interaction and your article! great result. I hope you'll have a chance to incorporate the new numbers/savings into your write-up.

</comment></comment></comment></comment></comment></comment></comment><comment><AUTHOR> Just did a bunch more testing, and I can get it to idle at ~40 mA if I disable HDMI (inside `/etc/rc.local`: `/usr/bin/tvservice -o`).

With a WiFi dongle (so it can operate on the local network at least) and HDMI disabled, it hovers around 120 mA. Very, very nice!

</comment><comment><USER7> > _The Raspberry Pi Zero - quite a small Linux computer!_

Okay, I have a smalllll complaint. In popular modern-day English, R-pi is not really a 'computer'. Because you can't run a full-fledged operating system like Windows or Linux on it -- you have to run embedded version of those OSs (like Windows CE or whatever). I think I may not be totally and technically correct in my point here, but I think it's an important point to make or at least mention somewhere along the way anyway -- because, as it happens, I told someone that R-pi is basically a cheap computer... and this person was eventually miffed to find that he couldn't run the Windows applications he wanted to run on it, because those applications only could run on a x86 Windows install.

<comment><USER8> The solution to "cheap windows" is the various Bay Trail devices; so far the best offer I've seen is 7" tablet for £45 which comes with a year's subscription to Office 365 (worth >£50).

<comment><USER9> do you have a link please?

<comment><USER8> [http://www.amazon.co.uk/gp/product/B00O636WJU](http://www.amazon.co.uk/gp/product/B00O636WJU) (price has gone up a bit)</comment></comment></comment></comment>


Command line tricks - <AUTHOR> http://www.tuxradar.com/content/command-line-tricks-smart-geeks

<comment><USER2> Regarding the SSH tricks:

If you change the port of SSH, change it to still use a port under 1024. On most Unix systems these are privileged ports that require root to open. This ensures that if there is a process listening there that it was opened by root and not some intruder hoping to get your password to sudo to higher privileges.

Regarding removing reserved space:

Be careful doing this on some file systems. Some filesystems may need to write more data to a journal or are set up with copy-on write and will not be able to delete files if you have no space left on the disk. Reserving a little extra space for this can be necessary. A little extra space also lets you have some space to work with if you need to temporarily create or move files before you can free up the space.

Also, if you're going to use a non-standard port, set up your .ssh/config!

        Host s
            hostname server
            port 666

That is a little more versatile than setting up an alias as it will work from any shell and you can specify any ssh daemon in there.

See `man ssh_config` for more info.

<comment><USER3> Additionally, (almost all) Unix filesystems do not fragment under normal use. This is far superior to FAT, which does have this problem, but also means that there's typically no way to defragment them short of recreating them.

A nearly-full disk is _not_ "normal use" in the above sense.

<comment><USER4> XFS (from SGI) has xfs_fsr for defragmentation. It's not perfect, but it's better than nothing.

</comment></comment></comment><comment><USER5> I loved these tricks. Here's a site dedicated to command line tricks:

<http://www.commandlinefu.com/commands/browse/sort-by-votes>

Being on a Mac, I didn't have the rename or prename command, but after a lot of searching, found it here:

<http://www.perlmonks.org/?node_id=303814>

(beware the '+' signs at the beginning of each line)

</comment><comment><USER6> It's a pet peeve of mine to see

      cat [one-file] | less

`less` takes a file as an argument.

      less [one-file]

only requires you to run one program, and I/O tends to be really slow.

<comment><USER7> You're not the only one, this inspired the Useless Use of Cat Award (a few other strange practices are covered here as well):

<http://partmaps.org/era/unix/award.html>

</comment></comment><comment><USER8> tar -xvf infers the filetype from the file; obviating the need for the smart untarring script.

edit: Also, another useful thing is <command> | xargs -n1 -I{} <stuff>, which runs stuff once on every element of the output of command, replacing every occurrence of {} with the element. Then you can do something like "ls *.mp4 | xargs -n1 -I{} mv {} `basename {}`.mp3" .

<comment><USER9> I quite like using for loops for that. I use it a lot for batch audio conversion with ffmpeg:

for f in *.wma; do ffmpeg -i "$f" "${f/.wma/.mp3}"; done

<comment><USER2> I end up using this template for a lot of operations. Where rename really seems applicable is when you want to do complicated renamings such that regular expressions really are appropriate.

<comment><USER9> Yeah. I used to love Bash but these days anything that takes more than a couple of lines I just use Perl. Its ubiquity helps a lot.

</comment></comment></comment></comment><comment><USER10> You can also expose a file in a one-shot webserver using netcat:

        cat somefile | nc -q1 -l -p8081

<comment><USER4> I prefer:

         nc -q1 -l -p8081 < somefile

</comment></comment><comment><USER3> The given syntax for prename is very odd, and e.g. <http://man.he.net/man1/prename> has a much more sensible explanation of what the command does (i.e. _remove_ _bak).

The script under "crash test dummy" is hackish and full of race conditions, but that's perhaps acceptable in that case.

Leaving webmin and X open to all local users (in the case of webmin, by choosing a bad password) is not a good idea.

</comment><comment><USER9> I love this sort of stuff. The first few "top ten" command line tricks articles I found back when I was enthralled with the power of the Bash shell have probably saved entire days of my life by now.

</comment><comment><USER11> I really liked this list, especially all the remote stuff. Though he did make an error: lower nice values get "more favorable scheduling". That's why +10 is default.

</comment><comment><USER4> Regarding "Editor redirection": why should we complicate ourselves with /etc/alternatives when the EDITOR and VISUAL environment variables can be used?

</comment><comment><USER4> The root account from RHEL and Fedora systems is configured by default with safe-delete command aliases.</comment>


Ships are illegally dumping plastic trash at sea, study suggests - <AUTHOR> https://www.cbc.ca/news/technology/sea-litter-ships-bottles-1.5318390 <comment><USER2> Related: International Maritime Organization (IMO) rules for garbage discharge overboard from ships...

[http://www.imo.org/en/OurWork/Environment/PollutionPreventio...](http://www.imo.org/en/OurWork/Environment/PollutionPrevention/Garbage/Pages/Default.aspx)</comment>


Show HN: Auditus – Ebook to Audiobook Conversion - <AUTHOR> https://auditus.cc <comment><USER2> I like your project, but I'd like it more if it was accompanied by a video of the text being highlighted as it is being read (a kind of dual modality reading - visual + audio). I created such a tool for myself on MacOS with the Alex voice, that works in browsers on any page and in PDFs. I find dual modality reading to enhance my focus a lot. Btw, I use this tool to read this very thread of comments.

And of course I would want to hear the amazing Wavenet voices used in this role.

<comment><USER3> Do you plan to share yours? Looks very interesting from the dual modality. I would definitely try it out.

<comment><USER2> It's only for personal use, but if you want to try:

[https://gist.github.com/Visarga/e6597edcb7ec6993521829ef5c17...](https://gist.github.com/Visarga/e6597edcb7ec6993521829ef5c1798db)

Also, it only works on MacOS because it uses the Alex voice which comes bundled with the system.

<comment><USER4> That's really cool, thanks for sharing!

I saw a site that highlighted the official audiobook/ebook of "The Lean Startup" on one page. I love this as a generic solution.

</comment></comment></comment></comment><comment><USER5> I have the habit of convert everything I write into audio first before I publish it. It's a good way to make your mistakes pop out.

I use fromtexttospeech[1] to convert to audio. Judging by the voices, it seems like the author is also using the same speech engine.

If I'm not mistaken, these are from Nextup's TextAloud software.

[1]:[http://www.fromtexttospeech.com](http://www.fromtexttospeech.com)

Edit: Though this beats the 50k character limit

<comment><AUTHOR> AWS Polly

</comment></comment><comment><USER6> It occurs to me that the mistakes in emphasis that I hear in these samples are the same mistakes I hear from young readers who are concentrating on decoding the words one at a time. The method that fluent text speakers use is to process the entire sentence while beginning to speak it.

I wonder if a better model of word emphasis considering whole sentences could move an automated reader out of the uncanny valley.

<comment><AUTHOR> Currently the text is run through a sentence segmented before conversion

</comment></comment><comment><USER7> Hey, I just created a similar application using the Cloud Text-To-Speech API from Google and the textract ([https://textract.readthedocs.io](https://textract.readthedocs.io)) library to extract text from lots of kinds of documents:

[https://github.com/danthelion/doc2audiobook](https://github.com/danthelion/doc2audiobook)

It runs inside a Docker container so fairly easy to try it out.

</comment><comment><AUTHOR> Hi! I built this to have an easier way to convert ebooks to audiobooks! The backend is powered by AWS Polly. If you have any feedback or feature requests, please feel free to drop me an email at <last 3 characters of username> @ <myusername>.com

<comment><USER8> This is great! I forwarded this to my girlfriend (who is blind) and she loves it.

<comment><USER9> On Android there is [http://www.hyperionics.com/atVoice/](http://www.hyperionics.com/atVoice/) which I really like and which seems on a similar level (at least judging by the example).

</comment></comment></comment><comment><USER10> I built something similar to listen to Paul Graham's essays It's a console app & uses OSX's "say" command for the TTS. Contributions are welcome. [https://github.com/hemantasapkota/awesome- essays](https://github.com/hemantasapkota/awesome-essays)

</comment><comment><USER11> I get a metamask (ETH Wallet) phishing warning on this site. Anyone else experience this?

<comment><USER12> Interesting. The MetaMask phishing detector keeps a blacklist of URLs/domains and compares a site's domain against it using the levenshtein distance algorithm. So it could be a false positive. After a quick check I didn't find Auditus on there:

[https://github.com/MetaMask/eth-phishing- detect/blob/master/...](https://github.com/MetaMask/eth-phishing- detect/blob/master/src/config.json)

</comment></comment><comment><USER13> This is cool and I figured we'd be heading down this path soon enough. A lot of the best audiobooks I've listened to were narrated by people that can do multiple voices well. I was thinking that being able to produce an audiobook that uses different voices for different characters would be great. Something like Narrator:

[http://marinersoftware.com/products/narrator/](http://marinersoftware.com/products/narrator/)

Narrator, though, uses Mac OS text to speech, which is nowhere near the level of Polly or Google Cloud Speech.

</comment><comment><USER14> There are a few IOS apps that do this in real time. The best one by far is 'Voice Dream' and they use the same voices. It is basically and audiobook in your pocket anytime, anywhere for any text file and shows the words as it is reading back, start/stop/pause, adjust speed, change voice, etc etc. All around awesome. When the new google voices or equivalent make it to IOS, it will be almost human-like.

<comment><USER15> This is a good example of a tool that was created for the accessibility community (vision impaired, dyslexic) and has subsequently been adopted by mainstream readers.

</comment></comment><comment><USER16> As a language nerd I would like to praise naming the project after a Latin past participle.

</comment><comment><USER17> It's really cool to see the applications made possible by the high quality, reasonably priced, and fairly licensed text-to-speech APIs offered by AWS, Azure, and Google Cloud.

The most fleshed out service of this type that I've found is narro.co, which offers web/pdf/epub/video/rss/email/text to audio conversions.

</comment><comment><USER18> What are the best practices for doing the reverse: taking audio and producing text? I don't mind the translation to be rough, the error rate can be quite high for my purposes, but I want the process not to get stuck and recover so it processes a full length talk.

<comment><AUTHOR> From my experiments generating subtitles from TV/movie audiotrack, 75% (worst case) to 95% (best case). If you model it as a standard distribution, somewhere around 85-90% accuracy. Most services provide much better accuracy for stuff like calls or conferences with proper microphones and minimal background noise than for things like TV shows and movies. If the input audio is noisy, I would do some noise filtering before piping it into conversion.

<comment><USER18> Which conversion tools/services do you have in mind?

<comment><AUTHOR> Google and Azure

</comment></comment></comment></comment><comment><USER19> As an easier problem, what I'd find useful is a way to keep a pirated audiobook and pirated e-book in sync, the way that Amazon does with WhisperSync. A single app where I upload the .epub and the mp3s and it keeps me in sync when I read in either format.

</comment><comment><USER20> You can also do this in iBooks with any of the built-in voices available on iOS.

Just turn on Speak Screen in Settings -> General -> Accessibility -> Speech and then swipe down with two fingers while reading your book. It'll even turn the page for you.

</comment><comment><USER21> This is awesome. But doesn't Amazon bill you for usage? What's keeping you afloat?

<comment><AUTHOR> Each conversion costs a couple dollars depending on length - cheaper than most audiobooks at the expense of human realism. You can listen to a sample of a human read version of accelerando: [https://www.audiobooks.com/audiobook/accelerando/210129](https://www.audiobooks.com/audiobook/accelerando/210129)

The one generated by auditus is too smooth, slightly unnatural

</comment></comment><comment><AUTHOR> Here's a human narrated sample for comparison: [https://www.audiobooks.com/audiobook/accelerando/210129](https://www.audiobooks.com/audiobook/accelerando/210129)

<comment><USER22> That link returns a 404 for me.

<comment><AUTHOR> Try again?

</comment></comment></comment><comment><USER23> I usually do this by hand with surprisingly good results: I use calibre to convert the ePub to txt and then fix some common problems (i.e. remove line breaks and page numbers) using regular expressions. Then I convert it to an audio file using the macOS Automator text-to-speech action (be sure to download the high quality voices first).

</comment><comment><AUTHOR> Update: Server's overloaded right now. Any conversion that has not been sent will be delivered by end of tomorrow.

</comment><comment><USER24> Love the idea about the project. I tried to upload a Epub and got an error page. I tried 3 times and different voices. I look forward to seeing more of it and think it's an awesome idea.

<comment><AUTHOR> Send me the epub and I will convert it for free! Thanks for catching the bug, will look into it soon. Edit: Was the epub in English?

</comment></comment><comment><USER25> This doesn't seem to be working. I have tried uploading a sample epub. After the epub is uploaded it sends me to a _conversions_ page. That page is just a copy of the homepage.

<comment><AUTHOR> Also, you need to select the file type on the left, currently epub is the only option but PDF support is planned soon.

</comment></comment><comment><USER26> Part of me would actually really enjoy having the option to use an old school voice generator. Imagine a horror novel narrated by MS Sam

</comment><comment><USER27> Really neat solution.

Is this in any way based on Amazon Polly?

</comment><comment><USER28> Looks like its down. I get a "The page you were looking for cannot be served." error.

</comment><comment><USER29> Metamask warns me that this site is on the Ethereum phishing list...</comment>


FUDsec - The Wikileaks bandwagon rolls into town - <AUTHOR> http://fudsec.com/the-wikileaks-bandwagon-rolls-into-town

<comment><USER2> This post cuts through the hysteria and offers a solid analysis of the issue</comment>


You are smart enough to be a software developer - <AUTHOR> https://medium.com/@emmarobbins_51556/you-are-smart-enough-to-be-a-software-developer-fed006501486 <comment><USER2> Here's a random observation from experience:

Focused _and_ persistent @ IQ 110 beats unfocused and dithering @ IQ 170 over time.</comment>


The Ten Commandments of Egoless Programming - <AUTHOR> https://blog.codinghorror.com/the-ten-commandments-of-egoless-programming/ <comment><USER2> > The only true authority stems from knowledge, not from position

Try telling that to the boss of your boss. Authority does not birth from knowledge. Compared to what you learn i school, processes and practices guide the vast majority of the decisioning you make day-to-day.</comment>


TSA to retest airport body scanners for radiation - <AUTHOR> http://www.usatoday.com/news/washington/2011-03-11-tsa-scans_N.htm

<comment><USER2> It's a distraction. They don't want the news talking about how they are randomly taking over Amtrak and bus stations around the USA and doing searches now.

[http://cs.trains.com/TRCCS/forums/p/188504/2059127.aspx#2059...](http://cs.trains.com/TRCCS/forums/p/188504/2059127.aspx#2059120)

It's working, not a single word about it in that article.

Somehow the scanners are going to "pass". Or they will ask for funding for new toys and mothball these with the other hardware they've bought but never installed. Anything for a show of force.

<comment><AUTHOR> Thank you for posting that. It's very interesting.</comment></comment>