Image unshredding using a TSP solver - <AUTHOR> https://github.com/robinhouston/image-unshredding <comment><USER2> A bit tangentially, this is also a great display of _just how good_ readily available approximate solvers have gotten for a wide range of combinatorial optimization problems (like TSP).

The LKH solver used here has rather impressive performance. From [http://webhotel4.ruc.dk/~keld/research/LKH/](http://webhotel4.ruc.dk/~keld/research/LKH/): "LKH has produced optimal solutions for all solved problems we have been able to obtain," and the studies linked there show that it really does find optimal solutions "with an impressively high frequency."

While the point here is to use an off-the-shelf solver, it can be nice to have visibility into what's actually happening:

\- This is a reasonable example for explaining some of the local search TSP heuristics. For instance a "2-opt" move corresponds to picking a contiguous range of columns and flipping them.

\- The previous HN post used simulated annealing, which would not be an outright terrible approach to TSP itself -- were it not for the better Lin- Kernighan-based approaches (like LKH).

\- If we want, we can tell LKH to start from Sangaline's "nearest-neighbor" approach ("INITIAL_TOUR_ALGORITHM = NEAREST-NEIGHBOR"). This does not make a difference here though.

<comment><AUTHOR> Not so long ago, we'd say "That's an NP-complete problem" with the implication that it was hopeless to expect an efficient solution. Now (to exaggerate a little) it's more like a sign that there probably _is_ an efficient solution using some powerful solver.

LKH is a lovely piece of work. I used it a couple of years ago to find a counterexample to an unimportant but fairly long-standing conjecture in combinatorics[0], and the list of scientific applications[1] is impressively diverse.

The best SMT solvers are also hugely impressive. Z3 has an excellent interactive tutorial and online solver.[2]

[0] [https://arxiv.org/abs/1408.5108](https://arxiv.org/abs/1408.5108) [1] [http://webhotel4.ruc.dk/~keld/research/LKH/ScientificApplica...](http://webhotel4.ruc.dk/~keld/research/LKH/ScientificApplications.html) [2] [http://rise4fun.com/z3/tutorial](http://rise4fun.com/z3/tutorial)

<comment><USER3> I always thought "NP-complete/NP-hard" metaphor only meaningful in theoretical work. In practice, it does not make much sense in guiding what/how the real world problem is to be solved.

<comment><AUTHOR> I don't know. I find it useful in practice. Apart from anything else it means there's (probably) no point looking for an exact polynomial-time algorithm, which can save a lot of wasted effort. But it also gives useful clues about what sort of algorithmic approaches might be useful, and allows you to relate your problem to a huge amount of existing research.

It's really basic taxonomy: what sort of problem are we dealing with? If you don't know that, how could you even begin to design a reasonable algorithm for it?

<comment><USER3> Maybe you are heavily on huge algorithmic work. In practice, most algorithms are polynomial-time, and we are looking for under-polynomial-time alternative - log(n). For exponential-time, we are just looking for an approximate alternative.

</comment></comment></comment></comment></comment><comment><USER4> I'm really surprised by double-shuffling can actually be solved. [https://github.com/robinhouston/image-unshredding/#double- sh...](https://github.com/robinhouston/image-unshredding/#double-shuffling) It looks very unintuitive.

<comment><USER5> Isn't the definition of insanity something about doing the same thing and expecting different results.

<comment><USER5> Funny, I thought more people would have gotten what I was saying here.

<comment><USER6> You're forgetting HN is serious discussion for serious people or something like that. Humour in a substantive post seems fine, but usually any humour on its own doesn't fare well here.

</comment></comment></comment></comment><comment><USER7> The 2D shuffling result is a bit surprising at first glance, but less so when you think about it - row comparison doesn't care about the order of the pixels in each row as long as the order is the same in both, and the shuffling mechanism guarantees that.

<comment><USER8> the important bit is you have to unscramble in reverse order of the scrambling. Unscrambling the columns and then unscrambling the rows would lead to smooth gibberish.

<comment><USER7> No. Column scrambling doesn't affect row ordering and row scrambling doesn't affect column ordering. They are literally orthogonal. You can't tell, from a scrambled matrix, whether the rows or the columns were scrambled first.

Here's a more explicit breakdown:

        (0,0) (1,0) (2,0)
        (0,1) (1,1) (2,1)
        (0,2) (1,2) (2,2)

Column scrambling: swap 1&2:

        (0,0) (2,0) (1,0)
        (0,1) (2,1) (1,1)
        (0,2) (2,2) (1,2)

Now row scrambling: swap 0&1:

        (0,1) (2,1) (1,1)
        (0,0) (2,0) (1,0)
        (0,2) (2,2) (1,2)

Notice how the x coordinates still all match along the vertical, and y coordinates still all match along the horizontal.

Let's try it in the other order, row first:

        (0,1) (1,1) (2,1)
        (0,0) (1,0) (2,0)
        (0,2) (1,2) (2,2)

And columns:

        (0,1) (2,1) (1,1)
        (0,0) (2,0) (1,0)
        (0,2) (2,2) (1,2)

It's the same result irrespective of the order.

<comment><USER9> Another way to think of it is that permuting rows corresponds to multiplying a matrix on the left by a permutation matrix, and permuting columns, on the right. Since matrix multiplication is associative, they commute!

<comment><USER10> Yet another way to think of it is that a matrix is an encoding of a linear (or affine) function, matrix multiplication is composition of such functions, and since composition is associative, they commute. :)

</comment></comment></comment></comment></comment><comment><USER11> Shuffling lines was the scrambling algorithm used in nagravision a while back... [https://www.youtube.com/watch?v=gFAinn6sddQ](https://www.youtube.com/watch?v=gFAinn6sddQ) there was a software that could descramble the video using a similar technique

</comment><comment><USER5> Now I'd like to see this run on a more realisticly shredded image. A real paper shredder creates strips that are more than one pixel thick are not straight on (I.e. the pixels don't necessarily align with the cuts) and possibly have cutting defects such as ragged edges or nicks.

<comment><USER12> Seems like with a high resolution enough image, some of these effects may actually make the problem easier.

<comment><USER5> Perhaps, but you would have to look at more than just neighboring pixel color to take advantage.

</comment></comment></comment><comment><USER13> Reminds me nagravision decoders (if i recall correctly), one of the first pay tv protection algorithms

Edit: [https://en.m.wikipedia.org/wiki/Nagravision](https://en.m.wikipedia.org/wiki/Nagravision)

</comment><comment><USER14> Could be used to compress images? Perhaps instead of shuffling the columns and rows randomly they could be ordered in ways that are better suited for compression.

<comment><USER15> That's what the Burrows–Wheeler transform does for text. A similar approach could work for images.

<comment><USER16> It's a good point. PNG uses DEFLATE internally for the compression, it would be interesting to see what would happen if you used bzip2 instead, which uses the Burrows-Wheeler transform.

</comment></comment></comment><comment><USER17> Roughly how long does this algorithm take to run?

<comment><AUTHOR> On my laptop (real times for a single run):

1.6 seconds to make the TSPLIB2 file representing the dissimilarity graph. (time make tsp/instances/blue-hour-paris.tsp)

0.3 seconds to run the solver and produce the optimal tour. (time make tsp/tours/blue-hour-paris.tour)

0.4 seconds to reconstruct the image from the shuffled version using the tour. (time make images/reconstructed/blue-hour-paris.png)

</comment></comment><comment><USER18> I wonder how well this would work with some image pre-processing. Wouldn't something like supersampling improve the speed per-pixel? Of course, it would require more pixels to be sorted, so I doubt it's any more optimal.

</comment><comment><USER19> This is super interesting; how does it compare to other sampling algorithms? While the problem is indeed reducible; it also has a _lot_ more structure than TSP, especially if we note that natural images are forced to come from some nice, underlying manifold which is probably very well-behaved.

I'd be inclined to believe that with some assumptions on the problem itself (e.g. the images satisfy some smoothness assumptions and of course, assumptions on the objective, itself), we can always come up with a PTAS for this particular instance for any accuracy, ε>0.

</comment><comment><USER20> I guess this could be used to reassemble images corrupted in other ways (other than shuffling).

I have a disk image of jpegs and other data I'd like recovered. It was Windows formatted a few different ways but any blocks that format didn't touch are still there.

There's other data there of course as well, so there would be lot of blocks that aren't part of images so would need to be ignored when reassembling the images.

<comment><USER21> I'd guess not, actually. In this a lot of the information is there. Just in a different order. Corrupting data is a different problem.

That said, I'm sure I could be surprised by what is possible.

</comment></comment><comment><USER3> I guess the difference with the simulated annealing approach would be that this approach(and the simple picking-the-most-similar-column approach) assumes you know the particular shredding method in advance, whereas the simulated annealing does not have this requirement. For example, does this still work out for shredding diagonally in some unknown degree?

</comment><comment><USER22> How good does it work with non-natural images where rows/columns are not similar to each other? White noise for example should prove rather difficult.

</comment><comment><USER23> Do you know of any image compression technique that would drop information that we expect to be redeemable in a reasonable time using a solver of this sort?

<comment><USER24> Since the shuffling adds information, it seems unlikely.

<comment><USER23> Don't you think that for any image there exist a shuffled version that compress better?

<comment><USER24> I don't see any reason why there necessarily would be. Generally speaking, compression algorithms do better when adjacent pixels have similar colors, which is the unshuffled state.

In cases where there is a version that compresses better, unshuffling might not work. If you had an "pinstripe" image consisting of alternating black and white columns of pixels, obviously it could compress better after being "shuffled" with all the similarly colored columns together. But then this algorithm would be unable to get the original image back.

</comment></comment></comment></comment><comment><USER25> Looks like it's time to go back to burning.</comment>


Show HN: Chrome extension to automatically copy and/or search the selected text - <AUTHOR> https://chrome.google.com/webstore/detail/selectgo/mipenehiegjhcbdlmbbalcimahclffaa?authuser=1

<comment><AUTHOR> Built this over the weekend, with one objective in mind - learn by doing. Would love to hear any feedback/comments/bug reports :).</comment>


Big Picture of Calculus (2010) [video] - <AUTHOR> https://www.youtube.com/watch?v=UcWsDwg1XwM&index=2&list=PLBE9407EA64E2C318 <comment><USER2> Now I know someone is browsing my Youtube history and posting to HN. Youtube really is the math teacher I never had.

Probability Models and Axioms [https://www.youtube.com/watch?v=j9WZyLZCBzs](https://www.youtube.com/watch?v=j9WZyLZCBzs)

The Exponential Function [https://www.youtube.com/watch?v=oo1ZZlvT2LQ&t=100s](https://www.youtube.com/watch?v=oo1ZZlvT2LQ&t=100s)

Vector Space [https://www.youtube.com/watch?v=ozwodzD5bJM&t=36s](https://www.youtube.com/watch?v=ozwodzD5bJM&t=36s)

Tree cutting fails: [https://www.youtube.com/watch?v=JHZkR6UVegY](https://www.youtube.com/watch?v=JHZkR6UVegY)

<comment><USER3> _Youtube really is the math teacher I never had._

Same here. I'm on a quest to run myself through the equivalent of a standard Calc I, Calc II, Calc III, Linear Algebra sequence, and I've managed to find great Youtube resources for all of those things (and more). It's times like these that you marvel at how awesome the Internet can be at its best. :-)

<comment><USER4> Hey I'd love to hear about how your learning experience is going! Care to share your experiences?

I'm about to start a self-directed learning exercise focused on CS related math. I've never used online resources to learn / brush up on complex topics before so I'm a bit unsure what to expect.

<comment><USER3> Sure, I'm at work right now, but I could write up some stuff about my experience later.

</comment></comment></comment></comment><comment><USER5> If you want to get into the details of Calculus

Professor Leonard [https://www.youtube.com/user/professorleonard57](https://www.youtube.com/user/professorleonard57)

Herbert Gross MIT Calculus Revisited: Multivariable Calculus [https://www.youtube.com/playlist?list=PL1C22D4DED943EF7B](https://www.youtube.com/playlist?list=PL1C22D4DED943EF7B)

Herbert Gross MIT Calculus Revisited: Calculus of Complex Variables [https://www.youtube.com/playlist?list=PLD971E94905A70448](https://www.youtube.com/playlist?list=PLD971E94905A70448)

Herbert Gross MIT Calculus Revisited: Single Variable Calculus [https://www.youtube.com/playlist?list=PL3B08AE665AB9002A](https://www.youtube.com/playlist?list=PL3B08AE665AB9002A)

MIT 18.02 Multivariable Calculus, Fall 2007 [https://www.youtube.com/playlist?list=PL4C4C8A7D06566F38](https://www.youtube.com/playlist?list=PL4C4C8A7D06566F38)

MIT 18.02SC: Homework Help for Multivariable Calculus [https://www.youtube.com/playlist?list=PLF07555F3CC669D01](https://www.youtube.com/playlist?list=PLF07555F3CC669D01)

MIT 18.01 Single Variable Calculus, Fall 2006 [https://www.youtube.com/playlist?list=PL590CCC2BC5AF3BC1](https://www.youtube.com/playlist?list=PL590CCC2BC5AF3BC1)

MIT 18.01SC: Homework Help for Single Variable Calculus [https://www.youtube.com/playlist?list=PL21BCE50ABFF029F1](https://www.youtube.com/playlist?list=PL21BCE50ABFF029F1)

<comment><USER3> Second the recommendation for Professor Leonard. He does a really nice job explaining things and making it accessible.

</comment></comment><comment><USER6> I was a teaching assistant at UC Santa Barbara for 3 years, I can tell you, students might slug through the calculations , but none -- or a precious few -- really get the big picture.

Mostly, it is that calculus pedagogy is a disaster. And it hasn't been updated to include recent advances in technology (such as Data Science)

Strang is an amazing lecturer. I can tell you, that far into your Masters and PhD these same basic issues resurface. There's at least one project I can think of where the entire problem hinges on taking one derivative.

</comment><comment><USER7> Gilbert Starng also has an excellent series on Linear Algebra on MIT opencourseware [[https://ocw.mit.edu/courses/mathematics/18-06-linear- algebra...](https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra- spring-2010/)]

</comment><comment><USER8> Honestly I couldn't slog through the video. It started off with "let's use letters" and immediately got to things like "the right letters for this case are 'df' and 'dt'". This just feels like "here's a bunch of magic that you'll just have to memorize". Reminds me of my favorite quote:

> And then Satan said "put the alphabet in math".

When I learned the basics of calculus it was via simple problems that we wanted to answer. And for a while we estimated the answers by brute forcing approximations. I still drop back to this when I can't remember how to do something and need to re-discover parts of calculus.

</comment><comment><USER9> Thanks for sharing such an amazing thing. I am done with studies but should be helpful to understand it properly to teach my kids in future so that they just fill up pages with dy/dx without getting an idea behind it.

<comment><USER10> For kids I'm a fan of [http://dragonbox.com/](http://dragonbox.com/) and their [http://www.dragonboxapp.com/](http://www.dragonboxapp.com/) They do a pretty good job of teaching symbolic algebra to kids who don't yet know arithmetic.

<comment><USER9> Thanks

</comment></comment></comment><comment><USER11> I find that most people understand the general concepts of basic calculus. What's hard to wrap around is when you start getting into proofs (ie. delta- epsilon, trig, e,...and that's just some general ones).

I suppose that's how they differentiate the good students from the just okay students. Personally, I don't think youtube videos will help that much when it goes down to the nitty gritty. You really have to sit down, think, and understand.

<comment><USER12> Keisler has a introductory text book (available free here: [https://www.math.wisc.edu/~keisler/calc.html](https://www.math.wisc.edu/~keisler/calc.html)) that uses non-standard analysis, which he claims is easier for beginners to understand than the notorious delta-epsilon proofs.

I can't say how effective it is at that goal, but it is all mathematically legitimate, and has a nice intuitive appeal to it.

<comment><USER13> Looks similar to this book, which I bought after a recommendation here: [https://www.amazon.com/Infinitesimal-Calculus-Dover-Books- Ma...](https://www.amazon.com/Infinitesimal-Calculus-Dover-Books- Mathematics/dp/0486428869)

</comment></comment></comment><comment><USER14> Having had a non-traditional route through learning various math topics, I'd like to advocate the teaching of calculus (the understanding of the accumulation of infinitesimally-small changes) before the teaching of trigonometry (the relationships between different geometric shapes, and the resulting association with waves).

In my (humbly biased) opinion, it's easier to learn how trigonometric concepts operate with a calculus background, than with merely an algebra-and-geometry background. Trigonometry complicates calculus education - knowing trigonometry may facilitate better calculus understanding to some, but not knowing trigonometry doesn't necessarily complicate learning calculus concepts. The trigonometry-first tradition in math education reflects a "number-phile" bias, not the most optimal route for humans to ingest and retain concepts.

Edit: The "number-phile" bias is clear - we want our teachers to be people with passion on the topic, as we're more likely to get a better understanding of the topic. Understanding sine waves as the result of an integral was much easier for me to understand than as the result of a complicated calculation.

<comment><USER15> It's somewhat misleading/unfortunate to use angle measures as the basic characterization of an angle/rotation, when just trying to solve basic geometry / triangle measurement problems. It's almost always easier to use a vector representation, written in terms of explicit coordinates if necessary (e.g. as a complex number). [http://www.shapeoperator.com/2016/12/12/sunset- geometry/](http://www.shapeoperator.com/2016/12/12/sunset-geometry/)

Where angle measures and sines/cosines start to be useful is with uniform circular motion, which as you say is a calculus problem.

The key to understanding the form of trigonometry courses is to understand the context for their creation. Namely, all calculations (e.g. for astronomy, navigation, engineering, mapmaking, ...) used to be done by hand by humans, which was very expensive. It was important to have someone with fluent knowledge of trigonometric identities simplify formulas to a form with as few arithmetic operations and table lookups as possible to save money, or just to match the available function tables. Today in a computer age, extensive memorization of trigonometric identities is an anachronism, and spending lots of time on practicing their manipulation is okay algebra practice but not anything directly useful per se.

</comment></comment><comment><USER16> I love Gilbert Strang, but _by far_ my favorite book on calculus is "Calculus Made Easy" by Silvanus Thompson. It supplies a very elegant and powerful set of tools to derive many of the "fundamentals" of calculus.

</comment><comment><USER17> What would be some practical "daily life" applications of calculus? I've been discussing a focus on teaching with applicable examples in a lot of subjects, but this is one where I've been struggling a bit.

<comment><USER18> I'm going to expand the definition of "daily life application" from things you actually use daily to things you experience daily. These could be engaging too!

For example... what are sunspots? How does the plumbing around your house work? How does a car battery work? How does a microwave oven work?

Another thought. If we are unable to find applications for a technique, should we place such an emphasis on teaching it? I look around and see all the very practical (and interesting!) math and science that is not being taught, e.g. signal processing, just because they are nontraditional. It's a shame to me.

<comment><USER17> That's kind've where I was going with it. Basic understanding of the building blocks of society type stuff: power, water, gas, construction, radio, biology, growing food, vehicles, finances, etc.

Thinking back, I just realized the main reason that I was bored to tears in high school wasn't so much that the subjects weren't worth knowing...just that at the time I was being taught formulas without any clear understanding of why it was useful to know it. If you don't know why it's useful to know it, there's very little reason to remember it beyond passing a test.

<comment><USER18> You might enjoy this. I actually think this is well done.

[https://www.grc.nasa.gov/www/k-12/airplane/index.html](https://www.grc.nasa.gov/www/k-12/airplane/index.html)

Obviously, it could use more work, but the kernel of something really good is there.

</comment></comment></comment></comment><comment><USER19> Thank you for this video, is there a platform that you could aggregate all this resources mapping them to certain knowledge tree ? So that you could have the big picture and then zoom in for specific topics videos. Something like Khan Academy's knowledge map[1], that you could customise to add you own contents, this video for example.

[1] [https://www.khanacademy.org/exercisedashboard](https://www.khanacademy.org/exercisedashboard)

<comment><USER20> I worked pretty hard on something like this many years ago. I am not a coder so it was a formidable challenge and I never really got it polished or complete. The goal was to create a blank canvas where anyone could organize instructional content from the web (i.e. KhanAcademy, YouTube, etc) into their own class. Each class topic was structured like a subreddit for each topic, with the best results hopefully filtering to the top

<comment><USER21> I just had that idea this weekend - also like a github branch-diagram of math/physics history</comment></comment></comment>


Ask HN: Can young entrepreneurs with no coding experience launch a startup? - <AUTHOR>

Here's my situation, briefly:
A few months ago, I watched customers deal with a widespread problem in what was my industry at that time.
I researched the idea fully for a week or so and found a few services that solved similar problems. I could only ever find one blog post (someone offering up the idea) with more or less the exact idea.
For the next few weeks, I mocked up the idea, wireframed the pages, and generally envisioned how this would work and refined the business model. I asked a few people with relevant industry experience for their suggestions, and made a few minor tweaks on the mock-ups. A dozen industry leaders have already pledged to me that, if I can deliver what I have mocked up, they will definitely be interested in a long-term relationship with my company.
This idea is nearly unique (and its proposed execution definitely has not been done before), but the technology required to execute on it is not particularly complicated.
I had hoped that in the next few months, I could familiarize myself with Ruby on Rails and code this thing myself. As I look ahead, though, I'm realizing that it would be a good idea to have someone on the team who is familiar with other technical aspects of the company - like handling the servers, the bugs, et cetera.
I would love to focus on business development, networking, sales, general admin tasks and marketing, since I have experience running my own (non-technical) businesses in the past. Unfortunately, young start-up founders all seem to be programmers who brought on gray-haired entrepreneurs.
I'm wondering where I should go from here. <comment><USER2> I see two potential next steps:

1) I agree with the other commenters that finding a programmer would be smart. Some places to find such a technical co-founder are:

\+ <http://www.linkedin.com/> \- Go through your network, or do a search for programmers with potentially an entrepreneurial streak. This will take some time and might even cost you some money for their premium account (so you can email people not directly connected to you). You could also join some LinkedIn groups, lurk for a bit, and see if you can find any good fits.

\+ <http://www.builditwith.me/> \- This is a new site and I'm not sure how effective it is. But it's an interesting premise and might not hurt to try.

\+ <http://founderdating.com/> \- This is also a new service and is limited to a handful of cities. If you happen to be in one of the upcoming locations, you could give it a try as well.

\+ <http://news.ycombinator.com/> \- I figured I'd just throw this one out there too. I'm not sure how effective it would be, but conceivably, you could find a technical co-founder here as well.

2) ig1 also brought up a great second option. Try to raise some funding so you can hire a programmer or development team. There are lots of ways to raise seed funding: equity on your mortgage, credit cards, friends & family, investments from those industry leaders, angel investors, etc. You can do some online searches to find a plethora of info out there.

Good luck!

</comment><comment><USER3> You should definitely, definitely, DEFINITELY find a developer. Designer can wait, but I personally believe (based on trying to do the same thing) that the time it takes to ramp up your development efforts will leave you so frustrated that by the time your idea comes to fruition, it will be 10% of what you want and take 300% longer than you expected.

</comment><comment><USER4> First, forget about the dozen industry leaders who pledged to you. Also be honest with yourself in the possible future of this product. Odds are you are not the first.

Make sure if this is built you can succeed without their pledges.

Second, research,research,research the idea extensively.

One you have done that do as much work as possible as you can in building it, as far as you can go, this will allow you to see your idea through.

Once you have done that if you are still interested in building it, either

1) ask a coder friend, or find a coder and put them on as co founder

2) outsource the development

If you have no $ than try to produce it yourself no matter what

\- non coder

</comment><comment><USER5> Find a co-founder with technical expertise or hire someone are the two main options you have.

If you can't afford to hire someone yourself than raise money, if you've got a solid business model with clients lined up you might be able to find an angel willing to finance. Or alternatively see if you can get some of the industry leaders who are interested to provide financing.

<comment><AUTHOR> I have $1000 that I'm willing to throw into development right now, but I'm not sure how to spend it wisely. It seems like $1000 might not be enough to code a fairly simple application. What happens if the money runs out before the project is finished? Would I end up with a non-functioning set of code, or would the feature set just end up severely reduced?

<comment><USER6> This is obviously not enough to find anyone worthwhile. That being said, your best bet is to pitch a programmer who is willing to join your team for equity.

I would suggest going to local events, meetups, etc and reach out to a bunch of potential candidates..

</comment></comment></comment><comment><USER7> My recommendation would be to think in terms of a minimum viable product. You said you have $1000. That amount might not be enough to build your entire product, but is enough to build _something_. And that something would be enough to get further feedback--feedback about demand, but also about more specific things like pricing and marketing.

While your mention of wireframes and mockups makes me think you might already be walking the Lean Startup path, I don't think it's necessary to make the leap to a full-blown product just yet, particularly since you're not in a position to build it yourself.

</comment><comment><USER8> If you really want to you can, but I would recommend working together with a experienced programmer. Anyone can hack something together after a few weeks of learning and basic example code, but if you want to program something you can rely on for the next couple of years you can do yourself a favor by working on a solid foundation. You say people are interested in your product, but will they still be when it's done? If you got the time and money to work on it for at least 6 months I say go for it, you're going to regret that you haven't tried if you don't ;)</comment>


On Gender in Minecraft - <AUTHOR> http://notch.tumblr.com/post/28188312756/gender-in-minecraft <comment><USER2> The fact that this even have to be said is retarded.

<comment><USER3> Now someone will have to write a blog post about your usage of "retarded".

</comment></comment><comment><USER4> This is not HN worthy.</comment>


How I Almost Destroyed a £50M War Plane and the Normalisation of Deviance (2016) - <AUTHOR> https://fastjetperformance.com/podcasts/how-i-almost-destroyed-a-50-million-war-plane-when-display-flying-goes-wrong-and-the-normalisation-of-deviance/ <comment><USER2> It's really nice to have a term to pin to a vague concept that had been floating around in my head for at least a decade. I've come across instances of this repeatedly but never knew quite how to describe it properly to others. It's a treacherous slope where the first step is usually completely un-noticed and by the time people do notice it is way too late. We've always done it like this is the oft heard excuse but on closer examination that usually isn't true. Typically at some point in the past there was a much tighter regime and the present day situation would have definitely not passed muster. But by inching towards it with very small steps critical thresholds can be exceeded without notice.

Thanks for posting this. It will help me professionally to formulate my opinions on such situations more clearly and will serve as a reference point for future discussion.

<comment><USER3> I think for every 100 people who talk about Vaughn's "Challenger Disaster" there are maybe 2 or 3 who actually read it.

Like many books in sociology it comes at the problem from multiple perspectives (in different chapters) such that you can't really hold the author to any one clear conclusion.

Big picture, the space shuttle was "unsafe at any speed" from the very beginning. It was estimated that it had a 3% or so chance of blowing up on any flight which turned out to be just about right in hindsight.

One point of the book was that the Space Shuttle had many things wrong with it: they had a committee who's job it was to review all the safety risks and decide which ones they could accept. "Normalization of deviance" was not a bottom-up matter of bad practices, but a formal part of the organization.

If things had gone a little different with the O-Rings then perhaps the "Challenger Disaster" wouldn't have happened, but then it would have been something else. I remember in the early days the thermal tiles were falling off, but after they had done a few flights they convinced themselves it was OK. It was the Columbia disaster that proved that they weren't OK and couldn't ever be OK.

<comment><USER4> " _...they had a committee who 's job it was to review all the safety risks and decide which ones they could accept._"

Having a committee dedicated to assessing risks seems like a good idea. Having a group consciously deciding between risks you can live with, risks you have to live with, risks you can mitigate, and risks that stop everything is a fundamentally good idea.

The problem is taking risks without thinking about them, or taking risks without knowing exactly why they haven't killed you before. And even more unfortunately, this is all on an experimental project That Cannot Ever Be Allowed To Fail, Ever™.

<comment><USER5> Do committees who do such reviews equally ask themselves - if this does fail - can we live with it and justify our initial risk assessment and stand by it?

So often - risky decisions are made - many pass without incidence and yet, problems happen. Often such risks are reviewed in hindsight and the weight of the realisation of risk that pass into actuality. Though when they don't fail, nobody passes a second thought and over time, such risks get normalised.

<comment><USER6> That's a fair point, but the fundamental fact is that at some point, someone/something/some process needs to accept a level of risk, or perform some type of cost/benefit analysis. Sometimes the analysis is straight forward or easy to criticize. For example, when the costs are high, and benefits relatively low. It's super easy to criticize the shuttle disasters, because the cost of cancelling the launches are so obviously lower than the lives of astronauts (at least in popular discourse).

The issue is that all mechanisms to do so are susceptible to some variant of the problem you highlighted. As so highlighted in the OP's submission, individuals can make the exact same mistakes.

I feel that a lot of things comes from the fact that we are constantly encouraged to over-quantify risk. That is we take things where REALLY we don't have a good grasp of the real numbers, and then make something up. If we're good, they'll even be in the right ballpark. If we happen to choose to follow the course of action, and everything is working "just fine", we go back to our mental risk model, which we always knew was kinda made up, and then say "well, I guess I guessed wrong", and assign a lower risk.

<comment><USER5> Yes, it does get down to risk factors and the parallels with the reinsurance/insurance market with actuaries, is an aspect not lost upon me. As we know you can't put a price on a human life - but they do in the end.

</comment></comment></comment></comment></comment></comment><comment><USER7> This was a really good reminder of engineering wisdom.

Over the year I find myself and my team (civil-structural engineers) doing even more and more riskier designs. Things we wouldn't have considered OK during our early years. We do get review and approvals from seniors, but maybe their acceptance is also "normalisation of deviance" as described in the article. Sharing it with my team.

<comment><USER5> Would it be a case of the norm was to over-engineer (thinking victorian times mostly as examples of structures standing and still in use) compared with today in which we have a finer insight into the maths, which allows us to engineer just-enough.

Or is it a case of market/management pressure pushing the limits that is the driver? With those pushing the limits getting a competitive edge - that draws others into competing in the same way.

</comment></comment><comment><USER8> When it was being described the checklist a pilot has to go through before taking off, it resembled, though quite crudely and without life or death implications, the lengthy checklist the developers at our company had to follow back when we deployed everything manually. We had synchronization steps to pull modifications, couple compilation steps, we had to check if configuration files were added in order to accommodate extra services, there was cache mechanisms that had to be cleared, after that we needed to check for database migrations, then restart the systems, run the little tests we had and manually check if nothing broke. It happened often to see code conflicts, 40 to 50 with varying difficulty every release. It was insane and never quite worked, we always got weary of release days and never wanted to go through them as we didn't trust the system (with good reason).

Automation and good practices brought back confidence, so I was wondering two things.

First most obvious, how come do pilots trust a system that has these layers of manual security checks with their lives. And second, can't it all be automated? I understand aircrafts must have independent modules and every level of integration implies extra risk, but at least in security critical tests, automation would come a great way in ensuring crew safety, wouldn't it? Is it too hard? Systems are not flexible to implement automatic security checklists or is it just a culture in the field to check it all manually?

<comment><USER9> > First most obvious, how come do pilots trust a system that has these layers > of manual security checks with their lives. And second, can't it all be > automated?

The trust comes from understanding the system and associated risk that contribute to conditions where the system is out of normal operating range. Pilots learn that doing simple checks on parts of the system that are likliest to contribute to abnormal operations is life saving in the long run. E.g. may seem simple that control locks werent taken off but it happens and it's why we check the controls prior to take off. It's why we put the engines to fill throttle to make sure oil pressure is as expected.

Anytime there is an accident many pilots will read the reports and add an additional check or reemphasize a current check that they need to continue to use.

As far as automation. It could fix it. But it increases system complexity so there will be this valley where for a while the automated system isn't much better than normal checks, infact might be worse due to over reliance. Also logic in software or hardware can lie to you in ways that mechanical realities of moving control surfaces, checking oil level, checking tire pressure can't be beat (at least with a cheap solution). This is why the FAA for critical components requires multiple systems doing the same thing and concensus algorithms.

Recently I bought a lemon for a motorcycle. The ECU started giving me error codes twenty five minutes into my first ride. Engine started cutting out. I couldn't start it. On the other hand in a Cessna I could lose all electrical power (assuming magnetos are still operating) and still operate the plane just fine, although without radio comms. The Cessna is from 1972, while the motorcycle from 2017. Granted I could've checked wires or inspected more carefully but non the less, the 2017 motorcycle is a more complex system.

<comment><USER10> The motorcycle engine is LESS reliable than a well maintained Cessna. But I suspect the motorcycle was not well maintained, perhaps not well designed, and you should not have been driving it.

<comment><USER2> Short of a teardown I don't see how the buyer of a second hand motorcycle would have a chance at spotting such a fault without driving the bike. And even a teardown might not show the fault, in fact could make it worse.

<comment><USER11> That's equally true for an aeroplane engine - the difference is that no pilot would skip performing maintenance at the required intervals and there are maintenance logs etc. in place to enforce this. What makes the Cessna engine more reliable than the motorbike engine in practice isn't a simpler design but a better maintenance and operation culture.

<comment><USER2> Sure. But the accepted norms for buying second hand aircraft versus second hand motorcycles are quite different as well, which is why it makes no sense to say that the buyer should not have driven the motorcycle. The accepted norm for cars and motorcycles is that you pay the seller and you drive them home, not that you inspect the detailed maintenance log or do an on-site teardown. Likely the seller was aware that this bike had problems and simply did not tell the buyer, something that is close to fraud, and on top of that this placed the buyer in a potentially dangerous situation, especially on a new bike.

When I buy cars I do a pretty thorough inspection and an extended test drive. Even so I once ended up with a car on which the clutch pedal would not return once every few hundred strokes or so, something that by chance did not happen even once in a pretty long test drive.

</comment></comment></comment></comment></comment></comment><comment><USER12> About 2 weeks ago a German Luftwaffe Global 5000 Businessjet missed the runway and nearly crashed in an emergency landing. The currently suspected cause: Faulty maintenance and incomplete preflight checks. [http://www.airliners.de/in-schoenefeld-jet- flugbereitschaft/...](http://www.airliners.de/in-schoenefeld-jet- flugbereitschaft/49799)

</comment><comment><USER13> An interesting paper about systems on the edge of failure is "'Going solid': a model of system dynamics and consequences for patient safety" by Cook and Rasmussen: [https://qualitysafety.bmj.com/content/14/2/130](https://qualitysafety.bmj.com/content/14/2/130)

There's a good overview in this talk: "Architectural Patterns of Resilient Distributed Systems" by Ines Sombra [https://youtu.be/ohvPnJYUW1E?list=PL1Fqq0rxeNehyA4eWbuCwizkq...](https://youtu.be/ohvPnJYUW1E?list=PL1Fqq0rxeNehyA4eWbuCwizkqLeL9-6aE&t=589)

</comment><comment><USER14> Nice read. I often notice something similar during software development. It's simple stuff like "I have to do a full rebuild or it won't work" or "I have to build twice" etc pp. I cannot understand how others can simply ignore what could indicate a fundamental problem.

It's fucking terrifying.

<comment><USER15> I notice this too. However I've come to see that the building twice folks are cranking on writing code and getting praise from management while I'm finding some bug in yarn and not getting any actual work done..

</comment></comment><comment><USER16> > Before you try and change the world just have a look at the foundation from > which you are starting it from.

So true. People who think they're doing the right thing can cause so much damage.

</comment><comment><USER17> Podcast seems to be 404 on libsyn. I was worried it'd gone Spotify exclusive but the correct feed URL is [https://feed.pippa.io/public/shows/5bb78b332630bb48399f39fe](https://feed.pippa.io/public/shows/5bb78b332630bb48399f39fe) and the episode URL is [https://feed.pippa.io/public/streams/5bb78b332630bb48399f39f...](https://feed.pippa.io/public/streams/5bb78b332630bb48399f39fe/episodes/5bb78b4fac8922f843285be0.mp3) (as an alternative to the linked article).

</comment><comment><USER18> I see this sort of complacent deviance all the time at my job. Thankfully we don't can't kill people with it. But insular organizations seem to thrive on doing things "their way" thinking it right, when in fact to anyone else it would be total lunacy. If people saw what we did I think people on HN would have a good laugh.

</comment><comment><USER19> This reminds me a lot of this idea: [https://www.lesswrong.com/posts/Kbm6QnJv9dgWsPHQP/schelling-...](https://www.lesswrong.com/posts/Kbm6QnJv9dgWsPHQP/schelling- fences-on-slippery-slopes)

"Normalization of deviance" would be the mechanism by which a slope becomes slippery; a schelling fence (in this case... a checklist?) is then a possible fix, even if it's just to trigger a re-evaluation of "normal"

</comment><comment><USER20> I'm amazed that outside a wartime situation that flying a plane like that with a known equipment failure (of that nature especially) was allowed. And if it wasn't allowed, I'm surprised it didn't end up in a grounding of the pilot.

<comment><USER21> It wasn't just the pilot. It was also a few people who heard the idea and said that's ok. You can't fix systemic issues by punishing selected few. Aviation seems to be pretty good at that.

</comment></comment><comment><USER22> IMO the article is excellently written and very compelling. I suggest people read it for enjoyment, despite the dry title.

<comment><USER23> What do you do on a daily basis that makes "How I Almost Destroyed a £50 million War Plane" sound dry?

<comment><USER22> Good point. I was focusing on the "... Normalization of Deviance" part when I wrote that.

</comment></comment></comment><comment><USER24> A lot of incidents seem to arise from normalization of deviance + self regulation (especially in the moment). And hubris.

The author's situation reminds me of the Tenerife airport disaster- absolutely macabre and sobering. [0]

[0]: [https://en.wikipedia.org/wiki/Tenerife_airport_disaster](https://en.wikipedia.org/wiki/Tenerife_airport_disaster)

</comment><comment> awinter-py First heard this term in the danluu 'wtf wtf' blogpost -- so useful. Also 'fast jet performance' is such a cool podcast topic.

<comment><USER25> This one: [https://danluu.com/wat/](https://danluu.com/wat/)

Great articles there.

</comment></comment><comment><USER26> Everyone here seems to be caught up in waxing poetic about how the normalization of evidence is a bad thing and talking about preventing it but there's another side to that coin. If everybody is breaking the process 9x/10 it's not "normalization of deviance", it's because you've got a half baked process that sucks and the people on the ground know it and work around it. Obviously you could fill a library with all the material written on how to write organizational policy but it's not as black and white as "violating policy = bad"

</comment><comment><USER27> _Think of the global financial crisis of 2008 when the economy collapsed because the banks hadn't been properly regulated because they convinced the authorities that they could do it themselves._

Also Boeing and the MCAS fiasco

</comment><comment><USER28> Excellent read and a key concept.

I'd had a general idea about this, but it was crystalized by a vehicle dynamics engineer and racer who mentored me in both engineering and race driving. He said:

"You need to know the difference between skill and getting away with something"

I.e., just because you survive an event does not mean that it is reliable. If it is based on skill, you can replicate it and reliably survive. If it based on luck, well . . . take the bit of grace you were given by the Gods and learn your lesson to not do it again. Mostly, THINK.

</comment><comment><USER29> "How I Almost Destroyed a £50M War Plane and the Normalisation of Deviance"

versus

"How I Almost Destroyed a £50M War Plane [while nearly killing myself and my colleague] and the Normalisation of Deviance"

<comment><USER11> Yeah I thought that was interesting. Perhaps an artifact of how soldiers are trained to regard their own lives versus their equipment?

</comment></comment><comment><USER30> "Normalization of Deviance" sounds like the name of a ship from the Culture series.

</comment><comment><USER31> Tried to read your article, but the way the email prompt loaded made it impossible to dismiss so I entered a false address.

I'm running the latest version of Chrome on Ubuntu 16.04 - hope that helps you diagnose this issue.

Here's a screenshot: [https://imgur.com/kwXdIsa](https://imgur.com/kwXdIsa)

</comment><comment><USER32> The concept of Normalization of Deviance was mentioned in this thread in March about some comments made on Twitter by a software engineer about the 737 Max crashes.

[https://news.ycombinator.com/item?id=19414775](https://news.ycombinator.com/item?id=19414775)

</comment><comment><USER33> Complacency creep.

Talking to a few Royal Australian Air Force folks last year, apparently they had a similar-ish culture of "she'll be right" arrogance/confidence, and then they had a cluster of accidents that caused a foundation level rethink and rebuild of a safety/process culture into every flight and every high risk activity.

It sounds like accident #'s have collapsed since then.

Be careful, aviation fanboy writing, not a qualified pilot/tech/engineer.

</comment><comment><USER34> A few past related discussions: [https://hn.algolia.com/?query=Normalization%20of%20Deviance%...](https://hn.algolia.com/?query=Normalization%20of%20Deviance%20points%3E20&sort=byDate&dateRange=all&type=story&storyText=false&prefix=false&page=0)

</comment><comment><USER35> What I find odd is that he chose to get his 0g flight by flying a parabola, in the style of the vomit comet, instead of just flying in a roll. For example, roll by 200 degrees and then raise the nose by about 20 degrees. (probably can't just roll 180 degrees without stalling)

</comment><comment><USER36> In my experience this also applies to dating, it's really easy to overlook things in the early stages of a relationship (or what might become a relationship) that turn out to be quite significant later.

</comment><comment><USER37> This read reminded me of one of the famous TOP GUN movie quotes:

"A good pilot is compelled to always evaluate what's happened, so he can apply what he's learned. Up there, we gotta push it."

<comment><USER38> Top Gun...

</comment></comment><comment><USER39> Anybody reminded of Captain Flashheart while reading this?

</comment><comment><USER40> When I think of "normalization of deviance" I focus more on mundane things like speeding.

I'm a cyclist. People often ask me why cyclists break the law so frequently. The studies I've looked at don't suggest cyclists break the law any more frequently than drivers do, but they do break different laws. (To my knowledge I follow every applicable traffic law.) When I ask the driver I'm speaking with if they speed I frequently get a justification of speeding as if it's perfectly safe because "everyone does it".

In terms of problems people should focus on, cyclists tend to break laws in ways that mostly harm only themselves (e.g., running a red light), while drivers break laws in ways that tend to harm others in addition to themselves (e.g., speeding, failure to yield, etc.). The fact that drivers vastly outnumber cyclists also makes the priorities even more clear.

<comment><USER41> > cyclists tend to break laws in ways that mostly harm only themselves (e.g., > running a red light)

Running a red light could easily harm others - a driver could swerve to avoid you, crashing in the process; if a driver hits you because you ran a light, that could also cause a chain of accidents; and if a driver killed you because you ran a red light, that could seriously harm their wellbeing.

<comment><USER40> > if a driver killed you because you ran a red light, that could seriously > harm their wellbeing

While there are people who would be seriously bothered if they inadvertently contributed to a cyclist's death, I think the number of such people is smaller than the number of people who claim they would be bothered.

One only needs to look at their revealed preferences to show this. Given the choice between waiting a minute or two (estimated 95th percentile; the median is likely around 15 seconds) to safely pass a cyclist, many drivers instead choose to pass a cyclist very closely and dangerously. If they actually would be bothered if they killed a cyclist, why do they do this? My life is not worth a minute of someone's time.

Of course, the bad drivers could claim that they didn't know passing a cyclist closely was dangerous, but in my experience few drivers claim that. I've spoken to many drivers when stopped at stop lights, trying to understand their perspective. Often they imply that I _deserve_ to be passed dangerous because I broke some rules of the road. So they can't claim ignorance.

(The "rules" I broke are not actually rules. For example, I've had people tell me that I was going far below the speed limit when I was going 17-18 mph in a 15 mph zone. I intentionally picked that road because of the low speed limit, yet some people want to drive 40+ mph there and apparently that makes me a jerk who deserves to be run over.)

<comment><USER42> You cannot use driver behavior in the absence of an accident to infer the emotional impact of killing someone...

<comment><USER40> Why? These drivers often show no remorse for their actions, as I stated. I don't doubt I'd be a victim of a hit-and-run if any of those drivers actually did cause a collision.

I've seen arguments that people are "different" when driving, so perhaps they're a jerk when behind the wheel but a normal person otherwise. I haven't seen any clear evidence for this but it would indicate that the emotional impact would be different.

<comment><USER43> > I don't doubt I'd be a victim of a hit-and-run if any of those drivers > actually did cause a collision.

The only way I can read this is that you believe, of the people that cause a collision with a bicycle (accidentally or otherwise), all (or almost all) of them would just leave you to die on the side of the road.

That is a staggeringly low opinion of humanity.

<comment><USER40> The low opinion is not of humanity, rather, certain drivers who are particularly dangerous.

I didn't have all drivers who'd be in a collision with a cyclist in mind, just a large fraction of those who deliberately pass cyclists closely. My wording was too strong.

Hit and run crashes account for roughly 12% of all crashes in the US, so they may be more common than you believe: [https://aaafoundation.org/hit-and-run- crashes-prevalence-con...](https://aaafoundation.org/hit-and-run-crashes- prevalence-contributing-factors-and-countermeasures/)

</comment></comment></comment></comment></comment></comment></comment><comment><USER44> Climate change is another example of this probably. Except the whole planet instead of a plane.</comment>


LispyScript - <AUTHOR> http://lispyscript.com/ <comment><USER3> Neat!

Now look at: [https://github.com/clojure/clojurescript](https://github.com/clojure/clojurescript)

And try it: [http://himera.herokuapp.com/index.html](http://himera.herokuapp.com/index.html)

<comment><USER4> Clojurescript has to be compiled by the google closure compiler on the server side. The closure compiler doesn't work out of the box with other libraries, you need some kind of bridge.

Lispyscript has the very nice advantage of being able to run directly in the browser. Since it's translated directly to javascript, I expect it won't have problems using other javascript libraries.

<comment><USER5> Some inaccuracies here. ClojureScript does not have to be compiled by Google Closure. That pass is an optional one for production builds. There is no "bridging" when using the Closure Compiler with non-Closure compatible code. The issue is that in production mode the Closure Compiler will make aggressive assumptions about what it can rename. So it's not about bridging it's about preventing renaming - again this is only relevant for advanced production builds.

That said for non-Web applications or applications where advanced compilation isn't that useful providing a bootstrapped ClojureScript is desirable. We've been working on that slowly for a long time now. In the coming months you'll see changes such that the ClojureScript compiler can itself be compiled into JavaScript.

<comment><USER6> > "again this is only relevant for advanced production builds"

So, all real-life builds that anyone would care about.

<comment><USER7> Think about where you would use CLJS - my use cases would not have an issue with extra 100kb of code - you don't use CLJS to do jQuery animations on your web page - you would use it to build complex single page apps or write server side code/scripts.

You don't use clojure for performance anyway, it's going to be slower by default (because of immutability/persistent data structures, and yeah I know about react benefits with immutability that's not my point - you're still going trough a lot more memory and stressing GC) - you use it to help you deal with your code because of it's semantics.

But in reality last time I tried CLJS I didn't really feel like it delivers on the productivity part and it's mostly because of implementation issues. IMO the decision to implement CLJS on top of Closure compiler and in Clojure (instead of going for a self hosting compiler) was a mistake - you can't overstate the value of REPL and fast iteration in a language like Clojure - and my last attempt to use CLJS the compiler/REPL environment far from what I would consider fast iteration : compiler took forever to start because of JVM and while it could run as a service it needed to be restarted frequently enough that it mattered, REPL was very unstable it would just die randomly - sometimes you'd need to refresh the page, sometimes you'd need to restart the server process. Oh and don't even get me started on the voodoo needed to get the damn thing running - install piggieback, then install austin and then add this weasel thingy then configure the server process all so you can get a halfworking repl and pray it doesn't break because good luck figuring out what's actually going on. Compare this to JS where I just go in to the devtools panel and test my code.

<comment><USER8> Things are progressing and I encourage you to have another look.

        $ lein new figwheel myproject
        $ cd myproject
        $ rlwrap lein figwheel

        Browse to http://localhost:3449

If you return to the terminal, you should have a connected REPL now running. Along with live code reloading on save of your cljs files.

<comment><USER7> Maybe, but things progressed way further outside of CLJS space. If you told me this 2 years ago when I was in to it I would be jumping right on it - <USER2> when people were saying coffescript fixed JS problems :D

Right now JS has persistent datastructure libraries and TypeScript is a huge productivity boost - tooling is top notch - it makes JS manageable, once it gets async/await (which it should in the next couple of months) I'll be pretty happy with JS development story.

I'll miss some niceties like collection operations, homoiconicity and macros but on the other hand I have working optional type checking, excellent tooling and I don't have to code in AST serialization format with macros :D

<comment><USER8> Interesting. Do you have a link to a very basic "get-started" in this space? Right now, figwheel / om or figwheel / reframe are very quick to get started with (although familiarity definitely plays a part..). Last time I looked at js there were 5 or 6 or so flux implementations competing for mindshare and I had to stumble my way through setting up a project with webpack / babel..

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> Coincidentally, I've been fiddling with a rather similar project, just as a hobbyist thing. It looks like this one is much farther along, kudos! Unfortunately from the contributions graph perhaps interest in it is dying off -- the last 10% (aka the last 90%) is always the hardest part to slog through...

I've been meaning to write up the various approaches to sexpressions+macros in JS. Mine differs from the others (and perhaps is closer to LispyScript) mostly in that it's close to JS in its names and semantics (e.g. "function" defines a function and you're still required to use a "return" statement), but then it lets you write macros to e.g. define "fn" where the return is implicit.

Anyway, here's some sample code from mine (which is itself defining macros used elsewhere in the compiler): [https://github.com/martine/pjs/blob/master/lib/macro.pjs](https://github.com/martine/pjs/blob/master/lib/macro.pjs)

</comment><comment><USER10> Doesn't look like Lisp, more like Clojure. Basically none of the functions, macros or syntax is from Lisp.

The documentation says:

[http://lispyscript.com/docs/](http://lispyscript.com/docs/)

> LispyScript is not a dialect of Lisp. There is no list processing in > LispyScript . LispyScript is Javascript using a Lispy syntax (a tree > syntax).

That's about right. It actually uses some kind of s-expressions, but not Lisp syntax.

<comment><USER11> > Basically none of the functions, macros or syntax is from Lisp.

Sounds like it's extremely well named, then. LispyScript is to Lisp as Javascript is to Java: an entirely different language with different syntax, semantics, standard library and performance characteristics, but with just enough superficial similarity to provide plausible deniability for the name.

<comment><USER10> Well said! ;-)

</comment></comment></comment><comment><USER12> See also: [https://github.com/vsedach/Parenscript](https://github.com/vsedach/Parenscript)

TLDR: s-expression syntax for javascript, macros are written in common lisp.

<comment><USER13> Macros can actually be written in Parenscript as well, AFAIR. But the language blends itself very naturally with CL code.

<comment><USER12> parenscript macros are in straight common-lisp. They expand to parenscript, of course.

</comment></comment></comment><comment><USER14> Also check out LiveScript.

[http://livescript.net/](http://livescript.net/)

and its awesome FP library inspired by haskell's prelude.hs

[http://www.preludels.com/](http://www.preludels.com/)

I have done all forms of projects using LiveScript - robotics, simple websites, blog, cryptography, computer vision.

Its actually becoming silently fairly mature.

It helps when it doesn't generate any hype like most languages.

The community around is also very helpful !

And LiveScript is awesome with React.js or any other virtual DOM based MVC framework.

<comment><USER15> Why did they name it that?

LiveScript was Netscape's original name for JavaScript, before Sun asked them to throw in some Java branding.

<comment><USER16>
      > Name
      >
      > LiveScript was one of the original names for JavaScript, so
      > it seemed fitting. It's an inside joke for those who know
      > JavaScript well.

</comment></comment></comment><comment><USER17> Doesn't it say something about JavaScript, dissatisfaction with it, and the overwhelmingly splintered ecosystem, when _every_ comment is suggesting alternatives to the solution in the article?

I guess we chalk this one up to "neat if you're a hobbyist or solo dev with no maintenance handover, but generally commercially unviable."

</comment><comment><USER18> > An inherent problem with Javascript is that it has no macro support, unlike > other Lisp like languages.

[http://sweetjs.org/](http://sweetjs.org/)

There are macro systems for Javascript, just not native ones.

<comment><USER19> Not that you're wrong -- you're correct -- but I want to bring up that this was likely true at the time this was written. LispyScript has been around for a while.

<comment><USER20> First commit for lispy - march 5, 2012 First commit for sweet.js - August 1, 2012

im actually pretty shocked they are that close together.

<comment><USER19> I bet they both started around the time CoffeeScript started to get big -- it showed there was a market for compile-to-js-but-almost-js langs.

</comment></comment></comment></comment><comment><USER21> I wouldn't say Lisp-like syntax is necessary for a macro system. It helps a bit... but all things considered, I believe pattern matching is a bigger boon to macro writing than syntax per se.

For those it may interest, I have made a language with mostly conventional syntax which supports macros: [http://breuleux.github.io/earl- grey/](http://breuleux.github.io/earl-grey/)

The macro system is modular, so you can easily write and publish macro libraries. I have written some for testing, gulp, and react. It's not _super_ mature but it's getting there.

</comment><comment><USER22> Also Spock (Chicken Scheme wiki): [http://wiki.call- cc.org/eggref/4/spock](http://wiki.call-cc.org/eggref/4/spock)

</comment><comment><USER23> Wisp should also be added to this list: [https://github.com/Gozala/wisp](https://github.com/Gozala/wisp)

</comment><comment><USER24> Nice. This is reminiscent of HyLang[1], which is like this for Python.

[1]: [http://hylang.org/](http://hylang.org/)

</comment><comment><USER25> There is no shortage of JavaScript-Lisps, that's for sure!

</comment><comment><USER26> LispScript | [https://bitbucket.org/ktg/lispscript](https://bitbucket.org/ktg/lispscript)

Try LispScript | [http://ktg.bitbucket.org/lispscript/lispscript.html](http://ktg.bitbucket.org/lispscript/lispscript.html)

</comment><comment><USER27> Nice! reminded me of [http://ympbyc.github.io/LittleSmallscript/](http://ympbyc.github.io/LittleSmallscript/)

</comment><comment><USER28> BiwaScheme is also pretty decent. [http://www.biwascheme.org/](http://www.biwascheme.org/)

</comment><comment><USER29> also [http://sibilantjs.info/](http://sibilantjs.info/)</comment>


Show HN: sMAILs - a HTML-Newsletter-Editor (weekend project) - <AUTHOR> http://smails.herokuapp.com/

<comment><USER2> This definitely looks useful. I have a bunch of templates generated with the mailchimp tools that I've not touched in ages. Doesn't help that I've since forgotten how their tools work. Something like this would be a great way to prototype and quickly update them. I can cope with remembering how to copy and paste into a window even if it is once every six months.

But the 'Generate HTML with Inline Styles' button isn't appearing as a button (and doesn't do anything when clicked on) when running the latest Chrome on Win 8.1

<comment><AUTHOR> Thanks for your feedback. The button should work now.

</comment></comment><comment><USER3> My only suggestion is to support LESS.

<comment><AUTHOR> Added LESS and HAML support.</comment></comment>


OpenSNP - <AUTHOR> https://opensnp.org/

<comment><USER2> Why would anyone upload their genome to an unaudited site?</comment>


Trippy 3D projections onto buildings - <AUTHOR> http://www.projectiononbuildings.com/en

<comment><USER2> Anyone have any idea how this works? The images are very bright, and there don't seem to be any issues with the buildings casting shadows on themselves.

If this is just a projector in front of the building, I've never seen a projector that powerful. And the effect will break down at any other viewing angle.

<comment><USER3> I've been curious about this as well, I found this post by a guy who gave a few details: <http://crustea.vjfrance.com/article-21479794.html>

      Tech specs:
      Here are a few few technical details, the biggest mapping I've ever done so far.
      * 6 * 20 000 lumens projectors, to project on the 160m x 10m facade.
      * We'll use high res visuals: 1024x768 on each projector, so the output image is 6144x768.
      * To stream such High res format, we'll have 2 "killer" media servers, to run very large video files.
      * We'll use the VJ software Arkaos 3.6 to control the video, with multi output and Midi support, we'll be able to synchronize the visuals together and stream Gigabytes of video very smoothly.
      * VVVV will be used to sequence and root midi MIDI signals between computers.

Edit: formatting

</comment></comment><comment><USER4> This is why I follow HN... people post a lot of stuff that I wouldn't give to hoots about but every now and then you find something really cool. This 3d projections onto buildings is freakin cool!

</comment><comment><USER5> Does anybody have any third-party video of these? I would like to see what the effect is like when not viewed from exactly the right angle (and also proof that it's not just fake).

<comment><USER6> There were many other similar installations, see for example here:

[http://www.youtube.com/results?search_query=3d+building+proj...](http://www.youtube.com/results?search_query=3d+building+projection)

Edit: BTW based on author's comments on Vimeo, the first video (top right) in the original post is indeed fake, second one (bottom left) is real one, from live performance.

</comment></comment><comment><USER7> Watching this in person, your eyes give you more depth information, so I would imagine the 3D illusion is weaker.</comment>


A Nuclear Jockstrap - <AUTHOR> https://carlwillis.wordpress.com/2017/02/03/a-nuclear-jockstrap/ <comment><USER2> I wonder what will be the equivalent a hundred years down the line.

In other words - do we do something today which will seem as hare-brained to our descendants as irradiating one's genitals does to us? (We probably do - but what will it be...)

<comment><USER3> Homeopathy.

<comment><USER4> I really wish there was a law that forbids using the Homeopathic label on a product that isn't really homeopathic, but just non-medicinal. For example, there are some teas or cough drops that are soothing when you have a bad cold (in a similar way to how lemon or honey helps you feel better). Often these are labeled as homeopathic, even though they contain no homeopathic poisons. I don't mind getting non-medicinal remedies, but when I see the homeopathic label I tend to shy away.

<comment><USER5> > even though they contain no homeopathic poisons.

Something that is genuinely homoeopathic doesn't contain any of the poisons either. Or at least at such a low concentration that it is insignificant.

From Wikipedia ([https://en.wikipedia.org/wiki/Homeopathic_dilutions](https://en.wikipedia.org/wiki/Homeopathic_dilutions)):

Hahnemann advocated 30C dilutions for most purposes (that is, dilution by a factor of 1060).

...

Dilution advocated by Hahnemann for most purposes: on average, this would require giving two billion doses per second to six billion people for 4 billion years to deliver a single molecule of the original material to any patient.

<comment><USER6> I think the parent's point was that things _marketed_ as "homeopathic" usually have their _active_ ingredient diluted into non-existence, so they're pointless to take for relief of the claimed symptom. Slapping the label "homeopathic" on something that actually _does_ something is really annoying, then, because it'd make you assume it did nothing.

<comment><USER4> Or, things get the homeopathic label when they really aren't homeopathic at all -- they use it as a synonym for natural, or non-medicinal. Which is what I want sometimes. But I never want anything that is homeopathic, because to me that means fake.

</comment></comment></comment></comment></comment></comment><comment><USER7> Posted earlier, 158+ points, 33 comments:

[https://news.ycombinator.com/item?id=13616574](https://news.ycombinator.com/item?id=13616574)</comment>


typeface.js -- Rendering text with Javascript, canvas, and VML - <AUTHOR> http://typeface.neocracy.org/

<comment><USER2> Looks respectable, although it breaks clipboard use and also doesn't work on Opera. (not quite sure why)

</comment><comment><USER3> "Can't call method "read" on an undefined value at /www/sites/typeface/lib/TypefaceJS.pm line 36."

(this upon attempting to convert a font to test it)

</comment><comment><USER4> This looks great, but I'd be hard pressed to use it on anything other than occasional headers until someone figures out a hack for copy/paste support.

<comment><USER5> I really keep hoping that embedded fonts carry on. The many workarounds I've seen all have a lot of downsides. Embedding fonts could fix all that immediately.

And it's about time. I'm sick of seeing Georgia on every single web page I visit.

<comment><USER6> Unfortunately, its a copyright violation to embed fonts. I'm surprised Firefox decided to implement the ability to embed fonts, but I'll be far more surprised if Microsoft decides to. And, if they don't, embedding fonts will remain something you cannot rely on.

<comment><USER7> IE already supports font embedding and has for years. Their font embedding tech was developed with input from the font making industry and they plan to make the technology available for other browser vendors to freely use. Bill Hill, an MS researcher has been a strong advocate of improving typography on the web.

<comment><USER6> IE's current support is limited to the EOT format I believe ([http://ajaxian.com/archives/the-fight-for-fantastic-fonts- or...](http://ajaxian.com/archives/the-fight-for-fantastic-fonts-or-lets-give- tahoma-a-rest)), which is almost completely unused in the real world. There would need to be a lot more work done in this area to make it practical I think.</comment></comment></comment></comment></comment>


Phoenix 1.2 - <AUTHOR> https://github.com/phoenixframework/phoenix/blob/v1.2.0/CHANGELOG.md <comment><USER2> Wow this news front-paged fast! I just tagged the commit, grabbed a quick lunch, and I haven't finished drafting the official release announcement yet :)

If you want to know what Phoenix 1.2 is all about, check my ElixirConfEU keynote which demos Phoenix Presence: [https://www.youtube.com/watch?v=n338leKvqnA](https://www.youtube.com/watch?v=n338leKvqnA)

José Valim and I were also on the changelog podcast talking about what's new: [https://changelog.com/208/](https://changelog.com/208/)

<comment><USER3> That talk was super interesting. Thanks!

[https://haslab.wordpress.com/2011/07/08/version-vectors- are-...](https://haslab.wordpress.com/2011/07/08/version-vectors-are-not- vector-clocks/) \- are you sure what you described is a version clock, and not a version vector? I'll readily admit to not having a thorough enough grasp of both to feel confident in making that call.

<comment><USER2> I believe they are technically version vectors, which I've mentioned in other talks, but I haven't been sure if the distinction is helpful and I haven't been confident enough yet to explain the nuances. Will check out the link

<comment><USER3> Ok - the reason I ask is because when I needed to figure some of that stuff out, I had more or less the same reaction you did in your talk. I'm still not 100% of everything :-)

</comment></comment></comment></comment><comment><USER4> I'm 10-years Python/Django and making the leap to Phoenix. I can't begin to describe how thrilled I am with the intelligence and warmth of this community—something I always treasured at Django and was afraid of losing.

Phoenix 1.2 looks ready for prime-time! I just finished setting up a continuous integration/deployment pipeline with edeliver and CircleCI and couldn't be happier with how everything is fitting together.

Very excited for the bright future of Phoenix!

<comment><USER5> Thanks for sharing!

What are the biggest pros/cons you've encountered so far? What do you miss from Django? What do you wish Django could copy from Phoenix?

<comment><USER4> I'm still learning, but I've already noticed some incredible response times. Phoenix is very fast. I wish I had actual numbers to give you that compare to Django but I'm just not that far along yet.

Elixir as a language is very nice. This is my first taste of "functional" programming and I really like it. Shifting my mindset from object-oriented has been difficult, but I find the code I write much more straight-forward so far for the types of apps I will be building.

What do I miss? Maturity. Django dotted its Is and crossed its Ts a long time ago. I wouldn't say Phoenix is buggy at all—I think it is production ready—but the recent work Django has been doing with things like Postgres libraries are several steps ahead of where Phoenix is at today. Also, Django documentation is bar-none.

There is nothing about Phoenix that jumps out at me and says "Django would be so much better if it had this!" Maybe Presence will be one of those things, but I have not had a chance to play with it yet. I am excited about how Elixir (on top of Erlang OTP) scales horizontally with virtually no effort, which is something that Python cannot replicate as easily without a lot of thought and some sort of bolted on message db—and in the end, slower. The robust concurrency story (and what that could mean long-term) is exciting to me.

<comment><USER6> I found Ecto to be one of the best Postgres libraries out there.

What do you miss from Django's Postgres libraries in Phoenix/Ecto?

</comment></comment></comment></comment><comment><USER7> Long-time rails guy here, really looking forward to playing around with Phoenix soon. I keep hearing really positive things, just haven't had the time dive in. I tried diving into react for a bit, but I found that it was really hard to retain unless I was working with it every day. Lots of people are telling me that the switch from Rails to Phoenix will be a lot easier though (I hope so!)

<comment><USER8> You'll feel instantly at home in phoenix, since you know more or less what MVC, migrations, ... are about.

Then you'll dig deeper into und the actual syntax (a week for rubyists before it clicks?). Dont give up, its awesome.

Then you'll be productive right away. And then you'll discover what OTP actually means in practice and be blown away. (Point of no Return)

And then you fully "get" what channels, distributed systems, umbrella- refactorings and all the latest talks are about.

Phoenix as a whole is so advanced compared with everything else we have currently its not even funny. I highly recommend the official book, it itself is one of the best books in web dev i've ever read.

<comment><USER3> > And then you fully "get" what channels, distributed systems, umbrella- > refactorings and all the latest talks are about.

I used Erlang on my last project, and it was an awesome fit for what we were doing. It's a really great system.

That said... "don't drink too much kool-aid" \- Rails is fine for lots of things, and continues to be fine. Not everyone needs a fault-tolerant, distributed system, or web sockets or some of the other stuff where Erlang is a win.

<comment><USER9> I agree with the "don't drink too much kool-aid" but you can also completely develop and deploy Phoenix without worrying about distributed systems, umbrella-refactorings, and so on. At least, I have. :)

This has been the whole appeal behind Phoenix: it truly is as productive as Rails/Django but much more performant. Better throughput, faster boot times, faster tests, etc. Fault-tolerance, channels and presence are the cherry on top.

I have seen Chris keynote about the presence implementation and they seem to have used really advanced techniques that I cannot not quite follow. Still, this week we are going to deploy presence into production and all we really care is that it just works (and without Redis or any other dependency).

<comment><USER3> > it truly is as productive as Rails/Django but much more performant

I'd be a little bit skeptical of both things: Rails has a huge, mature ecosystem, and Erlang is often faster than Ruby, but is not a "fast" language.

[http://benchmarksgame.alioth.debian.org/u64q/compare.php?lan...](http://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=yarv&lang2=hipe)

Where Erlang really shines is that it's running in a single Unix process, yet can handle multiple things going on at once thanks to its scheduler (and not have ugly code, because it's the scheduler doing the work, not the programmer writing spaghetti). This means that generally, it's going to handle more connections better and degrade more gracefully. This is a huge win for web sockets where you have a connection sitting open and _not_ tying up a huge Rails process. Although even there, I guess the Rails guys are working on some new stuff that should help alleviate that problem.

Erlang is cool stuff, and well worth a look. I certainly enjoy working with it. But be wary of throwing out Rails because "new! shiny!"

<comment><USER10> Jason's coworker here.

The Erlang benefits you have mentioned above also fully apply to regular web requests and applications. One of Ruby biggest flaws (and consequently Rails') is poor support for concurrency. And Phoenix performs well on all aspects Jason mentioned because running tests, serving data, etc is all done concurrently.

I was also skeptical at first. I've heard talks and benchmarks reporting Phoenix is from 10x to 20x faster than Rails (like this one: [https://gist.github.com/omnibs/e5e72b31e6bd25caf39a](https://gist.github.com/omnibs/e5e72b31e6bd25caf39a)). After porting our authentication service to Phoenix (our first), we saw similar gains with less CPU and memory usage (as well as better response time averages and 99 percentiles). Our deployment infrastructure for this service is now one fourth of what it was originally.

Other than that, Rails definitely has a huge ecosystem and that should be taken into account by those planning to move to Phoenix. Honestly, it has not impacted us in any significant way but YMMV.

</comment></comment></comment></comment></comment></comment><comment><USER11> Fantastic week for the Erlang/Elixir ecosystem!

Another thanks to the teams behind all these amazing releases, and particularly (for this news) Phoenix.Presence and Phoenix.PubSub, for the wide range of new exciting applications they'll have.

<comment><USER12> Just bumped my servers to erlang 19, Elixir 1.3 and Phoenix 1.2. The erlang and Elixir communities have done a great job shipping.

At my organization, I created our first internal service which uses Elixir and Phoenix. I've had a number of both junior and senior devs come up to me really excited about building on Phoenix and Elixir. It's rock-solid stable, fast and fun to use. Props to José, Chris and the rest of the team for their continued hard work.

</comment></comment><comment><USER13> > [CodeReloader] The lib/ directory is now code reloaded by default along with > web/ in development

Nice! I found the way Phoenix deals with iteration to be quite refreshing, glad to see lib is now a part of this too.

</comment><comment><USER14> I posted this in the Elixir 1.3 thread but also relevant here. If you're looking to understand why you/your company may want to consider migrating from Rails to Phoenix you can check out my RailsConf talk on the subject [https://www.youtube.com/watch?v=OxhTQdcieQE](https://www.youtube.com/watch?v=OxhTQdcieQE)

</comment><comment><USER15> Does anyone know if Programming Phoenix will be receiving an update for 1.2? Are there any backwards incompatible changes in 1.2 or Ecto 2.0?

</comment><comment><USER16> Phoenix is "A productive web framework that does not compromise speed and maintainability."

<comment><USER17> The readme doesn't even feature Elixir.

</comment></comment><comment><USER8> are the new calendar-types from elixir 1.3 already rolled into phoenix/ecto?

<comment><USER2> They are supported if you are on Elixir 1.3 since we ship with Ecto 2.0 out of the box, but we are waiting to make them the default with the Phoenix generators until we make Elixir 1.3 a hard dependency.

 _Edit_ : I was wrong! My comments will apply to Ecto 2.1. Life is confusing living on the edge sometimes

<comment><USER18> Is there a good writeup on how to use them somewhere?

<comment><USER19> Since that question is related to Ecto, I will take it. :) Right now it is not possible to use the new Calendar types in Ecto. The plan is for Ecto v2.1 to come in about a month, when v1.3 is a bit more stable, and it will introduce the `:datetime`, `:time` and `:date` types that will map to Elixir's v1.3 new Calendar types.

TL;DR - not yet, soon.

<comment><USER8> Sounds good! I am actually waiting for this exact feature before starting my next project, since I have to deal with a lot of time range queries (all items from the Last 48h, ...)

I really appreciate the work of you all!

</comment></comment></comment></comment></comment><comment><USER20> What is Phoenix (in this context)? Since it took me a few clicks to figure out, I'll share:

> A productive web framework [using node] that does not compromise speed and > maintainability.

CORRECTION:

> A productive web framework [using elixir] that does not compromise speed and > maintainability.

Apologies for incorrect info, I saw the "npm install" content in the instructions and assumed it meant phoenix used node.

<comment><USER21> Node? I thought Elixir was the underlying language/platform?

<comment><USER2> The tagline is definitely _not_ about using node :) We include brunch.io, a nodejs asset builder, for our css/js building/bundling. Phoenix is first and foremost and Elixir web and distributed systems framework, which is where we get our productivity, speed, and maintainability.

</comment></comment></comment><comment><USER22> Phoenix looks great, but seems like it's focus is on serving html to websites (like Rails). Is there something simpler for building REST apis for mobile/single-page apps (like Sinatra)?

<comment><USER23> I have just started working on a serious Phoenix project after watching from the sidelines for a while. It takes a completely different approach to Rails. First of all there's no magic (everything is completely explicit). It's extremely adaptable to different needs (JSON apis, real-time communication, etc).

Also note: Phoenix is not so much a singular framework, it's more of a collection of libraries (Plug, Ecto, etc.)... it's just that Elixir makes cohesive communication between independent libraries so dead simple it "feels" like a Rails-like monolith.

But once you actually look at the structure of the project, there's really not a whole lot that's hidden "behind the scenes" like in Rails. You can just add remove any of these components as you like.

Honestly give it a try. This is the first backend environment I've tried that feels "right" (and man I've been searching); everyone else I know who's tried it agrees. It's just a pleasure to work with.

So far every surprise has been pleasant!</comment></comment>


Show HN: Deploy dathttpd on now.sh - <AUTHOR> https://dat.jimpick.com/ <comment><USER2> Fascinating! The first web 2.0 website is online! Congrats to Tim Berners-Lee 2.0!

<comment><USER3> If you're interested, there are a actually a few other sites being hosted with Dat!

You can view them over https or if you download Beaker [1], you can view the peer-to-peer version of the site with the dat:// protocol.

\- taravancil.com \- creationix.com \- pfrazee.hashbase.io

1\. [https://beakerbrowser.com/docs/install](https://beakerbrowser.com/docs/install)

</comment></comment><comment><USER4> Hi. This seems very interesting. I followed the rabit hole and ended up here [https://pfrazee.github.io/blog/what-is- the-p2p-web](https://pfrazee.github.io/blog/what-is-the-p2p-web). And also checked provided git. Still doesn't understand how you published straight from the Beaker Browser so your dat:// site is accesible via http! Can you elaborate bit more? Thanks

Edit: I've got it :-)

Edit2: You've installed awscli, but never used in Dockerfile. Why?

<comment><AUTHOR> Unnecessary... I did a cut-and-paste from another project. :-)

</comment></comment><comment><USER5> This is the kind of post that triggers my imposter syndrome. I've been a programmer for 10+ years and I have no clue what now.sh or dathttpd are.

:(

<comment><USER6> Unless you are involved with the Node.JS ecosystem [1][2] not knowing what those projects are is irrelevant, at least it is for me and I still know what they are. I will make an assumption and say that you work with a different technology and/or in a different field that doesn't involves web development, in this case there is no need to think that, because you do not know something outside the focus of your work, your career will be affected.

[1] [https://zeit.co/now](https://zeit.co/now)

[2] [https://github.com/beakerbrowser/dathttpd](https://github.com/beakerbrowser/dathttpd)

<comment><USER5> Nope, I work in web development, and currently in node.js =( I've heard of Zeit though. Heard of Micro, Hyper and have actually used PKG. So maybe its not so bad</comment></comment></comment>