Factoring may be easier than we think (2016) - <AUTHOR> http://math.mit.edu/~cohn/Thoughts/factoring.html <comment><USER2> Imagine what you would do if you discovered how to factor efficiently?

Think carefully. You now how the power to decrypt much of the world's banking and internet traffic and spoof certificates. There are forces in this world that would kill you to have this power. Would you publish your findings for everlasting fame? Would you sell it to the NSA for money (remember you can prove your power without releasing your algorithm)? Would you use it for personal gain or power? Who would you tell first? Who do you trust?

<comment><USER3> 1\. Setup few servers operating in different countries, pay for a few years and set up a timer which will publish this algorithm on few public websites, then destroy all credentials, so it could not be undone.

2\. Steal bitcoins from very old wallets with some small amounts. Supposedly those wallets are lost. Steal enough to have enough money to live a good life. Well, if for some reason I would have enough money, skip this step.

3\. Break google.com certificate and mail hashes to Google Security team. Ask them to disclose that factorization is broken, so the rest of the world can prepare. Repeat with some other big companies.

4\. Disclose algorithm when the world is ready.

<comment> Ar-Curunir Factoring doesn't allow you to break discrete log; you won't be able to steal from old wallets.

<comment><USER4> It's been a long time since I looked at this but IIRC factoring and discreet log are equivalent.

<comment><USER5> It's also a long time since I looked at this, but I recall pretty clearly, that factoring is "easier" than DLP. "Easier" as in factoring is reducible [1] to the DLP.

I.e. if you could do DLP in polynomial time, then also factoring becomes polynomial (thanks to Shor's Algorithm [2]).

The reverse, however, is not currently known to be true AFAICS: having an oracle that computes the DLP does not help you to speed up factoring (at least not in a way that makes it polynomial).

[1] [https://en.wikipedia.org/wiki/Reduction_(complexity)](https://en.wikipedia.org/wiki/Reduction_\(complexity\))

[2] [https://en.wikipedia.org/wiki/Shor%27s_algorithm](https://en.wikipedia.org/wiki/Shor%27s_algorithm)

(EDIT: typo)

<comment><USER6> They are equivalent according to this: [https://crypto.stackexchange.com/questions/9385/reduction- of...](https://crypto.stackexchange.com/questions/9385/reduction-of-integer- factorization-to-discrete-logarithm-problem)

<comment><USER5> I think you are mistaken, they are not equivalent in any way that would be meaningful to assessing cryptographic strength.

The stack-exchange questions that you link to refers to [1] "Discrete Logarithms and Factoring". Section 1 "Introduction" already states many facts that imply that DLP is hard, even if you can factor:

* fastest known method for DLP is O(exp(c sqrt(log n log log n)))

* 1. 3c) "if we can factor in polynomial time, then to quickly solve a^x ≡ b mod n, all we need are solutions modulo the prime divisors of n"

Note that "solutions modulo the prime divisors of n" are still instances of the DLP with super-polynomial complexity, and in cryptographic applications N is usually a prime number anyway (DHE, ElGamal crypto-system), so 1 3c) does not actually apply.

See also the paper's section 6 final remarks "Conversely, one can ask for a fast algorithm for prime-modulus problems, assuming all needed factorizations. Both of these questions remain unanswered".

[1] [https://www2.eecs.berkeley.edu/Pubs/TechRpts/1984/CSD-84-186...](https://www2.eecs.berkeley.edu/Pubs/TechRpts/1984/CSD-84-186.pdf)

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> _Let 's say a serious attempt consists of several months of work by an expert, someone who knows enough number theory to read the literature on this problem. Then the number of people who have seriously tried must be on the order of magnitude of 100._

In academia, maybe. But I would not be surprised if millenia of experts' time has been spent on this problem in intelligence agencies.

<comment><USER8> But if intelligence agencies broke factoring, would we know? Maybe they have.

<comment><USER9> If they have then they are using it exceedingly sparingly.

I strongly suspect it would be impossible to use it to any moderate degree without being found out in one way or another.

I know this is a very very poorly worded question :) but I wonder what the most amazing secret was that was held for the longest time?

<comment><USER10> > but I wonder what the most amazing secret was that was held for the longest > time?

Very relevant, probably the Allie's breaking the Axis enigma code in WWII.

[https://en.m.wikipedia.org/wiki/Ultra](https://en.m.wikipedia.org/wiki/Ultra)

If the NSA broke RSA, they'd have a similar program set up to ensure nobody notices.

</comment></comment></comment></comment><comment><USER11> The US federal government once expended 10 percent of the US's electric energy supply in the Manhattan project for getting weapons grade nuclear material. This gives a rough ballpark for the amount of energy they are willing to invest into major strategic advancements.

However, if you apply Landauer's principle, current factoring algorithms would require enough energy to boil all oceans on the earth, that's a lot even compared to the US's energy supply.

So algorithmic improvements are the real danger basically. Even if we discovered a decryption method now, and immediately everyone stopped using RSA, there would still be an immense impact because all the past encrypted traffic that someone might have stored somewhere suddenly becomes decryptable. And usually, traffic from 20 years ago is still relevant today.

<comment><USER12> > current factoring algorithms would require enough energy to boil all oceans > on the earth, that's a lot even compared to the US's energy supply.

Interesting. For what algorithm & key size?

I'd love to quote this. I've heard it before but I don't remember the source.

<comment><USER11> [https://eprint.iacr.org/2013/635.pdf](https://eprint.iacr.org/2013/635.pdf)

> Boiling all water on the planet (including all starfish) amounts to about > 2^24 lakes of Geneva and leads to global security: 114-bit symmetric > cryptosystems, 228-bit cryptographic hashes, 2380-bit RSA. This needs to be > done 16 thousand times to break AES-128, SHA-256, or 3064-bit RSA.

I think this paper isn't using Landauer's bounds though, but conventional computers. So maybe my claim was wrong, because we aren't 16 thousand times away from Landauer's bounds but millions [1].

[1]: [https://web.archive.org/web/20141219043239/http://www.bloomf...](https://web.archive.org/web/20141219043239/http://www.bloomfieldknoble.com/nanomagnet- memories-approach-low-power-limit/)

<comment><USER13> But why the starfish.

<comment><USER14> Because if you're already boiling the oceans, you probably won't have enough left over in the budget for marine conservation.

</comment></comment></comment></comment></comment><comment><USER15> I've heard the following called "Aaronson's trilemma": Either the extended Church-Turing thesis is false, or quantum computers are impossible, or...there exists a classical polynomial factoring algorithm that runs in polynomial time.

One of these things must be true, and debates around quantum computing usually focus on the first two. But as argued, we don't have great reasons to believe factoring in polynomial time is impossible. We certainly don't have a proof that no such algorithm exists.

<comment><USER16> What is the extended Church-Turing thesis? We already know quantum computers give speedups beyond classical lower bounds.

<comment><USER17> It's basically that BPP captures all realistic polynomial-time computations. The speedup would have to be sufficient to show something like a problem in BQP that isn't in BPP. I don't think anyone has been able to show that unconditionally yet.

You'd also need to accept that Quantum computers are realistic, which is why Aaronson's trilemma includes quantum computers being impossible.

<comment><USER18> This sort of statement is exactly why serious people shouldn't take "quantum complexity theory" seriously. The complexity class BQP is bullshit: there are no quantum computers, the end.

Feel free to prove me wrong by building one which does useful calculations. No time limit, until you die, in which case "time's up."

Edit add for downvoters: the strong Church Turing thesis is also almost certainly, and very obviously bullshit. How does that make you feel?

<comment><USER17> Which statement are you saying demonstrates why people shouldn't take quantum complexity theory seriously?

<comment><USER18> BQP doesn't exist: there are no quantum computers. Saying BQP is like saying "magic fairy dust perfect unitary transformers that effectively encode a perfect complex number in the same sense a protractor theoretically can solve NP-complete problems by encoding real numbers."

<comment><USER15> The qbits and unitaries don't have to be perfect - it's about how things scale when you add qbits. Quantum error correction shows that you can add more qbits to make up for imperfection, and still be left with a quantum computer, i. e. the imperfection hasn't demoted the thing to a classical computer.

Of course they haven't built one yet, but none of the difficulties encountered so far have involved discovering new physics, which is what you would need to do to rule out quantum computers since the laws of physics as currently understood permit them.

<comment><USER18> You've just regurgitated Aaronson's statement on the topic. Aaronson never studied physics, and has probably never fiddled with an op-amp or tried to make entangled anything in the world of matter. There's zero evidence quantum error correction will work; it's just an idea with no basis in the world of matter. There's zero evidence you can meaningfully and usefully manipulate quantum coherent states, let alone manipulate an exponentially huge number of them with a polynomially large number of computational elements, which is the essential claim of quantum computing. They make great claims: great claims need evidence; not theoretical wankery. Building a couple of scalable error corrected qubits would be a great start; it might even cause me to shut up about it.

Never in the history of the human race has something as complex as a computer architecture existed in the theoretical world before it exists in some form in the physical world, let alone one for which we define complexity classes.

The entire field is intensely silly, and the last time I said so in a public place, the waiter turned out to be some dude who just got his Ph.D. in the subject. He didn't agree with me exactly, but the fact the dude had a job bringing people steaks for a living is a decent argument I'm right.

<comment><USER15> My arguments are my own. I am an atomic physicist, and I meaningfully and usefully manipulate coherent quantum states every day. Not for making a quantum computer mind you, but quantum states nonetheless. Quantum mechanics works. The atoms do exactly what the Schrödinger equation says they should, however much entanglement is added. We are yet to see it break down. Plenty of my colleagues are working on quantum computers, with atoms, ions, photons, and solid state systems, and from where I stand it doesn't look like nonsense. It looks steady progress, and if there are insurmountable barriers, they have not yet been encountered.

I am not certain that quantum computers are possible, but I am certain that you are _wildly_ overconfident that they are not.

<comment><USER18> How much money are you willing to stake on that statement? I'll make a market for you; if you think your education gives you an edge over the wildly overconfident guy -we can even stick it on a blockchain that is quantum future proof if you like. Your choice.

Saying "quantum mechanics works" is not the same as saying "I can manipulate exponential QM states with polynomial imperfect physical devices." In the early days, people sketched out optical quantum computers that totally worked, but had exponential growth in elements with quantum states. Which, I bet, is how the universe is always going to work.

Money where your mouth is: I haven't found any other good shorts for this shitty idea.

<comment><USER15> I asked you elsewhere in the thread what odds you're willing to bet at, so absolutely I'm willing to put my money where my mouth is. I'll happily bet that there will be a demonstration of 'quantum supremacy' within 20 years - that is, a quantum computer computing something faster than a classical computer, whether it's Grover's algorithm or factoring or something else. Let's say by the 1st of July 2039.

I'm not super confident there will be quantum computers, whereas you seem very confident there will not be. What do you think the probability of quantum supremacy within 20 years is? If you think it's 5 % and I think it's 50 %, perhaps we can take the geometric mean and bet at 6:1 odds (~15% chance).

Will you give me those odds? Let's say I stake $200. Then I'd give you that if I lose, and you'd give me $1200 if I win. Or we can increase the amount a bit. Today's dollars, we can inflation adjust since it'll be 20 years.

The terms might sound favourable to me, but you seem very confident that there won't be quantum computers ever, so less than 15% chance in the next 20 years seems consistent with your belief.

I wouldn't know how to put the bet on a blockchain, but if you know about that and want to, I'm happy. Otherwise I am happy to just take your word.

We can also shorten the duration of the bet, but I would want to shift the odds a bit since although I think quantum computers have a decent chance of being possible, there is considerable uncertainty in how long it would take to get to the point of demonstrating quantum supremacy. Probably I would accept doubling the odds if the duration of the bet were halved and so on.

<comment><USER18> We'd need a hard definition of "quantum supremacy" -I believe there have been several press releases claiming this already, and I think you agree with me that there are no such machines at hand.

There's this ethereum thing called Augur we could use to place the bet, though that's an interesting bet in itself (ethereum and auger being around in 20 years is not a sure thing). I suppose also "long bets." If you google my name you can find my contact info.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER19> In a sense, this is terrifying. I mean, the math nerd in me is delighted at the idea, but in all practical senses if someone were to stumble upon and share a usably fast factoring algorithm tomorrow, the sky would fall.

Sure, lots of crypto exists that isn't prime factoring based and we could move to that in a hurry- but it would be a lot like if we'd realized the Y2K problem on December 31st, 1999. Everything would need to be updated right now, immediately, today.

And yet part of me is kind of excited it could happen.

<comment><USER20> It wouldn't even be as tame as you mentioned. Any encrypted information that has been caught and stored would also become available.

</comment></comment><comment><USER21> > Of course, I have no real evidence for my views...

> On the other hand, the people who talk about the great difficulty of > factoring have equally little evidence...

This is a classic antinomy (paradox): one can argue indefinitely in either direction, because the question lies along the bounds of human reason (or so says Immanuel Kant).

The two sentences above, in themselves, provide a bit of evidence of the impossibility of solving the problem, and at the same time provide evidence for the possibility of handling this problem as a significant phenomenon of pure mathematics.

:)

EDIT: I mean only that the insolubility of the problem may itself be of mathematical use: it may (insofar as it is unsolvable, and insofar as it appears to be soluble) amount to a kind of 'anchor' for mathematics, a marker that indicates the boundary of the mathematical sciences, and that such a boundary would be of tremendous import to mathematicians and philosophers. Why is _this_ problem, _this_ problem specifically, unsolvable? (Rather than some other problem that has been solved?)

tl;dr The question of "why have we have trying to solve this problem for millennia?" is perhaps more significant for mathematics than the solution to the problem.

</comment><comment><USER22> This is the Henry Cohn that recently received recognition for his work on optimal sphere packings in 8 and 24 dimensions: [http://www.ams.org/journals/notices/201804/rnoti-p463.pdf](http://www.ams.org/journals/notices/201804/rnoti-p463.pdf)

</comment><comment><USER23> I am not a scientist, but in my team 15 years ago, many people much more talented than me were in love with public cryptography. For them encryption with a 1024 key was perfect, impossible to break. They did not even considered that several RSA challenges had already been found.

Even if I had no education in mathematics I tried to show that in fact it was feasible to factor some enough large numbers with "bc" (using square root and a few other simple tricks) so the risk of having encryption broken by professionals was quite serious.

My boss asked to another guy for its advice, which was essentially that for a start he would not try to break any encryption scheme anyway. And that was the end of the story. The unstated lesson was probably that there were no reason to expect a career boost by working on such topics.

<comment><USER24> I'm not following your story.

At the time when 1024-bit numbers used in RSA were 'perfect', it was infeasible to factor the number in a reasonable amount of time. The most straightforward approach is just to iterate over integers from 2 to your target number (call it _n_ ), and see if anything divides evenly. Now, you start looking for shortcuts. First, you can test only half the numbers, because the second half will give identical results (e.g. n=20, n/2 = 10; later, n/10 = 2; no need to even test the second half of the range.) Next, it becomes obvious that we only care about odd numbers (if it's divisible by an even number, it's divisible by two); but really, when it comes down do it, we only care about prime factors (for one thing, all non-primes can be decomposed into prime factors; for another, we used prime numbers to get _n_.) And lastly, for the simple shortcuts, you really only have to get to int(sqrt( _n_ )) + 1 or so. So we've cut down the number of integers we have to divide with.

Did we find the two prime factors of our _n_ in a "reasonable" time? If so, just double the bit length to get a problem twice as hard. Every publicly- known shortcut to factoring large numbers just means you need to make your _n_ larger to increase the workload on an attacker.

The question then becomes: has anyone found a shortcut that will factor any number within a "reasonable" time? We don't know.

As to your career-related comments, I read cluelessness from your boss, and carelessness from the 'other guy' \- if OG "would not try to break any encryption scheme," then he's not the person whose advice you want about the strength of cryptosystems. Your boss just lacked critical thinking skills.

<comment><USER25> Even 15 years ago 1024-bit numbers were not secure.

<comment><USER26> I'm working on RSA4096 and this is secure minimum up to 2040. Factoring is secure for all times, that's a numeric principle. You only have to make the key/number greater. I'm working for 50 years with prime numbers and now I'm where I was when I was 17. Only a little nearer. What Fermat didn't find and Euler missed (!) is the knowledge that there is no super'algorhythm' for factoring.

</comment></comment></comment></comment><comment><USER27> Nobody yet has mentioned a film that was premised on the invention of a black box "capable of breaking the encryption of nearly every computer system".

IIRC the movie plot was somewhat convoluted and confusing and I don't have any desire to see it again. I'm bringing it up because there are a number of "what would you do if" posts here.

In the end, the "sneakers" use the box to cause: _the sudden bankruptcy of the Republican National Committee, and the simultaneous receipt of large anonymous donations by Amnesty International, Greenpeace, and the United Negro College Fund._

[https://en.wikipedia.org/wiki/Sneakers_(1992_film)](https://en.wikipedia.org/wiki/Sneakers_\(1992_film\))

</comment><comment><USER28> > The first thing to realize is that until the advent of public key > cryptography in the 1970's, few people cared about factoring. Some people > were interested in it for its intrinsic beauty, but nobody thought it was > good for anything, and it certainly wasn't the notorious unsolved problem it > is today. If anything, it was mildly obscure.

"There is no branch of mathematics, however abstract, which may not some day be applied to phenomena of the real world." \- Nikolai Ivanovich Lobachevsky

Applied mathematics is a problem looking for a solution and pure or abstract mathematics is a solution looking for a problem. An instance of this is the extension of the set of complex numbers called the quaternions discovered long ago which eventually found their application in affairs that require the representation of orientations in three dimensions, such as in computer graphics.

It seems here then that a motivated entrepreneur can establish a remunerative business should he or she find a solution to this prime factoring problem.

</comment><comment><USER29> I think it's interesting how many people worry about factorization. A second preimage attack on SHA2 would be at least as dangerous, and nowhere near as many people know or care about its assumptions.

<comment><USER30> Breaking hashes aren't a decision problem, so they're not directly comparable, but sha2 isn't on the same shaky mathematical ground that RSA is.

<comment><USER31> SHA-2 was on related shaky ground. Remember that in relatively short significant advances were made in breaking MD-5 and SHA-1. SHA-2 is based on similar constructs as MD-5/SHA-1.

For this reason the SHA-3 competition was started to find a new hash function based on different principles.

In the end it was found that creating practical attacks for SHA-2 is too hard. But we don't know what the future will bring.

The difference between RSA and SHA-2 is that RSA is a very nice mathematical structure and we are still learning a lot about (prime) numbers. In contrast, SHA-2 is weird structure that has to solve a hard problem. It is hard to attack.

</comment></comment></comment><comment><USER32> EDIT: this comment was mistaken, bitcoin uses elliptic curve based key pairs, as hackcasual points out below.

One thing to keep in mind:

Bitcoin wallets are implemented with public/private key pairs. If you believed that you had a method to crack that, well you probably couldn't just take all the bitcoin (people would notice and the market value would evaporate), but you could probably figure out a way to make at least 1% (a couple billion). So if it can be broken with a group of smart people thinking hard, that sounds like a startup opportunity.

<comment><USER33> If Bitcoin was broken it would instantly become worthless.

<comment><USER34> Not if you did it in a smart way. Let's say you could arbitrarily make transactions. Probably the best way to do it would be to steal all the coins from a particular exchange, like Coinbase. Then everyone would think Coinbase pwnd but bitcoin is fine. Rinse and repeat with other exchanges once a year and you can make a hefty profit.

</comment></comment></comment><comment><USER35> I did a toy project on wheel factorization if anyone is interested. Through it I learned some interesting math and ancient algorithms. It is by no means the cutting edge of factorization, but it was a fun little project.

[https://github.com/mcastorina/wheel- factorization/blob/maste...](https://github.com/mcastorina/wheel- factorization/blob/master/README.md#performance)

</comment><comment><USER36> Discussed at the time: [https://news.ycombinator.com/item?id=12355431](https://news.ycombinator.com/item?id=12355431)

</comment><comment><USER37> With a grain of salt

> Of course, I have no real evidence for my views: [...]

<comment><USER38> Might want to include context there:

>Of course, I have no real evidence for my views; ... On the other hand, the people who talk about the great difficulty of factoring have equally little evidence.

<comment><USER37> When we talk about the 'difficulty of factoring' there's an implication. The direct meaning is that there may well be a simple algorithm for factoring that has yet to be discovered. No one disputes this.

The implied idea is that this discovery could happen at any time and all things depending on it are at risk. This is also true but it's unreasonable to think that it is likely given that much effort has been put into this.

Yes we don't know, but two unknowns are not 50:50. Of course regardless of how you estimate its truth consider the cost of being wrong when using anything depending on it.

</comment></comment></comment><comment><USER39> Purely tangential, speculative questions: If you did prove that P = NP would you tell anyone? If so, how? Why?

<comment><USER40> Yes, I'd check the proof with some trusted colleagues, because odds are I'm wrong and I'd want to know why.

P=NP is a very hard problem and there have been a lot of failed solutions (including some that are flawed for very subtle reasons that can be easy to overlook). Even many famous, well known people have fallen into the trap of thinking they have a viable solution.

</comment></comment><comment><USER41> Fast factorization is going to be one of the killer apps of photonic computers.

<comment><USER42> You mean quantum computing (for which photonics are not a leading candidate), or regular photonic computing? Because the latter doesn't scale any better than normal computers.

</comment></comment><comment><USER2> I have always wondered if the NSA has figured out how to factor effeciently.

<comment><USER31> One thing to remember here is that RSA can be used in 2 ways: to encrypt and to sign.

If RSA is used to encrypt (for example if you send an encrypted message using PGP) then factoring directly breaks the encryption.

In practice, a lot of encryption on the Internet uses RSA to sign the hash of a key obtained using Diffie-Hellman. In this case breaking RSA would allow the NSA to impersonate but not directly break existing communications. The problem with impersonation is that it is very noticeable.

What I find odd about the linked article is that it only talks about factoring. In practice, the discrete log. problem is just as important and is very much related to factoring.

</comment></comment><comment><USER43> The reason factoring is hard is because finding very large probably prime numbers is easier making number sieve based methods useless for the purpose of breaking large numbers used in cryptography.

</comment><comment><USER39> You can build a machine with a laser and a big mirror.</comment>


Nutrition start-up plans to launch free nutrition API for app developers - <AUTHOR> http://www.foodandtechconnect.com/site/2012/08/16/nutrition-data-api/

<comment><USER2> I use the fat secret API for one of my side projects, <http://www.glycemicloadcalculator.com>, so I'm very much looking forward to incorporating this new API.</comment>


Simple Ain't Easy, but Hard Ain't Simple: Leaving Clojure for Ruby - <AUTHOR> https://blog.appcanary.com/2017/hard-isnt-simple-ruby-clojure.html <comment><USER3> One thing he said I absolutely agree with - when you're in a crunch situation, either in a startup or a new high-pressure project at BigCo, you don't pick a tool set with which you're not familiar. Even if it looks very nice. You're going to have enough frustration making deadlines without the additional hassle of discovering the warts (and they're always there) of a new language.

Where I work there are always projects that are a little bit under management radar - things that need to be done but don't need to be done in a big hurry. That's the right kind of project to scratch your chin and say "You know, I've never written anything in whitespace."

<comment><USER4> Someone once gave me the piece of advice that a project should only have one 'difficult' part. If it's a difficult domain problem or a difficult deadline, use an easy language (as in one your already very familiar with). If it's an easy domain problem and an easy deadline, then feel free to use a difficult language or framework (meaning one that you'll have to learn while you're doing the project.)

</comment></comment><comment><USER5> His experience matches most of mine:

I've been learning & trying to use clojure(script) for ~2 years now. A few things I miss in every other languages:

\- The REPL: I usually don't need a debugger,

\- Figwheel: that makes frontend dev a pleasure,

\- reagent/reframe: are incredibly straightforward, there's no special syntax, no special constructs, just a few functions, a few concepts (dispatch / subscribed) and you're done,

\- core.async: based on a simple channels / pipeline paradigm

But then you get into the practicalities of testing & writing code. Shady special cases[1][2], annoying testing tools[3], useless stack traces. And other issues everybody knows. Simple things that become a 30 min discussion with 3 different libraries and 2 blog articles to go through[4].

To me, there's this hardcore group of clojur'ist that seems to be hyper productive and keep introducing new concepts. And the "rest" (me at least), that are floundering. You fight with useless stacktraces? Boom, transducers. You deal with buggy test runners? Bam, a new specification library with test generations.

I've started moving my code to ES6 and Go. The community is filled with helpful people, but it seems to me that if you're not writing Clojure professionally already, the language is becoming less and less relevant.

[1] [https://github.com/replikativ/superv.async/issues/2#issuecom...](https://github.com/replikativ/superv.async/issues/2#issuecomment-292112698)

[2] [https://clojurians- log.clojureverse.org/clojurescript/2017-0...](https://clojurians- log.clojureverse.org/clojurescript/2017-05-06.html#inst-2017-05-06T11:16:17.076103Z)

[3] [https://github.com/bensu/doo/issues/134](https://github.com/bensu/doo/issues/134)

[4] [https://clojurians- log.clojureverse.org/clojurescript/2017-0...](https://clojurians- log.clojureverse.org/clojurescript/2017-05-06.html#inst-2017-05-06T08:48:31.683316Z)

<comment><USER6> The examples you raised are all - I believe not by coincidence - related to how Clojure interact with the host platform (both Java and Javascript). It simply doesn't shield you from it. As a consequence tracebacks bubble up, and you're expected to know how to debug it, and hopefully use the debugger of the host platform. This is aggravated on JS since JS exceptions mean nothing most of the time.

clojure.spec (launching on 1.9) will allow better error reporting on some cases, but the philosophy of the language has always being about giving you a lot of power vs. shielding you from complexity.

Also, a lot of people being introduced to Clojure from a Ruby background (and others, ofc) are having to learn simultaneously about: functional programming, Lisp, immutable data structures, the internals of JS/JVM, advanced concepts like STM, CSP, and a whole host of new libraries. That is a steep learning hill, specially if one bought the argument that programmers are snowflakes who can't take frustration. Once you're on the other side though, the complex, messy real-world projects become more amenable, because Clojure was created thinking about those. That's what Rich meant by "Simple ain't easy".

IMHO, working at a company w/ a codebase of ~150 projects written in Clojure, the productivity has paid off w/ interest though.

<comment><USER7> _> specially if one bought the argument that programmers are snowflakes who can't take frustration._

I think that characterising a (clearly knowledgeable, well-referenced) account of frustrations - especially ones that do not occur in other languages - <USER2> the dismissive term "snowflake" is exactly where the stereotype of the "smug Lisp weenie" comes from.

I have used and loved Clojure for years now (I wrote the patch that made Clojure vectors implement java.util.List, back in the pre-1.0 days). The fact that these problems exist doesn't itself give me existential worries about the future of the language. The fact that almost every Twitter thread in the OP, and the parent comment, contained this sort of dismissive reaction makes me very worried indeed.

If we replicate the community of Common Lisp, we will end up replicating its level of industry adoption.

<comment><USER8> > If we replicate the community of Common Lisp

Peter Norvig, Paul Graham, Dan Weinreb, Edi Weitz, Jans Aasman, Patrick Winston, Kent Pitman, the people from Clozure/Franz/LispWorks/..., and many many others.

That would be hard to replicate.

<comment><USER7> And yet, even these luminaries were insufficient to propel significant industrial adoption in the face of a famously hostile/arrogant/condescending community. The same fate befalling Clojure would be terrible.

<comment><USER8> I find such a comment very hostile, arrogant and condescending.

</comment></comment></comment></comment></comment></comment><comment><USER9> Dreaming up new analogous phrasings to easy != simple now.

Fun > easy > simple?

</comment><comment><USER10> Revenge of the averages

</comment><comment><USER11> IMO The Clojure Community does care about developer happiness. I think "happiness" is very subject, Clojure makes me happy, i'm a dev, what more can i say on that? As Phillmv made his work about his feelings, i feel its appropriate todo the same @phillmv blog and talk made me angry and sad at the same time. I took a minute to try and understand why i felt this way and concluded it was because while his technical concerns were valid, his tone belittled my own experiences. He says he wasn't as productive in a new paradigm, with a new language, on a tight deadline and because of this the Clojure Community doesn't care about Developer Happiness? He says he isn't talking about he community, but the langauge itself, but languaages dont have feelins so i'm confused on how his complaint isn't being leveled at the people behind and using the language. From my perspective, Clojure enables people to do things that while possible in other languages, would require a lot more effort:

1\. Projects like Onyx and Jepsen where a small team was able to accomplish something beautiful. 2\. Property based testing and the awesomeness of ReFrame state charts?

What @phillmv tried todo was very ambitious, and its awesome that he gave it a shot, but his overarching conclusion seems to put the blame on Clojure and that seems both unfair and unproductive.

Here are his specific clojure concerns and how i feel the community is addressing them in a way that makes me feel very happy:

1\. StackTraces are hard to understand: Clojure Spec has been written about a ton in this regard 2\. Debugger: Sayid Pro looks really promising, maybe a big company could pick it up and support it? But from my limited perspective things could be better for this story. 3\. Startup Time: I don't know if anything is being done to address this, the current wisdom is that you work within the repl and structure your projects so they can be "reloaded" which has lot of nice properties.

Here are His Concerns i think are language agnostic:

1\. It hard to Structure large apps : While Clojure applied has some suggestions i think every project is context specific. 2\. New learning curve (lots of new ideas like reducers, map, transdecures, macros) : Their are a wealth of resources on all these topics, what in particular was he not getting answers about? 3\. JVM requires deep editor integration: I feel this is true of every JVM language. 4\. Dependency hell: True in every language? Clojure spec is actual designed to help with this eventually!

So his pain points are real, but thats not where he puts the emphasis. Rather he puts that on his feelings. And as John Oliver has warned us, that makes it easy to trivialize real problems, which i feel is what has been done here.

<comment><USER12> > StackTraces are hard to understand: Clojure Spec has been written about a > ton in this regard

developers cannot be expected to spec out every function prototype. that's just not a good use of time.

<comment><USER11> total agree, I hope Clojure Spec will eventually be applied to most Core Clojure functions, and that information will trickle up to user defined functions. But thats only a hope, and possible based on my limited knowledge of the tool.

However, aren't most languages with the best errors messages Static and Strongly Typed?

My point was more that I think Spec was created to help address the issue. Not necessarily that it was even a good solution.

<comment><USER12> calling spec upon every core function invocation will surely be a performance hit, and a big one for tight loops.

</comment></comment></comment></comment><comment><USER13> My fav is Hard is easy, easy is hard.

<comment><USER5> You mean complex is easy, simple is hard?

<comment><USER13> indeed</comment></comment></comment>


Putting the Tesla HEPA Filter and Bioweapon Defense Mode to the Test - <AUTHOR> https://www.teslamotors.com/blog/putting-tesla-hepa-filter-and-bioweapon-defense-mode-to-the-test?utm_campaign=Blog_050216&utm_source=Twitter&utm_medium=social <comment><USER2> "In other words, Bioweapon Defense Mode is not a marketing statement, it is real. You can literally survive a military grade bio attack by sitting in your car."

That is so fucking metal!</comment>


Operation Soft Cell: A Worldwide Campaign Against Telecommunications Providers - <AUTHOR> https://www.cybereason.com/blog/operation-soft-cell-a-worldwide-campaign-against-telecommunications-providers <comment><AUTHOR> good article[1] & reporter's thread[2]

1: [https://apnews.com/e72c0141f1c54ad68c9f5f78d6a6c7e6](https://apnews.com/e72c0141f1c54ad68c9f5f78d6a6c7e6)

2: [https://twitter.com/razhael/status/1143590451343220736](https://twitter.com/razhael/status/1143590451343220736)</comment>


Line drawing on a grid - <AUTHOR> http://www.redblobgames.com/grids/line-drawing.html <comment><USER2> I remember enjoying Abrash's write up in the mid 90s. [http://www.phatcode.net/res/224/files/html/ch35/35-01.html](http://www.phatcode.net/res/224/files/html/ch35/35-01.html) He starts with the algorithm, then a C implementation, then optimization in ASM, only to throw it all away in the subsequent chapter because of a better algorithm. He goes through this cycle several times. Splendid educational effort on multiple levels.

</comment><comment><USER3> Not sure why they author immediately dismisses bresenham. It's fast, quality, and well known. I feel like if the post continued for another few logical steps of optimization that's exactly where he'd end up!

<comment><USER4> Bresenham lines are not "quality." For one, they're obviously jaggy. As well, most implementations of the algorithm will give you different results if you invert the start and end points. This matters a great deal for the common use- case in games of casting rays to determine sight lines. It is quite annoying to have a tactical combat game where a foe can shoot from behind a wall and you can't shoot back from the exact same position. Amit notes lack of symmetry in the "2.1 Orthogonal steps" implementation, which leads him to the "Supercover" implementation.

<comment><USER5> Pray tell which tactical combat game determines the line of sight using a _discrete grid_.

<comment><USER6> An awful lot of top down 2D tile based games.

</comment></comment></comment></comment><comment><USER7> What a stroke of luck for me! Just today I was studying line drawing techniques on a grid for a prototype related to offline mapping using polygons. It's yet another reminder how topics and links that don't seem relevant to my interests at first (games in this case) often turn out to be relevant to other areas of my interest.

</comment><comment><USER8> don't forget to read his explanation of A*, it's really great, and pretty easy. explained in python so it was even better.

<comment><USER9> You are right. Thanks for bringing this to my attention. [http://www.redblobgames.com/pathfinding/a-star/introduction....](http://www.redblobgames.com/pathfinding/a-star/introduction.html)

Every now and then I visit Amit's pages and find something interesting that he explains in an awesome way.

</comment></comment><comment><USER10> great to see the examples. I am exploring a small JavaScript module approach to SVG drawing to: [https://github.com/mulderp/pinboard- grid](https://github.com/mulderp/pinboard-grid)

<comment><USER11> I used to draw the lines that way, but I've switched to drawing the squares instead. It tends to look cleaner, the code is simpler, and it also lets me color the squares individually. If I really need to draw the lines, I can set a background color. It's a cheap trick :-)

<comment><USER10> indeed good advise! I changed the implementation [https://github.com/mulderp/pinboard- grid/commit/2a9d2c256ce0...](https://github.com/mulderp/pinboard- grid/commit/2a9d2c256ce0ba18cf90d7eae28acb8e43c7bb48) \- just saw you use transform instead of adjusting x/y

</comment></comment></comment><comment><USER12> Too bad there was nothing about using color for anti aliasing.

</comment><comment><USER13> Great demo. I really appreciate the interactivity.

Is there something similar for antialiasing?

</comment><comment><USER14> Anyone seen a similar run-down for 3D grids?

<comment><USER11> One nice thing about linear interpolation is that the same algorithm works for 2D grids, 3D grids, and hex grids. For 3D, the diagonal_distance, point_lerp, and point_round functions can look at at z in addition to x and y. For hex grids, you need hex grid distances and hex grid rounding ([http://www.redblobgames.com/grids/hexagons/#line- drawing](http://www.redblobgames.com/grids/hexagons/#line-drawing)).</comment></comment>


Raspberry Pi PIXEL Desktop for PC and Mac - <AUTHOR> https://www.raspberrypi.org/blog/pixel-pc-mac/ <comment><USER2> This is pretty cool. I have some elderly Lenovo T60 laptops that this will give some new life to. I love Pixel on my Raspberry Pi. It's a really clean interface, easy to use and pretty snappy on the Pi. Hoping it's equally snappy on the x86 hardware.

</comment><comment><USER2> Came back to say that it runs well on the T60, and in Virtual box on the same T60 (win7) and a T61. Pretty spiffy, very nice build.</comment>


Cows graze aligned north/south - found by studying Google Earth - <AUTHOR> http://www.livescience.com/animals/080825-magnetism-cows.html

<comment><AUTHOR> Hm, on second thought I wonder if this will turn out to be a joke. Even if cows could feel the magnetic field, what would be the benefit of grazing in a particular direction? Maybe more efficient resource allocation?</comment>


Gab.com is under attack - <AUTHOR> https://gab.ai/ <comment><USER2> This is a pretty good example of "you have the right to free speech, not consequence free speech" IMO.

<comment><USER3> Not..really.

Gab has denounced violence from the beginning. Twitter STILL has many more instances of ISIS accounts, Muslim brotherhood members, and people like Louis Farrakhan..who regularly call people of the Jewish faith 'termites'.

Why no de-platforming??? Where is the outrage????

It really makes no sense to me why one crazy person person having an account on Gab means that they have to be de-platformed by so many providers.

The only logical conclusion is political bias. Gab is known for allowing conservative/right-leaning view points and this is one more way for people on the Left to ban them.

I also find it a little ironic that so many people on HN are pushing corporate rights for not doing business with anyone for almost any reason. Yet, if it was a Christian baker not baking a gay wedding-themed cake, it should somehow be illegal.

Trump Derangement Syndrome is alive and well.

<comment><USER4> Have you read Gab's twitter feed? The whole "We're just for free speech" spiel would be a whole lot more believable if they weren't also constantly spewing reprehensible rhetoric.

[https://twitter.com/Popehat/status/1056266704148262912](https://twitter.com/Popehat/status/1056266704148262912) [https://twitter.com/ClenchedFisk/status/1027690034718224384?...](https://twitter.com/ClenchedFisk/status/1027690034718224384?s=20)

(Note, they started deleting a lot of their own old posts)

<comment><USER5> Like it or not its free speech, I don't see why this shouldn't be allowed... You can talk shit about "white people" on twitter all day, but you can't say goy (unless you are a jew of course) and you can't say jews are fragile?

We need to go out in the sun more, we have all become so thin skinned.

<comment><USER6> > Like it or not its free speech, I don't see why this shouldn't be allowed

Like it or not, a platform choosing not to relay it is free speech as much as the originator choosing to express it.

<comment><USER5> How is no-platforming even speech? I guess when china shuts down access to websites, they are just expressing their free speech as well?

When a handful of companies can hold power over what is allowed to be online and what is not, without a court ruling, its about time to regulate the hell out of the sector.

</comment></comment></comment></comment></comment></comment><comment><USER7> The trouble with echo-chamber platforms is they preach to the choir by reinforcing biases, beliefs and wedge issues, further dividing and conquering society into various polarized, separate subcultures. There ought to be a microblog platform that supports written debate of every belief, such that arguments can be settled online rather than spill over into the real world. The powers that be love it when the "little people" argue over abortion, climate change, separation of church and state, criminal justice, the War on Drugs, et cetera ad infinitum. Without civilized debate, a civilized society is impossible.</comment>


Bitcoin biggest bubble in history, says economist who predicted 2008 crash - <AUTHOR> https://www.theguardian.com/technology/2018/feb/02/bitcoin-biggest-bubble-in-history-says-economist-who-predicted-2008-crash <comment><USER2> >Roubini said the sharp fall was the beginning of a crash that would see the value of the digital currency plummet "all the way down to zero".

It's really hard to take someone seriously when they say the price is going to literally go to zero. It's possible to imagine crashes of 90% or even much worse for an asset like Bitcoin, but a market price of zero would suggest not a single person in the world is willing to pay anything at all for a Bitcoin. Absent a world-changing event like hashing being completely broken, the price will never be exactly zero. I can guarantee this since I myself would be willing to pay at least 1 cent for every BTC that exists, and I know that many would be eager to outbid me with that offer.

If someone is so good at calling the exact extent of this crash, then they should place a short on Bitcoin. You don't have an excuse anymore. It's on regulated markets, unrelated markets, and everything in between. If you really think it's going to zero then put all of your money in a short and make sure not to close it until it gets there. Options are also available for some investors, which should be even more lucrative for you.

<comment><USER3> How much money would Second Life currency be worth if Second Life went bankrupt and offline?

How much money would World of Warcraft gold be worth when that game eventually shuts down?

How much money will any amount of Bitcoin be worth when the Bitcoin network frays apart and ceases to function?

<comment><USER2> The difference between your first two examples and the final one is that Bitcoin is decentralized. It only takes a single miner to keep the Bitcoin network functioning (in practice you'd need to wait for difficulty adjustment and it wouldn't work out very well, but the point is you don't need to rely on any single entity not going bankrupt).

It's true that is the price goes low enough mining would decrease significantly, but after difficulty adjustment it would still exist to a nonzero extent. Bitcoin can absolutely still function as a system even if 95% or even 99% of all miners end up quitting it over the next few years.

<comment><USER3> BitTorrent is truly decentralized. So long as there's one seed of something you can repopulate the network, you can rebuild from that singular copy. Even N partial copies could do it so long as the sum of all those parts is at least one complete copy. This is because all you care about is data, and data can be copied easily.

Bitcoin isn't. It's merely _distributed_ and it depends on having a singular, healthy, functioning consensus network. Where that network splits you suddenly have more than one blockchain, as is the case with Bitcoin Gold, and Bitcoin Cash.

It also depends on having a healthy balance of mining power in that network so that no one entity can take control and start to dictate terms.

If all that's left is a single miner then four other miners can easily take charge of that network and effectively destroy it.

> Bitcoin can absolutely still function....

The blockchain component of Bitcoin can technically function, sure, there's no problem there, but as a unit of value, as a unit of exchange, it's functionally useless. You can't trust it, you can't use it for anything.

There is, by design, _one_ blockchain. When that assumption fails, the network has to work to achieve consensus, and if that can't be achieved it splits. That can be fine with things like IRC networks or peer-to-peer exchanges, you can always do a messy reconciliation later, but with something that's supposed to represent a medium of exchange that's a problem.

</comment></comment></comment></comment><comment><USER4> > "Policymakers and regulators are getting worried. Pretty much every G20 > policymaker is talking about a crackdown," Roubini told Bloomberg > Television. "We can't allow it to become the next Swiss bank account for use > by criminals and people evading tax."

I've heard the "prevent the next Swiss bank account" thing a few times now, and it still looks like the most baldly corrupt aspect of this whole show. _Actual_ Swiss bank accounts are still a thing people have, and are still being used by the wealthy to avoid taxes on a routine basis, and everyone knows it. But _actual_ Swiss bank accounts have a stable legal status and no one's really going after them.

What's the difference between "the next Swiss bank accounts" and the current ones? Perhaps it's that Bitcoin is available to much less wealthy classes of people. Or perhaps the right people have big Swiss accounts and haven't gotten around to diversifying into crypto yet. But I haven't seen a _good_ account of the difference.

<comment><USER5> Swiss banks can no longer rely on banking secret like they used to anymore.

</comment></comment><comment><USER6> It makes for a convenient headline, however I fail to see how Bitcoin takes that crown versus Cisco in March of 2000. It was overvalued by a solid $450 billion, and that's coming up on two decades ago (for inflation adjustment purposes).

Alternatively, there's PetroChina's trillion dollar valuation during the peak of the commodity bubble:

[http://www.nytimes.com/2007/11/05/business/worldbusiness/05i...](http://www.nytimes.com/2007/11/05/business/worldbusiness/05iht- petrochina.1.8190012.html)

And if we're not talking about more individualized entities, the mother of all bubbles goes to the global real-estate market in 2004-2006. Or, possibly, the bond market today. The global real-estate market was overvalued by tens of trillions of dollars at its peak before the great recession. That rather comical gap between Bitcoin's value at the top and the overvaluation in just the US real-estate market in the Summer of 2005, speaks to the dramatic tone of the article existing solely for clickbait.

If Bitcoin goes to zero, it will not matter very much on a global economic basis. It impacts exceptionally few people, fortunately. When the real-estate bubble collapsed, it nearly took down the global economy. That nearly bankrupted half the largest banks on the planet. It has taken until just the last few years for much of the developed world to truly recover or begin properly recovering.

</comment><comment><USER7> People seem to conflate Bitcoin with cryptocurrency as a whole. It has been clear for a long time that Bitcoin's pace is unsustainable, especially with the exorbitant transaction fees and mining costs.

But if you look at other cryptocurrencies like Etherium for example, you'll see an interesting differentiation. Not all cryptocurrency values are tied to Bitcoin. Etherium, at the time that I write this, is up 2.4% over a month ago while Bitcoin is down 37% over the same period.

So, while I do think Bitcoin especially is pretty much definitionally a bubble, crashes and volatility are the nature of the game right now. We should be careful to lump digital currency as a whole into the the same category as Bitcoin. Bitcoin was the first, but probably wont be the last and certainly isn't the best.

I think there is value in cryptocurrency that is difficult to see for many people past the black market stuff and the gambling aspect people put to it. If you've ever transferred Etherium to a friend to pay them back for the pizza they bought you, you'll feel that value in a small way.

I'm certain there is a reason these currencies exist beyond the sketchy reasons and I'm quite sure they are here to stay, bubble or not. There was a massive housing bubble, but we still have houses and they are, mostly, recovered above the peak of that bubble ten years later.

It's not hard to imagine a (near) future where cryptocurrencies are a primary means of monetary transactions.

<comment><USER8> > If you've ever transferred Etherium to a friend to pay them back for the > pizza they bought you, you'll feel that value in a small way.

I think it's hard for people to appreciate this because this is already a pretty trivial task using tools like Venmo (or whatever your choice might be).

> There was a massive housing bubble, but we still have houses and they are, > mostly, recovered above the peak of that bubble ten years later.

There's not really an alternative to houses (assuming you include e.g. apartments under that umbrella). You have to live somewhere, you don't have to use cryptocurrencies.

I think a more apt analogy would be the dot-com bubble. If there truly is a use for cryptocurrencies (as there turned out to be for internet-based companies), they'll stick around and we'll see them again.

That said, I also think it's a bit dangerous to rely on historical bubbles for predicting the future here. There are plenty of "bubbles" that never recovered at all, and we just moved on to the next thing entirely. We tend to forget about them.

<comment><USER9> _I think it 's hard for people to appreciate this because this is already a pretty trivial task using tools like Venmo (or whatever your choice might be)._

Venmo is great - for people in the US who only transact with others in the US. I had the same problem in Belgium - my local friends used Payconiq, which of course only works if you have a Belgian bank account.

<comment><USER8> Sure, and I think this problem of transferring small amounts of money internationally painlessly is probably the strongest case for cryptocurrencies. But:

1) I don't think that's what cryptocurrencies want to be relegated to (a forex market essentially)

2) This also seems like a way around currency controls and things like AML compliance. If that became a problem at some point, I have to imagine governments will start putting the screws on any major exchange that interacts with fiat (and to some extent they already have).

</comment></comment></comment></comment><comment><USER10> Roubini was also advising to stay out of equities following the post crash recovery in 2009...

</comment><comment><USER11> "Ah, surely this professional, educated economist has not studied bitcoin and doesn't understand crypto and the value of unregulated currency as well as me, your local pothead with no economics or tech education! And in any case, surely Etherium doesn't suffer from the same problems despite being fundamentally similar!"

<comment><USER12> Pattern matching logic fails for both educated people and potheads alike. Bitcoin is unlike anything seen before and no one knows what's going to happen with it.

</comment></comment><comment><USER13> Take a breath.

The biggest bubble in history? Compared to the mortgage and banking meltdown in 2008?

History must be short.

</comment><comment><USER14> Recent drops in cryptocurrency valuation coincide with futures settlements dates on CBOE and the other exchange. Perhaps someone is gaming the market and exploiting arb opportunity?

</comment><comment><USER15> Antecdotal, but I knew something was wrong when a coworker of mine who thinks he can pick stocks bought into bitcoin at about 16k</comment>


Daemon Showdown: Upstart vs. Runit vs. Systemd vs. Circus vs. God - <AUTHOR> http://tech.cueup.com/blog/2013/03/08/running-daemons/ <comment><AUTHOR> Hi, author of the post here. The thing that really surprised me, when writing this, was how difficult it was to find the edge of the topic: where does process supervision end, and server monitoring begin? If most of these programs include something for handling logs, are they then also logging software? Where's the boundary between watching processes and gathering metrics on them? Good software is canonically supposed to do one thing well, but it seems like nobody can agree what handles what here. I even started writing sections on logging and server monitoring, before sanity prevailed and I chopped them out.

It's maddening that we have no clear separation of responsibilities for this stuff. And so we end up with things like Upstart's half-hearted logging, because it's not clear where Upstart's job stops.

<comment><USER2> The tangled world of service monitoring, process monitoring, process launching, configuration management etc etc et bloody c is waiting for a ZFS- style collapsing of multiple fighting tools into a single layer.

I noticed this most of all when I was writing puppet manifests that ... create Upstart scripts.

Why do I have one tool that configures and supervises the bits at rest and a completely different tool that configures and supervises the same bits in flight?

<comment><USER3> This!

Curious for people's thoughts on whether Solaris's SMF (or its general model) fits this bill.

</comment></comment></comment><comment><USER4> I still prefer Bernsteins daemon-tools. I first used them when I had to deploy qmail and now I use them to manage Python webapplications. The default logging option is a bit weird, but the simplicity is a real winner for me.

I don't like Upstart, I don't agree that it's "admirably simple to use", it needlessly complex. For what I do, I don't need or want to care about runlevels, I always want respawning and I don't like the syntax of the configuration.

systemd seems like a really complex solution to a very simple problem, but I never used it.

Supervisord while being well documented and popular again seems to complicate what I want to do. Again I dislike the configuration, I fail to see why a shell script isn't better.

Runit I never used, because why would I replace daemon-tools with, what was when I first read about it, just a daemon-tools clone, made by people who didn't like Bernsteins licensing. I guess it's what I would most likely go for, if daemon-tools where not available.

The rest I never heard of.

<comment><AUTHOR> Runit is very similar in design to Daemontools, but it seems generally more featureful and better maintained. If you like one, chances are you'll like the other. (Also, daemontools' licensing is no longer an issue; it's been public domain since 2007.)

I do take issue with your characterization of Upstart and Systemd as needlessly complex; almost all of that complexity is for their role as init.d replacements, and doesn't need to affect you if you just want to get a few programs running. Their big selling point, for me, is that one of them is usually the default and the config files are quite easy to write if you're not doing anything elaborate.

<comment><USER5> As a diehard daemontools user this comment made me take another look at runit. Just to be clear, runit is one of several forks of daemontools. Another one is daemontools-encore [1].

There are a couple nice runit features that stock daemontools lacks.

The ability to run each service in a separate process session with "runsvdir -P" is a big one. If your ./run file consists of a pipeline, bouncing the service with TERM will produce orphans otherwise. (I have some patches to daemontools-encore that do this on a per-service basis.)

The svlogd program from runit also looks nicer than multilog. I've tried to like tai64, I really have, but I'd rather just have human AND machine readable timestamps from the get go.

I like that service dependencies aren't statically declared in runit, but rather something you can block on in your ./run file.

[1] <http://untroubled.org/daemontools-encore/>

<comment><USER6> runit also has man pages.

<comment><USER5> daemontools also has manpages, thanks to Garrit Pape, the author of runit ;)

<http://smarden.org/pape/djb/manpages/>

The manpages are installed by default in every system I use -- debian, ubuntu, MacPorts, and FreeBSD ports.

<comment><USER6> Indeed. They are not part of the standard daemontools source though, which is why I thought it worth mentioning.

</comment></comment></comment></comment></comment></comment><comment><USER7> "Process Supervision: Solved Problem" [0] is a great rundown too (from one of the Chef Opscode guys).

His recommendation is to use runit (based on djb's daemontools).

He recommends against using Bluepill, God, Foreman, supervisor and others.

[0] [http://jtimberman.housepub.org/blog/2012/12/29/process- super...](http://jtimberman.housepub.org/blog/2012/12/29/process-supervision- solved-problem/)

<comment><AUTHOR> If you do what that guys says and use runit, you'll be happy. I use runit, and I am happy with it.

However, I also think you'll be happy with Upstart or Systemd, even if you find them over-engineered and inelegant. They'll do what they need to with minimal configuration, and you're probably already using one of them behind the scenes. Why not use one of them, if you have it sitting right there, already installed and configured?

</comment></comment><comment><USER5> "The one caveat to be aware of is that Runit expects daemons not to fork."

Nitpick: it's fine if your daemon forks, it just shouldn't background itself, which is different.

A nice succinct implementation of daemonization lives in the BSD sources: [https://github.com/DragonFlyBSD/DragonFlyBSD/blob/master/lib...](https://github.com/DragonFlyBSD/DragonFlyBSD/blob/master/lib/libc/gen/daemon.c)

But don't do this in your code ;) _Do_ use a service manager to do this for you. Programs which lack a foreground option are harder to test and interact with due to the action-at-a-distance -- one of my chief objections with init.d-style service management over what I'll call daemontools-style service management. Nearly all popular daemons will have an option to stay in foreground for this reason. The only exceptional case that comes to mind is nginx, which does have a foreground option, but you'll lose zero-downtime upgrades if you use it (due to some extreme cleverness).

</comment><comment><USER8> Like others have already said, monit also deserves a mention. Whilst it can start the process for you, it probably doesn't really fall under the same category. It's more about making sure things are running, do not consume too much resources etc.

However, the flexibility of monit beyond process monitoring is really great. Anything from filesystem usage, cpu, memory to monitoring tcp ports, file checksums, you name it. I'm yet to find something you can't do with this little toy, and it's rock solid. I even use it to pull graphite stats[1] and report when certain thresholds are reached, like the percentage of 500s compared to other response codes in my nginx logs.

I also personally like its configuration syntax better than most yaml-like DSLs. It feels almost like writing natural text.

[1]<http://blog.gingerlime.com/2013/graphite-alerts-with-monit/>

<comment><USER9> +1 to monit. I have an app that does a lot of scraping via Selenium, headless using xvfb and Firefox. If the Selenium scripts fail, Firefox never closes. On an EC2 micro instance, it doesn't take long before 40MB+ FF processes kill other things, like Tomcat (which runs my queue processor). I use monit to watch the root URL, and when it fails, it fires off a handful of scripts that restart the various pieces of the application.

</comment></comment><comment><USER10> highly recommend runit + monit. The beauty of runit is the ./run script is easy to run by hand to test. The beauty of monit is it can watch things like http urls and ports, and if something goes wrong, monit executes sv down myservice; sv up myservice. (sv is the actual launcher-thingy of runit)

If you use monit without runit, you end up in this weird world where monit starts and stops things in a very odd environment and it's flakier, environment variables are missing, etc. Also monit takes a long time to notice things are down, and doesn't start things right away on boot.

Finally if you don't boot your services with monit, you booted them with something else meaning that when a restart occurs, you're starting a service in a different environment than you booted in. weird.

so, let monit check health of things, let runit start and stop things.

<comment><USER11> are you using ruby with rhnit?

which server (mongrel, thin, etc) do you recommend for playing nice with runit - which the OP mentioned has a problem with fork.

<comment><USER10> passenger

</comment></comment></comment><comment><USER12> This didn't really seems like a comparison, more of a description. I was hoping for a bit more.

We're currently using supervisord but we've had to hack it to support more than 200 workers, and I'd like something with the ability to change the number of workers dynamically.

<comment><USER13> We use Supervisor integrated with Zookeeper to dynamically adjust the number of worker processes.

</comment></comment><comment><USER14> Systemd has socket activation, where it opens sockets and passes them to applications.

But the maintainers seem to have sub-zero interest in passing said listening sockets to multiple applications, which I'm sorry to see not be available. I'm thankful for this article, because it discusses Supervisord and Circus's willingness to do some pooled program handling.

</comment><comment><USER15> Amazed to see so many simple responses in this thread given that the single- system paradigm is sort-of heading out of date now (manual Unix systems administration is arguably a dying horse). For any serious (ie. highly available) systems, a major option that _should_ be under discussion is Pacemaker/Corosync. <http://clusterlabs.org/>

It's a pain to learn, easy to screw up, non-trivial to test exhaustively, but provides highly flexible daemon migration and monitoring functionality you aren't likely to find anywhere else, for free, and for any imaginable service. (Actually you can monitor anything with it, including hardware. Check it out.)

<comment><USER16> These projects seem to do a very poor job of presenting themselves. I spent a good deal of time reading about each and have yet to have a really good grasp of what they are for. They seem to be about maintaining a cluster of hardware servers to run arbitrary services with configuration and fail over.

Do you know of some decent article, blog post, video, whatever that presents these products, what they are good for and a typical use case?

<comment><USER15> I agree wholeheartedly about the failure to communicate clearly. At least one contributing factor to the present situation is the lucrative consulting that exists around these solution types and helps to fund their development. Simply put, _if it was that easy, everyone would be doing it (and we'd be out of a job)_. However, to be fair, things change fast and there does exist a lot of good documentation - just not necessarily perfectly up to date for your scenario. Your assumption is perfectly correct. Have a look at <http://www.linbit.com/en/downloads/tech-guides> or <http://clusterlabs.org/doc/> or try #linux-ha or #linux-cluster on freenode.

</comment></comment></comment><comment><USER17> I wouldn't use god. It's had issues crashing or misbehaving in the past. I've found monit very stable if a little awkward to configure correctly.

Today I would use systemd with monit for additional monitoring. systemd ensures that the same environment is used regardless of how it is invoked, and the units are extremely simple. Monit can ensure a running server is behaving (memory limits, CPU limits, testing HTTP) and rely on systemctl for restarting processes.

</comment><comment><USER18> One thing I always found annoying is, say I am on a shared host like WebFaction, and I use supervisord to manage my uWSGI (or whatever) instances. What do I do to ensure supervisord itself starts? Current approach is a script run by cron that greps ps for supervisord and if not found, launches it. But that seems rather flakey and unreliable.

Suggestions?

</comment><comment><USER19> Poor man's solution:

        echo 'pidof myapp > /dev/null || su appuser -c "myapp --arguments" &' > ~/check_services.sh
        echo 'exit 0' >> ~/check_services.sh
        chmod 700 ~/check_services.sh
        #Add ~/check_services.sh & to /etc/rc.local for startup
        #Add */1 * * * /root/check_services.sh to crontab

<comment><USER3> Interesting hack, sure, but "poor man solutions" would be interesting if "rich man solutions" cost too much. That is not the case here. For someone that can understand this, runit will take < 10 minutes to learn.

</comment></comment><comment><USER20> I would really like to hear from more people who use Circus in production. It seems interesting.

<comment><USER21> Tried it and I can tell you it's not production ready. It would randomly go rogue on our production servers and would start falsely detecting a worker crash and start launching more.

It took us a long time to trace it down and we quickly switched to runit and supervisor on a few servers. All of our problems went away.

<comment><USER22> Circus author here.

I am interested in any form of feedback on the issues you had. Falsely detecting a work crash sounds very weird and unprobable because Circus uses the system PID list to check on processes - so I wonder what happens in your case.

</comment></comment></comment><comment><USER23> Monit is another standard program, but I think these these things being compared are really not similar. Aren't upstart and systemd replacements for init.d runlevel scripts? While God & Monit make sure programs are running. I suppose there is overlap.

<comment><AUTHOR> There's no clear delineation here. Upstart and Systemd are init.d replacements, both of which can make sure that programs are up and running, and incorporate some basic process monitoring. Runit is similar, and _can_ replace init.d, but will happily run as just another process. God can't replace init.d, but it can easily handle the daemonize-and-keep-running functionality that's at the core of what we want from all this stuff. In addition, God can do some health checking, for things like memory and CPU consumption. IIRC Monit doesn't actually start other programs under itself, but it can be configured to watch running programs, and kick them if necessary.

If you have a clean and useful taxonomy for this stuff, I'd be interested in hearing it.

<comment><USER23> Monit will startup a program if it's not running according to it's pid file. In other words if the pid file doesn't exist it will start the program. I use it in deploys by killing a program and letting Monit start it again with the new version.

</comment></comment></comment><comment><USER24> 1\. Keep services running: Runit, even with daemontools differences, is hard to beat.

2\. Hard(-ish) resource limits and accounting: LXC w/ cgroups. Almost as good as full paravirtualization (Xen). There are still some issues with limiting resource contention impact between cgroups.

3\. Softer resource limits, rogue app restarter: We've heavily modified bluepill because it seemed to lack insight on the needs and challenges of large-scale production ops. Specifically, we've added optional total child process limits (a few issues reported, fixed and even submitted a pull request). It might be useful to add max # of processes, nic bandwidth, iops and couple other checks.

Also worth considering:

4\. Status monitoring: Icinga

5\. Performance: collectd

6\. Entropy injection: Chaos Monkey

</comment><comment><USER25> I'll throw my hat in the ring here and point out daemon-manager: <http://porkrind.org/daemon-manager/>

It was designed to be simple (creating a new daemon conf file is trivial), low over-head (it's very init-ish in function so it needs to not take a lot of memory or processor time), stable (can't crash or it loses track of everything that it launched), and secure (lets you launch your daemons as different users --usually at a lower security level).

I've been using it myself an almost every server I administrate (I wrote it to scratch my own itch) for a couple years now and it's been very stable.

</comment><comment><USER26> runit user here. For someone without a strong understanding of UNIX the runit documentation will strike her as a little low level. That being said, the learning curve is steep but short. What's missing is a blog post that explains UNIX from the point of view of runit. Starting & restarting and logging. It's actually a good place to start learning more about UNIX.

Zed Shaw has also starting a project of his own to tackle these things. Not sure how it handles multiple unrelated things. Seems to be a good idea to document in understandable Python code, what the requirements are to daemonize a process corrently.

</comment><comment><USER27> When I saw the title "vs. God", I thought it was going to include something like "Tup vs. the Eye of Mordor", a whimsical performance comparison between the Tup build system and "The Eye or Mordor" in the guise of a simple script that does no dependency checking because it already knows what has changed: <http://gittup.org/tup/tup_vs_mordor.html>

</comment><comment><USER28> "[Upstart] currently has no way to adjust the maximum number of file descriptors, or limit memory usage"

That's weird, because that's one of the things i like about Upstart. The ability to set pam limits and others in the same file. Upstart seems to support the same options Runit memory wise: stack, data, memlock ...

limit nofile 10000 10000

limit nproc 1024 1024

nice 3

chroot /var/roots/mychroot

[0] <http://upstart.ubuntu.com/wiki/Stanzas#limit>

</comment><comment><USER29> My favorite remains BSD style (rc.conf+rc.d/) init - which for some reason receives no mention here. Arch Linux used to have it too, but then inexplicably dumped it.

[http://www.freebsd.org/cgi/man.cgi?query=rc&sektion=...](http://www.freebsd.org/cgi/man.cgi?query=rc&sektion=8)

<comment><USER18> Because having long bash scripts and every process implementing it's own daemonisation in weird and wonderful ways is redundant and inconsistent, and makes making something a daemon frustrating.

<comment><USER30> >Because having long bash scripts

Has nothing to do with BSD style init? No part of the system should be either long or bash.

>it's own daemonisation in weird and wonderful ways is redundant and inconsistent

Calling daemon() is not that hard.

>and makes making something a daemon frustrating

That doesn't even make sense. People writing shitty software that should be a daemon but isn't makes that "frustrating".

</comment></comment></comment><comment><USER31> I have been using supervisor with my django apps for the past 2 years. I have no real complaints, and actually like how you can easily add new processes with a single conf file and then type

supervisorctl update to start the new deamon without having to restart the whole supervisor process over again ...

<comment><AUTHOR> Just FYI, that feature is shared by all the alternatives mentioned in the article. (The exception would be runit, which has one supervisor process per service, thus making the issue moot.)

</comment></comment><comment><USER32> We're using Bluepill, which has a friendly DSL to define CPU and RAM usage limitations. This can be quite handy if you need to occasionally restart a process with memory leaks or the like.

I believe God and Monit have similar capabilities, although I haven't used them personally.

</comment><comment><USER33> I'm extremely happy with supervisord, but now wondering how Runit is better.

</comment><comment><USER34> Don't forget about s6 which seems pretty interesting as well (I admit I haven't used it yet). <http://www.skarnet.org/software/s6/>

</comment><comment><USER35> Anyone have any experience with Bump's Angel?

<comment><USER36> Angel author here. It works very well for us and some others, but it must be said it is not as featureful as most of these. For example, it provides no assistance for log rotation (you'd need to do this via a shell pipeline with multilog or similar) and does not have a "control interface" (zmq socket etc) of any kind.

<comment><AUTHOR> I think that logging should be separate here, so I'm actually okay with built- in logging support being minimal to nonexistent. If you use multilog (from daemontools) it will handle logging to rotated files; if you use svlogd (from runit), it can do the same, and also includes syslog support. If you just log to syslog in the first place, and your syslogd is at all modern, it can also handle writing things to log files, here or on a remote machine, which is cool too if you're okay with configuring it.

Or you can do something event more radical, like using fluentd[1], which looks really useful and well-designed. I like having this decision decoupled from the process manager. So, I would not count that as a strike against Angel.

[1] <http://fluentd.org>

<comment><USER36> Yeah, I agree. To this end, I just added support for specifying a logger process, and angel will now pipe stdout/stderr to it ala daemontools. Definitely don't want angel in the business of being a log rotator.

</comment></comment></comment></comment><comment><USER37> what about <http://libslack.org/daemon/> ?

</comment><comment><USER30> I'm amazed at how far the linux world has fallen in the last decade. It is now "normal" for people to run an extra layer of buggy crap software to monitor their buggy crap software and restart it when it crashes. Rather than the long standing unix solution of not using buggy crap software that crashes constantly in the first place.

<comment><USER3> All that unix software was so great! Apache 1! Woo! bind! Yeah! sendmail! Nice! That's pretty much it! It never crashed and was so secure! Happy times!

Trololol.

</comment></comment><comment><USER38> I was hoping the daemon from Daniel Suarez's Daemon novel had been located.</comment>


Flawed Kinect Offers Tantalizing Glimpse at Gaming's Future - <AUTHOR> http://www.wired.com/gamelife/2010/11/review-kinect/ <comment><USER2> I'm not so much concerned with the Kinect hardware for gaming, but what it's potential might be for robotics sensing. This confirms my suspicions that this sort of active sensing technology doesn't work well with natural lighting, although anecdotally I have heard people say that this sort of system can be made to work outdoors if you pump out enough energy.</comment>


Exploding Git Repositories - <AUTHOR> https://kate.io/blog/git-bomb/ <comment><USER2> I wonder what the author means by "a lot" of RAM and storage. I tried it for fun. The git process pegged one CPU core and swelled to 26 GB of RAM over 8 minutes, after which I had to kill it.

<comment><USER3> Yeah I tried it too. Killed at 65G. Disappointed that Linux killed Chrome first.

        Oct 12 15:47:52 x99 kernel: [552390.074468] Out of memory: Kill process 7898 (git) score 956 or sacrifice child
        Oct 12 15:47:52 x99 kernel: [552390.074471] Killed process 7898 (git) total-vm:65304212kB, anon-rss:63789568kB, file-rss:1384kB, shmem-rss:0kB

Edit:

Interesting. Linux didn't kill Chrome, it died on its own.

        Oct 12 15:42:21 x99 kernel: [552060.423448] TaskSchedulerFo[8425]: segfault at 0 ip 000055618c430740 sp 00007f344cc093f0 error 6 in chrome[556188a1d000+55d1000]
        Oct 12 15:42:21 x99 kernel: [552060.439116] Core dump to |/usr/share/apport/apport 16093 11 0 16093 pipe failed
        Oct 12 15:42:21 x99 kernel: [552060.450561] traps: chrome[16409] trap invalid opcode ip:55af00f34b4c sp:7ffee985fb20 error:0
        Oct 12 15:42:21 x99 kernel: [552060.450564]  in chrome[55aeffb76000+55d1000]
        Oct 12 15:47:52 x99 kernel: [552390.074289] syncthing invoked oom-killer: gfp_mask=0x14201ca(GFP_HIGHUSER_MOVABLE|__GFP_COLD), nodemask=0, order=0, oom_score_adj=0

Seems Chrome faulted first, but it was probably capturing all signals and didn't handle OOM. Then next, syncthing faulted and it started the oom-killer which correctly selected 'git' to kill.

<comment><USER4> > [..] and didn't handle OOM.

How would Chrome 'handle' an OOM anyway? As far as I'm aware, malloc doesn't return ENOMEM when the system runs out of memory, only when you hit RLIMIT_AS and alike.

<comment><USER5> Or when you hit 4G VIRT on 32-bit.

Took me a good day's worth of debugging before some bright spark piped up and said "wait, you said you were on x86-32...?"

...yeah, I use really old computers.

<comment><USER6> I'm setting up my last machine for my wife for gaming. Athlon X4 630, and 16 GB of RAM. I loaded windows up and said it had ~2 GB free and I was like "oh crap, the RAM sticks must be dead" (because the last motherboard that I just replaced broke some RAM slots).

I fixed my old video card, a GTX 560, and wanted to see what it could run. I loaded steam and PUBG said "invalid platform error". It took me a moment. I hit alt-pausebreak, presto, Windows 32-bit. Whoops.

Hadn't had that problem in a long time except at clients running ancient windows server versions complaining about why Exchange 2003 won't work with their iPhones anymore "it used to work and we didn't change anything!" (Yeah... but the iPhone DID change--including banning your insecure 2003 Exchange protocols.)

</comment></comment></comment></comment></comment><comment><USER7> I'm curious how this was uploaded to GitHub successfully. I guess they do less actual introspection on the repo's contents than I thought. Did it wreak havoc on any systems behind the scenes (similar to big repos like Homebrew's)?

<comment><USER8> I too was curious about this.

[https://github.com/Katee/git- bomb/commit/45546f17e5801791d4b...](https://github.com/Katee/git- bomb/commit/45546f17e5801791d4bc5968b91253a2f4b0db72) shows:

"Sorry, this diff is taking too long to generate. It may be too large to display on GitHub."

...so they must have some kind of backend limits that may have prevented this for becoming an issue.

I wonder what would happen if it was hosted on a GitLab instance? Might have to try that sometime...

<comment><USER9> Since GitHub paid a bounty and Ok'd release, perhaps they've patched some aspects of it already. Might be impossible to recreate the issue now.

My naive question is whether CLI "git" would need or could benefit from a patch. Part of me thinks it doesn't, since there are legitimate reasons for each individual aspect of creating the problematic repo. But I probably don't understand god deeply enough to know for sure.

<comment><USER10> is this a git->god typo, or a statement about your feelings towards Linus?

<comment><USER11> Please don't let Linus read this

</comment></comment></comment></comment></comment><comment><USER12> Because that page is AMP by default, it takes about 7 seconds to load the page on my laptop. AMP is _really_ slow in some cases.

Edit: see my comment below before you downvote me.

<comment><USER13> Huh, I've tested on a bunch of devices/connections and haven't encountered that. Do you know what causes AMP to be that slow for you? I'll take a look at serving non-AMP pages by default. It will require tweaking how image inclusion works.

<comment><USER12> For people who use extensions or browsers that block third party JS, AMP pages will take many seconds to load in non-mobile Web browsers.

Here is information about some of the other problems with AMP:

[https://www.theregister.co.uk/2017/05/19/open_source_insider...](https://www.theregister.co.uk/2017/05/19/open_source_insider_google_amp_bad_bad_bad/)

[https://danielmiessler.com/blog/google-amp-not-good- thing/](https://danielmiessler.com/blog/google-amp-not-good-thing/)

[https://ethanmarcotte.com/wrote/ampersand/](https://ethanmarcotte.com/wrote/ampersand/)

[https://css-tricks.com/need-catch-amp-debate/](https://css-tricks.com/need- catch-amp-debate/)

[https://daringfireball.net/linked/2017/01/17/schreiber- amp](https://daringfireball.net/linked/2017/01/17/schreiber-amp)

<comment><USER14> Fix your browser /shrug

<comment><USER12> It isn't just my browser. AMP performs very badly in some non-mobile browsers (no extensions).

</comment></comment></comment></comment></comment><comment><USER15> Why not just always run git under memory limits?

For example:

      %  ulimit -a
      -t: cpu time (seconds)              unlimited
      -f: file size (blocks)              unlimited
      -d: data seg size (kbytes)          unlimited
      -s: stack size (kbytes)             8192
      -c: core file size (blocks)         0
      -m: resident set size (kbytes)      unlimited
      -u: processes                       30127
      -n: file descriptors                1024
      -l: locked-in-memory size (kbytes)  unlimited
      -v: address space (kbytes)          unlimited
      -x: file locks                      unlimited
      -i: pending signals                 30127
      -q: bytes in POSIX msg queues       819200
      -e: max nice                        30
      -r: max rt priority                 99
      -N 15:                              unlimited
      %  ulimit -d $((100 * 1024)) # 100 MB
      %  ulimit -m $((100 * 1024)) # 100 MB
      %  ulimit -l $((100 * 1024)) # 100 MB
      %  ulimit -v $((100 * 1024)) # 100 MB
      %  git clone https://github.com/Katee/git-bomb.git
      Cloning into 'git-bomb'...
      remote: Counting objects: 18, done.
      remote: Compressing objects: 100% (6/6), done.
      remote: Total 18 (delta 2), reused 0 (delta 0), pack-reused 12
      Unpacking objects: 100% (18/18), done.
      fatal: Out of memory, malloc failed (tried to allocate 118 bytes)
      warning: Clone succeeded, but checkout failed.
      You can inspect what was checked out with 'git status'
      and retry the checkout with 'git checkout -f HEAD'

</comment><comment><USER16> Run this to create a 40K file which expands to 1GiB

      yes | head -n536870912 | bzip2 -c > /tmp/foo.bz2

I would imagine you could do something really creative with ImageMagick to create a giant PNG file as well that'll make browsers, viewers, editors crash as well.

<comment><USER17> PNG has dimensions in the header so the decoder should know when it's decompressed enough.

</comment></comment><comment><USER11> Odd. It's surprising to me that this example runs out of memory. What would be a possible solution?

Admittedly I don't know that much about the inner-workings of git, but off the top of my head, perhaps something with traversing the tree depth-first and releasing resources as you hit the bottom?

<comment><USER16> You need a problem to have a solution to it. What do you consider to be the problem here?

This is essentially something that can be expressed in relatively few bytes that expands to something much larger.

Imagine I had a compressed file format for blank files "0x00" the whole way. It is implemented by writing in ascii the size of the uncompressed file.

So the contents of a file called terrabyte.blank is just ascii "1000000000000" ... or the contents of a file called petabyte.blank is "10000000000000"

I cannot decompress these files... what is the solution?

<comment><USER18> >You need a problem to have a solution to it. What do you consider to be the problem here? > >This is essentially something that can be expressed in relatively few bytes that expands to something much larger.

That seems to be the problem. I mean, if an object expands to something much larger to the point that it crashes services just by the sheer volume of the resources it takes... That is pretty much the definition of an attack vector of a denial-of-service attack.

<comment><USER19> There is a problem here, but it's not with data. It's with the service.

Being able to express trees efficiently in a data format is an useful feature, but it requires the code processing it not to be lazy and assume people will never create pathological tree structures.

</comment></comment></comment></comment><comment><USER20> Would this be possible with a patch-based version control system like Darcs or Pijul? Does patch-based version control have other analogous security risks, or is it "better" in this case?

<comment><USER21> If the patch language includes a recursive copy than it's possible to reproduce this problem in that setting.

<comment><USER18> If I understood correctly, this problem isn't caused by recursive copies but simply by expanding references. The example shows that the reference expansion leads to an exponential increase in resources required by the service.

<comment><USER19> This means the same in this context; if it was just expanding references one by one while walking through the tree this would not happen - the bomb requires copies of expanded references to be stored in memory.

</comment></comment></comment></comment><comment><USER22> Bare for the win.

        git clone https://github.com/Katee/git-bomb.git --bare

</comment><comment><USER19> Going to second level on Github breaks commit name for me - it gets stuck with "Fetching latest commit..." message. Curiously, go one level deeper and the commit message is again correct.

[https://github.com/Katee/git- bomb/tree/master/d0/d0](https://github.com/Katee/git-bomb/tree/master/d0/d0)

(INB4 The article suggests Github is aware of this repo, so I have no qualms posting this link here.)

</comment><comment><USER23> Directory hard links would "fix" this issue since `git checkout` could just create a directory hard link for each duplicated tree. I wonder why traditional UNIX does not support this for any filesystem.

(Yes you would need to add a loop detector for paths and resolve ".." differently but it's not like doing this is conceptually hard.)

</comment><comment><USER24> Has anyone tried to see how well BitBucket and Gitlab handle this?

</comment><comment><USER25> What happens if you try to make a recursive tree?

<comment><USER26> As in a tree that points to itself? You cannot, since a tree would have to point to its own SHA1. So this would require you to know your own tree's SHA and embed it in the tree.

<comment><USER27> Reminded me of the GIF that displays its own MD5 hash:

[https://twitter.com/bascule/status/838927719534477312](https://twitter.com/bascule/status/838927719534477312)

<comment><USER28> So it's possible, but impractical?

<comment><USER27> I think it's possible.

</comment></comment></comment></comment></comment><comment><USER29> If we all click "Download ZIP" on this repo we can crash GitHub together!

Just click here: [https://codeload.github.com/Katee/git- bomb/zip/master](https://codeload.github.com/Katee/git-bomb/zip/master)

<comment><USER30> Wouldn't that just do a `git fetch` and therefore not have the issue?

<comment><USER31> "Download ZIP" downloads the repository's files as a zip. No Git involved for the downloader.

<comment><USER32> i expect the download zip to be implemented as running 'git archive --format zip | write-http-response-stream'

<comment><USER33> Hmm I'd hope they do a caching step in between ;)

</comment></comment></comment></comment></comment><comment><USER34> I thought it would self destruct after cloning of forking before clicking :)</comment>


Type it Out: Extension that disables copying and forces you to type it out - <AUTHOR> https://chrome.google.com/webstore/detail/type-it-out/cakjfpgihbciegpnmholbaafghdbngjp?hl=en <comment><USER2> > _This extension helps you learn more efficiently._

The motivation makes sense, but the example screenshots show the user copying variable declarations, a rote task where forcibly typing it doesn't provide any insight or knowledge.

Therein lies the problem. Not all DIY tasks are constructive learning and might just waste valuable time. I'm not even sure if forcibly typing code from Stack Overflow would even be insightful unless the intent behind code was properly explained/documented.

<comment><AUTHOR> I find this extension most helpful for learning syntax for new languages. For when you know what you want to do, but don't know the specific syntax to do so.

</comment></comment><comment><AUTHOR> Hey author here, I built this to combat my bad habit of copy-pasting from stack overflow. It's open sourced here [https://github.com/jaxgeller/type-it- out](https://github.com/jaxgeller/type-it-out)</comment>


Will OpenCL help displace GPGPU? (2012) - <AUTHOR> http://yosefk.com/blog/will-opencl-help-displace-gpgpu-parallella-p2012.html <comment><USER2> That article is largely a trainwreck.

Yes, OpenCL is an API and shader language for GPGPU usage.

Yes, it is a competitor to CUDA.

No, CUDA is not the majority of GPGPU usage (although in 2012, 3+ years ago, this may have been true depending on who you spoke to), OpenCL now is.

No, CUDA and OpenCL competing is not an issue, as Nvidia is a founding member of Khronos's OpenCL committee, and Nvidia uses a merged compiler backend for all their shaders (including OpenCL and CUDA); neither API will produce better optimized binaries than the other on Nvidia hardware.

Yes, there are OpenCL implementations for CPUs: AMD and Nvidia's implementations both can be executed on x86 CPUs (useful for lock step full program debugging without needing to deal with the async nature of GPUs), Intel has one for x86, IBM has one for PPC/PPC64/Power.

Yes, there are OpenCL implementations for DSPs. That was already a thing in 2012. Sony/IBM has one for Cell SPEs.

Yes, GPUs are in fact highly parallel DSPs.

No, since 2012, Parallela nor P2012 ever took off.

No, OpenCL cannot displace GPGPU because that is like saying Electric will displace the Car. No, instead, we'll make cars that run on electricity instead of gasoline, ie, we'll be changing APIs to more universally supported ones.

Is GPGPU dying? 'Yes.' What is killing it? The march of technology into making EVERYTHING generic processing: floating point used to be a specialized application requiring an optional co-processor: it is now part of the CPU; SIMD used to be part of specialized DSPs, that, too, is now part of the CPU, and that part of the CPU becomes more complex every other generation or so; GPGPUs were merged on-die a few generations back and natively speak to the CPU's memory controller and socket bus as if they were just another core.

And merging GPGPUs on-die isn't even new, if we view them as just specialized highly parallel DSPs: MIPS and ARM SoCs have done this for like the past decade by including communications and audio processing DSPs in the package or on-die.

So, just like the FPU, and SIMD, GPGPUs as highly parallel DSPs are being merged into the CPU.

And, as a side note, you know what this looks like from the other side, a CPU becoming a highly parallel DSP? Intel's Xeon Phi series.

And, as another side note, since it is now a given that CPUs need to be able to do embarrassingly parallel tasks optimizedly, Intel is now tackling the fact that QPI is not scalable across a CCNUMA cluster like AMD's Hypertransport is, and is now moving to embed their Infiniband variant onto CPU and Phi dies, for native 40/100gbit on-die. There are already Xeon E5/E7s slated to have this on a post-LGA-2011-3 socket.

Disclaimer: I wrote DiabloMiner, that popular GPU Bitcoin miner everyone used until ASICs took over.

<comment><USER3> Why do you think AMD would bother with something like [http://www.anandtech.com/show/9792/amd-sc15-boltzmann- initia...](http://www.anandtech.com/show/9792/amd-sc15-boltzmann-initiative- announced-c-and-cuda-compilers-for-amd-gpus) so recently if OpenCL has become the dominant platform?

Nvidia still doesn't even support OpenCL 2.0 from two years ago. Surely if OpenCL was making significant gains they would be forced to support it, especially since Nvidia is far more popular in any kind of HPC setup with Tesla compared to FirePro?

<comment><USER2> Nvidia has been having problems implementing OpenCL 2.0 (DX12 support has been a higher priority for them, especially given that only Maxwell rev 2 properly supports things like Rasterizer Ordered Views and Tiled Resources).

Intel has had OpenCL 2.0 support since August 2014, and AMD since September 2014.

OpenCL 2.0's interesting features are shared virtual memory and generic memory space, which Nvidia hardware before Maxwell rev 2 may not be able to support, and Nvidia may have to wait until Pascal to support it (via the NvLink and unified memory feature set). Ergo, Nvidia may not have the hardware support to add OpenCL 2.0 yet.

OpenCL 2.1 is more of the interesting one because they are adopting a SPIR-V (Vulkan's IL representation)-based C++ subset, something Nvidia has been leading the charge on.

OpenCL 2.1 was finalized last month, so I don't expect drivers to support it properly until this time next year.</comment></comment></comment>