Sorry, Lisp. I tried.  - <AUTHOR> http://www.ketralnis.com/roller/dking/entry/20070609

<comment><USER2> _Without database access, sockets, threads, POSIX, or a portable FFI (which I could at least use to write many of those features), my programs can't talk to the outside world, and that makes them basically useless._

Is this a troll, or did the author simply not do very much research?

CLSQL can talk to MySQL, PostgreSQL, SQLite, Oracle and (through ODBC) MS SQL. What database does dking want to access?

I'm pretty sure all the major Lisps provide a socket interface. Usocket standardizes it.

I think the commercial Lisps provide threads. SBCL and CMUCL have threads on certain platforms, especially x86 Linux.

POSIX is available through the FFI, but some Lisps provide their own POSIX package. I think there's some work being done on a more portable library.

CFFI and UFFI are both good FFIs portable between all the major Lisps.

Everything the author asks for is easy to find. Many of these libraries are the first hit for a reasonable google search (e.g. "lisp sql"). I've had issues finding good Lisp libraries on occasion, but the situation is much better than the author suggests.

<comment><USER3> > Is this a troll, or did the author simply not do very much research?

It's not a troll, and I did do some research. I wrote this a few months ago (this is its second round on news.yc, and I'd make some addenda to it if it weren't actively linked to at the moment). My biggest problem was that I was insisting on portability across implementations, because that's something that I got for free in languages like Ruby that I didn't want to give up. Reading around, it looks like most Lisp users didn't have such a requirement, and mine was largely engineered because of bias given to me by the implementations of the other languages I was using.

When it comes down to it, I needed an implementation with those things that would work with FreeBSD, OS X, and Linux, and maybe sometimes Windows, and support threads, WebDAV, and some other stuff. While I still can't find a WebDAV client library, it looks like my requirements for portability across implementations may have been unfounded.

Since then I've gotten into Erlang, but if I were to try Lisp again it would probably be a Scheme after reading around.

<comment><USER2> Portability between implementations is a bit of a red herring (as you seem to have discovered); most libraries you'd actually want to use run on most implementations you'd actually want to use. Ruby and Python don't give you this either, but they do give you a standard implementation. I suspect a lot of Ruby code you might want to run doesn't work with JRuby or IronRuby.

Portability between operating systems is a more serious problem. I think ECL is the only free implementation that does this well, and most libraries don't seem to be tested on it.

<comment><USER4> >I suspect a lot of Ruby code you might want to run doesn't work with JRuby or IronRuby.

Not likely. Rails runs fine on jruby at least, and it does every dirty trick under the sun.

</comment></comment></comment></comment><comment><USER5> I had similar issues. But I find that the simplest way to get around them is to use VMWare or Parallels, depending on what your emacs is running on. Run Lisp inside the OS of your choice and connect to it using Slime.

If you don't use Slime+Emacs+Tramp, I don't think it's worth getting into CL at all. Slime absolutely rocks my world. I have 8 SBCL instances running on multiple machines (virtual and dedicated) on my network and the internet, and I don't have problems doing any lisp development.

My advice would be to pick the OS/Lisp implementation based on what you think you might need.

<comment><USER6> One of these SBCL instances is for telltrains dot com? :-)

<comment><USER5> 2 actually, running behind pound. I had a few more running on a couple of virtual slices on slicehost. But I recently got rid of them as SBCL has some issues with Xen.

<comment><USER6> wow! it's good! I need to know more about your impressions with SBCL to web development, I'm on it too.

</comment></comment></comment></comment><comment><USER6> Please, check <http://cliki.net>, <http://common-lisp.net> and <http://cl- user.net>. There are packages to all the things commented.

About SQL, for example, I'm using Submarine (ORM) with Postmodern and it's running like a charm. For Web development, there are Hunchentoot, AllegroServe and interesting frameworks like UCW or Weblocks (each one I'm using right now).

Maybe the guy didn't tried enough, more 5 min. at google would be enough ;-)

</comment><comment><USER7> Sounds like CL wasn't right for his particular requirements. He would have been better off with a Scheme implementation on the JVM -- Kawa or SISC.

<comment><USER8> Maybe Lush? <http://lush.sourceforge.net/lush-manual/f0288067.html>

Rather than a Lisp that runs on the JVM, I'd be very interested in knowing a Lisp that somehow runs as Python bytecode. Does such a monster exist?

<comment><USER9> Googling turns up a few lisp-in-python implementations: <a href="<http://www.biostat.wisc.edu/~annis/creations/PyLisp/>">Pylisp, <a href="<http://www.xs4all.nl/~yduppen/site/psyche.html>">psyche and <a href="<http://hkn.eecs.berkeley.edu/~dyoo/python/pyscheme/>">pyscheme (the last two being scheme, obviously). Haven't tried using them; maybe I will the next time I feel like frittering away a couple of hours.

<comment><USER9> clearly html links are the wrong way to go around here ;)

</comment></comment></comment></comment><comment><USER10> Lisp the Platonic ideal is powerful and important. Particular dialects like Common Lisp and particular implementations are akward because the ANSI standard is intentionally silent about common network, I/O, etc. stuff.

Old news.

</comment><comment><USER3> Submitter: thanks for submitting the (original) ketralnis.com article. The same article was submitted under my experimental domain wrotit.com to reddit, and I'd much prefer to reduce the incoming links to that domain until I've made final decisions about the link and URL structure over there :)

</comment><comment><USER11> Check out <http://programming.nu>. It has pretty much everything that this guy says he was missing.

</comment><comment><USER12> This guy is an idiot.</comment>


Lessons Learned – A Year of Going "Fully Serverless" in Production - <AUTHOR> https://dev.to/ketacode/lessons-learneda-year-of-going-fully-serverless-in-production-3mh8 <comment><USER2> I cringe every time I hear someone going "fully Serverless" or NoSQL or whatever is popular these days.

From the article, it mentions Torri, a site that is ~1.5 millionth in Alexa rank. I understand they probably have a backend with different server loads, but I highly doubt going fully Serverless has benefitted them for this particular site. A 1.5m Alexa rank is less than 5,000 page views a month.

> Serverless, because we like to sleep at night.

No, this is a solved problem already. Spin up a load balancer in front of your web servers, add health checks, and go take a peaceful sleep.

Using react, and then creating a static site, trying to beat the 50mb Lambda by deleting readmes and test, and the fact that you hope AWS would do something for the 500ms warm up time... They are big hints you are doing something wrong.

I'm not against Serverless, and I understand it has it's place. Just don't go overboard.

</comment><comment><USER3> Where's the feature that prevents Amazon from stealing your core business?</comment>


There Have Probably Been Aliens - <AUTHOR> http://www.nytimes.com/2016/06/12/opinion/sunday/yes-there-have-been-aliens.html?mwrsm=Facebook <comment><USER2> "In other words, given what we now know about the number and orbital positions of the galaxy's planets, the degree of pessimism required to doubt the existence, at some point in time, of an advanced extraterrestrial civilization borders on the irrational."

This is terribly silly. It reminds me of the Onion talks:

"Step 1: Devise an idea to create a car that runs on compost. Step 2: Create the car. We've already completed Step 1. We're half way there."

Increased certainty on three of the seven factors is irrelevant, because 99% of the uncertainty is concentrated in the other four. Specifically, the odds of abiogenesis are ~1/2^n, where n is the complexity in bits of the smallest self-replicator. n is bounded below by ~100 or so (or it would have happened in a lab somewhere), and is bounded above by ~1 million (the smallest existing bacteria), so we know it's somewhere in there. But where? We have only the vaguest guesses.

"Specifically, unless the probability for evolving a civilization on a habitable-zone planet is less than one in 10 billion trillion, then we are not the first."

"One in 10 billion trillion" sounds like a very large number, and it's put in italics for emphasis, but in information-theoretic terms it's tiny. log2(10^22) is just 73 bits. If all that we know about a number is that it's somewhere between a hundred and a million, saying that X would imply it's higher than 73 (well technically 73 + log2(#incidences of molecular combination), so probably a few hundred) is not a disproof of X, nor really even much evidence against X.

<comment><USER3> Your assumption that the odds of abiogenesis is given by `~1/2^n, where n is the complexity in bits' is utterly, completely false.

Statistical mechanical entropy and information theoretical entropy ('complexity in bits') are not the same quantities! But even if they were, your expression would still be incorrect because living system are definitionally not at equilibrium - so equilibrium statistical mechanics (whose predictions you are using to get that expression) does not apply.

No one is surprised when convection rolls spontaneously form as they boil a pot of water. But equilibrium statistical mechanics predicts that this should be extremely unlikely as convection roll states have very low entropy. That is, of all of the ways that the molecules in that pot of water could be oriented, vanishingly few of them correspond to convection roll states, where several Avogrado's numbers worth of particles conspire to all swirl around in a coordinated fashion to dissipate the heat of the burner faster than would occur just by diffusion.

The origin of life is very much analogous to the formation of convection rolls in the pot. And I believe is far more common than classical predictions imagine it to be - nearly everywhere there is an energy gradient in the universe, you will probably find life.

Citation: I have a degree in physics

<comment><USER4> This has nothing to do with physics. He's saying _if_ a first self replicator requires a very specific sequence (of something like RNA) to form by chance, then it can easily be incredibly unlikely to happen.

As an analogy, if I make a huge number of random files on my computer and execute them, the probability of getting a self replicating computer virus is really low. And one that successfully spreads itself to other computers over the internet should never happen.

If the first successful computer virus is 100 bits long, then probability of finding it in a single random search is 1/2^100. Which is just incredibly small and should never ever happen.

<comment><USER5> Purely hypothetical: where N is the fewest number of bits for a successful virus.

There could be 10 different ways to make a virus N bits long. There could be 1000 ways to do it in N+1 bits, and a billion ways in N+2.

The relationship between how many possible successful viruses exist to number of bits doesn't have to scale by any obvious pattern.

<comment><USER4> I don't see why adding more bits would make the program significantly more likely. Yes in theory that is possible, but I don't think it's relevant.

<comment><USER5> Knowing the size N of the smallest possible virus doesn't tell us much about the probability of a given random string being a virus (as long as it's longer than N)

There could be a magic threshold where if you're shorter than that you don't work.

For ex, maybe a valid file has to be at least 100 bytes, or the OS can't treat it as a file. Any random file 100bytes or longer could have a 10% chance of being a virus (the actual number depends on how the os works).

This would mean a really high chance of randomness creating a virus, even though N is 100.

The probability of vitality depends on the specifics of the operating system, and we can't infer it by looking at the size of the smallest possible virus.

</comment></comment></comment></comment></comment></comment><comment><USER6> If you parse the article carefully, the entire argument is summarized as

 _" There are more than 10^22 planets, therefore ETI probably exists[ed] if P(intelligent life|planet) > 10^-22"._

        Specifically, unless the probability for evolving a
        civilization on a habitable-zone planet is less than one in
        10 billion trillion, then we are not the first. [...] In
        other words, given what we now know about the number and
        orbital positions of the galaxy's planets, the degree of
        pessimism required to doubt the existence, at some point in
        time, of an advanced extraterrestrial civilization borders
        on the irrational.

<comment><USER7> That was extremely shallow from the nytimes.

</comment></comment><comment><USER8> Probably. Assuming they're correct. Our planet is 12,742km in diameter, yet we've only drilled 12.262km deep [1] (and we were wrong about how hot it was gonna be), and sent a human down ~10.900km [2] (and we still can't decide how deep we've actually been). Do we really think we understand the probability of alien life on another planet _that_ well, given the amount of data we have on those planets, and that we're constantly proven wrong on our own?

And yet (either way) I'm not sure it really matters. No human has left our own planet for longer than 437.7 days [3]. Say they're right, though. If an alien race exists, we're still a ways from being able to do anything about it. Unless we can make it there in 1.198 years. If that alien race can make it here, awesome. Let's talk then. If they can make it here and are hostile, then it's been swell. See you on the other side. Probably.

[1] [https://en.wikipedia.org/wiki/Kola_Superdeep_Borehole](https://en.wikipedia.org/wiki/Kola_Superdeep_Borehole)

[2] [https://en.wikipedia.org/wiki/Bathyscaphe_Trieste](https://en.wikipedia.org/wiki/Bathyscaphe_Trieste)

[3] [https://en.wikipedia.org/wiki/List_of_spaceflight_records](https://en.wikipedia.org/wiki/List_of_spaceflight_records)

<comment><USER9> You got the units of the borehole wrong, it's in meters not kilometers.

<comment><USER8> Well, we've certainly drilled more than 12 meters. :-P (Note the decimal point: 12.262km; I just converted to km to keep it consistent with the other measurements I cited)

Could be confusing if you're used to the inverse comma/period from what we use in the US (comma=thousands, period=decimal).

<comment> gh-lfneu28 Would be far less confusing if you'd just written "~12km" and "~11km". The meters add little to the point you're trying to make.

<comment><USER8> Well, that's probably true, but it was late and I thought the higher precision would help add to my point. Too late to fix it now, though. :-P

</comment></comment></comment></comment></comment><comment><USER4> >unless the probability for evolving a civilization on a habitable-zone planet is less than one in 10 billion trillion, then we are not the first.... the degree of pessimism required to doubt the existence... of an advanced extraterrestrial civilization _borders on the irrational_.

After working with probability theory and combinatorics a bit, you realize probabilities much smaller than this occur naturally all the time. It might be hard for people not familiar with that to appreciate this, and be wowed by such a small number.

The thing is that independent probabilities multiply together, and so approach 0 exponentially fast. The probability of winning the lottery is of course very small. But there are so many people, that someone wins a lottery every day.

But the probability of winning the lottery twice is really small. The set of people that have won twice is a very small set indeed. And the probability of winning 3 times is so small that we should expect it to be a sure sign of corruption of the lottery, or intervention by supernatural forces. It's also about 1 in 10 billion trillion.

It seems to me that the evolution of intelligent life from nothing, also requires many things to go right. We don't know what all these things are, and we can only guess what their probability is. But if even a few of them have relatively small probabilities, then the probability of _all_ of them happening could easily be much lower than "1 in 10 billion trillion".

There are many candidates for this so called "great filter". Some ideas I think are plausible are the evolution of multicellular life. The Earth was around _a very, very_ long time before multicellular life formed. And it's believed to have required some weird coincidences like a parasite evolving into mitochondria.

Another candidate is the evolution of intelligence itself. Most animals aren't intelligent. Intelligence doesn't actually seem to be very correlated with fitness beyond a certain point - and reducing brain size is actually beneficial. Animals with brains were around for a very, very long time before anything like humans appeared. Some scientists believe that humans required a really weird set of conditions to evolve. There are theories that we are a case of accidental runaway sexual selection, like the peacock's feathers.

And there are many other factors we can't possibly know, that might be unique to Earth and rare on other worlds. We also don't know a huge amount about the evolution of early life. Getting from RNA self replicators to the complicated machinery of the most recent ancestor was no small feat.

</comment><comment><USER10> I can't recommend "The Vital Question" by Nick Lane enough for people who are really interested in the question of life elsewhere. It does such a wonderful job of show how mechanically "life" can be started as a way of simply reacting chemicals to a lower energy state. It neatly ties together biochemistry and geochemistry in a very plausible (and testable) way.

Given the analysis, I would expect that bacteria and archaea, are present on any world with liquid water and a threshold amount of heat. Which would include Mars, Europa, and probably Io as well. There are a number of forms of bacteria living in the ionosphere which could be dropped on to Mars and they would do just fine there.

The odd bit comes from the origin of _eukaryotes_. Which, according to Lane, might arise if archaea were endosymbiotically living inside bacteria. That works but he doesn't offer any suggestion of what caused it to happen the first time. And he makes a reasonably good case that what ever it was that caused it, happened exactly once in the 4+ billion years of the planet and 3+ billion years of bacterial life roaming around.

<comment><USER11> I read his book 'power sex suicide'. Here is a good summary:

[http://ronbarak.tumblr.com/post/25996121029/life-is-it- inevi...](http://ronbarak.tumblr.com/post/25996121029/life-is-it-inevitable- or-just-a-fluke-by-nick)

In the book he expanded on what was needed to make the jump to complex life and his opinion on how unlikely it was. He made a case for how only a relatively small time window was available for it to happen at all.

I second the rec!

</comment></comment><comment><USER12> Here's a Drakes Equation visualizer I made. I know it doesn't address certain factors but it's fun to play with as a thought experiment. Even with some optimistic numbers there's not that much out there.

[https://tristaaan.github.io/ngDrakesEquation/](https://tristaaan.github.io/ngDrakesEquation/)

</comment><comment><USER13> Content of the article is pretty information, but worst clickbait title of the year. At the risk of alienating some fraction of HN community (no pun intended), I'm going to say this is what you get when trying to chase the likes of Gawker (again, no pun intended).

<comment><USER14> The irony here is that Gawker's relevant clickbait is actually far superior:

[http://io9.gizmodo.com/11-of-the-weirdest-solutions-to- the-f...](http://io9.gizmodo.com/11-of-the-weirdest-solutions-to-the-fermi- paradox-456850746)

<comment><USER15> I read a good short story recently about a weird Great Filter:

[http://www.begoodenough.com/the-great- filter/](http://www.begoodenough.com/the-great-filter/)

<comment><USER16> The first movie that I think plays around with the idea of a Great Filter (not quite; but close) is The Forbidden Planet, a 1950s scifi. The basic idea without spoiling it is that enhancing their own intelligence also made powerful their lizard brain of sorts. I assume freudian psychoanalysis was big in the 50s so they called it the ID in the film.

</comment></comment></comment></comment><comment><USER17> The universe is so big, that I think it's very likely that somewhere out there are, there are many other intelligent civilisations, in distant galaxies–but so far away we'll never be able to visit them, never be able to communicate with them, we'll likely never even know of their existence (not in a million years–and I mean that literally.)

In the long-run (of millions of years), the two most likely outcomes for humanity are extinction or galactic colonisation. If we do colonise the galaxy, we'll prevent any other intelligent species from evolving within it (by fully occupying that ecological niche.) If we go extinct, we'll leave that niche open for another species to maybe one day occupy as we would have failed to do. The galaxy is likely a natural hard limit for our species, since while interstellar travel is difficult it is arguably possible (not with present day technology and economics, but potentially within a few centuries from now), but intergalactic travel is likely never going to be possible.

Even if these authors are right that other intelligent life once existed in this galaxy, it's almost surely extinct by now. If that's true, it makes their conclusion profoundly pessimistic – the more ruins of dead civilisations this galaxy contains, the greater the likelihood that we'll meet the same fate as they did.

I think we'll always be alone in the universe. And it's probably better that way.

<comment><USER18> Why do you believe it is surely extinct by now ? Why do you believe that colonisation of the galaxy means eliminating all other advanced civilisations and life form ? Are you russian ;)

Your reasonning is based on many assumptions. Look at us. When did we discover how to fly ? When did we landed on the moon ? Human evolution in science and technology mastering is evolving exponentially. Things can happen much faster than we thought.

Another question is why is the UFO phenomenon consistently discarded in the reasoning ?

<comment><USER17> I'm not suggesting our species will go on a genocidal rampage through the galaxy. Rather, I'm suggesting that if we manage to spread through the galaxy, we won't encounter any other intelligent life, and our galactic colonisation will prevent any other intelligent species from evolving in this galaxy since we'll be fully occupying that ecological niche, just as being the dominant species on this planet makes it much less likely that other species (such as chimpanzees) will evolve to our intelligence level. (Galactic colonisation might plausibly result in speciation of homo sapiens, however.) It's extremely unlikely for two intelligent species to evolve within a single galaxy at the same time; if other intelligent life ever exists in this galaxy, it is almost certainly many millions of years older or younger than us. Any civilisation following our present trajectory is likely to colonise the galaxy within a few million years. Since we are here, no species before us has successfully colonised this galaxy, which implies either we are the first, or the others have already gone extinct before reaching that level.

I discount UFOs because I believe the alternative explanations–such as natural atmospheric phenomena, secret testing of military aircraft, confusion, hallucination and delusion–are collectively much more probable than the hypothesis that extraterrestrials have visited us.

<comment><USER18> > I discount UFOs because I believe the alternative explanations–such as > natural atmospheric phenomena, secret testing of military aircraft, > confusion, hallucination and delusion–are collectively much more probable > than the hypothesis that extraterrestrials have visited us.

I understand but the reality is undetermined. So assuming that UFOs are not real or ET manifestation is taking a risk of error.

My knowledge of the UFO phenomenon is different as I'm very close to the author of the following articles [1]([http://www.meessen.net/AMeessen/Propulsion.pdf](http://www.meessen.net/AMeessen/Propulsion.pdf)) [2]([http://www.meessen.net/AMeessen/Evidence.pdf](http://www.meessen.net/AMeessen/Evidence.pdf)) [3]([http://www.meessen.net/AMeessen/Production.pdf](http://www.meessen.net/AMeessen/Production.pdf)).

While I agree that a priori we don't know if we can attribute any reality and seriousness to the UFO phenomenon, these studies shows that we should. The author applied the inductive process. If the UFO phenomenon is real and the manifestation of unconventional flying objects, they must respect the law of physics that we know. The author investigated the a priori uncertain data to determine if it is possible to explain what is described by using the law of physics that we know and are valid in our environment (air, water, etc.). The only liberty degree accepted is that since the UFOs may not be of human origin, they could use the laws in an original way not yet known to us.

The referenced articles describe the result of this study. The conclusion is that it is possible to explain the propulsion mechanism of UFOs and its described properties. The mechanism is not yet used by humans and require a supraconducting surface at room temperature and above.

This is a theory explaining the observed phenomenon. The normal scientific process should now be to test that theory. This is the work of an experimentalist, not a theorist.

Of course this doesn't tell us anything on the real origin of the phenomenon. But its theorical validity give credibility to it. Since many reports signal the presence of non human looking beings, this give credit to the ET origin. Finally, supraconductivity at room temperature is not yet mastered by humans. I suppose it would be known by now if it was the case.

Why do these ET behave as they do ? I have no idea. One thing to keep in mind is that the rules and optimal strategies may be different once space travel and energy is not a problem anymore. There may be no benefit in colonizing the galaxy. There is enough space for everybody. My conclusion, influenced by the results of the author is that what matters is to be able to ensure the survival of ones civilization. This means that scientific and technological advance is primordial and determinant as we can see with the war in Syria. We can also see that colonizing doesn't work unless the population is fully under control. This may give credit to the abduction side of the UFO phenomenon, but I'm much more careful on this subject because we have much less data and independent data sources.

</comment></comment></comment></comment><comment><USER19> > a probability for civilizations to form of one in 10 billion per planet was > considered highly pessimistic

I think people underestimate how low probabilities can really be. According to wiki the shortest self-replicating RNA is 165-bases long. Even if you convert all observable universe in RNA this is still not enough for this RNA to form by chance (4^165 is much larger than number of atoms in observable universe).

<comment><USER20> Given enough time anything can form out of random fluctuations if it has >0 probability no matter how small.

<comment><USER4> "Enough time" can easily be longer than the age of the universe. A million monkeys typing on keyboards will eventually write shakespeare. But it will be longer than the universe before they even type a single sentence of it correctly.

<comment><USER21> You are postulating the existence of only a single universe, ours. But there could be any number of universes, and so far we have no scientific way of testing that. The anthropic principle and the possibility of multiple universes puts all probabilistic arguments to dust: anything with non-zero probability happens in an infinite number of universes.

> A million monkeys typing on keyboards will eventually write shakespeare. But > it will be longer than the universe before they even type a single sentence > of it correctly.

All it takes is _one_ tiny selection force to blow this argument out of the water. For example, if only _a single_ correct letter is retained from each attempt, the problem gets exponentially easier with every iteration, making the expected time to generate any work linear in its length.

<comment><USER4> I agree with you completely about the anthropic principle. But we are talking about the probability of aliens. If the anthropic principle is true, then we should expect aliens to be really really unlikely.

</comment></comment></comment></comment></comment><comment><USER22> The most likely reason that nobody seems to be out there is the great filter. The author asks if our civilisation will last millenia or millions of years - but I think a more apt scale would be decades or centuries.

It just seems pretty obvious that we have an ever increasing set of opportunities to extinct ourselves, and therefore the probability of a civilisation ending disaster only grows.

All the aliens either blew themselves up, starved, or are the kind of incredibly boring, mind-numblingly conservative people you wouldn't invite to a local cluster cocktail party.

Oh, either that or civilisations tend to disappear up their own simhole.

<comment><USER23> Conversely, by 2025 there might be living humans on Mars (if Elon Musk gets his way). And while that's not enough to form the basis of a backup civilization, I'm somewhat more interested in the effect on global zeitgeist it's likely to have. The feeling of the times is, I feel, an underestimated factor in how democracies pick their priorities - and the early 21st century has been very down on itself trying to figure out practical, post-conflict economic realities.

<comment><USER24> > And while that's not enough to form the basis of a backup civilization

Or that might accelerate the end of civilization due to our inherent tribalism - the US isn't quite a backup of Europe as a 16th century European would have imagined. Who is to say there won't be a Planetary Mutually Assured Destruction?

 _The Expanse_ (books and TV series) explores some themes around planetary identity & politics in a fictional future where Mars and the outer solar system has been colonized and Earth is seen as a meddling 'old country' by some.

</comment></comment></comment><comment><USER25> It's true that so far every step of the way we have found environmental factors that are generally in favor of life. However, there are still many factors we don't know anything about. Somewhere between the possibility of one of these remaining factors being extremely unfavorable to life and us just being in a very isolated location, there should be an explanation for the Fermi paradox we're actually experiencing.

Sensationalist title aside, the article actually plays it extremely safe, by asserting a very high probability of extraterrestrial civilizations having been existed somewhere in the past.

But even so, there are potential problems where that might conflict with observation. I'm not generally a fan of these "alien civilizations must construct huge machines and must have a huge energy output" assertions, _however_ , all it would take is _one_ of the previous civilizations to have built _something_ we can recognize. Sadly, nothing has been found, neither artefacts such as probes, nor radiological evidence in the sky.

Now, there are many possible reasons for why we haven't found anything yet, first and foremost being that the galaxy is vast in both time and space. But if I forced myself to be especially pessimistic, I would say that the probability of abiogenesis might be way smaller than even our bleakest estimates.

</comment><comment><USER26> By pure probability, if we're the first intelligent species here on earth (that is able to build radio communicators) out of what, ~4 billion species that ever existed, what are the odds of us being the first in the entire universe?

My logic is that intelligence (as defined above) is clearly rare here on earth where life is prolific, hence by no means a certain development of evolution. I wouldn't be surprised if we're the first in the universe where the conditions are certainly less hospitable for life.

This game of probability doesn't only depend on the sample size, but also on what you're looking for. For instance, you can say that if you find a _purple_ grain of sand, and you know there are trillions of other grains, then you'll be sure to find other _purple_ grains. But what if the purple grain has written on it "hello how are you" by sheer chance. You'd probably never find another grain of sand with that same writing.

And then you can say, "us being the first is egocentric". Well, but we're the first (on earth) when it comes to building radio stations, so we clearly won that lottery big time. Is it such a stretch to say we won the lottery in the universe as well?

<comment><USER27> I'm not sure you can draw the conclusion that since there is one intelligent (as defined above) species, out of many on earth, that intelligence is unlikely to develop.

The main issue I see is that there were other species that were close to as intelligent us, but we out competed them and they went extinct.

I wouldn't be surprised if there was a sort of "first one wins" effect where the first species to reach a certain level of intelligence quickly (compared to the time it takes to evolve that level of intelligence) comes to dominate the planet wide eco-system on a level far beyond every other species, thus preventing the development of other intelligent species.

<comment><USER26> Even if we outcompeted the Neanderthals (or other very intelligent species), they were still very close to us in the tree of life (they were cousins). For a trait to be a high probability outcome of evolution it needs to happen independently in many different species - take binocular sight for instance which happens in all kinds of very far related species.

Even if our close relatives had similar intelligence and we outcompeted them, it's still safe to say that intelligence (as defined) appeared in only a small branch of the evolutionary tree.

Why don't we see species underwater building radio stations? There's lots of life diversity underwater and they certainly weren't in our "competing" ground (being underwater) so they could've developed intelligence independently... but didn't. That's my point.

<comment><USER27> One species having binocular sight does not significantly effect the evolutionary advantage of binocular sight in other species.

And define what you mean by "high probability", I would agree that intelligence (as defined as building radio communications) does not have a high probability of evolving compared to binocular sight.

What I am getting it, is I would not be surprised if intelligence (building radio communications) was like a race. A race only has one winner. Just because a race may have a thousand contestants, does not mean that the probability of there being a winner if 1/1000\. Rather the probability of there being a winner goes to 1 as the duration of the race increases.

Or its like concluding that because life only appeared once on this planet, its very unlikely that life could have appeared at all. Another hypothesis is that once life appears, it makes it very difficult for new life to appear by preventing the conditions that lead to abiogenesis. For example maybe chemical evolution can not occur, because in all places where it would the are living organisms that interrupt the process because those chemicals are useful to the organism.

Similarly, I Humans have an unprecedented degree of control over the evolutionary trajectories other organisms. Maybe other apes, if left alone for hundreds of millions of years would evolve into a species that develops radio communications. But organsim's need evolutionary pressure to evolve, and humans really, really like preserving species. If a species of apes start dying off due to some evolutionary pressure, rather than let nature take its course and let the apes which can survive that pressure out compete the others, humans will intervene to keep them around, and as a result they will remain stagnant evolutionary.

</comment></comment></comment></comment><comment><USER28> > Specifically, unless the probability for evolving a civilization on a > habitable-zone planet is less than one in 10 billion trillion, then we are > not the first.

And … nothing is provided to indicate that said probability is higher than that.

</comment><comment><USER29> I'll agree that aliens might exist far away from us in space &|time, but so what? I'd argue there's no practical difference between being alone in the universe and our local galaxies.

<comment><USER18> Why do you believe there is non difference ? Do you believe they didn't vist earth yet ?

</comment></comment><comment><USER30> One in 10^22 seems really easy odds to beat. Basically anything should happen more often than that, right? But if we choose a random 16 letters, the odds that they spell "unscientifically" is less than that. If we gave 16 letters to each of the 10^22 stars, we expect the word to happen on only one star, if even that. It's not too hard to argue that life is more difficult to get by random chance than the word "unscientifically", so we can't simply argue that "of course life will happen with those odds!"

</comment><comment><USER31> In estimating a probability for the occurrence of life in the universe, the authors essentially claim to know the probability of their own existence.

For me, that's a red flag. I can't know the probability of a phenomenon which my existence depends on just by observing my existence alone, because I can't observe all of the cases where the phenomenon does not occur and I do not exist.

<comment><USER32> The quantity of interest in any probability problem can always be re-cast as the conditional probability of the event given that the person asking the question exists, which is necessarily 1.

<comment><USER31> Pardon me, but I'm not sure what you mean by quantity of interest. If the quantity of interest can always be re-cast as a value which is necessarily 1, isn't "quantity of interest" the same for all probability problems?

<comment><USER32> Sorry, I wasn't very clear.

> the authors essentially claim to know the probability of their own existence

The "quantity of interest" is the probability of aliens, but not the probability of aliens in all possible worlds, but rather the probability of aliens in _this_ world. In _this_ world, the probability that the authors exist is 1.

<comment><USER31> I respectfully disagree. I claim the probability of human existence could have been less than 1, just as the probability of heads when tossing a fair coin is less than one, even when heads is observed as an outcome. We simply can't observe "tails" in this case.

<comment><USER32> You are confusing two (perhaps three) different things. The probability of a fair coin landing heads is 1/2 before the coin has been flipped. After the coin has been flipped, the probability is either 0 or 1. You may not know which it is, but it is definitely one or the other (assuming classical mechanics).

Now, if you are being a strict Bayesian then the probability of the coin being heads is never quite 0 or 1 because there is always the possibility that your eyes are deceiving you, or you are suffering from delusions or some equally unlikely contingency. But I assume that's not what you mean, and that you accept that humans do in fact exist in this universe, in which case the probability that humans exist in this universe is exactly 1. P(A|A)=1 by definition.

</comment></comment></comment></comment></comment></comment><comment><USER33> There have been aliens -- and they don't care about us.

To assume that a collection of beings which have obtained the ability to travel amongst the stars would focus their attention on a single species on a single planet is fundamentally selfish.

What resources does our earth contain that are not (more easily) obtainable elsewhere? If Drake's equation "holds", then what novelty do _we_ provide?

<comment><USER4> There are many reasons alien races would interact with primitives.

Morality - they might be utilitarian and desire to relieve the suffering of other sentient beings. Or force their weird alien morals onto them.

Competition - we might not be a threat now, but given a sufficiently long time, we could evolve into one. If they really don't care about us, it would cost them very little to exterminate us early.

<comment><USER34> I see these mentioned a lot, and sci-fi is big on the latter. The one I rarely see come up is "entertainment."

If we were traveling amongst the stars today, with our values and morality as they currently are, and happened upon a world of what we'd consider to be primitive life, we'd probably scoop some up and put them in a zoo, or broadcast them on TV. We'd want to study them, learn what makes them tick, maybe try to see if communication was possible, but in the end, we'd get entertainment value from them.

Even if we left them be and studied from a distance, the human race would be fascinated. New life! A new place! We like new things. The news would cover these creatures ("Are they a threat to humanity? News at 11."), they'd be woven into stories and shows and movies in one form or another. If we could safely bring them back, people would want to see them in person.

We'd get a lot of entertainment value out of these new creatures. We're a curious race who manages to get bored of spectacular things quickly, which is a large reason we continue to innovate and push forward.

There's no way to know for sure what would motive another intelligent race to advance to the point where they'd reach for the stars, but if any of them shared a degree of curiosity, I doubt they'd pass us up if they found us. We'd be something new for them.

Maybe we'd end up in their stories, their drawings, their videos, or even their zoos. Maybe they'd be interested in how we smell, or the way we dress and decorate ourselves, or the things that we watch on TV. Our languages, our fights, our culture. Or maybe just the way we go about our business, missing some big cosmological truth that's so obvious to them, as if we're adorable little children.

Maybe we'd provide just enough entertainment value to be worth a quick trip. Or a long stay.

<comment><USER23> If we discover intelligent alien civilization, there'll be a massive effort put into simply tapping into their culture. Their internet, TV, magazines, history, everything we can our hands on. The sheer volume of cross-cultural exchange which would be suddenly and immediately valuable would be enormous.

</comment></comment></comment></comment><comment><USER35> If you are interested in these issues a great book to read is called "Rare Earth:Why Complex Life is Uncommon in the Universe" by Peter Ward and Donald Brownlee. They make a convincing argument (to me anyway) that bacteria and simple life forms are probably common in the universe but animals and complex life with intelligence might be exceedingly rare. We, quite possibly, could be the only intelligent life in the universe.

</comment><comment><USER36> ITT, people start with the assumption live is something incredibly special. Another way of thinking assumes live emerges everywhere an energy gradient and some matter exists.

Maybe we are not special at all.

[https://www.quantamagazine.org/20140122-a-new-physics- theory...](https://www.quantamagazine.org/20140122-a-new-physics-theory-of- life/)

<comment><USER37> So where is everyone?

If intelligent life is at all likely, there ought to be many civilizations that have had millions of years technological head start on us. Given one species with galactic ambitions, but technology only modestly beyond ours, it would only take them one million years to colonize the entire galaxy. So why do we not see any evidence whatsoever?

This is the Fermi Paradox.

<comment><USER36> Well, maybe intelligent does not necessarily include expansive.

</comment></comment></comment><comment><USER38> Protip: if the greatest possible discovery in astrobiology is confirmed scientifically, you won't hear it first in a New York Times article whose title is "Yes, there have been aliens"...

</comment><comment><USER39> What I always don't get about this, is the omission of the probability of life emerging. We know it's possible but do we know how likely it is?

</comment><comment><USER40> About Drake's equation: [https://xkcd.com/384/](https://xkcd.com/384/)

</comment><comment><USER41> What these guys clearly omitting is that our planet could be the host of not only human civilization. At least in the past.

</comment><comment><USER42> No cookies, no article. Couldn't read.

Comments seem sufficient for me to realize I needn't have bothered.

</comment><comment><USER43> No they have not been any aliens until proven so...

Science requires a theory and experimental validation.

The existence of aliens has been postulated by a crazy monk that also invented the concept of monads. It is much more metaphysics (religious thinking) than physics.

It strikes me as odd that most of flawed science is coming with the same flaws as religious thinking.

<comment><USER44> > _Science requires a theory and experimental validation._

You'd be surprised. That's 19th century thinking about science, and not really how science works in practice most of the time.

<comment><USER43> Well, that is at least what made relativity and quantum mechanic successful.

And yes there has been a lot of flawed science in the XIXth I am aware of it ... like the theory of the ether that Einstein dispelled through experimentation, or the superiority of some "races", or the morpho- psychology...

</comment></comment></comment><comment><USER45> So you can publish papers by making up numbers?

</comment><comment><USER46> i love u guys. really great discourse. thank u.

</comment><comment><USER47> Title is mild clickbait. There's no evidence for the claim, just a persuasive argument that it's probably true.

Interesting nonetheless.

<comment><USER48> it's not mild clickbait. it's the definition of click bait. it states a very controversial conclusion, as if the argument is finally settled, while the body has little new information and fudges around with statistic that "it's likely probable that there is or once was another species".

<comment><USER49> Since this is published in a newspaper, the terms that are generally used are "headline" or "title," not "click bait."

<comment><USER50> Their media kit claims 78 million online viewers vs. 5 million _wealthy_ newspaper audience. I think more people click to read the article than turn a paper page to read it. Newspapers are online media companies now that happen to also produce limited paper versions.

Perhaps I'm missing your point?

</comment></comment></comment></comment><comment><USER51> I enjoy these quasi statistical arguments. Can anyone remind me what is the probability that God exists, or that we live in a computer simulation?

<comment><USER36> Did god invent humans or was it the other way round?

<comment><USER16> See Isaac Asimov's the Last Question for a recursive god human relationship

</comment></comment></comment><comment><USER52> m - km, fix the units</comment>


What a Student Loan 'Bubble' Bursting Might Look Like - <AUTHOR> https://www.vice.com/en_us/article/qvqw3x/what-a-student-loan-bubble-bursting-might-look-like <comment><USER3> Just like the housing crisis... when are we going to learn that government backed debt is a terrible thing for our nation? Creditors _should_ turn away bad lendees or cap loan limits, or increase your interest rate based on risk factors. Some exmaples of this might be:

* Limiting the amount of money you can be loaned for majors that don't have good job prospects. People should turn to grants for these types of educations

* Increasing your interest rate if you miss class

* Increasing your interest rate for poor grade performance

<comment><USER4> I don't think GPA has very much of any effect on lifetime earnings except by being a proxy for IQ and conscientiousness.

Same with missing class, they don't reduce someone's lifetime earnings, they just tell you how conscientious they are.

But extroversion is actually one of biggest contributors to lifetime earnings. Maybe if they miss a party we should increase their interest rate too.

<comment><USER3> Well the difference between a 3.0 and 4.4 isn't likely much, but a gpa of 1.5 would very much indicate failure to focus

<comment><USER4> If you control for IQ and conscientiousness then it's probably what classes they took, how motivated are they by school, whether they found a girlfriend they really liked that semester and decided to blow off class, founded a company, decided to teach themselves french.

<comment><USER3> I mean, exactly

</comment></comment></comment></comment></comment><comment><USER5> The fact that people can't ever discharge their loans makes it hard to imagine it "bursting" in the way that the housing market did in 2008. The whole notion of bursting is people collectively realizing their folly and everyone scrambling as quickly as possible to make out with whatever they've got left - like a run on a bank.

In this case there is no bank to run on - people will just continue to have their wages/benefits garnished to continue to repay these loans that they are forever tied to. I feel like there's not really a "burst" that can happen - just that hopefully there can be some sort of debt forgiveness program or something that will alleviate the burden of the debts.

<comment><USER6> > The fact that people can't ever discharge their loans

They can, it just has a higher bar of unaffordability before it can be discharged even in bankruptcy. OTOH, the higher the ratio of student loan debt to income becomes for typical borrowers, the more likely it is that there would be a wave of discharges.

Even without discharges, though, you can have an escalating problem of loans becoming worthless to those with the right to collect them; that the loan isn't legally discharged doesn't mean that the borrower isn't practically judgement-proof.

> The whole notion of bursting is people collectively realizing their folly > and everyone scrambling as quickly as possible to make out with whatever > they've got left - like a run on a bank.

No, not like a run on a bank, more like a panic sale of a marketable asset class, in this case student loan assets (or student loan asset backed securities, just like the mortgage-backed securities that went through the same thing around 2008.)

OTOH, this is really only likely to be a big issue with private, non- government-backed student loans, which exist and have high per-borrower balances, but are a minority of student loan debt.

On the federal side, increasing defaults mostly reduce the difference between loans and grants and probably push public policy away from generally-offered loans to more targeted (by some mix of need, individual merit, and social desirability of course of study) grants. (These also are marketed and privately collectable after the government originates them, but the government guarantee limits the risk of value crash.)

<comment><USER7> >On the federal side, increasing defaults mostly reduce the difference between loans and grants

Interesting tidbit here is that half (yes, half) of the US government's financial assets are student loans.

It was discussed here previously -- [https://news.ycombinator.com/item?id=16136330](https://news.ycombinator.com/item?id=16136330).

Something I find very hard to comprehend, tbh.

<comment><USER8> I believe this does not include most of the mortgages. For example, Fannie Mae has 3.35 trillion in assets, which dwarfs total student loans.

[https://en.wikipedia.org/wiki/Fannie_Mae](https://en.wikipedia.org/wiki/Fannie_Mae)

<comment><USER6> > I believe this does not include most of the mortgages. For example, Fannie > Mae has 3.35 trillion in assets, which dwarfs total student loans.

Fannie Mae is a publicly-traded, privately-owned corporation, despite being federally chartered for a public mission.

So, yes, their holdings are not counted in government holdings, because they aren't the government.

<comment><USER9> Fannie Mae and Freddie Mac are currently under the authority of the Treasury, since the TARP program.

Which is why, their stock is worth a lot less than even their own book value.

</comment></comment></comment></comment></comment></comment><comment><USER10> > Meanwhile, some people are more than happy to spend $100,000 on a graphic > design degree that may get them a job paying roughly $40,000 a year

> A lot of that has to do with what Goldfarb likes to call the narrative—one > that convinced kids that the only route to middle-class respectability was > to get a college degree and that suggested failing to do so meant having > failed at the American dream of upward mobility. To keep using the > economist's terms, I would also say that 18-year-old high school seniors > definitely qualify as "novice investors." Their philosophy can be summarized > by Colin Hanks in the 2002 movie Orange County. When asked why he's so > obsessed with getting into Stanford University and college more generally, > he snaps back, "Because that's what you do after high school!"

Too often I see articles about student debt make a comment about how students have been tricked or misled into pursuing a degree. Why does no one talk about the alternative?

Let's talk about the alternative: You don't get a degree and now you're resume gets automatically thrown out of most applications. Only a small fraction of those without degrees manage to get well-paying jobs; they're the outliers. Albeit unrelated to employment rate, look at the salary differences: [https://www.bls.gov/careeroutlook/2018/data-on- display/educa...](https://www.bls.gov/careeroutlook/2018/data-on- display/education-pays.htm)

Some might say it's worth making $X/yr and debt-free than making $Y/yr with $Z debt but this only holds true when Y > (P*Z + X) where P is the percentage of the total initial debt being paid off per year (e.g. P = 0.2 if you're paying $10k/yr to get rid of a $50k debt). And these values are hard to predict.

<comment><USER11> Or you could go into the "blue collar" trades like plumbing, electrical work, etc. After you get some experience, the trades can be a very lucrative line of work. Albeit you have to sweat and get your hands dirty. Mike Rowe has a few very persuasive thoughts on the subject as well, and can do it much more eloquently than I can.

<comment><USER12> It's also dooming yourself to a life of little mental stimulation and lots of repetition. I have no problem with physical labor (love house projects and hobbies), but these jobs would suck the life out of me for that reason.

<comment><USER13> My father was an electrician by trade, but generally very capable. He loved the problem solving aspect of his job. Wiring and code are not so far apart as you might think.

I've often felt, given the pedestrian nature of the problems I have to solve in code, he got the better end of the bargain.

</comment></comment></comment></comment><comment><USER4> Summary of what a student loan bubble bursting would be from the article.

The government would guarantee less student loans. This would be means some for profit institutions might shut down.

<comment><USER14> The govt shouldn't guarantee any student loans. My tax dollars shouldn't be an ATM for student tuition.

<comment><AUTHOR> > My tax dollars shouldn't be an ATM for student tuition.

In _theory_ tuition is a great way to spend tax dollars because it creates better educated citizens who will earn more money (and pay more taxes). In practice it does seem to be more like monopoly money that props up sub-par schools and provides even worse education to those who could benefit from it the most.

<comment><USER15> Depends on the theory. If you believe that college education is a positional good - that it allows people to be classified as middle-class and be prioritized for white collar work at the expense of non-college-educated competitors - then tax subsidies is the _last_ thing you'd want to do.

<comment><AUTHOR> Wow. That's a really good point... subsidies lead to overconsumption and overconsumption of education leads to "degree inflation" which leads to... even more overconsumption of education.

Yikes.

</comment></comment></comment></comment></comment><comment><USER16> If the loans are guaranteed by the government and people can't discharge them, then there is almost zero risk to the creditor. How do they justify charging interest higher than a government bond?

I can see why there are rumblings of loan forgiveness, as it has metastasized into something truly horrible. [https://en.wikipedia.org/wiki/Debt_bondage#Definition](https://en.wikipedia.org/wiki/Debt_bondage#Definition)

<comment><USER17> I think that profitability at all levels is the core of the issue with crazy student loan debt. Why do guidance counselors steer kids toward expensive schools they can only afford through huge impractical loans? Why do overpriced schools exist in the first place? I suspect because some institutions getting rich off loan interest use their influence to encourage this.

Mass exploitation of a population is best done when you can advise them to act in your best interest while convincing them its in their own, and that's best done from a position of authority.

If the health of society was a primary concern then the conflicting profit motive would have been legislated out of existence, so we could maximize the benefits of an educated population for the least cost.

</comment></comment><comment><USER18> Unrelated to the topic of the student debt bubble, the opening paragraph left a bad taste in my mouth:

> I hated the film adaptation of The Big Short. The acting was good and it did > have a surprising amount of energy for a story that centers around men in > suits doing math. But I felt condescended to. Maybe director Adam McKay > tested an original cut—one minus Margot Robbie—and realized a good amount of > the audience couldn't follow the plot without having a sexy lady in a > bathtub break the fourth wall to explain subprime mortgages and the > financial crisis.

Sounds awfully pompous, like it came from r/iamverysmart. Much of the impetus behind The Big Short certainly was to break down the 2008 financial crisis to those that aren't economics/finance wonks, or for whom a narrative holds more weight than a several-hundred page non-fiction book about it.

It's not that I want journalists to coddle their audience, I just think it's a bad look to completely dump on other creators to inflate one's ego while trying to make a point.

How about this, which still includes a bit of criticism of the director (but milder):

"While the film adaptation of The Big Short had good acting and a surprising amount of energy for a story that centers around men in suits doing math, it wasn't exactly my cup of tea. Perhaps it's because I felt Adam McKay didn't trust his audience quite enough to understand the ins-and-outs of complex subprime mortgages and the financial crisis, instead falling back on gimmicks like having Margot Robbie in a bathtub break the fourth wall to explain it all.

"In some ways, America's student-debt crisis is a lot simpler than all that."

I don't know, maybe that's not any better: and maybe by criticizing the author I've become the thing I set out to...criticize. Whatever.

</comment><comment><USER19> At it's core, this is an economic problem. When there's an "endless" supply of money, prices go up.

Combined with the newfound belief that, you must, and in fact, are entitled to go to college, "demand" goes up too.

The result is students racking up needless debt when they probably shouldn't have gone to college to begin with. College is NOT the only option to build a successful career, but in America, with no degree, you're seemingly sub-par in the eyes of our backwards society.

So, who wins? All these "Not-for-profit Universities" whose profit margins (non-profit, doesn't mean that aren't profit-ABLE) are probably better than many Fortune 500 companies.

The solution is to dry up the source of loan money, reduce the pumped up demand for bogus degrees kids are getting and encourage young people to consider other career options. You don't need a piece of paper to be successful.

<comment><USER20> There was an article over the weekend about MIT and that 58% of their students receiving financial aid. And how the tuition has gone up over 1000% in the last 20 years. And that MIT is purposly setting the tuition so high because it enables them to be a tax exempt charity based on that 58%. Point is that it is in the school's best interest to make it unaffordable for most of the students that go there. More loan money is available, so the tuition goes up to keep it unaffordable for most of the students.

</comment></comment><comment><USER21> I cannot fathom what it must be like having near $100K in college debt. I dropped out in the middle of my 3rd year from a state college that only charged $279/credit hour, and a lot of my credits were already transferred from high school.

Putting people into massive amounts of debt in exchange for a shot at a prosperous future puts a lot of pressure on them to rake in the dough, which I fear is going to corrode the morality of those going into high-paying institutionalized positions. In effect you are looking at the upper-middle- classes of the economy starting out with a big incentive to stratify because everything they have worked for rides on being able to pay off those loans - <USER2> toxic praxis.

On the other hand, if you are a policymaker for a nation's central bank, then this might be your strategy for ensuring your workforce produces measurable value.

</comment><comment><USER22> While we may not see a big burst, it's possible that a generation of defaulting borrowers will gain steam. As more and more people justify and publicize non-payment, it may become politically untenable to collect on those loans.

Imagine a collective movement of 9,000,000 all ceasing to pay their student loans at once. If I thought all my friends weren't planning on repaying, I would be quick to join them. Is the government really going to dig thousands of dollars from the bank accounts of millions of people at once?

</comment><comment><USER23> I don't see this here, but could a student debt bubble result not in a wave of defaults but as a contributing factor in a market slowdown or recession because the average borrower will have less disposable income to spend on other goods and products due to the insane debt payments? A wave of defaults most likely won't hurt any major institution except the federal government, but the lack of consumer spending seems like a potential problem for our consumeristic economy.

<comment><USER24> I've had similar conclusions when considdering this as well. These loans are not wiped out by bankruptcy so they will be a permanent drain to personal wealth indefinitely. I think we will see lasting effects as more and more young people are unable to invest, save for retirement, and buy real estate but there won't be a sudden shock to the economy like the mortgage bubble.

<comment><USER23> I think we are seeing this drain already occuring actually. The popular news sources are all reporting that millenials are struggling to buy homes and build savings, though this may be due to low wages rather than debt servicing costs

</comment></comment></comment><comment><USER25> I'm thinking the solution will be political, not economic.

Once there are enough people in enough pain that it turns them into single- issue voters on this issue, someone will create taxpayer-funded relief.

If it's a Democratic-led solution, it'll probably be in the form of massively expanded support for universities that eliminates the cost entirely, or reduces it to what one might be able to reasonably earn/save at a minimum wage job.

If it's a Republican-led solution, it'll probably be in the form of creating a tax-deferred savings account for college, combined with subsidized work-study program.

Both parties will probably agree to bail out degree holders who can't pay after a certain number of years. They will probably also force cost cutting reforms on schools.

<comment><USER26> Debt should be forgiven, not relieved. We should not reward the debtors for constructing this byzantine nightmare.

</comment></comment><comment><USER27> I strongly believe that lower cost vocational education with group housing for 18-23 year olds is part of a solution to this problem.

You don't need the full college experience, but as the article mentions, getting away from home is a part of it for most.

<comment><USER28> > I strongly believe that lower cost vocational education with group housing > for 18-23 year olds is part of a solution to this problem.

In other words, the Army.

<comment><USER17> I don't think a bigger military is what the US needs, unless we were simultaneously shrinking the private sector money fire that the DoD provides fuel for. On an individual level, freedom-impeding loan debt is still less of a personal cost than what army recruits pay.

</comment></comment></comment><comment><USER29> I have $9k in student loans that bankruptcy wouldn't set aside for a degree I can't use (nothing to do with the degree, the defect is I) and Navient, Betsy DeVos' company, can go straight to hell.

</comment><comment><USER30> Aggregate student loan repayment should be capped as a percentage of yearly income. This should have been included as part of the same legislation that made student loans non-dischargable by bankruptcy. If you make 100k / year, and the rate is set at 10% (random number), 10k is the maximum amount you pay on your loans, regardless of the total amount you loans you have accrued.

Currently, there is no downward pressure on either universities or banking institutions for tuition rates or amount of money lent in student aid, because the government guarantees repayment. As a result, tuition rises every year far faster than inflation, and banks are happy to join in the reaping of the profits. Capping repayment rates would correct both of those issues, as the ROI goes down significantly over a certain total amount lent, if indeed the sum total can be repaid within a lifetime.

This would additionally ensure that while student loans are available to all students, all parties involved (student, bank, university) have an interest in ensuring the student will be able to pay back the loans, as opposed to now, where two parties have an interest in loaning as much as possible, and one child has conflicting interests.

This would additionally force some accountability on universities for their programs. For-Profit institutions, which currently largely exist to gather tuition money while providing no real educational value would suddenly become unprofitable, and collapse. Simply put: if your graduating students make little money after graduation, their rate of repayment will be significantly smaller than an institution that provides actionable skills.

Similarly, this would incentivize universities to ensure that all of their students are pursuing reasonable / coherent career paths. Currently, universities have the dual incentives to ensure that their students that go on to significant success and high paying industries do so (alumni donations), but also to collect as much tuition/room/board etc as possible from their existing students. This change would negate impact of the this second option.

This would allow students who go to expensive schools to go into lower paying professions. This would allow students with the best and most expensive education to pursue all types of jobs (for example, teaching), not just the jobs best fit to pay down their debt.

This would cap how punitive the accrued debt can be to the graduating student, and resolve the original issue mentioned in the article: student debt is now so overwhelming, it is preventing a generation from entering financial adulthood.

And lastly, this is just a guess and could be wrong, but I would paradoxically expect that in addition to all of the above changes, capping repayment rates would also result in lower amounts of student debt willingly taken on by students. This is for the simple reason that it is easy to rationalize "I'll be able to pay back {Insert_Any_Abstract_Dollar_Figure_Here} once I graduate and get my dream job", but everyone intrinsically would understand: "If I take out this loan, I'll be paying 10% of my income for the next 30 years"

<comment><USER31> This exists—income based repayment plans. There are different types [1], but basically you pay a maximum 10% or 15% of your income, and the remaining balance is discharged after 20 or 25 years. AFAIK, it's open to everyone—but you have to apply. It would probably help a lot of people avoid default if everyone was auto-enrolled in IBR.

Also, there's the public service loan forgiveness program (10 yrs public service job -> discharged loan balance).

[1] [https://studentaid.ed.gov/sa/repay- loans/understand/plans](https://studentaid.ed.gov/sa/repay- loans/understand/plans)

<comment><USER30> Correct, unfortunately, this is only for federal loans. I'm advocating that this program should be expanded to also apply to private loans, since those are also not dischargable by bankruptcy.

</comment></comment></comment><comment><USER32> Infinitely scary.</comment>


Ruby Security Have You Not - <AUTHOR> https://hakiri.io/blog/ruby-security-have-you-not <comment><USER2> Hey!

I'm one of the maintainers behind the Ruby advisory database: [https://github.com/rubysec/ruby-advisory-db](https://github.com/rubysec/ruby- advisory-db)

We're trying to build a common database for people building these tools; at the moment, trawling through CVE disclosures and various mailing lists is a largely manual process that we can reap economies of scale by pooling our efforts together.

It's free and volunteer run. I would like to encourage you and anyone else reading this who is interested in bolstering the security ecosystem to consider using and contributing to the advisory database.

It's the database that powers [http://github.com/rubysec/bundler- audit](http://github.com/rubysec/bundler-audit) and (disclaimer: I made this) [https://gemcanary.com](https://gemcanary.com) and the more people contributing the more we can all benefit from improving the ways we can notify end users of their vulnerable dependencies.

Thanks,

<comment><AUTHOR> Do you know of any apps like [https://hakiri.io](https://hakiri.io) that automate security scanning and serve as a security monitor for any future disclosures?

Also, how would one use bundler-audit if a Rails app is not actively maintained?

<comment><USER2> Yeah, like I mentioned above the app I built ([https://gemcanary.com](https://gemcanary.com)) automates disclosure notification, and I believe? [https://codeclimate.com/](https://codeclimate.com/) may use the advisory database as well, but I do not know for a fact.

There's a few others but they've slipped my mind at the moment.

I use bundler-audit when performing security audits for clients. You could wrap it in a git commit hook and have it publish results, but to my knowledge no one's done it that way.

</comment></comment></comment><comment><USER3> I can't help but feel that this could also be written as "How secure are Ruby and Rails apps? About as secure as all other apps."

In my experience, vulnerabilities are pretty much endemic to software that has not been hardened by experts.

<comment><USER4> I think the big thing here that threw me for a loop is just how many gems are pulled in by projects. 133 seems like a ridiculous amount of dependencies for a single project. Our entire company runs on Python in the backend and we don't even have 50 (and that's including ones I use for command-line utilities or corncobs, for example, which aren't in our public-facing software).

Even if the Python modules we have had the same number of vulnerabilities as gems, our exposure for a given system would be 1/10th, just from not using so many external libraries. Couple that with the fact that many of those gems are outside of end-user control, of generally unknown quality, might only be pulled in for one small piece of functionality, and might themselves depend on other gems which you weren't necessarily aware of and haven't necessarily vetted.

<comment><USER5> It would be interesting to know why this is. Are each of these dependencies really small on average? Or is there some big cascade of dependencies? Or is there a particular framework that's to blame?

I've had a similar issue when using the Spring Framework in Java, where including it in your project at all brings a whole host of other (often not very good) libraries in. This is a problem in Java, which doesn't have any built in capacity for loading multiple versions of the same library.

In Python, I suspect that I've been in a similar situation when using things like Pyproj, Numpy or Scipy, because those all depend on components written in C or Fortran. Perhaps next time I use any of those libraries I shall take some time to read more of the code and try to trace out what things they may be brining in.

<comment><USER6> > Are each of these dependencies really small on average? Or is there some big > cascade of dependencies? Or is there a particular framework that's to blame?

All of that, really. For better or worse, scores of gems have dependencies. Rails and a few popular others add large dependencies that cascade to other dependencies at that — some large, some not. It's extremely rare to run into quality gems with very few if any dependencies. (Sequel comes to mind, and that's just about the only 1st-class gem I can think of in pure Ruby.)

</comment></comment></comment></comment><comment><USER7> This doesn't surprise me, and I'm not even an experienced Rails developer.

I think a big part of the problem is that there are a ton of gems that are simply hobby projects that gained traction and became popular. They were originally architected by enthusiasts, rather than experts. Some of these projects may also have been abandoned by their authors, but still in use because they may be the only way to accomplish a complex task or integration. You can't expect stuff like that to be very secure. You just have to think carefully about whether the risks of using them are worth the gains made from not having to implement the functionality yourself.

<comment><USER8> This can be said of any open source library, but it is unfortunately true that the average Ruby gem seems to be more hobby-grade than enterprise-grade.

The fact of the matter is, if you are developing an application and are integrating 3rd-party code, always pay attention to what that 3rd-party code is, how well-maintained it is, and how the project has responded to bugs and vulnerabilities in the past.

<comment><USER9> Yup. I've done enterprise Rails consulting, which is almost a contradiction in terms. But it's possible and I've brought fixes like net ldap to work with AD.

Edit: Can't take all the credit, client hacked together the first version but we managed to extract it for "the greater good." Also fun stuff like XML-RPC (Xen API -> gem xenapi, VMware (gem rbvmomi)), which wasn't bad and worked OOTB. Wished MS exposed their APIs as RESTful endpoints, because WinRM + gem winrm just doesn't cut it with some products... generated powershell run by an agent instead. For some products, even having the (.someextiforgot) files that describe the API, there's no MS docs on them, so lots of trial-and-error in PowerGUI to find the right objects and methods (Yuck).

<comment><USER8> You sir are a saint. Dealing with net_ldap and AD was the bane of my existence for a while.

<comment><USER9> Thanks. I'm the 😈, really. }:) <\-- just in case you're using an ascii browser.

[https://github.com/steakknife/ruby-net- ldap](https://github.com/steakknife/ruby-net-ldap)

Shameless self-promotion: Clients call for this devil if something's hard or something's broken.

</comment></comment></comment></comment></comment><comment><USER9> Generally: don't trust anything from the outside world or anything that can transit untrusted infrastructure, that means check types and sanitize values before passing along. Break loudly and quickly to get attention for a fix. Keep the codebase as tiny as possible too.

Ruby: recompile with minimized OpenSSL 1.0.1+ (LibreSSL when possible) and with patches that improve Ruby's default OpenSSL security.

[https://gist.github.com/steakknife/8228264](https://gist.github.com/steakknife/8228264)

[https://gist.github.com/steakknife/10092587](https://gist.github.com/steakknife/10092587)

[https://gist.github.com/steakknife/10096008](https://gist.github.com/steakknife/10096008)

For Rails apps: use brakeman as one part of security audit strategy

For gem authors, sign them (please!): I wrote waxseal to make it dead simple

        [sudo] gem cert --add <(curl -L https://gist.github.com/steakknife/5333881/raw/gem-public_cert.pem) # adds my cert (do once)
        [sudo] gem install waxseal --trust-policy HighSecurity

For gem users, find which aren't signed

        Add this to ~/.gemrc gem line:

        --trust-policy MediumSecurity

        or just if there's no gem: .... already:

        gem: --trust-policy MediumSecurity

For anyone using git, sign your tags (git tag -s ...) and commits (git commit -S ...) por favor

<comment><USER10> For what it's worth, --trust-policy has been broken since RubyGems 2.0.0 and will not be fixed until 2.3: [https://github.com/rubygems/rubygems/issues/859](https://github.com/rubygems/rubygems/issues/859)

</comment></comment><comment><USER11> More alarming than vulnerable gems is the the number oh high severity vulnerability found in Rails itself: [http://www.cvedetails.com/product/22568/Rubyonrails-Ruby- On-...](http://www.cvedetails.com/product/22568/Rubyonrails-Ruby-On- Rails.html?vendor_id=12043). Compare this to Django ([http://www.cvedetails.com/vulnerability- list/vendor_id-10199...](http://www.cvedetails.com/vulnerability- list/vendor_id-10199/product_id-18211/year-2013/Djangoproject-Django.html)) per year.

<comment><USER12> number of reported vulns != number of existing vulns

</comment></comment><comment><USER13> The graphs of Gem distribution and Vulnerability distribution are utter nonsense. You can't just measure your mean and standard deviation and expect a bell curve that fits those two parameters to correspond to your distribution. _Especially_ if you cut off one of the tails.

<comment><AUTHOR> Do you mind sharing how you would present this data?

<comment><USER14> A histogram.

<comment><AUTHOR> Why do you think it would be better/more meaningful?

<comment><USER15> The current graph is actively misleading because it misrepresents the data as following a (truncated) normal distribution when that's very unlikely to be the case. A histogram would show the actual distribution.

The kind of graph being used here -- a "best fit" bell curve based on the sample mean/s.d. -- can be useful when overlaid on a histogram, to illustrate how closely the data can be approximated by a normal distribution. Without that context, it's essentially meaningless.

</comment></comment></comment></comment></comment><comment><USER16> I am not a ruby developer, but it does seem shocking that 66% of all gemfiles contain a known previously-disclosed 5+ exploit. Yes, most software is flawed, but you would think once it's been reported, it would be fixed.

<comment><USER17> The gems have probably been updated by the developers of the gem.

The data used in the article contains a list of the versions being used in production by real app, not the latest versions released by the gem developers.

<comment><USER18> Risk also depends on how you're using the gem.

If it's a bug in a YAML parser but you're not loading YAML from untrusted sources, then it would be a false positive.

</comment></comment></comment><comment><USER19> It's possible some of these are false positives, at least with regards to Rails, as most of the recent vulnerabilities I can remember come with workarounds and patch files. Some people need to stay on a specific Rails version, but may have fixed the problem.

I'm not ready to speculate which is more probable.

<comment><AUTHOR> Yep, totally agree, that's why I added a post scriptum :)

<comment><USER19> Ah! Missed it! My bad.

</comment></comment></comment><comment><USER20> Hi there I'm the man behind the codesake-dawn security scanner for ruby code. It will be great having some of you comparing the results obtainw with haikiri or other scanner and mine. Just for sake to reach an enterprise level tool.

Regard firstname.lastname@example.com

</comment><comment><USER21> This article kind of confuses me - what does it mean for a gem do be 'secure' ? The idea of many gems being 'secure' or 'non-secure' is a non-sequitur. Obviously for some projects, like Rails, it's fairly clear what is meant when someone talks about a vulnerability - that you can hacked somehow. However, if someone wrote a gem to wrap eval into a command line tool so they can use some random ruby commands or libraries from their shell, well, secure would be a non-sequitur, but also beside the point.

</comment><comment><USER22> Unpatched Rails apps are found vulnerable until patched... news at 11.

<comment><AUTHOR> I think the interesting part is how many unpatched, hence vulnerable, apps there are.

<comment><USER22> But this article does not analyze that. It's only scanning CVE's and most have already been patched when the CVE is issued. Of the 53 mentioned for Rails for instance, all have been patched.

This article is incredibly misleading.

<comment><AUTHOR> I don't understand... The article first analyzes _apps_ that use a certain number of vulnerable gems. Sure, most of them are probably patched, but it doesn't change the fact that certain apps use old unpatched versions of these gems.

The second part focuses on historical vulnerability data for Ruby gems.

In what way any of it is misleading?

<comment><USER22> > Sure, most of them are probably patched, but it doesn't change the fact that > certain apps use old unpatched versions of these gems

Why is that news?? That is true across every and all software platforms.

> In what way any of it is misleading?

Because the article assumes or implies the conclusion that there are a huge number of vulnerable Rails apps. The only way to discover that is to not simply count how many CVEs have been issued for each gem, but how many systems in production have not been patched.

From the comments here, many people seem to assume this is an analysis of the number of unpatched vulnerable systems in production, which it most definitely is not.

<comment><AUTHOR> > Why is that news??

It's not exactly news but it's nice to see it broken down in a cohesive way.

> Because the article assumes or implies the conclusion that there are a huge > number of vulnerable Rails apps.

I think the article quite clearly indicates where all data points came from (Hakiri Facets) and then analyzes them. There is a post scriptum at the end that encourages readers to not evaluate their or any other projects based on the number of CVE vulnerabilities and goes into detail why it's not always a great idea.

</comment></comment></comment></comment></comment></comment><comment><USER23> This doesn't surprise me in the least, but the increase in 2013 is obviously due to increased security scrutiny, which is good so long as people act on that information.

As a side note, I'd be interested to see a similar analysis of popular Java projects.

</comment><comment><USER24> It's neat (if not alarming) to see security metrics visualized like this. Nice work!

</comment><comment><USER25> STOP LYING! Everyone knows the only security issues nowadays are caused by developing in PHP, that's why we bash it every day, right guys?!

</comment><comment><USER26> Nice graphs. Meaningless, but nice.</comment>