1.5 TB of Dark Net Market scrapes - <AUTHOR> http://www.gwern.net/Black-market%20archives <comment><USER2> I've had a very slow-moving hobby project to parse and analyze a subset of this data: [https://github.com/rcompton/black-market-recommender- systems](https://github.com/rcompton/black-market-recommender-systems)

So far I've had some ok results along the lines of "91.7% of vendors who sold speed and MDMA also sold ecstasy" [http://ryancompton.net/2015/03/24/darknet- market-basket-anal...](http://ryancompton.net/2015/03/24/darknet-market- basket-analysis/) I am working on extending this to markets besides evolution now.

<comment><USER3> Just FYI, MDMA is ecstasy.

<comment><USER4> I don't do MDMA personally but I have some friends who like going to raves with some chemical supplies. On the streets, some make it a point of honor to sell/buy only "MDMA" because "ecstasy" is "crap".

The only precise name is MDMA. In my opinion the name users give to what they're ingesting is inconsequential. Most users have no idea what MDMA means anyway, nor would they know how to chemically identify if what they bought _might contain_ MDMA.

<comment><USER5>
        The only precise name is MDMA.

There's also "Molly," which is the street name for pure MDMA. (Though: most terms are tied to a specific geography.)

        ...nor would they know how to chemically identify if what they
        bought might contain MDMA.

True, but that's not unique to MDMA users. cf. the heroin in the Netherlands that was being sold as cocaine, and which lead to several deaths.

Organizations such as Dance Safe have existed for decades (in the U.S. at least) to let people test their drugs to make sure they're ingesting what they intend to ingest.

<comment><USER6> "Molly" is the US term for what should be (but usually isn't - very often some mixture of random RCs methylone, mdvp and other alphabet drugs) MDMA.

The UK term is "Mandy" "mud" or MD.

Ecstacy (which I have never ever heard used) refers to what should be MDMA in pill form along with some binders and is usually referred to in the UK as "pills" or "Es". I believe the US prefers the terms "rolls"

>the heroin in the Netherlands that was being sold as cocaine, and which lead to several deaths. How on earth can you buy coke and get heroin? No one buys coke and then rakes up a monster line, everyone does a dab test (lick finger, stick it in the powder and taste it)

Heroin tastes nothing like coke. Heroin sells for more so why mix it in?

</comment></comment></comment></comment></comment><comment><USER7> Gwern has a Patreon page now for anyone interested in supporting his research: [https://www.patreon.com/gwern?ty=h](https://www.patreon.com/gwern?ty=h)

<comment><USER8> Gwern does more interesting things with lower monetary burn rate than most anyone I've met.

</comment></comment><comment><USER9> gwern, you are an absolute force of nature when it comes to generating and collecting and presenting information in a useful way. Thank you.

<comment><USER10> I hope a he gets some good donations from HN, Ill send him a few dollars in BTC.

</comment></comment><comment><USER11> What amazing work! I am very interested in doing research with Tor and a dataset like this could make my job a heck of a lot easier. I have a legal question though: Are your scrapes text only? Before I work with this dataset, I want to make sure that there's no possibility it contains illegal images (child porn).

<comment><USER12> How about ascii art?

Actually, this is an interesting topic. Poisoning a dataset. CP would work for private security investigators, and to poison against government investigators you could use leaked classified secrets.

Could you work around this by operating on the files on VPS you don't own, streaming a very low-res ('Basilisk'-proof - [https://en.wikipedia.org/wiki/BLIT_(short_story)](https://en.wikipedia.org/wiki/BLIT_\(short_story\)) ) remote desktop image.

<comment><USER11> Possession laws are pretty strict and hard to decode. I wouldn't want to be the test case in court. The idea of "poisoning" a dataset is an interesting theoretical. But in practice, I just want to judge the likelihood that the dataset is poisoned by the presence of images. If it is then there's not much I can do with it.

<comment><USER13> Yes, this absolutely needs to be clarified by Gwern. This is a very dangerous thing to link researchers to if it contains any illegal content.

<comment><USER14> Um.. It says Black Market right on the tin.

<comment><USER15> Indeed, a warning that it may contain illegal content would be about as sensible as the standard "Warning: may contain nuts" label on a tin of nuts.

<comment><USER16> What is illegal in a download of an online drug marketplace? Are pictures of drugs banned where you live?

<comment><USER15> It's a general black market, not just drugs. For example, one of the sites described on that page is PEDOFUNDING, "A crowdfunding site for child pornography." Now the dump isn't supposed to contain any images, but it's hard to be 100% sure. In any case, whatever risk there might be seems to be clearly implied in the name and description there.

<comment><USER16> FreeeOW, that's what I get for skimming the list I guess.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER17> This is why I love the internet. This article has given me a fascinating glimpse into a world I have no idea about.

Author: thank you so much for taking the time to document this.

<comment><USER18> Unrelated to the darknet, but this Twitter account evoked a very similar feeling in me (random snippets from userboards in the 80's and early 90's):

[https://twitter.com/wwwtxt](https://twitter.com/wwwtxt)

<comment><USER5> "I heard about the Apple Watch recently and was going to check it out—but not now. It can't even transmit or input data. ☯93JAN"

Prescient.

<comment><USER19> Some of them a bit less clairvoyant about the future:

"The actual date for the end of the world is July 5, 1998. ☯92NOV"

It is interesting how many were accurate or onto something. I just didn't expect it.

edit: Found this gem after wasting more time that I cared to.

"I think the future of personal communications holds great things in store for us, but privacy won't be one those things. ☯94AUG"

<comment><USER20> July 5, 1998 is "X-Day", from the Church of Subgenius. It actually happened.

[https://en.wikipedia.org/wiki/X-Day_%28Church_of_the_SubGeni...](https://en.wikipedia.org/wiki/X-Day_%28Church_of_the_SubGenius%29)

<comment><USER19> Some of the quotes are from before I was even born so that is just one of the references I don't know and just assumed it was a date thrown out for no reason.

</comment></comment></comment></comment></comment></comment><comment><USER21> This is so cool. Thanks gwern.

If someone's feeling bored, you're welcome to put the entire archive on a web server for us to look at......

Or maybe I'll just do it.

<comment><USER22> You mean like this? [https://archive.org/download/dnmarchives](https://archive.org/download/dnmarchives)

<comment><AUTHOR> I don't think that works. It's not remotely browsable or searchable. It would be quite challenging to put these scrapes up, anyway. They're regular wget crawls with a regular directory/file structure, the problem is that there's so much material and so many files that it can be almost impossible to find what you are looking for... (Plus you need to rewrite links into relative links to make everything render properly.)

<comment><USER21> Hmm. Now I'm thinking that I might end up using your idea (scraping the dark web) and using something like httrack[0] to do exactly that: structure.

[0] [https://en.wikipedia.org/wiki/HTTrack](https://en.wikipedia.org/wiki/HTTrack)

<comment><AUTHOR> I once tried using HTTrack, but I found it was doing too much magic under the hood and was hard to work with. As dumb as wget is (that blacklist bug is over 12 years old now!), it at least is understandable.

<comment><USER21> Thanks for saving me the headache :)

</comment></comment></comment></comment></comment></comment><comment><USER23> What is the legality with respect to downloading this file? Could it contain material that would put us at legal risk?

<comment><USER24> I really don't know the answer. But I just want to remark that it seems to be a terrible situation that more than one of us have to wonder about the legality of downloading the dataset (there was a comment thread below on the same topic)

</comment></comment><comment><USER25> >collating and creating these scrapes has absorbed an enormous amount of my time & energy due to the need to solve CAPTCHAs,...

Have you considered automating this?

<comment><USER26> The whole point of CAPTCHAs is to be difficult to automate. Or are you suggesting automating farming out the CAPTCHA solutions to cheap workers?

<comment><USER25> There's a plenty of service providers that sell APIs to captcha solving services at reasonable prices.

<comment><USER27> Or if you don't want to spend money, you can always re-host things on your own site. Let your visitors do the work for you.

</comment></comment></comment></comment><comment><USER28> "HOW TO CRAWL MARKETS" section has good tips for general crawling as well.

</comment><comment><USER29> you could also use a library to handle captchas. have a look at 'tesseract'</comment>


Power outage stops game at Super Bowl - <AUTHOR> http://www.usnews.com/news/sports/articles/2013/02/03/lights-out-power-outage-stops-game-at-super-bowl

<comment><USER2> America spends trillions overseas on unjust wars while the infrastructure at home crumbles around us. Unbelievable.

<comment><USER3> only half the lights were out inside the stadium and the city didn't lose power so it's pretty safe to assume it's a problem in the stadium's lighting equipment and not the city's power grid.

<comment><USER2> well, I'm still a cynical bastard and I don't care. Just letting some frustration out and trying to make it a political issue.

</comment></comment></comment><comment><AUTHOR> More details: [http://sports.yahoo.com/blogs/nfl-shutdown-corner/power- outa...](http://sports.yahoo.com/blogs/nfl-shutdown-corner/power-outage-super- bowl-adds-unexpected-element-big-014914888--nfl.html)

</comment><comment><USER4> Second joke: Told them not to host their lights on AWS!

</comment><comment><USER5> I'm just waiting for "Anonymous" to say they did it...

</comment><comment><AUTHOR> Game just resumed after a 35 minute delay.

</comment><comment><USER4> Must have been built on Rails

<comment><USER4> Yeah, I didn't see this one going over well.

</comment></comment><comment><USER6> most people on twitter were commenting how this is going to trigger a hundred op-eds on crumbling infrastructure.</comment>


Zalando "fastest European company ever to reach €1 billion in sales" - <AUTHOR> http://venturevillage.eu/zalando-hits-e1bn-breaks-even <comment><USER2> Its a rocket company. Probably spent 2 billion acquiring those customers.

<comment><USER2> ah, there you go: "while the company's EBIT margin (a measure of profitability) improved from -12 per cent of sales in 2011 to -8 per cent in 2012, operating losses as a lump sum still increased from €60m in 2011 to €90m last year"</comment></comment>


Ways to Organize Books at Home, According to a Librarian - <AUTHOR> https://www.apartmenttherapy.com/book-organizing-tips-librarian-36721739 <comment><USER2> I like this. Tsundoku can be a true challenge at times.</comment>


Megaupload Readies for Comeback, Code 90% Done - <AUTHOR> http://torrentfreak.com/megaupload-readies-for-comeback-code-90-done-120923/

<comment><USER2> Unfortunately for Kim, finishing that last 10% usually takes 90% of the time.

<comment><USER3> I rarely experienced that with megaupload. I won't say it never happened though.

<comment><USER4> That's a programmers' saying about projects completion, not a comment on data transfers.

<comment><USER5> <http://en.wikipedia.org/wiki/Ninety-ninety_rule>

</comment></comment></comment></comment><comment><USER6> Who would be dumb enough to use it, given Kim Dotcom's criminal record? The man has been convicted of receiving stolen goods, insider trading, and embezzlement. If he's not convicted in the Magaupload case, it will be ENTIRELY due to procedural errors on the part of the prosecution.

The only times he appears to have not been engaged in criminal activity is when he was on parole waiting for a sentence to run out.

Why would you trust him with your data? Or if you are an investor, with your money?

<comment><USER7> _If he's not convicted in the Magaupload case, it will be ENTIRELY due to procedural errors on the part of the prosecution._

That, and the fact that it isn't entirely clear how enabling copyright infringement is anything other than a civil matter.

<comment><USER8> Pretty clear, actually: the law in the US, in the UK, and in New Zealand says there's criminal copyright infringement. The difference between civil infringement and criminal infringement tends to be "you tried to make a business out of it".

Hope that helps.

<comment><USER7> Yep, that is true, but there is no crime of enabling copyright infringement for profit in either the US or New Zealand, as far as I am aware.

<comment><USER8> This is an article of faith among advocacy groups but I'm not sure (I'm not a lawyer) that's actually true. People have been held criminally vicariously liable for other criminal actions despite the lack of clear statutory language to that effect. They've then argued to appeals courts, "but there's no statute that clearly says I can be held criminally liable vicariously", and lost. "Vicarious liability" is not some crazy idea invented by the copyright cartels; it's a basic principle of common law.

<comment><USER7> I am just basing this view on the Sony vs Universal and Grokster vs MGM trials, which were both civil. Admittedly there is a legal argument that could be made for criminal vicarious liability, however from my understanding, that is usually only used for situations that could endanger the general public, where the lack of direct criminal action is deemed to be outweighed by the need to protect people, and that does not seem to apply in this instance. However, that said, I am also not a lawyer, so I could well be misunderstanding the issues completely.

</comment></comment></comment></comment></comment></comment><comment><USER9> Now they just have to write the other '90%'.

<comment><USER10> Yeah, that was my first thought. Although they're essentially redoing the exact same code so they might have a fairly good idea of how long it would take.

</comment></comment><comment><USER11> It's interesting to know investors are still interested. The risk for them has shifted from "will this be successful", which they have experience in predicting, to "will this get shut down again", which they presumably don't.

<comment><USER12> He has enough money, he won't need investors.

</comment></comment><comment><USER13> Sometimes Torrentfreak takes a snippet of available information, then makes it into a full, long (and excruciating) article, nothing new is added after the opening paragraph here.

</comment><comment><USER14> Maybe they'll clean up the interface!

<comment><USER15> Do you mean the U.S. Dept. of Justice interface?

</comment></comment><comment><USER7> They should call it TeraUpload. Not only would the headlines be great, but also the server instances could then be called tera-cells.

</comment><comment><USER16> Buy Megaupload here: [http://www.amazon.com/gp/product/B007EGTDSS/ref=as_li_ss_tl?...](http://www.amazon.com/gp/product/B007EGTDSS/ref=as_li_ss_tl?ie=UTF8&camp=1789&tag=asdfdsa-20&creative=390957&creativeASIN=B007EGTDSS&linkCode=as2)

This IS a joke.</comment>


Teleport - opinionated server setup with Ruby - <AUTHOR> https://github.com/rglabs/teleport

<comment><AUTHOR> I am the author. Teleport sets up servers by "teleporting" to the machine and running itself there. It sets up Ruby, then follows the instructions laid out in your Telfile to customize the server. Here's a sample Telfile:

      user "admin"
      ruby "1.9.2"
      apt "deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen", :key => "7F0CAB10"
      role :app, :packages => [:memcached]
      role :db, :packages => [:mongodb-10gen]
      server "server_app1", :role => :app
      server "server_db1", :role => :db
      packages [:atop, :emacs, :gcc]

More details on github. Feedback welcome!

</comment><comment><USER2> I'm using an early version of this for a project, and it works a charm. If you work within its (sensible) constraints, it accomplishes much the same thing as Chef, with less effort and less overhead.

</comment><comment><USER3> This looks awesome. Going to try it out... Chef solo is painful.

</comment><comment><USER4> interesting. will have to look at this compared to chef.</comment>


Clocks for Software Engineers - <AUTHOR> http://zipcpu.com/blog/2017/09/18/clocks-for-sw-engineers.html <comment><USER2> One of my favorite undergrad electrical engineering classes [0] took an innovative approach to introducing this. Instead of learning about clocks/pipelines and HDL at the same time, we only looked at the former. We created our own simulators for an ARM subset, fully in C, where there was only a single for/while loop allowed in the entire codebase, representing the clock ticks. Each pipeline stage, such as Instruction Fetch, would read from a globally instantiated struct representing one set of registers, and write to another one. If you wanted to write to the same place you read from, you could only do so once, and you'd better know exactly what you were doing.

Because we didn't need to learn a new language/IDE/environment at the same time that we learned a new paradigm, we were able to keep our feet on solid ground while working things out; we were familiar with the syntax, so as soon as we realized how to "wire something up," we could do so with minimal frustration and no need/ability to Google anything. Of course, it was left to a subsequent course to learn HDL and load it on real hardware, but for a theoretical basis, this was a perfect format. Much better than written tests!

[0] [http://www.cs.princeton.edu/courses/archive/fall10/cos375/de...](http://www.cs.princeton.edu/courses/archive/fall10/cos375/descrip.html) \- see links under Design Project, specifically [http://www.cs.princeton.edu/courses/archive/fall10/cos375/Cp...](http://www.cs.princeton.edu/courses/archive/fall10/cos375/Cproject10.pdf)

<comment><USER3> I happened to do something like this in my CPU design class too. In my case I knew that writing a simulator of our design in C would be trivial compared to actually making the CPU itself + it could be used to test code much more easily (we had to write a GCD routine to "prove" it works).

You're right that it helps a LOT when it comes to implementing the actual hardware.

(I also put it through Vivado HLS, but I wasn't able to sneak that past the professor, rats! :)

I wish I had known about verilator back then, I could have compiled the verilog into c++ and ran my simulator test suite against it!

</comment></comment><comment><USER4> When I took Berkeley's EECS151 class (Introduction to Digital Design and Integrated Circuits), the first lecture actually did not go over clocks. Instead, it goes over the simple building blocks of circuits - inverters, logic gates, and finally combinational logic blocks that are made up of the previous two. These components alone do not need a clock to function, and their static functions are merely subject to the physical limitations such as the speed of electrons, which we package into something called propagation delay. It is entirely possible to build clockless circuits, otherwise known as asynchronous circuits.

From the perspective of an electrical engineer and computer scientist, asynchronous circuits theoretically can be faster and more efficient. Without the restraint of a clock slowing down an entire circuit for its slowest component, asynchronous circuits can instead operate as soon as data is available, while consuming less power to overhead functions such as generating the clock and powering components that are not changing state. However, asynchronous circuits are largely the plaything of researchers, and the vast majority of today's circuits are synchronous (clocked).

The reason why we use synchronous circuits, which may relate to the reason why many students learning circuits often try to make circuits without clocks, is because of abstraction. Clocked circuits can have individual components/stages developed and analyzed separately. You leave problems that do not pertain to the function of a circuit such as data availability and stability to the clock of the overall circuit (clk-to-q delay, hold delay, etc), and can focus on functionality within an individual stage. As well, components of a circuit can be analyzed by tools we've built to automate the difficult parts of circuit design, such as routing, power supply and heat dissipation, etc. This makes developing complex circuits with large teams of engineers "easier." The abstraction of synchronous circuits is one step above asynchronous circuits. Without a clock, asynchronous circuits can run into problems where outputs of components are actually wrong for a brief moment of time due to race conditions, a problem which synchronous circuit design stops by holding information between stages stable until everything is ready to go.

The article's point of hardware design beginning with the clock is useful when you are trying to teach software engineers, who are used to thinking in a synchronous, ordered manner, about practical hardware design which is done entirely with clocks. However, it is not the complete picture when trying to create understanding of electrical engineering from the ground up. Synchronous circuits are built from asynchronous circuits, which were built from our understanding of E&M physics. Synchronous circuits are then used to build our ASICs, FPGAs, and CPUs that power our routers and computers, which run instructions based on ISA's that we compile down to from higher order languages. It's hardly surprising that engineers who are learning hardware design build clockless circuits - they aren't wrong for designing something "simple" and correct, even if it isn't currently practical. They're just operating on the wrong level of abstraction, which they should have a cursory knowledge of so synchronous circuits make sense to them.

<comment><USER5> An asynchronous multicore chip [http://www.greenarraychips.com](http://www.greenarraychips.com)

 _NO CLOCKS: Most computing devices have one or more clocks that synchronize all operations. When a conventional computer is powered up and waiting to respond quickly to stimuli, clock generation and distribution are consuming energy at a huge rate by our standards, yet accomplishing nothing. This is why "starting" and "stopping" the clock is a big deal and takes much time and energy for other architectures. Our architecture explicitly omits a clock, saving energy and time among other benefits._

<comment><USER6> _saving energy and time among other benefits_

Wouldn't (bitcoin) miners be a good fit for async circuits? Simple logic, low power, high performance.

<comment><USER7> The quote totes an advantage whilst waiting for input. I take that to mean having really good idle performance. Miner ASICs don't make money when idle.

<comment><USER8> I don't know the details of the circuitry needed for bitcoin hashing, but it can contain steps that are not executed continuously and hence being idle until they have some work to do (i.e. they have some inputs). Reducing the power consumption of a miner directly translates into making more money.

<comment><USER9> Pipelining would make those idle circuits useful rather than less costly

<comment><USER10> And as far as I know, most of these are heavily pipelined already.

</comment></comment></comment></comment></comment></comment></comment><comment><USER11> I'm not surprised that software engineers find these concepts difficult to understand at first -- it's a very different way of thinking, and everyone has to start somewhere. But I do find it kind of odd that someone would jump straight into trying to use an HDL without already knowing what the underlying logic looks like. (My CS degree program included a bit of Verilog programming, but it only showed up after about half a semester of drawing gate diagrams, Karnaugh maps and state machines.)

Does this confusion typically happen to engineers who are trying to teach themselves hardware design, or is it just an indication of a terribly-designed curriculum?

<comment><USER12> How often do people jump into javascript coding without having the faintest idea of how a CPU or anything else works? When high-level facilities are made available, there are always going to be people who go to the high-level facility without having any understanding of the actual consequences of what they are doing. I'd recommend the software folks who want to get into hardware to check out the book 'Elements of Computing Systems' and the NAND2Tetris project that its based on. Its quite easy to follow along with building your own quasi-HDL things run in a simulator (with no generative for-loop stuff, you have a clock line and you use it), then build up from there to put together a CPU, memory, develop a basic OS, implement a programming language, and eventually play a game of Tetris! All with nothing but NAND gates and the convenience of being able to 'use' those tons of gates without actually having to deal with them physically.

</comment></comment><comment><USER13> TL;DR:

> _The reality is that no digital logic design can work "without a clock". > There is always some physical process creating the inputs. These inputs must > all be valid at some start time – this time forms the first clock "tick" in > their design. Likewise, the outputs are then required from those inputs some > time later. The time when all the outputs are valid given for a given set of > inputs forms the next "clock" in a "clockless" design. Perhaps the first > clock "tick" is when the set the last switch on their board is adjusted and > the last clock "tick" is when their eye reads the result. It doesn't matter: > there is a clock._

Put another way, combinatorial systems (the AND/OR/etc[1] logic gates that form the hardware logic of the chip) have a physical _propagation delay_. The time it takes for the input signals at a given state to propagate through the logic and produce a _stable_ output.

Do not use the output signal before it is stable. That way lies glitches and the death of your design.

Clocks are used to tell your logic: "NOW your inputs are valid".

The deeper your combinatorial logic (the more gates in a given signal path), the longer the propagation delay. And the maximum propagation delay across your entire chip[2] determines your minimum clock period (and thus maximum clock speed)

There exist clockless designs, but they get exponentially more complicated as you add more signals and the logic gets deeper. In a way, clocks let you "compartmentalize" the logic, simplifying the design.

[1] What's the most widespread fundamental gate in the latest fab processes nowadays? Is it NAND?

[2] or at least clock domain

<comment><USER14> > What's the most widespread fundamental gate in the latest fab processes > nowadays? Is it NAND?

Typical standard cell libraries have hundreds of cells, including cells representing logic such as muxes, full adders, and other frequently occurring clusters of gates. Logic is mapped to these in the way the tool finds most optimal. So I don't think it's right to say that there is a fundamental gate in modern processes.

Unless you are looking at it from the electrical perspective in which case the fundamental gate is the inverter.

<comment><USER13> Thanks! I'm probably confusing it with NAND vs NOR flash memory.

It's been a while since I last talked about this with my HW Eng friends :)

</comment></comment></comment><comment><USER15> This is such an important notion.

Another I try to explain hardware design for people coming from a software background:

You get one choice to put down in hardware as many functions as you want. You cannot change any of them later. All you can do later is sequence them in whatever order you need to accomplish your goal.

If you think of it this way, you realize that the clock is critical (that's what makes sequencing possible), and re-use of fixed functions introduces you to hardware sharing, pipelining, etc.

But it's hard to grasp.

</comment><comment><USER16> And here's "Clocks for Hardware Engineers": [1]

[1] [http://lamport.azurewebsites.net/pubs/time- clocks.pdf](http://lamport.azurewebsites.net/pubs/time-clocks.pdf)

<comment><USER17> Not really related to clocks (other than the fact I was watching one while waiting for that to load) but that link seemed very slow to load, I haven't seen a link to azurewebsites before but I'm assuming that's some sort of static file hosting on Microsoft's Azure platform?

<comment><USER18> Loaded fine for me, maybe it's been cached now, Azure end.

It is a pdf though which can be slower.

</comment></comment></comment><comment><USER19> Reading this would actually tremendously help software engineers improve their concurrent/parallel software design skills as well. I never had a particular desire to do hardware (my degree is CS) but some of the best C/C++ programmers who were able to squeeze out every last ounce of performance truly understood not just software languages but also computer architecture and I might even go as far as saying understood physics to a large extent very well. The LMAX software architecture is a product of this kind of hardware+software understanding. Awesome article.

<comment><USER20> Article: [https://martinfowler.com/articles/lmax.html](https://martinfowler.com/articles/lmax.html)

</comment></comment><comment><USER21> "The reality is that no digital logic design can work 'without a clock'. "

This is not true.

"HDL based hardware loops are not like this at all. Instead, the HDL synthesis tool uses the loop description to make several copies of the logic all running in parallel."

This is not true as a general statement. There are for loops in HDLs that behave exactly like software loops. And there are generative for loops that make copies of logic.

Also, the "everything happens at once" is not true either. In fact with out the delay between two events happening, synchronous digital design would not work. (specifically flip-flops would not work).

<comment><USER22> I guess a more accurate statement would be "no practical digital logic design can work without clock (unless you are doing seldom used, generally undesirable asynchronous design)"

HDL languages do have loops, but they are for testbench purposes only, in hardware implemention non generate loops would not be implemented!

I think by saying "Everything happens at once" the author meant that all your code executes at once. He is obviously trying to get the one line at a time sequential mindset out from people used with software.

<comment><USER23> I don't think asynchronous logic is undesirable.

For an example, the GA144[1] is an example of a practical computer completely implemented asynchronous logic.

Its Asynchronous nature is one of the features, benefits include lower power consumption, faster speed, and lower electromagnetic interference

[1][http://www.greenarraychips.com/home/documents/greg/PB001-100...](http://www.greenarraychips.com/home/documents/greg/PB001-100503-GA144-1-10.pdf)

<comment><USER22> I meant undesireable in the sense that it takes longer time to develop and harder to debug, but you are absolutely right about the power consumption and speed

</comment></comment></comment></comment><comment><USER24> I liked the article, but I feel like an argument for why you need a clock was really never made.

<comment><USER25> This is definitely how I felt as well. The discussion is kind of circular, "You need a clock because doing things without a clock doesn't fit well into a clocked paradigm."

It's not an argument against clockless design, but an observation on the limitations of doing everything in a single clock tick of an already clocked chip.

We can surely build a circuit without any clock, but what challenges do we run into? How does imposing discrete clock steps help? What exactly _is_ a clock? I would have liked to see the discussion drop a level or two of abstraction to EE or something.

<comment><USER24> Right, so what the author hinted at, but never really said, is that without a clock it is difficult to prevent spurious intermediate values output by circuits from having unwanted effects; for example, unbalanced combinational circuits typically shift between multiple intermediate values as the signals propagate through. Such hazards are normally avoided, though I can imagine such hazards might potentially contain information that could have adaptive advantage (in evolutionarily derived circuitry)

</comment></comment></comment><comment><USER26> Learning to think in parallel, and understand and design for procedures that don't run sequentially, would be good practice for concurrent runtimes and distributed systems too. Not only for HDLs.

</comment><comment><USER27> The zipcpu blog posts never ceases to amaze me, the content is so good. As a sw developer who plays around in verilog on my free time, the posts are extremely helpful to me. I just want to tip my hat to the author(s?), thanks!

<comment><USER28> Since we're getting meta on the post, I find the tone to be haughty and demeaning.

"I'll spare him the embarrassment of being named, or of linking to his project. Instead, I'm just going to call him a student. (No, I'm not a professor.) This 'student'..."

Why was this needed? What purpose did it serve the greater article except for the author to espouse their own superiority?

This kind of stuff needs to get the hell out of engineering. It turns away potentially many brilliant people that could join the field but fear the rejection by peers.

<comment><USER29> Would you still want it gone if it caused a net gain in proficient engineers, but a net loss in total engineers?

<comment><USER30> > Would you still want it gone if it caused a net gain in proficient > engineers, but a net loss in total engineers?

Do you have any reason to believe that arrogance and proficiency are correlated?

</comment></comment></comment></comment><comment><USER31> BTDTBTTS. Way back when in uni, we had to deaign a working CPU with everything at the time but superscalar, MIMD and reservation/retire. Pipelined CPUs can get faster clock rates by splitting up hardware into more (smaller) stages, but at the expense of total latency (due to adding pipeline regisers) AND slower pipeline stalls on branch prediction misses (pipeline has to be emptied of wrong micro-ops). The overall CPU can only be as fast as the slowest stage.

It looks like this Si are the mostly combinational logic for a stage and Pi are the pipeline registers between stages (nearly all signals between stages should be buffered by pipeline regs). IO is omitted but it's the same overall architecture.

        Clk </comment><comment>--+</comment><comment></comment><comment>---+---... ....
                    |               |
                    \               \
          +-> S0 --> |P0| --> S1 --> |P1| --> .... --+
          |                                          |
          +</comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment>+

</comment><comment><USER32> The Figure 5 in that article pretty much summarizes the main point - if you show that to the original (hypothetical?) student, then this should be sufficient to make them understand the downsides of their design.

</comment><comment><USER33> How does one go about starting a project in an HDL? I have always wanted to design and build a CPU but I've never figured out how to set up the "build chain" for VHLD. How do you implement, compile, and test different features? Is there an IDE?

Understanding the basics is important but I'm held up before the basics even start mattering.

</comment><comment><USER34> Conceptually, this is the same idea as concurrent network programming with futures, yes?

</comment><comment><USER35> Immediately thought it was referring to this [https://news.ycombinator.com/item?id=15282967](https://news.ycombinator.com/item?id=15282967)</comment>


Overuse of bullet points - <AUTHOR> http://writers.stackexchange.com/questions/7719/overuse-of-bullet-points <comment><USER2> A clear example of a class of problem in which one must simply bite the bullet.</comment>


Kdb+ and Python: EmbedPy and PyQ - <AUTHOR> https://kx.com/blog/kdb-python-embedpy-pyq/ <comment><USER2> I've never quite understood why banks still opt for kdb when (a) hiring staff is ridiculously difficult/expensive and (b) there are so many other cheaper alternatives available. They literally have a strangle hold on every eFX desk in London, and I cannot really figure out why!

<comment><USER3> Maybe you're wrong about (b)?

Sure, I could cobble something about as fast and functional out of postgres, C, awk, and a bunch of other things, but it'd certainly cost more _just in developer time_ than kdb.

<comment><USER2> Your comment made me chuckle. In fact, made me immediately think you were employed by Kx at some point; your profile confirmed this was the case!

I think Kx has built a wonderful business, but as an ex MD at a bulge bracket bank, I would not sanction its use any more. It is expensive, and it's difficult to hire good people to work with it (not every guy with kdb on their CV is great). The product is good, but so are many open source offerings, which are now matching it both in terms of performance and flexibility. Plus hiring good people is significantly easier and cheaper.

<comment><USER4> What are the open source offerings that you referring to? The ones that provide 1) no-copy analytics 2) i/o stack bypass i.e. memory-map.

<comment><USER4> Thank you for your answers. To summarize, we have 1) Hobbes 2) hacked LMDB 3) C++ memory-mapped store of arrays.

Given that options #2 and #3 require some (non-trivial) work, they are not really options.

We left with #1,-- hobbes, which was uploaded to GitHub about 5 months ago and has a whopping team of 2 contributors, both employed by Morgan Stanley.

This is more than nothing, but not much.

I do not have experience with KDB, and looking at the language syntax, not a fan. Integration with Python (depending on implementation) may push KDB towards larger acceptance.

So far I was mostly relying on a variation of the option #3.

<comment><USER5> You must be able to come up with better criticism than that. Number of people who work on the project, time it's been on github, contributors' employers ... these are completely irrelevant to the question of whether this project is a viable alternative. There's not a single technical argument here! :)

For what it's worth (not much), from a purely superficial standpoint, kdb itself started out as a one or two person project at Morgan Stanley! :D

We've managed to get this thing right in the hot path (not just for analysis off on the side, though that use-case is important too) where a significant portion of global trading happens, in one of the biggest investment banks in the country, and we've had it working in production for four years doing this (before recently open sourcing), having had to make the technical case to many people who are very aware of kdb and what it can do (as far as kdb goes, Morgan Stanley is Mount Doom!).

I mean, I take your point that it's not ubiquitous in the world yet, but in terms of the OP proposition that there are free and technically superior alternatives, it's proof positive.

</comment></comment></comment></comment></comment></comment><comment><USER6> I like it when kdb is mentioned because then I can post this link: [https://github.com/KxSystems/kdb/blob/master/c/c/k.h#L96](https://github.com/KxSystems/kdb/blob/master/c/c/k.h#L96)

It's one of my all-time favorites. A window into a certain type of mind.

<comment><USER7> I'm surprised you didn't link an actual c file; everything related to k seems to trigger an exorbitantly high number of "what the fuck"s. [https://github.com/kevinlawler/kona/blob/master/src/0.c](https://github.com/kevinlawler/kona/blob/master/src/0.c) for example.

[http://archive.vector.org.uk/art10501320](http://archive.vector.org.uk/art10501320) is one of my favorite articles, though

<comment><USER8> That article was enough to inspire me to write a K interpreter, and eventually land me a job working with K. If I ever meet Stephen Taylor in person I imagine it'll be an interesting story to tell.

<comment><USER5> That's great that your interest produced that result! When I made a K interpreter, Kx threatened to sue me and everyone I had worked for.

<comment><USER9> That's likely because yours was fast enough to threaten their sales, whereas RodgerTheGreat's is JavaScript and can not.

Nick Nickolov's one also disappeared off GitHub, though Kevin Lawler's Kona k3 implementation and Andrey Zholos's jitted weird dialect are fast and still up; also nils holm's klong.

</comment></comment></comment></comment></comment><comment><USER10> the idea that I could seamlessly move data between python and kdb+ is making me salivate. Please, make this work well and lower the cost-barrier to using kdb+ and I think you'd see this become the defacto "stack" for many database setups.

<comment> ah- All this needs now is a free (at least commercial use allowed 64bit version, ideally OSS) kdb+.

I'd love to use this, many things are so much more straight forward in q than they are in pandas. But if it means noone can run my software unless they pay for kdb+ it's a non-starter.

I wonder if this would work with Kona.</comment></comment>


Show HN: A "Hit Song" Generator Written in JavaScript - <AUTHOR> http://hitsonggenerator.com/

<comment><USER2> What is the purpose of this? So I generate a name and then?

</comment><comment><USER3> ummmmm ok... give us some context. Without context this is mix and match site that could be one of the tutorials in an online Javascript class.</comment>


Why Go for decision trees? (2013) - <AUTHOR> http://datascience.systemsbiology.net/data-notebook/

<comment><USER2> Hah, I wrote this article ages ago when we were experimenting with a data science blog. Surprised to see it pop up here.

I think most of it still holds true but we did do some benchmarks on public data and the implementation is competitive with or faster then other implementations including scikit-learn's cython implementation which wins a lot of benchmarks.

Code and some benchmark results in the README are here: [https://github.com/ryanbressler/CloudForest](https://github.com/ryanbressler/CloudForest)

<comment><AUTHOR> I submitted it because I'm playing with CloudForest right now. I really appreciate your work, it's fun to use.

I'm co-founder of a digital real estate brokerage[1] and as a weekend hack, I'm using CloudForest to predict sale prices for residential real estate using data from the MLS.

A lot of the predictions are really good with a few that are terrible (like double the list price, which is a feature). Any way to get a certainty value for each prediction? Perhaps I could look at the variance of values from each tree? I'm kind of a noob with decision trees so I'm probably totally off base but maybe that question makes sense. :)

[1]: [https://www.openlistings.co/](https://www.openlistings.co/) (YC W15)

<comment><USER3> Yes, you should be able to estimate a confidence interval by looking at the distribution of votes from each tree.

[1] [http://arxiv.org/pdf/1311.4555.pdf](http://arxiv.org/pdf/1311.4555.pdf)

</comment></comment></comment><comment><USER4> It sometimes raises eyebrows, but this sort of post is why I classify Go as a modern take on a scripting language. With GC and interfaces standing in for duck typing, it's got a lot of the good characteristics of the scripting languages, but it's also got just enough low-level performance stuff going on (real structs, real arrays, the slices the blog post mentions) that rather than taking the 50-100x hit you can easily take with Python you can take merely a 2-3x hit over C, and you'll write the code significantly more quickly than C or Rust too. The result is that where you might not need something that allows you, and correspondingly, _requires_ you to deal with all the issues of memory management or ownership that C, C++, or something like Rust may, but you might not be able to afford Python's performance issues or the general difficulty the scripting languages have with true concurrency and multicore, you have a viable choice in Go.

Go's not alone... it seems to me there's many languages taking a swing at this niche now. All the best to all of them... I love me the flexibility of Python or Ruby sometimes and may not need total system-level control but I can't always afford the resource gluttony of the older-school scripting languages.

<comment><USER5> This development/trend seems to show that the value of the "dynamicity" of scripting languages have perhaps been overrated. There seems to be a set of abstractions with reasonably/very good mechanical sympathy that are both easy to understand and use, and that doesn't constrain you that much with regards to factorability and coding iteration.

<comment><USER6> If there was a time when dynamic typing was overrated, then it is certainly possible that this time is a time when static typing is overrated.

<comment><USER4> "If there was a time when dynamic typing was overrated, then it is certainly possible that this time is a time when static typing is overrated."

I think that's actually a very important aspect of understanding _where_ dynamic typing as a hugely popular thing came from. Today, I favor static typing. However, given a choice of Python or 1990s-C++, I'll take Python. It could be argued that the static typing world needed to be dethroned and be forced to improve itself in the face of the dynamic competition to become what it is today.

On the other hand, pretending that "static typing" is the same today as it was 20 years ago is merely the complementary failure that people too bedazzled by static typing 20 years ago and dismissed the oncoming dynamic typing wave made.

</comment></comment></comment></comment><comment><USER7> We need generics!

</comment><comment><USER8> Don't know if the link was at a different place earlier, but it's now at: [http://datascience.systemsbiology.net/data- notebook/](http://datascience.systemsbiology.net/data-notebook/)

<comment><USER9> Thanks, we've updated the URL to that.

</comment></comment><comment><USER10> why go?

why is go being so heavily featured on HN lately? reminds me of that effort to ask a ton of R questions on stackoverflow.

We should have more stories about languages people actually get paid to use, like java!

jk (kind of)

<comment><USER11> i'm paid to write go :)

<comment><USER10> I wonder how many here are in your boat</comment></comment></comment>


How We Got Owned by a Few Teenagers (and Why It Will Never Happen Again) - <AUTHOR> http://blog.phpfog.com/2011/03/22/how-we-got-owned-by-a-few-teenagers-and-why-it-will-never-happen-again/ <comment><USER3> I feel really bad for the phpfog guys. But given the situation, I think they handled it admirably well - kudos to them. No software is secure and this could have happened to anyone. Especially startups who have to take shortcuts at the very beginning.

I know the attackers were just kids but I have to admit pursuing legal action sounds very tempting - even to just act as a deterrent to others. If they had just put up phpfogsucks.com, it _might_ have been ok. But tweeting trash from their twitter account, redirecting their root domain to phpfogsucks, etc - <USER2> all not cool at all and should have _some_ consequences.

<comment><USER4> What I find most disturbing about this whole situation is the way in which these teenagers are handling themselves, especially after the fact. The continued denial of responsibility and half-hearted mea culpa, coupled with the monetary damage to those businesses who had been running on PHPFog, leads me to sincerely desire that these teenagers face a penalty of some magnitude, not just a slap on the wrist.

Maybe then they'll stop with the half-assed apologies and recognize that there's a right way and a wrong way to do things.

<comment><USER5> _Maybe then they'll stop with the half-assed apologies and recognize that there's a right way and a wrong way to do things._

PHPFog built a castle out of sand and you're upset that a wave came and demolished it. I'm always surprised at how thin-skinned a lot of HN commentary is. "Oh, Zed shouldn't be so rude" "These kids' lives should be destroyed for playing games with an wholly insecure website." "I stopped reading that article because it used the word blowjob."

I don't get angry at my dog when he shits in the house. Being angry at something that can't understand only satisfies the urge to shift blame.

My dog shits in the house and it's my fault for not walking him sooner. If some children compromise every level of your company then getting mad at them is only trying to deflect the blame. PHPFog is the only responsible party in this mess. I feel for the customers who still trust them.

<comment><USER6> This isn't a wave knocking over a sandcastle or a dog shitting in the house. These are 16 year old kids, old enough to know right from wrong, and with the knowledge and skills to exploit the system. And once the exploit worked, they didn't then responsibly disclose the problem to PHPFog; they started vandalizing, changing passwords, and the works.

This is like someone finding an unlocked door to the apartment building's maintenance office, taking the master keys from there, rifling through a bunch of people's personal belongings, sticking signs in the windows saying "this building's landlords suck," and changing the locks on some of the doors to make it hard to clean up the whole mess.

They absolutely are the responsible party; you should never blame the victim of a crime just because the victim didn't take adequate steps to defend themselves. If I accidentally leave my door unlocked one day, that does not make it suddenly OK to come in and take my stuff and it's my fault for not having locked my door, instead of yours for taking my stuff.

Now, in this case PHPFog does bear some responsibility, because they have a duty to protect their customers as well as possible, and from reading about how this happened, it sounds like they were amazingly sloppy and irresponsible about it (passwords stored in the clear on the server, passwords shared between various accounts, leaving unsecured shared systems running after beta launch, etc). But that doesn't reduce the culpability of the attackers; they acted maliciously, with full knowledge of what they were doing, vandalized systems, changed passwords, and bragged about it.

<comment><USER7> I'm now nearly 25, and the amount I have changed since I was 16 borders on the immeasurable. Teenagers are glorified children. We seem to forget how little reflective capacity we all had when we were teenagers.

If I were 16 and hacking some stupid website, I would think, sure this is "wrong", and I might get in trouble, but I probably wouldn't think that it would matter 5 years from then, or 20. Truth is, a criminal infraction can fuck your life up worse than is deserved. You could be forbidden to immigrate to your wife's home country, denied any worthwhile job you can find, or spend a sizable portion of your life in a smaller cage than they house animals in.

There's a reason juvenile law exists, but we forget it. We charge 12 year olds as adults. This isn't _that_ different.

Hell, where I live, a lot of employers will check the sheriff's website for public arrest records. These aren't convictions, they're arrests. People are being denied jobs based on accusations.

I've never gotten so much as a speeding ticket, but even I realize what a load of shit this all is.

<comment><USER8> So when they turn 21, a "be responsible" switch will magically flip in their minds?

The job of parents and society is to teach children responsibility. That means having consequences for your actions. And the closer you get to adulthood, the more adult those consequences should get.

When I was a teen, some real estate developers tore down a bunch of woods where I had always played and started building a house. I was ticked and I vandalized the construction site. But my crime was discovered, and I had to work carrying lumber in the hot sun and scraping glue off windows to pay back the damage. Why did I have to do this? Because the developer talked to my parents. And my parents made me do it.

This was light punishment - I wasn't taken to court, and I didn't get a criminal record. But parents are to children as legal system is to adults: they set the rules and enact the punishments. If they don't, someday the legal system will have to address their failure to do so by locking up their grown children who never learned right from wrong.

I'm not sure what the consequences here should be, but the argument "they're kids, they can't be held responsible" is silly. If they're not expected to be responsible, they won't learn to be responsible.

<comment><USER9> What you're saying is fine. Its probably the better adult way to handle it - talk to the parents. Other people are suggesting FBI/Criminal law. How would you have fared/grown up if they sent you to court?

Do note that since he is tech oriented, he probably now knows what a S __* storm he has kicked up - the blog response pretty much ensures that he is aware. I assume you would be sweating bricks if you knew that the FBI was coming after you AND that you had hand delivered a bullet point confession. I assume thats a pretty strong deterrent (in his specific case)

(Incidentally fighting to stick up for your woods was a kinda nice thing to do, modulo the amount of vandalism you pulled off.)

</comment></comment></comment></comment></comment></comment></comment><comment><USER10> The blog post is riddled with the words "luck" and "timing" which brings doubt into my mind that the team can actually take full responsibility for their actions.

"aware of the potential security threat " but they left it for the next week, who honestly here would do that?

I have also seen comments around the web of migrating to Php Fog because of how they handled the situation. If you are one of these people please enlighten my mind as to how you came to such a logical decision or how much you get paid per year.

Also if Php Fog could enlighten us on how their terms of agreement will work in the case where our intellectual property is stolen on no fault of our own.

Save your sympathy for the sites that are still down, four days and counting

<comment><USER11> I couldn't agree more. The phpFog team cut corners to deliver quickly. We (devs) all do it. The important part is to clean up after yourself.

The whole blog post seems a bit melodramatic. I mean seriously, who here hasn't spent 3 all nighters in a row fixing a mistake? sack up and do what you should've done before deploying other people's data.

...and who would seriously sue these kids? they handled it poorly but they're smart (definitely smarter than i was at 16) you're lucky it was curious kids, rather than malicious (and experienced) hackers that would've been harder to catch. Do you really want to burden them with a criminal record for life?

<comment><USER12> Am I the only one who becomes functionally useless after the first 24 hours? I'm nearly 30 now, but the maximum I'd have done 10 years ago was 36 hours.

</comment></comment></comment><comment><USER13> I am bothered by some of the language in this post:

\- _we were aware of the potential security threat behind post-deploy hooks and were about to disable them [...] but..._

\- _we were days away from replacing this server_

\- _They were a short-term stopgap measure we had been planning to replace_

To me, it sounds like the real problem could have been stated as "We were lax on security," but almost worse than that is the lack of accountability that I sense from company. Yeah, maybe it won't happen again, but it's hard to be full of confidence to buy into a service like that.

<comment><USER14> The article starts out in that tone, but it changes to be pretty remorseful after that. For example the "why it won't happen again" part.

<comment><USER15> I agree with parfe's comment below.

They sound incredibly laxed on security and the "we were days away from fixing it" could be complete bull. To Lucas, it probably sounds better to say they were close to fixing it instead of admitting they were unaware of these exploits.

I find the disclosure in the blog post great, but the conditions they had leading up to the hack very disappointing.

If they were aware of the exploits, they should have taken quicker action. They'll probably be focusing on security big time now... they have no other choice.

</comment></comment></comment><comment><USER16> "We have hired professional white hat hackers with government level security experience to attempt regular pen tests on our system..."

I guess whenever I read this kind of statement from now on I'll be thinking of HBGary and chuckling a bit inside.

<comment><USER17> At the risk of that comment being taken as a joke, I've done a lot of work with the federal government, and I can assure you that while the level of hilarity that HBGary has generated, the typical level of talent in government cleared individuals is not necessarily great.

I don't mean to impune the capabilities of the people involved (I don't know who they are,) and it isn't to say that you can't find some AMAZING talent in the government realm, but as in all fields, it's the exception, not the rule.

<comment><USER18> Reminds me of when I was in the Navy and I went to Navy Security and Vulnerabilities Technician school. I was all excited, so I went out and bought a copy of Hackers Exposed and read through the whole thing, learning everything from how to determine what family and version of operating system a computer is running by what ports are open, to how a buffer overflow attack actually works.

Fast-forward to the class, and we're sitting there running tools like BackOrifice that exploit vulnerabilities that had been patched for _years_ , and learning that a SYN flood is "a malicious attack". That's it, just "a malicious attack". When I asked about the difference between a SYN flood and a Christmas tree attack, I got a blank stare and "they're both malicious attacks".

I spent the rest of the class in the back of the room, reading the Armadillo Book.

Also, I did not once in my brief Navy career get to hack an enemy computer. Hugely disappointing.

<comment><USER19> Yeah, military schools on scientific/technological subjects can be very disappointing. Not all of them, obviously, but computer science topics get distilled down to be accessible to the bottom 20% of attendees. Having spent over a decade in the Marines, I learned not to get excited about anything like that... unless the schools were taught by civilians.

</comment></comment></comment></comment><comment><USER20> I mentioned this last time, but I don't think anyone was interested, but the "John" guy is compwhizii (same handle on Twitter) who runs the forums (facepunch.com) for garrysmod, a very popular game. I will be curious to see how garry (owner person) responds to this, or if he already has.

Elliot is apparently VERY scared and blames John (compwhizii) (edit: not john, he blames someone else called supersnail1): [http://www.facepunch.com/threads/1071855-A-member-of- Facepun...](http://www.facepunch.com/threads/1071855-A-member-of-Facepunch- may-cause-me-to-be-sued)

Here is (compwhizii) Johns reply: [http://www.facepunch.com/threads/1071855-A-member-of- Facepun...](http://www.facepunch.com/threads/1071855-A-member-of-Facepunch- may-cause-me-to-be-sued?p=28754506&highlight=#post28754506)

<comment><USER21> And here's Elliot's "official statement": <http://elliotspeck.com/phpfog.html>

And for anyone who missed it, here's what Elliot posted in the previous HN discussion about the phpFog breach: <http://news.ycombinator.com/item?id=2346161>

<comment><USER22> _Hi, I'm <full name>! My site says what city I'm from. I've written publicly to admit that I committed multiple crimes, definitely without consulting legal counsel first! I even put them in a nice bulleted list that can be copied and pasted right into a complaint. They're pressing charges, but that's bullshit. I'm 16!_

Not too bright, are we? Instant message the company you just hacked and bust out from behind your handle, then provide evidence for the prosecution in the form of a Web page? What is with kids these days?

It only takes one episode of Law & Order to figure out how to proceed here. Clue: Attorney.

<comment><USER23> Somebody really needs to get in touch with that kid's parents and let them know that they need to find the mute button on their kid and retain counsel.

After reading that I think it's a pretty strong argument against those claiming adult status for him. He clearly doesn't understand the situation he's in or the second order impacts of what he's done.

</comment></comment></comment></comment><comment><USER24> The phpfog guys really deserve praise for being so open on this issue. As a fellow engineer, being able to learn from their mistakes and see exactly what they could have done _ahead of time_ to avoid the disaster is priceless.

Just goes to show that those with the time to spend are the most likely to break your stuff, even if you pay "professional white hat hackers" to test your system.

<comment><USER25> On the contrary, they knew of security vulnerabilities and intentionally left them unpatched, then blamed it on chance and timing when they got owned because of it.

Avoid phpfog if at all possible, in my opinion.

<comment><USER24> If I lost $1 for each time I left a known vulnerability unpatched because I was convinced I had more important work to do, I would be a very poor man indeed.

Honestly, there are very very few developers that fix security problems in beta environments before anything else. In my experience, it's more likely that you're fighting fires, handling outages, and dealing with problems of scale than fixing security vulnerabilities.

Besides, isn't a beta the correct time to find these security issues? (Design / Alpha would be the ideal time, granted, but sometimes that's not possible.)

</comment></comment></comment><comment><USER26> It seems like incredible coincidence that allowed this to happen but when I think back to all of the security incidents I've been involved in, it always seems this way.

I guess the best way to think of it is that badness on the internet is like water. It will flow into every tiny crack in your wall you haven't sealed up tight. A crack in a dam doesn't leak less because its in an "obscure" location.

</comment><comment><USER27> Goes to show you why the DRY principle (I might be stretching that analogy here, but bear with me) is important here - if you have old stuff lying around in production that was cloned a long time ago, you might forget about it and open yourself up to unfortunate incidents like this.

PHP Fog is doing great work to make the PHP ecosystem easier to work with, and I hope they didn't suffer too much from this mistake.

<comment><USER24> Still stretching the analogy, but the same could be said of their password reuse.

<comment><USER28> That's exactly what I thought of.

DRYP - Don't repeat your passwords.

</comment></comment></comment><comment><USER29> While it is admirable and good that they have learned from their mistakes and are taking steps to reduce the likelihood of getting hacked in future, to say "never again" is to paint a big red bullseye on yourself.

</comment><comment><USER30> Wait...their model is an EC2 instance per customer? The normal limits Amazon imposes are 20 reserved or on-demand instances and 100 spot instances per region. You can request more, but will Amazon really accommodate a one instance per customer model?

<comment><USER31> Amazon is happy to. The limits you cite are merely the point at which you need to have a conversation with Amazon staff. They are quite happy to accomodate _much_ heavier usage from customers.

<comment><USER32> When I asked for a raise to my limit they denied me, on the basis that my usage was insufficient. Of course, my usage was low because I hadn't launched my product fully because I didn't have enough instances to serve a lot of customers ... catch 22.

So I had to build out a rather convoluted architecture that used the loophole of deploying to multiple regions and failing over to whichever region would give me an instance ... which gives me up to about 80 instances ... just barely enough for me to get going with a trial beta program.

Which is all just to say, it is _slightly_ more than just a "conversation" that you need to have to get a higher limit.

<comment><USER31> Sorry you had a bad experience. I have gotten nothing but superb support from Amazon staff. I'm surprised they weren't willing to accommodate you, doubly so if you were ready to pay.

One thing that surprises me is when people talk about utilizing multiple availability zones in EC2 as some sort of burden. It's very clear from their documentation and architecture that you need to be capable running in at least 2 availability zones regardless if you want any sort of availability.

<comment><USER32> > people talk about utilizing multiple availability zones in EC2 as some sort > of burden

My use case isn't for an ongoing server where you require availability. It's purely about compute power - I don't care where the compute power comes from but preferably I want low latency to my customer. So ideally I would just get all instances for any given customer from a single region.

I did find in the end that, as you say, I would sometimes not be able to get an instance in a region even when I was below my 20 limit for reasons internal to Amazon, so the failover work was going to be something I had to deal with anyhow ... but it just added complexity to my life earlier than it would have otherwise.

Edit: I would also mention that I certainly don't think of it as a "bad" experience. I think it is something of a small miracle that Amazon offers the service they do in the first place and I certainly understand why they have caution about handing out large limits to just anyone. I only made my comment above as a kind of caution to not just assume you're going to get a raised limit from Amazon immediately and especially don't leave talking to Amazon about it until the last minute if you're planning to launch something.

<comment><USER31> Thanks, that's an angle I haven't seen since I've mostly used EC2 as a hosting service. What little batch work I've done on it hasn't involved instance counts where I ran into limits. (Now, EBS i/o limits and other things... :/ )

</comment></comment></comment></comment></comment></comment><comment><USER33> Leaving the doors to your house wide open does not grant every passerby the right to enter.

So, yeah, PHPFog screwed up and did that. Then these kids went in, threw paint on the walls, smashed some windows, etc.

PHPFog was stupid - they admitted that.

The kids were criminal.

The first is not illegal - the second is.

</comment><comment><USER34> What a crazy story. If the timelines are accurate there was an extremely small chance of this happening. Bad luck all around.

My site is still down, guess i'm in the unlucky 1%.

<comment><USER35> Yeah but the problem with 'things planned for the near future' is that they have a tendency to stay in the future until something like this happens.

<comment><USER36> Yes, and after the fact, they sound like excuses (even when they are true and they are not meant to be excuses as in this case).

</comment></comment></comment><comment><USER37> Ugh, you shouldn't try writing an apology after not sleeping for days. Sleep on it first, always sleep on it. Talking about prosecution and explaining this with a framing that it was all a fluke caused by the only person who was silly enough to IM you with a confession... add one more person who will never be a customer of yours with an apology like that. Now I know you're irresponsible.

Seriously don't write official blog posts for your company while you're experiencing "I was just in the field for days trying to fix this stuff" emotions.

Calm down, then try and be graceful about the fact that you were hacked by a few clueless kids. (Clueful kids don't let you know who they are.) Then try and figure out how to protect yourself against people with a clue.

</comment><comment><USER38> Wow, that is quite the list of security measures that they had almost but not completely/correctly implemented, or hadn't got around to yet.

I guess the real moral of the story is to finish what you begin, or don't keep putting security off until it is convenient for you.

</comment><comment><USER39> Never? I would be cautious about issuing a challenge like that.

<comment><USER40> They might still have a security hole big enough to drive a freight train through, but that specific attack will never happen again given that they've shut down the shared failover server.

</comment></comment><comment><USER41> Congratulations to PHPFog. They've managed to direct the attention to the 16 year old kids rather than their own incompetence.

Is it me or no one mentions the lack of expertise of the PHPFog team in PHP and Systems Administrations.

Sure kids broke in and the way they published their findings was despicable. The fact remains that PHPFog was utterly broken to pieces and the exact essence of the problem is simply the lack of knowledge in their field.

I am very disappointed by the tone of the blog post and think PHPFog don't really have a notion of what they are doing. I would much rather seem them where they belong, in the Ruby world where their experience is.

</comment><comment><USER9> Their response and abilty to turn the situation around is a case study in dealing with a difficult situation. Kudos! I'm saving their response and will use it when dealing with things. Being able to have a counter party to identify has definitely helped in handling the situation. I didn't realize how powerful that can be until I saw this, I learnt something new.

Its a brilliant piece and a great start/way to restore faith and recover from what must be a pretty grueling ordeal. Good job.

</comment><comment><USER42> Great to see disclosure. This can happen to anyone, and more so for startups, where labor is short, focus is on developing features. Using the phrase "Never Happen Again" is a bit strong though. Security is risk management; spend until you can accept the remaining risk while still maintaining profit and avoid being a hacker's low-hanging fruit.

</comment><comment><USER43> This post convinced me _not_ to use PHPFog. They reveal more in their lack of foresight and security prevention measures than their response to what was otherwise a fairly trivial exploit. I am not sure this blog post was helpful in convincing customers like me that want to feel that their infrastructure providers are on top of things.

</comment><comment><USER44> Here's an interesting tweet from one of their developers.

<http://twitter.com/ReinH/status/50348989366796288>

> Your password in the database is SHA512 encrypted, but we're not taking > chances.

I hope he knows what he's talking about and is just tired from the past few days.

<comment><USER36> Let's hope they are salted and iterated.

<comment><USER18> Or swapped over to Bcrypt

<comment><USER45> we've cleared the old SHA512-salted passwords out of our production database and have upgraded the password hashing to bcrypt, with a cost of 10.

<comment><USER18> Good call :)

Just in case anyone doesn't know why Bcrypt is so awesome, it's because it actually takes _longer_ to hash (based on the difficulty level you set, and you can bump up the difficulty level as hardware gets more powerful).

For other applications, you want hashing to be fast. But for passwords, you want hashing to be as slow as possible without compromising user experience.

<comment><USER45> I wanted Lucas to link to Coda Hale's post on bcrypt (found by googling "Coda Hale bcrypt") in the blog post, but he edited that out. So it goes.

</comment></comment></comment></comment></comment></comment><comment><USER46> I remain in two minds about idea of charging the kids.

There is no doubt they did some things they should not have. And I don't doubt there can be a decent case built against them. But as someone who actually had something from his teen years come to bite years later, it's not pleasant. At least in my case it was a MAJOR maturing moment(also the worst day of my life). May be it will take a lawsuit to get these kids to mature up...to that extent anything that gets em to mature up before they _really_ get screwed would be fair.

I'm not merely advocating another chance but actually something that gets these kids to be a tad more thoughtful about their actions. It's not always easy to do that when you are 16 and full of adrenaline.

</comment><comment><USER47> I am sure, many of the HN users here would have found at least a loophole in similar systems in the course of time. What I do in such situation is letting the service know about the flaw. Isn't that the ideal behaviour ?

</comment><comment><USER48> > Eliminate shared hosting failover server – We may never do shared hosting > failover again if we can not guarantee its security. We might do a non- > realtime failover to automatically launch a new instance for you, but this > experience taught us what a bad idea this can be.

What does _realtime_ mean in this case? Anyway, this isn't the only option. They could keep a few bare instances of their php stack online and simply run the deploy script instead of the image creation script. That ought to be able to run in under ten seconds I think.

<comment><USER31> Realtime means a request in flight is retried if there's a failure. Said a different way, the load balancer already knows about a spare at all times. This is compared to a ~5 minute downtime if spawning a replacement EC2 instance is required.

Nice idea on the hot spare instances however.

</comment></comment><comment><USER49> This feels like a business model where the lean/MVP approach isn't quite appropriate. A lot of things fall out of that decision, not the least of which is that the exposure surface area you get from an environment that allows user-sourced code on purpose is enormous. I feel for the guys going through this but there were a lot of errors in the wild all at once to allow this to happen.

<comment><USER50> IMO, in this sort of business it is important to define the right MVP.

</comment></comment><comment><USER51> There is no such thing as bad publicity! Kudo's for turning lemons into a viral blog post! Although, if I understand correctly, you were reusing passwords and storing them in plain text! This is an ABC123 computer security nono. Thank goodness it was just some young script kiddies and not someone with malicious intent!

</comment><comment><USER52> great to hear all the details so quickly so that others building similar systems aren't in the same situation. as fellow PHP'ers its also great to hear that you are not blaming it on PHP somehow (no fuel for the php haters).

</comment><comment><USER53>
      2:56:45 AM Elliot : then I used the method detailed by turby
      2:56:46 AM Elliot : to gain root

Has anything been said about what this method was?

<comment><USER21> If it's what was on the PHP Fog Sucks website, it worked something like this (I may be misremembering a step):

1\. Use the post-deploy hook to chmod _/home/ubuntu/.ssh_ so that it could be written to.

2\. Upload a PHP shell, use it to write your public key into _/home/ubuntu/.ssh/authorized_keys_ , and get the public IP of the EC2 instance.

3\. SSH into the box, _sudo su_ will get you root.

<comment><USER53> Ok, thanks. I was wondering if there was some other root exploit at play, not just bad system configuration.

</comment></comment></comment><comment><USER54> So, shouldn't the first thing you learn as a hacker include how to mask your physical location so as not to have the FBI knocking on your door?

</comment><comment><USER55> I feel for the people at phpfog.com, but this is a bigger blow to cloud computing.

Customers who are already pretty risk averse to their data being stored in the cloud would see this as another reason not to take the risk.

The cloud computing consortium needs to work on a stable stack as well as figure out how to audit that it works properly.

In addition, it calls for security ahead of features. Given that phpfog is funded, they'll need to implement the equivalent of a bleeding edge stack and a locked down stack.

</comment><comment><USER56> php... a language by amateurs, for amateurs. phpfog... a service by amateurs, for amateurs.

<comment><USER57> Your argument could also apply to the Linux kernel.

<comment><USER58> I know the point you're trying to make, but it's a weak one.

75% of the kernel is written by corporate employees.

<http://apcmag.com/linux-now-75-corporate.htm>

<comment><USER46> _75% of the kernel is written by corporate employees._

And that implies what about the kernel?

<comment><USER58> I'm not sure where you're heading with this. I was trying to tell the parent comment that arguing against 'amateur' coders writing php is the same as the Linux kernel was factually incorrect.</comment></comment></comment></comment></comment>


I went through YC as an intern, here's what I learned - <AUTHOR> http://swizec.com/blog/i-went-through-yc-as-an-intern-heres-what-i-learned/swizec/2345

<comment><USER2> I think there's nothing better when your young (or old for that matter!) to muck in with a startup. Although it isn't Silicon Valley, I interned with two businesses this summer a loved it.

With one, we were brainstorming ideas for the product and mapping out the development schedule. You guys know what it's like - your sitting in a noisy cafe with a pen, a piece of paper, all grinning at each other as you see ideas coming together. It's even more awesome now as the products get developed (we've got one developer. He's a real trooper!). You learn how to work with other people. You learn to expand on other people's ideas. You learn to ship.

The other business I worked with is a bit more established, (an SEO consultancy) where I was working on my own within the team to produce a one- off guide to creating linkbait. It took two weeks of research, interviewing the team on video, editing video, working with the design team and writing it all up... then about a weeks editing which included learning bits of CSS mighty-fast to fix formatting errors! Hehe...

We launched this afternoon - Seth Godin picked it up! WOW! That and a few hundred other tweets and some really quite humbling comments from the rest of the SEO community.

Why in the world that we live in would you not try and connect with people and do awesome, challenging, fun stuff like this. And why would you work in a corporation?!? You'd never get anything like that kind of flexibility you get with SME's and startups.

I'd love to do what this guy did in the valley someday. Better still, I hope to get more people interning in the valley and across the world with startups. Maybe there's some money in that?

<comment><USER3> I am one of the owners of one of the companies that Ed interned at. He is amazing - proving that opportunity is there if you just go and grab it. Here's what he produced: <http://www.distilled.net/linkbait-guide/>

</comment></comment><comment><USER4> I know it was probably sarcasm, but I'd assume most people won't be making a list of people to share wealth with when they strike it big. If you want something in return from those founders you should make it clear what you'd like (job referrals, industry connection, etc) now before your opportunity diminishes. Don't keep your card too long.

<comment><AUTHOR> Yes it was mostly a joke.

But it's never bad to be able to quickly relate to a certain summer when talking to such people some random time in the future.

It's a bit like how you see in the movies people relating over going to the same university.

</comment></comment><comment><USER5> One more proof that startups are for people that are into the experience. Not the money, not the credit, just the fun of doing it. If you're about calculating your percentage, figuring out if your're getting paid the largest sum of money around or have the best title - talk to the recruiters of big cos.

<comment><USER6> Totally agree with you. While you need to fairly sure you are on a path that is capable of providing you with a decent return at some point, the fact remains that you have to absolutely love the journey, otherwise there are a lot easier ways to earn a living... I'm involved in my third startup experience (one decent-ish consulting biz (not really a startup but a new biz nonetheless), one ecommerce biz that went fairly big (in UK terms) and then blew up, and now a social commerce play), and I still get the same feeling of being a kid in a candy shop that is just not present when you are doing (much better paid) work for other people.

</comment></comment><comment><USER7> Can you elaborate on what Fukime is and how you used the Parse launch in your favour?

<comment><AUTHOR> The page itself had to be taken down by request of certain people, so I'm not sure how much I can really elaborate.

But basically it was just this:

1\. Lots of buzz around a startup 2\. Use startup's name in your link titles and add a bit of controversy

It gets plenty of clicks :)

<comment><USER8> Was it taken down as a result of the controversy, or was it unrelated?

<comment><AUTHOR> The controversy.

</comment></comment></comment></comment><comment><USER9> This is a pretty good post, but I think a lot of these things can be said of any start up, not just of a startup going through YC. Interesting read, though. Can't wait to try myself.</comment>


Why Scala's Option and Haskell's Maybe types won't save you from null - <AUTHOR> http://beust.com/weblog/2010/07/28/why-scalas-option-and-haskells-maybe-types-wont-save-you-from-null/

<comment><AUTHOR> It appears that his primary assumption, NPE prevention, on Options is based on blogs rather than actual good quality code.

If you look at how Options are used in Scala core lib, you wouldn't make such claims and you will definitely not use it only for null checks.</comment>


OnLive awarded important patent on streaming gaming content - <AUTHOR> http://arstechnica.com/gaming/news/2010/12/onlive-awarded-important-patent-on-streaming-gaming-content.ars <comment><USER2> Well, that kind of blows. Aren't they just patenting charging for dummy terminals?</comment>


Ask HN: How to Automate Processing of Emails? - <AUTHOR> The simplest idea is: 1. Whenever I receive an email, some script should be ran. 2. The script takes the email as the input. It can do anything, for example parse the information and categorize it and store it somewhere else. 3. The script then move the email to different folders (for example: processed, error-occurred).
I'm using this to sort through hundreds of amazon shipping emails, and I know sometimes the processing of the email might fail, so I need to read a few manually.
Looking around I don't see any tools making this easy. <comment><USER2> You can use gmail + a forward to cloudmailin + trigger an aws lambda running a simple script.

Alternatively you could have it direct to cloudmailin if your usage is a bit different.

<comment><AUTHOR> Thank you this is exactly what I need!</comment></comment>


RUI, one ring to rule them all - <AUTHOR>

https://www.indiegogo.com/projects/rui-one-ring-to-rule-them-all/x/2069339#home
RUI(Ring Used Interface), the next generation wearable natural user interface for home automation and augmented/virtual reality.
RUI is the very first wearable device in the world that has seamlessly integrated remote control, gesture control, multi-touch control, augmented and virtual reality control into one. With the multi-touch surface,  RUI has the most natural way to detect human finger gestures, hand gestures, and arm gestures, and provides a powerful control system for all the electronics around you and your house.
It truly enable the futuristic scene of Iron Man 2 and Minority Report. Truly enable "one ring to rule them all, one ring to find them". Truly enable the "point anywhere, control from everywhere" remote control for all your home appliances. <comment><AUTHOR> It can naturally detect human finger gesture. It has a multi-touch surface acting as a full function trackpad, keyboard, mouse, and remote control. The multi-touch surface really bring all the functions together.</comment>


Gauging Gates's Health Grants Five Years In - <AUTHOR> http://www.nytimes.com/2010/12/21/health/21gates.html?hp=&pagewanted=all <comment><USER2> This is a fascinating look at some of the science that has been funded, but also I found 2 meta-points interesting:

1\. The fact that there is such careful evaluation going on is reassuring. But then this was to be expected from Gates.

2\. They're also ratcheting down the size of each grant and increasing the number of grants -- exactly the approach that YC has been promoting. Obviously, there are many differences, but I see the same trend of "diversification" (for lack of a better term) at work.</comment>


Lost Joys of the Screen Saver - <AUTHOR> https://www.theparisreview.org/blog/2017/05/23/salvation-mode/ <comment><USER2> I _love_ Screensavers, and I still have them enabled on my machines. Typically, the screensaver kicks in about 10 minutes in, and the machine sleeps properly about 20-30 minutes in. Laptops on battery power are the only time I forgo them; choosing to go direct to sleep modes 5-10 minutes in.

I have (like many of us) written fun some Screensavers in my time. My current stock one is actually boring - it's a Windows 8/10 style clock and date (think the one on the lock screen) in white lettering, that floats around on a black screen. Simple but useful.

In the past I've written fractal based one, game of life ones, youtube streaming ones, ISS live camera ones (they've since locked out direct access to the feed - boo). I've wanted to write one that plays recorded footage of all the flight paths in World of Warcraft - like a virtual DroneCam - but I never seem to find the time to record the footage...

Screensavers are a lost art form, and I totally agree with the article.

As an aside, how many of us used to get annoyed when people said 'I've got that picture as my screensaver' when you _know_ they mean 'I've got that picture as my wallpaper'? Grrr.

<comment><USER3> > a lost art form

Yes, totally, and one that only flowered for a quarter century! Although if somebody released a generalised program that allowed others to create their own screensavers as plugins, perhaps they could flourish once again?

<comment><USER4> I feel like screensavers and music visualizers are similar art forms, and both flourished for only a short period of time.

I guess music visualization still has a niche industry in live performance or creating video entertainment. But screensavers...

<comment><USER5> I think everyone in this thread should familiarize themselves with the demoscene ([https://en.wikipedia.org/wiki/Demoscene](https://en.wikipedia.org/wiki/Demoscene))

It's 30 years old now and still going strong, and it'd be right up your alley.

There's plenty of trippy, interesting, or technically demanding audio/visual demonstrations for you to run, emulate, or just watch on youtube.

</comment></comment></comment></comment><comment><USER6> Screensavers are great. I spent endless hours watching Screen Antics (aka Johnny Castaway)[1], and the silly things he did.

But I also watched countless hours watching this art screensaver that came with my Canon printer (I think it was Canon). It just animated the painting process. I think it was called something like Canon Creative, but I'm not sure. I'd love to find that screensaver again - That'd instantly take me back 20 years.

[1] [http://web.onetel.net.uk/~gnudawn/johnny/](http://web.onetel.net.uk/~gnudawn/johnny/)

<comment><USER2> Never heard of that one! I always felt that Little Computer People would make a good screensaver, which is the same concept I guess.

<comment><USER7> Always wanted to play LCP as a kid, never got the chance. Should really fire up the emulator...

</comment></comment></comment><comment><USER8> I _really_ miss After Dark. A couple are available (legitimately, apparently) through IIRC some Japanese site, though I recall the payment system being sketchy enough that I wasn't willing to give them my CC. Still, they provided Flying Toasters as a demo with the full version unlockable with a license key, and a little digging with "strings" and a write to a certain MacOS preferences plist later... and man do I like seeing those toasters when I come back to my desk after a meeting. Wish I could get the city skyline, though. :-(

Anywho, I really can't believe the whole package isn't on the App Store somewhere. I'd probably have dropped $20+ for the whole collection.

<comment><USER9> There's an in-browser CSS recreation of some of the After Dark screensavers here: [https://www.bryanbraun.com/after-dark- css/](https://www.bryanbraun.com/after-dark-css/)

</comment></comment><comment><USER10> Aerial [1] for macOS is nice and quite relaxing. It plays the fly-over/aerial screensaver videos from the Apple TV.

Note that it does download a whole bunch of HD videos -- don't start this on mobile broadband!

[1] [https://github.com/JohnCoates/Aerial](https://github.com/JohnCoates/Aerial)

<comment><USER11> Last year, I installed an older version of this and 3 months later I got a nasty email from COMCAST telling me I was using more than a TB per month and was going to get charged a fortune unless I stopped. It was really hard to pinpoint it as the cause because when I was investigating, the screensaver turned off. I think the newer version caches thankfully.

</comment></comment><comment><USER12> I still use xscreensaver:

[https://www.jwz.org/xscreensaver/](https://www.jwz.org/xscreensaver/)

Note, there's likely some anti-link mojo there, so you might want to copy- paste that link.

<comment><USER13> For the lazy: (this will hide the referral and not forward you to the image crying about HN traffic)

[https://anon.to/?https://www.jwz.org/xscreensaver/](https://anon.to/?https://www.jwz.org/xscreensaver/)

</comment></comment><comment><USER14> The PC screensavers were naff and showed the level of imagination that went with the PC. In comparison the SGI screensavers were inspiring. Not only did they have 3D but they did stuff with that 3D. Meanwhile, on the PC with Windows 95 you had some annoying spinning text as the go-to screensaver. Even though you could customise everything about the text including colour and font, it still looked naff.

The SGI screensavers had authenticity, that teapot was no normal teapot, it was The Utah Teapot!!! Every screensaver had some educational aspect of maths or graphics that probably went back to some SIGGRAPH paper or Conway's Game of Life. The cheesy PC screensavers were tragic because people didn't know better, they had sampled cheap sugar not the golden fruit yet they were easily pleased.

</comment><comment><USER15> Writing screen savers was a lot of fun too. There are some constraints, but you have a blank canvas. I wrote a rock climbing screen saver in the late 90s that used a very simplistic inverse kinematics model to animate a couple of climbers trying to find a route up your screen. I've long since lost the source, but it would be fun to rewrite, there's just nothing to target

</comment><comment><USER16> Most of my coworkers are in their 20's so when they see 8-bit winged toasters flying around my monitor they just assume it's birds or that I am a weirdo.

(Btw, I found it within the first few G search results for "after dark mac". Not sure how legal it is, but I couldn't find a anything in iTunes either. There should be a gog for screen savers).

<comment><USER17> XScreensaver has a mac port and an openGL toaster saver along with a lot of other goodies, should you be so inclined..

</comment></comment><comment><USER18> There was this wireframe rollercoaster screensaver that used to be available for Windows 3.1. Seems like I got it off AOL or Prodigy, or maybe even GEnie. It would only work on systems that had a math coprocessor (it worked on my -DX, but not my friend's -SX). I found that it you synced it up to this specific MIDI it would start and finish at the same time as the song, but for the life of me I can't remember what the song was.

It was amazingly primitive by modern standards, but blew my mind in the early 90s. I must have stared at that rollercoaster for hours.

</comment><comment><USER19> I don't know why more people don't use them. All those digital photos we take, we get to watch them on a photos screensaver on a computer in the kitchen. Just makes sense to do that if you have a centrally located interface.

</comment><comment><USER20> For some time, in a company where I worked, I used to have a screensaver called Mandala Screensaver. It was really colorful and good (IMO). Colleagues used to ask me for it.

Just did a search for "mandala screensaver" and got some results which look like it. Use caution if downloading from any site, though, and scan with anti- virus, etc.

After Dark was great too.

Earlier, Windows' MSDN docs used to have sample code for a screensaver (in C, I think), which could you adapt, and just change the graphics to your own. IIRC I tried it out some and made a simple one or two of my own. Good fun, and could be used for things like task reminders too.

</comment><comment><USER21> While my laptop screensaver is pretty boring (lock icon with blurred wallpapers), I have a lot of fun screensavers on my Kindle. Turns out the Kindle supports PNG (with transparency) screensavers, with the catch that you need to jailbreak to set them.

I made some transparent ones that show the book text what I'm reading along with some fun text.

For eg, this one asks people to guess which book I'm reading: [https://imgur.com/a/wIj4O](https://imgur.com/a/wIj4O). A few more are at [https://twitter.com/captn3m0/status/862587826801590273](https://twitter.com/captn3m0/status/862587826801590273) and the files are at [https://github.com/captn3m0/avatars/tree/master/kindle](https://github.com/captn3m0/avatars/tree/master/kindle)

</comment><comment><USER22> I wonder if the old skeleton adventures guy ever sold a copy:

[http://discuss.joelonsoftware.com/default.asp?biz.5.415151.5...](http://discuss.joelonsoftware.com/default.asp?biz.5.415151.56)

...surely one of the last big budget screen saver efforts. (Anyone else remember Joel's BoS forums? I wonder if those moderators are still moderating.)

</comment><comment><USER23> ... but they live on! Shameless plug for the Electric Sheep [1] (no affiliation except as a user) in [1]. Electric Sheep is truly a 21st century screen saver as new screens ('flames") are created algorithmicaly based on peoples' up/down vote.

I do remember learning Macintosh programming by writing an After Dark module to display randomly changing Truchet Tiles [2].

[1] [https://electricsheep.org/](https://electricsheep.org/)

[2] [https://en.wikipedia.org/wiki/Truchet_tiles](https://en.wikipedia.org/wiki/Truchet_tiles)

</comment><comment><USER24> > You can't consume a screen saver in an instant. You can't fast-forward or > rewind one.

+1 for this. This is exactly the experience that Apple TV's screensavers gets right. They're just gorgeous, but they're random, slow, and you can't pause or rewind or fast-foward (unless you download the movies manually, which takes the fun out of it).

</comment><comment><USER25> Johnny Castaway was awesome. I'd spend considerable time watching the antics. The other one I liked very much was for Win 95/98\. It was set in a undersea coral 3D environment and the camera was fixed on a great white shark you could control. It was fun swimming and looking around as the shark.

</comment><comment><USER26> Some of you may call me a heretic but I'd love to have the classic starfield simulation on my linux box.

<comment><USER27> xscreensaver probably has it

<comment><USER26> Afaik it hasn't. Therefore, I look at some colored, randomly placed smilies from to time time to elevate my mood:

function smilies(){ while true; do num=$(( $RANDOM % 80 )); for i in $(seq 1 $num); do echo -n " "; done; echo $(tput setaf $(( $num % 10 )) ) ; sleep 0.17; done }

<comment><USER28> XScreenSaver's closest equivalent is "rocks"; you might try something like this (unfortunately, when you ask for one color it's not white)

        rocks  -no-move -no-rotate -colors 1

</comment></comment></comment></comment><comment><USER29> If you want some nice screen savers that don't look like you're traditional stock screensaver, check out xscreensaver.

[https://jwz.org/xscreensaver](https://jwz.org/xscreensaver)

</comment><comment><USER30> Lost + Found: Screensaver- and demoscene-ish stuff that runs in your browser:

[http://www.arscalculanda.com/](http://www.arscalculanda.com/)

</comment><comment><USER31> The GL screensavers are great, but IMHO the best is Pixel City. Somewhere there's an article about how the author did it and the choices he made to allow efficiency and verisimilitude.

</comment><comment><USER32> I have always wondered - do Screen Savers actually _save_ anything?

Do they save significant amount of electricity? Do they make the screen "last longer"? What was their purpose?

<comment><USER33> Old CRT monitors were susceptible to burn-in. Having a screensaver or turning your monitor off when not using it was how you prevented burn-in.

</comment></comment><comment><USER34> > The ending of Borges's story, wherein the narrator is revealed as the slain > minotaur of Greek mythology, only reinforced the connection; to me, screen > savers have always afforded some tenuous connection to the afterlife. The > first one I can remember, on my family's household desktop, featured a > crimson psychedelia that overtook the screen's blackness, a kaleidoscope of > paisleys and helixes forever in a state of irresolution. Late at night, I'd > prepare an unhealthy snack and sit patiently in front of the monitor to > watch it, a child beseeching death.

... what?

</comment><comment><USER35> Johnny Castaway</comment>