Aluminum battery from Stanford offers safe alternative to conventional batteries - <AUTHOR> https://news.stanford.edu/news/2015/march/aluminum-ion-battery-033115.html

<comment><USER2> Likely as a direct response to the hype of this article (a few thousand comments in the related reddit thread), Elon tweeted:

        Battery "breakthroughs" need to state power *and*
        energy density (not the same thing), plus how long
        they last. They usually fail on energy.

[https://twitter.com/elonmusk/status/585185843649716225](https://twitter.com/elonmusk/status/585185843649716225)

<comment><USER3> It's important to note, though, that this kind of breakthrough is hardly the kind of invention that Musk is interested in. The world of batteries doesn't revolve around Elon and Tesla.

This _is_ sort of a big deal for those of us who design small electronic devices, not electric cars. Not having to factor in the potential fire hazard of a rechargeable battery for a keychain that the user has in his pocket or thrown in her purse or luggage is pretty useful -- and many such devices are thrown away along with their batteries, making them even bigger an environmental problem than they already are.

Even _if_ these devices offer no improvement in terms of energy density, they're still a pretty big deal. It's also worth remembering that Li-Ion and even Li-Po batteries weren't necessarily stellar in terms of energy density when they were discovered, either -- they reached their current status through quite some industrial development.

It's also helpful if they can really stay steady at 2V (before the discharge cycle goes down abruptly), that would simplify 1.5V designs _a lot_ , and help drive the space (and price!) further down.

<comment><USER4> The global energy storage market is and will continue to be driven by electric vehicles (no pun intended!) and consumer electronics. For these applications a faster charge/discharge is nice to have, a non-flammable material is nice to have but a high energy density is a MUST HAVE which Aluminum-ions don't.

My interpretation of a big deal is relative to the number of people it will positively influence. I wouldn't consider this a big deal, and in fact have heard the same claim at least once per month (ex. Titanium dioxide nanotube batteries [http://www.gizmag.com/quick-charge-li-ion- battery/34347/](http://www.gizmag.com/quick-charge-li-ion-battery/34347/)).

<comment><USER3> The last time I read about this, it was _expected_ that, at one point, in 2020 or so, electrical vehicle batteries are finally going to become a relatively important component of the battery market. There's a lot of money being poured into it because a lot of people are trying to move towards electric vehicles, but in terms of actual numbers being produced, the EVBs are dwarfed even by regular car batteries. Of course, EVBs are incredibly important, especially from an R&D perspective, but I don't think the market is being driven my them. My information may be a little out of date though.

As for consumer electronics, it's worth remembering that there are probably thousands of coin-cell operated keychains for every smartphone whose energy density capacity is not being satisfied by our current technology. There are a lot of consumer electronic devices for which current battery technology is OK -- better density would be an improvement, as it is everywhere, but it's no longer the hog it is for smartphones. For those, ease of recycling, lack of fire hazard and the slightly simpler circuitry really are important.

</comment></comment></comment></comment><comment><USER5> Those batteries have a lower energy density and power than the current li- ions, so you won't be seeing them in a phone, laptop or a car.

At the same time they are significantly cheaper to make and durable, which might make them a good solution for home energy storage - paired with panels for example.

<comment><USER6> Where does the article discuss the energy density?

I'm skeptical myself based on the number/style of claims they make, but I can't see any details.

<comment><USER5>
      "Our battery produces about half the voltage of a typical lithium battery," he said. "But improving the cathode material could eventually increase the voltage and energy density. Otherwise, our battery has everything else you'd dream that a battery should have. [..]

<comment> mark-r Voltage is not the same as energy density. It's one component of it certainly, but not nearly enough by itself to tell you anything.

</comment></comment></comment></comment><comment><USER7> With something like that, something like the a battery at home could become viable. Even with the lower energy density, it has a few properties that are really great if it proves to be correct: no degradation on cycles, inexpensive and reduced fire hazard.

So imagine the Tesla Home Battery that would currently be double the size, at a lower price point, and could theoretically last a decade or two?

So there are definitely commercial options if it ever proves truly viable and can be easily manufactured.

</comment><comment><USER8> I wish they bothered to state the expected energy density of these batteries. The fact that they didn't leads me to believe it's quite low compared to something like li-ion.

<comment><AUTHOR> I believe this battery's density is not super impressive that they didn't mention in in the article and the title of the article is about it's safety rather that fast charging time or capacity

<comment><USER9> Safe and "high-performance" as they claim are somewhat opposing; to a battery that can be discharged at high currents, with a high energy density, there's no difference between dissipating that energy in the usual load it's powering and through a short circuit of equivalent resistance. Even if the battery itself doesn't catch fire or heat up, all that energy still has to go somewhere.

<comment><USER10> Safe here doesn't mean "can't kill you". Instead it means this chemistry doesn't exhibit thermal runaway ([http://batteryuniversity.com/learn/article/lithium_ion_safet...](http://batteryuniversity.com/learn/article/lithium_ion_safety_concerns))

</comment></comment></comment></comment><comment><USER11> Ugh, getting tired of all these articles that may not be untrue, but ought to live in science journals, properly read as experiments with new battery tech that could deliver to consumers 5-10 years later, rather than get posted on social media as if they're consumer-grade product launches. I've read tens of breakthroughs in batteries and solar and so on, in channels that had no interest in the science behind it. HN is mostly the exception to the rule.

Anyway, is there a battery guy that could explain to the rest of us what the state of battery innovation is like the next 10 years? Is it realistic to expect some big leaps? I really don't know what to expect and it'd be awesome to get some insider insights as to what kind of consumer-grade products we may see the next decade :) Thanks in advance.

<comment><USER4> Battery innovation is actually bottle-necked by the fact that over 90% of all anode materials are produced in China. So global adoption is ultimately dictated if manufacturing in China follows suit.

Unlike Americans, where we innovate with a short time horizon, China innovates and invest for generations (25-100 years). Thus, once a technology has been chose to be invested in overseas, the Chinese go "all-in".

The current paradigm shift is that Chinese anode material manufacturers are switching from graphite based electrodes to silicon nanoparticle based electrodes due to the fact that silicon nanoparticles have a theoretical energy density of ~4,000mAh/g, a 10x increase when compared to graphite which is only ~350mAh/g. There has been substantial research on silicon based lithium-ion batteries in the US for the past 7 years; it is now hitting a true inflection point.

This is one of the few "break-though" battery chemistries that is now entering full scale production - Panasonic is producing an 18650 cell made of silicon used by Tesla. *(see also Amprius, Envia,XG Sciences, etc.)

Tl;dr battery adoption is limited by adoption by Chinese manufacturers which control over 90% of the global anode material manufacturing. Silicon nanoparticles are taking over.

<comment><USER11> Sweet, thanks! So what would that (e.g. 10x the energy density) mean for consumers in practice. Can we expect 4x the range on a car, or the lifetime on a phone? Is this something we'll see around 2020, later, earlier?

<comment><USER4> Unfortunately, there are several items that interfere with maximizing the 10X. Then anode is only ~16% of the entire battery, so increasing the anode energy density by 10X does not translate directly to 10X battery life. You must also consider the cathode, and separator which can hinder this. Remember the 10X is a "theoretical maximum". It is important to constantly increase the theoretical maximum energy density. Translating the theoretical to an actual energy density is about downstream processing - see how Tesla increases battery efficiency each year by make the processing more efficient.

Amprius was promising 10X increase, but once they reach ton-scale manufacturing they were only able to get a 30-50% increase. I am sure they will be able to gradually increase close to the 10X maximum by making processing more efficient.

tl;dr battery material processing into the actual battery is where most of the batteries lose there performance. Step (1) Increase theoretical energy density by finding new material; Step (2) focus on battery processing to approach theoretical maximum.

</comment></comment></comment></comment><comment><USER12> > "Our rechargeable aluminum battery generates about two volts of > electricity."

Now that's really good news. Stick it together with a 1.5V regulator inside an AA cell and never have problems with devices not accepting the 1.2V of a typical NiMH cell.

<comment><USER13> Alkaline batteries _start_ at 1.5 V, but they do not stay there. They are 1.4 V when they have discharged by about 10%. When they have discharged by around 30%, they are at 1.3 V. When they have discharged by around 60%, they are at 1.2 V. By 90%, they are down to 1.1 V. By 95%, they are down to around 1.0 V, and they pretty much nosedive from there.

NiMH also drop as they are discharged, but the curve flattens out for much of its range. A good charger should charge an NiMH to around 1.4 V. After about 10% discharge, it will be down to 1.3 V. By 30% discharge it will be around 1.25 V, and it stays around that until around 80% discharge. From 80% to 90% it will drop to around 1.2 V, and from there it will drop to around 1.0 V at around 99% discharge, and then it falls off a cliff.

Devices designed for alkaline batteries are designed to run fine on anything from around 1.0 or 1.1 V through 1.5 V, and so should be fine with NiMH, except if they have a battery remaining indicator that is hard wired to assume alkaline it won't give accurate estimates for NiMH because of the different shapes of the discharge curves.

</comment></comment><comment><USER14> Why do we see a new "super battery" article every two weeks or so?</comment>


Zombies apocalypse might wipe out humanity in 100 days - <AUTHOR> http://www2.le.ac.uk/news/blog/2017-archive/january/zombie-apocalypse-would-wipe-out-humanity-in-just-100-days-students-calculate <comment><USER2> I feel like the 90% susceptibility rate is high. Does this mean that if I come into contact with a single zombie, there is a 90% chance of it biting me?</comment>


HamWAN: A modern, multi-megabit, IP-based, digital network for amateur radio use - <AUTHOR> https://hamwan.org/ <comment><USER2> Since the site isn't that clear, the protocol is narrow-channel 802.11n on the amateur 5.9 GHz band. I wonder if the net throughput or range are any better than using the unlicensed 5.8 GHz band.

<comment><USER3> HamWAN founder here. The protocol is this:

[https://wiki.mikrotik.com/wiki/Manual:Nv2](https://wiki.mikrotik.com/wiki/Manual:Nv2)

And the sectorized spectrum plan is this:

[http://hamwan.org/Standards/Radio%20Frequency%20Engineering/...](http://hamwan.org/Standards/Radio%20Frequency%20Engineering/Spectrum%20Allocation.html)

When choosing a protocol for cheap long-range point-to-multipoint microwave, I wanted something that was TDMA'd and not CSMA'd like 802.11. That left me with WiMAX/airMAX/NV2 to choose from. WiMAX gear was overpriced, Ubiquiti's airMAX gear was very basic, and Mikrotik's NV2 was just right. Their $100 modems could run VRRP+OSPF+BGP+MPLS out of the box, offered more RF power than the competition, and could operate in the amateur part of the spectrum. HamWAN launched with 1.3W "Metal 5SHPn" modems on the air.

The speed advantage we get from these choices is not blocking on carrier sense, avoiding hidden-node collisions/retransmits, and operating in clean (high SNR) spectrum that allows for higher order modulations.

<comment><USER4> I have a full duplex system that I've developed. Advantages are:

1) Easy to add power amplifiers and preamps.

2) Works on any ham band.

Disadvantages:

1) Cost. Even the cheapest transmit capable SDR can't compete with consumer hardware.

2) Limited selection of bandwidths. 1.7, 5, 6, 7, and 8 MHz.

[http://www.w6rz.net/ofdm3.png](http://www.w6rz.net/ofdm3.png)

</comment></comment></comment><comment><USER5> Is it legal to use encryption over HAM in the US?

<comment><USER6> [https://hamwan.org/Administrative/Internet%20and%20Part%2097...](https://hamwan.org/Administrative/Internet%20and%20Part%2097.html)

</comment></comment><comment><USER7> Very cool I live in the Seattle area, looks like a fun thing to dink around with.

The web site has some broken links however:

Poynting 5.9 GHz 30 dBi grid or 2 ft solid dish antenna Mikrotik RouterBOARD Metal 5SHPn wifi radio

do not work

but over all really neat.</comment>


First past the post voting distorted British election - <AUTHOR> https://www.nytimes.com/2019/12/16/world/europe/uk-election-brexit.html <comment> core-questions Isn't it disingenuous to say that an election which proceeded according to the laws of the country, without any vote-counting scandal, is a "distortion"? This from the same people who like to say that the Electoral College is a distortion, as well?

If you don't like the result, it doesn't mean the system is broken. It might even mean that the system is doing exactly what it was designed to do.

</comment><comment><USER2> Well, this is going to get flagged pretty quickly since this orange site will suppress all debate due to it being 'off topic' and would most certainly keep the echo-chamber quiet of having an opinion.

In terms of this baseless claim from the New York Times about possible distortion in the election due to first past the post, there was a reason people were told to vote tactically to stop the one from getting a majority and the result was made clear regardless of that. Proportional Representation (PR) would still result in the Brexit Party winning anyway. So would that be a 'distortion' even under a different system?

Do they believe that the previous elections were also distorted, given that they were under FPTP? I can sense that this was written by those who cannot accept the result of a democratic process.

</comment><comment><USER3> Actual title of the linked article is "The UK Election Explained, In One Number".</comment>


Rämixx500 – An Open Hardware remake of the Commodore Amiga 500+ mainboard - <AUTHOR> https://github.com/SukkoPera/Raemixx500 <comment><USER2> Is this really 'open hardware'? The licence has complex and seemingly imprecise restrictions on right-to-sell.

The terms even become more restrictive as the Euro inflates!

<comment><USER3> As an outsider looking in for some reason this seems endemic to the Amiga community more than other retro systems. On the Atari ST even though it is a smaller community we have a completely open source GPL implementation of the operating system plus (EmuTOS) extensions for it (FreeMint, fVDI, etc.), and several hardware projects etc. with pretty open communities, if less of them.

On the Amiga side every project seems to have licensing which seems to always be defined such that someone thinks they're going to make a successful commercial project out of it Any Day Now. It's strange.

<comment><USER4> Yes! In my observations (also as a relative outsider), it has been like this at least from the early 1990s. Every little itty-bitty program for the Amiga cost money.

I am also reminded of how Carl Sassenrath thought it would be a good idea to release Rebol under only a proprietary license. The Rebol project, which initially had (IIRC) enormous hype and mindshare during development, tanked, and I think this was large reason why. This is despite the time being just right for a new good scripting language: PHP and Python were just beginning, and Perl was slowly dying.

(Rebol was eventually released under an Open Source license in 2012, when nobody cared about Rebol anymore.)

</comment></comment></comment><comment><USER5> Someone already made a new Amiga mainboard and have been selling them, the Retro Man Cave did a series on building "The worlds newest Amiga" [https://www.youtube.com/watch?v=ECu6i6WR7](https://www.youtube.com/watch?v=ECu6i6WR7)

So remaking the mainboard is certainly possible, getting the custom chips seems to be much harder. The chips can't be consider that advanced anymore, so I wonder if you can get some Chinese manufacturer to duplicate them, if you had working chips for them to copy?

<comment><USER6> > _The chips can 't be consider that advanced anymore, so I wonder if you can > get some Chinese manufacturer to duplicate them, if you had working chips > for them to copy? _

I believe reverse engineering the silicon still takes serious amount of money. But is it really required? Reimplementing the chips from scratch with FPGAs is a much more practical option, since everything is already pretty much documented by the simulator community.

And it appears that someone has already done something on it.

* [https://en.wikipedia.org/wiki/Minimig](https://en.wikipedia.org/wiki/Minimig)

> _Minimig started around January 2005 as a proof of concept by Dutch > electrical engineer Dennis van Weeren. He intended Minimig as the answer to > the ongoing discussions within the Amiga community on implementing the Amiga > custom chipset using an FPGA. The project 's source code and schematics were > released under version 3 of the GNU General Public Licence on 25 July 2007. > _

<comment><USER7> Why even go that far? Software emulation is good enough for the vast majority of the users.

What would be very interesting would be a way to compile software emulators that have very consistent sets of rules for how hardware is expressed (such as MAME) into an FPGA.

<comment><USER6> > _Why even go that far? Software emulation is good enough for the vast > majority of the users._

To replace a failed ASIC on a physical Amiga (or a Amiga replica), which will be unavailable in the future. Don't forget the original topic of the article is a recreation of the physical Amiga.

<comment><USER7> It gets trickier when you need to replace one of the ASICs, but if all you want is the physical object and the user experience, without much regard for the internals, software emulation combined with support for the user-facing aspects (keyboard, mouse, screen perhaps), the approach taken by the c64 mini people with their larger breadbox model seems interesting. Having a drop in replacement for aging Amiga motherboards that can run the software and replicate the user experience without necessarily replicating the underlying hardware seems to be a good 80/20 approach.

There will be people interested in the accurate replication of the hardware. For those there is no replacement for the real thing or something very close to it.

</comment></comment></comment></comment></comment><comment><USER8> This is especially useful as an A500+ replacement, as the A500+ comes with one of those varta barrel batteries that tend to leak and destroy the boards.

</comment><comment><USER9> Great README. I like these projects where the documentation clearly shows the passion and expertise involved.

</comment><comment><USER10> So you will still need to source the chips, CPU won't be major issue, though denise, paula, gary and agnus from what I can tell will still be a sourcing issue. More so as they are not made and only made for the Amiga, so be harder to source.

THough as there battery on the Amiga and many old kit are known to expire in epic ways and leak all over the place, which will quickly kill the board, and depending how stored - will take out some of the chips as well. So for many this will be good for those with access to those chips, those chips will be the hard part sourcing wise if you need those.

</comment><comment><USER11> There are a lot of mentions that this is untested, are there any hints about why the author hasn't gotten to try it out yet? Maybe it's unfinished?

</comment><comment><USER12> If you have an Amiga (or any other old electronic device), then you should remove any old batteries inside. Batteries leak and destroy electronics.

</comment><comment><USER13> > This mainboard was designed with reasonable - not maniacal - accuracy to the > original design.

And that's when I approved their effort.

</comment><comment><USER14> I guess the name is a pun on "remix".

<comment><USER15> Rämä also means "broken" on Finnish

</comment></comment><comment><USER16> Somebody needs to order a batch of these printed from China and try it out!

<comment><USER17> Actually, China is often a less-than-ideal option for prototypes, unless you live in or near China.

I can't talk for the US, but in Europe where I live there are some amazingly cheap options for prototype batches. You could probably order a prototype batch (2~3 boards) for around 150 euro with a 7 day delivery. Ordering from China from EU will probably take around a month to arrive.

<comment><USER18> Any you'd suggest in particular? I often find it cheaper to get 10 boards made in China than to get 1-2 made anywhere near remotely local. (And DHL from China isn't a lot slower than DHL from Germany)

<comment><USER19> Germany isn't a bad place to get 100 - 1,000 PCB's of this magnitude made .. perhaps you'd be better off doing the hardware assembly in Hungary or Croatia right now, albeit for the current cruise notwithstanding, of course ..</comment></comment></comment></comment>


'Shut up and drive'? Option for Uber riders to avoid small talk divides drivers - <AUTHOR> https://www.theguardian.com/us-news/2019/may/17/shut-up-and-drive-option-for-uber-riders-to-avoid-small-talk-divides-drivers <comment><USER2> Hrrm... I use Uber / Lyft a lot, and I usually wind up chatting with the driver the entire ride, with the exception of the occasional driver who either A. clearly doesn't speak much English and can't really carry on a conversation, or B. very obviously isn't into talking.

But, _sometime_ I get in the car with the mindset of "I'd really like to just plug my earbuds in and zone out and listen to some music / this podcast / etc.". On those rare occasions, I find that just letting the driver know "Hey, I'm going to have headphones in, listening to a book" is usually all it takes.

Still, I guess this could be useful for some people.</comment>


ScoutBot: Automated network scanning, environment assessments, and more - ko-da-za http://lansec.net/project/scoutbot/

<comment> ko-da-za Hello everyone, LANsec is proud to announce the initial release of ScoutBot - an automated network scanner that can be used as a quick and dirty scanning and network assessment tool, a packet sniffer, or a drop box.

Here are some of the features:

        Scans the target network for open ports
        Sniffs the wire
        Identifies Windows environments
        Identifies dynamic routing protocols
        Identifies devices by scanning with SNMP
        Identifies DHCP servers and their advertised scopes
        Checks egress filtering
        Identifies the network's outside IP address
        Performs a WHOIS to identify ISP
        Performs a traceroute to the outside
        Accepts static IP addresses
        Accepts DHCP
        Pre-configure for a completely automated attack
        Works on any Raspberry Pi, no setup necessary
        Free

Basically, you pre-configure the scan and then just plug the Raspberry Pi into the target network. This tool comes as a preloaded IMG which is ready to use. We also sell pre-configured Raspberry Pis.

Project Page [http://lansec.net/project/scoutbot/](http://lansec.net/project/scoutbot/)

SourceForge Download Page [http://sourceforge.net/projects/scoutbot/](http://sourceforge.net/projects/scoutbot/)

Other Projects [http://lansec.net](http://lansec.net) A Kodaza Managed Services Project</comment>


Did anyone apply to TechStars Summer 2011 program? - <AUTHOR>

Notices are sent out this evening. Did you apply? <comment><AUTHOR> Here -- I created one: <http://www.chatzy.com/937035240686>

<comment><AUTHOR> 9 People chatting now! Join us

</comment></comment><comment><USER2> We applied. Still waiting on the results to come down.

<comment><AUTHOR> GL! We should get them around 11 EST. Do you know of a discussion board for TechStars applicants - like YC applicants created last round? Someone should create one.

</comment></comment><comment><USER3> Check your emails now, you should have heard by now!

</comment><comment><AUTHOR> David Cohen joined the chat!!</comment>


Why a modern language like Kotlin makes a developer more productive - <AUTHOR> https://kotlinexpertise.com/kotlin-productivity/ <comment><USER2> I've been using Kotlin lately, and while I do enjoy it, I'm not sure that it actually makes me more productive compared to Java. I haven't yet used coroutines, but if anything in Kotlin is a killer feature, I suspect coroutines are it. The lack of a decent way to write concurrent/async programs lead Java programmers to use over-complicated solutions like Rx.

Rx is cool and definitely a good solution for some cases, but if what you really want is just to talk to your database without blocking the thread, then it's way overkill. Just give me async/await like C#.

I'm a bit concerned about Kotlin code tending towards being overly concise and clever, especially with the IntelliJ refactoring suggestions that constantly pop up to entice me to make my code a few characters shorter. Of course, "my IDE told me to do it" is no excuse, it's my responsibility to write clear code.

One of the things I like about Go is the tendency to write straight forward code that reads easily from top to bottom. Java has a bit of this tendency too (if you forego the kind of all too common libraries that require you to instantiate 14 objects to do even the simplest things). Java has already become much less noisy with type inference for local variables.

Java code doesn't have to look like this, Java programmers are doing it to themselves:

        HttpURLConnection httpConnection = (HttpURLConnection) new URL(url).openConnection();
        httpConnection.setRequestMethod("POST");

</comment><comment><USER3> A comparison needs a baseline, and the one missing from this headline is: ...than Java, the author's first language.

I don't think anyone would be surprised by that.

</comment><comment><USER4> In 5 years, we're going to have another post like FooLang is 3x better than Kotlin. And in another 5 after that, BarLang is 4x better than FooLang. And so on. In 30 years, we'll finally just be at LISP.

Jump ahead 30 years and just use LISP already.

<comment><USER5> Shhh.. Don't give our secret out! I plan to be gainfully employed 30 years from now, just in time for the hot thing then(with a resume showing 35 years of experience). Beat it, young uns!

</comment></comment><comment><USER6> There are languages for teams, and than there are languages for individuals.

Kotlin is a great language for individual productivity, but _can_ be terrible for teams.

You can do the same thing in Kotlin in so many different ways that it makes hard to read other's code, and a lot of time gets wasted in code reviews over preferences.

Compared to that, limiting languages like Go are great for teams since well there's only one (or max, two) ways you can iterate a collection, as compared to what, like, seven ways in Kotlin.

<comment><USER7> Problem is, in most companies "languages for teams" means "languages for the JVM". What is the Go-like team language for the JVM?

<comment><USER6> I agree with you when you say in most companies "languages for teams" means "languages for the JVM"

But I am curious, is that just out of familiarity or are there things that JVM ecosystem has that are just not there elsewhere?

I understand that some of the distributed data processing ecosystem (Kafka et al) are _built_ on JVM but there's nothing inherently stopping people from using these things from other languages.

</comment></comment></comment><comment><USER8> > "How Kotlin makes me a more productive software developer"

The title is categorical, but the content does not support that claim.

The content gets no further than — " _I feel_ much more productive with the language."

Whether or not the author is more productive seems unknown (and apparently unexamined).

</comment><comment><USER9> I remember I took a serious look at Kotlin for some hobby projects. On the paper it was a great language with many interesting features but it ended up being just another kid in the current generation of new high level languages, nothing else. The biggest turn off for me was to see that its main selling point, JVM, made it actually a problem since most of the shared code, developers, terms and ideas came from the Java community and turned it in just "a better java" on top of the java ecosystem... which is what Kotlin is after all.

<comment><USER10> But why is that a problem. So you get access to a big community and lots of libraries.

Are you working in an environment where you can't run a JVM at all for some technical reason? Maybe very constrained? But then you wouldn't be able to take many libraries regardless of platform. And in fact Kotlin also has Kotlin/Native which generates small(ish) native binaries.

There is also SubstrateVM which can create standalone, statically linked, natively pre-compiled binaries from Java/Kotlin programs. It even has an ARM backend.

</comment></comment><comment><USER11> What this article doesn't mention is that not only can you replace Java with Kotlin, you can replace JavaScript too. I've even gotten rid of ugly bash scripts with judicious application of kscript. And soon, Gradle scripts too. (Good riddance, Groovy.)

But compile-to-JVM is compelling enough on its own. The Java ecosystem that comes for free with Kotlin gives you an incredible amount of power. A massive collection of battle-tested libraries, the excellent performance of the JVM, and tooling. Beautiful, productive tooling.

<comment><USER12> I'm wondering about diving into the jvm ecosystem. Could you give some examples of the beautiful productive tooling you love ?

<comment><USER10> Most obviously IntelliJ itself. I read a lot of praise for Visual Studio Code these days and then when people describe what they're so in love with it for, they're describing basic features IntelliJ had years ago. It's quite clear a lot of VSCode users have come from Atom or Sublime or just console text editors and have never really encountered a proper IDE before.

VisualVM is very helpful. You can attach to any running Java program on your system and get graphs of CPU, memory usage with garbage collection information, you can take a snapshot of what every thread is doing with reliable stack traces, you can take a heap dump and then query it with the "Object Graph Query Language" or visualise it with tables and charts to understand where your memory is being used. You can do deep CPU profiling or lightweight CPU sampling. It understands mixed language programs and can show you the heap and stack in ways that make sense for not just Java and Kotlin but Ruby, JavaScript, etc.

[https://visualvm.github.io/graal.html](https://visualvm.github.io/graal.html)

Of course all these features work transparently over the network too.

Then there's Mission Control. It's a profiling system so fast and low overhead you're intended to keep it running all the time even in production. You can connect it to a live server and profile without disturbing users.

Then there are ultra-low-pause garbage collectors like ZGC, Shenandoah. Whilst Golang offers such a thing too, it comes with the _major_ caveat that it has a very simplistic design that creates enormous CPU and memory overheads. The Java ultra-low-pause time GCs have more sophisticated designs that impose much lower overheads on running applications.

Then there's SubstrateVM, which can turn Java programs into standalone native executables of small size, without any JVM dependency. This comes with some caveats: execution speed is slower and you can't do dynamic classloading, which quite a few advanced frameworks do require. You also have to tell it in config files what you want to reflect, so it doesn't throw away code accessed only reflectively. Also the garbage collector is fairly middle of the road and not so sophisticated as what you get in HotSpot. After all the regular JVM is large because it's giving you a lot of stuff. But it can be very useful for command line tools.

Talking of which, check out the combination of PicoCLI, Apache SSHD and JLine. They integrate together well to enable you to easily and rapidly create interactive shells for your program, accessible via SSH. There are of course many other powerful libraries available.

Finally you may like to explore things like the Chronon time-travelling debugger, which records what your app does so you can step forwards and backwards through a debugging session.

</comment></comment></comment><comment><USER13> Kotlin is amazing and I hope it will grow more and more popular. I'm in love with this language, I hope it can take over for web development and backend.

</comment><comment><USER14> I have to admit I haven't actually used kotlin, but I've taken a look at it and could see its lack of verbosity in syntax alone making it more productive java. I mean typing half or even a third of the amount of characters kind of implies that right?

I had to write something in java recently for the first time in a while. I'd forgotten just how much you have to type for the most basic things.</comment>


Ask HN: Is commercial nuclear fusion feasible in the near future? - <AUTHOR> <comment><USER2> Not really, depending on what method is used there are several outstanding issues: Assuming magnetic plasma confinement: \- plasma stability issues: it is not really know how to maintain a stable plasma in an energy efficient way that can maintain a reaction that produces a net positive energy generation. When hearing about interesting exotic fusion reactors, one should wonder about the plasma stability (ie the plasma goes into a failure mode and hits the wall or leaks out of the confinement). \- materials issues: the materials don't really exist that can sustain extended periods of radiation damage. This is also a limiter in many exotic fission reactor designs.

Laser confinement also has some issues.

So conceptually we know it should work but maybe a couple of the needed technologies are not ready. This is similar to Leonardo da Vinci knowing that helicopters should be possible in the 1400s but many of the required supporting technologies weren't ready.

</comment><comment><USER3> Technically it has been feasible for some time, the challenges are really political and social. I switched majors away from nuclear engineering when I realized that; political problems are not my thing.

<comment><USER4> Which fusion technology do you think is most promising, if all the sociopolitical problems could be overcome? How much would it cost to make it happen?

<comment><USER3> Oh I misread this as fission. Sorry about that.

</comment></comment></comment><comment><USER5> We don't even know if it's possible to build any fusion plant smaller than a star, let alone if it's possible to run one at a profit.</comment>


Ask HN: CSS overrides to make HN more readable? - <AUTHOR> I am having increased difficulty reading HN due to the font, background color and margins.<p>"reader modes" break threading to a degree, and zooming is a hassle, so I'm curious as to whether there are Stylish or custom CSS overrides that would make it easier to read HN.<p>("dark mode" would probably help, but the biggest issue I have are long lines in desktop browsers...) <comment><USER2> Looks like there are some on userstyles.org. You can use it with the Stylus extension and adjust as fits your preference. [https://userstyles.org/styles/browse?search_terms=hacker%20n...](https://userstyles.org/styles/browse?search_terms=hacker%20news)

</comment><comment><USER3> I use greasemonkey in Firefox. Tampermonkey does the same thing. See my comment [https://news.ycombinator.com/item?id=21191539](https://news.ycombinator.com/item?id=21191539) for changing font sizes. It's your browser (mostly). You can change anything you want (mostly).</comment>


Aeron: Open-source high-performance messaging [video] - <AUTHOR> https://www.youtube.com/watch?v=tM4YskS94b0

<comment><USER2> 49 nanoseconds, average, on a Fujitsu TX 200 with a Xeon E5-2420 hexacore proc, for adding 10,000,000 lines from a text file, at an average rate of 985,000 lines / second, at 315 bytes / line. Impressive.</comment>


Real-time visualization of Facebook friend activity - <AUTHOR> http://insanecrazyfriendwall.herokuapp.com/

<comment><USER2> I think I figured out how it works guys - it's going through all friend activity from when you opened your account, catching up. If you leave it open long enough it slows down to a more believable rate.

</comment><comment><USER3> There's no way that this is real. The activity is way too fast. My friends aren't even close to this interesting.

<comment><USER4> I don't know. I looked at the source code, and it pulls in from Facebook. I think it might be legit.

</comment></comment><comment><USER5> "I don't even see the code anymore. I just see blonde, brunette, redhead..."</comment>


Baidu beats Google in China - <AUTHOR> http://www.businessweek.com/magazine/content/10_47/b4204060242597.htm?chan=magazine+channel_top+stories

<comment><USER2> Not tough when the government periodically shuts down your competitors and redirects their traffic to your site.</comment>


Ask HN: Are you looking for a new job? Leave your Twitter handle here - <AUTHOR> <comment><USER2> I can of misread this and thought it meant if you're looking for a new job, delete your Twitter account. Strange times.

<comment><AUTHOR> Thanks for the heads up

</comment></comment><comment><AUTHOR> I want to help, retweeting/sharing profiles looking for new opportunities mainly due to pandemic.

Will also try to engage other potential Twitter users with thousands of followers to do so.

My Twitter handle, just in case: @leonagano

<comment><USER3> really nice and inspiring story, just read your blog! following you on twitter and also looking for new opportunities. @rafaelmelhem</comment></comment>


Migrating from PostgreSQL to GridDB - <AUTHOR> https://griddb.net/en/blog/migrating-from-postgresql-to-griddb/ <comment><USER2> > GridDB is an In-Memory NoSQL Database

> GridDB utilizes a "Memory first, Storage second" structure. "Hot" data is > kept in-memory, allowing for much faster writes/reads on the most-used data.

If that would result in being a in-memory database, than nearly 95% of all databases in the world would be a "in-memory" database.</comment>


Show HN: Nextprev.it: Share PDFs & control what page the viewer sees in realtime - <AUTHOR> http://www.nextprev.it/ We just launched Nextprev.it — A project we put together over a couple of days.
The project stemmed from a frustration with PDF file transfers and screen-sharing when giving design presentations to clients through Skype.
Please give it a go and let us know what you think - @nextprev
Any feedback is greatly appreciated! <comment><USER2> I've been testing this for the past week or so - used it for a couple of client presentations and it's absolutely fantastic.

</comment><comment><USER3> This is very cool!

It's not really the same purpose but a cool feature to me would be a sponsorship / deck feature so you can send it out to see who views it individually. So you can see which potential sponsors actually look at the info you send, and which pages.

</comment><comment><USER4> I will start using this immediately. Question- do the links expire after a certain amount of time, or can I control how long the PDF is available at that link? That would be an important feature to me (as a way to protect confidentiality etc)

<comment><USER5> Yes, we'll delete files and links after 24 hours. That's not implemented yet, but will be happening by the end of the weekend.

We're also planning some pro features like account-level permissions, and we could serve the PDF file dynamically from our servers instead of storing it on S3.

I'd love to know if you have any precise security requirements, feel free to email me: email@example.com

</comment></comment><comment><USER6> Very very nice, congratulations ! Could you share a little bit of the technologies used ? How to you make every browser go to the same page ? Do you think this would be possible with video as well, like making sure everybody is at the same point on a video ?

<comment><USER5> We host the PDFs on AmazonS3, and use PDJ.js to render them on the page. To keep everyone on the same page, we use websocket events via Pusher. Video would be quite different, so we don't have any plans for that now.

</comment></comment><comment><USER7> Great demo! I'd make the "You see this" "They see this" images larger. And add a favicon, if only so it's more clear that the "<>" at top left is a logo (and when the user clicks on the logo, go home).

</comment><comment><USER8> Doesn't seem to work in Chrome. This is what I get: <http://i.imgur.com/ps7LT.png>

<comment><AUTHOR> Ah, that sucks. Sorry about that. We haven't run into this.

This may be due to your PDF export settings or the library we're using to render the PDFs.

We'll look into it.

<comment><USER9> Are things broken right now? No matter the browser, when I try to upload one of my small PowerDot created presentations (90k), it just pops a banner up that says "Thanks for signing up" and nothing happens.

<comment><AUTHOR> Yea, that's totally our fault. We added an email signup form, but screwed something up. It should be working now!

<comment><USER9> Thanks, it looks like it works now. This will be great for my research group!

</comment></comment></comment></comment></comment><comment><USER10> Neat idea. It'd be nice if you could see how many other people were viewing the presentation and perhaps even chat with them.

<comment><AUTHOR> Definitely a feature we have planned for the near future as well as some handy pro features.

Thanks for the feedback.

</comment></comment><comment><USER11> Very nice. I did something similar as well a couple of weeks ago but didn't publish it. Good job!

</comment><comment><USER12> Love it. Great use of websockets to create a simple solution to a common problem.

</comment><comment><USER13> Great work Alex! This looks very useful and promising.

</comment><comment><USER14> Pretty awesome, does it come in black?</comment>


A better rat trap improves the lot of low Hindu caste  - <AUTHOR> http://www.sfgate.com/cgi-bin/article.cgi?f=/c/a/2008/01/06/MN32TVLQQ.DTL

<comment><USER3> This is a great thing, and it's wonderful how they're embraced it, and it's making a difference.

But it makes me wonder: what prevents people from coming up with such inventions on their own? Is a lack of raw materials (steel)? A lack of technical mechanical engineering expertise? A lack of an inventive mindset? It seems there are so many problems out there that could be solved, with a little effort and ingenuity sustained over time.

Actually, I think it is probably as simple as when you are struggling to find your family's nest meal, you just don't have the capacity to invent. Not the spare energy, not spare resources, not resources to invest in learning - <USER2> no buffer to make it reasonable to take risks. What's my excuse?</comment>


A Scourge Is Spreading. M.T.A.'s Cure? Dude, Close Your Legs - <AUTHOR> http://www.nytimes.com/2014/12/21/nyregion/a-scourge-is-spreading-mtas-cure-dude-close-your-legs.html?_r=0

<comment><AUTHOR> Even if you don't bother to read the article, at least enjoy the amusing (and hopefully effective) signage they're going to put up:

[http://static01.nyt.com/images/2014/12/21/nyregion/21MANSPRE...](http://static01.nyt.com/images/2014/12/21/nyregion/21MANSPREADING4web/21MANSPREADING4web- superJumbo.jpg)</comment>


Is it bad for prod systems to integrate via a data lake? - <AUTHOR> Help me either win or avoid an argument:
I work at a major tech company. We have a massive data lake for analytics / data science and essentially all of our systems send logs to it. A pattern that I'm seeing pop up sometimes is teams building systems that read other systems' data from the data lake.
This screams antipattern to me. The data lake introduces significant latency and many of the log streams in it have little to no SLA guarantees. If you read another team's logs from the lake without their knowledge, they could move, delete, or make breaking changes to their logs and not know that they've broken you until you complain. I think it's a lazy way for teams to avoid calling each others' APIs or even talking to each other to discuss integrations. I want to tell them they're making a very big mistake by doing this.
Is there a good argument for applications/services reading each others' logs from a data lake instead of integrating through APIs? Is it a valid pattern? Maybe if the data needs expensive processing before the client can use it and latency isn't important? Or just because it's easier/faster than a proper service integration? <comment><USER2> Late to the party, but someone at the office shared [https://storage.googleapis.com/pub-tools-public- publication-...](https://storage.googleapis.com/pub-tools-public-publication- data/pdf/43146.pdf) , which reminded me of this question. You're right that the system consuming data from the lake is taking on debt by building a system around what is by design an unstable signal. The consumer is also attaching technical debt (in the form of visibility debt) to the producer, since changes to the data fed into the lake now have an undeclared consumer that may break if the data format or cadence changes in ways that seem innocuous to the data originator. At the major tech company where I work, this would result in someone on the team responsible for the data originator getting paged.

I would recommend reading the paper linked to above and sharing it with teams that are taking the approach you describe.

<comment><AUTHOR> Thanks for the link, this is a good formal description of the one main problem I'm seeing with this type of architecture (undeclared consumers) as well as several others I hadn't thought of.

</comment></comment><comment><USER3> My company uses a common data lake for various teams and organizations to share their datasets. We enforce schemes, and owners are responsible for the correctness and quality of their data, although some consumers usually build some tests in their downstream pipelines to validate data sets from the data lake.

It has its problems. How do I know a data set I am consuming is complete or correct? As a consumer, how do I even know what to test for? Although producers are technically on the hook for the quality of their data, it is not clear which data sets in the data lake have what level of support from the owning team. You probably want to get in writing from the producer that it's safe to consume or take a dependency on their data.

Personally, I don't want to vend a service to provide data sets, especially when they're in hundreds of gigabytes or terabyte scale (for analytical use cases). I don't want to support all that infrastructure and have engineers supporting he service logic itself, and so a common abstraction makes sense, at least on paper.

One of my recent projects involved using this data lake as an integration point. The specific data lake implementation means that for consumers, the exact time that a dataset (after it is written to the data lake) is available for querying is non deterministic, and that resulted in a lot of friction.

What is the size of the logs you want to consume? Are you able to query specific log lines by wildcard or key? How fast do you need the data once it has been produced? Are you willing to support your own infrastructure (hosts, configuration, patching, etc)? If there is a datalake, is it a common/managed service, and so is the cost of supporting your own service worth it?

Does your data lake enforce schemes on write? If so, how easy is it to change schemes later on, and how does that impact consumers? How do you identify or track consumers of the data, whether you're using a data lake or a service? Is there support for versioning?

</comment><comment><USER4> IMHO, it's an antipattern in almost every way. Being "common" doesn't make it "smart". The preprocessing argument is invalid for a lake, and the upfront "integration" time savings are cannibalized later by maintenance and risk overhead vs. proper service integrations. And everything you mentioned aside, there's security and DPP risk. Lakes should not be used for collaboration between systems - that's what lakeshore data marts are for, at worst, or real service-to-service APIs, ideally. (We have a good word for this in German: Datensparsamkeit. See also [https://martinfowler.com/bliki/Datensparsamkeit.html](https://martinfowler.com/bliki/Datensparsamkeit.html)). The people and services regularly using the lake should be data scientists and analytics folks, probably not prod services.

<comment><AUTHOR> Security and privacy risk are also good things to consider, thanks. Our data lake does have processes and protections in place for this (GDPR tagging and deleting, retention policies, security groups for access control) but just because my streams are compliant doesn't mean downstream consumers are.

</comment></comment><comment><USER5> > You should use a data lake for analytic purposes, not for collaboration > between operational systems. When operational systems collaborate they > should do this through services designed for the purpose, such as RESTful > HTTP calls, or asynchronous messaging. The lake is too complex to trawl for > operational communication. It may be that analysis of the lake can lead to > new operational communication routes, but these should be built directly > rather than through the lake.

Source: [https://martinfowler.com/bliki/DataLake.html](https://martinfowler.com/bliki/DataLake.html)

</comment><comment><USER6> I know I'm late to this discussion, but you're right on the money. It's slow, you have no idea how often the data is refreshed. There's no SLA's, schema could change. It really should be for analytics and not transactions. It's the lazy way and folks will pay for it down the line. We got away from shared DB architecture and this is nothing more than a backdoor to the same old way of doing things but in a worse way.

</comment><comment><USER7> How much data are we talking about? If the data is small enough, then an API is better. If you need to read a lot of data GBs/hour, then you probably want to read data through the data lake.

However, it's still possible to define schemas, version, and set SLAs on the datasets in the data lake. If you don't do that, then you are going to have to fix this whenever the upstream dataset changes. This is even better if the upstream dataset is produced by a team and they have no idea that they are other people consuming their datasets. Welcome to the Wild West of Data Engineering.

<comment><AUTHOR> Yeah, my first two questions were: "Why don't you get the data from the upstream team's API? Have you discussed this design with them?"

It also happened to me once before--a production system took a dependency on one of my streams in the data lake without my knowledge, and no SLA, then when I made a change to it, they came out of nowhere, blowing me up with e-mails and meeting requests. I told them to go pound sand.

</comment></comment><comment><USER8> Integrating through a database or other data source is a time tested method of linking together services. For some services in critical paths that might not be the best way to do it but the pattern you've described here sounds pretty normal and standard. What exactly do you want to do to solve this problem? What's a "proper service integration?"

</comment><comment><USER9> the most effective way I've seen is to have a specific data eng team which builds core tables, drawing on data from different teams. they work with producer teams to ensure data quality and completeness, and work with consumer teams to understand what they need to see from data.

but this depends on what your use cases are (e.g. online vs offline).</comment>


I found two identical packs of Skittles among 468 packs - <AUTHOR> https://possiblywrong.wordpress.com/2019/04/06/follow-up-i-found-two-identical-packs-of-skittles-among-468-packs-with-a-total-of-27740-skittles/ <comment><USER2> >So, what's the point? Why bother with nearly three months of effort to collect this data? One easy answer is that I simply found it interesting. But I think a better answer is that this seemed like a great opportunity to demonstrate the predictive power of mathematics. A few months ago, we did some calculations on a cocktail napkin, so to speak, predicting that we should be able to find a pair of identical packs of Skittles with a reasonably– and perhaps surprisingly– small amount of effort. Actually seeing that effort through to the finish line can be a vivid demonstration for students of this predictive power of what might otherwise be viewed as "merely abstract" and not concretely useful mathematics.

<comment><USER3> Can I ask the point of quoting a snippet of the article without comment?

<comment><USER4> I admit; I don't read most articles. I prefer the HNer snippets like this. I generally find HN comments to be more cogent and provocative than most editorial nonsense. So, thank you parent, for choosing and posting a snippet which you considered worthy.

<comment><USER5> Yes, it's pretty obvious that lots of HN readers don't actually read the article.

<comment><USER6> At this point, it should be an official HN rule: "do not read the article, go directly to the comments and speculate what the article is about based on the character-limited title"

<comment><USER7> For many of us, submitted articles are just social objects for provoking a discussion on particular topics. Also, sometimes the articles are really worth a read, other times they're garbage; usually they're something in-between. Going straight to comments is the fastest way to discover which is which.

Commenting without reading the article is fine. Speculating about what's in the article without having read it is a problem.

<comment><USER8> so we're reddit, just without the memes?

that does seem a bit disappointing. I think anyone simply quoting the article needs to explicitly show that that is the case because it was very misleading.

still this article was fascinating to me, from the idea that someone would go to the effort to the results of such effort. then top off my fascination with the idea of trying elsewhere in the country should there be more than one manufacturing point or trying to buy up a production lot as seeing the results of that as well.

Now how many packs of M&Ms? There are six colors there. If you go with the peanut version it probably is worse because they you have the variability of the peanuts which would make the chances of encountering packs with more variation in just the number of candies.

<comment><USER7> > _so we 're reddit, just without the memes?_

Kind of? And with slightly higher average standards of discourse? And I believe this is actually a compliment.

In my experience, HN comments under article are almost always more useful and more informative than the original article. The same is the case with various subreddits. When I read, say, /r/SpaceX, I also immediately jump into comments, as there is better quality info there.

This applies to mainstream news stories in particular. On HN, there's a good chance you'll find someone who was - or knows someone who was - involved in the topic first-hand, and who then proceeds to debunk various nonsense a typical news story contains. That's a huge value-added.

> _I think anyone simply quoting the article needs to explicitly show that > that is the case because it was very misleading._

Sure, I think making it clear what text is quoted (and from where) should be an obvious rule. And it doesn't have anything to do with whether or not others read the article; it simply saves brain cycles trying to understand the comment.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> I'm so glad to see there are people who think like me. I have spent thousands of hours doing various projects like this.

The only difference is, I normally spend my time automating the process with computer vision (a skittle sorter/counter wouldn't be that hard to build; opening the bag is harder than identifying colors). And then I never really finish the project.

<comment><USER10> You might enjoy this :) [https://willemm.nl/mm-skittles-sorting- machine/](https://willemm.nl/mm-skittles-sorting-machine/)

<comment><USER11> That reminds me of this post about sorting 2 tons of lego: [https://jacquesmattheij.com/sorting-two-metric-tons-of- lego/](https://jacquesmattheij.com/sorting-two-metric-tons-of-lego/)

<comment><USER9> that post was a revelation to me and led me down the path of making object detection classifiers for various projects. I was shocked at how easy it was to build a classifier that was as good as a human, but faster (or more accurate than a human in the same amount of time).

I ended up learning there is a whole area of computer vision in industry- relatively "boring" stuff like just looking at a line of bottles going by. I went on a tour of the money making machines in DC and they stream sheets of bills by industrial vision cameras to detect whether the bills are within QC.

Still amazes me people get paid to do this. To me it's just like a big fun game.

</comment></comment></comment></comment><comment><USER12> The experiment is great hands on math but I would have enjoyed a discussion of variance versus expected value and the difference between short and long term averages... it's too easy to infer that everything is great because he was lucky enough to get within the target range but the likelihood of that occurring is not actually that high and was only implied by the shape of the Monte Carlo distribution in his previous post. When such experimental results are this conveniently "accurate", amateurs in the audience may take away the kinds of wrong inferences which create "it's a hot day so must be global warming" type of logical inaccuracies.

<comment><USER13> Author of the article here; this is a great point. This experiment initially stemmed from a nice analytical solution to the problem of computing the expected value (via generating functions as described in the post). Computing other moments, let alone the entire distribution, required some Monte Carlo simulation, as shown at the end of the first article ([https://possiblywrong.wordpress.com/2019/01/09/identical- pac...](https://possiblywrong.wordpress.com/2019/01/09/identical-packs-of- skittles/)) before I started the experiment.

And even this histogram assumes a distribution of _total_ number of Skittles per pack (that varies) that I had to guess at beforehand. In hindsight, the final sample distribution suggests that I probably initially overestimated the true variance, and thus also _overestimated_ the expected number of packs I would need to inspect. In other words, this experiment arguably took _longer_ than "average."

So you're right-- this experiment _could_ have extended into 700 packs, 800 packs... and still have been consistent with the assumed model, but I would have simply been in an unfortunate 90-th percentile possible universe where it took much longer than "average."

</comment></comment><comment><USER14> > _From 12 January through 4 April, I worked my way through 13 boxes, for a > total of 468 packs, at the approximate rate of six packs per day._

Honestly thought I was going to read a "just cause" blog on machine vision and process automation, e.g. 3 months to develop a functional prototype and train the system, 3 days to process 468 packs...and automated repeatability at the end of it all.

<comment><USER15> I assume most people read the title and we're expecting machine vision. I definitely did within the first two seconds of reading.

<comment><USER11> Yeah as soon as I saw the title I made the assumption that it'd be automated.

I can't imagine doing this manually - though it'd take me more time to write a Machine Vision solution rather than just doing it manually. :P

</comment></comment></comment><comment><USER16> This was really hard to read through because I kept dreaming up systems and applications which might automate the whole process.

</comment><comment><USER17> This guy missed an excuse to build a Lego sorting machine....

<comment><USER18> Sorting machine?, this is hacker news, a visual AI that would count each color is the rage these days.

<comment><USER19> Lego Mindstorms has a color sensor, so theoretically you _can_ build that AI in Lego.

<comment><USER14> I suspect the parent's suggestion is more along the lines of 1 snapshot that captures all Skittles in a pack, then ML object detection to classify and count outcome...processes an entire pack in a single shot, as opposed to 1 Skittle at a time.

<comment><USER20> > ML

Why? There is zero need for this. I know it's one of _the_ buzzwords of the recent years, but identifying colors? Come on.

<comment><USER14> Why not? Is the problem _really_ just naive color identification? It looks more like classifying uniformity, shape, size, orientation, character imprint, malformation, anomalous objects, etc. Can the problem be solved with a more traditional image processing toolbox? Sure, why not? But what fun is there if we're not swinging HN's favorite rubber mallet at the problem...and then doing it again and again just because.

Keeping with the context of the GP's remark, a solution is more about achieving process speed while minimizing error and human intervention in the face of uncertainty--i.e. single-shot at the package level, which is at least a 50x speed increase over individual mechanical sorting.

</comment></comment></comment></comment></comment></comment><comment><USER21> Stats is the science of sciences in my opinion, and anyone who brings it to life like this is awesome.

<comment><USER22> Have you seen Efficiency Is Everything's data on food?

[https://efficiencyiseverything.com/food/](https://efficiencyiseverything.com/food/)

<comment><USER23> That site is amazing and engrossing. Thank you for this link - I think I'm going to loose myself in excel for a few hours. Nutrients/calorie and nutrients/dollar are incredibly useful metrics for anyone while shopping!

<comment><USER24> It's not really useful. It's extremely easy to get plenty of nutrients and calories extremely cheaply, but that's not what people are paying for when they shop for food.

<comment><USER25> Please share your data on everyone's shopping habits.

I take it you don't engage in strenuous physical activities far from grocers. For instance.

<comment><USER24> How does distance traveled relate to willingness to pay a higher price for food?

<comment><USER26> It's almost as if it costs money to transport goods or something.

</comment></comment></comment></comment></comment></comment></comment><comment><USER27> I wonder if it would have been worth it to build a device to scan the skittles for you.

edit: would love an explanation for the downvotes

<comment><USER28> A device seems like overkill. Write code to apply machine vision to this grid: [https://possiblywrong.files.wordpress.com/2019/04/skittles_a...](https://possiblywrong.files.wordpress.com/2019/04/skittles_all.png)

<comment><USER29> I think the fastest and most accurate process would be to:

* empty a pack of skittles onto white paper and take a photo

* use some existing image recognition libraries to count each color

* export to csv or smn and do excel magic

<comment><USER30> Even without an image recognition library, it should be possible to simply count the pixels and how close to red, green, blue, etc. they are to know how many skittles there are in the picture.

</comment></comment></comment></comment><comment><USER31> It's interesting this person can program, but chose to manually count 27,740 Skittles instead of automate it in some way.

It feels like an almost-trivial computer vision problem.

<comment><USER32> computer vision is trivial? when did that happen?and how was it made trivial?

<comment><USER9> I've done a few computer vision projects lately and it's really amazing what you can get done with opencv for relatively clean data. In this case you're trying to identify circles and classify their color into one of several groups. This is pretty common, the main challenge is cleaning up the raw data so that the hough transform runs quickly and accurately, and the color cluster identifier is robust.

If you're pretty good with Tensorflow, you could do it with an object detector. I've built absurdly accurate object detectors using the TF Object Detector tutorial and just a little hand labelled data plus some good synthetic augmentation.

For this project's scope and scale it's probably not worth automating unless you're going to be repeatedly running the process.

</comment></comment></comment><comment><USER33> I like practical statistics, for example when betting red/black in a casino, what is the chance that you would lose 10 times in a row ? Just keep doubling they say, but eventually you will get a bad streak and wont have enough money or you'll hit the limit.

<comment><USER34> I remember an experiment in high school math of some variety.

The teacher had us all "guess" what twenty coin flips would look like. The longest any streak any student wrote on their paper was maybe 4-ish in a row or something.

He then had us all actually flip the coins and record the results. One student had like 11 in a row, most hit a streak of somewhere between 5-8 of the same result.

Lesson learned, we're really bad at guessing 50/50 streaks.

<comment><USER35> Empirically, you should (~85% of the time) be seeing a streak of about 3-6; anything 8 or larger has a probability of about 1 in 20.

</comment></comment></comment><comment><USER36> Big fan of skittles - never crossed my mind. Very cool! Perhaps try different kinds as another follow up? Machine learning and openCV can do the trick for counting without sorting.

</comment><comment><USER37> So if I'm a statistical stupid what books do you recommend for the beginner?

<comment><USER38> Andrew Gelman has some pretty good books/lectures, but it might be too advanced.

</comment></comment><comment><USER39> This immediately made me wonder what the odds are of getting a bag of all the same skittles. My stats are rusty so feel free to correct me, but here's what I got:

(1/5)^59 = 1/1.73*10^41

Apparently ~200 million skittles are made daily, so at that rate, we might expect to get a monochrome bag of skittles after ~10^33 years.

</comment><comment><USER40> Jesus Christ, apple and grape?!? Poor Americans... in the UK that's Lime and Blackcurrant.

<comment><USER41> Why poor Americans? It's purely a matter of taste.

Apple is culturally significant and Blackcurrants are banned in the US (due to then carrying a disease that would devastate one of the forests). So one is more important than lime and the other is totally unfamiliar.

<comment><USER42> Blackcurrants are _banned_ in the US? Do they not even have Ribena over there!?

<comment><USER43> You can consume blackcurrants in US. You can't grow it (in some states nowadays). Read more at [https://en.wikipedia.org/wiki/Blackcurrant#History](https://en.wikipedia.org/wiki/Blackcurrant#History)

</comment></comment></comment></comment><comment><USER44> The dataset itself is nice, but if weighting each pack would have been part of the procedure it would have been even more interesting, no? Anyway, this dataset is already in my favorite examples of illustrating a birthday attack for my crypto lectures ;)

</comment><comment><USER45> The picture he shows of all the Skittles' counts looks like something I would've seen at the local museum. He missed his chance of taking an ultra high resolution photo of it and putting it on display:

[https://possiblywrong.wordpress.com/2019/04/06/follow-up- i-f...](https://possiblywrong.wordpress.com/2019/04/06/follow-up-i-found-two- identical-packs-of-skittles-among-468-packs-with-a-total- of-27740-skittles/#jp-carousel-3199)

</comment><comment><USER46> that is a seriously skewed candy budget... do you have any teeth left?

<comment><USER47> OP didn't eat them, per the post.

<comment><USER48> He said, "Yeah, I learned from this experiment that I don't actually like Skittles, which is probably good, so a lot of Skittles were bagged and handed off to relatives." To learn from this experiment that he doesnt like them, that means he must have started out eating them. But yes, he didn't eat them all (thank goodness for his teeth!)

</comment></comment></comment><comment><USER49> This is brilliant. Takes a theory, iid colors, it turns out to be wrong, but still gets a hell of a lot of conclusions out of it.

Since this is a nerd site, the next step is to use this:

[https://www.planet-gbc.com/](https://www.planet-gbc.com/)

Build a Lego contraption to push some skittles through a sensor that counts them.

<comment><USER50> I don't think the author presented any evidence that invalidates the assumption of IID colors. They show a histogram of the different counts of each color, which seems more or less consistent with a uniform distribution. There are some fluctuations but they're not surprising; the Poisson error bars would be roughly +/\- 0.16 in that figure. With error bars that large, it would be surprising if the data were in more exact agreement with the flat line; it's actually related to the same question that the author is examining ("what are the odds to observe all 5 colors at exactly their expected rate of 1/5 within measurement error?").

They do speculate that the number of candies per pack is not IID, i.e., that there are (anti)correlations from one pack to the next. But without knowing more about the packing process, and presumably also having some lot/serial number information for each pack, it would be pretty hard to establish this.

</comment></comment><comment><USER51> Cool thing to do.

This is similar tot the birthday problem: How many people do you need to have a probability of > x% that two are born on the same day?

It's something like 50 people to have a probability of > 80%. You can conduct this experiment at a school, using each class as a sample experiment to see if there are two pupils born on the same day.

<comment><USER13> Author of the article here-- right! This was the key "real world" motivation for this experiment as an attempt at a pedagogical tool; from the article:

> As an aside, I think the fact that this particular concrete application > happens to be recreational, or even downright frivolous, is beside the > point. For one thing, recreational mathematics is fun. But perhaps more > importantly, there are useful, non-recreational, "real-world" applications > of the same underlying mathematics. Cryptography is one such example > application; this experiment is really just a birthday attack in slightly > more complicated form.

<comment><USER51> Sometimes I find these concrete investigations necessary for our brains to make peace with the _unreasonable effectiveness of mathematics_ , as it's been called.

I would say one of the first great discoveries for a person is the exponential series (a real world examples: population growth). Another is the divergence of the harmonic series 1/n and convergence of 1/n^2 (my preferred real world example: pizza slices that converge to 1 pizza or diverge to infinitely many). E.g. give me 1/n slices for the rest of my life and I'll pay you $100 (-:

When travelling, I also have go-to experiments that I like doing (e.g., elementary proofs that the earth is round/spherical such as: great circles; N-E-S-W always at 90 degrees; shadow angles [Erastothenes]; seasons; etc.)

There are other things to investigate that are not really "proofs" or "combinatorial evidence", but equally interesting. One example is using music (esp. the piano) as a physical logarithm device. The music "sounds" additive but the frequencies are multiplicative.

</comment></comment></comment><comment><USER52> I love this and it demonstrates the kind of statistics I wish I was a lot better at.

I've just ordered a huge box of fun sized M&M packets and will try using computer vision to count them to copy this study from start to finish for learning purposes.

</comment><comment><USER53> It's nice to read science like this without it being reduced to a once a year period of [https://www.improbable.com/ig/](https://www.improbable.com/ig/) related links.

</comment><comment><USER54> I love this. Hacking is not always practical. Let us have some fun. Or sweet!!!

</comment><comment><USER55> Fun article. The Birthday Paradox was my introduction to recreational mathematics.

</comment><comment><USER56> 60kg.

</comment><comment><USER57> What a time to be alive!

<comment><USER58> This is 2 minutes paper. And I am your host.

<comment><USER57> Haha. So this ingenious method started with packet number 1 and examined thank you to the Patreon supporters John Jim and Richard now onto Packet number 456 where the color red was found. Thank you and please subscribe like and share.

</comment></comment></comment><comment><USER59> Don't care. Kill green apple and bring back lime!

<comment><USER60> This is the real truth here.

</comment></comment><comment><USER61> Just out of curiosity, did you actually consume them?

<comment><USER62> OP said in the comments of his blog that he didn't actually consumed them, instead he gave to relatives.

<comment><USER61> Thanks, I skipped over the comments. Apparently, though, it appears that the OP may not have started immediately handing them out to his/her relatives:

"Yeah, I learned from this experiment that I don't actually like Skittles, which is probably good, so a lot of Skittles were bagged and handed off to relatives. "

</comment></comment></comment><comment><USER63> I would recommend openCV to anyone considering a similar experiment.

A few hours with a blob detection tutorial would have saved hours of tedium.

</comment><comment><USER64> I hope he uses all the Skittles to decorate a wall or something :) otherwise it's a lot of wasted food

<comment><USER65> High fructose corn syrup is barely food.

</comment></comment><comment><USER66> I would have loved to see all the counts of each in piechart form with slices for each color :-)

</comment><comment><USER67> What's the likelihood of getting a potato chip bag with only one chip?

<comment><USER68> Zero because QA :)

<comment><USER69> hmm, I'd disagree.

There is no formal definition of "a potato chip bag". Hence it's possible to "just" create "a single potato chip bag"...

</comment></comment></comment><comment><USER70> Uh, congrats, I guess...

</comment><comment><USER71> The best justification ever for making Skittles-based vodka :)

</comment><comment><USER72> This is important and high quality research. It deserves an Ig Noble nomination.

</comment><comment><USER73> ..... Let's get this guy some cancer microscopy data!

</comment><comment><USER74> I've always wanted to write an app that can take a picture of a poured out bag of skittles or M&Ms and not only count how much of each color, but use the relative color difference to tell you what brand (skittles/m&m) or sub-brand (regular, tropical, peanut, etc) and how many calories are on the table.

No reason, no monetary plan.

</comment><comment><USER75> No ones mentioned this, but it's unlikely skittles are "randomly distributed". Instead it's likely in the Skittles factory there's some system that attempts to reasonably distribute colours so no one bag is too skewed. So the whole premise is faulty.

<comment><USER76> No one has mentioned it? Did you read the article? The article discusses exactly what you just said.

<comment><USER77> Where? I don't see it anywhere. There's some discussion of total volume perhaps being non-independent from box to box, so an underfilled box is followed by an overfilled one, but this isn't the point the grandparent is making. The grandparent is making the point that there are probably systems in place to prevent a box from being 80% red, say, so that the assumption each individual skittle in a box is independently uniformly drawn from each possible color does not likely accurately model the dynamics.

<comment><USER13> Author of the article here; you have a point that there is no explicit discussion of validating this assumption, beyond the variability shown in the colored curves in the "count per pack" plot.

Having said that, this small sample is indeed reasonably consistent (or at least not inconsistent) with that iid assumption for the color of each individual Skittle. We would not _expect_ to see any 80+% red packs even assuming that color was perfectly uniformly iid, because the probability of observing such a pack is so small (less than 10^(-19)).

However, still assuming this model, we _should_ expect to see packs with very _small_ proportion of reds... and we do, with one pack having just 3 red Skittles, for example. The entire distribution of proportion of red follows the assumed binomial distribution very closely.

</comment></comment></comment></comment><comment><USER78> Kind of disappointed at the methodology. It would have been more impressive to turn this around in a day with ML.</comment>