SONiC – An Open Source Network Operating System - <AUTHOR> https://azure.github.io/SONiC/ <comment><USER2> FWIW, I had an absolutely terrible time setting up sonic on an edgecore switch which was "supported".

Everything was a battle. The configuration is very finicky and not well documented, and every time I went to edgecore or sonic for support, they would blame the other party.

We did eventually get everything working, but it took hours of tinkering on the configuration. There's all sorts of undocumented magic about how ports, interfaces, lanes, etc all have to line up with each other. If it doesn't like how you've configured BGP, it will silently fail and refuse to link. And God help you if you have breakout cables. And for some reason, all the services are run in docker containers, which makes it even more painful to debug. Reapplying configurations is slow and buggy in itself, so you have to reboot the switch constantly.

Ultimately I find sonic hard to recommend. Yes it's free, but I'm really not sure that's a benefit. Any home enthusiast will buy a consumer grade switch with an OS preinstalled, or use OpenWRT. Any business using this for production would be wise to go with a stable product with good support, like Cumulus. As much as I hate how Cisco does business, at least their products will mostly work how you need out of the box.

</comment><comment><USER3> So my problem with SONiC, Cumulus, and all these other "network operating system" Linux platforms is that they all seem to be designed to be fiefdoms and tend to be really stale. In my view, they bring almost nothing of value to the table.

Let's talk a bit more about SONiC...

For one, SONiC literally lists that pull requests that aren't already planned and approved will not be accepted[1]. This defeats a good chunk of the value of having a community project. People will want to contribute and extend your platform in ways you never thought of, and they'll do it in a completely decentralized fashion.

Another issue I see is that SONiC literally holds back everything to an old Linux kernel and ships random BSP blobs that are unvetted. This is a nasty combination for anyone who wants to consider their NOS trusted or secure. They're on a 4.9.x kernel, and while that is still maintained, it is far from the best option if you want to take advantage of innovation in Linux networking.

I'm also generally confused on why this whole project isn't just "let's get the networking tools and hardware support stuff into standard Linux distributions and leverage their tooling and communities". This was also a problem I had with Cumulus. When I tore apart Cumulus, I figured out that it was less than a dozen unique tools and a distribution rebuilt for 32-bit MIPS and PowerPC. It was pretty trivial to rebase to standard Fedora or Debian and get a better platform out of it.

And finally, I don't really think this provides any real innovation. It's not really different from Cumulus, Open Network Linux, and others. And ONL actually is using more up to date kernels (5.4.x as of right now!) and offers better networking tools!

What I would _love_ to see is all these people who keep doing this crap working in the actual Linux distribution communities to build and integrate with upstream projects so that everyone downstream gets all kinds of flexibility.

Imagine if you had a flavor of Fedora CoreOS for your network gear! The immutable OS, updated with RPM-OSTree, fresh software stack, and broad hardware support, all in one neat package.

If we treated the network gear like weaker servers, instead of specialty equipment, there's so many more interesting things you can do!

[1]: [https://github.com/Azure/SONiC/wiki/Sonic-Roadmap- Planning](https://github.com/Azure/SONiC/wiki/Sonic-Roadmap-Planning)

<comment><USER4> >This was also a problem I had with Cumulus. When I tore apart Cumulus, I figured out that it was less than a dozen unique tools and a distribution rebuilt for 32-bit MIPS and PowerPC.

Almost all of which are open source (or at least "source available"), with the exception of switchd, which cannot be open sourced because it links with proprietary asic sdk's. I don't see how having very few custom tools over a vanilla Linux distribution is a bad thing.

>It was pretty trivial to rebase to standard Fedora or Debian and get a better platform out of it.

If you enable upstream Debian apt sources in your sources.list then it effectively _is_ standard Debian - plus switchd.

Of course it is entirely possible to take all of the components of Cumulus Linux and use them on a separate operating system - enter sonic, vyos, etc - so if you build out such a system which can also drive ASICs and that you prefer over Cumulus, you can take full advantage of all of Cumulus's open source contributions.

>What I would love to see is all these people who keep doing this crap working in the actual Linux distribution communities to build and integrate with upstream projects so that everyone downstream gets all kinds of flexibility

If I read you correctly, Cumulus works upstream as much as it can:

       ~/linux$ git log --author "cumulusnetworks.com" --oneline | wc -l
       773

       ~/ifupdown2$ git log --author "cumulusnetworks.com" --oneline | wc -l
       1265

       ~/frr$ git log --author "cumulusnetworks.com" --oneline | wc -l
       8107

I like to believe Cumulus is quite active in the communities of projects it uses. I feel I may have misunderstood your point, though.

>If we treated the network gear like weaker servers, instead of specialty equipment, there's so many more interesting things you can do!

I completely agree, that's the dream!

Disclaimer: I work at Cumulus

<comment><USER3> > _Almost all of which are open source (or at least "source available"), with > the exception of switchd, which cannot be open sourced because it links with > proprietary asic sdk's. I don't see how having very few custom tools over a > vanilla Linux distribution is a bad thing._

The problem historically with Cumulus on this was that it was heavily obfuscated. In the past, when I talked to Cumulus sales folks, it was not quite as honest as what you've said.

I don't have a problem with the "shipping a Linux distribution you can support" thing. I have a problem with "not making it so the stuff you have is available everywhere (i.e. push into Fedora _and_ Debian to feed into all distros and ecosystems)".

> _If I read you correctly, Cumulus works upstream as much as it can. I like > to believe Cumulus is quite active in the communities of projects it uses. I > feel I may have misunderstood your point, though._

Cumulus is actually a nice exception to this rule. Most Linux-based network operating systems do not bother (including SONiC, VyOS, EOS, etc), but Cumulus does good work here. My only complaint is the focus on ifupdown2 instead of helping make cross-distro tools like NetworkManager support these things. It's been a long time since NetworkManager was only for desktop-only use-cases and only did Wi-Fi. It's the standard tool on a wide range of distributions and supports server use-cases very well. I personally use it over ifupdown and netconfig on my systems.

<comment><USER4> >I have a problem with "not making it so the stuff you have is available everywhere (i.e. push into Fedora _and_ Debian to feed into all distros and ecosystems)".

Almost all of our kernel patches are in mainline Linux, and ifupdown2 and FRR are packaged on Fedora and others.

>Cumulus is actually a nice exception to this rule. Most Linux-based network operating systems do not bother (including SONiC, VyOS, EOS, etc)

In defense of VyOS, they contribute to FRR and generously offer free licenses for people who work on the projects they use ([https://www.vyos.io/open- source-contributors/](https://www.vyos.io/open-source-contributors/)). I think in general there's a lot of goodwill between the people working in the open NOS space.

> My only complaint is the focus on ifupdown2 instead of helping make cross- > distro tools like NetworkManager support these things

Gotcha, I understand now. I can't provide any direct insight into why ifupdown2 was chosen instead of nm. I also use nm on my personal devices - though I can't say I've ever missed the ability to e.g. configure vxlan tunnels on my personal infra ;). I guess if we'd chosen nm 10 years ago then there would be similar feelings from people who prefer /etc/network/interfaces. Of course, at the end of the day Cumulus engineering time is spent primarily on things that ship in Cumulus Linux.

Btw, appreciate the feedback :)

<comment><USER3> > _Almost all of our kernel patches are in mainline Linux, and ifupdown2 and > FRR are packaged on Fedora and others._

I did see FRR recently make its way into Fedora, but I haven't seen anyone package up ifupdown2 there. Is someone working on that at Cumulus? I'd be happy to do the package review if someone hasn't already grabbed it before me once it's submitted. :)

> _In defense of VyOS, they contribute to FRR and generously offer free > licenses for people who work on the projects they use > ([https://www.vyos.io/open-source-contributors/](https://www.vyos.io/open- > source-contributors/)). I think in general there's a lot of goodwill between > the people working in the open NOS space._

Oh, I don't doubt it. But it's weird how many of them built on Linux are still not FOSS or collaborating with their upstreams...

> _Gotcha, I understand now. I can 't provide any direct insight into why > ifupdown2 was chosen instead of nm. I also use nm on my personal devices - > though I can't say I've ever missed the ability to e.g. configure vxlan > tunnels on my personal infra ;). I guess if we'd chosen nm 10 years ago then > there would be similar feelings from people who prefer > /etc/network/interfaces. Of course, at the end of the day Cumulus > engineering time is spent primarily on things that ship in Cumulus Linux._

Well, for what it's worth, /etc/network/interfaces is supported by NetworkManager. :)

As for VXLAN configuration in my personal network, I do it for homelab stuff. Setting up layered networking is kind of necessary if I am going to be messing around with things like OpenStack and Kubernetes.

I totally get that the engineering is primarily spent on things that ship in Cumulus Linux. I just want to see more work from Cumulus that benefits everyone, especially given that networking is so hard to get right! :)

> _Btw, appreciate the feedback :)_

You're welcome. I'm happy to see such an engaged person from Cumulus like yourself responding well to feedback! :)

</comment></comment></comment></comment></comment><comment><USER5> Shameless plug: the Cisco 8000 series [1] recently got support for SONiC [2].

Right now, I believe it's limited to fixed systems, but work is ongoing to get SONiC running on the distributed chassis. Exciting stuff!

[1]: [https://www.cisco.com/c/en/us/products/routers/8000-series-r...](https://www.cisco.com/c/en/us/products/routers/8000-series- routers/index.html)

[2]: [https://blogs.cisco.com/sp/cisco-goes-sonic-on- cisco-8000](https://blogs.cisco.com/sp/cisco-goes-sonic-on-cisco-8000)

</comment><comment><USER6> If you want an overview of of the state-of-the-art open source networking stack, you can check this last year FOSDEM 2019 presentation by one of the developers of BISDN [1]. Somehow I don't think they mentioned SONiC in their presentation though.

I teach computer networking lab but for now we need to resort using dual boot Linux Switch Appliance (LISA) that uses custom kernel and another vanilla kernel running Quagga on multi-port Ethernet embedded PC. The good thing is that both LISA and Quagga use CLI environment similar to familiar Cisco switch/router IOS. I really wish there is an open source alternative offering that can seamlessly support layer 2 and layer 3 without dual boot, perhaps using Software Defined Networking (SDN) concept with an intuitive CLI.

Recently the new Shortest Path Bridging (SPB) has been integrated with 802.1Q since 2018 (bye-bye TRILL). I reckon any reasonably good Linux based open source layer 2 and 3 network OS player will be extremely popular overnight for enterprise, consumer and education. Together with eBPF this thing should be flying on the new off-the-shelf whitebox supporting 40Gbps multi-port Ethernet. Imagine a LAN or Metro LAN party with this beast :-)

[1][https://archive.fosdem.org/2019/schedule/event/from_closed_t...](https://archive.fosdem.org/2019/schedule/event/from_closed_to_open_source_switch/)

</comment><comment><USER7> Not really a "network" operating system since it doesn't seem to manage a network consisting of multiple devices the way something like ONOS, OpenDaylight or FAUCET would.

More like a "switch" operating system - it looks like SONiC is Microsoft's answer to ONL, Stratum, OpenSwitch, Cumulus et al.. Basically open source software to run on your cheap whitebox (or expensive greybox) switch.

</comment><comment><USER8> Obligatory link to tutorial on how to run Sonic on an Arista switch:

[https://www.servethehome.com/get-started-with-40gbe-sdn- with...](https://www.servethehome.com/get-started-with-40gbe-sdn-with- microsoft-azure-sonic-for-under-1k/)

</comment><comment><USER9> Linux-based. How is SONiC positioned vs. Cumulus ?

<comment><USER10> Besides being cheaper and supporting more hardware, SONIC is more like a traditional NOS that runs on Linux but doesn't integrate with it. I would expect SONIC to be more familiar to Cisco CLI jockeys and Cumulus to be more familiar to Debian/Ubuntu sysadmins.

<comment><USER11> Incorrect. In Cumulus you have NCLU which is the Cumulus cli that just takes the commands from the bash prompt. If you know traditional Cisco you can sort it pretty quickly. They start with "net" so to show something "net show interfaces".

Here is a cheatsheet:

[https://cumulusnetworks.com/learn/resources/cheatsheets/nclu](https://cumulusnetworks.com/learn/resources/cheatsheets/nclu)

Also Cumulus runs FRR (as does sonic) but on Cumulus you can do sudo vtysh and pretty much be at a CLI for routing like you are at an IOS prompt.

Sonic sticks all configuration in different docker containers in Json files and can be a real pain. Also not all commands are hitless, ie some will restart forwarding. That is being worked on. SONiC is pretty much "what MS wanted for large scale ops" and still is rough around the edges for enterprise IT.

There are a number of companies that support Sonic in production enterprise such as Dell and Apstra.

</comment></comment></comment><comment><USER12> How does this compare with existing switch OSs from Cisco and Juniper? Linux seems like it would be a bit heavy for a switch.

<comment><USER13> Cisco is already Linux based. IOS-XE and NX-OS.

[https://en.m.wikipedia.org/wiki/Cisco_IOS_XE](https://en.m.wikipedia.org/wiki/Cisco_IOS_XE)

[https://en.m.wikipedia.org/wiki/Cisco_NX- OS](https://en.m.wikipedia.org/wiki/Cisco_NX-OS)

<comment><USER14> I like this comparison

[https://medium.com/@Meela349588204/using-cows-to-explain- the...](https://medium.com/@Meela349588204/using-cows-to-explain-the- differences-of-the-cisco-operating-systems-876d14145827)

<comment><USER11> If you want to enjoy even more pain, ah trivia I mean, NXOS and NXOS are not the same thing. Read it again. There is an NXOS train for the original 2/5/7K and current kit in that line, and there is another version for the 3/9K new kit from the ACI spin in. They are just different enough that it makes things difficult when you are trying to develop tools for them. Cisco has promised to merge the two for a long time now.

</comment></comment></comment></comment><comment><USER15> Arista's EOS is open source and up to the task, it would be awesome if there was a fork of it supporting whitebox gear.

<comment><USER10> No, EOS is not even close to open source. You might be able to run cEOS on a whitebox.

<comment><USER15> Maybe something changed but I rember downloading the source and also running it in a vm.

<comment><USER11> EOS core code never been open source. It it is built on Fedora Linux.

They do have vEOS and cEOS offerings but there is closed source Arista code in all of them.</comment></comment></comment></comment>


Future of Racket - <AUTHOR> https://www.greghendershott.com/2019/07/future-of-racket.html <comment><USER2> My feel from the Rust that I've done so far: you know you're in something like an industry consortium, they're trying to please all the stakeholders, and gain adoption, and one can even see community input partly as market research. I like that I know where I stand.

With Racket, the community relationship has been fuzzier, with all the various enthusiasts and volunteers, with money not a factor.

From the professors' perspective, I'd say they're pursuing their research and education ideas, as well as getting funding, publications, tenure, awards -- and I think all that is mostly the goals of Racket. Nurturing _a_ user base serves those goals.

From some other community people's perspectives, Racket is a community effort, to support, with the professors doing most of the work on the core, while everyone contributes in various other ways. I suspect many have assumed it's a partnership of aligned interests, towards some shared goals, or that the core is a benevolent dictatorship that will represent the interests of the current user community.

I recently saw someone mocking Racketeers for being alarmed about losing their precious parentheses, but I don't think that's the problem, for maybe half the people who are concerned. I think the problem for some is having their ideas of the goals of Racket and the community relationship shaken up, after they've invested.

<comment><USER3> Unfortunately, the person stepping down in this blog post (Greg) was one of the people who I looked up to when I was really enthusiastic about racket because he was obviously "getting things done" in the language -- solving problems, making things.

</comment></comment><comment><USER4> The mailing list traffic on this topic took the wind out of my sails for wanting to spend my tinker time in Racket. On the one hand, it's probably overreacting for me to throw in the towel on Racket at just the suggestion of this change. On the other hand, part of what makes me interested in writing Racket for projects where I have broad latitude is the sense that I'm both using an exciting technology, and a technology with a history of relative stability.

By signaling a switch to some bland surface syntax I feel that it's eroding the exciting part as well as the stability benefits of the language. For data science (the domain I'm most invested in) Julia has attracted a decent mindshare, is lispy (even if not as much as I want), and is fast. Racket with a bland syntax is likely to be about as lispy as Julia, much slower, and have fewer of the data science libraries I care about.

I only have so many hours in my life to dedicate to not-making-money programming endeavors. I was sort of hoping that Racket would be the vehicle for those hours. But right now, I'm worried that Racket is Perl 6-ing itself, and that's a hard place to spend hours. Especially with people like Greg leaving the community.

A while back there was a thread about a prominent member of the Common Lisp community saying that he'd probably eventually leave CL for Racket. But then later he signaled that he'd settled on Gerbil Scheme. Maybe that's the right life raft from Racket.

<comment><USER2> Please don't be turned off by the recent commotion. (I'm sorry some of the commotion that might've discouraged you was by me.) Commotion happens because people care about Racket, and there are reasons they care.

Racket actually has a couple-decade history of very good stability, other than a couple oopses that might've helped prompt the current attempt at greater community involvement in decisions.

The history is too long to summarize here, but some quick evidence you can look at is, when the language evolution really wants to change an interface or how something works, the old way either still remains in the main language/library, or is preserved in a legacy language or module, which you can see at the bottom of the page: [https://docs.racket- lang.org/](https://docs.racket-lang.org/)

For further evidence that's fairly easy to find, you can also look at the very careful and conservative process for introducing the new Chez-based backend.

<comment><USER4> No, to be clear, the commotion that has me concerned about Racket's future is the substance of the proposal that was signaled on the list, not any sort of flamewar fatigue. One of the nice things about the Racket community is how thoughtful and earnest people are. I also tend to view your commentary as reasoned and agreeable, so no apologies needed!

To me, I'm just not that excited about the "maybe direction." I guess I'm one of the parentheses clutchers who loves parediting in emacs. I'm sure the transition toward some other syntax will be cautious and careful, and that the resulting preferred #lang will be a fine one.

It just sounds like the center of mass for the Racket community won't be quite what I was looking for when I came to Racket. I've got plenty of experience in at least one language, R, that started as a Scheme and then syntaxed itself to something less than Scheme (the language I mean, not the libraries—I'd kill for a #lang racket with R's libraries).

Maybe Racket2 will be the bland syntax that succeeds in not obscuring the Music of the Spheres. If anyone can do it, this community will. On the other hand, I've got a healthy skepticism, uninformed by any deep understanding of the issues. I'm not sure what the benefits of sticking it out to see are.

The proposed change has anti-network effects in that Greg is already signaled his departure, and I'd already stopped reaching for Racket as the answer to "what lisp should I learn?" I simply don't have a good answer to that question where I felt like I did a month or so ago.

But I'd be disappointed if Racket changed direction based on opinions of people like me. It's not like I have any grand contributions to the community that are going away! I'm a dilettante who writes a little bit of Racket to scratch a personal itch once in a while. My primary takeaway from my one in- person RacketCon (Seattle) was, "woah, I don't understand anything but these people are smaaaart." I am not the right person to cater a language to.

<comment><USER2> Thank you for your thoughtful comments. The Racket commmunity has a lot of people using it on the side, like you do, and your interests and contributions matter. I've been in that category myself lately.

If you were drawn to Racket, but want to look around a bit, there are a few other Schemes I find especially interesting, including: Guile, Chicken, Gambit, and Scheme48. Guile is in pre-release of some nice new JIT work, including instant startup from native code caches, and has an active community. If the bulk of your code can be written in RnRS, and you avoid mutating lists, you should be able to move around between the various Schemes and Racket reasonably easily. Of course, there's also CL and Clojure, but those are pretty different.

I didn't know that R had some Scheme ancestry. (For one client, I made an interface to call R from Racket, for a data science system.) I think one of the things Racket was heading towards, around the time of the math modules work, was for Racket to become a newer R. And perhaps have a `#lang r`, as another way of bringing in some of the wealth of R libraries and programmers (and also a similar situation with Fortran). I'm pretty sure we couldn't get Fortran performance with anything other than a native Fortran compiler, and I don't know the state of R performance, but Racket could've had closer to one of the main selling points of Python today (i.e., a wealth of data science libraries are available easily through it).

</comment></comment></comment></comment><comment><USER5> Well, this is in the end a very subjective thing but I'm with Greg on this one. A syntax without s-expressions is not an innovative feature. For me, it's not even desirable, not at all. When I'm using non-Lispy languages like Rust, Ada, Nim, and currently a lot of Go, that's despite their annoying syntactic idiosyncrasies. All of those quirky little curly braces and special symbols to save few keystrokes. I'd much prefer if all of these languages used s-expressions. That syntax is so simple that it makes you focus on the semantics.

</comment><comment><USER6> It's a pity, because he has a few interesting projects like:

* Rackjure: " _Provide a few Clojure-inspired ideas in Racket. Where Racket and Clojure conflict, prefer Racket._ " [https://github.com/greghendershott/rackjure](https://github.com/greghendershott/rackjure) I don't use it, but I think it's a nice idea, and it show how Racket can be used for multiple programming "frontends".

* Travis-Racket " _[https://github.com/greghendershott/travis-racket*"](https://github.com/greghendershott/travis-racket*") [https://github.com/greghendershott/travis-racket](https://github.com/greghendershott/travis-racket) Travis don't support Racket directly, so his package is very useful for testing without having to study all the details about the naming and download addresses of each version/variant, I use it all the time.

<comment><USER2> Greg Hendershott has contributed a lot of software development, and also a lot of community participation. The latter doesn't appear on one's CV, but makes a big difference, and has network effects.

The community of Racket has long been one of its selling points, which new adopters often mention, and people like Greg are why.

</comment></comment><comment><USER7> I agree in the sense that the non-s-expression syntax proposal also rubbed me the wrong way.

Rust is an interesting language but I have spent less than 20 hours playing with it. I am mostly retired now but if I were just starting my career then Rust is a language I would invest a lot of time in.

Anyway, I can't even imagine myself using Racket with a non Lisp syntax.

</comment><comment><USER8> To me, Flatt's proposal for Racket2 smells more like adding tools to better facilitate infix languages than depricating S-expressions. Given Racket's pedagogical mission, it looks more like a move toward migrating the HtDP series of languages (Beginning Student, Intermediate Student, Intermediate Student with Lambda, and Advanced Student) to infix syntax than anything else. Not really the end of the world or a big change to the larger Racket community. Just another extension of an ecosystem that remains s-expression based despite Algol and Datalog shipping in the box. It's hard to see Flatt's work porting Racket to Chez Scheme as a prelude to abandoning Scheme as the basis for the Racket ecosystem.

On the other hand, the reality is that Racket's syntax makes for an uphill fight when it comes to adopting the HtDP curriculum. Anyway #lang racket syntax is just sugar on the more important concepts in the HtDP methodology. The HtDP engineering principles are matter much more than which language is used at the top level.

</comment><comment><USER9> I think this is probably a big testament to the draw that Rust has. Its a great language with big improvement and it feels like it will actually happen.

Whenever I dabled around with other languages, I quickly found reasons to not go deeper that where outside of the language itself. Like, small community, not many libraries, will never convince the boss to use it in production, doesn't run on embedded, ...

Rust is the language that really feels like investing in it will pay off.

(Still - do some Racket, or lisp or scheme, it's fun and exposes you to cool ideas and concepts.)

</comment><comment><USER10> The Lisp community will feel the loss of Greg Hendershott.

</comment><comment><USER11> Funnily enough, I read just recently (though I can't remember where) that John McCarthy originally intended to develop a syntax called M-expressions (derived from Algol or Fortran) for LISP that would be compiled to s-expressions, but users seemed content with the s-expressions and he didn't bother.

Ah, Wikipedia mentions something on the topic:

[https://en.wikipedia.org/wiki/M-expression](https://en.wikipedia.org/wiki/M-expression)

Plus ça change, etc.

</comment><comment><USER12> Since he mentions Rust, I have seen Rust being used for many different things and not only "systems programming", aking to a general purpose language.

Is this because the language lends itself easily to all this tasks? or is the Rust hype and devs using it for different things to learn the language?

<comment><USER13> Both, I think.

The language is fairly high level, and the borrow checker + RAII covers a lot of the ergonomic-gains of garbage collection. It's also appealing in that it moves towards functional, but not too much (beneficial for those who like the idea of functional programming, but can't be assed to pick it up properly -- eg me).

And the absurd hype cycle just keeps it permanently in mind.

<comment><USER14> A lot of people get caught up in Rust's low level implementation stuff but you really don't need to touch that stuff at all if it's not your thing. Just understand enough about what you're doing with memory and the various performance trade offs which become more prominent at that level of the machine.

There's more than enough high level stuff like the type system and traits which you can go harder into in terms of learning new/better ways to program.

</comment></comment></comment><comment><USER15> I feel like there's some paragraphs missing in this post between "I'm concerned the change won't help grow the community" and "I'll no longer learn or have fun in this community". He doesn't explicitly say how he expects his relationship with Racket to change, or why. He's _implying_ , though not even outright stating, that he's distancing from Racket because of Matthew Flatt's proposal. That's begging for further explanation. Mere proposals don't warrant this sort of reaction. Racket is hardly a stranger to radical proposals. What's going on behind the scenes that's so serious as to warrant a post like this? Why, in short, the vagueposting _drama_?

<comment><USER16> He didn't rehash arguments that he had made on the Racket mailing list in his blog post. Here is one example: [https://groups.google.com/forum/#!msg/racket- users/vN_1uUJZn...](https://groups.google.com/forum/#!msg/racket- users/vN_1uUJZnXo/5bXiMEBvCgAJ)

<comment><USER15> I'm not talking about his specific objections to the proposal. I'm talking about why those objections, which seem to be taking place in the context of a reasonable discussion where everyone's opinion is heard, have led him to the conclusion that he will no longer enjoy participating in the Racket community.

<comment><USER16> This is a key line from his post:

"Also to be fair, if you have some mileage on your odometer, your pattern- recognition machinery will quickly assign high probabilities to various outcomes."

His combination of personality and experience is leading him to draw conclusions about where Racket2 is headed that is causing him to press pause on using (and contributing to) Racket. I had a similar reaction (described in this post: [https://www.travishinkelman.com/post/exploring-scheme- implem...](https://www.travishinkelman.com/post/exploring-scheme- implementations/)).

It seems that you are not assigning as high of probability to the outcomes that Greg is concerned about. Or you are not as concerned about those outcomes.

</comment></comment></comment></comment><comment><USER17> S-expressions were the one thing that held me back from Racket. I like a lot that I've heard about the language, and I'm a believer in keeping syntax as simple and regular as possible, but in my book a language that doesn't allow infix expressions is not usable, while a "language" that requires users to implement text-based macros to work productively in it is incomplete.

The article doesn't link to the syntax proposal, so I don't yet know what it looks like. But for the first time I now hope that Racket might become a practical language that I could use. No S-expression language has ever gone beyond niche. I'm glad Racket is willing to step past them.

<comment><USER18> Racket will never be a practical language, but not because it is a lisp. Clojure is a lisp and is plenty practical.

Racket was designed to teach programming to young students, which unfortunately isn't a killer app for a language even though I learned a great deal from HTDP and Racket/PLTScheme.

There have been other languages that were not lisps like Dart that utterly failed or like Coffeescript, which faded away because the space that they were trying to overtake (Javascript) was already deeply entrenched.

If you can't appreciate Racket beyond it's syntax, it's unlikely that you'd appreciate it with a new syntax. Racket is academic niche in the same ways Haskell is.

If parentheses and lack of infix symbols are sufficient to keep you away from Racket and you want much of the same level of metaprogramming and macros I suggest trying out Julia, which is considerably more practical where infix expressions and operators make sense within the numerical context: Every infix operator is a function call and has a dual syntax.

But don't please don't lie to yourself that you'd ever adopt Racket if it had infix expressions. I love Racket and I still don't use it for many practical projects because the demands of work are different than academia, which I believe are the true barriers to adoption.

<comment><USER2> > _Racket was designed to teach programming to young students,_

I'd say that the "teaching languages" were designed for teaching students, and Racket was designed as the practical tool for implementing those languages and a cross-platform IDE to support them. As a result of this, and various subsequent community work, Racket is now practical for many purposes.

I said a little more about this recently: [https://news.ycombinator.com/item?id=20507549](https://news.ycombinator.com/item?id=20507549)</comment></comment></comment>


Precaution, uncertainty, insurance, and morality [re AGW] - <AUTHOR> http://www.samefacts.com/2009/12/climate-change/precaution-uncertainty-insurance-and-morality/ <comment><AUTHOR> This post makes the point that the greater the uncertainty in the climate models, _the greater the potential catastrophe, and the greater the need to insure against catastrophe_.</comment>


React: CSS in JS - <AUTHOR> https://speakerdeck.com/vjeux/react-css-in-js

<comment><USER2> I'm not sure the implementation is quite what we need, but I definitely agree with the rationale.

CSS is global, as they point out, and CSS makes assumptions that may not be true today. For example, most apps these days — at least the kind I build – gain zero benefit from separating the style from the DOM. We never, ever apply different stylesheets to a single app.

The few use cases where truly separate stylesheets would be useful aren't significant enough to justify the separation. For example, "themes" are implemented through SASS variables that drive the presentation -- not with separate stylesheets.

And generally speaking, apps are simply too complicated to reimplement the entire app's stylesheet multiple times. And CSS cascading rules are sufficiently gnarly (eg., the exactness of the selector guides priority, except !important, etc.) to make things too error-prone to work in practice.

CSS was designed for basic marked-up hypertext documents, not today's sites or single-page apps. It works fine for documents, but for everything else it's increasingly showing an impedance mismatch, especially with respect to isolation and layout handling. Even with "flexbox" it's very limited compared to constraint solvers like Cassowary. We have to face the fact that layout belongs to content.

Anyway, there are a few problems with the CSS-JS approach:

\- Media queries aren't possible anymore and must be done with JS. (I usually end up doing a lot of this in JS anyway, because media queries only deal with viewport sizes, not parent container sizes, so media queries break component isolation.)

\- When you have styles applied to almost every element in a complex component, the DOM will look increasingly messy. You can't do mass-application things like "div > li { ... }".

\- It's going to massively increase DOM size.

\- Surely browsers must have highly optimzied CSS processors these days. I can't help but think that applying entire style rules to DOM elements must be slower than just referencing an existing compiled style.

Personally, I suspect a better way to approach this would be a React-specific, component-oriented style language. Not the modest tweaks they demonstrate in the first slides, but something built from the ground up.

</comment><comment><USER3> At the last slide author says that the goal of that talk was to convince us to drop CSS and use JS.

Can't say that those slides were very convincing. Right now it looks way too overengineered.

I'm not a fan of muddying JS with bits of HTML in general, and now CSS in JS?

Yes, that may work for Facebook. They've been inventing their own way of doing things for a while now, but I very much hope that won't go beyond Facebook.

<comment><USER4> I agree. CSS is global, but that doesn't mean that it can't be managed. I think going down the road of having programmatically generated stylesheets is solving a relatively small problem with an extremely over-engineered solution. Soon enough, you can't actually develop without the use of dozens of different tools for generating source maps, transpiling, etc. In all honesty, any presentation that says "If I started with this, you would have written me off as crazy" is probably giving itself too much credit and is in fact based on crazy.

The whole "button in an overlay" section of the presentation shows just how mind-bending this becomes. In a sane universe, one simply needs to write `.overlay .button`. But of course, you can't do that with CSS that's referenceable from a single name (or where you're statically applying styles to an element rather than letting the CSS...well...cascade). I could see if the solution presented made me say "Oh, this is a much more intuitive way of styling a web application." It didn't. Instead, I read, "We have this crazy proprietary namespaced CSS system that has limitations that we can't work around without building an even crazier system on top of it," and thought to myself, "Well don't do that then."

So now you're at a point where everything is an inline style. Your browser's devtools are essentially useless because you can't see how the CSS cascades. What overrode what? God only knows, go look at the JavaScript.

Of the seven problems outlined at the beginning, I think few of them were really a problem to begin with. A codebase that 1. properly organizes its CSS into files with descriptive names and a sane directory structure 2. uses OOCSS or other best practices 3. is mindful of style bloat 4. has a sane DOM structure and 5. uses a preprocessor of some variety for variables does not suffer greatly from any one of those issues. I work on a codebase of >10^5 lines of CSS that minify+gzip down to 30kb. That's a single small JPEG worth of code.

I would be interested to see some of the alternative solutions that Facebook explored that don't dive straight into the Twilight Zone.

<comment><USER2> We organize our SASS files very carefully. One file per component, careful naming, strict rules about overriding a component's internal classes (".mycontainer .somecomponent .somecomponent_title" is illegal). It's still painful.

One reason is cascading. The way CSS applies to trees means that every component has to be aware of what properties it could accidentally inherit. So to code defensively, you override things like colors and fonts. Using SASS mixins helps.

But sometimes you do want contextual modifiers, for example styling all components differently when they're in a certain type of container. For example, if the container's background is light grey, we'll want all nested text to have a subtle bevel (ie., white text-shadow).

It's tempting to use cascading for this, but that breaks — by definition! — separation of concerns. In one way it's better to actually have each component be container-aware; for example, given that Form and SpecialContainer are both components you can do this in form.scss:

        .SpecialContainer {
          background: #f0f0f0;
        }

        .Form {
          .SpecialContainer & {
            text-shadow: 0 1px 0 #fff;
          }
        }

It's cheating, though; now you have a dependency between the Form and SpecialContainer that really wasn't needed. So when I'm not being lazy, I resort to explicit parameters:

        .Form {
          &[data-light-background="true"] {
            text-shadow: 0 1px 0 #fff;
          }
        }

Now I have to pass "lightBackground" as a prop to the component have it set the data-light-background attribute. Of course, this only works the component embedding Form actually knows that it has a light background or is embedded in one that has a light background. Not ideal.

</comment></comment></comment><comment><USER5> I can see that this approach might be viewed with a lot of suspicion, but this gels with a lot of problems we saw while building the MontageJS framework.

CSS _is_ global, and it is a big problem to manage while building a large web app (which of course Facebook is). I think everyone building a large site or app has come across a bug where a CSS rule is unintentionally overriding another. They are a pain to track down and almost impossible to prevent.

I haven't played with this approach (as it was only talked about today!) but I think it's on the right track. The only thing I would like to see is the "CSS" put into its own module and required into the component JS file. Not only would this maintain the separation of style and markup, but also make it more approachable for those CSS experts coming from the current way of doing things.

I'm really looking forward to seeing how this progresses.</comment>


Microsoft admits Mac was Windows 7 muse - <AUTHOR> http://www.theregister.co.uk/2009/11/11/microsoft_copies_apple/

<comment><USER2> I'm not sure the quote really means what they're interpreting it to mean. Emulating the Mac "in terms of graphics" could simply refer to Microsoft trying to create a more powerful graphics subsystem, akin to Apple's Quartz (which is strongly based on PDF). In other words, to let Windows programs more easily produce interesting graphical effects that don't tax the CPU needlessly.</comment>


Want unlimited scale and performance? Those are the databases to start with - <AUTHOR> https://indexoutofrange.com/Want-unlimited-scale-and-performanceThis-is-where-to-start/ <comment><USER2> There are many faults in this article.

For one thing, it is not true that relational databases always use b-tree indexes and that key-value stores always use hash indexes. You can find examples of both that let you choose.

Also there are many key-value stores that support transactions.

</comment><comment><USER3> Step 1: mentally redefine "database" to include memcache...</comment>


The End of Elliot - <AUTHOR> https://docs.google.com/document/d/1V3Sfm2ZNLxFjNFSExKxGkjZ3BRVBVIaAdvrdf8S5W44/edit <comment><USER2> For those who don't know, Elliot was a startup Shopify competitor. They gained notoriety through relentless Twitter self-promotion, trash-talking Shopify non-stop and promising an e-commerce revolution when their platform launched.

They really had mastered the art of Twitter promotion. I would see friends of friends recommend that people follow Elliot and their founders several times per week. They were frequently held up as examples of founders doing things the right way and tackling big problems.

They officially launched last week after hyping the company since 2017, but the site didn't work. The CEO tried to blame a DDoS attack, but later announced that their launch date was pushed back to December.

A few days later, the CEO posted a screenshot of his resignation letter from the Notes app of his iPhone. Shortly afterward, the company announced that they were shutting down via an e-mail to their waitlist. The shutdown email vaguely alluded to the CEO's personal problems, but stopped short of a real explanation.

This is a disappointing outcome for everyone, including those of us who like to cheer scrappy startups along from the sidelines. It's a good reminder that PR, charisma, and confidence are necessary but not sufficient to launch a startup in today's environment. It's hard to believe that Elliot raised $3.7 million from dozens of investors, yet no one noticed that they didn't have a functional product or even a team that could execute the product. Even their business model didn't make sense, with fee structures so low that they would never become a sustainable, scalable business until they were processing on the order of billions of dollars of shopping purchases through their platform.

People have joked that this is the Fyre Festival of eCommerce, given how much PR they did and how no one was willing to accept that it was doomed as they approached their launch date. It's fascinating to see how quickly the company imploded as soon as it made contact with the real world. At minimum, I would expect investors to appoint another CEO to turn the IP into something useful if they were actually that close to launching. The fact that they let it implode completely suggests that the company had much deeper problems.

</comment><comment><USER3> What an incredible journey. All that hype for absolutely nothing.

Grade A Vapourware.</comment>


A pair of cave lion cubs has been found mummified - <AUTHOR> http://www.bbc.com/earth/story/20151119-first-mummified-cave-lions-discovered-in-siberia <comment><USER2> Title clarification: They were found frozen in ice. They were not preserved by artificial means.</comment>


Ask HN: Is there any input method which can guess language - <AUTHOR> I often need to switch typing between English, Japanese (Romanji) and Traditional Chinese (Changjie). Is there any input method which can guess the language by keystroke?
For example, "apple" (English), "ringo" (Japanese), "tyhc wd" (Chinese).
Thanks. <comment><USER2> I just so happen to be working on a universal input method that covers different languages.

All the letters are divided into trees. One thought that I've had but haven't gotten around to implement as of yet is to merge different languages, along with their statistical attributes into one tree.

This is in a prototype stage but if you feel like this might be something go and subscribe and I'll let you know when there are new updates.

I'm having a new version that'll come out soon, that's rewritten and in a more serious manner.

[http://sigma.eruditenow.com/theory.html](http://sigma.eruditenow.com/theory.html)

</comment><comment><USER3> This is a use case for a programmable editor. Emacs might be the easiest to program for this type of task. Whether or not learning Emacs is justified is an orthogonal issue.

A programmable keyboard is another, but less universal option.

If you're willing to give up WYSIWYG, then parsing the input is an option. Near real time (for some definition of "near") is probably possible in Emacs.

</comment><comment><USER4> I'm not sure what do you mean with 'switch typing'.

On Windows, you can set Hotkeys to switch Input Language fast. [https://www.thewindowsclub.com/cannot-switch-between- input-l...](https://www.thewindowsclub.com/cannot-switch-between-input- languages)

<comment><AUTHOR> I mean I can use 1 input method to type text for multiple languages (i.e. without switching the input method).

</comment></comment><comment><USER5> It would be great to have an adaptive engine that would guess what language you type in from first letters or words . But what to do with Apple and Ringo, both are legit words in English (Ringo Star)

<comment><AUTHOR> Yeah. I just thought if anything similar to this exists. Actually there are no space separators in Japanese sentence, e.g. "Konoringohaakaidesu" (This apple is red).</comment></comment>


Show HN: LittlePSD, made by 17/21-yr-olds either side of the Atlantic in 1 week - <AUTHOR> http://littlepsd.com

<comment><USER2> Seems quite interesting, and could turn into an excellent resource.

Could do with an 'About' page explaining a little perhaps?</comment>


Filler Word Use as a Function of Age, Gender, and Personality - <AUTHOR> http://jls.sagepub.com/content/early/2014/03/26/0261927X14526993.abstract

<comment><USER2> the article is paywalled, the link only gives the abstract.

there is an informative summary here: [http://bps-research- digest.blogspot.fr/2014/06/women-young-p...](http://bps-research- digest.blogspot.fr/2014/06/women-young-people-and-conscientious.html)</comment>


Ask HN: Why does HN not use similar algorithm for comments as submissions?  - <AUTHOR>

HN's algorithm for ranking submissions is as effective of any I've seen for UGC websites, but comment ranking seems very heavily weighted to first submitted.<p>Perhaps HN does this already and time is just less of a factor than for submissions? <comment><USER2> _comment ranking seems very heavily weighted to first submitted_

Comment rating used to be much weighted to first submitted, implicitly, because later submitted comments almost never got enough karma to move into visibility. Something was done to the comment karma algorithm in the last year or two to make recency LESS weighted than before, and I think it has improved the quality of comments in many threads.

<comment><USER3> The following links are very old posts by pg, so we can't assume the algorithms stated are still accurate. None the less, they should give you a rough idea of what was tweaked to put less weight on time.

2074 days ago

<https://news.ycombinator.com/item?id=38704>

908 days ago

<https://news.ycombinator.com/item?id=1781417>

Also, it's kinda neat to see the progression by comparing the two.

<comment><AUTHOR> Thank you for the links. I was trying to find a 'source' but came up empty handed.</comment></comment></comment>


Beyond the Turing Machine  - <AUTHOR> http://www.mathcomp.leeds.ac.uk/turing2012/WScie12/Content/abstracts/kauffman.html

<comment><USER2> Not sure where the linked abstract actually was from (it's hosted with the Turing Centenary Conference, but I couldn't find a link in their public pages) - but here's the author's blog that may shed some more light:

[http://www.npr.org/blogs/13.7/2010/03/how_can_mind_act_on_ma...](http://www.npr.org/blogs/13.7/2010/03/how_can_mind_act_on_matter.html)

And then some more digging turns up reviews of his 2008 book:

[http://www.scientificamerican.com/article.cfm?id=sacred- scie...](http://www.scientificamerican.com/article.cfm?id=sacred-science)

Yep, standard "Gawd in the gaps" stuff, complete with a helping of quantum woo. Also, the originally linked abstract is full of howlers like this:

"I believe Trans-Turing Systems are constructable, go beyond Turing, and may have powerful implications for the Mind-Brain system."

I believe you believe that too, d00d - but absent something resembling even an outline of a proof or an existence argument you're just tossing quantum word- salad.

</comment><comment><AUTHOR> The article is very sparse on details, but they set up a good conceptual framework for post-classical computing.</comment>


86% of "people" dislike Facebook changes, 55% of IT workers "like". - <AUTHOR> http://www.digitaltrends.com/web/survey-vast-majority-of-facebook-users-dislike-new-redesign/

<comment><USER2> With 300 million people on Facebook (and there's probably more than that), you'd need to survey 3 million people to get at just 1% of them. This survey covers 1000, which is a fraction of 0.0000033.

In other words, a survey isn't going to tell you anything. What matters is trying it yourself, and deciding if _you_ like it or not.</comment>


The Gervais Principle III: The Curse of Development - <AUTHOR> http://www.ribbonfarm.com/2010/04/14/the-gervais-principle-iii-the-curse-of-development/ <comment><USER2> This guy is remarkable at both analysis (which is pretty common in our line of work) and rhetoric (which isn't so much). I'm not sure why but my mind keeps returning to the thought of him having the makings of a ridiculously effective attorney. Anyways, this was very well written. Definitely worth a read.

<comment><USER3> I thought this paragraph was a work of art:

 _The term "arrested development" is now deprecated by professional psychologists, and has been replaced by "developmentally disabled." This is good news for us pop psychologists hawking crude over-simplifications about functional adults, since we can safely steal the term. I assume you are not retarded enough to read this as a theory of clinical developmental- disablement._

<comment><USER4> The juxtaposition of the nuances of lay psychology against the word 'retarded' is jarring.. but I like it.

</comment></comment></comment><comment><USER5> "We can state the root cause of the Dwight-Phyllis dynamic as follows: the depth of any transaction is limited by the depth of the shallower party."

There's some interesting analysis to the post - which seems as relevant to game theory as to developmental psychology, however...

Its practical application / warning seems to me (as it resonates with my personal experience, too):

Know when not to enter a game on terms that are stacked against you.

cf: A jogger beside my car challenges me to see who can get to the office faster. When I accept he says, "well, we have to level the field: get out of your car," knowing that with us both on foot his regular training makes him the winner.

i.e., know that there are some discussions where rational-analytic ability are irrelevant. Try talking rationally about politics/religion with a dedicated know-nothing. Or convincing a 4 year-old that he'll someday be interesting in engaging reproductive parts with another human being for the pleasure of it.

<comment><USER6> The irony is that he says "well, we have to level the field: get out of your car" knowing full well that he's actually just unleveling it in his direction.

</comment></comment><comment><USER7> These are interesting, but I can't shake the feeling that someone could produce an equally well-written series that drew completely opposite conclusions and I would find it just as convincing.

<comment><USER8> This is why we see so many articles on HN about writing well. It is a skill I want to have!

<comment><USER9> My friend Jon's response:

This is really a stupid article. Attempting to generalize institutional truths from a show that sensationalizes the workings of one individual company is just silly, almost as silly as basing any kind of argument on moronic caricatures of "personality types" like "the sociopath." The show isn't brilliant because it examines the modern corporation, its brilliant because it takes a very generic, very relateable institutional setting and then uses it to examine the characters, which are not abstract caricatures but in fact very complicated and dynamic.

"If you leave out the clear marked-for-clueless characters like Dwight and Andy, you are left with the two most interesting characters in the show: the will-he-won't-he sociopath-in-the-making, Jim, and the strange Toby. Toby is a curious case — intellectually a sociopath, but without the energy or ambition to be an active sociopath."

Dwight and Andy are much more interesting characters than just "they're clueless because they like the company"- Dwight especially- and Toby is actually one of the least interesting on the show, hes mostly just there as a foil for Michael. Note as well, that this is sort of a vulgar Marxist argument, "corporations are made of sociopathic exploiters and clueless exploited," except that, unlike Marx, it rests upon the notion that the institution can be explained through the actions of specific individuals or "character types," rather than the complex social structres with which the institution interacts, most importantly the dynamics of a global economy. This is what so absurd, ultimately, about those stupid fucking books in the airport about "packing your parachute" and shit, individual firms might perform better than others based upon managerial skills, but an entire sector or economy over a longer term is going to be determined by much bigger factors than just "can the sociopath find saps to do his grunt work." Continuing with the vulgar Marxism, he points to the idea of 'alienation,' that 'clueless' people like Michael and Dwight have a masochistic relationship to the firm, which he uses to argue that this is why Michael gets promoted, the firm needs idiots in the middle because they are company men despite being idiots. Alienation, in this context, that they want to find their work meaningful but it simply isn't, it exists for some purpose other than the one that they wish to find in it. This is, of course, essentially the absurdity of Michael's character, he is a hyperbolic representation of the alienated reality of working at a corporation insofar as he has no separation between his public and private life. This isn't because "organizations are essentially run by stupid people because only an idiot would believe in an organization instead of rationally maximizing his utility," but because particular organizations- capital - are subject to a logic that is alien to the interests of the people that operate the institution, the profit motive. I think part of the reason for these idiot management books is that they attempt to deal with this basic reality- people have to be managed despite it being ultimately despotic- without being honest about the inequalities that constitute capitalism. How can you get someone to "believe" in a corporation if they aren't part of its decision making process? Anarchy in the market and despotism in the factory. It is asinine to generalize "institutions" in general without recognizing the historical specificity at work. Not only that, but it misses why the Office is such a fantastic show.

</comment></comment></comment><comment><USER10> That was just brilliant. I hadn't read the previous installations before, but I finally did. I bought him a coffee.

</comment><comment><USER11> Is anyone else troubled by the fact that these psychological studies, once you've been exposed to them, seem indispensable to everyday life, but you just can't get used to the idea of actually studying them much?

Maybe it's just the clinical or analytical nature of formal psychology that makes it hard to swallow. Perhaps reading the Proverbs or other sage literature dispenses the same helpful advice, but without the clinical self- awareness.

And, aren't psychologists the ultimate sociopaths, distorting the metaphysics or their surroundings?

<comment><USER12> Speaking as a former psych major, I don't think there's any real psychology behind his writing, aside from the basic concept of a sociopath. His writing is closer to philosophy or pop psychology, and there is nothing clinical or formal about it...

> And, aren't psychologists the ultimate sociopaths, distorting the > metaphysics or their surroundings?

No. Writing a paper about SQL injection attacks is much different than using a SQL injection attack to grab credit card numbers.

<comment><USER13> _there is nothing clinical or formal about it_

But do you think he's wrong? and if so, where?

<comment><USER12> I don't think my psych major qualifies me to say that any more than anyone else.

<comment><USER13> It's one of those essays that _sounds_ deep, but I am questioning exactly how deep it really is. Oh well, I am still questioning then.

<comment><USER14> It's another prism. Add it to your toolkit, apply, observe. Don't ever settle for just one, though.

<comment><USER12> Exactly. His ideas aren't science, even social science.

His philosophy is a prism--like analyzing a story in terms of Jungian archetypes, or a picture in terms of phallic imagery. There isn't a falsifiable theory in there anywhere. Yes this applies to Freud too, which is why his ideas have largely been discredited and ignored in mainstream psychology for years.

</comment></comment></comment></comment></comment></comment></comment><comment><USER15> What I really enjoy about these is that they are written from the point of view of his "sociopath" rather then the more conventional "loser" observer.

You get a sense of being disgust at all these "transactions" leading nowhere.

</comment><comment><USER16> If you read the previous articles on a small little screen like my laptop, this will turn the scroll bars into a reasonable size by hiding all of the comments:

      javascript:void(jQuery('dd, dt').hide())

<comment><USER17> You might enjoy viewing the article using Readability. Not only does it make the text bigger and easier to read, it suppresses comments and other additional content.

<http://lab.arc90.com/experiments/readability/>

</comment></comment><comment><USER18> Hollywood loves to take a small, clever European movie and turn it into a big, stupid American one. This strikes me as the application of the Hollywood process to Stephen Potter's _One-Upmanship_.

</comment><comment><USER9> Yet another emotionally unpleasant yet viciously necessary dose of reality.

<comment><USER15> Which part did you find unpleasant.

<comment><USER19> The ones that hit home ?

<comment><USER15> Do you want to elaborate?</comment></comment></comment></comment>


"Google just started mass banning/limiting Archive Team downloads" - <AUTHOR> https://twitter.com/textfiles/status/1112494767601053696 <comment><USER2> According to ArchiveTeam's own tracker, there was a temporary dip around 1 AM PT (when this tweet was posted), but the speed has recovered and they are again crunching through 100k+ items/hour.

[http://tracker.archiveteam.org/googleplus/](http://tracker.archiveteam.org/googleplus/)

(the graphs show up in the pink block at the bottom, which can take a while to render)

Where is the graph in the tweet from? If it's just measuring successful downloads, how did it conclude that it's Google at fault? Is there another tracker of download failures that shows quota errors/DOS blocking etc?

<comment><AUTHOR> I've been in the ArchiveTeam's IRC channel for ... the past three months or so. This wasn't just a temporary dip.

Workarounds are applied, but Google not fighting us would be a Really Nice Thing presently.

(Google's performance throughout this episode has been ... poor.)

The tracker is also only _very_ partial, showing about 1/50th of all activity at any one time.

A better picture is at the Grafana tracker:

[https://atdash.meo.ws/d/BQbN9QEiz/archive-team-tracker- chart...](https://atdash.meo.ws/d/BQbN9QEiz/archive-team-tracker- charts?orgId=1&from=now-6h&to=now)

<comment><USER3> Is there anything that says how many items haven't been added to the download queue yet? Or a rough sense of the overall percentage somewhere that I'm missing?

Edit: I see the post a few days ago said about 80%, that's at least something.

<comment><AUTHOR> 88% complete, on target to hit 92% of targets.

<comment><AUTHOR> And actually hit 98.5%

</comment></comment></comment></comment></comment><comment><USER4> The front page of Hacker News - the only way to get your support issue with (insert big tech company name here) fixed.

As I keep suggesting, these big companies each need their own internal ombudsman, because at the moment they provide no way to fix things when their systems have gone crazy.

The other reason (big companies X/Y/Z) need to each have an internal ombudsman is that it's getting tedious reading on Hacker News about how (big company X/Y/Z) has done something crazy and the user/developer/customer cannot get it fixed.

Maybe its an entrepreneurial opportunity - someone could make a website that pays a fee to someone with Google/Microsoft/Amazon who is willing to use their influence to solve a crazy problem. "Rent a manager friend at Google".

<comment><USER5> You just gave me an idea actually...

I don't have time to implement it but if you like it go ahead and steal it.

Basically a Hacker News style-site for complaining about companies that don't fix their problems or own their support issues and their user base can upvote them to embarrass them into actually fixing the problem.

<comment><USER6> Doesn't this exist?

<comment><USER7> "Get Satisfaction" partly fits this description. [https://en.wikipedia.org/wiki/Get_Satisfaction](https://en.wikipedia.org/wiki/Get_Satisfaction)

</comment></comment></comment></comment><comment><USER8> >talk to Google

Is that even a thing?

Google doesn't seem to want to be talk'n to outside of my information and in some cases credit card.

<comment><USER9> Historically if Google wrongs you, the only ways to get it fixed are:

1\. Get lucky and some googler sees your official bug report

2\. Know a googler who can file a bug internally

3\. Make noise on social media till someone at Google notices

<comment><USER10> I've uploaded a lot of data to Google(mostly photos) and someday if Google decides that I'm in violation of their policy and chooses to lock me out of system I'm screwed.

I need to start thinking about proper storage system for my pictures and other data.

<comment><USER11> I use hard drives.

</comment></comment></comment></comment><comment><USER12> It is 4pm on a Sunday at Google HQ. It seems a bit premature to start reading deep policy decisions into this.

</comment><comment><AUTHOR> Context:

Saving of public Google+ content at the Internet Archive's Wayback Machine by the Archive Team has begun

[https://old.reddit.com/r/plexodus/comments/az285j/saving_of_...](https://old.reddit.com/r/plexodus/comments/az285j/saving_of_public_google_content_at_the_internet/)

Previously on HN:

[https://news.ycombinator.com/item?id=19407865](https://news.ycombinator.com/item?id=19407865)

<comment><USER13> Ah thanks, context helps. Maybe the G+ servers just have security measures to limit bandwidth usage? Do you think it's malice?

<comment><USER14> No, it's just stupidity if they weren't stupid this entire effort would be unnecessary as they would be sending over the content to the Internet Archive themselves, probably along with a cheque that looks big to the IA (10 mil/yr budget) and is not even a rounding error to G (109B cash at hand).

<comment><USER12> Why would that be good?

<comment><USER3> Because there is a huge amount of important public posting on there. It didn't get billions of users but it had enough. Keeping up a read-only blob, or even better letting someone else do it, should be in the shutdown considerations of any major product.

<comment><USER15> G+ had a nerdier audience and many interesting communities and posts. I don't get this ghost town line. For example HN is a niche site, with almost no audience, with almost all accounts dead. Right?

<comment><USER12> I ran several G+ groups and they were destroyed over time by unfettered harassment, unstoppable spam, and just a lack of interest.

I've yet to see these things happen to HN.

<comment><AUTHOR> HN has niche appeal, a strongly focused discussion (there's one story feed, and effectively about 30-40 stories that really register on the front page, though many more are submitted), and pretty dedicated professional moderators. As of 2015:

 _Roughly 2.6M views a day, 300K daily uniques, 3 to 3.5M monthly uniques. It depends on how you count, of course._

[https://news.ycombinator.com/item?id=9220098](https://news.ycombinator.com/item?id=9220098)

Which ... actually probably compares favourably with Google+, which had a core of about 50-100k highly active users (posting 50-100x monthly), and maybe an extended set of as much as 100 million who'd interacted with the site at one time or another significantly.

I've done a fair bit of measurement (limited by available resources and indicators), and one conclusion I'm coming to is that raw numbers do a _pathetic_ job of indicating media or forum vitality. Most especially raw census numbers.

Looking at G+ communities, and running grid plots of _engagements_ , it turned out that _posts_ drove other engagement, not _members_ , and in fact it seems as if there's some kind of fall-off (at least on a per-member basis) when a given forum gets above about 5,000 members (though I need to check this).

Google had more users. But they were spread out over a vastly larger set of forums and discussion, there was no central "square" (as with HN's "new" or "news" pages), moderation was exceedingly uneven, and often entirely absent, and there were (and remain) huge barriers for like minds to come together.

HN's overall focus is fairly (but not excessively) narrow, and much of the conversation takes itself too seriously (and certainly myself), but _relative to the rest of the Net_ it's an exemplar. Good conversation remains exceedingly hard to find.

<comment><USER15> > it turned out that posts drove other engagement, not members

Yes, this was something particular to G+. Incidentally see these in Linus interview [0], HN post [1]

" The whole "liking" and "sharing" model is just garbage. There is no effort and no quality control. In fact, it's all geared to the reverse of quality control"

"I'm not on any social media (I tried G+ for a while, because the people on it weren't the mindless usual stuff, but it obviously never went anywhere)"

[0] [https://www.linuxjournal.com/content/25-years-later- intervie...](https://www.linuxjournal.com/content/25-years-later-interview- linus-torvalds)

[1] [https://news.ycombinator.com/item?id=19559970](https://news.ycombinator.com/item?id=19559970)

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER16> Did they bother talking to anyone at Google before they started doing this? It sounds like they're making a massive number of requests in a short period of time which is probably indistinguishable from abuse to Google's automated systems.

<comment><USER17> "Did they bother talking to anyone at Google before they started doing this?"

How would you go about contacting Google? As far an I've ever heard they're notoriously impossible to contact by anyone who isn't paying large amounts to run ads.

<comment><USER18> You don't even have to pay much for ads to get help. They will do a call and walk you through the steps for setting up everything to start spending. Its just that advertisers are the only customers google has.

<comment><USER19> Only in the past year or two. Before that you could only contact them via a support form and someone in India would answer the support.

</comment></comment></comment></comment><comment><USER20> What's really stupid about this is that it would have been less effort and expense for everyone involved if Google had arranged to work with Archive Team directly instead of forcing them to scrape content via HTTP over the public internet. That includes Google themselves. What's happening now is probably that they're ramping down the number of servers handling Google+ content, which they could have done even sooner if they'd cooperated on a proper archiving strategy.

I guess sometimes Google does this kind of crap just to prove (mostly to themselves) that they can. Truly, the bully of the internet.

<comment> identity-haver There was a claim [1] that the G+ terms of service might legally prohibit them from doing this after the service is shut down. I haven't verified it.

However, it's clear that for an archiving effort this big, people at Google are explicitly allowing it. The user agents and fetch patterns of the Archive Team crawler were clearly distinct enough to get caught by an automated tool, and someone knew someone at Google in order to get it unblocked.

Unfortunately, any archival effort that requires the "Warrior" crawler (and not just a guy with a 4TB disk) is at the mercy of the website's remaining staff and management. Just ask Soundcloud. Archive Team started to archive their stuff when it looked like they were going to go under, but Soundcloud shut them down.

[1] [https://news.ycombinator.com/item?id=19410050](https://news.ycombinator.com/item?id=19410050)

<comment><USER20> That's a really good point. OTOH, I think it would be nearly impossible for anyone to make a claim that their privacy has been violated by archiving _public_ posts. In that case, rights have been granted to everyone (i.e. the rights Archive Team is currently exercising without issue) so limitations on rights granted to Google itself are irrelevant. OTOOH, IANAL. ;)

</comment></comment></comment><comment><USER21> In my experience, scraping Google data is _hard_. I did it years ago for a project. And I had to lease a huge block of private proxies. Each one only lasted a few minutes. But with a large enough block, they'd come back.

<comment><AUTHOR> Google Web Search especially -- I've found that more than a query every 5-10 minutes will start throwing CAPTCHAs. For what I was doing, there was no other way to get the information I was looking for, so I just resigned myself to very slow crawls.

For Google+ itself, over a period of _years_ , I'd hammered it with 100s to ~100k requests from residential IP space without ever throwing rate-limiting, at a rate of 2-20 queries/second or so, roughly.

We've started getting news of rate limiting over the past few months as archival activity has proceeded.

</comment></comment><comment><USER22> I mean enforcing traffic control restriction is everyday operations for big companies, why would they assume that Google will treat them differently?

</comment><comment><USER23> I'm getting lots of 500 errors on Google Plus these hours, so maybe they're not getting throttled, just Google servers crapping out.

<comment><USER24> > Google servers crapping out.

This seems....highly unlikely for Google.

<comment><USER25> You've never opened Youtube, have you?

</comment></comment></comment><comment><USER26> You don't get it guys, when Google scrapes the web and downloads everyones data then serves up parts of it with sponsored ads next to it in searches its OK because they are Google. But if you scrape their data it's not OK because you're not Google. Once you understand this it makes perfect sense.

<comment><USER27> In case this is meant to be a serious comment, there's a standard mechanism called the robots.txt file to tell crawlers you don't want them to scrape your website. You don't have to let them if you don't want to.

<comment><USER28> Except archive.org _doesn't_ obey robots.txt files any more [1], and they also ignore requests to remove content.

[1] [https://blog.archive.org/2017/04/17/robots-txt-meant-for- sea...](https://blog.archive.org/2017/04/17/robots-txt-meant-for-search- engines-dont-work-well-for-web-archives/)

<comment><USER29> They don't obey robots.txt files _posted after-the-fact by domain hoarders that have zilch to do with the original content_. This is entirely proper on their part.

</comment></comment></comment></comment><comment><USER15> How can I archive my photos from G+ posts i.e. this [https://get.google.com/albumarchive/110322266958783287132/al...](https://get.google.com/albumarchive/110322266958783287132/albums/photos- from-posts)

<comment><USER30> In theory you can use Google Takeout. In reality Takeout has NEVER been able to backup my YT comments, and there is no way or reporting a bug/appealing/speaking to anyone about it.

<comment><USER15> I made available all the sources for all those animations, but this will not bring back 150K views/day. It's a shallow metric of interest, I know, however I am human too and I appreciated that, seen that presently there is, technically, no scientific publication avenue for this. [https://doi.org/10.6084/m9.figshare.4747390.v1](https://doi.org/10.6084/m9.figshare.4747390.v1)

</comment></comment></comment><comment><USER31> Banning downloads of what, exactly?

<comment><USER32> Google+ before Google deletes it in a few days.

<comment><USER33> A few days? I think it's tomorrow.

</comment></comment></comment><comment><USER34> What do they mean by this? I did notice that looking for archives of a few domains, including my own returned no results.

<comment><AUTHOR> Google+ is shutting down on April 2.

There's a bunch of folks who are interested in preserving content from the site. Some personally/individually, a whole host of communities, and, well, because their mission is to preserve the world's data, the Internet Archive and an unaffiliated though closely-working group, the Archive Team: [https://archiveteam.org](https://archiveteam.org)

That's a bunch of volunteers who basically suck the guts out of failing websites, and they've got some big ambitions.

I'm from the Google+ user community side and have been helping organise information and activity on behalf of others. There's a pretty comprehensive (and messy) wiki at [https://social.antefriguserat.de](https://social.antefriguserat.de) with FAQs and directories and guidance and a whole bunch of other stuff, plus other resources -- G+ communities, subreddits, and more.

We (the G+ community side) stumbled across Archive Team this past January and were delighted to discover they existed. (They've told us they're also glad they exist.) And we've been working together to coordinate this pull, with AT providing the technical capabilities -- code, servers, bandwidth, and storage -- and us mostly pestering them with questions but also knowing a good bit about the characteristics of Google+ data and some of its organisation. (I've made a hobby of ... measuring much of that.) I was finally able to answer my mother's persistent question "but who _cares_ about any of this" with "arkiver, of the Archive Team".

Anyway: the Archive Team grab is in full swing, we've got about 24-48 hours left, depending on when Google shut shit down, and all of a sudden all the archiving agents ("Warriors") stop collecting data. We're 86% through the working set, and are on track to complete 92% of all scheduled data requests by the earliest anticipated shutdown window. (Earlier estimates, before the throttling, were over 100% completion, meaning we had some slack space.)

Answer your question?

<comment><USER34> Yes, thank you</comment></comment></comment>


48 people were shot during yesterday's 15-hour filibuster on gun control - <AUTHOR> http://www.vox.com/2016/6/16/11952166/filibuster-gun-control-shootings <comment><USER2> Yes, but did they deserve it?</comment>