Starlink WiFi Router FCC Approved - <AUTHOR> https://fcc.report/FCC-ID/2AWHPR201 <comment><USER2> It looks like starting out they're going to prioritize rural Washington State. From the FAQ which is buried for some reason at [https://www.starlink.com/main.16ae0a3588c339b10118.js](https://www.starlink.com/main.16ae0a3588c339b10118.js).

q:Who can participate in Starlink Beta?

a:Starlink Beta will begin in the Northern United States and lower Canada, with those living in rural and/or remote communities in the Washington state area. Access to the Starlink Beta program will be driven by the user's location as well as the number of users in nearby areas. All beta testers must have a clear view of the northern sky to participate.

<comment><USER3> > prioritize rural

It's worth pointing out that this will _probably_ be something only people in rural areas will be interested in. "High speed" is in reference to the utter garbage available to these people. The throughput won't be close to what you can get from copper/fiber in an urban/suburban setting.

If you want to stick it to Comcast+co, this is unlikely to help you cut your cable [internet]. We'll still need to vote and apply pressure to end this oligopoly.

<comment><USER4> Nope.

"As confirmed by the company, Starlink will be able to provide speeds of up to a gigabit per second with latencies ranging from 25 milliseconds to 35 milliseconds. Elon Musk has previously stated that it has been designed to run real-time, competitive video games."

It will be plenty good to replace anything else available out there for home internet.

<comment><USER5> As has been pointed out on every Starlink thread, problem isn't latency or speed, it's contention. Each satellite only has 20gbit/sec per beam of capacity. It's likely even in small cities that will not be enough (it's only 1-3k users streaming Netflix for example). There is no way you can connect a city of even 100k people with 20gig/sec of capacity these days, never mind major metro areas many times that size.

Elon Musk has admitted this himself, and the problem gets worse over time (bandwidth requirements grow every year, but satellites capacity is fixed), so the 20gbps they have now will seem even more limited in 3-5 years when the satellites are becoming EoL.

This is not to say that Starlink is fundamentally flawed, it has great potential for rural access.

<comment><USER6> I have friends on the Indiana Ohio border that are only offered ISP from a monopoly, ISP offers one plan $45 mo 60/40mbps. They have the first fiber optic Network ever laid in the country from adelphia cable but Time Warner Cable bought it and turned it off 20 years ago.

<comment><USER7> I just gave Starlink my address as per the email request they sent out today.

I'm currently on an 5Mb down 1Mb up LTE connection with a total data limit of 250Gb/Month for the low price of 100 bucks a month. I do have very low latency < 20ms, but last month my friends kid was over and managed to burn through 95 gigs playing and updating some game called PubG that ended up putting me over my limit and I got to pay an extra 50 bucks for the next tier of usage.

If I get the opportunity to try out Starlink I will be more than happy to do so as it's very frustrating being forced to use an ISP monopoly provider, especially when there is fiber available to properties less than 5 miles away from mine.

<comment><USER8> You should really check out Visible. Unlimited 4G data, even on the hotspot.

$25 per month if you join a party, which takes about 5 minutes after searching for "Visible Party".

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> I really hope Starlink works out. I have relatives whose only option is satellite internet and with the ultra-low data caps streaming video is just not an option at home. Imagine living without Netflix or Youtube.

<comment><USER10> _Imagine living without Netflix or Youtube._

You mean, the way I and several billion other people do right now?

<comment><USER11> if you're implying that people in the developing world live without internet access, you'd be dead wrong. Cell service is ubiquitous and data is cheap. Those people are probably better-connected than rural Americans (not suburban, but truly rural, the people living out in the sticks).

American data connectivity is just that bad. It's worse than eastern europe. Worse than Asia. Worse than Africa.

<comment><USER10> _if you 're implying that people in the developing world live without internet access, you'd be dead wrong_

I imply no such thing. You're projecting your own biases.

I'm merely stating a fact: That not everyone uses Netflix and YouTube, and to consider those life essentials is absurd.

<comment><USER12> Did you forget the part where you said "several billion other people"?

If you're not talking about the developing world, and just look at the rural and those who don't want a connection, that's not _nearly_ enough people. So it's not wrong at all for people to assume you meant the developing world.

And nobody said they were 'life essentials'.

<comment><USER13> To be fair to the other person, China alone (in which neither Netflix nor YouTube are widely available) is almost 1.4 billion people. That could easily be part of their calculus, though I guess you could argue over the meaning of "several" in this context.

<comment><USER12> I see that as pedantry. We're talking about internet speeds, and the _equivalent_ of youtube.

<comment><USER13> _> We're talking about...the equivalent of youtube._

Well, _you_ certainly are, but I don't think the person who originally brought it up was.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER14> Today I received an email message from SpaceX/Starlink to obtain my "service address." Previously I had provided my email (more than once, actually) on the Starlink "beta" sign-up page. This is the first communication I've seen from them.

<comment><USER15> Got one in London too. Seems like just a standard thing though, not getting too excited... yet

<comment><USER16> Yep, same in Milan, Italy! I think they just realized zip codes are not so useful once you're dealing with the whole world ;)

</comment></comment></comment><comment><USER17> TBH, I'd really like to work for the FCC doing this stuff. I know it's grunt work, but it would be such a great end-of-career job to scrutinize hardware to the nth degree, and just have to worry about doing a precise job, not a fast one.

<comment><USER18> It's a third party commercial test lab that does this stuff, the FCC just receives the filings. There's a high degree of trust put on the test lab to meet certain engineering standards.

<comment><USER19> Not sure if that's a good thing. I know that structure exists all over the world though, in countless regulation contexts. So in practice, maybe it's OK.

<comment><USER20> how else could you do it? you expect bureaucrats to inspect the hardware themselves?

</comment></comment></comment></comment><comment><USER21> Interestingly, this device seems to _only_ be a WiFi router. The test results don't mention any protocols other than 802.11, or any frequency bands other than the 2.4GHz/5GHz bands used for WiFi. So the actual satellite communications must be handled by something else.

<comment><USER22> The actual modem that talks to the sats is in the antennas you mount on the roof. They are calling that the "terminal"

</comment></comment><comment><USER23> What's significant about this? Seems to be plain 802.11ac router.

<comment><USER24> The significance is that it's related to the much anticipated (by people with poor internet access) Starlink satellite internet service.

[https://www.starlink.com/](https://www.starlink.com/)

People would be just as interested if it were just a plastic mounting bracket with the "Starlink" branding.

<comment><USER25> I have fiber to the home that is super reliable... I am still anticipating this and will be signing up to supplement my fiber

so it is not just for people with poor internet

<comment><USER24> Well... wait until you see pricing. Even if it comes in at the speculated $80/month price point, many people will find it cheaper and faster to stick with their terrestrial internet. There aren't many people willing to pay $80/month for a backup connection they'll only use once every month or two. (I'm one of them, I usually have at least one Comcast outage a month, from a few hours to a few days). I'm in a bit of a celluar dead zone, I get one bar of signal from AT&T or Verizon, and only if I walk up to the second floor)

<comment> asdfk-12 In Alaska, specifically, gigabit internet costs ~$200/mo through Liberty Media's subsidiary GCI in the three major cities. In rural areas, GCI offers plans through local co-ops averaging $150/mo for 6mbit down, 100gb data cap over terrestrial microwave relay. Starlink will normalize this market.

<comment><USER26> Starlink will not cost $80 for the gigabit plan. I'd wager there won't be a gigabit plan. They need _thousands_ of customers per 20Gbps satellite for this to even start to make sense. You cannot fit even a couple hundred 1Gbps customers on a single satellite.

<comment><USER27> You can certainly fit a couple hundred 1Gbps customers on a 20GBps satellite provided you make sure that those people don't completely saturate the entire line at the same time.

<comment><USER26> Yes, I know how the modeling works. For a 100Gbps subscriber they will need to guarantee that they can achieve 100G peak speeds when nobody is using it. That means the beam the user is in always has to be illuminated with as much power as they need for 100G. Remember, this power is shared between all the other beams on that satellite.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER28> If anyone from Starlink is reading this thread: Add a Lat/Lon option for addresses in the service address entry form.

Many rural homes have mailing addresses that don't resolve correctly. For example, the input form forced me to use "Anacortes, WA" for my house in the San Juans. The islands are a perfect Starlink test market, whereas Anacortes has plenty of broadband options.

</comment><comment><USER29> I am buying an RV this week. Both me and my wife work fully remotely for 5 years now. Having a satellite internet terminal on the roof would make it... way more interesting.

</comment><comment><USER30> Starlink will be great in rural Canada where the ISP oligopoly has no incentive to provide fast internet. I am curious how it will perform in stormy weather. Does it cut out like satellite TV?

<comment><USER31> Heck just living in the wrong neighborhood in freaking Orange County California can mean the only options are att with 5mb dsl or cox charging ridiculous amounts for cable.

Im not even talking about the outskirts of the OC. If you happen to live in the wrong part of town or even the wrong building; your current internet choices could be drastically worse than the house 100 yards away.

A couple blocks down my neighbors get att fiber and google fiber. I get tons of ads for those fiber services just so they can rub in my limited choices of paying att ridiculous amounts for bad dsl or Coxs even higher prices for the lowest tiered cable.

<comment><USER32> I used to live in walking distance of downtown Mountain View (a short bike ride to Google HQ). It was similar to what you describe.

I think we ended up with less than 6MBit theoretical download. Later, we moved a few blocks and got 10x that.

</comment></comment></comment><comment><USER33> I really like that competition is coming, but hate that it is in area that again no one else can compete.

To bring competition all what's really needed is a legislation mandating that the last mile can be leased to any company (for a reasonable fee of course) and we would get back once again to times when we needed sites like dslreports.com to decide which ISP of so many in your area to use.

<comment><USER34> OneWeb went bankrupt, but is being apparently bought by a consortium that includes the UK government. It may well compete on some level.

Amazon is also considering a LEO megaconstellation called Project Kuiper.

<comment><USER33> It doesn't bring confidence to me that the other competitor is company run by the richest man in the world.

I meant a real competition.

Most likely the way it will happen is that it will be really nice for customers with Amazon offering lower prices until SpaceX is driven out of business and only Amazon is left.

</comment></comment></comment><comment><USER35>
        Frequency Range
        2412.0-2462.0
        5180.0-5240.0
        5745.0-5825.0

Looks like your standard wireless AC router (2.4ghz and 5ghz).

Whatever this is, it doesn't "talk" to starlink, per its FCC application:

> _In this application, SpaceX proposes to operate in the 10.7-12.7 GHz, > 13.85-14.5 GHz, 17.8-18.6 GHz, 18.8-19.3 GHz, 27.5-29.1 GHz, and 29.5-30 GHz > bands._

[https://www.fcc.gov/document/fcc-authorizes-spacex- provide-b...](https://www.fcc.gov/document/fcc-authorizes-spacex-provide- broadband-satellite-services)

<comment><USER22> I'd bet this is the box you place inside your house to serve Wifi, and run ethernet up to the roof to connect to the modem/antenna on the roof, hence this being essentially a bog standard AC Wifi Router.

</comment></comment><comment><USER36> For those unclear on the concept: Starlink is SpaceX's satellite network. The Starlink Wifi router will connect to that satellite network to provide wireless home internet.

<comment><USER37> Would this be that UFO-looking thing you stick in the ground outside. Or this is literally a router(box in home) that connects to what before connecting to the satellite(s)?

<comment><USER38> box in home router. Though it could have special equipment to further process the satellite signal, that's likely to be primarily on the 'dish' itself.

</comment></comment></comment><comment><USER39> I can't wait for it to reach me in rural Arkansas. I'm on a 4g lte connection. Best option at my house.

</comment><comment><USER40> So they're probably going to use the same scammy techniques as other ISPs where they require you to rent a $100 standard router for $200 per year.

<comment><USER22> Unclear until the pricing comes out, they may just include it for free when you buy internet service. But ISPs generally need to package at least something to serve Wifi for customers that don't already have their own Access Point, looks like SpaceX just put together their own, rather than provide something off the shelf.

</comment></comment><comment><USER41> Presumably the NSA wouldn't let something like Starlink get off the ground unless they had a way of listening. How are they addressing this?

<comment><AUTHOR> AFAIK "end-to-end encryption encoded at firmware level" is the current official answer: [https://www.reddit.com/r/Starlink/comments/8049u6/starlink_t...](https://www.reddit.com/r/Starlink/comments/8049u6/starlink_to_be_endtoend_encryption_encoded_at/)

<comment><USER42> I find the second part of that statement more interesting:

> If it is (and we learn about it), a crypto fix will go out immediately via > network-wide firmware update.

This implies that the SpaceX holds private keys for signing firmware updates, and a mechanism for pushing them.

I hope those keys are kept very far from anything Internet-connected.

<comment><USER43> I mean even if you own your own private modem, Comcast (and others) can and do push new firmware directly into your modem. I don't see how this would be any different.

<comment><USER42> I'm not sure how you think those ISPs supposedly push firmware to private 'modems' of all shapes and sizes, but I would like to know because I had a hell of a time reflashing my router with OpenWRT.

<comment><USER44> It's because cable modems are standardized with a set of upstream management protocols. They have a looong history of fleet management [1].

If you have cable internet, try going to 192.168.100.1. [2]

[1] [https://en.wikipedia.org/wiki/DOCSIS](https://en.wikipedia.org/wiki/DOCSIS)

[2] [https://www.computerworld.com/article/2880475/talk-to- your-m...](https://www.computerworld.com/article/2880475/talk-to-your- modem.html#:~:text=Most%20modems%20in%20their%20list,10.1%20or%20192.168).

<comment><USER42> You're referring to the CPE which is owned and managed by the ISP just as they own and manage other routers in their network.

The other person said that they can push firmware to private modems though, which is not possible without that modem being configured to accept such pushes.

Perhaps you're thinking of CWMP (TR-069) which is a protocol that ISPs often use to push _configs_ to modems configured to receive them, but again, it cannot be used to arbitrarily flash the firmware of private modems.

<comment><USER44> Motorola's documentation says that firmware updates to _my_ modem are pushed by the ISP, with the rollout managed by the ISP. It may be CPE, but I purchased it retail and it is not linked to any particular ISP.

[https://motorolamentor.zendesk.com/hc/en- us/articles/2160917...](https://motorolamentor.zendesk.com/hc/en- us/articles/216091737-How-do-I-update-the-modem- firmware-#:~:text=All%20firmware%20updates%20are%20handled,update%20cable%20modem%20firmware%20manually).

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER45> The one thing I find iffy is that the current generation of Starlink Sats aren't supposed to do Sattelite to Satfelite communication, so the actual latency benefits around the globe aren't going to materialize the first couple of years. I'm really interested when they plan to have their service to be actually competitive latency-wise.

<comment><USER18> The latency will be about 20ms higher than a terrestrial pure fiber based internet customer in the same location as the earth stations. For example whatever the latency is from the north bend WA site to downtown Seattle (a few milliseconds) plus 20ms. This is a considerable improvement over the absolute bare minimum rtt ping time on a Geostationary based VSAT which is 492ms.

There are a lot of terrestrial wireline last mile access technologies that already introduce 15-25ms latency on the last mile segment. Such as ADSL2+ on old POTS copper wiring or a heavily loaded docsis3 cable network. That is just from the home modem to the neighborhood's closest network node.

<comment><USER46> Yep just tested this morning, 33ms to my next hop on the network. ~15M/800k ADSL2 in Ohio.

I think folks in regions of Ohio will provide an interesting test bed as we're in fringe coverage of 4-5 of the total set of planned ground stations.

(Speaking of ground station, would be interesting if we could talk to starlink over AWS's ground relay service.)

</comment></comment></comment><comment><USER47> Good, the best live on a paved road and I can't even get DSL. I got a garbage mobile connection with a 80ms ping time to google and im paying out the ass for it and it is unreliable as fuck.

</comment><comment><USER48> So, will they look like little rockets?

[https://fcc.report/FCC-ID/2AWHPR201/4805891](https://fcc.report/FCC- ID/2AWHPR201/4805891)

<comment><USER10> Or Star Fleet comm badges?

</comment></comment><comment><USER49> Indoor operation only. This is just a router that will probably have an external device connected to another antenna.

</comment><comment><USER50> Military should be interested in this, they also pay more than sparse civilians in the middle of nowhere.

</comment><comment><USER51> Can someone ELI5 what is going on? Is it that I am non-US resident or did I miss something important?

<comment><USER52> See [https://news.ycombinator.com/item?id=23841492](https://news.ycombinator.com/item?id=23841492) for FAQs. Summary: SpaceX are bringing out broadband for folk in remote areas / some progress has been made.

</comment></comment><comment><USER53> I'd give my left arm for a 12v or 24v DC announcement. IPx67 would be incredible.

<comment><USER32> Seriously. I don't want to run a new trench for this when PoE would do.

<comment><USER43> The dish is powered by PoE apparently, single cable for the dish itself.

</comment></comment></comment><comment><USER54> My mom lives on Fox Island, and has no access to the Internet aside from LTE, which is very expensive and unreliable. Hoping she'll get a look from the Starlink team!

</comment><comment><USER55> Nice. But I want the application for the satellite transceiver. I can see their signal but I'd like to be able to demodulate them too.

</comment><comment><USER56> Does anyone know if all the Starlink satellites are operational? Did they send up any duds so far? Or have any failures post launch?

<comment><USER57> They've deorbited 5 of the 540 launched so far, and a few more of the first launch seem to be derelict.

</comment></comment><comment><USER58> A Starry employee told me they're going to start offering Starlink Internet next year in rural areas. In my city they're offering a $50/month 200-300 mb/s service using Air Fiber from Ubiquity.

<comment><USER18> that's not the same thing at all

<comment><USER58> I asked him about 30 questions on how the system worked and what he was doing. He said Starry Internet was going to MARKET a service with Starlink / SpaceX. He wasn't talking about using Air Fibers in rural areas -- he said satellites. Maybe he was mistaken but I was not.

<comment><USER18> starlink very definitely has not committed to working with any third parties for anything - either install services or value added services.

<comment><USER58> Yes I was skeptical too. Just saying what he told me.

</comment></comment></comment></comment></comment><comment><USER59> Awesome news !!

</comment><comment><USER60> In 5 yrs Bezos will be polishing Musk shoes

</comment><comment><USER61> Who else is hoping to run a bonded internet connection with their existing ISP and Starlink? :)

</comment><comment><USER62> No, I do not consent to yet another source of EMF blasting me at all times. I also do not consent to my view of the cosmos being permanently altered. Hope this idea, Starlink and all future competitors (it won't be just Starlink of course) go belly up.

<comment><USER63> Do you not consent to space existing? You have far more exposure to all sorts of interesting radio frequencies from space every day than you'll ever see from a small radio.</comment></comment>


Running Zoom in a (simple Ubuntu) sandbox - <AUTHOR> https://github.com/alexjung/Run-Zoom-in-a-Sandbox <comment><AUTHOR> (Not needed for most readers here, but the instructions are simple enough for almost anyone to follow.)

</comment><comment><USER2> I used to do this, now I use a QEMU VM.</comment>


How to boost your Vim productivity - <AUTHOR> http://sheerun.net/2014/03/21/how-to-boost-your-vim-productivity/ <comment><USER2> Step 1 for those who _really_ want to boost productivity (but let's be honest, most of us don't!): forget about all those plugins that are time sinks to configure and integrate with each other, about obscure tweaks that take half a day to get right, and about elaborate .vimrc's that take you so far away from 'standard' that you're crippled without them. Stick to a fairly basic standard install and a .vimrc of 50 lines, tops.

I've been using vim for 15+ years now, since late 1990's. I use viemu in Visual Studio and I've written code in 10+ languages in vim. I've written non- trivial vimscripts and integration tools with other parts of my workflows over the years. I love how comfortable it feels, and I feel crippled when I have to use a 'normal' editor; like walking with a little pebble in your shoe - not a big a deal enough to prevent you from doing anything you could otherwise, but still damn uncomfortable.

With that said, I'm not convinced that I've actually _saved time_ with it. I shudder if I think of the many man-days I spend 10 years ago on getting this and that plugin working, tweaking settings and keybindings for things I use twice a month tops and automating things that I could have done manually 10 times faster. It's not like regular editors are _that_ much slower to use. It felt like throwing off a yoke when I dumped all those plugins so that I could focus on writing software, instead of tweaking the editor so that I could write programs with a few keystrokes.

So yeah, that was gramps' advice I guess...

<comment><USER3> I started feeling the same, about the configuration time sinks ... my solution was to create ansible-configurable docker images with everything setup ... so I can throw away and recreate the images once a month, when the plugins get screwed up (which they really do, about once a month or so!):

\- Go version: [https://github.com/samuell/devbox- golang](https://github.com/samuell/devbox-golang)

\- Python version: [https://github.com/samuell/devbox- python](https://github.com/samuell/devbox-python)

I love being able to SSH in to a fresh docker (yes, I use fat SSH:able docker), and enjoy full auto-completion for python and Go! :)

But yeah, except for the auto-completion and a few minor tweaks, there's not too much extras there.

<comment><USER4> Why not handle the configurations with git? Handling dotfiles with git is pretty common nowadays, and you can roll back/forward as you please.

Plus, the overhead is a lot less.

</comment></comment></comment><comment><USER5> In my 6 years of coding and writing LaTeX in Vim, I have found that the greatest boost to my Vim productivity is learning all about the vi and ex foundation of Vim. My Vim knowledge applies 100% to any default Vim installation on a modern distro, and my vimrc contains mostly trivial tweaks.

This means using :normal and macros instead of :s for most of my search/replace actions; using H, M, L, {, } to navigate quickly in the lines visible on the screen, and using f/F/t/T/;/,/% to navigate quickly within a line.

I would say that 98% of my autocomplete needs are fulfilled by token completion (:help i_CTRL-N) and line completion (:help i_CTRL-X_CTRL-L).

I frequently use the command-line window (:help c_CTRL-F) and listing matching names (:help c_CTRL-D).

I specifically don't have mappings that involve the leader key, and I don't use the Ctrl-P plugin or a package manager or anything like that -- I honestly don't think that mapping <Leader>w to :w<CR> will make me any more productive in Vim.

<comment><USER6> > This means using :normal and macros instead of :s for most of my > search/replace actions

Interesting. Can you please elaborate more on this?

<comment><USER5> Suppose a range of lines contains function calls in some programming language:

        i = foo(a, 42, 1)
        j = bar(b, c, 100, 2)  # this is a comment (well, duh)
        bazbar(etc, 3)

You want to edit the last parameter in each function call -- change it to 'hello'. I have two ways of approaching this: macros (interactive) or :norm (less interactive).

1\. Put your cursor on the first byte in the first line, type qq to record a macro of your change, q to finish. Select the rest of the lines in visual line mode and type :'<,'>norm @q to run the macro on each of the remaining lines.

2\. Select the lines in visual line mode and type, for instance, :'<,'>norm %F,ws'hello' This is the "non-interactive" version of the above since the macro keystrokes are given directly to :normal. Pro: ability to undo keystrokes unlike when recording a macro. Con: cannot actually see the effect of the keystrokes (but with practice, this is not a problem).

(Note that colon in visual mode automatically inserts the :'<,'> so you only have to type :norm ...)

I find the declarative nature of regexes/:s to be too restrictive -- I much prefer the operational nature of macros/:normal, since that lets me make the change I want directly, without me having to phrase the change as a regular expression.

I could also use CTRL-A in normal mode to increment the last argument in each line: :'<,'>norm %b^A (where ^A is the literal, typed CTRL-V CTRL-A)

I feel like a wizard whenever I use :normal in a non-trivial manner.

<comment><USER7> care to suggest any resources that helped you reach wizard-like levels of usage? I very much wish to work towards that kind of ability in vi/vim.

<comment><USER5> I am sorry, but I don't know any resources... I have collected a few things on the following page: [http://users-cs.au.dk/rav/vim/](http://users- cs.au.dk/rav/vim/)

Basically, I achieved my wizard-level C++, Vim and Git knowledge during my three years as a part-time student programmer in the basic research center MADALGO. I took the time to study the documentation (respectively the C++11 draft, :help and the git man pages) whenever I was curious. For Vim in particular, I made sure to eliminate all repeated keysmashing in my daily workflow, which "unfortunately" required me to learn about macro wizardry.

My only advice is to keep practicing, to keep trying to spot inefficiencies in your own workflows, and to actively eliminate these inefficiencies by studying the Vim help pages.

</comment></comment></comment></comment></comment><comment><USER8> I'm all for bending your text editor to your will, but a few of these tips gloss over the core of vim: the shortcut _language_ it offers and the composability of its parts of speech [1].

For example, remapping the paste key to _paste_ \+ _move to end_ is cool, but `ppppp` to paste 5 times circumvents vim's killer repetition. Want to paste 100 copies? (hopefully you do this rarely). Just type `100p` and you're done. I won't bother typing out the obvious sequence of repeated `p` presses, but you get the drift.

Also, remapping v to progressively select larger surrounding text objects will keep you from learning that you can perform any action on the 3rd parent curly braces by following the action with the text object `3a{` or `3a}`. Select it: `v3a{`, delete it: `d3a{`, comment it out `gc3a{` ([2]), whatever. If there's an action in vim you can perform it on a text object which can usually start with some count.

I really do mean language, too. When I'm using vim, I feel a lot more like I'm communicating with my computer than trying to figure out how to do what I want to do.

The actions are there: delete, change, select. The direct objects are there: braces, this line, line number N, end of this word, the next character C, this paragraph. The prepositions are there: inside, outside, up to (inclusive, exclusive).

Heck, the sensible defaults feel a whole lot like the "understood" words that you can leave out of sentences. Instead of "navigate to line 30", you can just use the shortcut for "line 30" (30G) and be done.

One of my favorite things about vim is the number of times I've literally been _surprised_ that vim didn't read my mind and go where I was looking. Once you're familiar with the different pieces, it's so good that you might expect it to read your mind.

</almost-blog-post>

[1] [http://ferd.ca/vim-and-composability.html](http://ferd.ca/vim-and- composability.html)

[2] [https://github.com/tpope/vim-commentary](https://github.com/tpope/vim- commentary)

<comment><USER5> > I really do mean language, too. When I'm using vim, I feel a lot more like > I'm communicating with my computer than trying to figure out how to do what > I want to do.

I tell people learning Vim or Emacs that they should treat it like learning a new programming language -- a programming language for text manipulation. (In particular, don't attempt a first project in C++ while writing your first report in LaTeX, both in Vim for the first time -- it will not be a pleasant experience!)

<comment><USER8> On the other hand, if you like crying yourself to sleep, then that's definitely a good idea. :-D

</comment></comment></comment><comment><USER9> This post has some valuable tips (I'm probably going to try out <Space> as a leader), but one or two near the start rub me the wrong way in terms of how I use vim and how it's my impression that it's intended to be used.

>It seems like vvv is slower than vp but in practice I don't need to think beforehand what to select, and what key combination to use.

The way I've always used vim and always thought it was intended to be used is that you do think beforehand. You sit at your editor, think about what changes you want to make, and then key in a set of precision commands in vim-editing- language and it happens.

>This way v replaces viw, vaw, vi", va", vi(, va(, vi[, va[, vi{, va{, vip, vap, vit, vat, ... you get the idea.

I kind of like the precision of having all of those different things, and of course the option of using them for more than just visual select but also change, delete, and so on. Although I suppose this doesn't remove any of those keymappings, I must protest remapping Ctrl+v: I can't even use an editor without block select.

I imagine there's a plugin (or even builtin feature) that at least generalises "s, (s , [s, tags and things of that sort though.

>Stop that stupid window from popping up: >map q: :q

I know it's a weird and irritating thing to have that window pop up when you meant to quit, but it's actually a very neat interface: a whole vim buffer for recomposing commands and your command history for later execution (almost acme-like). Give the poor guy a chance.

As a counterpoint to what I've pointed out above, I'd like to recommend Drew Neil's [ _Practical Vim_ ]([http://www.amazon.com/Practical-Vim-Thought- Pragmatic-Progra...](http://www.amazon.com/Practical-Vim-Thought-Pragmatic- Programmers/dp/1934356980)) to anyone who hasn't read it already. It's got a lot of great content, and really goes a long way to explain vim's quirks and methods of doing things.

One of the useful tips I learnt from that was the ex command "normal", which allows you to execute a string of normal mode commands over a range of lines. So, for example, you can append a semicolon to each line in a visual selection by entering

        :'<,'>%normal A;

A small thing, but one that I've used a lot since learning about it.

<comment><USER10> +100 for Practical Vim... Hands down the best vim guide. After 10 years of kinda-knowing-vim, that book changed how I use vim entirely.

</comment></comment><comment><USER11> You may also like spacemacs. They've got a pretty nice <SPACE> \+ a lot of keys binding system. [https://github.com/syl20bnr/spacemacs](https://github.com/syl20bnr/spacemacs)

</comment><comment><USER12> This was great even just for the

        <Leader>p "+p

mapping. I do this all the time ("+p). It's weird because I tend to go through waves of realizing "Oh! I can just remap that!" I feel like I have fairly good vim skills but somehow always forget I can keep making it _more_ efficient than it already is.

Thanks for the reminder =].

<comment><USER13> I've got vim set up to use the system clipboard automatically. Want to visit a URL in a readme file? yiW, tab over to chrome (with vimium), hit P to open it in a new tab. Want to add a URL to documentation? yy it from chrome, tab to terminal, p it in.

</comment></comment><comment><USER14> It is a long long time ago that someone taught me something really new about vim and used plugins that are not just fancy (looking at most status line plugins or nerdtree) but actually seem to improve productivity in a vim-ish sense. Kudos!

<comment><USER15> python-mode or jshint2.vim? I don't think that they are "just fancy. Bringing linting into the editing window is more helpful (to me personally at least) than having a separate process running with it.

(Note: I turn rope off in python-mode because it's horribly inefficient, and sometimes causes Vim to delay reacting to user input)

</comment></comment><comment><USER16> This is one of the best "how to boost you Vim productivity" articles I have ever seen. It really gets to the heart of the types of customizations that a lot of people don't consider that make a massive difference.

</comment><comment><USER17> With bash, if 'fg' is too long to type you can just type '%'.

<comment><USER18> I use 'fg' a lot. Thanks for the '%' tip. Now to practice and grow some memory muscles!

</comment></comment><comment><USER19> Why don't people seem to use the F keys? I have mappings for very frequent functions on F1-F5, and it really helps:

        map  <F1> <Esc>:w<CR>|      "Fast save
        imap <F1> <Esc>:w<CR>|      "Fast save
        map  <F2> :make<Up><CR>|    "Fast compile
        map  <F3> :bn<CR>
        map  <S-F3> :bp<CR>
        " Same mapping for gnome-terminal ( see     http://stackoverflow.com/q/12813126/164171 )
        map ^[O1;2R :bp<CR>

        map  <F4> :bd<CR>|          "Close buffer

        map  <F5> :cnext<cr>
        map  <S-F5> :cprev<cr>
        imap  <F5> <Esc>:cnext<cr>
        imap  <S-F5> <Esc>:cprev<cr>

<comment><USER8> One of my favorite bits for saving my own butt: commit more.

        "Function to commit.
        function! GitCommit ()
          execute 'write'
          let message = input('Enter commit message: ')
          execute '!git add ' . bufname('%') . ' && git commit -m ''' . message . ''';'
        endfunction

        imap <F7> <Esc>:call GitCommit()<CR>
        map <F7> <Esc>:call GitCommit()<CR>

<comment><USER20> Have you tried fugitive (a vim wrapper around git)?

</comment></comment></comment><comment><USER21> A few of these are good suggestions but I can't live without C-v, whether I am already in visual mode or not. Block selection may not be used that frequently but when it is, it's a lifesaver.

<comment><USER22> I use block selection so often that I've switched the mappings in my vimrc. I have v mapped to visual-block selection and ctrl-v mapped to visual selection. It's very rare that I miss normal visual mode as I use block and line selection more than anything.

</comment></comment><comment><USER22> I have hacked my tabline to be a list of my open buffers. It still functions as a tabline, for those of you who use tabs, but it's kinda nice to see all my open buffers in a nice list across the top of my screen.

[https://github.com/yramagicman/dotfiles/blob/master/.vim/aft...](https://github.com/yramagicman/dotfiles/blob/master/.vim/after/plugin/tabline.vim)

<comment><USER23> See also: bufexplorer, MiniBufExplorer, buftabs

</comment></comment><comment><USER24> I disliked the part about setting up Vim on servers. Do you really want to be adding all the dependencies for your editor on every server? I'm an advocate of using the netrw plugin to edit over ssh directly and avoiding logging into servers when not necessary.

vim scp://host//etc/whatever

<comment><USER25> Plugins aren't installed until you call :PlugInstall

You can also use .nvimrc instead .vimrc, so only nvim instance on server gets plugins.

</comment></comment><comment><USER26> The most productive mapping ever: imap jj <ESC>.

No more having to reach for the escape key.

<comment><USER27> Even better:

        imap jk <ESC>
        imap kj <ESC>

I just hit both and j at the same time with my pointer and index fingers so it's essentially one keystroke.

<comment><USER28> How does this work? What if you are in insert mode and actually want to type j or k?

<comment><USER20> It's the combination jk (or kj) that is set up to switch from insert to normal mode. The idea is that this combination rarely occurs in English text. The drawbacks are a slight delay in cursor movement when typing j or k as vim waits to see what comes next, and momentary confusion when you inevitably need to actually insert these sequences.

</comment></comment></comment></comment><comment><USER29> This is great. One thing the author should do is remap enter to something better than "jump to end of file." I remapped mine to ":w<CR>" to save the current file, and it's stuck.

</comment><comment><USER30> Here is a way to do the Ctrl-Z trick in Bash:

[https://gist.github.com/mastensg/6d9030e25699c2eed5c4](https://gist.github.com/mastensg/6d9030e25699c2eed5c4)

<comment><USER31> Thanks! I am looking for this

</comment></comment><comment><USER22> Does anyone have any advice on bindings to change split size? Ctrl-w + and Ctrl-w - just don't work well, and my current configuration shadows Ctrl-a, which I really would like to avoid.

<comment><USER5> I rarely resize windows, but when I do, I set mouse=a and use the mouse to drag the window borders around. There's also CTRL-W _ (underscore) to maximize the current window and CTRL-W = to make window sizes equal.

<comment><USER22> Also, did you know that Ctrl-w | maximizes the a window horizontally? So if you have splits open above and beside you ctlr-w _ ctrl-w | will maximize your current split.

 _immediately maps this to ctrl-w m_

<comment><USER32> yes i did, having read the manual. :) there's also `:on[ly]` (i have `set hidden` in my .vimrc)

</comment></comment></comment></comment><comment><USER33> The zsh Ctrl+Z binding is brilliant!

The Space and Ctrl+Space thing is interesting, but not interesting enough for me to switch (from , and Ctrl+q – with Ctrl being on Caps Lock).

</comment><comment><USER34> Learning that makeprg can be anything (even a script) made a huge difference to me. That and errfmt means I dont need to leave vim to compile or lint

</comment><comment><USER35> Very useful, thank you. I'm spending more and more time using Vim through ssh and trying to re-learn as much as I can.

</comment><comment><USER36> Lately I use neovim, because it allows using 24 bit color themes in the terminal.

<comment><USER16> So neovim is usable now? Do you run into many problems with it?

<comment><USER37> I have been using it for months without any issues. It is actually a bit faster than regular vim.</comment></comment></comment>


Node.js 8.10 runtime now available in AWS Lambda - <AUTHOR> https://aws.amazon.com/blogs/compute/node-js-8-10-runtime-now-available-in-aws-lambda/ <comment><USER2> Was waiting for this update! Running Puppeteer related projects will now be a ton easier for ad hoc project.

I'm pretty deep into this with [https://checklyhq.com](https://checklyhq.com) so not really an "ad hoc" user. Some open concerns for "real" production use are still:

\- cost: memory heavy (1,5Gb) lambda get costly quickly

\- stability: No reason to assume this is bad, but running a Chromium browser in Docker is/was pretty tricky

\- startuptime: Booting Chrome might be a time suck

Coupling this with SQS triggers for Lambda might make spinning up VM's obsolete for my business.

</comment><comment><USER3> This is massive for AWS Lambda (and we've waited a long time for this). We can now use async/await without transpiling our code. Thereby making it more readable.

We should hopefully see improved performance and startup times.

<comment><USER4> async/await is still slower in 8.10 than promises though, isn't it?

Also, I'd be interested to find out why this is so massive? We've been waiting a long time for this, and I get that Lambda is complex with lambda@edge and all that, but how much tweaking does AWS do to the node runtime before they can deploy this?

Node 8 has been out for 10 months, and it's been LTS for half a year - or did they simply wait for it to be stable enough to run, or anything specific in 8.10?

<comment><USER5> In a world where cold starts have a substantial impact on performance, the runtime cost of async/await is negligible. If your function takes even a second longer to execute because of await use, I'd be incredibly surprised.

</comment></comment></comment><comment><USER6> It was said back in a blog post for the introduction of aws batch, but when will we be able to also use lambda functions in a batch environment? (The job isn't too complex, it just needs to run longer than 5 minutes (an hour or two))

</comment><comment><USER7> Has anyone already managed to get this working? I currently get:

        expected runtime to be one of [nodejs4.3 nodejs6.10 java8 python2.7 python3.6 dotnetcore1.0 dotnetcore2.0 nodejs4.3-edge go1.x], got nodejs8.10

It might be due to my using TerraForm and that needing an update, but it's also not yet listed in the docs: [https://docs.aws.amazon.com/lambda/latest/dg/API_CreateFunct...](https://docs.aws.amazon.com/lambda/latest/dg/API_CreateFunction.html#SSS- CreateFunction-request-Runtime)

Or perhaps it's not available in all regions yet?

<comment><USER8> The AWS provider for Terraform doesn't support the new runtime yet, yeah.

See:

[https://github.com/terraform-providers/terraform-provider- aw...](https://github.com/terraform-providers/terraform-provider- aws/issues/4019) (issue) [https://github.com/terraform-providers/terraform- provider-aw...](https://github.com/terraform-providers/terraform-provider- aws/pull/4020) (PR)

<comment><USER7> Ah, thanks - I didn't know where to look.

</comment></comment></comment><comment><USER9> Perhaps this will make headless chromium easier

[https://www.npmjs.com/package/@browserless/aws-lambda- chrome](https://www.npmjs.com/package/@browserless/aws-lambda-chrome)

</comment><comment><USER10> This update was a very long time coming, so I'm grateful that it finally happened.

Still, it's disappointing that they couldn't change course (ever so slightly!) to accommodate the 8.11.0 _security update_ from a week ago: [https://nodejs.org/en/blog/release/v8.11.0/](https://nodejs.org/en/blog/release/v8.11.0/)

Really hope they'll be keeping up-to-date with incremental Node 8 releases, now that they've (almost) caught up with the times.

<comment><USER10> Wow, I commented about this on the blog post, and they apparently deleted my comment. Cool.

<comment><USER11> We review and then approve all on-topic comments, but are working through a bit of a backlog this morning from what I understand.

</comment></comment></comment><comment><USER12> Have been waiting on this for a very long time. Hopefully AWS will keep up-to- date with LTS versions more quickly in the future. :-)

</comment><comment><USER13> Looks like this isn't available for use with Lambda@Edge/CloudFront. Very disappointing :\

</comment><comment><USER14> New LTS versions of node have been released in September/October for the last two years and AWS Lambda has added support for them around April/May for the last two years.

I wonder if they purposefully wait for x.10 update (6.10 and 8.10) or it has something to do with internal management of projects.</comment>


Flake: A Decentralized, K-Ordered Unique ID Generator in Erlang - <AUTHOR> http://blog.boundary.com/2012/01/12/Flake-A-decentralized-k-ordered-id-generation-service-in-Erlang.html <comment><USER2> A couple of questions about this:

\- Why [time, node id, seq] and not [time, seq, node id]? That would improve ordering if you have approximately equal load on each box.

\- Isn't a 16 bit seq number overkill? Handing some of those bits to the unique ID would have made unique ID assignment easier. Duplicate MACs can and do exist (especially if you buy a lot of hardware from the same vendors).

\- The quality of the ordering is going to be restricted by the quality of time synchronization within the cluster. Relying on NTP for this is OK, but experience suggests that a secondary monitoring system will be needed. Similarly, relying on monotonic time needs some care in system administration - care that could potentially be avoided with a different unique host ID assignment scheme.

<comment><USER3> > Why [time, node id, seq] and not [time, seq, node id]

Would you rely on sub-millisecond synchronization between nodes _and_ an almost exact load amount?

In other words for a particular millisecond seq order is only relevant on that particular node, so node should come first.

<comment><USER2> > In other words for a particular millisecond seq order is only relevant on > that particular node, so node should come first.

It still seems to me that T_0,0,A (issued by node A at T_0 with seq number 0) would tend to come before T_0,1,B so putting the sequence number first does add some value. On the other hand, the ordering T_0,A,0 < T_0,A,1 < T_0,B,0 < T_0,B,1 is arbitrary.

> Would you rely on sub-millisecond synchronization between nodes _and_ an > almost exact load amount?

Not rely on. Absolutely not. Maybe it's better to go further, and make it more explicit that the IDs are K ordered. Say you could synchronize your host clocks reliably to a maximum delta of 512 ms, then you could chose a scheme like:

[ milliseconds >> 9, host id, milliseconds & 0x1ff, seq id ]

The value here is that you make it much harder for consumer of the IDs to make incorrect assumptions about the precision of their ordering. Basically taking away the temptation to make statements about their ordering with false precision, by making a simple sort only provide a meaningful ordering within the real available precision.

<comment><USER3> > On the other hand, the ordering T_0,A,0 < T_0,A,1 < T_0,B,0 < T_0,B,1 is > arbitrary.

Still don't see it. We are already making the assumption that the millisecond part is the same T_0. So then seq will depend strictly on the count of previous such IDs issued by that node only (would you agree with that?). So that is why I said that it doesn't make sense to compare seq number from A -- (0) and a seq number of B -- (1) before considering the identity of A & B. You would effectively order by relative load between all your machines at that particular time, at that time frame it would be things like disk access, cache states and so on.

EDIT:

In addition. Ordering T_0,A,0 < T_0,A,1 < T_0,B,0 < T_0,B,1 will be stable though, machine A,B mac addresses don't change. For any particular millisecond you will get all the results for a particular machine then those results will be sorted by sequence # so you would even get some kind of a relative load measurement.

Let's look at a longer example:

T_0,A,0

T_0,A,1

T_0,B,0

T_0,B,1

T_0,B,2

T_0,B,3

T_0,C,1

T_0,C,2

T_0,C,3

T_0,D,0

I think is a lot better order than

T_0,0,A

T_0,0,B

T_0,0,C

T_0,0,D

T_0,1,A

T_0,1,B

T_0,1,C

T_0,2,B,

T_0,2,C

T_0,3,B

T_0,3,C

</comment></comment></comment></comment><comment><USER4> There's also Instagram's implementation in PostgreSQL:

[http://instagram- engineering.tumblr.com/post/10853187575/sha...](http://instagram- engineering.tumblr.com/post/10853187575/sharding-ids-at-instagram)

</comment><comment><AUTHOR> Thanks for the interest and the feedback. I've updated the readme and added a roadmap as well as a faq after getting some good input over the last couple of days.

<https://github.com/boundary/flake>

</comment><comment> google-1 This is quite similar to the BSON ObjectId specification used by MongoDB:

[http://www.mongodb.org/display/DOCS/Object+IDs#ObjectIDs- BSO...](http://www.mongodb.org/display/DOCS/Object+IDs#ObjectIDs- BSONObjectIDSpecification)

</comment><comment><USER5> For those interested, there is also Snowflake by Twitter on GitHub <https://github.com/twitter/snowflake>

<comment><USER6> Snowflake is referenced in the article "Credits" with a distinction between the products -- though, the product name is a bit dubious. _It [Flake] differs primarily in that we allow ourselves a wider address space in which to fit ids which means we don't have to think about timestamp truncation or coordination of worker ids._

<comment><USER7> Yeah. But the hard part is the 64bit representation. If you have 128 bits, you can no longer be a standard int. And you can just use UUIDs if you don't care about k-ordering.

</comment></comment></comment><comment><USER3> Why not use uuid1 then reorder its bits ?

</comment><comment><USER8> Is it Erlang day already?

</comment><comment><USER9> as mjb said, mac addresss can be cloned so not sure whats that buying we are using uuid1 which does more or like does what these guys are doing...

<comment><USER10> UUID1s don't sort by their generation time.</comment></comment>


After throwing Microsoft under the bus, Google won't patch flaw - <AUTHOR> http://www.neowin.net/news/after-throwing-microsoft-under-the-bus-google-wont-patch-flaw-affecting-nearly-1bn-users <comment><USER2> Except for the fact the flaw was patched by Google. It's the OEM's that aren't putting out the patched version.</comment>


The Rise and Fall of the Blog (2017) - <AUTHOR> https://daily.jstor.org/the-rise-and-fall-of-the-blog/ <comment><USER2> It still feels like social media, facebook especially, took away a lot of blog traffic. At the same time, social media took away a lot of traffic from traditional forums. Add the reality that not everyone was making as much money as they thought, prolific spammers, blog content thieves, and the continued rise of mobile web traffic, and you have a lot of abandoned blogs.

Blogging takes creativity and a lot of time, more so than many people realized. I tried to do it when I was under-employed. Getting traffic was very difficult, and ad revenue was nowhere to be found.</comment>


"Air Carbon" Planter Made with Direct Air Capture Technology - <AUTHOR> https://air-miners.myshopify.com/collections/frontpage/products/air-miners-planter <comment><AUTHOR> Excited to launch this! We worked with Carbon Engineering to get their carbon captured using direct air capture. We believe it's the first consumer product to use direct air capture.</comment>


Doe Device – A Multipurpose Kitchen Device - <AUTHOR> https://doedevice.com <comment><AUTHOR> HI guys, this is Rish, a mechanical engineer from New York. I am looking for your suggestions over the product. Also in search of an advisor in the same space. Thanks</comment>


Twitter or Tricker? WHy? - <AUTHOR>

<comment><USER2> u mean like... a business?

Trying to provide a service that is of value?

</comment><comment><AUTHOR> Is Twitter a service that adds value for you or just a trick to get you hooked so they can make money of you later?</comment>


Yahoo CEO + EBAY + FAKE Degree = totally awesome - <AUTHOR> http://nexthotstartup.com/2012/05/04/yahoo-ceo-computer-science-degree-from-ebay-image/

<comment><USER2> Education does not matter! talent is important to run a company.</comment>


Advice from me (now) to me (then) - <AUTHOR> http://nikibeth.com/2009/08/11/advice-from-me-now-to-me-then/

<comment><USER2> Would've been helped by concrete examples. It seemed a little hand-wavey.</comment>


How Extra.ai sends onboarding email - <AUTHOR> http://www.kevanlee.com/extra-onboarding-email/ <comment><AUTHOR> Although I wouldn't normally post marketing-related stuff on HN, I think this one should be quite useful to a large audience here. Liam really figured out how to write engaging and truthful copy, which makes you want to keep reading, and doesn't feel like a typical marketing newsletter. Wish more companies could use this style (genuinely).</comment>


Court Bans Whatsapp, Facebook and Instagram in Germany - <AUTHOR> https://translate.googleusercontent.com/translate_c?depth=1&nv=1&rurl=translate.google.com&sl=de&sp=nmt4&tl=en&u=https://www.sueddeutsche.de/digital/urteil-facebook-whatsapp-instagram-verbot-1.4713341&xid=25657,15700021,15700043,15700186,15700190,15700256,15700259,15700262,15700265,15700271,15700283&usg=ALkJrhjbeqYH0bbXCKD81_0n23SPpPJwnA <comment><USER2> From what I can tell (the translation isn't perfect), one of the patents BlackBerry sued them over is being able to "email entire chat history". Also, "switching from one chat to another" and "friend proposals".

Lmao. This stuff is ridiculous. I'm so glad my country (India) doesn't have software patents.

Edit: so I looked it up, and the software patent situation is kinda weird here:

> Software Patents in India is granted for an embedded software in a mobile > application, and/or software plus hardware combination. However, patent law > in India does not allow patent protection for software per se, whereby > patenting a computer program is prohibited.

<comment><USER3> We also don't have software patents per se in Europe, but that doesn't keep patent offices granting bullshit patents like this one

<comment><USER4> But in this case it's not just the patent office granting a patent, but actually winning a case in court as well. How does that work when supposedly software patents are not applicable?

<comment><USER5> [https://fsfe.org/campaigns/swpat/swpat.en.html](https://fsfe.org/campaigns/swpat/swpat.en.html)

It's complicated, The European Patent Convention states that software is not patentable. But laws are always interpreted by courts, and in this case interpretations of the law differ.

<comment><USER6> I'm not really sure how it works but can't Facebook take it to the EU court and win this case?

</comment></comment></comment></comment></comment><comment> zupa-hu Clickbait title. They are not banned. Blackberry won a patent case but there is no reason the apps can't be changed.

(Current HN thread title: "Court Bans Whatsapp, Facebook and Instagram in Germany")

<comment><AUTHOR> The title indeed sounds quite clickbaity.

I chose to use the exact translation of the original article's title as the title of the HN thread since "Süddeutsche Zeitung" usually is quite a reliable and transparent resource.

Original source without translation: [https://www.sueddeutsche.de/digital/urteil-facebook- whatsapp...](https://www.sueddeutsche.de/digital/urteil-facebook-whatsapp- instagram-verbot-1.4713341)

</comment></comment><comment><USER7> Fuck software patents.

</comment><comment><USER8> Anyone knows wat's the state of software patent laws at the european parliement ?

I remember they tried to create some laws that would be enforceable in all countries of the union, but i'm not clear on what's the situation now.

I'm actually pretty surprise software patent exist in germany.. i was pretty sure europe was software-patente free..

</comment><comment><USER9> The DeepL translation is a bit easier to read: [https://pastebin.com/raw/8PngEFcs](https://pastebin.com/raw/8PngEFcs)

</comment><comment><USER10> Interesting to see that Blackberry still has enough money to dare something this ridiculous. I thought they were completely irrelevant by now.

</comment><comment><USER11> How would this be enforced?

Is the Play Store going to remove those apps from the devices that are located in Germany?

<comment><USER12> Under this ruling, Facebook may no longer offer its apps in their current form in the German version of the Google Play store.

As the article remarks, the case doesn't involve core functionality of any of the apps, and FB claims to have compliant versions of their apps ready in case RIM seeks to have the provisional ruling enforced. (They can, but only if they pay 1.6 mil caution in case they lose in FB's appeal).

So, FB would only have to push an update for each app. Not doing so would clearly fall afoul of this ruling.

</comment></comment><comment><USER6> Software patents shouldn't even be a thing. Actually there shouldn't be any patents at all.

They only hinder development. If someone comes up with a solution/product/whatever that is better than the original, they should be able to create that.

If this harms the original creator they'll be forced to keep up and make their own thing better again.

Competition is always good. It makes people strive to be better.</comment>


Is Boeing's 737 an Airplane Prone to Problems? - <AUTHOR> http://www.thedailybeast.com/newsweek/2012/03/19/is-boeing-s-737-an-airplane-prone-to-problems.html

<comment><USER2> You don't say... The 737 has and always will suck. I can't believe they discontinued the awesome 757 and kept the 737.

<comment><AUTHOR> Were these models in competition somehow?</comment></comment>


Famous quotes, the way a woman would have to say them during a meeting - <AUTHOR> https://www.washingtonpost.com/blogs/compost/wp/2015/10/13/jennifer-lawrence-has-a-point-famous-quotes-the-way-a-woman-would-have-to-say-them-during-a-meeting/ <comment><USER2> I don't see the male stereotype as being all that great. In practice you have men declaring as fact things that are easily debatable. The inevitable response comes back as a challenge, an argument of semantics ensues, and at the end of the day you reach a mutual détente where nobody has lost face, but nobody is wiser for the argument either. The original point obliterated by the posture.

Whereas if the discussion had been framed as "Listen, I have a very strong opinion on this one but I respect that you may disagree, and I want to understand why," then presumably you reach an understanding and possible both parties emerge the wiser.

</comment><comment> r-w I've never seen this gender stereotype in action, but then again I'm not in the workplace yet. However, I do see that some of my classmates, including me and other guys, put their thoughts this way in order to avoid sounding too sure of themselves if they end up being wrong or to avoid inciting a debate. Maybe this signifies a parallel between student–teacher and woman–man relationships.</comment>


T-mobile asked me for my PIN over the phone - <AUTHOR> I wanted to change plans and have it activated the same day, which required that I call them directly.  They said that they could only "open" my account with the PIN and I had to verbally tell them my PIN.  I thought this was a security issue and told them that.  Their response was, "sorry, no can do."  Since I didn't know what my PIN number was, they issued me a temp number, and required that I give them a new PIN number over the phone, right after they issued me a temp PIN.  Isn't this considered a bad practice? I'm not a security expert, but have always been told not to give out passwords/PIN over the phone to anyone. <comment><USER2> This is standard practice, asking for your PIN. If you used a temporary PIN via text response, that should be enough for that task. The PIN protects you from somebody acquiring your phone and then taking over your account.

If you're in North America, call before 7pm PT to improve your odds of getting to their Oregon support people. They are _excellent_ in my experience, and I've been an Advanced Support engineer.

If you land on their overseas support (Philippines), just hang up unless it's a very basic question for which a wrong answer or botched change won't damage you. I have been repeatedly lied to, had critical account settings screwed up that took hours of rework, and disconnected in the midst of hour-long calls. They are literally worse than useless. I've had handset manufacturer support personnel confirm similar experiences.

The contrast is _astounding._</comment>


Do any of you use Salesforce.com regularly? - <AUTHOR>

SF was created to be a less sucky version of SAP.  Their motto is "The end of software."
I've been playing with it recently and I have to say, it's absolutely horrible.  The most complicated and unfriendly piece of software I have ever had the displeasure of using.
Do your sales people actually like this thing?
Are you developing applications for their platform? <comment><USER2> Yes, sales people love it because they hear from other salespeople 'Oh you have to get on Salesforce dot com'.

I haven't developed anything using their API.

I have used it a bit, and it is much better than it was ~5 years ago, but still is pretty much horrible.

</comment><comment><USER3> We use it, we like it, we don't customise it much and use it more as a tool for maintaining contact and pipeline management than a full on CRM, which is probably why it works well for us!</comment>


Amazon interviews are using programming tests that watch you - crooked-v This is something that I've seen come up on HN before [1], and something that people working at Amazon have said on here will be changed. From personal experience, I can say that it hasn't. My contact with an internal recruiter for a new AWS team led to an assessment test run by PAN [2], requiring use of a webcam.<p>I haven't gone through with it yet, and haven't decided if I will. That recruiter didn't even try to offer an alternative when I expressed my discomfort at the idea, which doesn't seem to bode well for any work experience that would follow it.<p>[1]: https://news.ycombinator.com/item?id=13130865 [2]: https://www.panpowered.com <comment><AUTHOR> I work in assessment industry and taking candidate snapshots or recording their actions during the assessment, with the candidate's permission, is a very common practice. Platforms like HackerRank, HackerEarth, Codility, etc, all do that.

The problem is, companies were initially comfortable with no such feature to record candidate actions (snapshots/videos), candidates however started cheating in dozens of ways and eventually get rejected when they are shortlisted for the first F2F interview after initial screening. Some candidates can't even explain how they scored well in screening test.

Due to that, companies became insecure and are now asking for features for proctoring the assessment. Things are never one-sided.

<comment> crooked-v There are plenty of ways to verify for identity and skill level that don't come with the automatic implication of dishonesty. For example, even exactly the same test could be done using a testing center... such as those physical testing locations administered by PAN, one of which I visited once upon a time as part of a job application in a different field.

</comment></comment><comment><USER2> Dang, I was thinking the test "watched" you in terms of typing speed to infer thought process or something.</comment>


Ask PG: Applying second time, why is the application url up after deadline - <AUTHOR>

Just a question I was wondering both the times. Is it something technical or are you being generous :) <comment><USER2> This should be changed to "Ask SA:"

<comment><AUTHOR> wouldn't SA just do what the big guys tell him to?</comment></comment>


J - A modern, high-level, general-purpose, high-performance programming language - <AUTHOR> http://www.jsoftware.com/index.html

<comment><USER2> So my first question was "Why?", Then after some digging "Why not just use R?". The best answer I found to that question was on stack overflow: [http://stackoverflow.com/questions/1427276/j-programming- lan...](http://stackoverflow.com/questions/1427276/j-programming-language-vs- r-programming-language-vs-incanter) .

Which for me said - "Just use R".

</comment><comment><USER3>
        * an integrated development environment
        * standard libraries, utilities, and packages
        * a form designer for your application forms
        * an event-driven graphical user interface to your application
        * interfaces with other programming languages and applications
        * integrated 2d and 3d graphics
        * memory mapped files for high performance data applications

This list doesn't strike me as particularly "modern" anymore. I know firsthand that there are environments from 10 years ago with lists like this, and the stuff wasn't new and hip even back then!

<comment><USER4> I don't have to tell a Smalltalker that not-new-and-hip is a poor metric to go by. The serious anachronism here is the lack of open-source. It's hard to see how any LFSP can get off the ground that way any more.

<comment><USER3> "not-new-and-hip" is always a poor metric for an environment. But it's relevant for PR that uses adjectives like "modern." (And this includes quite a bit of Smalltalk PR as well!)

<comment><USER4> Oh, I see. You were responding to the M-word.

</comment></comment></comment></comment><comment><USER5> See also: "The World's Most Mind-Bending Language Has the Best Development Environment" (<http://prog21.dadgum.com/48.html>)

</comment><comment><USER6> I wish they would call it a notation rather than a language. Language implies that you can read it like a sentence. We call symbols in math "notation" not language for a reason.

</comment><comment><USER7> Appears to be closed source.

Edit: stricly speaking it seems you can get a look at the source if you pay: "The price range is from $10,000 to $400,000. Regular updates to our current source levels are available for separate update fees."

However, what I mean is there won't be a lively community of open source developers working on J. If the company crashes with that holiday jet, nobody will go on working on J. To me it is effectively the same as closed source.

</comment><comment><USER8> I wish J were used as an embedded language for solving mathematical problems within a larger, simpler to read language for input/output etc. Like the way we embed SQL within Python, or whatever.

</comment><comment><USER9> I love using J when I'm solving Project Euler problems, but other than that, the language just isn't practical.

<comment><USER6> Yes it is. It solves problem doesn't it? Maybe those problems don't frequently come up, but I'll be damned if it isn't practical for a niche set of problems.

You may want to change your definition of "practical" so that's it broader than business applications.</comment></comment>


Brainpool curves - grey-area http://bada55.cr.yp.to/brainpool.html <comment><AUTHOR> The link just points at the Brainpool curves, but the paper describing the 'new curves' etc is lots of fun. ([http://bada55.cr.yp.to/pubs.html](http://bada55.cr.yp.to/pubs.html), _How to manipulate curve standards: a white paper for the black hat._ ) It takes a swipe at the wider issue of how a procedure for picking constants like Brainpool's can be gamed, by using a template procedure with lots of 'justifiable' choices - the hash function, the seed, etc, but only presenting the one that leads to your 'known-weak' curve.

 _Our experience is that Alice and Bob, when faced with a single procedure such as [BADA55-VPR 's] (or [Brainpool's]), find it extremely difficult to envision the entire space of possible procedures (they typically see just a few dimensions of flexibility), and find it inconceivable that the space could have size as large as 2^20, never mind 2^30._ (s5.3)

I find it pretty hilarious that someone found a way to weaponize the old quote "The wonderful thing about standards is that there are so many of them to choose from"

</comment><comment><USER2> If anyone wonders about the potential practical impact of the subversion of the Brainpool curves, it's pretty bad, as the curves have been standardized for use in TLS by RFC 7027.

Any other standard using Brainpool curves?

<comment><USER3> But nobody uses the Brainpool curves for TLS. Everyone uses NIST P-256 and P-384, and everyone will move to Curve25519 "real soon now".

key exchange:

[https://tools.ietf.org/html/draft-irtf-cfrg- curves-09](https://tools.ietf.org/html/draft-irtf-cfrg-curves-09)

signatures:

[https://www.ietf.org/mail- archive/web/cfrg/current/msg07364....](https://www.ietf.org/mail- archive/web/cfrg/current/msg07364.html)

<comment><USER2> Someone actively avoiding the NIST curves might use them, but it looks more and more like chosing between Scylla and Charybdis.

<comment><USER3> Looks like Brainpool curves are not supported by NSS, SChannel or SecureTransport, so they're useless for TLS when the client is a browser. If the client is something that uses openssl, you can use them.

[https://en.wikipedia.org/wiki/Comparison_of_TLS_implementati...](https://en.wikipedia.org/wiki/Comparison_of_TLS_implementations#Supported_elliptic_curves)

</comment></comment></comment></comment><comment><USER4> So in the original specification, if B failed the non-square test, it would start over with the next seed and regenerate both A and B.

In the RFC version, it will regenerate B until it is square, then do the other security checks.

That's unfortunate, but the generated curve should still satisfy all the requirements. There is little room for this to have an actual security impact. You would need an unknown attack that the original specification did not guard against, but randomly happened to avoid by its choice of seeding for the RFC method to be worse.</comment>


Hacker releases new batch of climatology e-mails just before climate conference - <AUTHOR> http://arstechnica.com/science/news/2011/11/hacker-releases-new-batch-of-climatology-e-mails-just-before-climate-conference.ars <comment><USER2> Sigh. All this climate doubting stuff is ridiculous</comment>


Jupiter flyby video – Juno Perijove 06 - <AUTHOR> https://www.youtube.com/watch?v=3kQbTBt418o <comment><AUTHOR> From [https://www.dpreview.com/news/6846393455/mathematician- turns...](https://www.dpreview.com/news/6846393455/mathematician-turns-juno- images-into-stunning-jupiter-flyby-video)

When NASA released the latest batch of images, last month, German mathematician Gerald Eichstaedt got to work, turning them into into a video....

London-based filmmaker Seán Doran saw the video when Eichstaedt uploaded it to unmannedspaceflight.com and spent another 12 hours smoothing the thousands of frames, before adding a soundtrack.</comment>


Milestone: Tesla makes 5000 Model 3s in a week - <AUTHOR> https://news.alphastreet.com/milestone-tesla-makes-5000-model-3s-in-a-week/ <comment><USER2> It would be fair if they would provide accurate weekly production numbers for the last 6 consecutive weeks. Maybe the pattern was: 3000, 2000, 1000, 1000, 3000, 5000... Which is quite different than pattern: 2500, 3000, 3500, 4000, 4500, 5000...

<comment><USER3> [https://www.bloomberg.com/graphics/2018-tesla- tracker/](https://www.bloomberg.com/graphics/2018-tesla-tracker/)

</comment></comment><comment><USER4> previously: [https://news.ycombinator.com/item?id=17436604](https://news.ycombinator.com/item?id=17436604)

</comment><comment><USER5> Tesla also expects to produce 6,000 Model 3 sedans a week "next month." - Sources

</comment><comment><USER6> What ever happened to.the $35,000 price? Are these only being offered with premium upgrades, or are they not able to produce them at a lower, more attainable price?</comment>


The Riemann Hypothesis (Part 2) - <AUTHOR> https://golem.ph.utexas.edu/category/2019/09/the_riemann_hypothesis_part_2.html <comment><AUTHOR> You can head over to cocalc.com, fire up a sage worksheet, and stick this in:

        for n in range(1, 13):
            field = GF(2^n)
            count = 0
            for x in field:
              for y in field:
                if y^2+y == x^3+x:
                    count += 1
            print n, 2^n, count

if you want to play around with some of these numbers.

</comment><comment><USER2> Discussion of Part 1:

[https://news.ycombinator.com/item?id=20912380](https://news.ycombinator.com/item?id=20912380)

</comment><comment><USER3> I love reading John Baez, he makes me feel like I can almost understand this stuff.

</comment><comment><USER4> If you're interested in a lengthier intro to the Riemann Hypothesis, I highly recommend the book "The Millenium Problems" by Keith Devlin.

<comment><USER5> _Prime Obsession_ by John Derbyshire is another in-depth look. Part biography, part history, part math.

<comment><USER6> Excellent read. I also liked his history of algebra: _Unknown Quantity_ ([https://www.amazon.com/Quantity-Real-Imaginary-History- Algeb...](https://www.amazon.com/Quantity-Real-Imaginary-History- Algebra/dp/0452288533)).</comment></comment></comment>


Fuck trends, long live decisions - <AUTHOR> http://restreitinho.com/2011/12/26/predictions-decisions-and-settling/

<comment><USER2> His list of decisions is too vague! They need concrete instantiations, things he can actually _do_ , or they're going to be just nice-sounding generalities to ponder. (I've been burned by this personally.)

I'll give an example. Suppose a college student decides "I will take charge of my education and be motivated to learn." This is a nice idea, but it's easy to lose sight of such high ideals when your daily routine is so immediate and distracting. Something more concrete might be, "I will read at least one interesting journal article each day for the next month, marking each day on the calendar." With this sort of decision, you will know if you are doing it or not, and it can become a routine.

Don't decide to get better at writing; decide what you're going to write, then write it while deliberately trying to write well. Don't decide to "become relevant or GTFO"; come up with some plausible ways to become relevant, try some non-null subset of them, and see how it goes.

</comment><comment><USER3> Making decisions is active.

Spotting trends is passive.

They complement each other, like a feedback loop.

</comment><comment><USER4> This guy doesn't know the difference between tracking trends and "predictions". Trends are based on decisions already made.</comment>


How you execute an idea is everything - <AUTHOR> http://pebblecode.com/blog/how-you-execute-an-idea-is-everything/ <comment><USER2> Netflix only has 36% of the market down from 56%. Google's market share is only at about 63%-64% according to comscore. Definitely impressive but not "winner take all" And Lyft has like a 40% market share in San Francisco. Me personally, I'd stay out of the bloodbath called ride sharing until the companies have to answer to profitability and actually market realities.

</comment><comment><USER2> I agree that ideas aren't as important but execution is a worthless word. It's basically a tautology for success. You can't predict if a company is "going to execute" or be successful. All you can say is "they were successful" or "they did execute well" Which is they same thing.</comment>


Ask HN: Navigating US Work Authorization Questions as a Canadian; TN vs. H1-B - <AUTHOR> This might be somewhat obvious but I haven't been able to find a lot of information online: almost all companies ask in the application process about work authorization, and as a Canadian citizen with relevant experience and an engineering degree, I'm eligible for TN authorization.
A TN, if I understand correctly, is not nearly as much of a hassle as an H1-B visa. I can only assume then that when companies are asking about work authorization, they are trying to determine whether there will be added hassle to hiring a non-resident alien candidate (if they aren't currently authorized to work in the states).
If this is the case, then what should my response to the question about authorization be? Technically, I would still require a TN, which I do not currently possess, but if the company filters applications based off responses to that first question, then should I perhaps answer in the affirmative and then proceed to explain that I would only require TN authorization as opposed to a visa sponsorship?
Do all companies have the ability to hire eligible candidates through the TN process, and do they distinguish it separately from the H1-B sponsorship process? I'm mostly curious about smaller companies and startups since most large companies have an immigration department to deal with international candidates.
I've worked in the states before on both J1 visas and a TN, so I'm familiar with the process as an applicant, just not with the companies' perception of it on their end. <comment><USER2> I've worked at a few companies in the US on various visas. If they ask if you are legally allowed to work in the US I'd generally suggest that you say no but also mention that you would qualify for a TN visa. Because a TN visa is cheap, quick, and relatively straightforward to get I've never personally experienced or heard of a company rejecting a Canadian applicant for this.</comment>


Readings in Database Systems, 5th Edition (2015) - <AUTHOR> http://redbook.io <comment><USER4> While this is a seminal work, I definitely wouldn't approach this like I would approach a textbook: it's definitely not meant to be friendly introductory material. That said, once you have a bit of background, it's a goldmine of a survey. You'll notice that each section is a short, few-page long introduction, but the bulk of the material is in the papers themselves, which can be significantly tougher to read. Though it's great that the summaries are friendly and help you contextualize the papers. My tip is to read papers starting with the introduction, and then the conclusion, and then decide if you want to dive into the rest of the paper to track down the evidence for specific claims.

<comment><USER5> It's organized as the most likely papers that you'd read in a graduate level database seminar. _Readings in Computer Architecture_ is the same.

<comment><USER6> Hi Chris, I searched far and wide for a PDF / soft copy of Readings in Computer Architecture. I searched for the same thing 6 months ago, but to no avail. Would you kindly provide a PDF or a soft copy? Sincerely,

<comment><USER7> How about your local library?

<comment><USER6> I live in Pune right now. Can't think of any such library. Libraries here aren't equipped well with such technical subjects.

</comment></comment></comment></comment></comment><comment><AUTHOR> Yesterday, [https://news.ycombinator.com/item?id=15428526](https://news.ycombinator.com/item?id=15428526) hit number 1 on HN. Having read the two books, I strongly believe that they not only complement each other, but also must be required reading for any data engineer.

</comment><comment><USER8> Does it have something along the lines of building your own RDBMS from scratch? (If not, any recommendations?)

edit: google search has potentially promising results, [https://www.google.com/search?q=build+your+own+rdbms](https://www.google.com/search?q=build+your+own+rdbms)

<comment><USER4> This book is more about different types of tradeoffs you can make in terms of your system design. I'd recommend looking at grad databases courses instead, e.g:

\- [http://db.cs.cmu.edu/courses/](http://db.cs.cmu.edu/courses/) \- [http://daslab.seas.harvard.edu/classes/cs165/](http://daslab.seas.harvard.edu/classes/cs165/) \- [http://daslab.seas.harvard.edu/classes/cs265/](http://daslab.seas.harvard.edu/classes/cs265/)

<comment><USER9> I really wish this class (or the Harvard class referenced below) were offered as a MOOC w/ some certification. I rarely find classes around OS/databases offered as MOOCs, which is a pity because those are the things I'd love to spend time on.

<comment><USER4> Part of the problem is that doing a decent MOOC takes a TON of preparation and effort, much more so than a regular class. The professor who runs the class (Stratos Idreos) has a billion things that he's working on, so turning it unto a MOOC would require some outside support, probably. That said, releasing the videos might be a possibility, I'll ask and see. I believe Andy Pavlo's class has videos online already.

The other part is that in the Harvard classes specifically, the class discussion is a huge part of the class.

</comment></comment></comment></comment><comment><USER10> Everyone I have met who have worked a long time in the database industry considers stonebreaker to be

\- Overrated \- Overly Self Promoting \- Mostly not credible

That being said I love his work and his history. The red book is super famous. What gives?

<comment><USER11> Not credible? Stonebraker is a Turing Award winning MIT professor who's worked in the database industry for ~50 years.

Ask your friends what gives

<comment><USER12> Stonebraker has done a lot of fine work, but also has a lot of false negatives; he got down on recursive queries, on data-parallel compute, and generally most other things that he's not directly working on. It ends up with an in-crowd of folks who think he is great, and an out-crowd of folks frustrated by his constant unfounded dumping.

I've personally found it is hard to take him seriously when he says a thing, and you should first check which company he is currently flogging. Doesn't mean he is wrong, .. but check.

</comment></comment></comment><comment><USER13> Is there an epub of this?

<comment><USER14> This seems like what you are looking for.

[https://unglue.it/work/153041/](https://unglue.it/work/153041/)

<comment><USER13> Thank you very much.

</comment></comment></comment><comment><USER15> Is this new edition? What have been changed?

<comment><USER16> New in 2015. See also: [https://news.ycombinator.com/item?id=10694538](https://news.ycombinator.com/item?id=10694538)

</comment></comment><comment><USER17> Redbook is too biased, there is just too much perspective from RDBMS people, which is not relevant in modern distributed environments or even outright incorrect.

Redbook inspired list by Christopher Meiklejohn [1] is a better alternative, or Aphyr's course outline [2].

[1] [https://github.com/cmeiklejohn/cmeiklejohn.github.io/blob/ma...](https://github.com/cmeiklejohn/cmeiklejohn.github.io/blob/master/_posts/2013-07-12-readings- in-distributed-systems.markdown)

[2] [https://github.com/aphyr/distsys-class](https://github.com/aphyr/distsys- class)

<comment><USER18> (Disclaimer: I work at Databricks.)

RDBMS techniques are absolutely relevant in modern distributed environments. It has become increasingly clear that MapReduce is too low-level a programming model for query processing, so modern distributed dataflow systems are increasingly hybridizing with RDBMS-like interfaces and optimizations (e.g. Spark dataframes).

<comment><USER4> For OP's benefit, here are some excerpts from the red book that agree with that premise:

> Google MapReduce set back by a decade the conversation about adaptivity of > data in motion, by baking blocking operators into the execution model as a > fault-tolerance mechanism. It was nearly impossible to have a reasoned > conversation about optimizing dataflow pipelines in the mid-to-late 2000's > because it was inconsistent with the Google/Hadoop fault tolerance model. In > the last few years the discussion about execution frameworks for big data > has suddenly opened up wide, with a quickly-growing variety of dataflow and > query systems being deployed that have more similarities than differences

[http://www.redbook.io/ch7-queryoptimization.html](http://www.redbook.io/ch7-queryoptimization.html)

Also see Stonebraker's comment at the bottom here:

[http://www.redbook.io/ch5-dataflow.html](http://www.redbook.io/ch5-dataflow.html)

edit:

To be more charitable, Mapreduce's main concern was fault tolerance (and recovery) and massive scalability, at the cost of all else. Since it's so simple, you could have subtasks die, disappear, and yet you can just respawn them and keep on chugging through the query. You also don't think too hard about job allocation. It's easy to build and use, easy to reason about. You can throw more computers at it when you have a spike of jobs, and it scales fairly predictably. Not many people were really running infrastructure and jobs at the scale google did, and that's quite different from the traditional "data warehouse" style application, and so it wasn't entirely unjustified. The other benefit, of course, is that you can perform arbitrary computation, which is quite different from most RDBMSes which often don't have great UDF support or are frequently highly restricted and, frankly, horrific to deal with.

Of course, they quickly found that "no query optimization" is sort of an extremist and unproductive position, and that you can have a bit of either or both cakes as needed.

<comment><USER12> For further reading about the high quality of database query optimization, and how far back MR et al must have set things, recent SIGMOD work managed to get to within 1000x of a single-threaded implementation (and so, not quite within that of data-parallel systems):

[https://github.com/frankmcsherry/blog/blob/master/posts/2017...](https://github.com/frankmcsherry/blog/blob/master/posts/2017-09-23.md#paper-2-graph- processing-using-sql)

I don't use databases because they are really quite bad at computation.

In my opinion, the main recent novelty in query planning has been the work on worst-case optimal joins, stuff like EmptyHeaded[1] and the recent FAQ work[2].

[1]: [https://arxiv.org/abs/1503.02368](https://arxiv.org/abs/1503.02368)

[2]: [https://arxiv.org/abs/1504.04044](https://arxiv.org/abs/1504.04044)

<comment><USER4> That's not quite what I was going for. I enjoyed your article because I agree with your overall point that distributing stuff has an overhead, and also it points to a definite problem in how some research work is portrayed - <USER2> having to do with what the incentives are in reviewing and publishing.

However I think you're taking your argument to the extreme in a way that doesn't really apply here. First, from what I understand, graph databases are still far from well understood and don't really represent the best in query optimization. This is in no way representative of RDBMS query optimizers for usual OLAP/OLTP tasks, and not what we're talking about right now. Something like SAP HANA or Redshift or experimental systems like HyPer and MonetDB or even something like impala would represent that literature better here. Or check out MapD, which uses GPUs for parallelism. Or kdb+, which has existed for forever and is well-known to do a great job parallelizing and offers a rich query syntax.

Indeed, when I look at the emptyheaded paper for example, I see SIMD parallelism, query compilation join optimization, all stuff that was first developed in the context of RDBMSes. Surprise, surprise, when you apply tried and true techniques in the context of a new problem, you see drastic improvements. This is pretty much exactly the point that Stonebraker and the others above are making: MapReduce was kinda like the graph databases you tested: they were hyper-focused on one functionality, and missed the memo on decades of many other basic optimizations. They're certainly not the only one guilty of this.

> I don't use databases because they are really quite bad at computation.

Well, if computation's all you need ... I mean, I hope you're kidding here, but there are reasons other than performance that you'd want to have a parallel system, e.g. your working set doesn't fit in memory, or you need to minimize downtime. Granted, these are not problems that are common. Also there's many reasons you want a database over a hand-rolled solution: you need to concurrently serve a lot of queries, including insertions and updates, you have to do well on many different types of queries rather than just a single one, etc etc etc.

Also, /what/ system? Bad at /what/ computation? There's so many different systems for so many different workloads that I can't believe you can seriously make such a statement. If you're saying RDBMSes are bad at graph computations, then sure. That's unsurprising. But that's not what we were talking about! :-/

<comment><USER12> > Indeed, when I look at the emptyheaded paper for example, I see SIMD > parallelism, query compilation join optimization, all stuff that was first > developed in the context of RDBMSes.

The main contribution of EH is not the use of SIMD, it is the implementation of new WCO join execution strategies that _hadn 't_ been developed in the past 40 years of RDBMSes.

If you wanted that behavior, with its orders-of-magnitude performance improvements, you _could not get it_ from an existing optimizing RDBMS---not HyPer, nor MonetDB, nor anything else in your list---but you could get it from a more programmable data-parallel system.

> Well, if computation's all you need ...

It is _a_ thing I need, which is exactly the point. If the RDBMSes don't provide the performance (or anything within 100x) you can get from a more programmable system, you need a different solution.

Stonebraker's claim was that MR was a huge step backwards, which is BS to the extent that RDBMSes weren't solving the problems Google (and others) had. No amount of fantasy query optimization was going to take SQL to the performance of MR or MPI codes (even circa 2009, Vertica still had no support for UDFs).

You are of course welcome to list other things that RDBMSes are good at, and that's great. However, Stonebraker's claim isn't that RDBMSes have some value (which everyone I know agrees with), his claim is that MR was a shit model and everyone should be using RDBMSes instead (preferably his).

> If you're saying RDBMSes are bad at graph computations, then sure. That's > unsurprising. But that's not what we were talking about! :-/

Remind me what that was, then? It seemed like we were talking about whether there was a heavy pro-RDBMS bias in the redbook, which I think is (i) fair, and (ii) fine. I also think Stonebraker is wrong in his claim that MR set things backwards because (as I referenced) RDBMSes weren't _there_ to be set back from. If anything, it _prompted_ a great deal of new work that led to improvements in areas he was blind to. A concrete example of this (e.g. iterative computation) seems totally on-topic.

<comment><USER4> > MR was a shit model and everyone should be using RDBMSes instead

Ah, yes, sorry, I didn't mean to make it sound like I agree with this.

> If you wanted that behavior, with its orders-of-magnitude performance > improvements, you could not get it from an existing optimizing RDBMS---not > HyPer, nor MonetDB, nor anything else in your list---but you could get it > from a more programmable data-parallel system.

Of course! I now realize that you're using terminology in a way that I'm not familiar with, e.g. "computation" meaning something like "arbitrary computation", which upon rereading, makes me understand and agree with a lot more of what you're staying.

What bothered me about your comment was that it sounded a bit like "wow, I found that RDBMSes suck at this specific type of computation that they're not built to deal with, therefore query optimizers suck in general", which seemed like an over-reaching argument.

When the claim is "there exists computations that RDBMS query optimizers suck at", then absolutely, I agree with you to the ends of the earth. If it's also "there's reasons why you want a more MR-like model", again, I agree completely. The point is that having query optimizers and different computational models are separate decisions that don't affect each other - <USER3> can have both.

> Stonebraker's claim was that MR was a huge step backwards, which is BS to > the extent that RDBMSes weren't solving the problems Google (and others) > had.

I guess what I took away from his claim was that the contribution of MR itself was not the problem, but the fact that while creating that model, they ignored a lot of other learnings: e.g. blocking operators can be detrimental, indexes are handy to have. Plus the fact that everyone /else/ who didn't have Google's reasons to forego all those niceties still dove head-first into "let's use MR for everything".

And that's what Elvin is talking about above - you're now seeing examples of tools where lessons from both camps are being applied: "MR-like but be smart enough to not scan everything" (e.g. spark).

> It seemed like we were talking about whether there was a heavy pro-RDBMS > bias in the redbook

Ah, for me the main question and discussion above was "are RDBMS techniques even relevant at this point", to which my answer is yes, absolutely. That doesn't mean you have to take every concept from it wholesale: many techniques that developed in one context are applicable in others, regardless of Stonebraker's opinions.

I think also maybe you see everything as firmly in the RDBMS / SQL camp or firmly in the "not" camp, but I really don't think that's the case. E.g. stuff like flink, where we have a lower layer API for arbitrary computations, and higher level APIs for stuff like SQL which compiled down to the lower level language, and get query-optimized different ways in different layers, for example. Or they do some neat join-optimizations. So even with newer computational models there's stuff to learn from old ways, so it's worth it to read the darned book. That's my point here.

<comment><USER12> > What bothered me about your comment was that it sounded a bit like "wow, I > found that RDBMSes suck at this specific type of computation that they're > not built to deal with, therefore query optimizers suck in general", which > seemed like an over-reaching argument.

Gotcha. Yes, it was more "I have some things I need to do, and RDBMSes can't do some of those things, which rules them out as a solution". There is for sure lots of great stuff in query optimization, and it makes some queries lots better.

> I guess what I took away from his claim was that the contribution of MR > itself was not the problem, but the fact that while creating that model, > they ignored a lot of other learnings: e.g. blocking operators can be > detrimental, indexes are handy to have. Plus the fact that everyone /else/ > who didn't have Google's reasons to forego all those niceties still dove > head-first into "let's use MR for everything".

They did not ignore them, they just weren't building a database. MR was much more a scalable HPC replacement than a data management product.

The main reason that the DB community took a huge step backwards is that they (incl Stonebraker) had doubled-down on mediocre compute abstractions, and found they needed to revisit much of what they'd done, because it just didn't work.

> Ah, for me the main question and discussion above was "are RDBMS techniques > even relevant at this point", to which my answer is yes, absolutely. That > doesn't mean you have to take every concept from it wholesale: many > techniques that developed in one context are applicable in others, > regardless of Stonebraker's opinions.

I totally agree with you that they are relevant (and am active in the area). Stonebraker takes a much stronger position, and the appeal to his authority was what triggered me. I didn't mean to point that at you as much as it may have turned out.</comment></comment></comment></comment></comment></comment></comment></comment>