Grammar Books for Technical Writers - <AUTHOR> https://medium.com/technical-writing-is-easy/grammar-books-for-technical-writers-9f299a29d54d <comment><USER2> Yet another collection of affiliate links (tag=fivboo-21). Second submission in less than a day.</comment>


How to Use [Google's] Wi-Fi Assistant to Keep Your Phone Safe on Public Networks - <AUTHOR> http://www.howtogeek.com/275474/how-to-use-androids-wi-fi-assistant-to-keep-your-phone-safe-on-public-networks/ <comment><AUTHOR> Google's Support page on Wifi assistant:

[https://support.google.com/nexus/answer/6327199?hl=en](https://support.google.com/nexus/answer/6327199?hl=en)

      To make open Wi-Fi networks safer, Wi-Fi assistant uses a virtual private network (VPN).
      The VPN protects your data from being seen by other users of the open network.

      When a VPN is active, you'll see a "Network may be monitored" message.
      Google monitors system data. When you're securely connected
      to a website (by HTTPS), VPN operators, like Google, can't record your content.
      Google uses system data sent through VPN connections to

      - Provide and improve Wi-Fi assistant, including the virtual private network (VPN)
      - Monitor for abuse
      - Comply with applicable laws and regulations, or as required by court or government orders

What does HN think of this model?</comment>


AMA: O-1 and EB-1A "Extraordinary Ability" Visas - <AUTHOR> http://visabuilder.com/blog/ama <comment><AUTHOR> A couple weeks ago there was a really good US immigration AMA here hosted by Peter Roberts, the immigration attorney for YC. [https://news.ycombinator.com/item?id=14703052](https://news.ycombinator.com/item?id=14703052)

This inspired me to do this separate AMA on my site, specifically for O-1 and EB-1A "extraordinary ability" questions.

If you ask questions here I'll answer them too.</comment>


Notes from Startup School NY 2014 - <AUTHOR> http://myusings.tumblr.com

<comment><USER2> Good job! I did Startup Notes ([http://startupnotes.org/](http://startupnotes.org/)) in the MV one, and thought about doing them for the NY and London ones. Glad someone picked up my slack!

<comment><AUTHOR> Your notes are great! Thanks for sharing. Yeah I actually had no intention of sharing my notes while I was taking them so they are not very comprehensive. My friend asked me to post them so I did.

</comment></comment><comment><USER3> Also few notes in russian, if anyone interested [http://andrebrov.net/blog/central-park-notes-yc-startup- scho...](http://andrebrov.net/blog/central-park-notes-yc-startup-school/)

</comment><comment><USER4> My less artistic but more comprehensive notes: [http://personalopz.com/blog/startup- school-2014-notes/](http://personalopz.com/blog/startup-school-2014-notes/)

</comment><comment> k-mcgrady Very nice. What stylus did you use for these? Also, does anyone know if/when the videos will be made available online?

<comment><USER5> The full video is still available on livestream: [http://new.livestream.com/mbp/events/3083143](http://new.livestream.com/mbp/events/3083143). I haven't seen individual archived videos yet.

<comment> k-mcgrady Thanks!

</comment></comment></comment><comment><USER6> My key takeaways:

[http://www.brevedy.com/2014/06/19/key-takeaways-from-yc- star...](http://www.brevedy.com/2014/06/19/key-takeaways-from-yc-startup- school-2014-in-nyc/)

</comment><comment><USER5> Very nice! Those are the most elegant set of conference notes I've ever soon. You definitely have talent.

</comment><comment><USER7> I think I can summarize this in one word 'Respect'</comment>


New York City in Timespace - <AUTHOR> http://blog.gradientmetrics.com/2017/12/07/new-york-city-in-timespace/ <comment><AUTHOR> Hello! I am the author. This was a super fun project and something I've been randomly obsessed with for a while. Any Qs and I'd be happy to answer!</comment>


What are u thinking about this platform? - <AUTHOR> https://www.cofoundme.org
Please tell me your opnion about missing features, design etc
Thanx in advance <comment><USER2> When I read your misspellings and poor punctuation in the post, I thought it likely a waste of time to review the site.</comment>


IPhone hackers pick wrong target - <AUTHOR> http://www.therecord.com/news-story/6815795-iphone-hackers-pick-wrong-target-a-uw-expert/ <comment><USER2> tldr:

"Cryptography. Security. Privacy. Human computer interaction." ... and picking shitty iCloud passwords "Those are his specialties."</comment>


How Australian Tech Companies Can Think About Silicon Valley - <AUTHOR> https://muru-d.com/blog/article/how-australian-tech-companies-can-think-about-silicon-valley/ <comment><USER2> While the US will always be dominant, there are some advantages for Australian Tech companies to take advantage of:

Cheaper tech teams with less competition. Australia has a lot of talent in the software space, but without the eye watering wage bills. Even in the most expensive cities of Sydney and Melbourne, software developers generally cost a lot less than in the Bay Area. You can get even better value if you can build your team outside of Sydney/Melbourne.

You can build more slowly, there is alot less pressure to build grow fast and fail type startups. While venture capital heavy startups are the most celebrated in the press, there are plenty of great smaller scale self/revenue funded startups that can build up over time. Most of our celebrated tech successes built their business models over a long time and focused on building a good business first.

There are also great incentives to keep the development team in Australia. As a pre revenue startup you can get over 43.5% of your development cost back through the R&D Tax Incentive. While its a tax incentive, the way that they designed it was to provide tax refunds to loss making companies. Plus there are other great grants/incentives you can also take advantage of. (disclaimer: I'm a consultant for software startups/companies in this area)

The best course of action is to take a global approach, don't limit yourself to selling or developing in only one location.

<comment><USER3> This is a great post. I'd like to add:

> You can get even better value if you can build your team outside of > Sydney/Melbourne.

So much this, but talent outside the big cities is _rare_. I lived in Queensland for a while and would have taken a massive pay cut for _any_ tech work... but there just isn't any. I knew more people working in Sydney (fly- in-fly-out) than working in local companies.

You can pick up uni students cheaply. They're grateful for relevant work.

I knew a few founders living cheap but pretty much all knew that moving to the US was an inevitability.

> focused on building a good business first

This is a massive plus of starting in Australia. The business _has to make sense_. Local funding sources aren't as willing to build perceived value and flip the company -- it's important to have a genuine sustainable business with customers and cashflow.

> 43.5% of your development cost back through the R&D Tax Incentive

Probably half of my lifetime earnings have been paid for with the R&D Tax Incentive and other grants. It's such a good deal that every incorporated company with a tech component should be trying to claim it.

I'm not grumpy or anything, but in Queensland, claiming government grants for tech companies is probably more lucrative than working in a tech company. There's an abundance of funding and a shortage of people able to use it.

> don't limit yourself to selling or developing in only one location

I forget who gave the talk, but it was from Muru-D at Co Spaces in the Gold Coast, and guy said (paraphrasing):

"New Zealanders have an advantage over Australians because they _know_ from _day one_ that they will have to sell overseas to make their company a success. Australia is big enough that you can fool yourself for a while, but it won't take long before you reach saturation and need to move anyway. Better to get it done sooner."

<comment><USER4> Where in Queensland? There is plenty of tech going on in Brisbane if you look for it... I suppose it depends on your specialisation though.

<comment><USER5> Just curious: what is some cool stuff happening in Brisbane these days? I grew up there but I moved away to find more interesting work a long time ago. It would be nice to hear about interesting things happen there nowadays.

<comment><USER4> We have dev offices for quite a few majors (Oracle, SAP, IBM, ToughtWorks, Boeing, Boeing Defense etc.), but lots of little interesting companies too.

I'm in Systems programming/electronics design - it's probably a different story for a front end dev - I don't think there's much of that at all!

One little company I used to work for was in traffic systems, basically IoT/sensor networks before it was cool (all on private networks though!) and we developed a system that GPS tracked ambulances and fire engines, and predicted their likely path (destination was known) and ran interventions on traffic signals to both minimise travel time for the emergency vehicle and minimise disruption to other traffic. It's a lot more effective than the dumb IR blaster systems used elsewhere. We also had a ramp system that staggered cars coming in down on-ramps near chokepoints that significantly increased highway flow and reduced nose-to-tail accidents (from the stop-start traffic) by something like 30%. I think they're rolling out that tech now with some US cities departments of transport.

Now I work at a little company that makes class-leading satellite/microwave equipment, including maritime and road-vehicle satellite-tracking terminals, frequently up- and down-converters, etc.

There's actually a surprising amount of microwave engineering, and lots of aviation/defence work (some in the city with Boeing Defense and some out in Ipswich at Amberley), as well as Virgin Australia being headquartered here.

<comment><USER5> That's great to hear about, thanks!

</comment></comment></comment></comment></comment></comment><comment><USER6> One of the common mistakes I see Australian (and other incoming) startups make is "hiring/setting up a sales person in SF".

It seems like a highly leveraged thing to do. Surely someone in-market, full- time, with sales experience should do 10x.

But you learn so much in the sales process. If you have someone remote, new and purely incentivized on sales you lose much of that information.

Quite often the sales person "fails". The company notches it up to being the wrong person, fires them and tries again. However, more often it's the product, the positioning, so many other things.

<comment><USER7> So what is the correct thing to do here? Do the selling yourself or go along to sales meetings?

<comment><USER8> You can't sell remotely from Australia via a US sales team. You either sell directly from Australia (with all the downsides this involves) or you move the management team to the USA.

<comment><USER9> The third option is to acquire a US competitor, let them sell your product into the US market, and make sure they have the same executive decision making power as the team in Australia

<comment><USER8> Yes - I am actually considering doing this right now. The difficulty is the massive mismatch in valuation between Australian and US companies makes this difficult.

</comment></comment></comment></comment></comment><comment><USER4> One massive advantage of our company doing the build local, sell global strategy from Australia is that in our game (satellite/microwave comms, mostly for the defence industry) is that if we were in the US, our products would be under ITAR. Not having to do all that paperwork is a huge selling point for our European and Asian customers. I do spend a lot more time on planes than I might otherwise, but at least we are close to Asia.

</comment><comment><USER10> The liveability of places like Adelaide (and even Melbourne, Sydney, Brisbane, and Perth) is such an undersold secret. Personally I'm so torn between wanting to tell everyone how good we have it and wanting to keep it to ourselves.

Melbourne, Adelaide, and Perth are all in the top 10 cities to live in the world.

Taking Adelaide for example (1.2 million people), we can live within 10-15 minutes of the CBD in a roomy townhouse for highly-affordable rent, get paid pretty well, be 20 minutes from the beach, 20-60 minutes from four different world-class wine regions, have a great festival culture, delicious local food and wine, and a highly safe and welcoming culture.

If you want a larger city you have Melbourne (3.9 million) with a fantastic bar and food culture and endless high-paying work.

As a company, and this is somewhat anecdotal, you have high availability of highly-educated workers who are used to working collaboratively and doing whatever's needed to get things done to cover the work of N other people (low population after all).

<comment><USER3> I want to drink your kool-aid.

I feel like they are highly livable _if you do not need to work for a living_.

Otherwise:

\- Adelaide: It's cheap but there are no jobs.

\- Brisbane: It's cheap(ish) but there are no(t many) jobs.

\- Perth: It's expensive but there are no jobs.

\- Sydney: You can't afford it. A one-hour commute is starting to look appealing.

\- Melbourne: Can't complain.

They are lovely cities to visit as a tourist. As a worker, there are much better opportunities out there. Australia tech workers have easy access to working visas in practically every country in the world.

Australia will always be there for retirement.

<comment><USER10> I can agree there are more opportunities elsewhere, but ive never met a developer or similar in Aus struggling to do well, and have generally found US devs shocked by our salaries and living costs.

On the other hand, if you want career advancement there's something pretty cool to the US idea of "have managed 15 people after 2 years experience" which just doesn't happen here in Aus.

<comment><USER3> As soon as we had a child, we went from "doing OK on two professional salaries" to "losing money even with a modest lifestyle."

Outside Aus, we only need one income to support the family. That's a pretty amazing quality of life improvement.

</comment></comment></comment></comment><comment><USER11> In my view with Australia cancelling the 457 visa that a lot of tech talent uses its pretty much over for everything tech related in AU. If you're looking for a fast growing tech city thats less expensive than the valley then head to Toronto

[https://www.gostudy.com.au/blog/australian-457-sponsorship-v...](https://www.gostudy.com.au/blog/australian-457-sponsorship- visa-to-be-cancelled/)

<comment><USER12> It was being abused to bring down local salaries. Good riddance.

</comment></comment><comment><USER13> I think that an Australian business going deeply into the US market is going to struggle with the E3 visa. It is so easy for a talented technologist to be recruited by a US firm that once someone works it out, you will see a rapid turnover unless you offer US rates.

</comment><comment><USER14> > Build local, sell global. This is the ideal. Hire the local talent (keeps > government people happy) but sell into the big market.

Yes. But if you're selling components (pretty rare TBH, most sell a service provided via a web page, or maybe a desktop or mobile app), you have to pay "royalty" tax to the IRS. Only about 5% IIRC under the us-aust tax treaty, and you're not double-taxed by the ATO - but it's paperwork, complex, and a little scary. Being subject to one jurisdiction's set of tax legislation is bad enough....

<comment><USER15> Can you provide some reference, or even the law/statute name that you're referring to here?

<comment><USER14> Was a few years ago, I'd have to google it myself.

google: US australia tax treaty; search: "royalty" within it (has a particular meaning in the act). The term "component" is _not_ actually used, IIRC. When the IRS audited a US firm I know, they had to sort it out with their non-US component suppliers.

I'm not at all sure, but maybe "w8-ben" is the relevant form (though it serves several purposes). Whatever the form is, I noted that Atlassian had a pdf of it on their site, and avoided all the complexity because their products were used directly by users (i.e. aren't components).

HTH!

<comment><USER15> All I can find is about if a contract between parties involves royalties, it may be taxed. Maybe that's what you meant?

<comment><USER14> Doesn't sound like it but unfortunately that's not enough information for me to tell. It might be. BTW a "contract" includes pretty much every transaction, even if immediately performed.

From ATO tax treaties: [https://treasury.gov.au/tax-treaties/income-tax- treaties/](https://treasury.gov.au/tax-treaties/income-tax-treaties/) to USA DTA (double taxation avoidance) "Convention between the Government of Australia and the Government of the United States of America for the Avoidance of Double Taxation and the Prevention of Fiscal Evasion with respect to Taxes on Income" [http://www.austlii.edu.au/au/other/dfat/treaties/1983/16.htm...](http://www.austlii.edu.au/au/other/dfat/treaties/1983/16.html)

> Article 12 Royalties

> (4) The term "royalties" in this Article means:

> (a) payments or credits of any kind to the extent to which they are > consideration for the use of or the right to use any:

> (i) copyright, patent, design or model, plan, secret formula or process, > trademark or other like property or right;

So, by 12.(a)(4)(i), if a customer pays you so they can redistribute your copyrighted code (i.e. a component), it's a "royalty".

BTW Reading it literally, this "right" includes the mere _use_ of copyrighted material - but here it's meant as more serious aspects of copyright, like the right of (re)distribution.

IIRC, there's a wrinkle: in something like a compiler, which is used by the customer and not redistributed, but includes some part that _is_ redistributed (like stdlib), this anxilliary distribution doesn't alter the character of the primary transaction. IIRC, the ATO stated this in what I think is called a "tax note" \- a decision made internally (i.e. not held by a court). tl;dr you can get away with a _little_ redistribution

<comment><USER15> Right. As usual the software is never the fucking hard part, its the ancillary shit like law/tax etc.

Open source code + paid services/support seems like it'd be the easiest way around this for small organisations, no?

<comment><USER14> It's a barrier to entry, favouring big caps that can afford international tax law specialist accountants/lawyers.

I think that works: free means no royalty paid; and the convention above specifically excludes ordinary services (IIRC). It includes "scientific" etc advice in the next point (ii), and I never looked into exactly what that covers... services/support for an ordinary component doesn't sound like it's covered (and if it was, it wouldn't be because it was a component - the issue I raised).

As a separate business issue, there's other problems with getting people to pay services for something they already have - in that sense, it's not "a way around". Of course, some market segments do really want that support. And if that's already your business model, well no problem.

I can't think of any examples of an open source _component_ with services/support. SleepcatDB and ghostscript are opensource components, but are dual-licensed (i.e. have a second proprietary license permitting redistribution as closed source). MySQL? Cygwin? Hibernate/JBoss? Red Hat?... not sure these are components that are often redistributed (though of course users could, license permitting).

</comment></comment></comment></comment></comment></comment></comment><comment><USER16> > You're starting a journey of effort, consideration and perseverance > because...

> You can succeed there.

Just bear in mind that "you can" doesn't mean "you should".

</comment><comment><USER17> One of the strange things about Australia's startup scene over the last decade or so is the sheer number of what I like to call "meta-entrepreneurs" \- people who help other people become entrepreneurs, and present themselves as experts on technology, business, Silicon Valley etc, but have declined to actually do the hard work of setting up a business, let alone growing it to success.

So when Australians give advice about "how to start a tech company", the first question to ask is "How many have you personally started? And how did they go?".

One of the things that I've come to understand about Australia since leaving (I've live in Berlin now, before that I was in London) is that the average level of professional work is quite mediocre by global standards - this is not just in tech, but across all industries. Australians have had it very easy for about 2 decades now, getting rich through a property boom and a mining boom, but most have never had to really compete on a global level. An awful lot of people cruise in their jobs, doing something that's good enough for the Australian market, and they've had a comfortable career doing it. So why change?

The problem is, that if you do achieve PMF in an economy that operates primarily at this level, you're not going to have PMF in an economy that operates at a higher level of sophistication and competence. That's why I think the strategy of "start local, grow global" is a fool's errand, and risks turning into "start local, burn some cash trying to go global, stay local".

I also dispute that there is an advantage to keeping a tech team in Australia. The salaries are still quite high (particularly in Sydney) but the talent is again, not that good. This is both in tech and tech management, however there ARE exceptions[1], and the situation is improving - Atlassian have produced a lot of people who know what it takes to run a global, successful tech business, and now we are starting to see a second generation of top-notch people being developed in fast-growing Australian startups like Canva and SafetyCulture[2].

There are plenty of companies who have sold into the US market from day 1 from outside the US. HotJar from Malta. My current employer, ChartMogul. Travis CI, also from here in Berlin. There are hundreds more, but those are the ones that just jumped to mind.

If you want to be a global company, then think like a global company from day one. This is as true in Australia as anywhere else, but it's taken a long time to sink in there.

[1] Of course, the exceptional people generally know it, and command matching salaries. As an aside (to this aside), I'm currently reading Camille Fournier's excellent "The Manager's Path", which is like a museum of management practises that I've never seen in Australia.

[2] SafetyCulture is an exception to my "start global" thesis. They did start locally, then grow. I think this is because of two main things: In health and safety things, Australia does generally lead the world, or come close to it. I used to work in construction in Australia, and am continually shocked at the practises I see in the EU and the US when I visit. The other thing is that I've seen SafetyCulture hire a lot of very good people, which is never a bad thing for a business to do.

<comment><USER18> I find your comments about technical competency of Australians being less than elsewhere interesting. My own experience over the decades is the opposite. I have had to deal professionally with various fly-in gurus (from the US and Europe) and they have had little success in understanding how we implemented the projects we were involved in.

what was more ridiculous was the major discrepancy in pay. If US or European, then you were paid 3+ times the Australian rates. Yet you couldn't solve what we thought were simple problems.

There is no doubt that there are many who claim technological acumen here who have much less than they advertise. But there are many here who have much more technological acumen than what you find overseas.

The biggest problem is the extensive need to use buzzwords in the various technologically based industries. If you are not up with the latest buzzword craze, you are considered backward and inferior, even though those buzzwords are just a renaming of various "stuff" that has been around for decades.

<comment><USER19> > they have had little success in understanding how we implemented the > projects we were involved in

I don't think I've ever had trouble helping people to grok the implementation/architecture/whatever of any system I've been involved in.

<comment><USER18> What I actually found was that they were unable to think outside of the box they were stuck in. They were in a position of "you can't do that" when the problem at hand required that "you need to do that" so find a way to do it.

I think one of the problems of today is the limited education that many in technical fields have. They are taught how to do something in one language or system and they do not get to see that other ways are possible in other languages and systems.

One of my former colleagues (from decades ago) was a brilliant programmer. We had a particularly pernicious problem to solve. He had spent some hours programming a solution around the problem and had written about half the code needed. In discussions with him, I brought up that old chestnut of Karnaugh Maps and Don't Cares. He had never heard of either. Once explained to him, he fixed the entire problem with 5 lines of code, throwing away all the previous work. He was very capable of thinking outside the box.

I have worked with many very capable Australian technical people over the years and they have been less valued than foreign people even though the Australians were vastly superior to their overseas counterparts.</comment></comment></comment></comment>


Stop Buying Load Balancers and Start Controlling Your Traffic Flow with Software - <AUTHOR> http://bits.shutterstock.com/2014/05/22/stop-buying-load-balancers-and-start-controlling-your-traffic-flow-with-software/ <comment><USER3> I'm really reaching back into the depths of my memory, but I've implemented this in the past. It's not quite as simple as they make it sound - there's a lot of sticky edge cases that crop up here (some of which have no doubt been addressed in subsequent years).

\- It heavily limits the number of nodes you can have - that is something the article does say, but I want to highlight here. It strikes me as a really bad strategy for scale-out.

\- I've run into weirdness with a variety of different router platforms (Linux, Cisco, Foundry) when you withdraw and publish BGP routes over and over and over again (i.e. you have a flapping/semi-available service).

\- It is true that when a node goes down, the BGP dead peer detection will kick in and remove the node. _However_ the time to remove the node will vary, and require tuning on the router/switch side of things.

This is a fairly crude implement to swing - machete rather than a scalpel. You lose a lot of the flexibility load balancers give you, and depend a lot more on software stacks you have less insight and visibility into (router/switches) and are also not designed to do this.

My suggestion would be that this is a great way to scale across multiple load balancers/haproxy nodes. Use BGP to load balance across individual haproxy nodes - that keeps the neighbor count low, minimizes flapping scenarios, and you get to keep all the flexibility a real load balancer gives you.

One last note - the OP doesn't talk about this, but the trick I used back in the day was that I actually advertised a /24 (or /22, maybe?) from my nodes to my router, which then propagated it to a decent chunk of the Internet. This is good for doing CloudFlare-style datacenter distribution, but has the added benefit that if all of your nodes go down, the BGP route will be withdrawn automatically, and traffic will stop flowing to that datacenter. Also makes maintenance a lot easier.

<comment><USER4> > _My suggestion would be that this is a great way to scale across multiple > load balancers /haproxy nodes. Use BGP to load balance across individual > haproxy nodes_

Exactly. BGP, while it may work like the OP said, was not meant to live this close to the actual server nodes.

You could push BGP even further away. In a more traditional model, it's meant to be used to switch (or load balance) between geographically separated datacenters.

</comment></comment><comment><USER5> This works well if you need to push high bit rates and are lookig for relatively simple load balancing. A trident box, ala juniper qfx, can push a few hundred gbs for ~$25,000. Thats an incredibly low price point compared to any other lb solution.

Some caveats and comments about the technique.

BGP & ExaBGP are implementation details. OSPF, quagga, & bird will all accomplish the same thing. Use whatevere your comfortable with.

Scale out can get arbitrarily wide. At a simplistic design youll ECMP on the device (TOR) where your hosts are connected. Any network device will give you 8 way ECMP. Most Junos stuff does up to 32 way today, and 64 way with an update. You can ECMP before that as well, in your agg or border layer. That would give you 64 x 64 = 4096 end points per external "vip."

ECMP giveth and taketh away. If you change your next hops expect all those flows to scramble. The reason is that ordering of next hops / egress interfaces are generally included in assignment of flows to next hop. In a traditional routing application his has no effect. When the next hops are terminating TCP sessions youll be sending RSTs to 1/2 of your flows.

For this same reason youll have better luck advertising more specific routes, like /32s instead of a whole /24\. This can help limit the blast radius of flow rehash events to a single destination "vip."

There are more tricks you can play to mitigate flow rehashes. Its quite a bit of additional complexity though.

For the same reason make double plus sure that you dont count the ingress interface in the ECMP hash key. On Junos this is incoming-interface-index and family inet { layer-4 }, IIRC.

You _really_ dont want to announce your routes from the same host that will serve your traffic. Separate your control plane and data plane. Its terrible when a host has a gray failure, say oom or read only disk, and route announcements stay up while the host fails to serve traffic. You end up null routing or throwing 500s for 1/Nth of your traffic.

<comment><USER6> Great post, you said things better than I ever could.

So I'll nitpick instead :)

> You really dont want to announce your routes from the same host that will > serve your traffic. Separate your control plane and data plane. Its terrible > when a host has a gray failure, say oom or read only disk, and route > announcements stay up while the host fails to serve traffic.

This was probably the largest area I spent architectural time on before deployment. Was it better to run a tiny little healthcheck script on my HAProxy boxes to tear down bgpd, or should I run a route server.

In the end, I went with what I felt was the simpler solution of the two for our scale (~60 total HAProxy boxes spread around the world) and used an extremely simple "is haproxy accepting connections on the VIP or not" script that stayed in memory. It was also tossed in init, just in case it did get oomkilled or crashed - and in the end we never had an outage based on a service check that should have been caught. Knock on wood. Completely agree that a route server is better than a naive script ran from cron as a service check.

The route-server method is more interesting to me at scale, but adds additional complexity to the problem. Now people need to know how this all works, your service checks need to execute remotely (not a bad thing!), and probably a few other things I'm forgetting. However, it makes management and scaling much better and is the way I'd go if I did it all over again.

The failure model for this setup is basically is HAProxy up? Yes? Then announce routes. If not, pull routes. HAProxy was responsible for detecting the health of the application itself and deciding what to do with an app failure. We did add some code later on to down haproxy should it be unable to reach _any_ webservers, but honestly the complexity and additional failure modes this adds usually is not worth it due to it being such a rare event.

<comment><USER5> Ha! I _did_ have an outage because route announcements and data plane were on the same host a few years back. Having a separate health check service & route server is a trade off of complexity vs control. I could see the argument when you only have a couple hosts total. With dozens of endpoints in a fleet its quite nice to tolerate more wonky grey failures.

Unfortunately I dont know of any existing public lib/application/framework that does this type of layer 2/3/4 load balancing for fleets of endpoints. The vrrp/carp/keepalive/heartbeat crew seem focussed on master/slave failover which is totally uninteresting to me.

<comment><USER6> Yeah, it's pretty custom. We started doing this nearly 10 years ago and when explaining it to vendors their eyes would glaze over. These days it seems quite a bit more common - and I hope I had a tiny bit to do with that evangelizing it wherever I could.

Curious how you solved the hash redistribution problem? We never came up with anything good (some clever hacks though!), but luckily for our uses it wasn't a big deal and we could do away with a whole shedload of complexity.

The best we came up with was pre-assign all the IP's (or over-assign if you wanted more fine-grained balancing) a given cluster could ever maximally utilize. Then distribute those IPs evenly across the load balancers, and have the remaining machines take over those IPs should there be a failure. This was complicated as hell, and obviously broke layer3 to the access port so was a non-starter.

I'm sure we had better/more clever ideas, but we never had reason to chase them down so I honestly forget. At this point, if someone needs to refresh a page once out of every 100 million requests I'm pretty happy.

</comment></comment></comment></comment><comment><USER7> Half a million dollars of load balancers? Either you are buying from the wrong vendor, or you have some wonky ideas of how many load balancers you need per data center, or are not using them correctly. ( hint: check a10 networks and Zeus)

The reality is, that if your problem is only L3, then arguably this can be solved many ways. For example, networks have been doing 10s of gigabits of L3 load balancing using DSR for ages. Dynamic route propagation doesn't have a hold on this ( albeit it's more "elegant" ).

But most people do more than L3, and really do L4-L7 load balancing, and most modern "application load balancing" platforms are really software packages bundled up in a nice little appliance. This is where packages like varnish with its vcl/vmod's and caching, Aflex ( from a10 networks ) and Traffic Sscript from Zeus , amongst others, come in. Shuffling bits is the easy part! Understanding the request, and making decisions on that is the harder part.

If you split the problem, and are using varnish or nginx as your application load balancer, you can't claim you've gotten rid of them, you were either not buying the right initial platform, or using it correctly. When you put a "stop buying load balancers"… you must first define what you mean by "load balancer" ;)

For the record, I've used both commercial load balancing platforms as well as contributed patches and used OSS load balancing platforms..

<comment><USER8> Half a million dollars of Netscalers is extremely easy, and given that Google originally ran all Netscalers, a lot of ex-Google people that run operations at startups now default to them as a kneejerk. Similarly, F5 is a pretty easy budget torpedo, too.

<comment><USER9> Nah, $20K and free shipping: [http://www.amazon.com/F5-Networks-Standard- Contract-F5-BIG-L...](http://www.amazon.com/F5-Networks-Standard- Contract-F5-BIG-LTM-1600-4G-SBDL/dp/B00B9HZ328)

<comment><USER8> You'd have a compelling case if I only needed one.

</comment></comment></comment></comment><comment> at-fates-hands This is actually a great idea. The last company I worked for ran into several issues with load balancing servers. Two out of the three major releases I was around for were an unmitigated disasters.

The first was because of old load balancing servers getting bogged down with traffic. CIO got pissed, dropped three million on brand new spanking SSD drives and new "state-of-the-art" servers. Cue to the next release.

Pretty much the same issue. It was two lines in a program that was calling a file from Sharepoint - thousands of times a second to the server and bogged down all three of the load balancer servers with traffic within minutes of the release. Took the back-end developers a week and some help from Microsoft to fix the bug. I just sat back and giggled since the CIO spent two hours in a meeting with the whole IT department lecturing them on the importance of load testing immediately after the first releases failure.

Needless to say, they didn't do any load testing for the applications either time which contributed to the issue. Of course, it just goes to show even with the bestest, newest hardware, you can still bring your site/applications to its knees.

<comment><USER4> > _It was two lines in a program that was calling a file from Sharepoint - > thousands of times a second to the server and bogged down all three of the > load balancer servers with traffic within minutes of the release._

That type of issue can be quickly identified if you have, on the team, the sort of mind who is inquisitive of what goes on at low levels, and is not afraid of poking around with tcpdump and network interface traffic counters and stuff like that.

</comment></comment><comment><USER10> This article is a bit low on details, so it's hard to judge the quality of the proposed solution (without having tested a similar setup).

We faced the choice of either upgrading our aging Foundry load balancers or building our own solution a few years ago and came up with a very stable and scalable setup:

2+ load balances (old web servers sufficed) running Linux and:

* wackamole for IP address failover (detects peer failure with very low latency, informs upstream routers; identical setup for all load balancers works, can be tuned to have particular IP addresses preferably on particular load balancer hosts) [http://www.backhand.org/wackamole/](http://www.backhand.org/wackamole/)

* Varnish for HTTP proxying and load balancing (identical setup on all load balancers) - www.varnish.org

* Pound for HTTPS load balancing (identical configuration on all load balancers, can handle SNI, client certificates etc. ...) [http://www.apsis.ch/pound/](http://www.apsis.ch/pound/)

This scales pretty much arbitrarily, just add more load balancers for more Varnish cache or SSL/TLS handshakes/second. We also have nameservers on all load balancers (also with replicated configuration and IP address failover). Configuration is really easy, only Varnish required some tuning (larger buffers etc.) and Pound (OpenSSL really) was set up carefully for PFS and good compatibility with clients.

The only drawback is that actual traffic distribution over the load balancers is arbitrary and thus unbalanced (wackamole assigns the IP addresses randomly unless configured to prefer a particular distribution), but the more IP addresses your traffic is spread out over, the less of a problem this becomes.

<comment><USER6> This solution works, but as you point out balancing over your load balancers is a huge hack and relies basically on DNS.

If you replaced wackamole/DNS RR with BGPd on your varnish/pound boxes, you would achieve the same goals but be able to fully direct your traffic flows yourself vs. a random RFC-busting DNS cache somewhere.

The other big downside to this solution is being forced to run a l2 broadcast domain for failover to work. Fine at your scale here, but when you get into dozens or hundreds of switches and larger scale I firmly believe (if at all possible) dropping L3 down to the access port is the way to go. Debugging STP issues on such networks is about the last thing in the world I'd like to be doing on a friday night.

</comment></comment><comment><USER11> Really he's talking about layer 4 load balancing, not 3, and assuming your juniper router has an Internet Processor II ASIC to juggle tcp flows. You're still buying hardware to do the load balancing, you just use software to do the BGP announce.

Honestly it all seems a bit crude and unreliable. If i'm writing a software load balancer i'm not going to use curl, bash scripts and pipes to do it. But this is why devops people shouldn't be designing highly available traffic control software.

<comment><USER12> It's a cool hack because you probably already have bought the hardware to do this. Most decent switches these days can run BGP (or at least OSPF, which is capable of the same thing). And switches are usually way cheaper than hardware load balancers, at least per-port. Sure it's not perfect but for a startup trying not to spend a lot it can get you pretty far.

<comment><USER11> I don't know. From an infrastructure perspective I don't like the idea of having too many eggs in one basket, like combining the router/switch with the lb vips. Ideally you'd get a couple commodity boxes and configure them with LVS or pfsense or something. That way things like maintenance and access control of different parts of your network are separated based on the resource, and stability of one component won't necessarily affect another. It would also probably be cheaper to buy a couple servers than buy a couple routers/switches for redundancy.

<comment><USER6> You're not understanding how this works at all. Your router/switch doesn't "combine" the vips.

Ignore the running of bgpd on the webserver - that's really an extremely specific use-case that is not useful for most folks.

However, imagine your scenario where you have two routers, two switches, and two load balancers configured in failover (LVS per your example) - <USER2> webservers behind that stack.

Now you need more than a single load balancer worth of capacity? How do you scale it?

Generally, you're pretty much stuck doing DNS RR to load balance across VIPs, and you add at least one VIP per load balancer you have. Need to do maintenance? Good luck not directing traffic to the load balancer you want to take out of service :) You can wait 3 days for all the DNS caches in the world to purge, or you're going to be killing sessions when you fail that VIP over.

Now consider instead running BGPd vs. DNS RR.

You have a single VIP, and as many load balancers as you like. I enjoy HAProxy, so I'll use it here.

haproxy01 184.108.40.206 haproxy02 18.104.22.168 haproxy03 220.127.116.11

All these machines advertise the VIP to the switch they are connected to. You set up path cost on your network so all these VIPs share an equal cost at your routers, and your router will ECMP to each. Doesn't matter what switch or rack your HAProxy box is connected to in your network (I suggest paying attention for traffic management reasons) - as long as it can speak BGP to the switch/router, the traffic flows.

Need to do maint? Kill bgpd on one of the HAProxy boxes. Current sessions stay up, you go get a coffee, and when you come back you have an empty session table and are free to do whatever you like to the machine. Turn it back on by starting up bgpd, and watch your traffic instantly rebalance.

You are completely correct that if you don't need more than one (in a pair for HA) load balancer worth of capacity, this solution is likely overkill. But once you need to start scaling, you're quickly going to learn that it will either be prohibitively expensive, or come with lots of downsides. There pretty much are no downsides to this architecture, other than needing someone with a small amount of clue to operate it.

<comment><USER11> What I was saying with that "combining" is that your router is now essentially the vip, in the sense that it is the load balancer and the peers are chosen and routed to from it. As opposed to a normal router which is merely passing routed traffic into the network and letting a different device handle load balancing. The idea being that different devices are used for different purposes, and separation of their functions may improve overall stability and increase flexibility of your network services.

One downside here is ECMP assumes all paths cost the same, which is ridiculous in real-world load balancing. One of your haproxies is going to get overloaded and then traffic to your site is going to intermittently suck balls as sessions stream in to both under-loaded and over-loaded boxes.

Of course, you have the same problem with round-robin DNS to load balancers, but in the case of a DR LVS load balancer for example, at least it's just starting the connection and handing it off to the appropriate proxy instead of randomly pinning sessions to specific interfaces. With DR it's the backend proxy that determines its return path; the LVS VIP isn't in the path. With LVS it can pick a destination path based on real-world load.

The other downside that you seem to gloss over with regard to scaling is the maximum of 16 ECMP addresses in the forwarding table. I'm sure we'll never need more than 16 of those, though....... (For reference: the company I used to work for had up to 23 proxies just for one application... might cause some hiccups with this set-up)

Doing maintenance on a vip address and doing maintenance on one of these bgp peers work about the same. You stop accepting new connections, let old connections expire, then take down the vip. As far as changing DNS records, instead of that you can either add a hot-spare vip with the IP of the one you want to maintain, or add the ip of that vip to an existing load balancer.

<comment><USER6> Ok, I understand what you meant now. I do disagree - it's simply doing what routers do, and has no specific knowledge or configuration for the VIP. It's simply forwarding traffic based on a destination table just like any other packet. If this was problematic in any way, your average backbone would implode - ECMP is utilized extensively to balance busy peers. Also routers already do redundancy (at least via L3) extremely robustly - so it's basically a "free" way to load balance your load balancers. You simply are not going to get the same level of performance out of a LVS/DR solution, as it's competing with very mature implementations done in silicon. We'll have to agree to disagree here.

Of course in ECMP all paths are the same - I don't see this as a downside though. Most router vendors do support ECMP weights if really needed, but there are better ways to architect things. I've run this setup with over 1500gbps of Internet-facing traffic, and never ran into a full 10g line because it was engineered properly. An in-house app that lowers my hashing inputs would probably require a different setup though, I agree.

16 ECMP is a decent number, but these days most routers I work with support 32. Some are supporting 64 now. But that's almost irrelevant, unless you're stuffing all your load balancers on a single switch. It's per-device, so you have 8 load balancers connected (and peering via BGP) to one switch, 8 another, and so on. Those then forward those routes up to the router(s) which then ECMP from there (up to 16/32 downstream switches per VIP). I've never needed more than "two levels" of this so I haven't really played with a sane configuration for more than 1024 load balancers for a single VIP (or 512 in your 16-way case). It scales more than perhaps a dozen companies in the world would need it to. Note that this explanation may sound complicated, but in a well engineered (aka not a giant L2 broadcast domain that spans the entire DC) network it just happens without you even specifically configuring for it.

Since my knowledge is dated - how do you "stop accepting new connections" with the LVS/DR model? I'm sure you can, just can't mentally model it at the moment. You need to have the VIP bound to the host in question for the current connections to complete, how do you re-route new connections to a different physical piece of gear at the same time utilizing the same VIP?

There are certainly downsides to this model as well, I don't want to pretend it's the ultimate solution. But, it's generally leaps and bounds better than any vendor trying to sell you a few million dollars of gear to do the same job. The biggest downside to ECMP based load balancing is the hash redistribution after a load balancer enters/leaves the pool. I know some router vendors support persistent hashing, but my use case didn't make this a huge problem. There are of course ways to mitigate this as well, but they get complicated.

In the end, for the scale you can achieve with this the simplicity is absolutely wonderful. It's one of those implementations you look at when you're done and say "this is beautiful" since there are no horrible-to- troubleshoot things that do ARP spoofing and other fuckery on the network to make it work. ECMP+BGP is what you get, you can traceroute, look at route tables, etc. and that displays reality with no room for confusion. No STP debugging to be found anywhere :)

</comment></comment></comment></comment></comment></comment><comment><USER13> This is a cool setup, but with the caveat that Allan stated, it forces you to think a little more about a layer that most systems people are less experienced in. The software approach is particularly useful because one could take the "healthcheck" setup and have it keep your alerting/dashboards in sync with reality (e.g. do healthcheck, fork: return exit code; POST {$hostname: 'ok'} to metric collector).

I also see that Shutterstock is actively hiring. For anyone looking, Shutterstock is a great place to work and employs some really brilliant people.

Disclaimer: Ex-Shutterstock employee

</comment><comment><USER14> I can boil water for tea in my oven, but should I?

This is totally nonstandard and it will be a nightmare to document and also difficult to hand this off to a new team mate.

<comment><USER15> This is becoming increasingly standard, and is quite easy to document.

<comment><USER16> +1

</comment></comment></comment><comment><USER17> This can shift complexity to elsewhere in your stack. A couple points to add.

Be mindful of the specific routing hardware you're using:

Announcing and withdrawing prefixes can cause the router to select new next hops (i.e. servers). This is mostly a problem with TCP and other connection oriented protocols (or even connectionless if you're expecting a client to be sticky to a server).

You may also lose the ability to do un-equal cost load balancing.

<comment><USER16> I think that's an important catch, flow breaking can happen dependent on what hardware you're using and how you swap in next-hops. It definitely requires a deep understanding of how flows are hashed.

<comment><USER6> Agreed. Also many applications for this you might just not care if a flow gets broken. Really depends on your use case.

Persistent hashing is key if you care about it though, I honestly don't know which routers do this off the top of my head :)

</comment></comment></comment><comment><USER18> You can also do this (Equal Cost MultiPath to servers) without a dynamic routing protocol but you are at the mercy of whatever health checks your top of rack switch supports.

On Cisco switches you can use a IP SLA check to monitor for DNS replies from a DNS server and then have a static route that tracks the SLA check. If your DNS server stops responding the route would be withdrawn and traffic routed away. This can happen within a few seconds. Slides from a NANOG talk about this (PDF): [http://www.nanog.org/meetings/nanog41/presentations/Kapela-l...](http://www.nanog.org/meetings/nanog41/presentations/Kapela- lightning.pdf)

</comment><comment><USER19> >"it is actually more of a load-balance per-flow since each TCP session will stick to one route rather than individual packets going to different backend servers."

This strikes me as expensive, does this mean packets no longer pass through the ASIC only side of a router and thus the software in the router has to do some of the heavy lifting, thus limiting the capacity/throughout to a mere fraction of what the route is really capable of?

disclaimer: i have only a high-level overview of router tech

</comment><comment><USER20> Been running software load balancers for over a decade. I started with LVS (Linux Virtual Server) now called ivps. Now we run HAProxy and we're looking Apache Traffic Server.

Some of the load balancers have even run BGP as called out in the OP. Nothing really fancy but enough to be interesting.

One of the coolest things I built was a Global Server Load Balancer to balance Load Balancers. We needed it initially to move data centers. It was built on top of PowerDNS and ketama hash.

</comment><comment> vhost- As someone who works with NetScalers on the regular, I have to say I love this idea. Citrix support is terrible and NetScalers are such a pain to configure. Then I see the bill and it frosts the cake.

We recently upgraded from version 9 to version 10 and it took down our production site because of some asinine undocumented rate limiting they "finally enforced" in version 10.

I'd like to play with software load balancing in the testing facility.

</comment><comment><USER21> _Even though the above says load-balance per-packet, it is actually more of a load-balance per-flow since each TCP session will stick to one route rather than individual packets going to different backend servers. As far as I can tell, the reasoning for this stems from legacy chipsets that did not support a per-flow packet distribution._

Is this not fairly risky? It's essentially relying on a bug?

</comment><comment><USER22> Yeah, I spent the last weekend configuring Cisco gear which basically I feel should have been done in software on Linux. The era of the hardware firewall slash load balancer is over as far as hardware goes. Buy a dedicated box (or two) and configure .. it's faster and more predictable/reliable.

</comment><comment><USER23> AWS's offering which is software based - [http://aws.amazon.com/elasticloadbalancing/](http://aws.amazon.com/elasticloadbalancing/)

<comment><USER24> Doesn't work for datacenters. Also is implemented with round-robin DNS to nodes (1-N, check X-Forwarded-For) in each AZ, which then handle balancing.

Also worth noting that unless you turn on cross-region balancing, if an AZ doesn't have a node in it and the RR DNS points them at that AZ, they'll be turned away. Additionally, without it you need to scale by multiples of AZs you run in otherwise you'll have unbalanced traffic.

On another note, I've always been curious if they're just abstractions around HAproxy at the per-node level.

<comment><USER25> FWIW, Amazon recently added the ability to load balance across AZ's (Still round-robin though)

<comment><USER24> Very true. Cross-AZ load balancing works quite well. I believe Amazon has said it's RR across the servers with the least connections, but degenerates to a simple RR without many nodes per AZ.</comment></comment></comment></comment>


Show HN: I built this standing desk for my cubicle - <AUTHOR> http://www.lifeisaprayer.com/articles/miscellaneous/2014/build-cubicle-standing-desk

<comment><USER2> Nice desk. I recommend buying a rip fence for your saw. It will make your cuts even more sweet.

[http://www.amazon.com/Hitachi-302691-Fence-C7SB- Circular/dp/...](http://www.amazon.com/Hitachi-302691-Fence-C7SB- Circular/dp/B0000223KC/ref=sr_1_2?ie=UTF8&qid=1388810369&sr=8-2&keywords=hitachi+c7SB2)

<comment><AUTHOR> I saw one of those at the hardware store when I was looking at blades... looks like a good poor man's substitute for a table saw, at least for some kinds of cuts. I'm sticking that on my wish list for my birthday :)</comment></comment>


MapQuest: A Symbol Of Everything That's Gone Wrong - <AUTHOR> http://www.businessinsider.com/mapquest-a-symbol-of-everything-thats-gone-wrong-2009-2

<comment><USER2> Yes, MapQuest has had an immense downturn in quality. It used to be my favorite, but Google Maps has won me over.

"I think there are a variety of things to pursue:

"1. Fix the consumer experience ASAP - make consumers love MapQuest again

"2. Find your voice in the market - maybe strive to be 'the most accurate directions on the web'."

MapQuest could still beat Google Maps in accuracy if it tried. (Google Maps doesn't know where my house is, for example.) This is a good space in which to have competition to build the best user experience.</comment>


Build an autonomous, voice-controlled, face-recognition drone for $200 - <AUTHOR> https://www.oreilly.com/ideas/how-to-build-an-autonomous-voice-controlled-face-recognizing-drone-for-200 <comment><USER2> Lots of applications! Personalize greetings of visitors to a store or museum. Find a friend in a crowd. Search for a lost child in a park.

Instead of badges at a secure facility, the drone can patrol the place and call out anyone it can't recognize.

Brave new world.

</comment><comment><USER3> Great write up! If you're not up for the read, it's worth watching the 45 second demo video at the bottom.

</comment><comment><USER4> Looking forward to the follow up walk through on how to miniaturise this to the shape of a bee.</comment>


Mode Media screws its employees on the way out - <AUTHOR> http://nypost.com/2016/09/16/mode-media-screws-its-employees-on-the-way-out/ <comment><AUTHOR> _The memo also defends its decision to keep employees in the dark — until the bitter end. "Had we notified you earlier of the possibility that these efforts were taking place and might not be successful, we believed the potential financing sources and acquirers would be unwilling to provide any financing or engage in any transaction," it said._</comment>


Looking for advice on selling domains within a website creation app - <AUTHOR>

I'm working on a rails app to help people create websites, similar to weebly, but focused on a specific niche. I want to make it as easy as possible for my customers to buy a domain name through the app. Can anyone recommend a domain registrar with a great reseller program/api? Are there any rails plugins that could speed up development of this piece of the app? Anything I should look out for? Thanks! <comment><USER2> We use Register.com, which was recommended to us by the awesome Weebly guys. The API is excellent and their developer support people are top-notch. Domains are pricey (though not as bad as it looks when you look at their retail prices), but everything works very well.

</comment><comment><USER3> enom has a pretty decent API. You do need to buy credit in advance. I think it was around $200. As people register domains through your site, the money gets deducted from your account.

<http://www.enom.com/resellers/Interfaceinfo.asp>

If you have specific questions for me, take my username and add it to gmail.com.

</comment><comment><USER4> Just avoid Network Solutions. They have an atrocious history regarding their users. If you want details, google them and see what comes up.</comment>


European Court of Justice: Cookies Need Active Consent (FR, DE; no EN?) - <AUTHOR> http://curia.europa.eu/juris/document/document.jsf?text=&docid=218462&pageIndex=0&doclang=FR&mode=req&dir=&occ=first&part=1&cid=1439205 <comment><USER2> Here, English: [http://curia.europa.eu/juris/document/document.jsf?text=&doc...](http://curia.europa.eu/juris/document/document.jsf?text=&docid=218462&pageIndex=0&doclang=EN&mode=req&dir=&occ=first&part=1&cid=1439205)

<comment><AUTHOR> Thanks. Unfortunately, I can't edit the story link to point to the English language version.

<comment><USER3> hopefully the admins maybe able to fix that</comment></comment></comment>


Revealed: The Internet's Biggest Security Hole  - <AUTHOR> http://blog.wired.com/27bstroke6/2008/08/revealed-the-in.html <comment><USER2> Huh? There are tons of operational security measures in place to protect BGP peering sessions. ISPs filter TCP to their routers. BGP itself has been aggressively filtered at the AS/prefix level since the mid 90's. If there's no new vulnerability here, how are they choosing arbitrary prefixes to redirect?

If all they're saying is that Verizon can inject arbitrary prefixes into BGP, what's the news there? That's how routing works today. The NSA isn't capturing everyone's traffic in a room in San Francisco --- AT&T is. The NSA is just a customer. Didn't we already assume telcos could do this? Presumably, it's the rule of law that protects us from this attack, not a new PKI.

</comment><comment><USER3> ::patiently waits for "hole in the Internet" to become a 4chan meme::</comment>


Ask HN: What advice would you give to a college freshman? - <AUTHOR>

I'm writing an article: https://medium.com/p/4c5df90c6c00 for my website (http://www.collegeanswerz.com/) about advice I'd give to a college freshman.
1) What advice would you give?
2) What do you think of my article? <comment><USER2> \- Live everyday like it's he/she's last. Always remember that the sole purpose to life is to enjoy it and be happy.</comment>


Gerbil – An opinionated dialect of Scheme designed for systems programming - <AUTHOR> https://github.com/vyzo/gerbil <comment><USER3> Here is some context on Racket vs Gerbil by a well-known Common-lisp Programmer, François-René Rideau : [https://fare.livejournal.com/188429.html](https://fare.livejournal.com/188429.html)

<comment><USER4> Some excellent comments there.

Looking around a bit, I came across a couple of implementations that might be of interest for those wanting to mash up "some kind of system programming" and "scheme": Bigloo (interpret / compile to executables, java byte code or experimentally dot.net) and larceny (interpret / direct compile to machine code / optionally via c):

[http://www-sop.inria.fr/mimosa/fp/Bigloo/](http://www- sop.inria.fr/mimosa/fp/Bigloo/)

[http://www.larcenists.org/](http://www.larcenists.org/)

<comment><USER5> Chicken Scheme

[https://www.call-cc.org/](https://www.call-cc.org/)

is another good one - compiles to binary, many libraries (eggs)

<comment><USER6> And can't compete in speed. But does have more libraries.

</comment></comment></comment></comment><comment><USER7> Can anyone explain why there are so many implementations of Scheme written in Scheme? What is the point of doing that (apart from learning purposes)?

I know, for example that people want Racket VM to be implemented in Chez Scheme because Chez is super fast. But what about all other implementations?

Also, as I'm currently writing R5RS/Clojure hybrid in Kotlin, can anyone please share any _simple_ standard algorithm of implementing r5rs macro system and macro expander?

The only thing I could find is [https://www.cs.indiana.edu/chezscheme/syntax- case/](https://www.cs.indiana.edu/chezscheme/syntax-case/)

<comment><USER8> I'm afraid I literally laughed out loud at your request for a _simple_ standard algorithm for r5rs macro systems. However, it's a sympathetic laugh. Macro Hygiene is still very hard, principally because (I claim) there's not yet a clean and widely accepted model for it. Matthew Flatt's "sets of scopes" model is (IMNSHO) the current leader. Time will tell whether he or anyone else comes up with a simpler and more widely accepted model. But yes: as an implementor, it "feels" _very heavy_ , and you keep thinking that there must be a simpler solution to this problem (aside from just throwing the problem out and giving up on hygiene and language composability).

<comment><USER7> Yeah, you are right. I've already implemented everything from r5rs, but macro system. And I don't know where to start, just can't wrap my head around it.

And people say that Scheme has a very minimalist design and is easy to implement...

<comment><USER9> You may find this paper helpful [1]. As noted there, syntax-rules consists really of two different parts: (1) the hygiene-preserving macro expander and (2) the pattern-matching facility. Once you wrap your head around the two pieces, it's actually pretty straightforward to implement them both. It's _very_ common to write some "low-level" macro facility as a stepping-stone to syntax-rules (such as explicit-renaming macros or syntactic closures): the implementation of syntax-rules then becomes a composition of the pattern- matcher and the low-level facility. A tutorial implementation of an appropriate pattern-matcher can be found at [2],[3].

There's lots of good reading to be found at the ReadScheme Library [4]. Most if not all of the references in [1] can be found there.

Tangentially: yeah, syntax-rules kinda flies in the face of the oft-mentioned minimalism of Scheme, but there's a reason for it: at the time syntax-rules was standardized, there was no consensus on which (if any) low-level facility should be standardized (and, really, there _still_ isn't any such consensus). The reason syntax-rules operates as a pattern-template rewriting system rather than as a procedural system is that such a system is able to _guarantee_ that the macros it produces are hygienic in a (comparably) simple and intuitive way. The idea was to standardize on a high-level system so that Scheme could have a standard macro facility, whilst leaving the low-level systems open for further exploration and experimentation. The two main contenders are still, after all these years, syntax-case and syntactic closures: the latter being easier to implement and arguably easier to grok, the former being potentially more powerful (in that an implementation of syntactic closures within syntax- case is known, but not vice-versa (last I checked)).

\--

[1]: [http://mumble.net/~jar/pubs/scheme-of-things/easy- macros.pdf](http://mumble.net/~jar/pubs/scheme-of-things/easy-macros.pdf)

[2]: [http://blog.theincredibleholk.org/blog/2013/02/11/matching-p...](http://blog.theincredibleholk.org/blog/2013/02/11/matching- patterns-with-scheme/)

[3]: [http://blog.theincredibleholk.org/blog/2013/02/12/patterns-w...](http://blog.theincredibleholk.org/blog/2013/02/12/patterns- with-ellipses/)

[4]: [http://library.readscheme.org/](http://library.readscheme.org/)

<comment><USER7> Hey, thank you so much for that! Haven't seen some of those papers. Looks promising. Cheers!

</comment></comment></comment></comment></comment><comment><USER10> [https://ecraven.github.io/r7rs-benchmarks/](https://ecraven.github.io/r7rs- benchmarks/) recent addition to the charts, and is doing quite well.

<comment><USER11> Yes, Chez & Stalin are the fastest in these benchmarks, which seems to match what the community usually answers when asked about quick implementations.

Sadly Stalin is unmaintained. Its whole program optimization techniques were really advanced. I remember it even got my ivory-tower professors, who were big in the static analysis field, excited.

Now, a tricky question. I'm mostly unfamiliar with Scheme for writing real- world code. Will the ongoing merger of Chez with Racket make the latter a clear winner in the Scheme camp? How are libraries and FFI?

A problem with Scheme is excessive fragmentation. Having a clear winner would be cool for library support. Racket is great due to multiple paradigms and DSLs [1]. I hope it eventually becomes a very practical Lisp with all Mozart/Oz semantic goodies.

I am trying to do all my projects in a Lisp. Lately this is either Clojure or SBCL. Both have good libraries and decent FFIs. Clasp (Common Lisp on LLVM) [2] has gotten me excited, as good interfacing with C++ will be great to access a lot of quick numerics code.

[1] [https://beautifulracket.com/appendix/domain-specific- languag...](https://beautifulracket.com/appendix/domain-specific- languages.html)

[2] [https://github.com/drmeister/clasp](https://github.com/drmeister/clasp)

<comment><USER6> R7RS ftw.

<comment><USER12> I feel that not standardising a low level FFI for r7rs large is a mistake. They should at least recommend some reasonably low level stuff that can be used to build abstractions.

<comment><USER5> Agree with the sentiment, but at the same time understand the rationale - <USER2> implies alot of GC/memory internal interfacing stuff, and to some extent expects a c-based implementation (not e.g. JVM,CLR,etc).

If r7 & library interface are widely adopted, some flavors of FFI could evolve within the library/srfi process and gradually become defacto standards..

Probably hoping too much.. but in any event..

<comment><USER13> That didn't happen with R5RS, why would it happen now?

Face it, by refusing to be opinionated they ensured Scheme will remain a toy.

<comment><USER12> Because r7rs large is an effort that probably is bigger than common lisp. It is very much not a toy language.

<comment><USER13> If it doesn't have an ffi it will remain a toy that relies on implementations to make it useful.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> (Mostly just hoisting Noel Welsh's comment to the top level) How will the expected upcoming move of Racket to the use of Chez affect the comparison between Gerbil and Racket? I'm trying not to set my expectations too high. BTW: RacketCon! This weekend!

<comment><USER14> I am super excited for Chez. If we see the speed we are look at a possible doubling in speed.

There is certainly a reason why he has stuck with Gambit and not my beloved Racket. This seems perfect for using Racket's macros. I just don't know Gambit's Macros well enough to compare them to Racket.

Here is the Bench Marks for Chez, Racket and Gambit [https://ecraven.github.io/r7rs-benchmarks/](https://ecraven.github.io/r7rs- benchmarks/)

<comment><USER10> Gerbil is there as well.

</comment></comment></comment><comment><USER15> Already a handful of "how does this compare...?" postings but the standard for a systems-programming scheme is scsh, a find abstraction layer over POSIX dating from the nineties.

[https://scsh.net/](https://scsh.net/)

<comment><USER16> I think maybe there is a different interpretation of the words "systems programming"?

I took it to mean a scheme used for developing close to the hardware programs, where as maybe others are taking it to mean a language which can be used to write common scripts and tools used by Systems Engineers.

<comment><USER3> Close to the hardware can also be done with scheme and indeed has been done in the past. Look at all the smartcards, running a JVM. For scheme, it just requires some engineering effort and there is not much demand for it. Btw, there is picobit([https://github.com/stamourv/picobit](https://github.com/stamourv/picobit)) which can be run on micro-controllers.

<comment><USER17> PreScheme serves the same sort of role within the Scheme48 ecosystem. It's been a while since I've looked at Scheme48, but if I remember right, the VM is written in PreScheme, which is usually compiled down to either C or machine code. However... you can also run PreScheme code within Scheme48, so that it's possible to test/update the VM semantics without actually compiling down to C for each dev cycle.

</comment></comment></comment></comment><comment><USER18> How does this compare to prescheme from scheme48?

[http://www.s48.org](http://www.s48.org)

[https://en.wikipedia.org/wiki/PreScheme](https://en.wikipedia.org/wiki/PreScheme)

</comment><comment><USER6> The link should have been to [http://cons.io](http://cons.io) See Freenode #gerbil-scheme for active development and help. So far it is very nice and about twice as fast as sbcl.

</comment><comment><USER19> For the uninitiated, how does this compare to guile?

<comment><USER20> Based on Gambit-C: A scheme to C (or javascript... and I think there may be other targets) compiler. Comes with a Scheme->target compiler, then compile that code, and deploy. So, you write in Scheme, and deliver in C...

Comes with an interpreter. Supports macros, etc. and Gambit supports easy C FFI, option for infix syntax, massive threading, full numeric tower

Easy to hack on (in my opinion), and performs very well. Gambit-C has been my "goto" Scheme for the past decade.

Could be used as an "extension language", but not its main purpose. Mostly, a way to deliver Scheme code that runs fast.

Racket has a better IDE -- but I like the simplicity of Gambit-C.

<comment><USER10> Gambit is amazing. And gerbil brings a lot of batteries to it.

</comment></comment></comment><comment><USER21> Why do people keep thinking they can use a garbage-collected language for "systems programming"?

<comment><USER22> I always wonder exactly what kind of "systems" these people are programming, clearly not any with real-time constraints. It almost seems like "systems programming" got co-opted to mean "not interpreted". At least that's what I noticed with Go's marketing.

There are a few HFT firms, most notably Virtu, that use Java. But my understanding from having interviewed people that worked there is that it's so convoluted to avoid GC pauses that you might as well be using C++.

<comment><USER23> It seems to md that systems (with a 's') programming was a term lntroduced by Go, and I always understood it has something with a wider scope than system programming - actually "intermediate level" applications for which a GC is acceptable both time-wise and space-wise. Yet for system (without 's') programmers (kernel, driver or embedded systems devs for instance) a GC is still a no-go.

<comment><USER5> > lntroduced by Go

no. further popularized perhaps, but no.

perhaps a shorthand for 'distributed systems' programming, idk.

Definately heard the phrase applied in this area 10+ years ago, by people who had been around for 10+ years (see also go authors)

</comment></comment></comment></comment><comment><USER24> Can anyone briefly summarize the points of debate and optimization? I'm not familiar with the competing kit, and the primary goals.</comment>


Why I use Firefox - <AUTHOR> http://mattryall.net/blog/2012/06/why-i-use-firefox

<comment><USER2> > _The most infuriating aspect of Chrome's search suggestions is that they > change after you've stopped typing._

That!</comment>


The Facebook Reckoning - <AUTHOR> http://stratechery.com/2015/facebook-reckoning/

<comment><USER2> I wish the publishers and their social network enablers luck. As a member of their target demographic, I can't help but feel they're in deep trouble. I quit Facebook when I realized most of the content it showed me was worthless. I also stopped visiting his example site (the Verge) when they abandoned their original purpose of covering technology news.

<comment><USER3> I don't even check Facebook anymore. I use their API (until the personal version goes away with 2.0) to grab photos I'm in or friend's pictures, and push it into Flickr and S3 for backups.

If I want news, I'm going to Google News, reddit.com/r/worldnews, or Hacker News.

<comment><USER4> Just to point out that some people are the opposite: I gave up on reddit (which I now never look at at all: I just can't stand the people there) and _started_ using Facebook as a news source. I follow a number of news sources I like, and many of my friends share interesting articles on the subjects they know a lot about; I then have conversations with people I know about the content of the articles, as opposed to random people, which I find much more valuable, as we slowly build up a shared knowledge of what is actually happening (as opposed to constantly feeling like I am talking with a blank slate: on Hacker News I finally decided I now just "fire and forget" comments, as otherwise it is just too frustrating to have to have the same conversations over and over again with people about complicated corner cases of subtle topics).

<comment><USER3> A suggestion: Use reddit for the links, and completely ignore the comments.

<comment><USER5> Another suggestion: Stop thinking of Reddit as a community and start thinking it as a bunch of different communities. The average quality of discussion on mainstream subreddits like worldnews is pretty low, but I find it to be quite good for niche communities (eg. the games I like to play).

<comment><USER6> this is the answer. i can't believe it when friends tell me they don't log in. /r/all is just brutal.

</comment></comment></comment></comment></comment></comment><comment><USER7> The author implies that publishers might take the deal because Facebook is far more powerful than they are--they have no choice. I don't think that is true at all.

If you're on Facebook, look through your feed. How many of the items are personal information someone has entered into Facebook (status updates, photos, etc), and how many are published articles that someone has shared? In most feeds it is a healthy mix. A quick survey of my own showed about a 2:1 ratio in favor of published content.

Shared articles are an inherent part of the value of Facebook today, which means Facebook has no leverage. They can't hurt publishers without also hurting themselves.

Ask yourself this: if Facebook has all the power, why are they bothering to offer to host published content on their servers? Why not stick with the status quo, where the publishers do all the work and Facebook just serves ads?

In fact, Facebook is creating this new service to help Facebook. Because, like publishers, Facebook also faces growing competition for attention. A world in which publishers self-host their own content is a world in which Facebook is just a middleman...and middlemen are easily replaced on the Internet. Readers can share links on Snapchat just as easily as they can on Facebook.

Content is still king. People want interesting things to read. Publishers, not Facebook, are the companies who give them that. Facebook has to become a publisher itself in order to compete for the long term--just like Netflix is now creating its own series.

<comment><USER8> 1) If the same user visits facebook.com and some newspaper site, let's say mercurynews.com, advertisers will pay premium for the facebook.com user since Facebook most likely will show that user the most relevant ad. San Jose Mercury News does not have access to the same trove of data, and their best hope is to display some ads from San Jose realtors in an effort to do geotargeting, or just go the retargeting route and show that user an ad for headphones he checked out on Newegg a few days ago.

At some point the premium for highly targeted ads is so rich that even with Facebook's cut of revenue share, the media company will make more money off the same content, than displaying it on a standalone Web page of their own.

2) Some mobile operators in some locations whitelist (or offer at a special subsidized rate) content from facebook.com. Some publishers might not be indifferent to those extra eyeballs (and potential rev share).

3) Publishers faced with same dilemma with Google News excerpt. Want to exclude your own headlines from excerpts? Go ahead, there are 15,000 competing news outlets around the world waiting in line to receive traffic from Google News. The publishing (especially news) world is becoming commoditized, and online aggregators have learned to play the content generators against one another.</comment></comment>


Show HN: Discover and learn more about music you listen to - <AUTHOR> https://lyricly.net <comment><AUTHOR> A simple interface to watch music videos, read lyrics, discover new music and learn more about the music you listen to. It's a prototype, but wanted to share!</comment>


A Mental Model for Video - <AUTHOR> https://tjkrusinski.com/articles/general/mental_model <comment><USER2> Nice article and as others have commented, love the simple and clear aesthetic of the gifs within!

This is just the entrance to that rabbit hole though; for anyone interested in continuing the journey all the way down I'd highly recommend: [https://github.com/leandromoreira/digital_video_introduction](https://github.com/leandromoreira/digital_video_introduction).

</comment><comment><USER3> The image that tries to illustrate how frames per seconds affects motion is wrong. At least in my browser, it is crossfading between images in the 30 and 15Hz illustrations. That's not how video works: you show individual images sequentially with no mixing. Simplistic attempts to "improve" motion are universally worse than leaving it alone.

<comment><USER4> _> That's not how video works: you show individual images sequentially with no mixing_

In the DVD era, PAL/NTSC used to interlace frames, which is a ghetto crossfade. I have no idea how this works in the Bluray era.

<comment><USER3> Interlace does not crossfade either :-)

The purpose of interlace is to double the frames per second and capture motion better. The downside is some vertical resolution loss and some odd, but rarely seen, visual artifacts.

<comment><USER4> Doesn't interlacing alternate even lines from one frame with the odd lines from the next, thus showing 50% of each image? Which would make it a ghetto crossfade as described.

<comment><USER3> Not really. On a CRT monitor, the odd lines fade before the even lines begin lighting up, and they are on different physical locations on the monitor. Recombination of the two images to improve resolution occurs entirely in the human vision processing system.

It's worth noting that this takes advantage of the human vision system's features. When an object moves quickly across your vision (for example, when turning your eyes quickly down a field), there is resolution loss that your brain makes up for, And when focusing on a still object, vision allows you to see more resolution.

So when a player runs across the camera view, your brain correctly processes the lower resolution and higher motion resolution. And when a player is still, your brain correctly processes the improved resolution.

</comment></comment></comment></comment></comment><comment><USER5> Amazing gifs! Just scanning it is enough to understand it.

<comment><USER6> Came here to make the same comment. Really love the aesthetics of this page.

</comment></comment><comment><USER7> This is great article but the contrast on the body text is horrible on the eyes (ironically)

<comment><USER8> It's 92% contrast, and passes Web Content Accessibility Guidelines at the highest AAA standard for contrast. What more do you want?

[https://contrast-ratio.com/#black-on- rgb%28246%2C%20246%2C%2...](https://contrast-ratio.com/#black-on- rgb%28246%2C%20246%2C%20246%29)

<comment><USER9> Not OP, but at default zoom level, the text is too thin - most of the pixels aren't actually black - they are subpixels smeared across two columns.

<comment><USER10> Yup - for me, each ascender of an "h" is a line of red pixels next to a line of blue, no actual black at any point.

Removing the unnecessary font-weight CSS makes it better. Personally I also find it more readable if I remove the font specification and let it fall back to Serif, and it also takes up less vertical space, but that's a matter of personal taste. Not tried it on mobile.

<comment><USER8> > Yup - for me, each ascender of an "h" is a line of red pixels next to a line > of blue, no actual black at any point.

Each of your pixels is made of a tiny red, green, and blue pixels. The 'black pixel' is the gap between the red and blue sub-pixels (the green being off is black.)</comment></comment></comment></comment></comment>


Why Facebook Will Never Fix Facebook - <AUTHOR> https://www.buzzfeednews.com/article/charliewarzel/facebook-time-well-spent-dashboard-wont-fix-problems <comment><USER2> There's something ironic about Buzzfeed advocating for content distribution reforms on Facebook. Buzzfeed owes its entire existence to gaming Facebook's engagement algorithm via clickbait.

What if the fix Buzzfeed is asking for involves less distribution for outlets like Buzzfeed? I'd support that.

<comment><USER3> The irony isn't lost on me, but it does not mean the article is wrong in some way because it is published by Buzzfeed News.

If anything, I would be inclined to trust Buzzfeed's expertise in garnering views, clicks and ad revenue from Facebook when it comes to them criticizing the system they thrive in.

Additionally, if those changes drive less traffic to Buzzfeed, it will likely hurt publishers across the board and they will all have to change to survive. Perhaps that could be in the general best interest.

<comment><USER2> I agree it's not wrong. While this article is light on specific proposals, I would likely agree with the author on many reforms.

But if those reforms actually happen, I bet the executive team at Buzzfeed won't be happy

</comment></comment></comment><comment><USER4> _Perhaps this shouldn't come as a surprise. Facebook, after all, is a platform historically powered by prioritizing and rewarding a very simplistic idea of user engagement. You react to something, or you don't. And if you do, you'll see more of the same._

This again. It's another variant on the themes I discuss in "Is there an actual Facebook crisis, or media narrative about Facebook crisis?" [https://jakeseliger.com/2018/11/14/is-there-an-actual- facebo...](https://jakeseliger.com/2018/11/14/is-there-an-actual-facebook- crisis-or-media-narrative-about-facebook-crisis/). Facebook will change when users quit or dramatically cut back using it. Until then, actual users are telling Facebook, "Full steam ahead."

<comment><USER5> The users are not the only driver of change. Government regulation is very likely to happen at this point, in both the US and the EU.

<comment><USER6> Content regulation is very unlikely in the US given the Court's near- absolutist First Amendment jurisprudence. Regulation of advertising and privacy is more plausible, but something like the GDPR is a constitutional and political non-starter in America.

<comment><USER7> GDPR comprises many facets many of which are completely constitutional.

And it maybe a political non-starter federally but all it takes is a few states in particular New York and California to impose restrictions and you will have the same effect.

<comment><USER5> Especially considering Facebook is based in California, which is controlled by the Democratic party. Many people in that party blame Facebook for helping Trump win the 2016 election.

</comment></comment></comment></comment></comment><comment><USER7> I don't understand why Facebook is being singled out here.

Because the underlying issue is time spent on the internet as a whole. Each of the negatives that can be attributed to Facebook apply equally to sites like Youtube, Instagram, Snapchat, Twitter etc. And ultimately Facebook is just a website that exposes humanities's neuroses and problems to itself.

And I still love the hypocrisy of this coming from Buzzfeed whose entire existence centres around driving traffic through jealousy, fake news, clickbait etc.

<comment><USER8> > _I don 't understand why Facebook is being singled out here._

From where I sit, they're not. Facebook is the canary in the coal mine because of its current outsized influence, but you can find similar pieces about all of the services you mentioned.

Here's an example, but I ran across a bunch more just looking for this one: [https://www.wired.co.uk/article/youtube-for-kids-videos- prob...](https://www.wired.co.uk/article/youtube-for-kids-videos-problems- algorithm-recommend)

</comment></comment><comment><USER9> The real question is: What is the fix? People want social networks but they are opposed to giving their data away or paying for the service. How does a company survive without revenue?

The least path of resistance is giving the service free to users and selling their data. The only fix comes from the users, not Facebook, and that's to stop using the service. Facebook will continue to monetize the user's data as part of their business model. There is no fix from their side.

<comment><USER10> Do we know people wouldn't want to pay? I know Facebook never asked me if I would. People have a track record of paying for services like television. They _might_ pay for social networks if given a chance. Charge me two dollars a month, make all the creepy spying illegal, and the remnants of Facebook are useful (staying connected to friends, etc.).

</comment></comment><comment><USER11> There's also probably a lack of dogfooding. "Oh yeah sure it's a good idea to publically display people's likes/comments/everything..."

Not to say they haven't also been corrupt to increase engagement because they have...

</comment><comment><USER12> Interestingly, the page <title> is different: "Why Mark Zuckerberg Can't Fix Facebook".

</comment><comment><USER13> huh? facebook isn't broken. it's a monetization platform, not a force for good. they _will_ DTRT to keep money rolling in. this talk about addictive patterns is a sideshow.

</comment><comment><USER14> Tldr: _the incentives that govern the platform (gamifying and monetizing attention, as well as prioritizing clicks, shares, ads, and money over quality of information) will largely remain the same._

<comment><USER7> Except this isn't true at all.

Facebook recently changed their algorithm to deliberately focus on the quality of engagement.

It had a material effect on marketers and news organisations so it was pretty significant.</comment></comment>