"We are left with no choice but to shut down Flytenow" - <AUTHOR> http://blog.flytenow.com/the-beginning-of-the-end <comment><USER2> They're shutting down. Good.

They weren't "innovating". They were trying to run an air taxi service without meeting the pilot training requirements and aircraft equipment safety requirements for carrying passengers. Private pilots should not be carrying passengers for money - the accident rates at the low end of general aviation are too high.

(Never go flying to a destination with someone who isn't IFR qualified. The weather can always change.)

This was on YC a few days ago, at [https://news.ycombinator.com/item?id=10769333](https://news.ycombinator.com/item?id=10769333)

<comment><USER3> > Never go flying to a destination with someone who isn't IFR qualified. The > weather can always change.

This is an absurd statement.

<comment><USER4> That's not an absurd statement. If you're trying to fly "to a destination" and don't have an IFR-qualified pilot, and the weather changes, you'll be in IFR conditions without someone who doesn't have the training to keep you in the air in those conditions.

If your destination doesn't matter, or isn't set in stone, then a non-IFR pilot is free to choose to fly somewhere else, where the weather is better, or land at the closest airport.

For those who don't know: IFR means "Instrument Flight Rules" or flying by instruments only. An IFR-qualified pilot is allowed to fly in conditions where he/she cannot see out of the aircraft, such as at night or through low- visibility weather.

<comment><USER3> Proper planning and weather monitoring makes VFR perfectly acceptable, to say never to fly anywhere without an IFR qualified pilot is absurd. It's akin to saying jr. devs should never write code that will be run in production.

<comment><USER5> More like "Junior devs should not be shipping nonreviewed code into lifesaving medical devices."

</comment></comment></comment></comment></comment><comment><USER6> _In the Opinion of the Court, Judge Pillard held that pilots sharing expenses on Flytenow were engaged in common carriage, making them the only common carriers (i.e., commercial airliners) in history to not seek a profit._

Profit seeking is not a requirement to be considered a common carrier. The FAA and the Court are quite clear in that all that is required for a common carrier classification is "(1) a holding out of a willingness to (2) transport persons or property (3) from place to place (4) for compensation" and that all four are satisfied by Flytenow. Posting an itinerary to the website qualifies as holding out, the flight is transportation, the individuals go from place to place, and the cost sharing is compensation. Flytenow's response to this is just needling and attempting to spin.

 _The current state of the law is extremely deferential to regulatory actions, at the expense of innovation. The Court relied on that regulatory deference, and the result is less choice for consumers, and less innovation in general aviation._

This is true. And it's for good reason. Flying is dangerous and should not be subject to the race to the bottom and rent-seeking behavior that is the "sharing economy".

<comment><USER7> What do you mean by rent-seeking behavior?

<comment><USER8> Companies that use their property – distribution networks, in this case – to profit from others without providing wealth creation back to society.

AKA Landlords, Realtors®, Middlemen, Licensures etc.

<comment><USER7> I know you're not OP and I understand this is the definition of rent-seeking behavior, but your examples are empirically and theoretically invalid. Landlords (renters of capital) and middlemen provide _huge_ benefits to society.

Of course, this doesn't mean that it's impossible for a landlord it middleman to be a rent-seeker, but they would need to pursue some type of artificial benefit/restriction on others in order for this to be the case. This does happen often, but I have no idea why OP would suggest rent-seeking is the primary activity of sharing economy companies. One of Uber's greatest accomplishments has been significantly weakening the rent-seeking taxi industry.

<comment><USER8> I should have clarified by "Landlords" and "Middlemen" I meant the groups that lobby local governments to restrict zoning and growth, so they can retrieve higher profits.

Uber is very much within the "virtuous cycle" – but they may also become rent seekers via monopolization of private transportation.

</comment></comment></comment></comment></comment><comment><USER9> Ride sharing in aviation is perfectly legal, the FAA isn't interested in the commercialization of ride sharing in aviation. However there have been websites offering that for many years, none are popular. Whoever thought of suing the FAA for something that is already legal... probably listening to an investor who didn't want to invest unless if it was given the FAA seal of approval...

[http://www.risingup.com/fars/info/part61-113-FAR.shtml](http://www.risingup.com/fars/info/part61-113-FAR.shtml) Sec. 61.113(1)(c)

This is a good article on this [http://speednews.com/article/6966](http://speednews.com/article/6966) (2011) essentially two people have to be intending to go to the same destination, the passenger can only pay half the costs and the pilot has to pay the other half since he/she is going to the same destination anyway.

<comment><USER10> Read their older blog posts. It was 16 months ago that the FAA told them that, effectively, they couldn't operate under their model.

</comment></comment><comment><USER11> Wow I never knew such a thing existed. I wish I could sign up, this is amazing!

</comment><comment><USER12> Customary, I'm probably going to get downvoted for this but..

What part of there being a "transaction" makes the flight inherently unsafe? Does a "transaction" imply that the private pilot will be flying more to make money, therefore be less safe? If that's the case shouldn't there just be a limitation on the number of flights private pilots can do?

Are private pilots not supposed to take passengers with them? If that's the case, maybe there can be additional safety/certification requirements imposed for private pilots (without requiring them needing to learn all the ins and outs of commercial 747 flights).

A blanket ban just because there is money involved seems silly.

<comment><USER13> I don't see the blanket ban...dozens of carriers operate in the US daily.

<comment><USER12> Blanket ban on private pilots having paying passengers.

</comment></comment></comment><comment><USER14> What about in other countries? It's perplexing to me why the green lobby isn't up in arms about hostility to Uber et al.

Services like Uberhop save energy, reduce traffic, pollution, cost. There are no downsides.

<comment><USER15> The competition here is not everyone flying their personal plane (with "sharing" introduced to maximize efficiency), it's commercial aviation, whose cost structure is already more efficient compared to general aviation.

It's as if Uberhop was launched in a market that used predominantly city buses.

</comment></comment><comment><USER16> Back when I ran a consulting/web dev biz I had a client who came to me with an idea like this. I didn't like the sounds of it and didn't pursue it. Glad I didn't. There's a myriad of technical legal issues to go through. Uber ("rideshare") vs taxis are majorly different from Flytenow ("flightshare") vs commercial airlines.

The FAA will never allow "flightsharing" on the commercialism that Uber is at. It's simply too dangerous.

<comment><USER17> I don't buy the "dangerous" argument at all. Show me the danger.

edit: I'm referring to the danger of adding a passenger to an _already_ scheduled flight by a private pilot, which is what I thought this flytenow service was about. (Assuming there's not a weight issue and someone isn't trying to intentionally crash the plane...)

<comment><USER18> Fatal Accident Rate (Table 5 [1]):

      Commercial Aviation (per 100,000 flight hours): 0.022
      General Aviation (per 100,000 flight hours): 1.11
      Motor Vehicle: (per 100 million miles): 1.5

General Aviation (which Flytenow falls under) has a fatality rate 2 orders of magnitude greater than commercial airlines (Delta, Southwest, Skywest, etc)

1: [http://enhs.umn.edu/current/injuryprevent/aviation/magnitude...](http://enhs.umn.edu/current/injuryprevent/aviation/magnitude.html)

<comment><USER19> You're misreporting that data.

From the same table the overall risk of death (Fatality Rate) is the same for both forms of flying:

       Fatality Rate (Commercial Aviation): 1.9

       Fatality Rate (General Aviation): 1.91

The reason the "Fatal Accident Rate" is so different is because when a jetliner crashes, dozens or hundreds of people die. So a single accident is very deadly.

<comment><USER20> No, you're mis-interpreting the data. The commercial aviation record is as low as it is _in spite of_ the large number of people on board. So the chances of accidents in general aviation are many times higher for any given flight using the one or the other mode.

In other words - and to make it really simple - if you are given the choice of going with a commercial carrier or with a general aviation craft to the same destination in a single flight then you would do wise to take the commercial carrier.

<comment><USER19> You're correct in the narrow sense (a commercial flight is less likely to crash) but wrong on the big picture (a commercial flight is safer for passengers).

Here's the thing: we don't care about the number of crashed airplanes, we care about crashed people (the number of deaths). And the number of deaths is about the same per passenger mile flown.

Passengers are no safer in either form of flying, according to this source.

<comment><USER21> Why would we care about the number of crashed people? You said it yourself, the number of deaths for commercial flight is higher because there are more people on board. If I'm on a plane, I don't care about how likely I am to be one of the people who died in airline crashes this year, I care about how likely it is that this plane is going to crash.

<comment><USER19> I understand it's not intuitive, but the statistics are clear: If you have 100 times more people on commercial flights, then the flights would have to be 100 times less likely to crash to have the same risk to each passenger.

Let's break this into two specific questions (assuming the source is correct):

1) How likely is it that a specific flight will crash?

Answer: it's more likely that a private flight will crash than a commercial flight.

2) How likely is it that I will die on a private vs commercial flight?

Answer: your likelihood of dying is the roughly the same for both private and commercial flights.

<comment><USER21> I'm not sure you're reading the replies here. Nobody is arguing that your likelihood of dying is significantly higher on a private flight. We're arguing that the thing you should care about is whether the specific flight that you are on will crash, which has a significantly higher likelihood in general aviation.

<comment><USER22> _Nobody is arguing that your likelihood of dying is significantly higher on a private flight._

I might be wrong, but this is indeed what I am arguing, and I think 'jacquesm' is saying the same. 'nostromo' says _your likelihood of dying is the roughly the same for both private and commercial flights_ , and I don't think this is true if the metric is getting from point A to point B without dying.

If my memory of previous research is correct, a passenger on a private plane has approximately the same risk per mile travelled as a passenger on a motorcycle[1], which is something much greater than the risk per passenger mile of a normal automobile, which is in turn something significantly riskier than traveling the same number of miles by a commercial jet.[2]

 _We 're arguing that the thing you should care about is whether the specific flight that you are on will crash, which has a significantly higher likelihood in general aviation._

Yes, each flight you take in a private plane is significantly more likely to end in a crash than each flight you take on a commercial airline. Your chances of being killed if the plane is involved in an accident are greater in a private plane (fatalities per accident are greater on the commercial flight, but risk to each passenger is lower). Therefore, your chances of dying per flight taken are greater in private flight than in commercial aviation. Is this the same as what you are saying?

[1] Edit: Found a seeming good source at [http://www.nianet.org/ODM/presentations/Overview%20SVO%20Ken...](http://www.nianet.org/ODM/presentations/Overview%20SVO%20Ken%20Goodrich%20and%20Mark%20Moore.pdf), page 8. Looks like I was slightly wrong to say that motorcycles and GA are comparable risks. If we trust their estimates, motorcycles are about 2x more dangerous per hour than GA, and 3x more dangerous per mile.

[2] I put the specific numbers from in my other comment here: [https://news.ycombinator.com/item?id=10781106](https://news.ycombinator.com/item?id=10781106)

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER23> Called it 549 days ago: [https://news.ycombinator.com/item?id=7923297](https://news.ycombinator.com/item?id=7923297)

</comment><comment><USER10> Did Flytenow ever even "make it off the ground"? Was there ever a single flight? I'm not sure that there was.

</comment><comment><USER24> Other sites offering the same service for more than 5 years on USA that now have to shutdown:

[http://www.skypool.com/](http://www.skypool.com/) (Since 1999- 16 years) [http://www.pilotsharetheride.com/](http://www.pilotsharetheride.com/)

</comment><comment><USER25> Ohhh, that mean old grinch Mr. Government. Always telling entrepreneurs their dumb ideas are dumb.

</comment><comment><USER26> Consenting people, using private property in compliance with the law. Why exactly is this banned, other than it cutting into the pocketbooks of carriers?

<comment><USER27> It's banned because the general public cannot assess the risk of private flights, and the flight is not under an AOC.

The FAA maybe a lot of things, but they're 100% right about this.

<comment><USER28> The general public cannot assess the risk of riding in a Taxi, bus, uber, etc. any better than taking a flight with a private pilot.

<comment><USER29> Near-100% of the public owns a driver's license. 100% of the public has logged thousands of hours on the road, be it at the wheel, or in the passenger's seat. If you feel like your life is in danger, you can always ask your driver to stop, and let you out.

Near-100% of the public wouldn't have the first clue about what it takes to fly a two-seater aircraft, how dangerous the conditions they are flying in are, and most definitely can't ask the pilot to pull over half-way through the flight.

Your claim is farcical.

<comment><USER28> Not a lot of of time to respond, but my first observations about your comment are:

1) How many pilots are actually unlicensed? I suspect given the relative difficulty of getting access to aircraft vs. cars that the more pilots in the air are of greater competence than the average driver by a long shot. Moreover, I'll bet that the examination standards for pilot licenses are somewhat more rigorous than the average drivers license.

2) Have you failed to assess that the greatest likely danger in a road trip is not the driver giving you a ride, but other drivers on the road? I use to commute between LA and the Bay Area every week for a couple of years. I guarantee you that not everyone you encounter is sane or competent to be on the road. And yeah, in the middle of nowhere on I-5 (or better still, I-15 between LA and Los Vegas close to Death Valley) you can ask to get out of the car if you are uncomfortable with the way the person is driving... but get out in the middle of nowhere and then what?

The complexity of the driving environment and the relatively low barrier to entry for cars don't make the risk assessment as simple as you would like to believe.

Finally, unless you are a pilot (you may be) seems like you're merely a member of that ignorant general public that can't make such assessments. If that's the case, how can you possibly be in a position to make a judgement? Or are you just better than everyone else and it's your paternalistic instincts taking over?

EDIT: To be clear... I am not arguing that general aviation is safer than driving (last I looked it's not). But I am arguing that the risk assessment may not be a difficult as people here are making it out to be, especially in light of the complexity of driving on the road.</comment></comment></comment></comment></comment>


Ask HN: Why aren't recipes written as timelines? - <AUTHOR>

Recipes would be a whole lot easier to follow if their notation was in timeline form.<p>Why instead are they written as narrative? <comment><USER2> [http://www.cookingforengineers.com/](http://www.cookingforengineers.com/) has an interesting condensed recipe card format: [http://www.cookingforengineers.com/recipe/89/Cheesecake- Plai...](http://www.cookingforengineers.com/recipe/89/Cheesecake-Plain-New- York-Style/trn)

</comment><comment><USER3> Gracipe was made by two students from my Data Vis course at the TU Delft: [http://www.gracipe.com/](http://www.gracipe.com/) (it only started during my course, their app is far more extensive now)

They parse and show recipes as icon-oriented graphical timelines.

</comment><comment><USER4> There is a style that writes recipes as a list of steps and lists ingredients for each; AFAIK it's a common format in Germany and it's also used in the original "Mastering the Art of French Cooking" book. Otherwise recipes came from handwritten notes. Quite often the general process is known by heart, described elsewhere, or deducible, but the exact formula have to be written down. It's also convenient to have the list of products (and maybe equipment) in one place so you can make a shopping list. And the paper medium can only have a single representation, so it has to be a compromise :) But electronic recipes should be more fluid.

</comment><comment><USER5> Just off the top of my head, being narrative gives greater scope for the cook to put their stamp on the recipe.

Like the Bible, much of the 'recipe' is open to interpretation - there is the recipe-maker's original intentions and then there is everyone's personal take on it.

In timeline form I believe there would be the propensity to see the recipe as a formula rather than a guide open for modification and exploration.

</comment><comment><USER6> I use a service called cooksmarts.com (weekly recipes so you only have to shop and cook, very little planning) and they use an optimized timeline. I.e. everyone takes a different amount of time to do certain things so there is room for mistakes.

That's my best guess for why recipes don't have a timelines. You're also probably cooking more than one thing at a time I.e. entree, sides, desert etc. If each of those had their own timelines you'd be screwed.

</comment><comment><USER7> Because no cooking equipment can deliver energy at a known precise rate. That means you cannot say "t=0 start frying meat. t=300s add liquid" because you might cook the meat faster or slower depending on equipment and what shape and size the meat was cut into.

Most recipes I have read are written as a less precise timeline though with exact times traded for estimates relative to the previous step or "until done"

</comment><comment><USER8> I imagine after you detail the events on a timeline according to the recipe, you'd likely end up with a colorful list of instructions and little else - some events would also be quite detailed and - in many recipes - optional. I'd much rather see a recipe depicted in a mechanical 'exploded view' format though - but maybe that's just me.

</comment><comment><USER9> Recipes are usually written chronolically

So you mean timeline the visual sense?

</comment><comment><USER10> Tradition, probably</comment>


Ask HN: How to train my students to be Unicorns - <AUTHOR> I have a free dev bootcamp for exceptional people who have a dream of becoming a software developer. I only have the front end curriculum ready but wanted to get some input on what I might add for front end or back end. I don't have most of the books listed but the learning is a combination of the best books with the best online videos.
Thank you so much for you input. I really appreciate it.
1. Setup Workstation
2. Learn Linux or OSX at the command line
3. Learn vim
4. Learn Git
5. HTML
6. CSS
CSS Frameworks
L. Bootstrap Framework 	M. Foundation Framework
7. Javascript
Javascript Frameworks  	A. Angular.js 	B. Backbone.js 	C. Knockout.js
8. JQuery
9. Pro Developer Tools 	a. Yeoman 	b. Grunt 	c. Gulp 	d. Bower 	e. Coggle 	f. Balsamiq
BACK END
10. Node.js
11. Databases and Caching Technologies
12. Webserver, reverse proxies and load balancers <comment><USER3> Several people have written nice supporting responses with good suggestions. One person (that I have seen so far) has written a testing question. I don't want to discourage you, but sometimes with ventures like this, I think testing questions can be more valuable than the good answers to your original question.

In that spirit, I hope you will allow me to explore some questions that might help you narrow down what you are doing. To reiterate what the other poster asked, "What do you mean by Unicorn" and I will tack on, "and why is that the goal?"

You say you want to target "exceptional people". What do you mean by that? If a person is "exceptional", why do they need your help? What gap are you filling?

Finally doing this kind of thing is often difficult and time consuming. What is your motivation for starting? How will you measure your success? How will you sustain your motivation over time?

Now some advice: Like some others, I have started many projects. Some have lasted a weekend. Some have lasted for years. Quite a few projects (even ones that lasted for years) had very few participants other than me. This was OK for me because I enjoyed building stuff. None of my projects (so far) became super popular. That's pretty normal I think. Exceptional things are exceptional by definition. Making sure that you are OK just building stuff because you enjoy building it is a good way to make sure you don't get jaded.

Having said that, if your motivation is driven by having a following, learn something from the lean startup guys. Do the tiniest thing that could possibly work and just go. See if you get any interest at all. Fiddle with the parameters until you start to get some interest. Don't build a whole big website and gamble on the all-or-nothing. Build it and your following together over time and don't engage in too much risk.

I hope the questions I asked will help you narrow your scope and help you find the fastest way to deliver something and start testing the waters. Good luck!

<comment><AUTHOR> Thank you for taking the time to comment. My motivation behind this is to teach people who have a goal or dream to be a software developer that can't go to school for whatever reason and can't afford a bootcamp.

By Unicorn I mean rockstar or be in the top percentage among peers instead of just being average.

It has been hard to find quality people who have the motivation, passion and drive to be able to have the determination to teach themselves.

What I meant be exceptional is someone that has tried and failed to teach themselves but is willing to give it another go. I should have used dedicated instead of exceptional.

I have already done the Front end curriculum this past year and weeded out all the bad videos and books to find the best material. It would take a part time student learning at night about 9 months for the bootcamp.

Thank you for your input.

</comment></comment><comment><USER4> "Unicorn" is a term usually used by experienced developers who think specific job listings are asking for unreasonably broad skill sets. i.e. "I've been doing this job for 15 years and even _I_ can't fill this Jr. Dev role."

EDIT: oh god, I just glimpsed a future where asshat recruiters on LinkedIn have co-opted the word "unicorn" and are starting to use it like "rockstar".

<comment><USER5> I can only look forward to the time when all recruiters are looking for "Full- stack ninja unicorns" to crush code.

<comment><USER4> It will at least make for a useful Gmail filter to auto-send to the trash.

</comment></comment></comment><comment><USER6> Don't overload them. Present an easy to get started IDE and prepare some code that already generates a hello world in a GUI/browser. Then teach them a little fancy stuff like color changing, making a ball jump by pressing space, etc. If you hit a newcomer with vim and expect him to do something without giving him 2 years, then he will just think that programming is not for him.

<comment><AUTHOR> I picked vim because of all the plugin support and its long history. A person is going to have to learn an editor anyway so might as well learn the best. The learning methods I use for vim are just one book and several videos. I do not expect anyone to move on to the next learning topic until they feel comfortable and positive about learning vim. So if it takes a few months to learn vim they will retain the knowledge before they move on.

Thank you so much for your comment.

<comment><USER6> Look, I'm a huge fan of vim and use it 99.999999% of the time. But think about the first time you tried to quit from vim or to enter something, or to navigate while in command mode. That takes a huge effort. If you want to make your course about vim, that's also fine. A human brain can only take a certain amount of pain, though.

</comment></comment></comment><comment><USER7> This list is more than a 4-year CS program would hope to teach a dedicated student. It's not realistic to expect someone to go from learning how variables work to load balancing a server they wrote from scratch, in vim on a shell, in a bootcamp.

<comment><USER8> Totally.

Just getting enough context to know what you don't know is often unattainable for those who have been in the field for years.

I have mixed feelings about bootcamp/codecamps, mostly because I think they are Web 2.0 University of Phoenix that preys on people that are in a disadvantaged state. My experience with people who have bootcamped is that they exist in an uncanny valley, some stimulus responses they react to, others result in a blank stare.

Wanting to learn how to code so the IT guy can no longer fsck with you, great! Make awesome macros in Python for Excel, cool. Learn SQL so you can reduce your own data, all the more power.

But to say that someone can learn "to code" in even three months is preposterous and potentially very damaging. Only use codecamps that take payment __after __you have been hired.

</comment></comment><comment><USER9> What exactly do you mean by the term "Unicorns"?

In my mind, rare and mythic creatures (and their human analogs) cannot be trained into existence. Sure you can take a group of people and show them how to do something, but the result would be closer to a squad of foot-soldiers than a herd of Unicorns.

<comment><USER8> In The Valley, Unicorns are devs that also have some graphic design skills.

In other places, they are often devs that also have deep domain knowledge. Like an elevator repair tech who also codes, esp useful for your elevator repair startup.

</comment></comment><comment><USER10> You're setting yourself and the students up for a really bad experience.

People have to experience repetitive text editing before realizing why vim would be valuable. They need to copy and paste code and make file backups by hand to get a clue why git would help. They need to try to build a big JavaScript application to appreciate why all the frameworks exist.

Don't rob them of having to experience why all these projects were created and what problems they're trying to solve. Ignite the spark with some fun, playful basic stuff.

<comment><USER11> Anyone who has ever lost or accidentally saved over a file should appreciate git. They might not appreciate the complexity of git, but they can see the purpose.

</comment></comment><comment><USER12> You can find a lot by researching what other web frontend / basic backend dev bootcamps offer: [https://www.google.com.br/search?q=programming+bootcamps](https://www.google.com.br/search?q=programming+bootcamps)

</comment><comment><USER13> IDK, maybe pull down a little on the frameworks and add another "traditional" back end language like python maybe? Or dare I say php since it's easier to pick up. Because all those frameworks look a lot of work for a bootcamp IMO. People can't go from learning command line and html to handling servers and complex frameworks in the course of a single camp.

</comment><comment><USER14> Hmm, are you advertising that this is a series on web development? Do these students have prior development education? I fear that you're agenda is bloated.

* Vim deserves a class of its own. My personal experience is that it takes months of using Vim before it starts paying off. Your students will struggle with and be distracted by Vim.

* Git is an invaluable, industry-standard tool for collaboration. Still, is it helpful for learning to code or is it also a distraction? Pastebin/Dropbox/etc. are sufficient for collaborative efforts when your students are writing no more than 100 lines of code a day.

* Angular/Backbone/Knockout are all opinionated and assuming. jQuery is also too much "magic" for someone who just learned what a variable is a few weeks prior. These are just getting in the way.

* Yeoman/Grunt/Bower are cool. But there is something much more intuitive about just manually linking in a .js file to learn about dependencies.

* Gulp/Coggle/Balsamiq are similarly unnecessary.

In general, I think you're trying to teach opinionated tools that will frustrate, distract, and mislead your students. There will be a lot of "wait, which one's Bower, again?"

I hope I haven't been discouraging and unnecessarily critical... my doubts would certainly need to be revised if these students indeed had prior experience. My idea of a schedule for starting web dev:

* Very short overview of how a web browser and a server communicate.

* HTML: creating interactive elements for humans with textual markup.

* Node.js/Sinatra/etc. (students might benefit from using Node.js here as to not be overwhelmed with languages later): code a simple server to respond to a POST.

* CSS: modifying the visual representation of HTML elements for a better user experience.

* JS: react to HTML element events.

* Expand your Node.js server to respond with JSON instead of HTML, and rig your JS to asynchronously communicate with the server.

* Throw in a simple JSON/SQLite database. Flat file databases are great. Make the tried and true to-do app.

* Teach deployment (real-world servers). Git should probably be fit in here!

* Bonus: a whole class on how to solve problems using Google.

If your students are intermediate/advanced, ignore me completely.

<comment><AUTHOR> Thank you for taking the time to comment. You gave me some really good ideas.

I thought maybe the agenda was kinda bloated too especially since this is just the Front End Development. If you add Back end development you would probably be looking at 2 years to complete. Luckily my bootcamp is free and you can take all the time you need and only move on to the next topic when you feel comfortable with the current topic.

Vim - Here is what I have for vim and surprisingly enough is just enough to be proficient in vim.

3\. Learn vim a. Do vimtutor b. Read Learning the vi and Vim Editors, Seventh Edition c. Up and Running with vi d. Venture Into Vim e. OReilly - <USER2> Vim f. Vim for Advanced Users g. Learning vim plugins

Git - I use two video tutorials that will make anyone an intermediate person within 2 weeks.

HTML - We do about 12 websites CSS - We learn almost everything about CSS. It does take about a month just to do the videos and books. Javascript - We learn it in depth with three of the top books and several videos. This is where most of the time is spent. We do not move on to JQuery until we are proficient in javascript so we do not use JQuery as a crutch.

Bonus - We have the students sign up for HN, Reddit, and StackOverflow to teach them how to troubleshoot problems.

Im running out of room here but thank you so much for your input.

</comment></comment><comment><USER15> Maybe for a small backend project, you could use Ruby on Rails? RoR gives you easy JSON API capabilities, an app server, and an out-of-the-box sqlite3 db.

I figured I'd throw it out there because for a beginner, all of these things would be wrapped into one application and it might be easier for a beginner to maintain; however, you'd need to call out that not all frameworks have everything that RoR has.

And I'm not saying RoR is a 'best practice' backend approach, because I'd never use it as a real backend service, but it might fit in nicely with your bootcamp!

</comment><comment><USER16> How long is the boot camp ?

Supposing is not very long I won't teach any framework, but I would let them discovered it.

After they manage the CSS I would make them build some simple page, and I will let them know that there are some CSS framework already done...

Similarly for the Js part, teach them how to inject html in a page first and finally let them know about framework...

Also I will cover react.js, not because is particularly cool or new or hot, but because is a completely different approach, actually closer to the CSS / front end way to code...

Also, I won't really teach them vim... Maybe sublime is a safer bet...

</comment><comment><USER17> Why Vim?

Edit: downvotes, so .. it's the only thing I've not personally used in his list, I've been using nix environments for over 20 years and never got beyond <esc>:q! (or whatever it is). It seems the author considers it an important tool for being an accomplished web-designer - enough to have it as a prereq - so I'm interested in the OP's reasoning.

<comment><USER18> I am of the same opinion.

I can use Vi / Vim enough to get basic stuff done on a server, but I'll stick with eclipse for doing my main development. Having a reasonably intuitive interface means I can concentrate more on the code and less on trying to remember strange key combinations.

</comment></comment><comment><USER19> You're not going to train an unicorn teaching technical stuff. This can be learnt easier and better using the internet.

</comment><comment><USER20> Looking at that curriculum if my newphew wanted to learn programming, I sure as hell wouldn't send him to your bootcamp.

You can't seriously expected anyone to learn 10 disciplines in one go.

</comment><comment> Warewolf-ESB Encourage them to participate in open source projects! They will learn from some pretty awesome developers, and also contribute to the greater good :)

</comment><comment><USER21> This is a lot of work. How long is the bootcamp?

<comment><AUTHOR> The bootcamp really does not have a time because it is free, and what they are really getting is a mentor who can help them and guide them to all the best resources. I put together the curriculum after searching for the best books and videos and it took me 9 months.

</comment></comment><comment><USER22> It would be awesome if you would include ReactJS for the people who lean towards app development.

</comment><comment><USER23> How to get in touch with you?</comment>


UX Design for Data-Intensive Applications - <AUTHOR> https://twitter.com/antdke/status/1252666650366816257 <comment><AUTHOR> Hey HN, writer of the tweet here.

Although I did a good amount of googling, I'm almost certain that I missed a couple best practices. Anyone else know a few more good ones?

Using these to write an issue for my product newsletter.

Thanks in advance :)</comment>


Dr. Fauci: Chance of coronavirus vaccine being highly effective is 'not great' - <AUTHOR> https://www.cnbc.com/2020/08/07/coronavirus-vaccine-dr-fauci-says-chances-of-it-being-highly-effective-is-not-great.html <comment><AUTHOR> "The chances of it being 98% effective is not great, which means you must never abandon the public health approach."</comment>


'Google Concepts' - the linguistic intersection calculator - <AUTHOR> http://adnam.twenex.org/adam-hayward/google-concepts/yp-pg.html

<comment><AUTHOR> Just a toy I made.</comment>


Python Documentation Using Sphinx - <AUTHOR> https://leimao.github.io/blog/Python-Documentation-Using-Sphinx/ <comment><USER2> I usually generate new Python projects with a cookiecutter; such as cookiecutter-pypackage. I like the way that cookiecutter-pypackage includes a Makefile which has a `docs` task so that I can call `make docs` to build the sphinx docs in the docs/ directory which include:

\- a /docs/readme.rst that includes the /README.rst as the first document in the toctree

\- a sensible set of default documents: readme (.. include:: /README.rst), installation, usage, modules (sphinx-autodoc output), contributing, authors, history (.. include:: /HISTORY.rst)

\- a sphinx conf.py that sets the docs' version and release attributes to pkgname.__version__; so that the version number only needs to be changed in one place (as long as setup.py or setup.cfg also read the version string from pkgname.__version__)

\- a default set of extensions: ['sphinx.ext.autodoc', 'sphinx.ext.viewcode'] that generates API docs and includes '[source]' hyperlinks from the generated API docs to the transcluded syntax-highlighted source code and links back to the API docs from the source code

[https://github.com/audreyfeldroy/cookiecutter- pypackage/tree...](https://github.com/audreyfeldroy/cookiecutter- pypackage/tree/master/%7B%7Bcookiecutter.project_slug%7D%7D/docs)

</comment><comment><USER2> There are a few styles of docstrings that Sphinx can parse and include in docs with e.g. sphinx-autodoc:

`:param, :type, :returns, :rtype` docstrings (which OP uses; and which pycontracts can read runtime parameter and return type contracts from [https://andreacensi.github.io/contracts/](https://andreacensi.github.io/contracts/) (though Python 3 annotations are now the preferred style for compile or editing-time typechecks))

Numpydoc docstrings: [https://numpydoc.readthedocs.io/en/latest/format.html](https://numpydoc.readthedocs.io/en/latest/format.html)

Googledoc docstrings: [https://sphinxcontrib- napoleon.readthedocs.io/en/latest/](https://sphinxcontrib- napoleon.readthedocs.io/en/latest/)

</comment><comment><USER2> You can use Markdown with Sphinx in at least three ways:

MyST Markdown supports Sphinx and Docutils roles and directives. Jupyter Book builds upon MyST Markdown. With Jupyter Book, you can include Jupyter notebooks (which can include MyST Markdown) in your Sphinx docs. Executable notebooks are a much easier way to include up-to-date code outputs in docs. [https://myst-parser.readthedocs.io/en/latest/](https://myst- parser.readthedocs.io/en/latest/)

Sphinx (& ReadTheDocs) w/ recommonmark: [https://docs.readthedocs.io/en/stable/intro/getting- started-...](https://docs.readthedocs.io/en/stable/intro/getting-started-with- sphinx.html#using-markdown-with-sphinx)

Nbsphinx predates Jupyter Book and doesn't yet support MyST Markdown, but does support Markdown cells in Jupyter notebooks. Nbsphinx includes a parser for including .ipynb Jupyter notebooks in Sphinx docs. nbsphinx supports raw RST (ReST) cells in Jupyter notebooks and has great docs: [https://nbsphinx.readthedocs.io/en/latest/](https://nbsphinx.readthedocs.io/en/latest/)

Nbdev is another approach; though it's not Sphinx:

> _nbdev is a library that allows you to fully develop a library in Jupyter > Notebooks, putting all your code, tests and documentation in one place._

> [...] _Add %nbdev_export flags to the cells that define the functions you > want to include in your python modules_

[https://github.com/fastai/nbdev](https://github.com/fastai/nbdev)

A few additional sources of docs for Sphinx and ReStructuredText:

Read The Docs docs > Getting Started with Sphinx > External Resources [https://docs.readthedocs.io/en/stable/intro/getting- started-...](https://docs.readthedocs.io/en/stable/intro/getting-started-with- sphinx.html#external-resources)

CPython Devguide > "Documenting Python" [https://devguide.python.org/documenting/](https://devguide.python.org/documenting/)

"How to write [Linux] kernel documentation" [https://www.kernel.org/doc/html/latest/doc- guide/index.html](https://www.kernel.org/doc/html/latest/doc-guide/index.html)

awesome-sphinxdoc: [https://github.com/yoloseem/awesome- sphinxdoc](https://github.com/yoloseem/awesome-sphinxdoc)

... "Ask HN: Recommendations for Books on Writing [for engineers]?" [https://news.ycombinator.com/item?id=23945580](https://news.ycombinator.com/item?id=23945580)</comment>


A simple no-login listmaker built by the folks at FriendFeed - <AUTHOR> http://simpli.st Built by the folks at http://FriendFeed.com Simpli.st makes and manages lists. It aims for simplicity and requires no login. Each list gets a new link for sharing and linkmarking (social bookmarking). I like it especially with the Del.icio.us Firefox add-on and Del.icio.us live feeds of selected tags into Firefox. <comment><AUTHOR> Built by the folks at <http://FriendFeed.com> Simpli.st makes and manages lists. It aims for simplicity and requires no login. Each list gets a new link for sharing and linkmarking (social bookmarking). I like it combined with the Del.icio.us Firefox add-on and "Live Bookmark" feeds of selected tags back into Firefox. Combining these two simple tools makes an even more flexible and powerful tool for managing lists.</comment>


Rearchitecting GitHub Pages - <AUTHOR> http://githubengineering.com/rearchitecting-github-pages/ <comment><USER2> Pretty similar to how Neocities serves static sites ([https://neocities.org](https://neocities.org)).

There's a few differences. We don't use SQL in the routing chain, we use regex to pick out the site name and then serve from a directory of the same name (this is NOT as bad as it sounds, most filesystems can do this quite well now and take MUCH more than half a million sites to bottleneck).

DRBD is also a little hardcore for my tastes. Nothing wrong with it, I just don't know it well, and I don't like being dependant on things I don't know how to debug.

An alternative I wanted to show uses inotify, rsync and ssh combined into a simple replication daemon. It's obviously not as fast, but if you enable persistent SSH connections, it's not too bad. If it screws up, you can just run rsync. Rumor has it the Internet Archive uses an approach not too far away from this for Petabox. Check it out if you're looking for something a little more lightweight for real-time replication: [https://code.google.com/p/lsyncd/](https://code.google.com/p/lsyncd/)

We're still working on open sourcing (!) our serving infrastructure, so eventually you will be able to see all of the code we use for this (sans the secrets and keys, of course). I've just been having trouble coming up with a good solution for doing this. For now, enjoy the source of our web app: [https://github.com/neocities/neocities](https://github.com/neocities/neocities)

<comment><USER3> > _Nothing wrong with it, I just don 't know it well, and I don't like being > dependant on things I don't know how to debug._

Yep, this is basically our approach as well.

We've been using DRBD for quite a long time now on our Git fileservers (which also run in active/standby pairs - in fact, they look a lot like our Pages fileservers) so we have quite a lot of in-house experience with it and it's a technology we're pretty comfortable with. Given this, using it for the new Pages infrastructure was a pretty straight-forward decision.

<comment><USER2> Yeah, I've read the engineering posts about DRBD for the git fileservers, so I assumed that's why you made that decision. Makes total sense to me.

<comment><USER4> This exchange is wonderful, and absolutely what I'd expect out of you two, and maybe I'm just in a bad mood, but it stands in such contrast to the way I often see technologies discussed online.

This kind of thing is the way engineering should be. Kudos.

<comment><USER5> Yay for acknowledgements! Yay for recursive enthusiasm!

</comment></comment></comment></comment></comment><comment><USER6> All of github pages was run off of _one_ server? (with one failover standby).

That's pretty amazing. If all you're serving is static assets, apparently you have to grow to pretty huge scale before one server will not be sufficient.

I'm curious if there was at least a caching layer, so every request didn't hit the SSD. They didn't mention it.

<comment><USER3> We do have Fastly in front of *.github.io, but there's still a significant amount of traffic (on the order of thousands of requests per second throughout the day) that make it through to our own infrastructure.

We don't do any other caching on our own, although the other replies are correct in that the Linux kernel has its own filesystem cache which means not all requests end up hitting the SSD.

<comment><USER7> What percentage of the requests is handled by CDN?

<comment><USER8> I believe that only CNAMEs use the CDN for custom domains (naked domains do not) [0]. So that rules out a chunk but there will be lots of cache misses too. Maybe someone from GH can confirm.

[0] [https://help.github.com/articles/about-custom-domains-for- gi...](https://help.github.com/articles/about-custom-domains-for-github-pages- sites/)

<comment><USER9> github.io will still also hit the CDN. It's just custom naked domains that will miss.

</comment></comment></comment></comment></comment><comment><USER10> I'm amazed that Github Pages ran on just two servers (well, aside from the MySQL clusters). That is absolutely incredible given the sheer amount of projects and people who rely on it for their sites (me included!). I love the philosophy behind Pages of abstaining from over-engineering and sticking to the simple, proven solutions. It's a great service and I'm a massive fan.

<comment><USER11> It's cool that they were able to do this with two machines, and I don't want to detract from that, but it's probably worth pointing out that a "machine" is not a very useful unit of capacity. These two machines could be dual core i5s or they could be 20 core xeon boxes with hugely varying amounts of memory and storage. Too bad they don't clarify, I'm curious.

<comment><USER12>
      The fileserver tier consists of pairs of Dell R720s
      running in active/standby configuration. Each pair is
      largely similar to the single pair of machines that the
      old Pages infrastructure ran on.

[http://www.dell.com/us/business/p/poweredge-r720/pd](http://www.dell.com/us/business/p/poweredge-r720/pd)

Two Xeon E5-2600s per machine.

<comment><USER13> The ES-2600 series comes in 4 to 12 cores, so that's still a pretty wide range of compute power.

</comment></comment></comment></comment><comment><USER14> Well done GitHub. Also a special mention to the invisible workers making nginx such a cornerstone of the modern infrastructure, it's a project that I don't hear about often, probably due to the fact that it's not the sexiest piece of technology, but it really seems solid and battle-tested. Kudos.

</comment><comment><USER15> I've been using GitHub pages for a while now and I always wondered why they had the "your site may not be available for another 30 minutes" message on creating a new GitHub pages site while pushes to an already-existing gh-pages branch were displayed instantly. Neat to see that explained here.

</comment><comment><USER16> This is all sitting behind a CDN correct?[1] Might explain why it was able to survive on two servers for so long.

Or is that automatically assumed when reading about a static hosting setup?

1\. [https://www.fastly.com/customers/github/](https://www.fastly.com/customers/github/)

<comment><USER17> It's mentioned in the article.

> We also have Fastly sitting in front of GitHub Pages caching all 200 > responses. This helps minimise the availability impact of a total Pages > router outage. Even in this worst case scenario, cached Pages sites are > still online and unaffected.

</comment></comment><comment><USER18> Only tangentially related, but I sometimes wonder if GitHub's management regret making GitHub Pages available for free, now that it's being used so heavily for personal and even business blogs, rather than just companion sites for open-source projects. They could be charging for static websites, as Amazon S3 does.

<comment><USER19> I never heard anyone gripe about it... not even once. The cost is pretty negligible, and there's a lot of halo benefit (i.e., you just get more people involved on GitHub the platform itself).

The fact that a lot of non-technical employees in marketing and other fields are using it for corporate blogs is actually a nice bit of pressure on the organization to make Pages and web editing even simpler for those users. It becomes harder to lean on "oh it's a developer site so they'll figure it out".

Mostly, though, I think it's just a matter that we wanted it for ourselves. It's pretty awesome from an industry bystander's perspective to have something free, simple, and static, so we can all benefit from more stable docs, blogs, and so on. Maybe that'll change in the future and something Totally Different will change the industry, but for right now I think it's pretty rad, and totally worth the investment.

<comment><USER20> You've just optimized for my happiness. Thank you.

</comment></comment></comment><comment><USER8> Nice! Does HN still run off of a single server and CDN too?

The CDN is key here, which you get if you use a CNAME (or ALIAS) instead of an A record for your custom domain on GH pages. I've found pairing pages with CloudFlare works great if you want to use a naked domain and you get HTTPS too. You can set up a page rule on CF to redirect all HTTP to HTTPS as well.

</comment><comment><USER21> It's time for github to start offering some basic hosting infrastructure of small projects, a light heroku, at least for JavaScript (which kind of already works).

I'd pay extra for that, I (we all) have a bunch of personal sites, landing pages, marketing sites and tiny side projects that'd love to not have to deal with hosting – I think they'd make a killing, but also think must be in the works.

<comment><USER22> I think the leap they'd have to make in infrastructure and architecture to support that is not worth it in their mind. But who knows.

<comment><USER23> Also the mental jump. Just because something is easy for them to do doesn't mean it is worth the distraction cost.

Github builds tools for developers. Atom, chat (abandoned), Pages, Gists, and github.com all fit within this. They tie into how teams operate. Serving JS is tangentially related — certainly something a web developer does — but not really core to their mission.

</comment></comment></comment><comment><USER24> Great summary of your architecture. Thanks for sharing!

A few questions:

\- Is everything in the same datacenter or in different datacenters? What happens if the datacenter is unavailable for some reason? Are data replicated somewhere?

\- You moved from 2 machines to at least 10 (at least 2 load balancers, 2 front ends, 1 MySQL master, 1 MySQL slave and 2 pairs of fileservers). That's a lot more. Do you need more machines because you need more capacity (to serve the growing traffic) or just because the new architecture is more distributed and requires more machines by "definition"?

\- I understand the standby fileservers are idle most of the time: reads go the active fileserver, only writes are replicated to the standby. Am I understanding correctly? If yes, it looks like a "wasted" capacity?

</comment><comment><USER8> Something I would really like is to be able to set the custom MIME type for an app cache manifest file. That way you could easily host offline web apps from GH pages. Anyone know a way to do this without using S3 or similar?

[https://en.wikipedia.org/wiki/Cache_manifest_in_HTML5#Basics](https://en.wikipedia.org/wiki/Cache_manifest_in_HTML5#Basics)

<comment><USER25> You shouldn't need to specify a custom one. GitHub Pages will automatically serve the file with the appropriate mime type given its file extension. Here [1] is the list.

[1]: [https://github.com/jekyll/jekyll/blob/master/lib/jekyll/mime...](https://github.com/jekyll/jekyll/blob/master/lib/jekyll/mime.types)

Edit: As you can see in that link, both .manifest and .appcache file extensions map to text/cache-manifest mime type.

<comment><USER8> Nice. Very helpful. I must have been using the wrong (or no) extension before.

</comment></comment></comment><comment><USER26> Have you thought about using bind as the db for the routing ? an internal dns lookup for the storage node storage.url -> 10.0.12.1

</comment><comment><USER27> Seems odd to me that the router hits a MySQL database on every single request rather than just hashing the hostname as the key for the filesystem node.

<comment><USER28> Hash-based partitioning has a big problem that when you change the hash size all of the data moves around. Eventually you'll need to do a lookup-based partitioning scheme. You also probably want control over where some users live since you don't want two super hot users on the same server.

<comment><USER29> There are hash schemes that prevent that, however. Look up consistent hashing.

And you could probably have a (small) hashmap that allows overrides for the most used pages.

<comment><USER30> My thought too. If they have half a million sites, and only 1% are hot enough to need deliberate placement, then they only need to store five thousand special cases, which is a few megabytes of memory, easily stored on each load balancer and loaded at boot.

Use hashing for the rest. Ideally consistent hashing, or rendezvous hashing, which i just read about on Wikipedia so must be good:

[http://en.wikipedia.org/wiki/Rendezvous_hashing](http://en.wikipedia.org/wiki/Rendezvous_hashing)

</comment></comment></comment></comment><comment><USER31> I thought github pages is running on riak and webmachine from 2012[0].

[0] [https://speakerdeck.com/jnewland/github-pages-on-riak-and- we...](https://speakerdeck.com/jnewland/github-pages-on-riak-and-webmachine)

<comment><USER32> I threw out that prototype soon after the talk. At the time, there weren't a lot of other engineers at the company doing Erlang, so maintenance was considered to be a long-term problem. I'm glad we made that call.

<comment><USER33> There are so many presentations of the form "We're using Erlang/Scala/whatever, it's so awesome!", but so few followups when they give up on the idea for production..

<comment><USER34> It's hard to sustain some alternate technology in the face of common knowledge. Rarely does technical advantage outweigh hard-won operational experience.

</comment></comment></comment></comment><comment><USER35> It seems to me, they could have gone a farther step removed via something like Cassandra. With a Cassandra cluster, they could have used a partition key that is the domain name + route in question, they could then do lookup against that entry, with the resource path (excluding querystring params) could be used to find a single resource in cassandra, and return it directly.

A preliminary hit against a domain forwarder would be a good idea as well, but for those CNAME domains, dual-publishing might be a better idea... where the github name would be a pointer for said redirect.

While Cassandra itself might not be quite as comfortable as say mySQL, in my mind this would have been a much better fit... Replacing the file servers and the database servers with a Cassandra cluster... Any server would be able to talk to the cluster, and be able to resolve a response, with a reduced number of round trips and requests... though the gossip in Cassandra would probably balance/reduce some of that benefit.

<comment><AUTHOR> Adding a database that is new to GitHub would not be a pragmatic move.

</comment></comment><comment><USER36> I remember the time I mistakenly drove huge amounts of traffic to Github Pages, believing they had the infrastructure to handle it. I apologise for last year's downtime :)

Glad to hear it's being improved. I'm impressed that it was able to run on such simple infrastructure for so long.

<comment><USER37> I doubt you caused any downtime. All the page content is behind a CDN.

<comment><USER36> I did, confirmed by the angered emails from GitHub and the suspension of the relevant repository.

</comment></comment></comment><comment><USER38> I'm wondering why GitHub prefers MySQL over PostgreSQL.

<comment><AUTHOR> Stable, proven, popular, great roadmap, great replication story, great tooling and an awesome community.

<comment><USER39> Would postgreSQL be meaningfully faster?

<comment><USER30> My heuristic is that MySQL is faster for inserts and single-table reads by primary key, but PostgreSQL is faster for more complex queries, particularly if they involve joins or subqueries.

Basically, MySQL is a key-value store in an RDBMS's clothing.

<comment><USER35> On the flip side, PostgreSQL's high availability options aren't in the box, or are at the very least varied, problematic and/or cost more than other options for support contracts.

Every time I've worked with mySQL I've seen some irksome behavior... just the same, setting up failover options is miles ahead of PostgreSQL. And when you already have in-house talent, it becomes even more obvious.

My only thought was that using a clustered database (such as Cassandra) as the store with the data itself might have been better. domain/url (minus querystring) would hash/distribute fairly well, and with even a relatively small cluster with 2 replica nodes for the shard would be pretty effective. Also, it would be easier to manage a replicated database, in my mind, than tracking sites to pairs of static servers. GoDaddy is/was moving to something similar with new development on one of their applications when I worked there, and able to serve a huge number of static requests (hundreds of thousands per second) off of a relative few servers with a sub 10ms response time, for content not backed by cdn.

In the end it just goes to show that serving static content on modern hardware can scale _really_ well, with a number of options for technology. Which is why I'm somewhat surprised that something hasn't taken over the tide of poorly configured Wordpress blogs.

</comment></comment></comment></comment></comment><comment><USER40> Question from Githubbers:

Have considered using cluster file systems such as GlusterFS or Ceph?

<comment><USER2> I looked into GlusterFS at one point. GlusterFS is a no-go for static file serving in hostile environments. It asks every node to look for a file, even if it's not there. You can imagine the DDoS attacks you could build here using a bunch of 404 requests for files that don't exist.

One story I heard from a PHP dev is that it would take 30 seconds to load a page while it looked for all the files needed to run it.

<comment><USER41> Yea, GlusterFS is terrible at PHP, or anything involving lots of small files. Like, static sites or say git repos.

</comment></comment></comment><comment><USER42> Surprisingly, they uses mysql (instead of current hype k/v store) to map hostname and fileserver.

<comment><USER3> MySQL is actually a really good key value store!

Here's the schema we use for the routing information:

        CREATE TABLE `pages_routes` (
          `id` int(11) NOT NULL AUTO_INCREMENT,
          `user_id` int(11) NOT NULL,
          `host` varchar(255) NOT NULL,
          PRIMARY KEY (`id`),
          UNIQUE KEY `index_pages_routes_on_user_id` (`user_id`)
        );

Since we use MySQL for everything else, we decided it made the most sense to keep this routing data here rather than introducing a new database.

<comment><USER43> > Since we use MySQL for everything else, we decided it made the most sense to > keep this routing data here rather than introducing a new database.

Very smart. "Perfection is Achieved Not When There Is Nothing More to Add, But When There Is Nothing Left to Take Away"

</comment></comment></comment><comment><USER44> I remember seeing a talk about a Github Pages rearchitecture at Erlang Factory in SF in 2012: [http://www.erlang- factory.com/conference/SFBay2012/speakers/...](http://www.erlang- factory.com/conference/SFBay2012/speakers/JesseNewland)

I don't see anything about those components in this post. Did that architecture never make it to production?

<comment><USER45> answered elsewhere - [https://news.ycombinator.com/item?id=9612914](https://news.ycombinator.com/item?id=9612914)

</comment></comment><comment><USER46> Can you explain a bit more about the load balancer tier? The blog post doesn't say much about it and I'm curious about your haproxy config.

<comment><USER47> This sounds like a great idea for a future blog post. Thanks for the suggestion!

</comment></comment><comment><USER48> I was hoping to see something about optional SSL/TLS. I'm even willing to pay for it.

<comment><USER8> You can use CloudFlare with flexible SSL in front of it and set up a page rule to redirect to HTTPS.

<comment><USER48> I had no idea. That's great news!

<comment><USER49> Be warned, though: this will only create a secure connection between the user and CloudFlare, not between CloudFlare and GitHub Pages.

<comment><USER35> While a potential for concern... I would guess that CloudFlare's own routing would take them to an exit node close to Github's... that does make quite a few assumptions though. It would still be more complicated to mitm between CloudFlare and Github than ISP-X (or China) and Github...

</comment></comment></comment></comment></comment><comment><USER50> Somewhat related, what's the SLA for Github pages? Let's say someone wants to move a highly successful blog, are there some quotas? Maybe I missed them but I don't remember seeing any...

<comment><USER51> Ben from GitHub here. We don't offer an SLA for GitHub Pages (or GitHub.com generally).

We've talked a bit about our typical traffic in the past ([https://github.com/blog/1992-eight-lessons-learned- hacking-o...](https://github.com/blog/1992-eight-lessons-learned-hacking-on- github-pages-for-six-months)), but suffice to say, we host some highly trafficked sites like the Bootstrap documentation.

If your blog isn't on that scale, and conforms to our terms of service ([https://help.github.com/articles/github-terms-of- service/#g-...](https://help.github.com/articles/github-terms-of- service/#g-general-conditions)), we generally don't enforce hard usage quotas, but if you're concerned, feel free to reach out to email@example.com any time.

<comment><USER52> I know users are using Travis-ci to automatically build when there are new commits in master and push back the results to the gh-pages branch with a lot of success.

Github pages is so cheap and easy it is the disruptive technology that is eating the lunch of stand alone hosting. Users don't think about servers, deployment or other things, they simply are pushing a branch and poof it is on the web.

Do you have a wishlist of features to add to GitHub pages? Maybe allowing minimal sandboxed server side computation with a max runtime of say 1ms, setting headers, redirects or other stuff? I am guessing every little addition would eat away at the alternatives.

<comment><USER51> > Do you have a wishlist of features to add to GitHub pages? Maybe allowing > minimal sandboxed server side computation with a max runtime of say 1ms, > setting headers, redirects or other stuff? I am guessing every little > addition would eat away at the alternatives.

Moving from WordPress, the fact that you couldn't constantly tweak a thousand little things was extremely liberating. That's the zen-like simplicity of GitHub Pages that to me, makes it an attractive option over heavyweight alternatives. Just push and your site is live. Fewer things to break and fewer things to worry about means more time to focus on what matters: your content.

</comment></comment></comment></comment><comment><USER53> I've never known about this way to extend nginx - I've been looking at a way to get nginx "smarter" on our mesos config as dns-based configuration had some holes.

<comment><USER54> Curiously, I remember sticking with lighttpd on a project for quite a while simply for their Lua integration (nginx didn't have it at the time).

</comment></comment><comment><USER55> I love this write up. Awesome example of how to build the minimal, simple thing first and expand as traction and needs develop.

</comment><comment><USER56> I find that for such tasks it makes sense instead of trying to hack around nginx config and its lua scripting, to throw it away and write a small app in, say, Haskell+Warp that would do the job. It's as fast as nginx (and probably faster than nginx+lua), would have much more static guarantees, express logic more clear.

</comment><comment><USER57> If you use the magic of hashing you can lookup where user's pages are without the dependency on the database.

<comment><USER3> Unfortunately this isn't an option for us. We need to be able to move sites between fileservers periodically to keep disk usage and load balanced. Adding new fileservers when using a hash modulus based routing scheme is also quite complex as it would require copying quite a lot of site data between fileservers.

Pages also supports custom domains for both user sites and per-project sites, so we'd still need a way to resolve domains to users.

<comment><USER58> Think about moving around hash buckets, not sites nor files. You can decide what you want to keep in memory data or metadata. Now you need both sites+file metadata, neither of those need to be stored when serving static files you yourself rename and place everytime they are published.

</comment></comment></comment><comment><USER22> Great example of YAGNI, related to the post by martin fowler posted earlier today on HN.

</comment><comment><USER59> Why did they choose the 98th percentile, specifically? Why not 99th and friends?

</comment><comment><USER60> I hadn't heard of Fastly before. How do they compare to Cloudflare?

<comment><USER61> Fastly is actually really great but they cost $50/month to start with. Their throughput and routing is amazing through.

<comment><USER62> I hear many great things about Fastly from users, but whenever I look at CDN performance comparisons (e.g. [https://cloudharmony.com/reports/editions/cdn- performance-re...](https://cloudharmony.com/reports/editions/cdn-performance- report/basic/cdn-performance-report-0914.pdf)) they never do brilliantly for US/EU regions. What gives?

</comment></comment></comment><comment><USER63> Even if Lua is all but a footnote, it's nice to see that, yet again, things that rely on it work as advertised.

</comment><comment><USER20> Guess I'm learning Lua now. :D

</comment><comment><USER64> awesome write up. I thought i was the only crazy guy to use lua to route subdomains to internal servers. Little surprised on the MySql part instead of redis like fast look up key.

</comment><comment><USER65> off-topic, but since we're talking about github pages.. i own github.id, and i'm thinking of making "linkedin for github users", do you guys think there's a market for that?

<comment><USER66> Wouldn't there be huge copyright issues with that?

<comment><USER45> No. "GitHub" is a trademark, not a copyrightable work.</comment></comment></comment>


Three questions that keep me up at night - <AUTHOR> https://blog.evjang.com/2020/04/three-questions-that-keep-me-up-at-night.html <comment><USER2> > Q1: Can we imitate "thinking" from only observing behavior?

If you believe that the human mind is the result of Darwinian evolution, then you implicitly believe that thinking can arise from interacting with the environment. That does not exactly answer the question, but if anything, it goes beyond it.

On the other hand:

"This could also be addressed via other high-data environments:

1) Observing trader orders on markets and seeing if we can recover the trader's deductive reasoning and beliefs about the future. See if we can observe rational thought (if not rational behavior).

2) Recovering intent and emotions and desire from social network activity."

Before you could even begin to get anything from these streams of symbols, you would have to have some way to give them semantics - to understand what they signify to the actors. I suspect that these would be among the last things you would achieve, not a means to the end.

</comment><comment><USER3> Question 2: This is better left unanswered. We've been building superhuman AIs for at least 300 years which are exploiting reality to their advantage and have started out-competing humans for resources (including by changing laws in their favour). These things we've been building are called corporations, and they're certainly smarter than any individual, richer than any human can become and are now working hard to figure out how their humans can be replaced by more reliable components.

</comment><comment><USER4> Q1) By the mere copying of behaviors, no. It is clear that there are at least the a priori intuitions of space, time, and causality necessary for humans to obtain any knowledge. It is within that a priori framework where concepts and then decisions are derived from.

Q2) This would require the knowledge of what is sufficient & necessary for AGI.

Q3) Minsky, Neumann, and most sophisticatedly, Norbert Weiner answered this question. Read page 33 of Weiner's Cybernetics. The real insight, where in what I will now explain in brevity is at the expense of the insight's resolution, is that in the nature of microscopic phenomena there are an exorbitant quantity of fickle components ("equal particles") whose positions, accelerations, velocities are all impossible to record because of that sheer quantity by the nature of this sphere of study. More so, there are then at least second-order coupling effects between all these fickle components due to their constant interaction. As a resolve, we have developed terms of statistical understanding. Weiner cites "cloud, temperature, turbulence, etc." all being concepts which are specifically defined in a statistical context.

Where as in the macroscopic sphere there are far fewer particles much further apart from each other with near certain initial masses, positions, and velocities.

The result? In the microscopic systems, because their properties are precisely defined statistically the distribution which predicts the future nor past for a given system isn't unique. That is, the future state (predicted by the distribution provided by some initial system) could have also been predicted by other distributions describing many other initial systems unless by "some miracle" there is a "tightly defined statistical range".

So are microscopic physics condemned to this paradox? Not necessarily, there may be some way to construct scientific objects from systems phenomena which don't require a statistical conception because they can be precisely defined from a microscopic systems point of view temporally & spatially. However, this would require a revolution in theory and likely mathematics

<comment><USER5> I dislike the cybernetic approach because I find that it first commits to epistemic helplessness, saying that we know nothing of particles. Perhaps that was reasonable when QM was so young, but these days we can use QM for a better answer. This isn't complete, but it helps a lot.

First, note that CPT-symmetry can be extended up to the macroscopic level. Einstein's thought experiments about time involve macroscopic entanglement. For example, given that a red ball and a blue ball are each in their own boxes, and one possesses one box, then the box is entangled to have either a red or blue ball inside. We can use linear logic to reason about these situations; one holds the multiplicative disjunction of red and blue.

Second, note that in order to make MWI's numbers work, we need not just branching of possibilities, but also coalescing. Imagine that, in empty space, we suddenly have a single virtual-particle pair; the diagram would look like () with time vertical. At the beginning, we have one universe; during the split, two universes, one for each chirality of the pair; afterwards, one universe. When we split, the probabilities of each possible universe decrease. Now, imagine doing this with more pairs, and verify that the Binomial Theorem manifests. This gives the same rise to macrostate-like entropy as Boltzmann's approach to thermodynamics, but only using QM particles and QM logic; a QM state is likelier when more of the many worlds in MWI would support its existence.

Putting these together, we may conclude that Loschmidt's paradox is an incomplete part of a bigger paradoxical complaint: We can see that there are time-reversible _and_ time-irreversible behaviors, both in particles and in macroscopic systems. The time-reversible behaviors seem to govern resources; the time-irreversible behaviors seem to govern entropy. The two logics for these two sets of behaviors are distinct; linear logic and intuitionistic logic can intertwine, but here they don't appear to do so. Why not?

<comment><USER4> > I dislike the cybernetic approach because I find that it first commits to > epistemic helplessness

The confusion here is that my statement has nothing to do with cybernetics. Although it's taken from his book titled Cybernetics, almost 60% of the book isn't about cybernetics but instead a philosophy and analysis of Science as it progresses through history.

Though it's fair to Wiener to preemptively note there is no failure in epistemology in one of the most successful theories which holds ground in neurons, to electrical circuits, traffic engineering, the abstract feedback loop. There's no assumption that "we know nothing of particles". Wiener originally wrote the book in 1948 and completed the second edition in 1961. He made significant contributions to quantum mechanics[0]. It's quite suspect to make such a defamatory remark about Wiener's cybernetics given his quantum knowledge and the field as a whole.

In fact, what would cybernetics possibly be predicting about the interactions between a particular system of which each component cannot exhibit a property? Surely some property must be being reinforced by means of itself or another auxiliary property.

QM & Cybernetics answer completely separate questions.

You're missing the bigger picture. The irreversibility in any physical phenomenon is only through scientific paradigms which are probabilistic—such as thermodynamics, temperature, cloud, turbulence etc. These concepts are simply more powerful in dynamical systems, which are always due to microscopic phenomena, where there are no analytical, deterministic solutions for the otherwise favored deterministic paradigms (e.g. three-body problem).

"while when a great number of atoms (of the order of Avogadro's number) is considered, this energy related to irreversibility becomes so large that its order of magnitude must be taken into account. Consequently, macroscopic irreversibility results related to microscopic irreversibility by flows of photons and amount of atoms involved in the processes."

[0] [https://www.ams.org/books/psapm/052/psapm052-endmatter.pdf](https://www.ams.org/books/psapm/052/psapm052-endmatter.pdf) [1] [https://www.nature.com/articles/srep35796](https://www.nature.com/articles/srep35796)

</comment></comment></comment><comment><USER6> Lame, only true question is will this universe die out and all information in it be lost or can it be maintained indefinitely/transfer information to the next universe iteration.

<comment><USER7> > only true question

you mean like, _the last_ question?

<comment><USER8> I guess, the only question that matters. Unfortunately it does not matter for us, we are too transient.

</comment></comment></comment><comment><USER9> Richard Feynman has a good explanation for 3.

[http://www.informationphilosopher.com/solutions/scientists/f...](http://www.informationphilosopher.com/solutions/scientists/feynman/past_and_future.html)

<comment><USER10> Thanks! I had a chat with a mathematician friend today, and we came up with a fairly simple resolution actually that I'm pretty satisfied with: the trick here is to stop thinking of particles as T-symmetric billiard balls, and just to assume that there exists microscopic T-asymmetric interactions.

If we assume molecular chaos hypothesis (independent particle velocities prior to collision) and take into account _probabilistic_ collisions occurring at the 2-particle microscopic level, then post-collision velocities are now dependent, thereby imposing an ordering to time.

I have to think more about this though, and I'm still struggling with understanding the circular dependency between entropy "being time" and entropy being "caused" by "motion" (with respect to ... time?)

<comment><USER9> Why can't you assume time symmetric billiard balls? If you take a lot of them, black and white, put all the black ones on one side and the white on the other, give them random velocities, and wait a bit, you will end up with them all mixed together. The entropy has increased even though all interactions are reversible. The macroscopic transformation is time asymmetric (the balls will not sort themselves back again) even though the microscopic transformations are time symmetric (elastic collisions and frictionless movement).

The way I think about this (from Feynman) is that entropy measures "how special" a macroscopic state is, on average. When you apply many microscopic independent fluctuations to a given state you are unlikely to end up with a "more special" state. That's how states are ordered at the macroscopic level giving the arrow of time a clear direction.

<comment><USER10> _The macroscopic transformation is time asymmetric (the balls will not sort themselves back again) even though the microscopic transformations are time symmetric (elastic collisions and frictionless movement)._

I agree with your observation of what happens to the black and white billiards. My question is - if the interactions are all time-symmetric (implies conservation of entropy), where does the entropy come from in the macroscopic system? My belief is that the microscopic model is flawed; if we simply introduce some randomness into the collision dynamics of two particles (which is believable given particle accelerator experiments / QM), then microscopic interactions are no longer T-symmetric.

It comes down to a philosophical interpretation of whether you can treat a system of two particles probabilistically (i.e. thermodynamically in aggregate) or not.

</comment></comment></comment></comment><comment><USER11> Sometimes I wonder if we are the critters related to the second question. Of course the next question is how far away are we from meeting the simulation's spec.

</comment><comment><USER12> > If you don't know how to grasp an object, you can't bring food to your > mouth.

Counterpoint: hummingbirds.

It might sound cheap, but discussions of the "basic necessities" of anything so often just overlook obvious counterexamples.

<comment><USER10> Hummingbirds actually proves my point here, on how life and death solve the symbol grounding problem.

What is a "stable grasp"? An entire body of classical robotics literature is devoted to answering this problem. I work on robotic grasping research and defining what it means to grasp something successfully is a little tricky unless you have a broader context of what the grasping is for.

Nature doesn't care whether hands exist or not - only that the animal can feed. That is why we see such a spectrum of appendages in the animal kingdom. Some are hand-like, others not at all, and some in between. In some sense, "hand" is a mental construct of human language.

</comment></comment><comment><USER13> Ask HN: what are the questions that keep you up at night ?

For me: How do viruses really work? How to antiviruses work? I've gone into basics of DNA -> RNA -> Ribosome -> protein -> cell organelles -> Cells. We know how to sequence DNA and it's pretty cheap right now. How do we simulate compiling DNA -> proteins and how they interact with each other? How do you find weak spots in a virus and figure out what proteins will inactivate them ?

2) what is the best course of action for people who lost their jobs? Taking <20k from 401k/IRA should be free of taxes and penalties. Everyone has lost >30% of value since Feb already. Sure most people don't have 401ks, but those who do could weather this storm a bit better. Should we stop payments for all loans for 2 months. No mortgage, auto loans, student loans etc. stop those foreclosures from happening. No rent.

3) will US have the most deaths? Most likely. A large population + most states haven't locked down. This means unless >80% of population gets infected and recovers to build herd immunity, or we have a vaccine, we're in deep trouble. Should we infect everyone >10 and under <30 and in clusters so we build up herd immunity in weeks? And get most of the young population back to work? Like if you're gonna be sitting at home for two weeks, may as well get sick and build up immunity while we're at it.

</comment><comment><USER14> Do we already posess the hardware required to run full AI?

Deep fakes can be done on older hardware, we just didn't have the code. Does the same thing apply to AI?

</comment><comment><USER15> I can imagine many people would not be comfortable answering that question, particularly depending on the company and their culture. I can easily see this type of question leading to dramatically different conclusions ranging from "this person is a deep thinker on varied topics" all the way to "this person wouldn't be a good cultural fit". Don't think too deeply about the wrong topics.

</comment><comment><USER16> #2 is explored by John Searle's _Chinese Room_ argument, which is unfortunately completely misunderstood by most technologists working in AI, to their own peril.

This article is an excellent introduction:

[http://www.dreamsongs.com/Searle.html](http://www.dreamsongs.com/Searle.html)

A relevant excerpt:

 _Searle goes on to contend that intentional behavior must be in response to real contingencies, not formal simulations of reality. If so, it would seem to follow that the capacity for intentional behavior is the result of what might be called phylogenic contingencies, and so would only be found in products of real evolution. In short, what Searle is saying is that, just like other biological activities such as digestion and photosynthesis, thought is intrinsically dependent on the biochemistry of its origin. Just as a formal simulation of digestion is not really digesting, a formal simulation of thought is not really thinking. And to tie this to the gender thought experiment, a formal simulation of gender is not gender, because it lacks biological structural and functional characteristics._

The "Simulation, Duplication and Evolution" section of the SEP article is also useful:

[https://plato.stanford.edu/entries/chinese- room/#SimuDuplEvo...](https://plato.stanford.edu/entries/chinese- room/#SimuDuplEvol)

<comment><USER9> See Mike Adler answer to Searle. To use your digestion metaphor, as long as the input is food and the output is energy and byproducts, AI researchers don't care whether philosophers would call it "real digestion" or a "formal simulation of digestion". Same goes for thinking.

[https://philosophynow.org/issues/46/Newtons_Flaming_Laser_Sw...](https://philosophynow.org/issues/46/Newtons_Flaming_Laser_Sword)

<comment><USER16> This essay is filled with so many falsehoods, assumptions, and lack of knowledge that I don't know where to begin. It's also written with a snarky, arrogant attitude which is not helpful. To list a few:

\- "there is every reason to believe that a human brain is a machine"

Given what evidence?

\- Platonism died for mathematicians some centuries ago, and simply looks silly.

And yet Frege and Gödel, two of the most accomplished mathematicians of the past century, could be described as Platonists. Clearly this isn't a closed question.

[https://en.wikipedia.org/wiki/Philosophy_of_mathematics#Plat...](https://en.wikipedia.org/wiki/Philosophy_of_mathematics#Platonism)

[https://plato.stanford.edu/entries/platonism- mathematics/](https://plato.stanford.edu/entries/platonism-mathematics/)

Regarding digestion: if the Turing Test is determining whether a computer is thinking like a human being, then focusing only on input and output, and not the 'being', sort of misses the entire point. Remember that Searle's Chinese Room is primarily a criticism of the Turing Test.

As always, the truth is: it's more complicated than it seems.

<comment><USER9> I certainly agree that Adler's provocative tone is not helpful and his strokes are sometimes too broad to be fully correct as you have rightfully pointed out, but the general point stands.

To me the Turing test gives a falsifiable experiment that defines what "thinking like a human being" means.

It does so by focusing only on the input output. To do that, it puts the system under test out of view of the experimenter and mediates the experiment with written text.

So yes, the input output is the important part, the Turing test, albeit flawed, is interesting, and any claim that machines can't think based on a definition of "think" that does not lend itself to a falsifiable experiment, such as Searle's, is uninteresting.

I will leave it at that and make a quick exit, as Adler suggests.

</comment></comment></comment></comment><comment><USER17> Q1: Can we imitate "thinking" from only observing behavior?

NO as you are measuring actions and not the process that instigated those actions.

One example - chess - you see the move, but the thinking behind that move may elude. More so a bad move, as their thinking was that it is a good move when it is not.

[EDIT format/spelling]

</comment><comment><USER18> > Many AI researchers, myself included, believe that competitive survival of > "living organisms" is the only true way to implement general intelligence.

My gut translates that idea as: Progress is made though War not Markets.

Which seems dubious to say the least.

<comment><USER19> Isn't a market a kind of competitive environment?

Given an appropriate reward function, and multiple independent actors seeking advantage, some might converge on an optimal strategy for attaining AGI.

Anyway, that's the way I read it.

<comment><USER18> Sure but exchange is a more important part of it. Even a predator offers protection services.

</comment></comment></comment><comment><USER20> In this set up, the answer to q1- Intent: No, intent can't be inferred.

If intent could be inferred- the meaningful and material portion of human privacy would cease to exist.

</comment><comment><USER21> Tldr:

>Q1: Can we imitate "thinking" from only observing behavior?

>Q2: What is the computationally cheapest "organic building block" of an Artificial Life simulation that could lead to human-level AGI?

>Q3: Loschmidt's Paradox: How does T-asymmetry in macroscopic phenomena arise from T-symmetric (microscopic) physical laws?

</comment><comment><USER22> 1\. What is your name?

2\. What is your quest?

3\. What is your favorite color?

Watch out for that last one.

<comment><USER23> >Watch out for that last one.

This must be a reference I'm not getting?

<comment><USER24> Monty Python and the Holy Grail

[https://youtu.be/0D7hFHfLEyk](https://youtu.be/0D7hFHfLEyk)

</comment></comment></comment><comment><USER25> I know it's just the moment we're living in right now but as I read those questions I just shrug, they couldn't feel less relevant.

The ones on my mind are "how long will large populations be locked down?" "How many people are going to die?" and "how long until this all happens again?"

<comment><USER26> "The greats never sacrifice the important for the urgent. They handle the immediate problem and still make sure to secure the future."

The media is incentivized to distract you. Turn off your computer, read a book, and don't forget to live.

<comment><USER25> What an absurd statement. This crisis matters. Social distancing matters. Lockdowns matter. "Stick your fingers in your ears" is awful advice.

<comment> m-p-3 But at some point, the medias are rehashing the same statements: Stay home, and follows good practices.

As an individual, there's only so much you can do and there's no need to keep yourself distracted from other matters.

<comment><USER25> The advice around wearing face masks has changed in the last 48 hours.

You can stay tuned in to the world around you without collapsing on the ground in hysterics.</comment></comment></comment></comment></comment>


NYTimes talks about Apple's former CFO and Steve Jobs - <AUTHOR> http://www.nytimes.com/2007/04/26/technology/26anderson.html?ex=1335240000&en=4db43cadc825d354&ei=5090&partner=rssuserland&emc=rss <comment><AUTHOR> I'm still nervous about all of this. I know the Board has cleared Jobs a couple of times, but that doesn't mean the SEC has. Hopefully it all goes away.</comment>


How make your web-app become available faster: Let go of the DOM Ready events - <AUTHOR> http://thanpol.as/javascript/you-dont-need-dom-ready/

<comment><USER2> As long as your JavaScript doesn't touch the DOM, there is no need to wait for DOM ready. BUT if you mess up and try and modify the DOM before it's ready, things break pretty horrifically. If your code is loaded async, you open yourself up to ugly race conditions that are tough to debug.

Start with the best practices (<http://developer.yahoo.com/performance/rules.html>). Once you've nailed those, then you can start thinking about optimizations like this.

<comment><USER3> hi josh,

as one of the people responsible for publishing those rules, I feel obligated to mention that they're somewhat out of date. The more correct rule today, given browser enhancements is, "It depends".

Asynchronously loaded code is obviously better. Stick it into the head, and allow it to download in parallel with the rest of the page. It doesn't need to execute until much later, but it's the downloading and parsing that takes the most time, so get that out of the way as soon as possible.

Within your script, you can check for the existence of the elements you need to manipulate. YUI does this quite well, but if you're using something else, just poll infrequently for the nextSibling of the node you want.

<comment><USER2> Philip, thanks for jumping in and well said. Looking at the state of the web right now where a typical site might have a couple dozen CSS/JS files, I think the rules are still well worth evangelizing. We still have a long way to go in helping people understand the fundamental principles, even if the exact implementations change over time.

</comment></comment></comment><comment><USER4> An important caveat to the "load scripts at the end of the document technique": The visual elements of a page will render before the functional components (i.e., JavaScript) are available.

Imagine a site the sets a custom `submit` handler for a `<form>` tag, possibly for client-side form validation, an AJAX file uploader, date selection, etc. A user submits the form _before_ JavaScript has loaded completely and doesn't get the benefit of the JavaScript-enhanced experience. In local development or on a high-speed connection this will probably never happen. But a user accessing the site with their phone over the slow U.S. cell data network has a much higher chance of hitting upon this undesired behavior.

<comment><USER5> Would it be acceptable to set the form's submit button to disabled in the HTML, attaching the custom event handler and enabling the submit button in a document-final script? Seems like you still get reasonable behavior in the low-bandwidth environment -- maybe even better, since the user can now access the other elements of the form while the script is being loaded and run.

<comment><USER3> That's an accessibility issue for users with JavaScript disabled (or where JavaScript fails to load).

On the other hand, it's also the recommended method to protect your users from a click-jacking attack across all browsers (read those that don't support CORS).

</comment></comment></comment><comment><USER6> "However when a script is loading, the browser will not start any other downloads, even on different hostnames!"

Orly? I guess he missed the async attribute (first one listed) in the MDN documentation he sited.

"As per the HTTP/1.1 spec browsers can download no more than two components in parallel per hostname."

Fortunately, browsers ignore the spec here and make more than two connections at a time per hostname.

"Faster page rendering, faster time when page becomes usable, faster page loading, better user experience. It's time to let go of the DOM Ready Event."

In my experience, getting things loading in parallel is one of the best ways to speed up page load times, and a DOM ready event can really work out in your favor when you set the async attribute on a script. Each situation is different, putting all the script elements at the bottom of body and forgetting about DOM Ready events may work best many cases, but making blanket statements isn't really helping anyone. I wonder how setting the defer attribute on both scripts and moving them to the head would work out in the author's benchmark.

<comment><AUTHOR> ...so why don't you try it? You can fork and perform tests...

You are right about what the spec sais and what happens in reality as far as "loading" is concerned. Modern browsers can open multiple requests to a webserver. However "Painting" and "Rendering" is another area which most developers miss its significance.

Script parsing is blocking to rendering.

Especially when scripts are included in the HEAD the delay becomes pretty apparent.

Check those slides out: <https://perf-metrics-velocity2012.appspot.com/#16> (move left/right with cursor keys).

The point of the article was to illustrate how DOM Ready is not needed when loading and executing scripts synchronously. I am not convinced that loading scripts asynchronously will result in a faster, absolute time of when the page is ready to be used by the user (events binded, complex ui that need JS are rendered.

And i am definitely not sold on the concept of polling every 5ms for the existence of elements in the DOM. Which is required for async loaded scripts if they want to be as fast as possible (and hoping that they were loaded before the elements are rendered). If I had that much a burden of manipulating an element right there and then when it was rendered, i'd plainly add a SCRIPT tag right beneath it.

</comment></comment><comment><USER7> So as a summary, Javascript loaded (except for loaded via document.write) can immediately access the DOM if the Javascript is loaded in the bottom of the body-element making the use of a ready-event like jQuery's ready-event obsolete. I have always wondered if that really is the case but haven't tried it out myself, seems that it's a green light then. :-)

</comment><comment><USER8> With one caveat: all your code that deals with positions and dimensions now needs to be smarter and handle the pushing around that comes during the loading of CSS and images. This will probably break the jQuery plugins that you're using if they deal with sizes and dimensions.

</comment><comment><USER9> This would require some discipline in a shop using this. For one thing, one would want to have a convention where write() is not used or perhaps is disabled until the operation is safe. Also, this may be a form of optimization which is applied based on profiling. An app would benefit most from this on the pages most frequently viewed the earliest. ROI would increase for mature code which is changing less frequently.

<comment><USER10> I wouldn't ever think of `document.write` as 'safe.' If it happens to be called after the DOM is ready, you'll purge the DOM and blank the page. document.write can be useful for dynamically inlining new <script> tags in a way that older versions of IE are kosher with (which is why it's common in advertising), but in almost every other case, using the W3C DOM methods (or innerHTML) is a better decision.

</comment></comment><comment><USER11> Like dynamic web apps aren't already flakey enough.

</comment><comment><USER12> worth reading in relation to this, including the impact of the CSSOM [http://calendar.perfplanet.com/2012/deciphering-the- critical...](http://calendar.perfplanet.com/2012/deciphering-the-critical- rendering-path/) and using the DOMContentLoaded event

<comment><USER13> "Marking scripts with "defer" and "async" makes an implicit promise to the document parser that you will not use doc.write, which in turn allows it to unblock DOM construction."

Would it be reasonable to ask for an opt-in feature to make document.write usage explicitly prevented, perhaps through an exception?

</comment></comment><comment><USER14> Consider'd: [http://javascriptisawesome.blogspot.com/2011/07/faster- than-...](http://javascriptisawesome.blogspot.com/2011/07/faster-than- jquerydocumentready-wait.html) ?

<comment><USER15> It should be noted that the event used in that, DOMNodeInserted, is deprecated: [http://www.w3.org/TR/DOM-Level-3-Events/#event-type- DOMNodeI...](http://www.w3.org/TR/DOM-Level-3-Events/#event-type- DOMNodeInserted)

<comment><USER14> Ah, good point!

</comment></comment></comment><comment><USER16> Let go of JavaScript.</comment>