Google's Indexing Javascript more than we thought - <AUTHOR> http://www.distilled.net/blog/seo/google-stop-playing-the-jig-is-still-up-guest-post/

<comment><USER2> I have been saying this for years, but most people have refused to believe me

Like most people a long time ago I also held the belief that robots were just dumb scripts, however I learnt that this is not the case when I had to trap said robots for a previous employer.

See at the time I was working for one of the many online travel sites; now most people probably are not aware that there is quite a bit of money that can be made in knowing airline costs. The thing is that to get this information is not actually cheap, see most of the GDS (Global Distribution System) providers are big mainframe shops that require all sorts of cunning to happen to emulate a green-screen session for the purposes of booking a flight.

The availability search (I forget the exact codenames for this) is done first, this search gives you the potential flights (after working through the byzantine rules of travel) and a costing or fare quote for your trip. This information is reliable about ~95% of the time. Each search costs a small amount against a pre-determined budget, and the slightly more over the limit (kinda like how commercial bandwidth is sold), if my memory serves it was 0.001 euro cents for each search.

During the booking phase (known as the GDS code FXP) the price is actually settled, the booking is a weird form of two-phase commit where first you get a concrete fair quote. This quote "ringfences" the fare - essentially ensuring that the seat cannot be booked for roughly 15 minutes. In practise there are a load more technicalities around this part of the system and as such it is possible for double bookings and over bookings to happen, but lets keep it simple for the sake of this story. These prebookings are roughly 99.5% accurate on price but cost something like 0.75 cents (there is a _lot_ that happens when you start booking a flight).

So with that in mind if you are in the business of trying to resell flights it can be to your advantage to avoid the GDS costs and scrape one of the online travel companies. You also want the prebook version of the fare as its more likely to be accurate, the travel sites mind less about people scrapping the lookup search.

Thus begins the saga of our bot elimination projects, first we banned all IP's that smash the site thousands of times, this is easy and kills 45% of the bots dead. Next up we start proper robots.txt and ways to discourage googlebot and the more "honest" robots, that gets us up to dealing with 80% of the bots. Next we take out china, russia etc as ip-addresses, we find that these often have the most fraudulent bookings anyhow so no big loss, that takes us up to 90% of the bots.

Killing the last 10% was never done, every time we tried something new (captua's, JS nonce values, weird redirect patterns, bot traps and pixels, user agent sniffs etc etc) the bots seemed to immediately work around it. I remember watching the access logs where we had one IP that never, ever bought products, just looked for really expensive flights. I distinctly remember seeing it hit a bottrap, notice the page was bad, and then out of nowhere the same user session appears on a brand new IP address with a new user agent, one that essentially said "netscape navigator 4.0 on X11" (this was firefox 1-2 days so seeing unix netscape navigator was a rare sight), it was clear the bot went and executed the javascript nonce with a full browser, and then went back to fast scraping.

A few years later, at the same company but for very different reasons I wrote a tool to replace a product known as gomez with an in house system. The idea of gomez and similar products like site-confidence is to run your website as the user sees it, from random ip's across the world and then report on it. I wrote this tool with XulRunner which is a stripped down version of firefox. Now admittedly I had the insider knowledge of where the bot traps were, but I was amazed at how easy it was to side-step all of our bot-detection in only a few days, I also had unit tests for the system that ran it on sites like Amazon and Google and even there is was shocking how easily I was able to side step bot traps (I am sure since they have got better, but it surprised me how easy it was).

I am not saying all the bots are smart, but my mantra since then has been that "if there is value for the bots to be smart, they can get very smart". I guess its all about the cost payoff for those writing the bots, is it a good idea to run JS all the time as a spider - probably not, does it make sense to save you from 0.75 cents of cost per search - very much so !

<comment><USER3> > I am not saying all the bots are smart, but my mantra since then has been > that "if there is value for the bots to be smart, they can get very smart".

I was once actually on the other side of the fence as you were, around 5-6 years ago (in a different industry, though). You're right, if there's value to be gained by scrapping other people's pages there's almost always a way round the obstacles.

I remember the day when my boss presented me a link to a strange-named FF extension, called Chickenfoot (<http://groups.csail.mit.edu/uid/chickenfoot/faq.html>). It allowed one to very easily write FF extensions that would programatically click on whatever links you wanted to be scrapped, all this from inside the browser, like a normal user would have done. I used to run FF with this extension installed on a dedicated cheap PC, saving the data to our servers, and from time to time automatically restarting FF because the machine was running out of memory. Fun times :)

<comment><USER4> Sure, automated browser testing is a whole industry, and I think we all know those tools aren't always used for testing sites you control.

Take a look at Selenium and Watir.

<comment><USER5> And phantom.js

</comment></comment></comment></comment><comment><USER6> It occurs to me that if GoogleBot is executing client javascript you could take advantage of Google's resources for computational tasks.

For instance, let me introduce you to SETI@GoogleBot. SETI@GoogleBot is much like SETI@home except it takes advantage of GoogleBot's recently discovered capabilities. Including the SETI@GoogleBot script into your web pages will cause (after the page load event) the page to fetch a chunk of data from the SETI servers via ajax request and proceed to process that data in JavaScript. Eventually, once the data has been processed it will be posted back to the SETI servers (via another ajax request) and then repeat the cycle. Thus enabling you, for the small cost of a page load, have GoogleBot process your SETI data and enhance your SETI@home score.

Obviously, this isn't a new idea (using page loads to process data via JavaScript) but it is an interesting application to exploit GoogleBot's likely vast resources.

<comment><USER7> One would assume that they are clever enough to have built in safeguards to prevent anything going too long, or using too much processing power.

<comment><USER8> Not just that, I would also assume that PageRank will penalise your site if a JS takes so long to execute.

<comment><USER9> Looking at Google Webmaster tools, I see a significant decline in my reported site performance starting in September, even though my site's speed has improved significantly since then by my own measures. Assuming this is due to our 'next' feature that AJAXs in, I'm going to disallow the 'next' urls in robots.txt and cross my fingers.

<comment><USER10> The Google Webmaster Tools site performance is being measured through multiple data points which can include: \- People on dial up (yes these still exist) \- People in other countries, if you have not taken care of cashing or CDN, your website might load slower for far-far away visitors This represents an average across all the data points being measured.

The way the data is being captured, is through: \- Google Toolbar \- Google Chrome \- Google DNS services

Your observations of the speed and the improvements are not always similar as the ones from the aggregated data Google has access to.

I'm not sure what you are trying to accomplish with the dis-allowment of the next urls in robots.txt. Can you explain more what your hypothesis is in this test, and how you would measure success?

<comment><USER9> I guess disallowing won't work if what you say is correct i.e. Google doesn't measure site performance with Googlebot. However, this wouldn't explain the slowdown since September unless changes were also made to include JS execution time in site performance within the services you mentioned.

Perhaps a solution then would be to trigger the AJAX on mouse over, but that seems kludgy. In my case, I need make the AJAXed content part of the initial page load anyway, for the sake of user experience. But I can see cases where Google should not be counting AJAX as part of the page load time. God forbid somebody uses long polling for example. Maybe Google is doing this in a smart way, looking at the changes after the AJAX and determining if they should count as part of the page load.

<comment><USER10> It really depends on what you are trying to accomplish if you need to worry what Google is reporting with respect to the page load time. In general, it's always good to pay attention to page load times, regardless what Google finds of it!

You can experiment with Asynchronous calls, or slow load jQuery scripts, which kick off after the headers and html framework already have been loaded.

Overall, I would not worry about the reports in Google WMC that much, just try to get faster in general.

If you are serious in delivering an ultra speedy service online, there are services which can test your site or application on multiple locations, different OS and connection speed or using a different browser. But these services are pricy, trust me on that one!

</comment></comment></comment></comment></comment></comment></comment><comment><USER11> Is there any example of a site having their dynamically generated* disqus comments indexed by google? Disqus is probably one of the most common form of ajax-generated content on the web, so if this were the case that googlebot was actively indexing dynamic content like this, I would expect to see disqus supported.

* disqus has an API to allow you to display disqus comments serverside, so some disqus implementations - think mashable is one - will have comments indexed without the aid of Javascript.

</comment><comment><USER4> I don't buy this argument. Wanting to have a more complete rendering engine for their crawler might have been a factor in designing Chrome, but I can't imagine it was in any way the driving force. The costs of developing a browser that runs well on millions of different computers and configurations are far beyond what it would take to make a really great headless version of WebKit for your crawler.

<comment><USER12> Yeah, I took that part of the original article (the one this one is citing) to be a dumbed-down explanation of googlebot's new behaviors. That article's audience were SEO people, not "engineers". It's unfortunately misleading enough that we get articles like these once you try to extrapolate from that.

</comment></comment><comment><USER13> i wonder if it also means we don't need to implement _escaped_fragment_ anymore [http://code.google.com/web/ajaxcrawling/docs/getting- started...](http://code.google.com/web/ajaxcrawling/docs/getting-started.html)

<comment><USER14> Just because GoogleBot _can_ crawl and execute/index javascript, doesn't mean that it will on your site. The best bet would be to keep them. Or take them off and see what happens. If you don't see negative effects, then you will have discovered something interesting.

<comment><USER15> Yeah I'd definitely say we should continue to follow our established best practices until G gets better at this but Josh's evidence and our continued testing on this subject is very compelling.

<comment><USER16> I'd argue that best practice in web development is not requiring JavaScript to load a page, but I'm sure that issue has been done to death in the past.

</comment></comment></comment></comment><comment><USER17> Why the assumption that "GoogleBot" is a single thing? Of course we know that google has a headless browser running, we see it's output in the instant previews, but I'm sure they still do plenty of standard crawling (and probably some half way partial JS execution and/heuristics too).

</comment><comment><USER18> > My personal favorite example of this is Google Translate, which is one of > the most accurate machine translating tools on the planet. Google almost > sacked it because it was not profitable, and _had it not been for public > outcry_ we may have lost access to this technology altogether.

I kind of missed this "public outcry", when did it happen? And if Google listens to public outcry, why did we lose Google Code Search?

<comment><USER19> When the shutdown of Google Translate API was announced, a few months ago. (Just the API, note. Not the tool itself).

It was saved because people care enough about the translation API that they're willing to pay for using it.

</comment></comment><comment><USER20> You might be able to check what the Googlebot executes by adding javascript to your site and checking the thumbnail.

EDIT: Removed comment about the bot's user-agent. The article links to a Google FAQ which answers the question.

<comment><USER21> They execute absolutely everything you put in Javascript; it looks exactly as it looks in Chrome. And it looks like it takes the snapshot after all the initial processing is done.

Javascript-heavy site with perfect snapshots: <http://goo.gl/xNUIM>

But they also index and take a snapshot of the no-javascript version: <http://goo.gl/eP84M>

</comment></comment><comment><USER22> I think they have it backwards. What if Chrome is GoogleBot? You get quality measurement on pages based on user behavior on the page. Crowdsourcing beats crawling!</comment>


Landing on a U.S. Aircraft Carrier Is About to Get Easier - <AUTHOR> https://warisboring.com/landing-on-a-u-s-aircraft-carrier-is-about-to-get-easier-1778ac6e771f#.ad3cbth81 <comment><USER2> How exactly does Precision Landing Modes [PLM] make landing on an aircraft carrier easier?

<comment><AUTHOR> I'm in awe of how the USN land F/18s at sea, at night in rough weather. When the pilots on ship are sitting down with pop-corn watching night landings in rough seas of their flight team-mates using IFLOLS... watch this video on landing on pitching decks. [0] The real problem is being able to land in all weather consistently. The advantage of Precision Landing Modes (PLM) is described this way:

 _" "Every aircraft is continually on glideslope with a stable and predictable energy state. Ultimately, this makes landing at the aircraft carrier safer," said Lt. Greg "Cinder" Blok, CVW-8 Paddles."_ [1]

Because landing is visual [2] and aided by LSO (Landing Signal Officer) the precision on landing is more variable than using PLM.

 _"All results showed benefits in touchdown dispersion reduction of more than 50 percent when compared to current landing control techniques,"_ [3]

References

[0] Video showing USN in the Pacific in rough seas [https://www.youtube.com/watch?v=v4HTXBTkcpg](https://www.youtube.com/watch?v=v4HTXBTkcpg)

[1] _" Fleet Flies Magic Carpet"_ [http://navalaviationnews.navylive.dodlive.mil/2017/03/14/fle...](http://navalaviationnews.navylive.dodlive.mil/2017/03/14/fleet- flies-magic-carpet/)

[2] _" IFLOLS—the improved Fresnel lens optical landing system—is a stack of 12 light cells, which produce a single ball-shaped image used by carrier pilots to determine the glideslope as they approach the carrier to land."_

[3] Ibid, _" Fleet Flies Magic Carpet"_</comment></comment>


If you hope to win a lottery, don't use a stolen credit card to buy the ticket - <AUTHOR> https://www.ctvnews.ca/canada/woman-charged-after-stolen-credit-card-used-to-buy-winning-lotto-ticket-n-l-police-1.4278343 <comment><USER2> almost as stupid as the guy who did a robbery without mask in his own neighbourhood, who was swiftly arrested, only to declare his utter amazement his face was visible on the cameras... after applying lemon juice! I think he made a brainfart with "invisible writing with lemon juice"

</comment><comment><USER3> I think the winnings should have gone to the victim of the credit card theft.</comment>


SporTV Play Front-End Architecture for the Olympic Games - <AUTHOR> https://blog.guilhermegarnier.com/2017/04/sportv_play_front_end_architecture_for_the_olympic_games/ <comment><USER2> Amazing!</comment>


Ask HN: Brilliant team, good product, terrible market. Need new market - <AUTHOR>

My 2 co-founders and i have been running our startup (angel funded) for about 3 years. We've released 4 products, and none of them have stuck with users. Our present one has around 5000 users in 3 months, not brilliant.
We've decided we need to stop. Take a week off, and do something entirely new. Our previous products were all in the same space-ish.
We're struggling to come up with a market/product to develop in. We're all technical, and one of us is a designer. We can create anything and sell it and win, but we have been working so much in our current space we can't think of a relevant problem to solve.
Any ideas how to fix this? We've not burnt out, just a bit tunnel vision on the current space - and we don't have the resources to take a month off to relax. We'd love someone to just say "x product sucks, make it better" - we've raised funding and understand passion etc, but the first step of a product is actually the hardest, when you have a product resonating with a small market everything falls into place.
tl;dr - tunnel vision team needs new product for new market, ideally b2c of something cool. <comment><USER2> If you've got the awesome team you say you do, then you should aggressively do some Lean Startup-style validated learning.

1) Come up with an idea

2) Create a Wordpress landing page for that idea (<http://news.ycombinator.com/item?id=1697302> ). Sure WordPress landing sites are ghetto, but if it's good enough for groupon...

3) Buy ads on Google/Facebook to drive traffic to your landing page. If you're angel funded, you're racing against time, not money, so buying targeted traffic is a way to get validated learning in a shorter amount of time.

4) If people are responding to your landing page (clicking through the ads, signing up for an email, etc), make a survey to find out their thoughts on your idea, it's competitors, and what they currently use to solve that problem.

5) When these marketing efforts help you find an audience that is reachable and responsive to your idea, start building! When you have something, monitor usage and solicit feedback to find the next thing to build. If something isn't getting used, scrap it. Repeat as often as possible.

If you're looking for a small tight market, I would highly recommend using Facebook ads. You can target by interests and groups, which means you're showing your ad to people _who have self-selected_ to show their interest in the market you're looking at.

I'm making an online trading card game, and I made a survey on Wufoo, posted it on my site, then made a FB ad pointing to the survey along with a $25 Amazon gift card to a random person that completed it. I spent $75 on ads targeted to Pokemon, Yu-Gi-Oh!, and Magic: The Gathering players, got 50K impressions, over 200 completed surveys, and this was in only 3 days. $100 total got me validation of my idea plus other useful information, and 50 or so people signed up for my newsletter and to playtest when I'm ready.

<comment><USER3> Great message, but one thing threw me for a loop. "got me validation of my idea" <\-- why? 50k impressions to 200 surveys is a 0.4% conversion rate, and I'm sure at least some percentage of those respondents were not entirely positive. Where did the validation come from? Or was it the $0.50 CPA for your "users" (survey completers)? (Which seems low, assuming you can muster something like $1 revenue per user on average.) (edit: Or maybe you can think of the CPA as $2.00, since only 50 out of the 200 people signed up for the newsletter.)

<comment><USER2> My "idea" is an online trading card game with a science and technology theme. So the "validation" I got from the survey results were things like:

\- I asked how interested people were in playing games with different themes (medicine, business, ancient history, science, etc). Science and technology had the highest interest, both in magnitude (very interested) and percentage of favorable

\- I was worried that people would not try new games, but would stick to the ones they already played. This was an issue for a minority of people but most people were excited about trying new games

\- I was worried that people wouldn't want to play TCGs online. This was a valid concern, more than half of users had concerns. I got stats on what people like about in-person gaming and what they dislike about computer gaming

\- I didn't know if people spent money on online TCGs. Turns out they do, but not as much as cardboard games.

This was not validation of a specific business model - this was answering the question: Is there a plausible market for this? I'll again survey for specific feedback on _my game_ when it's ready to show, but if I found out things like "I'll never play another game besides my favorite" or "No one spends money on any online TCG", then I would have to reconsider.

You can see the whole survey here: <http://geekstack.com/survey-for-trading- card-game-players/>

</comment></comment></comment><comment><USER4> _We can create anything and sell it and win_

Show. Don't tell.

If you've got 5000 users in 3 months, surely you can get a lot more. You clearly know your domain and have built something which a lot of people found useful.

Build on your strength, don't jump ship because you are bored. Stick it out.

Most importantly, please please don't try to hide your projects. Just link to them, get some more users. I really don't understand why people do this on HN. You are just losing out on loads of useful feedback and connections.

</comment><comment><USER5> What are your current products/markets? I have found, the best way to go is generally tweaking the work you have already done vs. scrapping and going towards something completely new because you will run into the same unknown pitfalls every time.

</comment><comment><USER6> Please make something to extend some of the more popular web-based time tracking systems out there. We already got sites like tick/harvest, etc... which does a great job at project based time tracking.

But a project compromises of;

\- benefit management. what each employee is entitled as part of the company, and how can they make the claims, etc.

\- training management. how many training hours each employee is entitled, and how they can start to apply for training course, etc.

\- leave management. how to apply leave in the system and sync it to the time tracking system, etc..

All these areas have plenty of opportunities to grow. If you have the skills, do make something in these areas.

</comment><comment><USER7> Go build a cross platform mobile device game. There is a whole industry around game development and tons of new "physical" games are invented each year. Take one of those ideas and port it to tens of millions of customers with a cool UI and experience. Then tie it into the social online world. Partnering with a game developer should be pretty easy as they probably aren't expecting to be the next Monopoly, but instead do it for love.

</comment><comment><USER8> Look at the hodgepodge of answers I got to this question: [http://programmers.stackexchange.com/questions/9371/what- is-...](http://programmers.stackexchange.com/questions/9371/what-is-the- proper-way-to-create-requirements-documents)

I was surprised that there wasn't a polished product in this space.

<comment><USER2> Requirements are mostly about trust between parties and understanding the problem domain and business needs by developers, and technical costs by business. It's much more about relationships and communication than documents.

</comment></comment><comment><USER9> One option (not saying it's the best) is you could try unashamedly following someone who's getting good early traction. You can be the mouse that gets the cheese if you can learn the space and out execute them.</comment>


US and UK suspend non-lethal aid for Syria rebels - <AUTHOR> http://www.bbc.co.uk/news/world-middle-east-25331241

<comment><USER2> So the Arab states were propped up by the Americans and the British following the 1st World War, left to their devices until the 2nd World War and then left again. A couple of decades later, amidst a few revolutions, with more western influence dictating the regions stability - the Arab Spring kicks off, with the population coming out of the woodwork demanding real democracy. Their actions are latched onto by the Islamists looking to invoke Sharia Law, and by Western powers to have some form of presence and support amidst the growing influence of Russia, China and within the next decade or two - the Indian sub- continent.

Somewhere along the lines the locals, the Islamists and Western Powers realise they want the current government overthrown, but soon thereafter their differences come out in what they want the end result to be.

Syria is already is a flashpoint for a Russian backed government battling Western backed rebels, and potentially any other hard-line Islamist state / government / group backed Islamists blending in with the rebels.

I'm not even sure who's playing what game anymore - the west is actively partaking in removing leaders it helped come into power - and has already done so with a few.

The whole region is ____ed and we are nothing more than mere spectators for whatever end-goal whoever has in mind, provided they actually get what they want.

To think, in about a century and a bit the regions oil reserves will have been depleted and it will be forgotten about - nothing more than a dot in history of leaders coming and going, a populace that never got the opportunity to develop or progress and will have severe bitterness of the powers that sought their own means at their expense.

\---

The first few statements are off the top of my head so please someone correct me if I went wrong, but thats how I remember the gist of the history of the region.

<comment><USER3> > "So the Arab states were propped up by the Americans and the British > following the 1st World War"

Actually, The United States had little to nothing to do with propping them up after the First World War[1] and the Middle East became temporary colonies of the British and French. The actual agreements behind that started not long after the outbreak of the war and was a joint British/French/Russian operation. They came up with the decisions about how to carve up the former Ottoman Empire (much of that was done prior to the US entering the war). Syria itself became a French Colony.

The US was a minor player (comparatively speaking) in influencing the outlook of post war Eastern/Western Europe and the Middle East. It would have probably been to the long term advantage of the victorious European Powers to actually listen to the initial suggestions of President Wilson[2] and The United States about terms for peace with Germany. If they had, we would have most likely prevented a Second World War, as Wilson recommended against harsh punishment and restitution towards Germany.

If interested in knowing more, I'd recommend reading the writings of T.E. Lawrence and/or watching Lawrence of Arabia. There's a number of other good sources as well, but those are a starting point.

[1] [http://en.wikipedia.org/wiki/Partitioning_of_the_Ottoman_Emp...](http://en.wikipedia.org/wiki/Partitioning_of_the_Ottoman_Empire)

[2] [http://en.wikipedia.org/wiki/Woodrow_wilson#Treaty_fight.2C_...](http://en.wikipedia.org/wiki/Woodrow_wilson#Treaty_fight.2C_1919)

[3] [http://en.wikipedia.org/wiki/Fourteen_Points](http://en.wikipedia.org/wiki/Fourteen_Points)

</comment></comment><comment><USER4> Sounds like the rebel groups most in line with Western values just got pushed out of one of their bases by more Islamist rebels, who might have grabbed some of their weapons. Essentially, since the groups we support the most lost a minor battle, we'll give them less weapons and equipment, thus decreasing their influence even more, and making it even less likely that the end result of this war will be a government that has any interest in democracy.

God knows we have plenty of reasons for being weary of supporting causes in the middle east, but the more we step away, the more the fight will be between Russian, Chinese, and Iranian backed dictators and Saudi, Al-Quada, and other Sunni extremist backed Islamic fundamentalists.</comment>


On Deep work: Rules for focused success in distracted world - <AUTHOR> http://www.stringcode.co.uk/on-deep-work/ <comment><USER2> Mods, can we instead link directly to Cal Newport's page? [http://calnewport.com/books/deep-work/](http://calnewport.com/books/deep- work/)

<comment><USER3> This request makes no sense. stringCode is not Cal Newport. This is part- review of Newport's book, and part-blog about it's impact on the reviewer. And seeing that you're not the original poster, why is it appropriate to change the link to point to the book's author?</comment></comment>


USA Today goes social: lots of complaints in the comments, but it's actually pretty well done - <AUTHOR> http://www.usatoday.com/news/community-features.htm#uslPageReturn <comment><USER2> USA Today completely misses the boat.

The reason the traditional first date used to be dinner and a movie was because the movie gives you something to talk about during dinner.

People read the paper for the same reason. It gives them social currency when they're talking to their friends. If you want to design a social community around your product, you have to design it so that enhances your product's core value proposition.

Instead of being about all about the reader, this new redesign is all about USA Today. The website looks like it was designed by a team of DMV employees and then sprinkled with banner ads.

Not to mention that it's five years too late.

</comment><comment><USER3> It's a positive thing that they're attempting to involve their community, but it's another thing when they don't understand their community in the first place.

<comment><AUTHOR> I think the question is, are the complaints they're getting facebook news feed type complaints? Or netscape.com relaunch type complaints?

One goes away when people get used to the newness. The other destroys any hope you had for a community.</comment></comment>


Everything Should Have An API - <AUTHOR> http://words.scottjackson.org/post/232078864/everything-should-have-an-api <comment><USER2> I agree with 95% of what he says, but I'd make one change to the Unix philosophy: text files aren't the best primitive.

Text files have a super low-level API (stat, seek, read). I think the system he describes would be much more usable if he picked slightly higher-level data abstractions. I would recommend Python's built-in types: list, tuple, dict, set. Most data can be easily represented by these types and they provide a higher level API than streams (files), which means the barriers to using them are much lower.

<comment><AUTHOR> That's probably fair. I honestly don't think we're at the point where we can abandon text files altogether, but you're right -- a lot of data could definitely be stored in slightly higher-level data structures.

I still _really_ like text files though :)

<comment><USER3> Text files are just a wire protocol. It would be nice to have a standard (similar to http) for what you send over the wire in general.

Great post by the way, very thought provoking.

</comment></comment></comment><comment><USER4> The printing press enabled more people to read, though still few could write. The net has enabled more people to write, though still few can program. A net where everything has an API is an extremely helpful step in creating a system which enables more people to program.

</comment><comment><USER5> The scenario descriptions are distinctly reminiscent of Berners-Lee's scenario descriptions for the 'semantic web' several years ago. Of course, that is now rather out of fashion, but not entirely justifiably. The 'semantic' angle was somewhat fanciful, but the core idea was _the data web_ : It is one thing to put APIs on everything, but another for all the data types and formats to work together. That was more the aspect that RDF was addressing. It is neither thoroughly convincing nor completely apposite, but worth factoring in to the consideration anyway. (The essential RDF concept was elegant, whatever anyone says.)

</comment><comment><USER6> Having a good API for everything would be almost as good as The Singularity (tm), but making a good and secure API is very hard and comes with a massive hidden cost.

1) An API can require even more planning to design than the application or service itself.

2) Mistakes made in the design or implementation of the API turn into liabilities you can't solve without breaking users of the API.

3) The cost of maintaining an API so that it and its documentation doesn't fall out of date with the app/service can be as high (and sometimes even higher) as the cost of maintaining the app itself.

4) It widens the attack surface.

And those are just the ones off the top of my head.

<comment><USER7> I agree with all of your points, but I think many of these costs are minimized by the other point mentioned in the OP: following the UNIX philosophy. The smaller and simpler your service is, the better. If you have a service that can't be accessed with a simple API, you probably have more than one service, and would benefit from splitting it into multiple APIs.

Another thing worth mentioning is the possibility of building your API first, and then building your service on top of it. If you are providing a web service, then it [usually] boils down to data access and management. Figure out what data you are managing for your users and what you need to do with it, then build your API around that. If you plan to have a public facing API from the start, this approach makes sense (in the same way that using javascript only for progressive enhancement of a working site makes sense).

Of course, because you are an "internal user" of the API, you can bend the rules a little more than the average client, but the less you do this, the more useful your API is.

Regarding points 2 and 4: if public APIs for data access become the norm, this will simply become the cost of doing business. Secure API's aren't _that_ difficult, and updating an API while maintaining backwards compatibility can be done, especially if you transition these changes and notify all broken clients.

<comment><USER8> For the UNIX philosophy I like the idea of small, useful tools loosely coupled. A big part of UNIX is that there are a lot of little tools that do _one_ thing well. The smaller the tool is, less attack surface there will be, and each tool can interact with a common authentication/authorization interface to further simplify things.

</comment></comment></comment><comment><USER9> Should creating email accounts have an API? How would the captcha be presented? How about adding reviews of products?

<comment><USER10> Don't captchas and open APIs fundamentally oppose each other?

The point of an API is to have a clear interface so that programs can be built on a service, and captchas _exist_ to try to block bots from using services.

<comment><USER9> That's my point. Not everything <should> have an API since there are so many things we put captchas around.

</comment></comment></comment><comment><USER11> I want, I want, I want. How about you propose how this should be done? It's harder than it looks, and no, Unix is not the one-and-all answer to everything.

Take one of his example scenarios:

"When my alarm clock goes off in the morning, get the latest tech news podcast (downloaded overnight) from my computer via WiFi or a wired network connection. Play the podcast through the alarm clock's speaker/s. If I hit "snooze" on the alarm clock, pause the podcast. Once I stop the alarm, remember where I was up to in the podcast and give that information back to my computer. Now when I go to listen to the podcast while I'm eating breakfast, it will continue playing where it left off."

This has a number of hidden assumptions:

* What is "the latest news podcast"? One user-selected source, or a blended filtered combination, or a crawled set selected based on previous behavior?

* How does a (not _the_!) computer know how to talk to the alarm clock? I hope I don't have to create a home network myself, cause I know my Mom certainly can't. How is this connection authorized? What happens if the power goes out on the computer - what will the clock do? What if the clock batteries fail - will the computer play the podcast instead?

* Is the saved position synced only the one source computer (I hope not). What if the network to the computer is down - can the alarm clock sync somewhere else?

* Can I resume listening to the podcast on a different computer at breakfast? My partner/sibling/friend's computer? How do I log in? How is the position data saved along with my identity? How does the breakfast computer address the first computer, or the alarm clock?

The problem here is not access to the external APIs (not that hard) but federation, identity management, authorization, privacy considerations, and addressing between these disparate systems. Proposal adopting existing standards for these requirements, or new proposals for standards are, of course, appreciated.

<comment><USER12> On the existing standard front, there is Universal Plug and Play (<http://www.upnp.org/>) that aims to "to enable simple and robust connectivity among consumer electronics, intelligent appliances and mobile devices from many different vendors." I haven't played with this stuff at all, but there appears to be fair amount activity in this space: <http://en.wikipedia.org/wiki/UPnP_AV_MediaServers>

I don't know how much of the example scenario you could implement within the confines of UPnP, but perhaps it could be part of the solution.

</comment></comment><comment><USER7> As previously said, the financial incentive is simply not there. As much sense as it makes for you to have easy access to your information, there are too many businesses that rely on controlling that access.

On the other hand, tools to do this with almost any site already exist. RSS is common enough, and where it doesn't exist, screen scraping isn't that hard (and getting easier all the time). If I'm not mistaken, this is the exact problem that Yahoo pipes are trying to address.

Of course, sites that require a complicated login process (banking sites) are still out.

<comment><USER13> I think Twitter is an example that owes its success to its API; it depends what your business plan is.

As more companies become Google-esque data hogs, APIs will probably be encouraged.

</comment></comment><comment><USER14> As right as you are, making an API for your service so people can replace your crappy interface with their own good interface probably isn't the best business decision. Online banking sites can display ads. The bank can't inject ads into another client though.

Also consider that the bank would get blamed in popular opinion if someone wrote an insecure client that was determined to be leaking data.

<comment><AUTHOR> I think it could be done -- I mean, Pine's an example of a secure client application that uses a kinda-API (SMTP) to deliver data.

If we don't make APIs because someone _might_ write an insecure client, would anyone make any APIs ever?

I do agree with you though :) It's probably not going to happen in the real world any time soon. C'est la vie.

<comment><USER14> I'm sure it could be done.

But consider for example if Bank of America did this and someone created an awesome client app that basically became the main way anyone used their Bank of America account. If a bug is discovered in this application that means everyone's data is no longer secure, are users going to blame the app creator or Bank of America? Can Bank of America afford to take that chance?

<comment><AUTHOR> You're right -- that's almost certainly the reason you don't see stuff like this happen.

Though, didn't Bank of America make that iPhone app a little while ago that allowed you to photograph a cheque in order to deposit it? They're obviously OK with sending financial data around. I guess maybe it was OK in that case because they were the ones that wrote the app?

In any case, you're right. Security is definitely the reason this kind of stuff hasn't been implemented. I still think that it's possible to write an open, secure interface to things. Email is the example I fall back on here. Like I mentioned above - sensitive information gets sent around in emails all the time, yet there are countless email clients and they're almost all secure (I say "almost all" to cover my ass if there's some that aren't). I think that if it can work for email, then maybe it can work in other domains.

<comment><USER7> Unfortunately, email is a terrible example. While it's possible to have a modicum of security in regards to receiving your e-mail, spoofing emails is trivially easy (unless you use cryptographic signing, which isn't well supported by the majority of clients afaik)

</comment></comment></comment></comment></comment><comment><USER15> For my own personal use I think text based interfaces (with pipes) are superior, the higher level pluggable components in AppleScript/Automator seem more usable in general.

That said, there was an excellent article recently with the experiences of a trainer in getting people to use *nix, who found the minimalistic "command mode" more usable due to it's text oriented simplicity.

</comment><comment><USER16> I want a build system with an external API.

</comment><comment><USER13> API is kind-of a misnomer in this case. API implies each service has its own set of rules, the data is open, but the access is different.

Everything should talk to one another, but a service's name or url should never have to appear in any code. That's the user's job to input.

</comment><comment><USER17> Yes. I was just doing my college applications and thinking this exact thing. The Common Application would look like a piddly idea if the college application process just had an API.

</comment><comment><USER18> Personally it sounds like a security and privacy nightmare to me.

<comment><AUTHOR> Maybe. I think that if we get enough smart, awesome people working on it, it could be secure, safe and open all at the same time.

</comment></comment><comment><USER19> Couldn't agree more. This is what my next startup is based on.

</comment><comment><USER20> They shouldn't just have an API, they should use the same APIs where possible. Each site having it's own API is a major barrier.

</comment><comment><USER21> for anyone who hasn't seen it, [http://www.ted.com/talks/tim_berners_lee_on_the_next_web.htm...](http://www.ted.com/talks/tim_berners_lee_on_the_next_web.html) provides a nice context

</comment><comment><USER22> well, most data is exposed via a website, so if the api doesn't exist, just mechanize it. its not really feasible to rely on goodwill for these grand visions.

</comment><comment><USER23> This is ridiculous, we can't just let people have access to their data!... i mean OUR data. Just imagine, they may learn things we could have forced them to pay absurd fees to get! :P

</comment><comment><USER24> But at what cost?</comment>


Real world DNSSEC+DANE for secure inter-domain mail transport [pdf] - <AUTHOR> https://static.ptbl.co/static/attachments/169319/1520904692.pdf <comment><USER2> Standing reminder that friends don't let friends use DNSSEC, because it is crazy and provides no real security. With that in mind, it's fun to dive in and see where the problems are ...

Fundamentally this DANE idea swaps out the problems of a PKI CA hierarchy, the kind your browser is using when you actually log into gmail to read and send your e-mail, with the problems of DNSSEC.

The biggest problems with PKI are that there are too many CAs, who can each go rogue, and that X509 is a horrifically awful format that takes a gigantic TCB to parse. Mitigations include browsers firing CAs and severely impacting their business (so effectively, there is regulation), certificate transparency and more and more testing. Either way you pretty much have to deal with these problems today though, an awful lot of e-mail is sent/received over HTTPS.

The biggest problems with DNSSEC are that DNSSEC doesn't actually protect DNS queries at their most vulnerable points, that you have to trust your TLD provider, and the root operators completely, with no effective means to fire them, and finally that they almost certainly are using embarrassingly stupid crypto that even most ICOs could avoid. This very deck has several 1024-bit RSA keys, right there on a slide with a February 2018 timestamp, as if it's not an incredibly obviously stupid thing to do since the LogJam paper was published, or you know, when everyone stopped using them for TLS years ago.

Actually wait, there's even a 512-bit key on another slide, using alg-5. That's RSA + SHA1!! What's the point of using all of those SHA256 MACs at the TLS Layer if everything depends on utterly breakable RSA plus SHA1 anyway? It's hard to take this seriously.

PS. Yes, this is also my best impression of tptacek

<comment><USER3> It's still an improvement over the way things where before (nothing at all, and complete blind faith in every part of the DNS and internet routing) and arguably better than CA which isn't that useful for email.

Furthermore, the browsers replace the TLD in who you trust with CAs and in many cases it's practically difficult (or impossible) to "fire" your browser.

I will agree that no one seems to use good crypto with DNSSEC though. The only people I know using it are using 512 bit RSA which is pretty horrifying.

<comment><USER4> >it's practically difficult (or impossible) to "fire" your browser.

You can reconfigure your trust store, and you can distrust a CA in a few clicks.

<comment><USER5> Not only that, but Firefox and Chrome have "fired" some of the largest, "most important" CAs on the Internet. We know there's no TBTF problem with the Web PKI. But there is literally no way to fire a DNS TLD operator. In this scheme, they are _prêt-à-porter_ TBTF.

<comment><USER6> > We know there's no TBTF problem with the Web PKI.

Hardly.

Comodo is still around, and Symantec took years to be gradually distrusted. Hell, even the WoSign/StartSSL debacle started in Jan 2015, and they weren't removed before Jan 2017.

<comment><USER7> The WoSign issues became public around early September 2016 (maybe last week of August): [https://wiki.mozilla.org/index.php?title=CA:WoSign_Issues&ac...](https://wiki.mozilla.org/index.php?title=CA:WoSign_Issues&action=history)

<comment><USER4> not to mention that wosign came out in early 2015. [https://news.ycombinator.com/item?id=8982013](https://news.ycombinator.com/item?id=8982013)

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> Would adopting web-style security model really be that horrific for email too. By that I mean that the servers would have private keys & certificates matching the domains of recipient addresses[1] and using SNI to solve vhosting. Sure, most mail providers might not be equipped to do so right now, but it is hardly an impossible thing to ask for.

[1] Heck, if you are really clever then you could also ask sending server to present a (client) certificate matching the sender address domain.

<comment> ietf-dane Why is SMTP not like the web: [https://tools.ietf.org/html/rfc7672#section-1.3](https://tools.ietf.org/html/rfc7672#section-1.3)

<comment><USER8> Yes, I've read that and disputing if the things mentioned there are really such blockers. I was specifically referring to the point 1.3.2 in my suggestion to use "envelope recipient domain" for verification as I really do not see it as impractical as the RFC supposes</comment></comment></comment>


Review My Startup: TweetHook - Twitter Search ping service - <AUTHOR>

I have spent the last several months bootstrapping my own company and developing TweetHook.com to be a Twitter Search ping service to push search results to webhook/callback URLs.
There are several levels of paid plans. There is no free plan mostly to avoid spammers/abuse, but also because I am running on a tight budget and don't have tons of cash to burn. I realize I may turn away potential users with this "paid only" barrier, but the users, developers, and businesses that would want this service should be willing to pay for it. I already have a few very happy clients from my beta testing phase that are sticking around as paid customers.
Please let me know what you think! <comment><USER2> Looks very, very useful. I've been looking for something like this.

A few notes on copy:

At <https://tweethook.com/th/webhookinfo>, under 'POST data arrays', 4th paragraph, you've got "concatinated." That should be "concatenated".

At <https://tweethook.com/privacy>, you probably mean "Privacy" and "Children Under Age 13". Also, the two links to the TOS point to /privacy, not /terms.

At <https://tweethook.com/terms>, you've got what is supposed to be a permalink to the Terms under the "Changes to the User Agreement" section. It's not a link and the URI points to something that doesn't exist.

Best of luck sir.

<comment><AUTHOR> All fixed. Thank you very much for catching those.

</comment></comment><comment><AUTHOR> Clickable link: <https://tweethook.com/>

Press release: <http://pitch.pe/27243>

</comment><comment><USER3> It is an interesting app and I think a lot of twitter developers will find it useful.

On the UI side you need to rethink things. Twitter apps tend to be on the really well polished side, and your site is lacking here. There are lots of different fonts and effects, alignment issues, things like that. Maybe hire a designer? Or if you want to do it yourself, really simplify things.

</comment><comment><USER4> How did you structure your pricing? Is it based upon cost or are they random?

<comment><AUTHOR> The prices are based on cost. I spent a lot of time determining price points based on bandwidth, resource usage, etc. Compared to other data providers like this, I have found these price points to be much less expensive.

</comment></comment><comment><USER5> Do you have access to the firehose feed?

<comment><AUTHOR> No, I do not.

</comment></comment><comment><USER6> congratulations on the launch!

</comment><comment><USER7> Over the next few months this feature will be added to <http://tweetminer.net> \- one way or another.

Of course it would be great to use a service like this to do the heavy lifting rather than code it myself...

But why would I pay $400 per month for 500 searches per month... when

a) 500 searches per month cost of $4,800 per year

b) 500 searches wouldn't be enough to service my target user-base of 10,000 users.

c) I could roll my own in less than 2 weeks

To be viable for an app developer like myself - the pricing structure would need to make it impossible for me to even consider rolling my own.

BTW - I'm not trying to dis the service, looks nice, just pointing out the pricing structure seems to be wayyyy too high</comment>


Ask HN: Do you work with a "hands-on" VP/CTO? - <AUTHOR> >500-employee company, 100 of those in engineering/IT roles.
Both CTO and VP are "hands-on" meaning they will get into the nitty gritty details of how we develop software or deploy infrastructure.
No roadmap to be seen. No high-level goal. No security policy in place. No offboarding process. No onboarding policy. No central auth server. A few different job schedulers. Few different container orchestration solutions. etc
My theory: they got promoted pushed to management roles (first as founder becoming CTO, then manager becoming VP) and now the salary is too high to go back. <comment><USER2> 700 person company, half of which are software engineers. Yes our CTO is extremely hands on, and one of the most insanely gifted developers I've ever met. He was the first hire at the company and has led engineering since.

This works as he is a strong, opinionated, intelligent, yet empathetic person who leads the charge on our engineering standards from the front line. He and the org have realised the problems that come with this approach, so have a head of engineering and the engineering leadership board (including the CTO) that fulfill the other functions of a CTO, such as the people management, resource planning, general process and procedure.

This leads to a great situation where we have a gun of a technical hands on CTO that rallies the troops, but the other typical CTO functions aren't neglected either

</comment><comment><USER3> Sadly it sounds like your options are to leave or to stop caring so hard about your job. I wish I had better advice, I am also familiar with micromanagers who make bad macro decisions. They are too into how work makes them feel. Some people actually enjoy sitting in meetings and watching people carry out their arbitrary commands, because it makes them feel powerful. On the other hand they don't enjoy asking whether the way they are spending their time is useful, probably because it would make their entire self-worth collapse.

</comment><comment><USER4> >No roadmap to be seen. No high-level goal. No security policy in place. No offboarding process. No onboarding policy. No central auth server. A few different job schedulers. Few different container orchestration solutions. etc

Have you asked about these things? Maybe they are there, just not visible to _you_. A 500-employee company is pretty big already.

<comment><AUTHOR> Engineering is only 100 people or so. I've been here for a while, there are no such things. Unless they're making a point to exclude it from me only, for years.

</comment></comment><comment><USER5> > 500-employee company, 100 of those in engineering/IT roles.

> No security policy in place.

How is that possible? Are you saying there's no security policy or a minimal security policy? I can't imagine how a company gets to 500 employees without a single security incident that forces your hand to create some kind of formal policy.

<comment><USER6> Let's be honest, Enterprise sales companies have to get a soc2 or equivalent. Otherwise, if your customers aren't asking for it, it's easy to go without. Especially at under 1000 people. I've worked at companies with ~200 employees that didn't have a security policy and were incredibly insecure. 500 is typically when b2c will start thinking about security programs, but the reality is that unless security is revenue generating it becomes incredibly hard to build out security policies

</comment></comment><comment><USER7> A 500 person thats missing what you've posted -> they're in trouble.

Even a 50 person company without that is in for a bad time.

Do they have good advisors?

</comment><comment><USER8> Is your company on auto-pilot and making good money? If not, sounds like it's going to crash and burn.

<comment><AUTHOR> It's funny that you mention this because yes, it is on auto-pilot and making good money. The costs of maintaining systems is slowing growing. If it crashes it will be in the long run. Meanwhile, it's a bit depressing in certain ways.</comment></comment>


T-Mobile G1 - <AUTHOR> http://gizmodo.com/5053612/t+mobile-g1-walkthrough-shows-smooth-user-interface

<comment><AUTHOR> The interface actually exceeds my expectations. And the keyboard makes such a difference.

</comment><comment><USER2> It's terrible , cannot even compare to the iphone !!</comment>


Shale Hacks Diesel Using Gas-to-Liquids Plants - <AUTHOR> http://www.bloomberg.com/news/2012-10-24/nazi-technology-turns-cheap-shale-gas-into-sub-2-diesel-energy.html

<comment><USER2> Shell has spent decades working on this tech after south africa and Nazi germany used this technology when they were cut off from oil supplies. Since I am an actual Energy Economist, I'm glad to see this happen. I think Gas to Liquid technology could be critical in some possible futures as a method to store energy in a dense compact fuel that works with massive existing infrastructure.

<comment><AUTHOR> _a method to store energy in a dense compact fuel_

\-- Energy density is almost always overlooked.

<comment><USER2> Unless you hang out with some peak oil guys, they scream about energy density - until you get something that creates dense energy then they scream about how efficent it is.</comment></comment></comment>


One Page Love – One Page Website Inspiration and Templates - <AUTHOR> https://onepagelove.com/ <comment><USER2> To all the people complaining that One Page Websites are bloated and brittle js monstrosities: turn off JS on your browser and access the site. Sure, there are lots of problems but some of those sites are actually quite light on loading and a pleasant UX experience (e.g: the Werner's nomenclature of colors).

<comment><USER3> Thanks for the comment Diego. I really should write a case-study on this but Parallax Scrolling really put the One Page website trend on the map BUT at the same time paved the way for so many more unnecessary effects in web pages. And I think this is what the majority of Single Page website haters refer to and are not wrong. But yeah, one can't generalize saying all One Pagers have heavy JS and effects. There are some stunningly simple, well executed Single Page websites out there.

</comment></comment><comment><USER4> I'm not entirely sure what to make of this post, but the link to Werner's Nomenclature site was very interesting.

<comment><USER3> Hey - I'm Rob who started OPL back in 2008. I'm personally fascinated with One Page websites and how designers/developers tell a story within them but I can safely say Single Page websites are not for everyone. I am absolutely not trying to convince every use-case to use a One Page website instead of multi- page. The essence of a One Page website starts with identifying attention spans are diminishing online. You get to the point with the perfect amount of text and imagery. An example I always use is traditionally people want a dedicated testimonials page with 12 testimonials. One Pagers encourage selecting your top testimonials only that highlight your product/service features. This effort organically creates a better website, and in-turn helps with better conversions. It's all about getting to the point with your message.

</comment></comment><comment><USER5> This is an extensive listing. On mobile, the page keeps loading more and more themes when scrolling through, I can't get to the end!

<comment><USER3> Hey! Yeah ha! I've collected 7000 Single Page website references since 2008, safe to call it a little obsession:) Really interesting use-case though on mobile, I'm unsure if numbered pagination is better as an indicator how deep you've gone? On desktop I LOVE the infinite scroll though. Especially effective when browsing tags and categories. You really get inspiration MUCH faster than clicking to page to next...

</comment></comment><comment><USER3> Hey! Thanks @hunvreus for posting about One Page Love. I'm Rob Hope who started the website back in 2008. There are around 7000 references now and I'm adding around 1-2 new websites per day. If you want to know anything about Single Page sites hit me up in the Comments, Twitter (@robhope) or Email: email@example.com Cheers! :)

</comment><comment><USER6> Please don't make more one page websites, every single one I come across fails me in one way or another and makes my job as a user more difficult.

<comment><USER3> Hey! There are definitely some "flashy" examples where designers/developers try push the limit (with the current tech available) but there are hundreds of practical examples of excellent, lightweight One Page websites. I personally love the App category as there are features displayed much better than app stores but still simply link to the download of them: [https://onepagelove.com/gallery/app/](https://onepagelove.com/gallery/app/)

</comment></comment><comment><USER7> No one loves one page websites except for-profit institutions who prioritize cost savings over user experience and accessibility. Not a single one of these will display _any_ information as a fallback when javascript is not available. And even if you do blindly let all javascript run it's still near infeasible to link to specific sections or content.

Templates for people that don't care if people can use their site but just want to put something up that will render in chrome.

<comment><USER8> Single page apps and accessibility are completely unrelated concepts. With any app, single page or otherwise, you must take a11y into consideration when designing and coding the site. Again, nothing really specific to SPAs here. What you're complaining about here are designers and developers that don't strongly prioritize (or understand) UX and a11y. I've seen plenty of more traditional multi page/server driven apps with terrible UX and no concern for accessibility. Many, oh so many of those, in fact. And I've also seen plenty of SPAs with excellent care for UX and a11y. So, you seem to have an axe to grind here, as nothing you've said here is really specific to SPAs. As far as disabling JS... well, I suppose for a static site or one with a simple form that's certainly something to consider. But otherwise: it's 2019. Let's not hold the all web apps back just because some feel the need to block JS. Responsible developers and designers: that's the goal.

<comment><USER9> > As far as disabling JS... [...] it's 2019.

Ads, tracking, and things like Magecart are still a huge scourge. Meanwhile that SPA isn't such a big "value add" to my life. I agree that devs working with JS need to be responsible, need to accept downsides and the upsides. And they've repeatedly proven they can't be trusted with this when business interests conflict.

I realise this probably makes me a luddite, but blocking or tampering with JS, however crude that is, is one of the only avenues left for controlling web content as a consumer.

<comment><USER10> Go ahead and control JS and your web experience. Just don't complain when sites break because of this and don't be surpriyed when this very niche case is poorly supported.</comment></comment></comment></comment>


Campaigns go digital, but ballot petitions can't - <AUTHOR> https://www.sandiegouniontribune.com/columnists/story/2020-04-12/column-campaigns-go-digital-but-ballot-petitions-cant <comment><AUTHOR> There's already a signature verification process. What difference does it make if the signature is inked or not?

Even without social distancing, we should all want more access to democracy.</comment>


New Social Polling Startup - <AUTHOR> http://www.polladium.com/

<comment><USER2> Cool idea for a Website. I like that I can add this to my blog and my theme for my website sticks throughout the visit on polladium.com. I also like that my polls can make it on the homepage of polladium.com to increase exposure to my blog. The theme support could be better. I'm also still a bit confused on how it all comes together, but I'll poke around a bit more.

Tell your friend, nice job so far, but work on simplifying the site. It has a lot of potential, but people won't stick around if they don't get it within 10 seconds.

</comment><comment><AUTHOR> I'm a long time reader/lurker, first time submitter. My friend created a startup and I thought I'd help him out by posting it on Hacker News for some feedback.

His website is called polladium.com and it borrows features from the social aspects of Twitter and the core of Reddit to define social polling. Please take a look and let him know what you think. Comments, suggestions, likes, dislike?</comment>


Seoul's full cafes, Apple store lines show mass testing success - <AUTHOR> https://www.bloomberg.com/news/articles/2020-04-18/seoul-s-full-cafes-apple-store-lines-show-mass-testing-success <comment><USER2> To be clear, the Korean government officially still recommends social distancing. They are especially worried about the family picnics mentioned in the article, and have officially closed many public parks in response.

To repeat: don't take this as a sign that Korea believes it has successfully managed the outbreak.

<comment><USER3> The article and discussions about Korea seem to mainly revolve around testing and contact tracing. These are complex and expensive, and substantially more difficult to manage in a country like the US which is 7x the population, and is somewhat decentralized (50 states).

Meanwhile, the on the street observations in the article are: masks, masks, masks. Masks are cheap and easy. So let's start with masks, masks, masks first while we figure out and ramp up testing and contact tracing.

 _the first Apple store to reopen outside China had lines snaking out the door as many South Koreans -- almost all wearing masks_

 _At Han River park in Seoul's Banpo district, families -- also in masks -- were having picnics_

 _including requiring voters to wear masks and disposable plastic gloves while casting their ballots_

 _People are still wearing masks and mind talking face-to-face with strangers_

<comment><USER4> > _"masks, masks, masks."_

Exactly. I wonder how many lives could have been saved if we (western/European countries) didn't have such an aversion to wearing face masks, even in a time of pandemic?

<comment><USER5> _I wonder how many lives could have been saved if we (western /European countries) didn't have such an aversion to wearing face masks_

My wife and I were one of the few people wearing masks at the start of the pandemic, before the closing and work from home orders came. We were harassed by random people shouting at us.

Also, I don't really get the societal penalty for disaster preparation. It's completely illogical. Just a few weeks before the news was really onboard with the pandemic, a woman at my apartment was looking at how many amazon boxes I had on my hand cart. I told her, "In times like this, it pays to be prepared." She looked at me like I was scum. My understanding is that some news outlets were actually denigrating disaster preparation at that point.

<comment><USER6> > Also, I don't really get the societal penalty for disaster preparation.

It makes a lot of sense: if you're preparing for a disaster, you won't go down with everyone else when disaster strikes. This creates some feeling of resentment which caused people to lash out at you.

<comment><USER5> _you won 't go down with everyone else when disaster strikes. This creates some feeling of resentment which caused people to lash out at you._

A well known psychologist was castigated a few years back for his likening people to lobsters. Yet here we have a perfect example of people acting like the proverbial lobsters who drag the escaping lobster back into the pot.

<comment><USER7> Crab pot metaphor.

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> Very surprised to find out country (city) in HN's top page!

For people who are interested in current Korea's situation:

There were 18 new cases two days ago, and 8 new cases yesterday. The number of new cases have been consistently falling since April 14th. 'Intense social distancing' was stopped yesterday (April 19th), and the Korean government is planning to drop the 'social distancing' policy on May 5th.

Schools are planned to open on May 6th, and the ban of churches, bars, etc. are expected to be dropped in the following days (probably tomorrow).

The KCDC is saying that to drop the 'social distancing', there should consistently have less than 30 cases (which is true for a week) and cases which infection routes are unknown should be less than 5% of all new cases. (It was about 5%~10% when it was announced, and AFAIK it's now true.)

There's still some anxiety because people started to become dull to the social distancing movement - so people are worrying that new cases might increase.

<comment><USER9> > There were 18 new cases two days ago, and 8 new cases yesterday.

> Korean government is planning to drop the 'social distancing' policy on May > 5th.

Are the people divided about whether it was a good idea to wait until new cases got this low before making decisions about moving forward? Or is there very broad support for the kind of discipline it takes to wait until May 5 despite the cases being so low already?

Have people's opinions about the appropriate level of response become politicized? Have the facts and statistics and science of it all become politicized?

<comment><USER10> As GP said in another reply, the incumbent party won a landslide victory in recent congressional election (180 out of 300 seats), so it seems South Koreans generally approve how the government handled it. (Of course the timing was fortuitous for Moon: had the election happened ~45 days ago things would have been very different.)

Even for the opponents, it seems the majority criticism was that the government _didn 't do enough_ \- somehow they're very angry about not blocking visitors from China. (Many conservatives have a strong pro-America, anti-China sentiment, and frequently accuse the president of being the opposite.)

<comment><USER11> You don't have to be a conservative to see how the gov't screwed up early on -- the SK gov't went against the advice of the Korean Medical Association to restrict travel from China; instead the current Moon administration made a political decision to keep their door open to China to please Xi JinPing of China. The problem for the minority party was their link to Shincheonji, a cult responsible for the majority of all coronavirus cases, and they really had to keep their head down pretty much throughout the whole ordeal.

I consider myself a newbie and neutral on SK politics and had dismissed all the criticism of Moon being a commie and all before, but I'm starting to see through the garbage now.

<comment><USER10> Well, AFAIK, expert opinions were divided: many doctors were against border closing, saying that it will cause more people to enter illegally and hide, and then we'll have a harder time tracking them. (It's just a short boat ride across the Yellow Sea from China: you can't block the entire coastline.)

We may argue which side was right in retrospect, but it's not like Moon ignored the expert consensus.

Besides, Korean Medical Association is not exactly a neutral expert organization. The current head of KMA is this guy here [1], with pickets reading "Impeachment is void! Stop framing innocent President Park! Let's rescue innocent President Park with people's power and restore her honor!" blah blah, referring to Park Geun-Hye, kicked out in 2017 after corruption scandal.

[1] [https://namu.wiki/w/%ED%8C%8C%EC%9D%BC:%EC%8B%9C%EC%9C%84%ED...](https://namu.wiki/w/%ED%8C%8C%EC%9D%BC:%EC%8B%9C%EC%9C%84%ED%95%98%EB%8A%94%EB%8C%80%EC%A7%91%EC%9D%B4%ED%98%95.jpg)

</comment></comment></comment></comment></comment><comment><USER12> South Korea demonstrated from the beginning on how to deal with this (despite their slow start - which still is super quick compared to most western countries).

But its completely wrong to think they passed it. They also have massive impediments at the moment. Back to normal? Far from it.

<comment><USER13> I am stuck in my flat in Spain for the 5th week now. Just read they extended it another two, to 8 in total. I am wondering how other countries can be a bit more relaxed (UK and Belgium are allowed out for an our for exercise), or in the case of Sweeden a lot more relaxed, and are still managing to flatten their curves.

<comment><USER14> This is something I have not seen discussed or explained anywhere:

Spain has such a hard lockdown and yet even after 5 weeks there are still thousands of cases every day. How are so many new people getting infected? The same has been true in Italy.

<comment><USER7> It's extremely difficult to tell whether the number of cases are increasing, or if testing is revealing more cases. It sounds like a tautology, but unless you have a very strict, consistent testing rubric, the outcomes are hard to measure in terms of growth. At best you can determine total case count.

</comment></comment></comment></comment><comment><USER15> South Korea started testing very early:

"South Korea's foreign minister, Kang Kyung-wha, speaking to the BBC last week, said the key lessons from her country are that it developed testing for the virus even before it had a significant number of cases. "In mid-January, our health authorities quickly conferred with the research institutions here [to develop a test]," Kang said. "And then they shared that result with the pharmaceutical companies, who then produced the reagent [chemical] and the equipment needed for the testing."

</comment><comment><USER16> > U.S. has swelled to more than 700,000 while Korea [...] have slowed to just > over 10,000.

Comparing absolutes is not very useful so I thought I'd double check this. The difference is obviously less when normalized but the US is still an order of magnitude worse than SK:

      USA:
      700e3 / 328.2e6 * 100 = 0.213%

      SK:
      10e3 / 51.64e6 * 100 = 0.019%

<comment><USER17> Looking at the graphs shows a whole other story though. South korea is on the way down for weeks now, the US are still on the way up. (active cases)

[https://www.worldometers.info/coronavirus/country/south- kore...](https://www.worldometers.info/coronavirus/country/south-korea/)

[https://www.worldometers.info/coronavirus/country/us/](https://www.worldometers.info/coronavirus/country/us/)

</comment></comment><comment><USER18> This worries me. South Korea's mass testing alone was not enough to contain the coronavirus outbreak; they relied on social distancing measures too: [https://www.reuters.com/article/us-health-coronavirus- southk...](https://www.reuters.com/article/us-health-coronavirus- southkorea/south-korea-extends-intensive-social-distancing-to-reach-50-daily- coronavirus-cases-idUSKBN21M02P)

<comment><USER19> South Korea: 234 deaths 8042 recovered. China: 4632 deaths, 77k recovered[1]. Their treatment protocol is an order of magnitude better than every other country. What are they doing that we're not? This is a huge glaring discrepancy. What's the difference between their treatment protocols and everyone else's?

[1][https://www.worldometers.info/coronavirus/](https://www.worldometers.info/coronavirus/)

<comment><USER20> _What 's the difference between their treatment protocols and everyone else's?_

It's mind boggling anyone has to ask now this has been going on so long but here goes.

1) They had an approach plan beforehand because they had observed the previous SARS epidemic and took the danger seriously.

2) They had "quarantine hotels" freely available for those who were positive to prevent them from infecting others. (Seattle has a few of these, I've heard but they aren't widespread in the US. The idea of housing people freely is anethema to the US.)

3) They did testing early and often.

4) Their doctors and nurses worked with full PPEs, full anti-contamination suits, so they didn't themselves massively spread the illness.

5) They reserved hospitals specifically for the epidemic and had protocols for sending people to these hotel.

6) They had drive through testing and other testing easily and freely available (the US plan of profiting from testing is just so "I can't even").

7) They did systematic contact tracing with an app for people to discover where infectious people had been.

8) They did social distancing _with mask_ from the start.

\-- The US had literally done NONE of this. What the US has done is last-ditch efforts by states after the ostensibly responsible parties (the CDC etc) failed massively. And the Federal government is now literally sabotaging the states.

<comment><USER19> The thing I don't get is this explains how people didn't get sick. What it doesn't explain is how so many got better after getting sick compared to other countries.

<comment><USER20> The fatality rate of countries with active epidemics is skewed by a variety of factors:

\- With exponential growth, most cases will new and so not become potentially lethal. This can lower the observed fatality rate (I think this explained Germany's original "great" rate which doesn't look at great any more).

\- With cases rising quickly, most countries don't have infrastructure or the time to increase testing (US testing is failing on multiple levels but even France, Spain and Italy are just overwhelmed with the sick and testing is less important).

\- With cases rising quickly, the fatality rate increases as hospitals are no longer able to provide adequate care.

\- Different countries have different age-profiles and the disease hits the elderly harder (but once hospitals break down, the odds for the young decrease).

South Korea's fatality rate is similar to the ostensible fatality rate of Turkey and Luxembourg but likely for different rate. We probably won't have a complete idea what's happened until these events are done.

</comment></comment></comment></comment></comment><comment><USER21> Meanwhile, the US still has very little testing capacity, with no signs of if and when that will change.

One problem with the death of journalism is that no one is even reporting on this.

<comment><USER22> On a per capita basis yes, but the US has done more tests than any other country. Once the US doubles testing capacity again it will be on par with other large countries.

<comment><USER21> Per capita is all that matters.

<comment><USER22> Sure. On a per capita basis the US is ahead of SK. To reach Germany levels the US will need 8M+ tests. That's a large percentage of all tests that have already been done worldwide. Unfortunately a large country like the US (similar to countries like China and India) is going to run into production/technology constraints.

The US dropped many balls and started late with ramping testing in particular, but now the ramping is happening and just takes time. There was also no way the US could test enough people using the original set of tests that simply took too long. New technology had to be developed.

<comment><USER21> > _Unfortunately a large country like the US (similar to countries like China > and India) is going to run into production /technology constraints._

A large country has a lot of people to test, but also a lot of people to manufacture and perform the tests. So that argument doesn't work for me. Especially when we're talking about the richest country in the world.

Now, if the test making and executing is done by a single centralized organization, it does make some sense. Which is why that way of doing things is an anti-pattern.

</comment></comment></comment></comment></comment><comment><USER23> In this NPR podcast they expand on the methodology applied in S-Korea: [https://www.npr.org/2020/04/18/837905422/the-coronavirus- gui...](https://www.npr.org/2020/04/18/837905422/the-coronavirus-guilt-trip)

</comment><comment><USER24> 3 new cases of coronavirus in Korea today

1 person has died this year from coronavirus in Seoul

Looks to be near the end of the outbreak for Korea

<comment><USER25> The US will hit 40,000 deaths today.

<comment><USER26> and 22 million unemployed

<comment><USER27> For businesses that will probably never return.

<comment><USER28> They'll be replaced, but the owners, employees, and loved ones who die won't.

<comment><USER29> Eventually, but an immense amount of capital (not just in a cash sense) that allows those businesses to operate has been irreparably destroyed. Rebuilding and bootstrapping that again from almost nothing will take years in many cases. Businesses don't exist in isolation, some sectors are seeing destruction not just of the businesses but the entire institutional structure of their ecosystems, which is _much_ harder to replace.

There will be second-order effects that people aren't considering. In Seattle, for example, all of the builders I know are saying multi-family residential construction projects have become indefinitely non-viable due to the systemic collapse of the business ecosystem they rely on. Even if things opened up tomorrow, most of their projects will stay dead for the foreseeable future. This will have a large impact for housing costs, construction worker employment, etc many years beyond the term of the lockdown.

I think many people are oblivious to the long-term second-order damage that is being inflicted in some industries that cannot be fixed on any kind of timeframe that matters to ordinary people.

<comment><USER28> I'm not. Plenty of countries have shown that we can reopen without killing those at risk. The false choice of saving the economy vs saving the at-risk is an entirely political position and part of the propaganda machine you've fallen for.

Unfortunately, in the US, the federal government has presented the false choice of safety vs economy. Yes there are trade-offs in tactics, but it's not one or the other. Stop playing politics, unify messaging, and increase spending money on _both_. I'm helping organize projectdomino.org as part of staving off the currently-likely upcoming shitstorm of reopening without public health funding, coordination, infrastructure, and compliance. I encourage aiding local causes and telling your local Republican leaders that you're tired of their shit that has now stepped up to pathogens literally attacking us at home.

</comment></comment></comment></comment></comment></comment></comment><comment><USER30> how was south korea able to scale up their testing so fast. Where did the extra testing capacity come from. I've been really curious about this but haven't seen any good answers so far.

<comment><USER3> It is am extremely stark example of the value of maintaining manufacturing capacity in your country. Korea can makes tests, so they have tests.

Seegene, a company that makes Covid-19 testing equipment is in Korea. They just needed approval and to ramp up manufacturing. For most countries, like the US, we don't have the manufacturing ability anymore, so we have to go beg and buy tests while we figure how to make again. Seegene alone has been responsible for 80% of the tests in Korea.

<comment><USER30> Afaik, the shortage in US is mainly of raw materials and commodity items like pipettes. How did Seegene ramp up its raw material pipeline so quickly. Really fascinating success.

<comment><USER3> Korea has a broad swath of medical manufacturing capacity. A large part of the supply chain is already there, so it is easier for them to ramp up. You can double or triple output just by putting more shifts into the factories. Then you start retooling, repurposing to really ramp up.

</comment></comment></comment></comment><comment><USER31> Won't this reignite the spreading of the virus? Like, they missed a few people who were asymptotic and not tested, and are roaming the streets with other people?

<comment><USER32> No, because they test people regularly and do accurate contact tracing. Thanks to a favourable regulatory environment and a profusion of capable people, South Korea scaled up testing and tracing capacity very rapidly. Furthermore, they were socially prepared to do the basic social distancing.

Also, Taiwan has done an excellent job, and they didn't need to perform nearly as many tests. AFAIK there hasn't been a single additional confirmed case in Taiwan for the last three days.

<comment><USER33> Taiwan also has a lot of preventative measures, i.e. mandatory mask usage in mass transit systems, government issued masks and mask rationing system, automated temperature checks at various public spaces, mask usage in schools, plastic dividers at restaurants... on top of that they have a very sophisticated scalable method of tracking people who have the virus and ensuring that they are abiding by quarantine rules via tracing the location of their cell phone.

In my city, people are still arguing whether or not people in the mass transit system should be wearing masks.

<comment><USER34> what are the arguments against wearing masks on transit? i'm no fan of masks, but wearing them on mass transit (of any kind) makes sense. lots of random people in a relatively small, enclosed, and cramped airspace elevates transmission risk materially (as opposed to wearing them outside, where it's negligible).

</comment></comment></comment></comment><comment><USER35> Does it show the success of their approach? Given that new cases hit the hospitals 2 weeks after exposure it seems like it is far too soon to know that.

Also, over what timeline are we talking? South Korea has fewer deaths now, will that be the case in two years time?

I've seen a lot of articles lately drawing conclusions from the state of the world as it exists now. But this is a highly dynamic situation. Today's assumptions are next month's fallacies. We need more humility.

</comment><comment><USER36> [https://archive.md/0Kygf](https://archive.md/0Kygf)

<comment><USER37> This is an archive of the above article. Thank you!

</comment></comment><comment><USER38> On the other hand, they have something like 65 re-infected people. Could it be that they have a different/new strain ?

Edit I don't see why the downvotes. It is easily fact-checkable. Those 65 persons were hospitalized for their symptoms, considered treated and released and about 2 weeks later showed symptoms again.

<comment><USER27> We need to wait until we have antibody testing to get a clear picture about this.

A theory is that these asymptomatic cases simply had latent virus in their nasal passages (thus testing positive), and not actually be building any antibodies from an infection.

<comment><USER39> Yeah - Id love to know if there are any cases where someone was sick enough that they were treated, recovered, and then got sick enough they need actual treatment again.

</comment></comment></comment><comment><USER40> > show mass testing success

And public masks, good contact tracking, and they say gloves but hand PPE, like alcohol, also I assume a good isolation system that doesn't scare people but seen no reports on this.

Investing on making testing to be cheap and quick has however been something people missed, compared to pie in the sky things like vaccines.

Also compared to masks which are obvious but have been religiously attacked in the West.

</comment><comment><USER41> How does testing prevent the disease? Also, if someone tests -ve, is that mean he/she is immune to the disease?</comment>