Stories from our first two years - <AUTHOR> http://exposure.watsi.org/watsi-turns-two <comment><USER2> I know I've beaten this drum on HN in the past, but one of the things I most love about Watsi is that you can set up a monthly automatic donation: [https://watsi.org/monthly](https://watsi.org/monthly) This makes it absolutely trivial to set it and help someone out every month, without having to do anything yourself (and thus, no way to forget it!)

They really do amazing work; congrats on two years, guys.

</comment><comment><USER3> Chase, Watsi's founder, spoke at start up school last year.

Donate first, then watch his talk and then donate again. Make sure you look for the tip option on the checkout page if you want to help cover Watsi's operational costs.

[https://www.youtube.com/watch?v=WlT3UhC7NwQ](https://www.youtube.com/watch?v=WlT3UhC7NwQ)

<comment><USER4> One of the best startup up school talks. Hard to watch this and not feel like you want to work for Chase by the end.

</comment></comment><comment><USER5> A problem with internationalization: One difficulty with wanting to contribute to projects like this is that you don't get tax credits on donations outside of your home country, which means I or my business cannot contribute any significant amount. I understand it would be a mountain of paperwork (and probably a loss in terms of accounting costs) for Watsi to be registered as a non-profit in every country.

<comment><USER6> Is this some kind of Scrooge McDuck excuse? If your income tax rate is 30%, on $1,000 donation, you get only $300 back in tax credits. If donation is not tax-deductible, then you won't get $300 so don't donate $1,000. Donate only $700.

<comment><USER5> Are you not then further taxed on that $700 as it must be classified as cash spending?

<comment><USER7> No, because my country (the USA) doesn't tax "cash spending". It taxes income, and it (well, in most cases, individual states), taxes certain _sales_ (generally, this tax is collected by the seller, but when its a foreign seller the buyer may be resposible for it as a "use" tax.) If I don't get a taxable good or service in exchange, its not a taxable sale, and the seller doesn't have to collect sales tax (and, if the seller is foreign, I don't have to pay use tax.)

For a foreign "non-profit", I won't get a deduction from taxable income for income tax purposes for the donation the way I would if the donation was to a local non-profit [1], but I don't have to pay an extra "cash spending" tax the way you describe.

[1] Actually, strictly this is a local _charity_ , which is a narrower category than non-profits. Donations to local non-charity non-profits are generally not tax deductible to the person making the donation, even if the non-profit is itself tax-exempt.

<comment><USER5> I'm genuinely not understanding this. Taxing "income" and "cash spending" is the same (at least, where I am taxed).

If I am a business and have earned $1000, keeping it in the bank is exactly the same as if I withdrew that $1000 and put it under the bed in $20 notes. I pay tax on that $1000. I can give it to Watsi, who is outside of my country's tax jurisdiction, and I must pay tax on that $1000.

<comment><USER8> You were taxed when you received that money, so you don't have $1000, you only have $700 (assuming a 40% tax rate - sub in your own rate as applicable). If you get a tax deduction for giving to Watsi, you can give them $1000 and only lose $700 out of pocket. If you do not get a tax deduction for giving to Watsi, then you can give them $700 and lose $700 out of pocket. In either case, you are giving them $700 out of pocket.

<comment><USER7> > You were taxed when you received that money, so you don't have $1000, you > only have $700 (assuming a 40% tax rate - sub in your own rate as > applicable).

30%, actually, or $600 @ 40%.

<comment><USER8> yea, oops.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> Truly awesome work and an inspirational journey. You guys are a huge motivation for folks in tech to strive to actually make the world a better place.

<comment><USER10> agreed. I funded Watsi on a recurring basis just because of the awesomeness of the story and it's impact.

</comment></comment><comment><USER11> I don't often tear up reading something posted on HN, let alone a company's website...

I really wish my mobile internet was good enough to watch Chase's talk right now.

Edit: Also want to express that I'm proud of the Humble Bundle community! The games industry has made us lose faith in humanity a bit recently. It is heartwarming to hear we're regaining ground.

</comment><comment><USER12> What a wonderful thing to have created. I know I'm just some random dude on the internet, but you should all be incredibly proud of yourselves. I've been meaning to start donating for some time now, and the $50-to-funded thing pushed me to do it.

The act of funding a person rather than an organisation really connects me with the importance of giving the gift - it de-abstracts the situation somehow, and brings home the fact that for not-that-much-really money you can completely change someone's life.

</comment><comment><USER13> Pretty cool stuff, guys. Congratulations on two years.

</comment><comment><USER14> In celebration, most of the patients on their website are just $50 away from being funded: [https://watsi.org/](https://watsi.org/). Go fund one of them!

</comment><comment><USER15> Watsi is incredible. I really love what they do and envy those who created it. It's also a nice surprise to see Watsi and I share the same birthday.

I'm a 26 year old, somewhat privileged white male. I have accomplished nothing in life. I'm still struggling to graduate from college which I'd spent 6 years of my life for. I get depressed thinking about it, and just want to dedicate everything I have as a human being to work for some humanitarian organization similar to Watsi so I could feel like I'm good for something. Of course, then I realize these organizations don't exist to help privileged people like me to figure out what we want in life, but to help people who have real problems. I'm such a cliche. I applaud Watsi team not because they're a bunch of wonderful people who have great talent, but because they wanted to use their talent to make the world a better place. I wish I had your courage.

I'll donate whenever I can, which doesn't amount to much, but every bit helps. I hope.

PS: This thread should be about Watsi but I couldn't help share what I feel. I hope you don't take it the wrong way. Thank you.

</comment><comment><USER16> Love this. Great story telling.

A side question, I love that 100% of donations go to medical care. How does the team fund their operational costs?

<comment><USER17> From their Google Doc transparency file.

"100% of every donation to patients on Watsi directly funds medical treatment - our overhead costs are covered by optional donor tips, cause marketing partnerships, and private donors, foundations and corporations."

[https://docs.google.com/spreadsheets/d/1KQ2uWkJf5-5f34uttGmA...](https://docs.google.com/spreadsheets/d/1KQ2uWkJf5-5f34uttGmAhPcQPxmgq8HsnsIfwFr9Do4/pubhtml#)

</comment></comment><comment><USER18> The greatest YC startup to date, by far. Good job and congrats!

</comment><comment><USER19> This is a really incredible and moving use of technology. I will be contributing soon!

</comment><comment><USER20> I am a student in Boston, born and raised in El Salvador, and I can attest the extent to which organizations like Watsi are helping save the lives of people like you and me. Chase, I'm giving up a cup of coffee a week; that's twenty bucks you can count on.

</comment><comment><USER21> A great startup.

But the linked webpage... when did design like that become normal?

<comment><USER22> Looks like they are using [http://exposure.co](http://exposure.co)

</comment></comment><comment><USER23> Awesome, for me Watsi is one of the most inspirational startups right now. Reminds me that if someone can, doing amazing startups is the right thing to do!

</comment><comment><USER24> You inspire me everyday, Watsi.</comment>


Ask HN: What's a good place for South African programmers to emigrate to? - <AUTHOR>

I am a South African programmer. Due to growing instability in South Africa (which I don't want this thread to become about), me, and some of my friends would like to set up a "plan B".<p>What countries are good fits for experienced South African programmers, currently developing enterprise systems, who speak English as a first language and have university degrees? Australia, New Zealand, the UAE and England seem to be favourites, but I'm sure there are other suitable options as well. I'd also be interested in hearing from programmers who have emmigrated already, about unexpected challenges and surprises that they have encountered. <comment><USER2> Looking for a job or working as remote contractors? I'm a U.S. citizen living abroad but since I do remote contract work I can work and live anywhere on a tourist visa. I have to do visa runs but that's just another excuse to travel somewhere different.

<comment><AUTHOR> I'm asking about a permanent move. I have a young family, so they would have to move with me.

</comment></comment><comment><USER3> Check out Panama. Seriously. Get a base here and either work locally or do remote work in the same timezone (u.s./canada). Ping me if you have any q's. (Ad: I'm hiring and open to remote workers)

</comment><comment><USER4> Canada could also be an option, if you can stand the winter. Though the immigration process can take a long time unless you have a job offer from a Canadian company.

If you are still interested in improving "plan A" (i.e., staying in South Africa), we are hiring system software engineers (compilers, object persistence, concurrency, UNIX/Linux, C++, etc):

<http://www.codesynthesis.com/jobs>

</comment><comment><USER5> England is not the preferred nomenclature. UK is your best bet; size of industry, proximity to continental Europe, ties with US and all...

Depending on your age, income and/or ancestary, you can make use of Tier 2, Tier 5 or ancestary visas, all of which lets you to work in the UK without finding a sponsor.

<comment><AUTHOR> I said England specifically because I don't know of programmers who have moved to other parts of the UK, but you make a valid point. I hadn't considered the other countries of the UK.

<comment><USER5> I had a couple of job offers from companies in Glasgow in the past. I had a feeling that the local goverment must be subsidizing the tech companies in Scotland. The pay package was comparable to London. Cost of living looked a lot less compared to London. If you are looking into building cash reserves it might be ideal.

</comment></comment></comment><comment><USER6> A friend of mine thought he had a well paid job in Australia, then moved to the UK to do contract work in a similar field to yours and earned 4.8 times as much with much less responsibility and stress.

The exchange rate isn't as good anymore, but I would recommend the UK over the other options.

</comment><comment><USER7> If your grandparent was Irish (unlikely in South Africa I guess), then you are entitled to an Irish passport. You are then a EU citizen and can work in any EU country with no visa issues. Just something that might be of benefit.</comment>


Mesos: Cluster management platform - <AUTHOR> http://www.mesosproject.org/

<comment><USER2> * Isolation between tasks using Linux Containers.

I would just like to point out that lxc offers zero isolation of tasks. UID/GID mapping isn't done thus a root user in a lxc container has all the power as root on the host among many other things. LXC is glorified resource control with a few minor isolation features (syslog in 2.6.38 and posix caps I think and current namespacing in Linux) right now and nowhere realiable as a real isolation environment

<comment><USER3> The standard setup for compute clusters is that there is an admin ops team that is assumed to be trustworthy and then many untrusted users. Is this a problem for these use cases? I think the untrusted admin is less of an issue for most organizations.</comment></comment>


Lapis – A Lua, Moonscript Framework built on OpenResty - <AUTHOR> http://leafo.net/lapis/? <comment><USER2> Hello, I create lapis.

I'm actually on the verge of releasing a new huge update. New features include:

* first class lua support

* built in support for a lua templating language: [https://github.com/leafo/etlua](https://github.com/leafo/etlua)

* a new postgres driver written in pure lua, should play nicer with luajit and and be available in more stages of nginx request cycle: [https://github.com/leafo/pgmoon](https://github.com/leafo/pgmoon)

* lots of other smaller changes

All of this code is in master I just need to write all the documentation for it, _sigh_.

The biggest site I've created with lapis is itch.io: [http://itch.io](http://itch.io) it's currently 43,209 lines of code (excluding libraries).

I run it on a really crappy 10 dollar a month vps, memory and cpu usage is tiny. I've handled well over a million requests per day in the past with no issues. The app communicates with a lot of third party payment services over http in the server, because of the non-blocking nature of openresty throughput is not affected when these services are slow (eg. paypal).

Another interesting lapis project is MoonRocks: [http://rocks.moonscript.org/](http://rocks.moonscript.org/) it's a public lua module hosting site. It's opensource so you can check out what a developed lapis application looks like: [https://github.com/leafo/moonrocks- site](https://github.com/leafo/moonrocks-site)

Thanks for checking out lapis!

<comment><USER3> We've started using openresty quite a bit in our shop. It's great for making certain things very fast. We looked at lapis, too, but it seems to be more than we need right now. Really, I'm afraid of doing too much complex development using a framework that doesn't provide you with an interactive debugger (like pdb).

Is there something out there that I'm not aware of? The following comment by the creator of openresty seems to indicate that there is not:

[https://groups.google.com/forum/#!topic/openresty- en/VSIM8jY...](https://groups.google.com/forum/#!topic/openresty- en/VSIM8jYtXo0)

<comment><USER2> It's not something I've experimented with yet. Lua comes with a very low level debugger: [http://www.lua.org/manual/5.1/manual.html#pdf- debug.debug](http://www.lua.org/manual/5.1/manual.html#pdf-debug.debug) but I haven't tried using it inside of an nginx worker. Because the workers are forked you don't have immediate access to their standard in.

For lapis I did build a server side console for running code in the same environment as the worker though, it might be useful: [http://leafo.net/lapis/reference.html#lapis- console](http://leafo.net/lapis/reference.html#lapis-console)

This project looks interesting, maybe it can be used: [http://keplerproject.org/remdebug/](http://keplerproject.org/remdebug/)

</comment></comment></comment><comment><USER4> As a sidenote, openresty (nginx+lua) is probably one of the most underrated technologies we have available nowadays. In my experience, every webapp can offload a ton of work(1) to openresty. The lua code is usually much simpler that the equivalent code in the app.

(1) think caching, auth/acl, security filters, some of the stats, some of the logs, etc

<comment><USER5> +1 At apitools.com we use openresty as well. We chose lua (over javascript) because it's super fast, and because it has good sandboxing/ isolation properties which make it safe to handle arbitrary middleware.

</comment></comment><comment><USER6> We've been using it in production for about 6 months as the endpoint for an analytics system. We went from 5 medium EC2 app servers down to 1 micro instance in testing for our load at the time (we have an HA setup in production). Blazingly fast. Less than 100 LOC for the service.

The author of Lapis is also very responsive to PRs and bug reports.

<comment><USER7> What framework were you using before that required 5 medium ec2 instances? I'd really appreciate knowing what you were comparing Lapis to.

<comment><USER6> We use Django behind gunicorn for most of the stack. Not that Django is slow (we still happily use it for 99% of our public facing system) -- it's just that OpenResty is _so_ fast[1].

Besides the fact that OpenResty is built on C/Lua, Nginx's evented architecture makes it scale concurrently really well. We use Gunicorn in a multiprocessing configuration for the rest of the stack -- so each worker consumes much more RAM than the equivalent evented configuration.

We were toying with the idea of implementing the service with Tornado (also evented, but in Python) -- but OpenResty/Lapis has been so great for our use case that it's tough to justify the test at this point.

[1] [http://www.techempower.com/benchmarks/](http://www.techempower.com/benchmarks/)

</comment></comment></comment><comment><USER8> I just need to say that OpenResty is great, and the mailing list is awesome.

After a topic here in HN saying about doing auth in lua+nginx, we successfully migrated our auth from inside the applications (all java) and now it's a 100 LOC of Lua, easy and decoupled from the application.

<comment><USER9> Doing auth in OpenResty is a good gateway into using it for more stuff, it is really easy to do whatever auth you need without your application having to know.

<comment><USER10> Are there any Lua libraries targeting openresty that makes it easy to use integrate with oauth2 services of Google, facebook, 4square etc ? Since everyone does oauth differently it would be nice to have a go to library for this.

</comment></comment></comment><comment><USER11> If someone is eager to try Lapis I have a Dockerfile available here:

[https://github.com/torhve/lapis-docker](https://github.com/torhve/lapis- docker)

it includes a readme for getting started with Openresty in general.

<comment><USER12> For plain openresty you can try [https://github.com/3scale/docker- openresty](https://github.com/3scale/docker-openresty) It does not have fancy readme, but has batteries included. Luarocks works, supervisor is set up, cron is there...

</comment></comment><comment><USER13> Lua within Ngnix is amazingly fast. I developed a custom solution for a client last year that basically did bot detection (similar to CloudFlare). It's amazing what you can do within nginx without touching an external fastcgi server or proxy.

</comment><comment><USER14> I don't understand. How is Lua's performance compared to NodeJS, or compiled languages such as Java or Go ? Does it make sense to stuck everything into a (used to be ) reverse proxy server ?

<comment><USER15> > How is Lua's performance compared to NodeJS, or compiled languages such as > Java or Go

This might help: [http://www.techempower.com/benchmarks/](http://www.techempower.com/benchmarks/)

> Does it make sense to stuck everything into a (used to be ) reverse proxy > server ?

Just like putting stuff into Apache or whatever other webserver you like.

</comment></comment><comment><USER16> Is Unicode ever a problem when you do real-life web development in Lua? Lapis looks appealing otherwise but the lack of a built-in Unicode string type in Lua makes me somewhat wary.

<comment><USER17> Not particularly. Lua doesn't do anything to make storing utf-8 in strings difficult. You have to bring your own Unicode library for doing anything nontrivial, but that's true of everything in Lua. Using ICU from Lua isn't substantially different from using it from any other lanugage.

<comment><USER18> And they said, "Lua doesn't come with Batteries included"

</comment></comment></comment><comment><USER19> This is great! I'm curious about CPU/RAM usage too

<comment><USER4> An order of magnitude less than any framework you have in your mind

</comment></comment><comment><USER20> I've been experimenting with this with a fairly large website. The lack of MySQL support is the only thing keeping me away.

<comment><USER21> You migh look at this nginx module wiki.nginx.org/HttpDrizzleModule. It should be possible to access mysql this way with lua as well

</comment></comment><comment><USER22> Performance aside, does anyone have experience with how Lapis/OpenResty compare to, say, Django or Rails in terms of stability, included batteries, 3rd party libraries and overall developer productivity and production readiness? I guess this overlaps a lot with the respective question for lua vs python/ruby ecosystems.

<comment><USER23> As a person that uses Django to make a living (at my full-time job, and for my freelance work) and who has tinkered with a lapis/openresty site on my spare time ([http://graphcake.com](http://graphcake.com)), I'd say productivity is on the side of Django. There are too many included batteries (form validation, django ORM more powerful than lapis ORM, staticfiles) and high quality 3rd party libraries (task queue (celery, python-rq), image processing, caching) that I'd hate to reimplement.

That said, performance is totally on the side of lapis/openresty. The performance of my side project totally depends on how fast postgres responds to the needed queries.

</comment></comment><comment><USER24> I was just looking at this yesterday. It seems awesome. My one question is how you do forking dependency graphs since everything is non blocking but looks blocking. Eg. I have takes A, B and C. C depends on A and B so I want to do A and B concurrently but then wait on them for C.

<comment><USER9> You can use [https://github.com/openresty/lua-nginx- module#ngxlocationcap...](https://github.com/openresty/lua-nginx- module#ngxlocationcapture_multi) to do multiple parallel requests

<comment><USER24> Interesting. Thank you!

</comment></comment></comment><comment><USER25> Absolutely amazing work. I have been meaning to give this a go for ages.

</comment><comment><USER26> Interesting, and I found it in the techempower benchmark too:

[http://www.techempower.com/benchmarks/#section=data-r9&hw=i7...](http://www.techempower.com/benchmarks/#section=data-r9&hw=i7&test=fortune)

<comment><USER24> I don't get how it does so badly in the queries and writes section when compared to node.js while the serving plaintext and json is so high.</comment></comment>


Ask HN: Who uses Chaos Monkey in production? - <AUTHOR> With Netflix announcing an upgrade [1] to Chaos Monkey today, I would be curious to know:<p>- Is your team using Chaos Monkey in your production/staging infrastructure?<p>- If not, do you use a variant tool or any interesting implementation of "Chaos Engineering" [2], and to what degree of success?<p>[1] https://news.ycombinator.com/item?id=12743693<p>[2] http://principlesofchaos.org<p>Previous discussion (2014): https://news.ycombinator.com/item?id=8713950 <comment><USER2> I commented about Failure Fridays at PagerDuty in the older thread (not Chaos Monkey precisely, but similar concept). We still do that, with a few modifications: 1) ChatOps is used to execute the commands from Slack in order to preserve history and help interested parties follow along. 2) If we're not testing a specific service/AZ/region, a "reboot roulette" bot is run that reboots any host from our production infrastructure at random. Every single production host is game. 3) This is now scheduled to run automatically at certain times of the week.

Many of the things we do regularly used to be terrifying. They no longer are precisely because we do them regularly. That's the value of chaos engineering.

</comment><comment><USER3> I did something similar and had a test suite across a CoreOS cluster that would fail machines in patterns for hours whilst checking service and data integrity. One can run the suite against the replica cluster when new deployment features are added or changed.

Doing it on live wasn't possible for me because failover resulted in some down time although I consider the former to have been very useful.</comment>


A new type of solar cell is coming to market - <AUTHOR> https://www.economist.com/news/science-and-technology/21736122-perovskites-have-potential-outshine-silicon-solar-panels-new-type <comment><USER2> Good article, but it's misleading on one very important point. It talks only about Oxford Photovoltaics, so it leaves the impression it is the only organization working on perovskites. Actually, lots of other companies and research laboratories are.

</comment><comment><USER3> Paywalled article I have no idea what you are talking about.

<comment><USER4> Could you please stop posting these? They're off topic and the rules on HN are settled. See sibling comment's quote from FAQ, plus:

[https://news.ycombinator.com/item?id=10178989](https://news.ycombinator.com/item?id=10178989)

[https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...](https://hn.algolia.com/?sort=byDate&dateRange=all&type=comment&storyText=false&prefix&page=0&query=by:dang%20paywall)

This solution has turned out to work well in practice because, although paywalls with workarounds are annoying, it's possible to work around them.</comment></comment>


Practical Invalid Curves Attack on Java TLS Implementations - <AUTHOR> http://web-in-security.blogspot.com/2015/09/practical-invalid-curve-attacks.html <comment><USER2> Note the date: September 14th, 2015.</comment>


Bitcoin exchange account of Coinpal shut down by Paypal - <AUTHOR> http://www.bitcoin.org/smf/index.php?topic=2555.msg101084#msg101084 <comment><USER2> I can't honestly think that anyone is surprised? PayPal plays it safe and this has been proven again and again.

At the end of the day, this Bitcoin hype is getting slightly tiresome. I do apologize for being negative here, but the only thing I take away from this whole spectacle is that investing in energy is a wise move. The amazing thing is, that was news a long, long time ago.

<comment><USER3> Investing in energy? huh?

<comment><USER4> Essentially, Bitcoin generation is tied to energy usage, which means not only do the power companies profit from Bitcoin generation, they can essentially regulate the price of it (until BTC is mined out, anyway).

<comment><USER5> Power companies can't regulate the price of Bitcoin because they don't control the price of electricity, which is determined by the government and/or the market.

Nobody ever regulates or controls the price of anything, except if they're the only supplier or the government.

<comment><USER6> You forgot the "only buyer" case.

<comment><USER5> Well, if there's only one buyer, then it's a negotiation between the seller and the buyer. The seller has to answer the question, "Where is the cut-off point where it's no longer in the buyer's economic interest to buy from me," and charge right below that. Of course, in such a situation, new sellers will typically emerge.

</comment></comment></comment></comment></comment></comment><comment><USER7> Paypal's arbitrary policies and ability to freeze on whim or even pull money out of your account with no comeback are the reason my latest startup[1] doesn't take Paypal at all. It's just not worth the hassle of dealing with customer (or $deity forbid, your own) paypal problems.

[1] - <http://www.44con.com/>

</comment><comment><USER8> I'd be surprised if PayPal didn't see Bitcoin as some form of competition - so, this is fairly unsurprising. Most companies prefer not to facilitate their competitors.

</comment><comment><USER9> This seems like a pretty clear-cut case. The Coinpal creator didn't do his homework, and PayPal acted well within its stated policies to protect its business.

The Coinpal creator chose to email Paypal to ask a question that is clearly answered in the company's acceptable use policy. Under Prohibited Activities, Section 3(f): "You may not use the PayPal services for activities that (...) are associated with the following Money Service Business activities: (...) currency exchanges.

Second, the operator expresses in his post that he had kept his agreement to keep chargebacks low, at 0.9%. Consider for a moment that PayPal and the credit card company are splitting somewhere around 3% of the transaction amount to cover all of their expenses and make a profit. Let's assume that, out of 100 transactions for $100, one gets charged back. If it costs the companies $50 to investigate the customer complaint and process the chargeback, then the combined companies are losing 16% of their revenue to cover chargebacks.

</comment><comment><USER10> > After about two weeks, I received no response to either email. I decided to > start operating the site, thinking they wanted to see something working > before spending time on a response.

I don't follow the logic here... no response != assume whatever the hell you feel like.

<comment><USER11> What "the hell" should he have done?

There's only 2 possibilities here - 1. They didn't see the emails or 2. They saw them, but chose to ignore them. Either way, his choice boils down to killing his idea or launching it and seeing what happens.

It would be utterly moronic of him to kill his idea simply because they didn't respond. Might as well leave that decision to them by launching it and seeing how they respond. If they don't shut him down, then his service isn't breaking their rules. If they do, he's no worse off than not launching in the first place.

He did the right thing.

EDIT: _HN is not letting me respond to wmf below, so I'm responding here_ \- Where does he say he knew paypal would freeze the account sooner or later? In fact, the whole point of his emails to them was to specifically ask if his service was compliant with their TOS.

Let's stick to facts and not get into subjective arguments. Your idea of 'ordinary' can be completely different from his or anyone else's idea of 'ordinary'. This is why TOSs, FAQs etc exist in the first place. But since he couldn't find an exact answer anywhere, he emailed them.

<comment><USER12> Hacker News has a timeout for responding to nested comment threads. The idea is to prevent long and drawn-out reply bickering between two people. If you can't reply to one of your comment's children, calm down, take a breath, and wait 5-20 minutes.

<comment><USER11> Thanks for the explanation, had no idea. You really shouldn't get so angry tho.

</comment></comment></comment></comment><comment><USER13> I chose to focus on his last point:

"CoinCard will be easier to bring back, minus PayPal payments. ... A possible PayPal freeze was one reason I focused on "gift cards" rather than PayPal exclusively. This whole situation has reminded me once again why distributed, resilient systems like Bitcoin and OTC are so important."

Paypal frustration led Michael to Bitcoin, and his service ended up dependent on it. That was probably not his original intention.

This makes MtGox's integration of (ACH) Dwolla even more important to BTC's liquidity. MtGox is receiving record traffic right now, nearly on the order of a DOS attack.

<comment><USER14> The guy running mtgox says it's an actual ddos.

</comment></comment><comment><USER15> I'm really interested to see if bitcoin will survive the law. I'm not a lawyer, but my understanding is that creating an alternate currency is strictly illegal. Maybe this is the first indication of the end of bitcoin?

(Edit: This is incorrect. See comments.)

<comment><USER16> _I'm not a lawyer, but my understanding is that creating an alternate currency is strictly illegal._

IANAL either, but I would be interested in a citation on that. My understanding is that say in the US (a) you can't print US dollars (because the Fed has a monopoly on that) and (b) you can't refuse to take US dollars (because they are legal tender). That said, I don't know of a law that says you can't also use an alternative currency. For example if a school had "school money" where "credits" are earned for good grades and are exchanged for lunch or snacks, that is a form of alternate currency that surely is not illegal.

<comment><USER17> Of course, the line between "illegal" and "can get you in legal trouble" is never a crisp one.

<comment><USER18> exactly. thats why point B does not work in most government facilities nowadays. try to pay a court case, get drivers license, pay for taxes or your license plate with cash --- a nice clerk will tell you "check, money order or credit card only".

</comment></comment></comment></comment><comment><USER19> There's a major disconnect here: PayPal is not digital cash, it's credit card processing. There is a major difference.

<comment><USER20> Paypal is both a credit card processor and an online wallet (stored value, digital cash, etc), both aspects of which complicated their anti-fraud and regulatory efforts.

There exists money in my Paypal account which can be spent or transferred at my direction and which does not exist elsewhere at an account under my control.

<comment><USER21> But the money in your paypal account is denominated in US Dollars (or Yen, or whatever real world currency), isn't it? Paypal's 'online wallet' is just another way of saying 'checking account'. I have a checking account which has a stored value that I access via debit card or paper check, and can transfer or spend the money at my direction. The money in that checking account does not exist elsewhere at an account under my control.

I don't see the difference here?

<comment><USER22> I thought Paypal had argued strenuously that it is _not_ a bank, so it's, according to them, _not_ just another way of saying checking account.

</comment></comment></comment></comment><comment><USER23> First wikileaks and now Bitcoin. I'm boycotting paypal.

<comment><USER24> This attitude comes up again and again when PayPal is in the news for doing something (ignorant/unpopular), but those who utter such statements fail to understand the power of the network effect.

If you wish to buy something and the seller only accepts PayPal, then good luck with your boycott. In my experience, the larger the retailer, the more infinitesimal the chance that they will "listen to reason" and consider accepting an alternate form of payment.

Oh, and did I mention eBay has this affinity for PayPal?

<comment><USER25> The larger the retailer, the more likely they will offer payment methods other than/in addition to PayPal. As the retailer, it's in your best interest to give the customer the greatest possible number of ways to give you their money.

</comment></comment></comment><comment><USER3> So long CoinPal, and thanks for all the bitcoins!

[http://www.bitcoinmoney.com/post/5086167006/paypal- freezes-o...](http://www.bitcoinmoney.com/post/5086167006/paypal-freezes-out- coinpal)

</comment><comment><USER26> It may still be possible to exchange bitcoins through mtgox and then through liberty reserve and bank money orders. Each stage along this route eats up 1-5%. Otherwise, informal transfers through the OTC ( <http://bitcoin- otc.com/> ) do work.

Tying too much cash in paypal is a mistake. There is no guarantee that you won't snag on some tripwire in their rules engine. That and all BTC-cash exchanges have so far been shut down by PayPal.

But more fundamentally, Bitcoin is incompatible with the credit card system. CC security is very lax, so CC transactions have to be reversible (the onus is on the merchant rather than the buyer). Banks (and Paypal is a bank) are ill equipped to parse the math and the crypto that go with a proof that a transaction has occured. Not technically, but institutionally they are a different thing.

<comment><USER27> This is exactly correct. I got seriously down voted for making the same point in less words. Bitcoin and exchanging e-currencies is nothing new. People have been getting screwed over exchanging Paypal and credit cards for other currencies for a long time. That's why Paypal doesn't allow it. But yet, these services keep popping up. Poor business idea, it's been tried many times and it doesn't work.

</comment></comment><comment><USER27> Seems like a silly idea anyways. There is a good reason why Paypal doesn't allow it. There is also a good reason why most other exchangers don't allow credit cards. In both cases there are too many people running charge backs.

<comment><USER28> He writes, that his chargeback rate was 1.5%

<comment><USER27> I have worked a lot with exchanging e-currencies for certain niche reasons. Exchanging can be quite a pain. Generally you have to tap into the various exchange communities to figure out your options and lookup reputable exchangers. In these communities you will find that a widely accepted rule is that you don't exchange with currencies which allow charge backs (unless perhaps you personally know that person.) That includes Paypal and credit cards.

Yes, there are people who will exchange for Paypal and credit cards. Yes, there are some services which can start out with low charge back rates. The problem is the scammers will find you as the service grows. This is a business with low margins, eventually you will lose out.

Again, Paypal isn't perfect, but the service has plenty of experience with this issue. Too many people are getting screwed over. Unfortunately this is a lesson that people have to learn over and over again. Do NOT use Paypal for exchange.

I don't understand why I'm getting down voted for this.</comment></comment></comment>


Oral History of Donald Knuth (2007) [video] - <AUTHOR> https://www.youtube.com/watch?v=Wp7GAKLSGnI <comment><USER2> Here is the transcript, for those who do not feel like viewing a three hour video ;)

[https://www.computerhistory.org/collections/catalog/10265805...](https://www.computerhistory.org/collections/catalog/102658053)

An excerpt:

Feigenbaum: One of the things worth observing -- it's off the track but as long as we're talking about history -- is that our current generation, and generations of students, don't even know the history of their own field. They're constantly reinventing things, or thoughtlessly disregarding things. We're not just talking about history going back in time hundreds of years. We're talking about history going back a dozen years, or two-dozen years.

Knuth: Yeah, I know. It's such a common failing. I would say that's my major disappointment with my teaching career. I was not able to get this across to any of my students this love for that kind of scholarship, reading source material. I was a complete failure at passing this on to the people that I worked with the most closely. I don't know what I should've done. When I came to Stanford from Caltech, I had been researching Pascal. I couldn't find much about Pascal's work in the Caltech library. At Stanford, I found two shelves devoted to it. I was really impressed by that. Then I came to the Stanford engineering library, and everything was in storage if it was more than five years old. It was a basket case at that time, in the 60's.

</comment><comment><USER3> And my favorite story about him:

[https://news.ycombinator.com/item?id=2856567](https://news.ycombinator.com/item?id=2856567)

</comment><comment><USER4> Another video interview with Knuth, [https://www.webofstories.com/story/search?q=knuth](https://www.webofstories.com/story/search?q=knuth)

</comment><comment><USER5> The man likes his hardware:

[https://www-cs-faculty.stanford.edu/~knuth/organ.html](https://www-cs- faculty.stanford.edu/~knuth/organ.html)</comment>


Cost-benefit analysis of airport security: Are airports too safe? - <AUTHOR> http://www.sciencedirect.com/science/article/pii/S0969699713001270

<comment><AUTHOR> More information at [http://www.businessweek.com/articles/2014-01-10/considering-...](http://www.businessweek.com/articles/2014-01-10/considering- the-odds-of-terrorism-airports-spend-too-much-on-security)</comment>


Trump's FCC Votes to Allow Broadband Rate Hikes for Schools and Libraries - <AUTHOR> https://motherboard.vice.com/en_us/article/trumps-fcc-to-vote-to-allow-broadband-rate-hikes-for-schools-and-libraries <comment><USER2> What's alarming is how silent the giant Silicon Valley companies have been as the FCC proceeds to destroy the foundations of the open web. It was little noted at the time that Google actually sent a letter in support of the new policy giving ISPs the right to sell customer browsing data. Without Sillicon Valley in our corner there's not much hope. It's all corporations in lock step aligned with the government against consumers.

<comment><USER3> SV is full of people like Pai. Their culture is one of haves vs have nots. Hope you're in with the right caste, it's coming to a town near you.

</comment></comment><comment><USER4> Can someone explain to me why an FCC that is currently made up of three Obama appointees is "Trump's" FCC?

Trump did name Pai as chair, but he was already on the commission (appointed by Obama in 2012).

That's a terrible, click-baity headline, even by the low standards normally set by vice.com.

<comment><USER5> > Can someone explain to me why an FCC that is currently made up of three > Obama appointees is "Trump's" FCC?

Because it has a Republican majority as a result of Trump's election, and Trump selected the chair from among the two Republican members, so Trump's election and Trump personally have set the direction of the current FCC. Hence, Trump's FCC.

<comment><USER4> "Because it has a Republican majority as a result of Trump's election"

Sorry, its makeup has nothing to do with "Trump's election". All three current members were appointed by Obama.

Now, Trump is likely to get to appoint at least three members. At that point, it could, perhaps, justifiably be called "Trump's" FCC.

Until that happens, though, calling an independent agency in which all of the members were appointed by Obama (and over which the executive branch has no direct authority) "Trump's" is so disingenuous it crosses the borderline into dishonesty.

<comment><USER5> > Sorry, its makeup has nothing to do with "Trump's election".

Yes, it does. It's not a 3 seat commission, it's a five seat commission, and the Senate blocked the renomination of a Democrat whose term ended in December to keep the seat open for Trump to name a replacement, and Wheeler resigned at the end of Obama's term, as is customary (but technically not required) for FCC chairs at a Presidential transition unless asked to stay on.

Had Trump not been elected, the FCC would either be in a 2-2 deadlock (had the Senate blocked Rosenworcel's renomination despite a new Democratic administration waiting in the wings, which would be unusual but not implausible), or still a 3-2 Democratic majority.

This is Trump's FCC, even if he hasn't appointed the two members (only one of which can be a Republican) he can to fill vacancies.

<comment><USER4> "Yes, it does. It's not a 3 seat commission, it's a five seat commission"

And? All three of the members were still _appointed by Obama_. Not Trump.

</comment></comment></comment></comment></comment><comment><USER6> It's going to be fun cleaning all this up in 4 years.

<comment><USER7> The previous 3 presidents all had 2 terms. Any reason to thinks Trump or at least the republican party won't too? Just because you don't like him doesn't mean everyone else doesn't either. Real humans who really wanted him voted him in and they're probably happy with their choice.

<comment><USER8> You don't have to guess, there are plenty of polls. He is historically unpopular.

<comment><USER9> "Trump will never run for President"

"Trump will never make it past the first debate"

"Trump will never win the Republican primary"

"Trump will never win the general election"

(You are here)

<comment><USER10> "Trump will never fulfill his promises"

(I'm over here)

<comment><USER11> He doesn't need to.

</comment></comment></comment></comment></comment></comment><comment><USER7> Maybe the reason the market is uncompetitive is because of the price caps? Fighting monopolies with regulation constraining their prices seems like a risky solution. Isn't there some action that would open up the market to more competitors? Even if another company had to lay their own cables, that happens and isn't impossible.

<comment><USER12> Are you serious? I'm not being flippant here, but your question doesn't seem to be an honest one.

Laying cables is really expensive. It takes a lot of capital and the return on investment is long-term. Two things that aren't very popular among Wall Street or VCs. Even assuming you undertook such a venture, Comcast/ATT/Verizon have really _really_ deep pockets. They can undercut any price you care to offer.

The idea that "regulation" is causing high prices is farcical on its face. There aren't enough schools and libraries to make any difference to the bottom line.

The market is uncompetitive because broadband is a natural monopoly. Just like roads, water, sewer, and electricity. The capital costs are so high there is usually no business justification for having more than 1-2 providers in a given area. There are a few exceptions but they prove the rule.

A "free market" in broadband would look much like the "free market" in electricity: Either the government or a regulated utility granted monopoly power by the government should run the fiber. Any ISP that wants to compete can hook up at the local CO. The equipment on both ends of that fiber can even be owned by the ISP so if new technology allows faster speeds nothing blocks the upgrade (the favorite excuse for pro-corporatist shills).

The big three have fought anything that smells like a free market tooth and nail, outright purchasing legislation at the state level to squash municipal attempts to do anything like this.

The funny thing is the big ISPs rely on government-granted taking of private property in the form of right-of-way to run their cables... something for which property owners receive no compensation. So much for "using our lines" as the ATT CEO likes to claim.

<comment><USER13> If it's a natural monopoly, why are there all these laws only allowing one provider?

BTW, before the law enshrined AT&T as a "natural" monopoly, there were lots of phone companies who seemed to have no trouble at all running wires everywhere.

<comment><USER2> If you're referring to the Kingsbury Commitment[1] that was an agreement with the government after AT&T was already a monopoly. After Alexander Graham Bell's patent expired there was a brief period where competition sprouted but then AT&T started consolidating and buying up the smaller competitors. By 1913 AT&T was already in a dominant monopoly position.

[1] [https://en.wikipedia.org/wiki/Kingsbury_Commitment](https://en.wikipedia.org/wiki/Kingsbury_Commitment)

<comment><USER13> It was not already a monopoly. Your link says there were other long distance companies, just that AT&T's was more extensive.

Note that scores of companies were running railroads everywhere, it doesn't make sense that running a telephone wire was infeasibly expensive.

<comment><USER14> A couple things: 1) While AT&T was not the only provider left standing, it was the largest and had scale advantages against its competitors. The Kingsbury Commitment was designed to halt AT&T's expansion and ensure that those competitors were able to compete. However, it failed: AT&T was able to continue to improve its dominant position by selling subscribers in certain areas while purchasing competitors in more strategically advantageous areas. 2) One of the reasons for "natural" monopoly regulation is that we do not want telephone wires, railroads, and pipelines running everywhere. Building a utility requires gaining access to rights of way. Society subsidizes all private utilities by creating rights of way and infringing upon the rights of property owners along the route. 3) The scores of companies running railroads everywhere were subsidized even further. At least in the West, the federal government gave land along the route to the railroad companies for construction. [1] In some ways, building a railroad was more about land speculation than private construction of transportation infrastructure.

[1] [https://en.wikipedia.org/wiki/Pacific_Railroad_Acts](https://en.wikipedia.org/wiki/Pacific_Railroad_Acts)</comment></comment></comment></comment></comment></comment>


Move Over, Bitcoin. Ether Is the Digital Currency of the Moment - <AUTHOR> https://www.nytimes.com/2017/06/19/business/dealbook/ethereum-bitcoin-digital-currency.html <comment><USER2> I think this is just a compounding of a pump and dump</comment>


Is it a new particle, or just a fluke? - <AUTHOR> http://www.cnn.com/2011/US/04/08/particle.physics.tevatron/

<comment><USER2> so far respected scientists suspect a statistical ghost.. but dr cnn already announces possible energy revolutions. they have too much airtime to fill</comment>


Things I Wish Someone Had Told Me When I Was Learning How to Code - <AUTHOR> https://medium.com/learning-to-code/565fc9dcb329 <comment><USER2> Nice post. A few more of my own:

1\. There is no substitute for the hours spent coding. Sure, you can waste a lot of time, but there's a high correlation between hours spent and skill attained.

2\. You can do almost anything with almost any tool. The energy spent debating tools would probably be better spent building something.

3\. One of the biggest differentiators between a good coder and a poser is the willingness and ability to go several levels down. Deep understanding comes from deep diving.

4\. One of the best ways to deep dive is to benchmark. The milliseconds gained will be a byproduct. The real benefit is your understanding of what goes on under the hood.

5\. Write a framework. You may never use it, but you'll never be the same programmer either.

6\. Almost everything you do will take longer than you expect, no matter how experienced you get or how good you get at estimating. If it doesn't, then you're not taking on challenging enough work.

7\. Once you regularly deliver great value to others, you will often feel like an imposter. The compliments will exceed what you think you deserve based upon how hard it was for you to build. That's OK. Accept it and enjoy it. It's payback for all those hours you slaved away and no one knew what you were doing.

8\. If you maintain a sense of wonder and delight every time you get something new working, you will probably never get burnt out. Don't let anyone tell you otherwise.

9\. When you look at your old code, most of it will suck. That means you're always getting better. But some of it will still amaze you. Embrace and harness that energy. Try to recreate the conditions under which you wrote it.

10\. Name your variables properly. That will always be half the battle.

<comment><USER3> Here are my two cents:

11\. Learn to read, navigate and understand code! Unless you are working alone (but even then) you will spend a lot of time "uploading" code to your brain. Code is read more than written.

12\. Algorithms are not nearly as important as some people make it look like. I can't really remember if I ever wrote a sorting routine that is used in production and I am still waiting for the moment A* will be useful. And when that moment comes, I will just need to _read_ it on wikipedia.

<comment><USER4> Re 12: It happens (sometimes? eventually?). It took me 15 years though until that point came. And the answer was just looking at Wikipedia and other code samples in other languages to get some understanding of it. I also implemented it wrong and it kind of worked (not as well, but well enough) and only realized my mistake months later.

</comment></comment></comment><comment><USER5> Biggest lesson after my 29 years of programming is the difference between the best solution, and a successful solution. I.e. getting familiar with _system_ patterns (architecture) that work for your chosen stack/area of interest.

There are a million resources for specifics, like doing a server redirect or creating a jump list for Windows Phone, but precious little on the end-to-end implementation semantics of, say, a smartphone app that syncs content to a server that scales out and up. Also, adding security, monitoring hooks, and exception and configuration management. The realisation that open standards are far, far more important than open source.

Similarly, the web is full of people _telling_ you to use Mongo, but again hardly anything about being able to reference indexes in a document, or that partition tolerance and availability come at the expense of consistency.

Get these things right and suddenly even using GOTOs, or the differences between closed and open source, become non-issues because if your solution _works_ , and is _successful_ , any ciritics of such semantics can go to hell.

<comment><USER6> _a smartphone app that syncs content to a server that scales out and up_

This made me chuckle. I recently designed an entirely new way of doing just that (to me, at least), and it's probably the 20th or more end-to-end approach I've developed over the last seven years on that specific problem.

I don't know why scalable sync interests me, but it does. :)

<comment><USER5> I'm in _exactly_ the same place.

</comment></comment></comment><comment><USER7> A key for learning anything is being _curious_ and _exposed_ , in combination. A great way to get that as a beginner (in coding or in a new environment) is to take on bug fixes and CRs. While not always everyone's favourite it forces you to dig into others code, see things you like/dislike, grok new concepts, and challenge the inner-you with doing what's right vs. what is faster. Perfection vs pragmatism - knowing what to side with and when is a key to a long career.

<comment><USER8> What are CRs?

<comment><USER9> Change Requests?

</comment></comment></comment><comment> vezzy-fnord Any particular reason for linking to a primer on C++ pointers, instead of C pointers?

It's rare that you'll need to use pointers that much in C++ (compared to C where they're definitive and essential), and type casting is stricter in C++.

I also laughed a little at the Joel Spolsky bit. But overall, sound advice. It beats most of the empty fluff that the "learn to code" movement is producing.

<comment><USER10> Lack of pointers in C++ has been the bane of my C++ experience.

<comment><USER11> What do you mean by "Lack of pointers in C++"? C++ has most of the pointer stuff from C (except for some unsafe casts).

<comment><USER12> You should avoid it though.

</comment></comment></comment></comment><comment><USER13> Nice piece. This definitely resonates:

"Braces should go on the next line. Braces should go on the same line. Use tabs to indent. But tabs are evil. You should use stored procedures, but actually you shouldn't use them. You should always comment your code. But good code doesn't need comments."

<comment><USER14> Apart from some of them _are_ the wrong way.

Work on an over-commented code base and come back and tell me commenting everything is good.

Work on a large crud app where it's 'SPs only' and come back and tell me SPs are good. Worse still that particular debate is _dead_. The parametrized queries not only won, they blew SPs out of the park with ORMs. Hence the articles linked are 7-8 years old. And yet I still come across the occasional programmer who uses SPs (who will generally think ORMs are evil rather than massive time savers).

As for commenting, new coders _need_ to comment because they forget what they did. New programmers can't read code properly, they need help from comments. So it's good general advice to tell newbies to comment while they're learning. But not in production code.

New coders also tend to get a revelation when they go back to some of the first code they wrote and can't understand it. The revelation is 'I should have commented it all!'. But the revelation is false, there are two reasons they can't understand it. Firstly, it's crap code because it's their first code. Secondly, they can't read code properly. But they can't admit it to themselves because they think that _now_ they're good coders.

I've seen this time and time again, programmers with less than 2 or 3 years experience often won't try to read the code and figure out the logic when something goes wrong. Be it their own code or a bug in someone else's they've been asked to fix. They start changing random variables or commenting out lines. And yet they honestly think they are good coders.

Many time I have sat down with them to help, read the code and then pointed at a line and said 'Hmmm, that looks wrong'. And that will be the bug. 5 min fix, if you can read code. It's just part of the learning process and now I'm older I can see the Dunning–Kruger effect as stupefyingly obvious in programmers. I don't begrudge it, I don't think they're stupid or unskilled, it's just a stage every programmer seems to need to go through. The stage where they still haven't realized they can actually read the code to figure out what it does.

What's funny is that I can see _I_ suffered from it constantly in my early years and was a right arrogant idiot.

<comment><USER15> I disagree with some of your points, and I have experienced what you pointed out should be needed to bring that conclusion. This stuff isn't cut and dried, so i just go with whatever the project is already using. Seems easier that way :)

</comment></comment></comment><comment><USER16> > Before you learn to code, think about _what_ you want to code

This is and always has been my problem. I like to look at what other people do, and I like coding in theory. But the little I need for myself isn't enough to keep pushing myself. I don't work anywhere near programming, and I don't particularly want to, either. Because of that, though, I haven't progressed very far into SICP, my finished Scheme programs can be counted on one hand, and most of my shell scripts are simple <1h (including looking things up) hack jobs.

I only really script for personal use, and my personal usage patterns are either not terribly demanding or too complex for me to satisfy.

<comment><USER17> Try automating some of your work. Pick one simple thing that you do a lot, and write a small script or program to automate it: getting a handful of files from somewhere; putting a handful of files somewhere; opening one or a handful of files; creating the skeleton of a file that you will later flesh out.

I'm guessing you probably work in Windows because most people do at work; their shells are either too simplistic and awkward (cmd) or nuclear-capable (psh). Install cygwin and use the bash shell. Or do everything in Scheme/Racket and don't worry about a shell.

Automate something today, something almost too simple to automate. Repeat.

<comment><USER16> openSUSE & zsh all the way! ;) In all seriousness, though, my setup is a wee bit unusual for somebody who _isn 't_ a programmer, since I work exclusively in emacs via LaTeX & org-mode, stuffing all of it into git, on a openSUSE system, zsh as my standard shell, i3 as my window manager etc.

The thing is, I do have a script for example to push the current git project to various remote repositories or to create a new empty remote repository on those servers, set it up locally etc. But there are few other things that need to be scripted. I'm sure some of my things violate all kinds of sensible programming standards, but they do work (e.g. looping through shell arguments and then using case "$1" instead of using getopts, because it only allows long arguments in a hackish way).

The problem, essentially, is that of the plateau. If I wanted to do something that pushes things further, I'd need a _real_ project, not the odd, occasional shell script. But for that, I lack the motivation, since, after all, it's not really what I should be doing. It's a bit of a circular problem.

edit: My "job" is studying literature.

<comment><USER18> > _edit: My "job" is studying literature._

I thought someone in your position might use regex fairly frequently.

<comment><USER16> Mh, I don't really. There are few things that need to be changed all at once. The only recent time I needed regex on a wider scale was when I botched org- mode radio targets after a recent syntax change, which was a simple case of replacing >> with >>> and << with <<< respectively, as well as extracting some text from within square parenteses and put it on its own instead. But 1 to 2 lines of sed -i are not thaaat challenging! ;)

<comment><USER19> That poster probably meant things such as quantitative text analyis, comparing language patterns used in literature, etc - which is mostly a question of interpretation, but requires some skills in scripting for the initial data gathering.

</comment></comment></comment></comment></comment></comment><comment><USER20> I agree with all of the points of the author. I'm currently in the process of teaching myself iOS development (nathanmorgan1.wordpress.com - for anyone who's interested) and have experienced some of the issues the author addresses first hand. I'd like to say little on 'teach yourself to code' programs.

I often see these type of programs criticised here but personally, I can't say enough good things about CodeAcademy and it's ilk. Even though I'm learning iOS development its learn Javascript tutorials proved invaluable for me. It's true, I wasn't building a 'real' project but the courses demonstrated to me that I had an aptitude for coding and encouraged me to keep going with their light-hearted approach and 'rewards'.

When I came to start learning iOS development, I had a range of transferable skills and knowledge that I could call upon, for example I was comfortable with loops, declaring variables and manipulating/defining objects.

So I think while these courses won't take you from beginner to employable or experienced, they do serve their own purpose.

<comment> FLUX-YOU >So I think while these courses won't take you from beginner to employable or experienced, they do serve their own purpose.

This has been the deceptive part to me (and maybe it's only me). They're a great way to get new syntax down, but as I've learned syntax from a few languages now (Python, JavaScript, C), I really don't need these programs to serve _this_ purpose. I learned how to learn new syntax from learning syntax.

Determining when you're employable can be difficult (entry-level at Google vs. some LAMP web shop) and I'm sure that some portion of people trying these programs are looking to switch careers. I found it easy to get the notion that "If I learn through a bunch of courses and books and maybe do a few projects, I can put these languages on my resume and get hired". That, unfortunately, turned out to not be the case.

<comment><USER20> I agree with you, it is equally possible to learn syntax in other ways, whether that's from a book or tutorial etc but I think these programs reach people that may not otherwise have considered starting to code. They produce a zero barrier to entry, needing zero expense and minimum effort to get started.

They also make the initial stages of learning to code 'fun' (I'm aware that's a highly subjective term however) when learning syntax, often without having grander goal in mind e.g. an app, can be a dry subject at best.

On your second point, are you referring specifically to people who complete these courses or people who teach themselves to code and build a few projects?

EDIT:typo

</comment></comment></comment><comment><USER21> The experienced programmer knows it won't work first time, and does not wait until its finished before testing it. The time spend finding a bug is cubic in the distance you have coded since last thorough testing. You test the foundations are rock solid before building a cathedral on top of it. I find the newer blood writes way too much code in one go.

</comment><comment><USER22> I'm finally at a point where some programs I write work the first time. It's very disconcerting.

<comment><USER23> Oh familiar feeling. "There must be something wrong, it can't just WORK!"

</comment></comment><comment><USER24> Good post. Though my experience with this one was slightly different:

 _Someone will always tell you you're doing it wrong_

In my case, it wasn't that people were telling me I was doing it wrong, it's that I saw all these people asserting the Right Way to do things and I was so worried about doing it wrong that I would get bogged down worrying about tiny irrelevant problems. But in the end what people cared about most was getting things done, and usually didn't care as much about the methods as one would believe reading the web.

The most common traits of all the best code I've worked with is _consistency_ and _thoroughness_.

</comment><comment><USER25> The executing process belongs to _you_ \- you should allow it no secrets in its basic running.

First thing in any lang: learn the tools available for debugging and quickly opening library code to trace execution. Set breakpoints; dive into stack traces.

</comment><comment><USER26> Like everything in life... Practice Practice Practice.... best advice to get good at what you do.

</comment><comment><USER27> Well, HTML is not a programming language, given that it is not turing complete. Hence, knowing how to write HTML doesn't make someone a programmer.

<comment><USER28> A researcher in programming languages once told me that when she encounters the "HTML isn't a programming language" meme she always thinks: Agda and Coq aren't Turing complete, are they also not programming languages?

The point being, I think, that drawing this bright line at Turing completeness doesn't hold up when you give it any thought, and so it's not very useful. Aside from that, you've utterly missed the point of the OP.

<comment><USER29> How shall we define a programming language? At least as powerful/expressive as a DFA? I kind of like that. (A bit off topic, but not really - writing out a regex is sometimes harder for me than C++/Java/JavaScript. +1 for DFAs!)</comment></comment></comment>


Show HN: Scrape the web by injecting JavaScript into web pages - <AUTHOR> http://espion.io/ <comment><USER2> This is cheerio with node.js and extra bells. Example: [http://projs.hackhat.com/the-making-of-the-hackernews- crawle...](http://projs.hackhat.com/the-making-of-the-hackernews-crawler- interesting-tips-to-know/)

<comment> dale-cooper Sounds more like phantomjs/casperjs.

<comment><AUTHOR> Espion is similar to PhantomJS in that they're both headless browsers that you can inject JavaScript into.

Espion comes with a lot more. First there is the infrastructure: processing power, storage, connectivity and IP addresses that you don't have to provision, set up or manage. Second, Espion includes the features that surround extracting data from a site such as job scheduling, data quality monitoring, online debugging and problem resolution and data delivery.

PhantomJS is perfectly viable, but if you need the features I highlighted and use it, you'll have to build a lot yourself to get the job done.

<comment><USER3> So, is the Espion headless browser based on an existing one? If this is the case, which one?

<comment><USER4> it's just PhantomJS wrapped up behind a cloud

</comment></comment></comment></comment></comment><comment><USER5> Or for scraping anything non-trivial, do it yourself with Casperjs, AWS/Linode/DO, RabbitMQ and a bit of monitoring/alerting from someone like Datadog. It will be cheaper and a lot more flexible.

Edit: realised above sounds a bit harsh. Am sure Espion can fill a gap where clients need to scrape a limited amount of non-volatile data and don't have the time to setup and manage something on their own.

<comment><USER6> you will need to manage the pool of IPs what is sometimes the hardest part as it has to be maintained. all the setup is usually one time job

<comment><USER5> An easy solution to this is to have your scrapers on one network and a fleet of squid proxies on another. As and when an IP gets banned you just cycle in another squid instance on a new VM with a fresh IP (which can be from a completely different netblock/geolocation if you want). You don't have to touch the rest of the scraping apparatus.

<comment><USER7> I deal with this every day. Eventually we gave up on the proxy pool idea, and started running the headless browsers in the pool as Selenium nodes. It's definitely not easy- for example, we've also had to build infrastructure that helps keep track of IPs and their history.

We've open-sourced part of it. [https://github.com/cardforcoin/shale](https://github.com/cardforcoin/shale)

</comment></comment></comment></comment><comment><USER8> Since this looks to be a product targeting businesses, I would say you might want to dumb down your techie speak on the front page. Talk to more of the benefits and value that you will bring to your customers.

<comment><AUTHOR> Thanks, we'll definitely do that.

</comment></comment><comment><USER9> We run in-house data extraction structure very similar to this (spiders, phantomJS, ocr, anonymous proxies, etc), and it indeed takes some time to set it up properly. Main problem I see with turning this operation into SaaS product is that no matter how big IP pool you have, if you have significant number of clients, those IPs will eventually all get blacklisted. Unlike the small players who create small amount of traffic and can run below the radar (and thus offer the same service cheaper).

</comment><comment><USER10> It sounds a bit in a grey area, in-between legit use and something slightly dodgy (mentioning IP blacklisting and overriding CAPTCHAS...)

I'm curious to know what kind of companies / websites would need this kind of service. And wouldn't this put the provider (the website being scraped) and consumer (the people using this service) at an arms race? Can one build a solid business on this basis?

I'm genuinely curious about the use-case, not trying to criticize or form any judgment.

<comment><AUTHOR> CAPTCHAs and IP blacklistings are things you encounter routinely in perfectly legitimate and legal web scraping projects. A typical example are businesses that want to monitor their competitors' prices or the second hand market for their products.

We have plans to actively prevent the use of our platform for illegitimate purposes (fraud, spam, etc.).

<comment><USER11> > CAPTCHAs and IP blacklistings are things you encounter routinely in > perfectly legitimate and legal web scraping projects

It's only legal if the site TOS says it is.You dont get to decide wether you can scrap websites legally or not.People got sued for scraping,trust me. And it's not even about fraud or spam.

<comment><AUTHOR> It's more complex than that. Whether the site's TOS are enforceable is a matter of jurisdiction and may depend on the intent of the web scraper.

Ryanair for example has lost several cases where they tried to forbid scraping their website on the groups that data scraping promoted free competition and served consumers in general. See the latest decision here: [https://uk.finance.yahoo.com/news/ryanair-suffers-setback- ge...](https://uk.finance.yahoo.com/news/ryanair-suffers-setback-german- screen-141456801.html).

<comment><USER12> Does the location of Espion in Mauritius have anything to do with this? Are the servers located there?. Good design of the site and good work.

<comment><AUTHOR> Actually no, and I don't think our location would protect us from legal liability. Our servers are outside Mauritius, hosting locally would be very expensive and induce latency.

</comment></comment></comment></comment></comment></comment><comment><USER13> What if instead of scraping, customers use the JS injection features for spamming?

How are you approaching the liability issues involved, given that potentially anyone can sign up, create workloads of varying levels of evil, and without you necessarily vetting your customers?

I'm genuinely curious, not ranting.

<comment><AUTHOR> We approach this the same way e-commerce or gaming companies approach fraud: we'll actively monitor the infrastructure and set up safeguards to prevent spam and other illegitimate activities.

</comment></comment><comment><USER14> Can you talk about performance? Having written both vanilla non-js scrapers (which get you far enough) and js scrapers for those pesky web pages, I can confirm that our js scrapes took 10-20x the time.

Also, I am assuming dynamically changing IP addresses when you may get blacklisted is built in?

<comment><AUTHOR> The entire anonymous IP infrastructure is indeed built-in. Yes, performance is lower than non-JS-enabled scraping. To counter this we offer a variety of options to turn on/off performance trade-offs. They can be set per job and per page.

</comment></comment><comment><USER15> At first I thought it was going to be a framework like this [https://medialab.github.io/artoo/](https://medialab.github.io/artoo/), but it seems a little more nefarious than that.

</comment><comment><USER16> This is an interested idea, but what I would pay for is a GUI for scrapping, for example: clicking an element, seeing what classes it got, and be able to click each class to see if it matches the elements I need. And then generate the code for those actions I did in the GUI, maybe even be able to do both (modify the JS and still work with the GUI when required, yeah, is hard but possible)

The other major feature I need is paths of execution, for example if there are two possible pages after certain step (think if-else) I want those views visualized as interconnected nodes.

<comment><USER17> If you are looking for a GUI for scraping, check out kimono labs. It's a GUI tool for scraping, and requires no code to set up and can find all similar elements from the one you click on. It supports pagination and other types of scrapes too.

</comment></comment><comment><USER18> All I can say is ...

1) Inject javascript into webpages 2) IP pool to hife your own IP address 3) Extract text from images and solve CAPTCHAs

Do you pay these guys in bitcoin on tor?

</comment><comment><USER19> Just wondering, can the scraper record video of the site it's on? I feel like that's something I have a lot of use cases for. Even if it's not possible is there some hack you could do by taking a snapshot every 1/30th of a second and stitching it together into a 30fps video? Always been curious about this.

<comment><AUTHOR> That's a feature we never thought of. I can't promise it will be available at release time, but quite possibly later next year.

</comment></comment><comment><USER20> The packages don't state if the number of pages are per month, day or hour? We currently scrape well over 5 million pages an hour for a lot less (although, much like you we are geared up for such loads) but it would be interesting to see the cost per number of pages per hour you charge for odd-jobs/one-offs.

<comment><AUTHOR> The packages are for a set number of pages, with no timeframe. I don't have prices yet for a load in the millions of pages per hour. What kind of system do you feed data into?

<comment><USER20> A very large array of mysql databases. Basically, each month we fire up a new fresh database and start streaming data into it. Were currently pulling around 700Gb a month. Our reporting tools/systems run queries across this array. Its actually not that bad speed wise (reports of over 9000 keywords over a 1 week period for top 100 positions on a per hour basis)

</comment></comment></comment><comment><USER21> What's the benefit of injecting JavaScript into the page when it comes to scraping?

<comment><AUTHOR> You gain access to the page's own DOM and JavaScript, which lets you call the site's functions to fetch data if it helps you. You can also code your scraper with the same techniques you use when building a page: jQuery, CSS selectors, etc - basically all the good interfaces that have been developed over the past 20 years are available to your scraping code.

</comment></comment><comment><USER22> Who is your target customer?

<comment><AUTHOR> We target businesses that want to fulfil their web data extraction needs in- house rather than hiring a third-party provider – which we originally are. Espion was built for our own needs at first and we're still in the customer discovery phase. I expect we'll find many use cases in the coming months.

<comment><USER22>
        businesses that want to fulfil their web data extraction needs in-house

Well, since you offer a web scrape tool, that's obvious :) But who is the typical customer who wants to do that?

<comment><USER9> As someone who provides the scrapping services for years I can indeed confirm there's a lot of totally legit businesses who need the ability to compile the various online data. From press clipping/ soc. media trends monitoring, over various data-mining & analysis tasks, to price comparison and building dropship inventories. This days everyone talks about big data and data analysis, but you first need to collect that data.

</comment></comment></comment></comment><comment><USER23> How is it different than Phantom.js or Zombie.js ?

</comment><comment><USER24> Does this scrape website behind encapsula?

<comment><AUTHOR> Yes!

<comment><USER24> That's interesting. You should advertise that.

For anyone else wondering, it is actually spelled Incapsula, I would edit my original comment if I could anymore.

</comment></comment></comment><comment><USER25> you can do this with selenium also btws</comment>


Apprentice, Journeyman, and Master: The Medieval Guild (2018) - <AUTHOR> https://blog.philosophicalsociety.org/2018/01/10/apprentice-journeyman-and-master-the-medieval-guild/ <comment><USER2> I followed the apprenticeship model in one field, but was entirely autodidactic when it came to tech as there is so much material/content online to teach yourself if you're motivated and inclined and would prefer it remained as such. I mainly pursued one technology only to find that I had to learn so much more as I went deeper and it eventually snow-balled into starting my own fintech startup and then as a bootstrapping founder working professionally to fund the former. This was only possible because there was so much freedom within our community and free content available.

With apprenticeships you're really at the mercy of the Master who will instill in you their 'version' of how something must be done, whether you agree with it or them is entirely irrelevant to the process of edification and leaves very little room for creative problem solving and rule breaking that I think the most important work in tech actually requires.

This rigid structure is well suited for certain crafts (like electricians or plumbing) and perhaps some arts (culinary, masonry/architecture), but I fear the power that FAANG already has exerted in corrupting an entire generation of tech workers with salary and supposed _prestige_ has done enough damage already to the World, so to have them act as gatekeepers of information seems entirely myopic and self-serving to an already entrenched Industry.

<comment><USER3> 100%

Tech companies could hire and train students straight from high school instead of making them go through the academic priestly class, but nope, everyone wants expensive degrees _and_ pay more for it.

<comment><USER4> If Google did that, then Google would also be to blame for the resulting demographics not meeting everyone's standards for equality. It's really useful to be able to outsource some of the filtering process.

<comment><USER5> Agreed. Any company that can pay six figure salaries for graduate wants to be able to choose a combination of outstanding smarts and grit. Getting into an Ivy League CS program demonstrates that - and any demographic issues are provably due to the pipeline.

Exchanging that filter for hiring straight out of school but only taking, say, people with top 5% of SAT results (1420) would also demonstrate it, but it would result in even more stark over-representation of Asians (about 20% of them meet the cutoff) and under-representation of Blacks (only 1% of them meet the cutoff).

</comment></comment></comment></comment><comment><USER6> People tout Guilds and Apprenticeships as a solution to improving quality and filterting out subpar coworkers, but I don't think it would be any better, and maybe worse. Obtaining membership seems like an enormous gatekeeping device, similar to those that modern equality movements are trying to tear down. It does widen sharing skills from father to son to a community, but there is nothing stopping the same sort of exclusionary practices that we have problems with today.

Apprenticeships often boil down to grunt work and "watch me do it" for the first few years. This kind of work mode is acceptable as a teenager. But today the apprentice level would probably be until 30 or so. This education phase is currently owned by higher level academia. Many people choose industry over academia to avoid a lengthy education. If industry became the same, I think this would frustrate and burn out many people.

Those running the guild are probably not the best at a craft, just best at politicking and with the most vested interest. Those that would run a guild 900 years ago and VPs and CSuites today.

Also tech is too new. A technology can rise and fall within a decade. This makes lengthy learning periods impractical. We don't value consistency across generations (in-fact this would be now considered stagnation). This also means customers are less able to discern (or even care about) high versus low quality products.

On the flip side, education is much more readily available. A dedicate Master <> Apprentice relationships was key, because they were likely your only source of learning. This is just no longer true. Today we have so many knowledge sources that no single one gets such an illustrious title as master.

Most engineering disciplines have a professional community that requires you to take a proficiency exam (usually on graduation) and continued education. Companies will outright refuse to hire any candidates without the certifications. Programming will eventually have something similar, but the field needs to crystalize more.

<comment><USER7> You say "gatekeeping device" like it is negative. All companies I have worked for have had a system of gatekeeping that includes things like university qualifications. I don't see why this system can't be improved.

Many companies now invent their own proficiency exams. We do.

Medical graduates work in a specific setting under supervision for a period after extensive education. I don't think the conditions of their work are ideal, but many people make it through without burn out.

Although technologies are changing, there are some things that transcend technology. Concepts about like documentation, naming, decomposition, components, interfaces, concurrency, locking, algorithm efficiency, many considerations of user experience.

<comment><USER8> > Many companies now invent their own proficiency exams. We do.

Companies are not doing it to artificially restrict supply. They have no incentive to raise the cost of labour.

(On the other hand, the process could be captured by incumbent employees who do have that incentive).

<comment><USER9> (On the other hand, the process could be captured by incumbent employees who do have that incentive).

That seems to be way more common than most people want to admit. So many software engineering interviews have very little to do with the job. Worse of course is when there is no process and it seems like the whole thing is just designed to get the hiring manager friends hired.

</comment></comment></comment></comment><comment><USER10> People of the same trade seldom meet together, even for merriment and diversion, but the conversation ends in a conspiracy against the public, or in some contrivance to raise prices. —Adam Smith, observation on urban guilds, 1776

Are we out of imaginative new ideas for organizing society?

We have the ability to organize a social heirarchy more generally across the public thanks to computers. The necessity for highly specialized master/apprentice guilds looks limited to a few areas.

<comment><USER11> > _People of the same trade seldom meet together, even for merriment and > diversion, but the conversation ends in a conspiracy against the public, or > in some contrivance to raise prices._

I.e. a case of a phase-locked loop. I think viewing it like this can help spot other such loops in the society.

</comment></comment><comment><USER12> I understand there are some difficulties with this ideology in terms of technology companies and onboarding, but I have also seen the potential benefits of such an approach without explicitly attempting to engage it.

We have hired entry-level engineers that went from supporting basic production issues to writing low-level parts of the platform within 2-3 years. At no point was I consciously aware of any explicit knowledge transfer occurring, but somehow these individuals are able to pick up a tremendous amount of capability via proximity to the day-to-day process. One day it clicked for me that a pull request I was reviewing for one of these individuals was something that I would expect to see from a highly-skilled developer on the team. Seemingly overnight, we converted a QA/Support engineer who chases down basic null refs into a proper developer, and all it required was exposing them to the full depth of the process every day.

Perhaps the master/apprentice game doesn't make sense in terms of on-boarding. But, I see massive amounts of value in exposing willing employees to increasingly-difficult problems on a daily basis. Make your most junior employee install the same tools and work through the same processes that your most senior employees do. There is no reason you cannot ramp someone from zero to hero in a few years with passive exposure to your business processes.

I find that this model of constant exposure combined with probationary employment strategies provides an excellent blended approach for ramping in new talent that would otherwise be filtered out by your typical big corp HR department. Just because someone doesn't have a computer science degree or got burned by our incredibly shitty legal system doesn't mean they aren't willing to bust their ass and try to prove something to the world. I would like to give as many people a chance as possible at this. Programming is an art and no one policy is going to be the best. The more perspectives we have the better.

<comment><USER13> I don't think it has to be codified, just recognized. Too many people in this submission read into the article and assume that the guild is a gatekeeper whose primary interest is keeping trade secrets. It's the opposite in reality. The entire idea behind a guild is that you give young people with zero skills an opportunity to learn the craft. Right now the American system is the opposite. People have to take care of their own education before they can get the job by going to university. There are jobs where this makes a lot of sense and there are jobs where its just pointless busywork.

</comment></comment><comment><USER14> This history is interesting to read. The last place I worked had trade unions (electrical, mechanical) and still used those terms. There's a huge difference between a master electrician and an apprentice.

<comment><USER15> Care to tell us more ?

</comment></comment><comment><USER16> "Thus, craft guilds provided the goods and merchants fed the need: the beginning of real capitalism." Good piece but please allow me to nitpick : Not blaming the author because it is a very common and well established narrative that capitalism arose from the development of a merchant class but Ellen Meiksins Wood makes a pretty good case for an agrarian origin to capitalism in The Origin of Capitalism. She argues that the roots are instead a combination land rental and the reliance on waged labour IIRC. If the subject is of interest to you I recommend you check it out.

</comment><comment><USER17> Codify similar structures for programmers and engineers and we will reap the benefits. Organize.</comment>


Amazon details drone delivery plans - <AUTHOR> http://www.bbc.co.uk/news/technology-32653269

<comment><USER2> First sentence: "According to the patent, the drones will be able to track the location of the person it is delivering to by pulling data from their smartphone."

Further down: "Using Audi's...communications system...DHL delivery drivers would track a customer's vehicle over a specified period of time and then use a digital access code to unlock the boot. This code would then expire as soon as the boot was shut."

...well this is all a little more unsettling than having your mailman or a FedEx driver drop by your house

<comment><USER3> If the issue is someone knowing your whereabouts during the day, hopefully you consider it pretty unsettling that everyone who knows your address can be reasonably certain where you are between 6 PM and 7 AM.

That being said, I can see being uncomfortable with them having access to your trunk.

<comment><USER4> There is reasonable expectation that people _can_ drop by when you are "at home".

But when driving around on 5th avenue, seeing a DHL truck chasing you around? That's just not a reasonable expectation.

<comment><USER3> Why not?

The reason these companies deliver to an address is because historically, that is the most convenient way for them to get your package to you.

Now it would be more convenient for you (and less so for them, given the amount of effort that goes into route planning) to deliver to wherever you are.

A truck would hardly 'chase you around', but might follow you until you've parked, and give the package to you then.

I'd say the service places an unreasonable expectation on the shipping company, rather than on the consumer.

</comment></comment></comment></comment><comment><USER5> I don't see any implementatin details in that patent, only a very rough sketch of how such a system could work (but many important details missing!) It could be me though.

<comment><USER6> Yeah, I am wondering about the limits of range, package weight, recharging operations and more for this kind of use. There are probably many headaches involved in implementing such projects.

<comment><USER5> Range could be handled by having recharging stations spread out through a city (a drone could also swap batteries). However, the challenge is to keep the solution as low-cost as possible, because drones will be lost to weather, crashes, etc.

None of this is addressed though.

<comment><USER7> Why is the challenge to keep costs low?

If this means I can have something delivered in 45 minutes, or maybe 20 minutes!, they can charge a premium for that. Otherwise I wait for the big truck to bring it by within 2 days and it's free. Right now there is no solution at any cost, in my market, by which I can order almost any product from Amazon and have it within a few hours.

<comment><USER8> Amazon already does same-day delivery in more areas than you're likely to see drone-delivery in the next 5-10 years.

In most places, the fancy technology has zero advantage over employing a handful of couriers. Exceptions would include extreme geography or traffic conditions.

<comment><USER9> Straight line delivery and Amazons ability to mark a job finished more quickly are both valuable in any area.

</comment></comment></comment></comment></comment></comment><comment><USER10> Title is misleading.

Given that you can patent without implementing, and unimplemented patents can leave you open to lawsuits, organizations are forced to patent everything they they may want to do someday.

</comment><comment><USER11> A patent isn't necessarily what someone will implement. It's just one possible embodiment.</comment>


Ask HN: Sites like product hunt to get early users for your beta launch - <AUTHOR> What are sites or forums that you know I can submit my beta saas web app in order to acquire early users <comment><USER2> betalist, betabound, indiehackers, /r/alphaandbetausers, /r/startups monthly thread

<comment><USER3> To add one more into list is betapage.</comment></comment>