Bitcoin contracts - <AUTHOR> https://curiosity-driven.org/bitcoin-contracts?utm_source=hn

<comment><USER2> If I'm understanding correctly, these scripts are very long lived in that days, weeks, or years may go by before segments of the contracts are resolved. This does seem like a powerful mechanism. Unanswered in the paper is a look at the wallet implementations; can I expect wallets to correctly report the balance in a payee's wallet, if they have outstanding contracts?

The escrow mechanism strikes me as particularly powerful; it's like traditional escrow but even better because the buyer doesn't quite give up control, and the seller knows that the escrow company can't just disappear with their funds.

Activation of the escrow agreement can take minutes, rather than days or weeks for traditional escrow (usually involving wire transfers), once the escrow agreement is set up.

It doesn't solve the problem of the escrow company being in cahoots with one of the parties, but I believe the mechanism can be extended to N-way escrow where multiple escrow companies could be arbiters over the transaction. Then the buyer and seller can both have a trusted escrow company in the transaction and know that their interests are indeed being represented, and the escrow companies can determine if the other escrow company is one that they trust as well, before committing to fulfilling a transaction.

<comment><USER3> Escrow contracts in Bitcoin are now more and more used because they add real security without taking much convenience. Shameless plug [0] - I work for a startup that does exactly that, utilizing multi signature Bitcoin addressed for added user security. But you could also build a service for a true escrow very easily and as you say, that wouldn't mean putting all the trust into the escrow party, which is great. These services, in fact, already do exist [1].

Also, at least for P2SH (multi signature/escrow addresses), getting their balance is no different than for normal addresses. Completed transactions to them are perfectly visible on the network so you can sum them up. Uncompleted (partially signed) transactions are not sent to the network though, so you cannot estimate how many "pending" Bitcoins are there for a given address.

[0] [https://bitalo.com](https://bitalo.com)

[1] [http://cosign.co.in](http://cosign.co.in)

<comment><USER4> What I wonder is what happens to the money locked up in these transactions. I understand that the money is effectively inaccessible to both the sender and the receiver until the transaction is resolved. Is that right ?

I wonder about the implications of that, since it effectively takes money out of circulation, during whatever length of time it takes to finalize the transaction. That presumably includes any conflict resolution process that may exist ?

<comment><USER3> The money is sitting on special multisignature address before transaction is resolved. It's a 2 out of 3 address, which means that two signatures are required to move the funds further. The good thing about it is that you can totally skip the escrow party here if buyer and seller agree - they both just sign the transaction and send it to the network. Escrow is needed only in case of a conflict - he would have then to investigate and sign a transaction with either buyer to refund the Bitcoins or to seller to forward them to him as contract described (in a case that buyer disappears for some reason).

This setup has one downside though - if one party sides with the escrow, they could agree to sign a transaction to scam the other party. That's why it is not 100% bulletproof and you have to carefully select your escrow partner so that both parties trust him to an extent.

</comment></comment></comment></comment><comment><USER5> For many, the flexibility of Bitcoin in terms of what sorts of scripts can be executed with transactions is much of the appeal. I'm of the opinion, however, that it represents Bitcoin's greatest weakness.

In order for bitcoins to be useful for money, it must be possible to determine how many bitcoins I have, or, more exactly, how many I have available to spend. The fact that there is a language associated with bitcoin redemption, and the redemption happens at the time of the creation of the outbound transaction makes this very difficult to determine.

So, more clearly, if I want to spend some coins that have been nominally sent to me, at the time that I attempt to send them to another address, I have to have all the information necessary to redeem the coins that were sent to me, for each instance of an incoming transaction to my address.

If this sounds complicated, it is because it is. Ethereum, one of the many post-Bitcoin cryptocurrencies, does a much better job of this on the pure currency side -- spending coins at an address to which I have the private key is just a matter of saying how many coins I want to send and where. Ethereum, unfortunately (in my mind) complicates this with a much more sophisticated language that lives in a side-chain.

I think the ultimate Bitcoin successor (or modifications to the Bitcoin protocol itself) will have to prune down the scope of allowed transactions, rather than expanding it. This isn't just a question of the "my grandmother won't be able use it" \-- although having an unspendable incoming transaction from the "publishers bitclearing house" for a million bitcoins will definitely be a bit offputting.

In the end, even for a sophisticated user, it will be nearly impossible to present a view of how many bitcoins you have available to spend. This is why the core devs have been very reluctant to allow different kinds of scripts; multisig is the first real expansion.

Colored coins, while kind of fun, live outside the blockchain, so don't really bother me. It's basically the equivalent of a (for example) car saying "anyone who holds a dollar bill with serial number xxxxxx from year yyyy can start the engine", in a world where it is impossible to forge dollar bills (as is the case with bitcoins).

<comment><USER6> > In the end, even for a sophisticated user, it will be nearly impossible to > present a view of how many bitcoins you have available to spend.

This is not necessarily true. It is a UI challenge. You can imagine a UI that, for example, understand that you are participating in a crowd-funding contract, and denotes the bitcoins locked down in that contract in a special area. For the crowd-funding contract, you can exit at anytime before the contract closes, so the UI could present that option as well. This approach can be applied to other contracts as well.

<comment><USER5> I'm going to stick with nearly impossible. Yes, it is theoretically possible to have a fixed number of built-in contract types that follow a certain schema, and to make a UX corresponding to each one. But in the full generality -- you get one of these crowd-funding contracts, and the client has to figure out what kind of counterparty signing has to happen in order to spend these damn things. Without prior knowledge of the structure, I'm guessing that this verges on intractable.

Of course, with a finite number of transaction types, this is easy, right? Well, not even then. Because you don't redeem these things when you _receive_ them. You redeem them when you _spend_ them. There's no easy solution here; most likely the clients will have to do a one-time sweep of receptions that it knows how to complete to an unencumbered transaction input, so that they can be spent without having to gather a bunch of documents. But at the very minimum, a very challenging experience compared to cash, or even the "standard" transaction inputs.

</comment></comment></comment><comment><USER7> On a related note - all the blockchain contracts are limited by their inability to access real world data (e.g. stock prices, websites). For that you need distributed oracles: [https://github.com/orisi/wiki/wiki/Orisi-White- Paper](https://github.com/orisi/wiki/wiki/Orisi-White-Paper)

<comment><AUTHOR> Exactly - scripts are pure functions. Reasoning behind this is explained by Satoshi himself at Bitcoin Talk [0]. Contracts page at Bitcoin wiki has an example of defining a will using Bitcoin and an oracle [1].

[0] [https://bitcointalk.org/index.php?topic=195.msg1611#msg1611](https://bitcointalk.org/index.php?topic=195.msg1611#msg1611)

[1] [https://en.bitcoin.it/wiki/Contracts#Example_4:_Using_extern...](https://en.bitcoin.it/wiki/Contracts#Example_4:_Using_external_state)

<comment><USER7> Yup. The concept of a blockchain is fundamentally incompatible with external inputs. That's because at any given moment you need two things: \- everyone to agree that the new block is valid (and you can't really do that if blocks contained functions that had to be verified with external/changeable outputs) \- everyone has to agree that all the past blocks were valid, which is even harder, because if blocks referenced external outputs, the sites/sources they relied on may have changed

That's why not only Bitcoin, but also Ethereum and all the other blockchain- based currencies need oracles for passing external inputs in.

By the way, here's our repo, if anyone's interested in the whole concept: [https://github.com/orisi/orisi](https://github.com/orisi/orisi) \-- there are tutorials there explaining how to set up an example oracle, and an example client sending stuff to oracles to sign.

There's also one of the core devs explaining this stuff: gavintech.blogspot.com/2014/06/bit-thereum.html

</comment></comment></comment><comment><USER8> Wow, this is superb educational content.[1] Is the author anonymous?

1\. also [https://curiosity-driven.org/low-level-bitcoin](https://curiosity- driven.org/low-level-bitcoin)

<comment><USER9> The author does appear to be anonymous. By the systematic lack of articles (a lot of noun phrases seem to be missing "the"), I would guess a speaker of a Slavic language. Possibly Polish, given the DNS.

</comment></comment><comment> a-b [http://trustatom.com/](http://trustatom.com/)</comment>


Know your tools: JVM's jstack - <AUTHOR> https://devolution.tech/know-your-tools-jstack/ <comment><USER2> jstack does only one thing but does it well :) It is one of the most useful tools while debugging JVM apps and everybody should get to know it.

<comment><USER3> yeah but it somehow very hard to automate/use it in containers where access is strictly censored. APMs like Newrelic provides similar capabilities.</comment></comment>


Organizing Go code - <AUTHOR> http://blog.golang.org/2012/08/organizing-go-code.html

<comment><USER3> This _almost_ feels like it's burying the lede:

      (The gofmt command has a -r flag that provides a syntax-aware search and replace, making large-scale refactoring easier.)

I've been writing Go on some toy projects off and on, and I'm not an expert by any means, but I had no idea this flag existed. I must've missed when reading about gofmt. :(

Anyway, it sounds like it could be pretty useful for refactoring.

</comment><comment><USER4> "A good name is the most important part of a software interface" - <USER2> true. Naming is so important and yet so many apis spend seemingly so little time on this - win32, OpenGL I'm looking at you guys ;

<comment><USER5> "There's only two hard problems in software engineering: Naming things, cache invalidation and off-by-one errors"

</comment></comment><comment><USER6> This all makes a lot of sense for non-main packages. I am curious though what people are doing with their main packages and their non-code dependencies. Say you were writing a web app in Go, or some other executable. Is the convention to put main packages in src/cmd/somename? And is there any convention for the location of related files such as templates, configs, etc.?

What I've been doing recently is keep web apps in separate gopaths, and having code in src/ like usual (with src/cmd/frontend as an example, then a 'go install' outputs to bin/frontend), and then relevant dirs to the app: config/, templates/, etc. all in the root of the gopath, much like a typical project directory tree I suppose. However, I get the impression this is not the 'right' way to do it.

<comment><USER7> You should put your entire project under $GOPATH/src/path/to/project, including non-code dependencies and main packages.

<comment><USER8> This annoys me immensely. I understand a standardized format for installed packages, but I should be able to dictate the hierarchy of my own code....

<comment><USER9> If everyone dictated their own hierarchy for their code base, utilities like go build and go get wouldn't work.

The guidelines are just guidelines - you can still do whatever the hell you want, but don't expect the standard tools to automatically handle all your own personal preferences for project structure.

Go had has this structure ever since the Go 1 release - your source goes in $GOPATH/src/$YOUR_PATH, and the go get/build tool will compile any main packages to $GOPATH/cmd, and all packages to $GOPATH/pkg/$YOUR_PATH.

If you really want to isolate the code for a specific application/project, why not use a folder structure like $GOPATH/src/$PROJECT/$PACKAGE or $GOPATH/src/$PROJECT/$EXECUTABLE. I had problems initially adjusting to the Go 1 workspace changes, but it's really not a big issue once you adapt a little (like anything in Go).</comment></comment></comment></comment>


Microsoft-Amazon patent deal covers Kindle, Linux - <AUTHOR> http://arstechnica.com/microsoft/news/2010/02/microsoft-amazon-patent-deal-covers-kindle-linux.ars?utm_source=rss&utm_medium=rss&utm_campaign=rss <comment><USER2> _Microsoft says it has reached more than 600 licensing agreements since launching its intellectual-property licensing program in December 2003._

Wonder how much they've made so far from this new line of business.</comment>


Rhode Island expands car stops from only New York to all states - <AUTHOR> https://www.bloomberg.com/news/articles/2020-03-29/rhode-island-ends-policy-of-stopping-cars-with-new-york-plates <comment><USER2> It feels like Cuomo is on the wrong side of this. Like it or not, national and state boundaries are the only boundaries we can reliably use to segment the general population right now, and while it may be rudimentary, it's one way of identifying those coming from a very dense area with a very high rate of infection.

<comment><USER3> But Rhode Island's stops _aren't_ segmenting the population. They're not turning anyone around, just giving out-of-state travelers a verbal warning that they need to self-quarantine for 14 days.

It's the worst of both worlds: the extremely shaky legal situation of a real border lockdown, without any of the supposed benefits.

<comment><USER4> A simple sign can probably accomplish the same thing.

<comment><USER5> Most people won't pay attention to signs. When the police stop you, you'll remember.

</comment></comment></comment></comment><comment><USER6> I live in Rhode Island, and it's been interesting to see my own gut reactions to this.

The _short_ summary is:

\- I'm sympathetic to their urge to escape ground zero.

\- But I'm angry at those who put Rhode Islanders at greater risk when they ignore the stay-at-your-residence order from our governor.

\- It's a little frightening to see the xenophobic and reactionary feelings this brings me. I feel a strong tug towards us-vs-them mentality.

</comment><comment><USER7> I'm interested what the legal basis is for this? I thought that the US constitution provided fairly unconditional protections, unlike say in Europe. (Not saying this is right or wrong, just wondering how the technicalities work in the US).

<comment><USER8> One of the fairly explicit exceptions to federal jurisdiction on interstate travel is precisely to prevent the spread of diseases, both human and agricultural. Agriculture inspections and quarantine type cost recovery are pretty much the only "taxes/tarrifs" the states are allowed to prescribe on interstate (and international) travel and trade.

From US Constitution Article 1 Section 10:

No State shall, without the Consent of the Congress, lay any Imposts or Duties on Imports or Exports, except what may be absolutely necessary for executing it's inspection Laws

Like much of the US Constitution, it's both a very loaded and ambiguous statement, with much of it's meaning implied.

</comment></comment><comment><USER9> The ironic thing is that while the wealthy 1% are driving to their second homes in southern Rhode Island, the ultra-wealthy 0.1% are flying in their private jets to the state's regional airports and are ignored. The governor has admitted that the state can only afford to police one airport, TF Green.

<comment> cbm-vic-20 The 0.1% only go to Rhode Island during yacht racing season in Newport. I wouldn't worry about the three of them going to RI during all this.

</comment></comment><comment><USER10> The 'house by house' bit with the national guard is disturbing. I'd be concerned with someone pitching girl scout cookies door to door, much less something like this.

As folks are getting back from spring breaks... I suspect it is too late to really stop the spread. If I had vacation property in a sparsely populated area, I'd take full advantage of it. (As things stand for us, we have more ID'd infections in our county than RI, but several orders of magnitude under what is happening in NYC right now)

</comment><comment><USER9> If all New York residents are under a shelter-in-place directive, then why is anyone with a home in New York driving to Rhode Island or to any other state? In other words, why isn't Cuomo policing his own border?

<comment><USER11> Why would Cuomo spend his resources policing the rather enormous NY state border? What does NY have to gain from that? This whole thing smacks of the decision to suspend travel to and from Europe. Maybe not the worst idea, but also ultimately not going to make that much of a difference. This virus is already everywhere. Rhode Island and Florida want to blame all their trouble on New York rather than their own social distancing decisions.

</comment></comment><comment><USER12> Will the police stand six feet away while they issue these instructions?

I mean what could possibly go wrong by systematically exposing your law enforcement to every person that enters the state at the moment of their arrival?

<comment><USER2> They probably will. I don't think it's reasonable to think they are throwing caution to the wind and not taking some measures to protect the folks working at these checkpoints, who are trained professionals following orders.

</comment></comment><comment><USER13> Not to be obtuse but I'm from eastern Connecticut and Rhode Island's "southern border" is with the Atlantic, not CT.

</comment><comment><USER14> So the real question is.. what happens if you do not stop? As I stated before, most reasonable people allow for a lot of leeway to the government during a crisis, but.. the government response has been inconsistent ( at best ). I don't feel good about it at all. It makes me feel like Super Sad Love Story was basically a prophecy.

</comment><comment><USER15> Can't they introduce border controls like in Europe?</comment>


Decentralized, end-to-end encrypted Cloud Storage - <AUTHOR> https://storj.io/ <comment><USER2> Interestingly, just yesterday this was trending on HN: [https://shitcoin.com/storj-not-a-dropbox- killer-1a9f27983d70](https://shitcoin.com/storj-not-a-dropbox- killer-1a9f27983d70)

<comment><USER3> FWIW I did a similar test with IPFS recently and had no problems and it worked phenomenally. I know people have issues with Filecoin (IPFS's native token) and I myself am confused about some of the outstanding unsolved issues with it, but IPFS itself works really well.

<comment><USER4> I've been very curious about IPFS, but I can't find any descriptions of the limitations and guarantees of the protocol.

For instance, what is the durability of the files I store there? How long do they exist? How much can I store? What is the availability of the files stored? What about latency?

<comment><USER3> Great questions, and not to be unnecessarily pedantic, but IPFS is not a method of storage. IPFS is a replacement for HTTP and has a lot of attractive qualities and works in production today.

Filecoin is a method of incentivizing decentralized storage on IPFS and many of the questions you've asked are either totally or partially unsolved.

Those questions will ostensibly be solved by groups (possibly Protocol Labs, the creators of IPFS and Filecoin, possibly others) over time. That's kind of the bet you have to make if you buy Filecoin, but it's not a bet you have to make to use IPFS today or build a solution on IPFS that works at least for your ends but not all ends.

Hope that makes sense - it's worth digging in more on the forums themselves as to which of those problems are more or less solved and what the possible solutions are.

<comment><USER5> What is to stop it from just being used for storage though? I see no reason to just use IPFS as a place to dump static files, besides ethical ones.

<comment><USER3> Sorry just to be clear - you can literally not use IPFS for storage, it doesn't "have" memory or compute or anything. It's a protocol.

</comment></comment></comment></comment></comment></comment><comment><USER6> I may have missed something in the whitepaper, but using a confidentiality- only encryption scheme like AES-CTR seems bad because it enables trivial attacks on file integrity (bit-flipping attacks and such). How does Storj protect the integrity of a file? I see that proofs of retrievability are used, but PoRs don't guarantee protection against integrity attacks in general.

</comment><comment><USER7> Hard drives are famously prone to failure: I understand that filecoin and storj pay 'miners' to store files, but what happens in case of failure? It doesn't seem that there are any penalties for losing files (other than losing out on the expected income) – this does not seem very resilient.

</comment><comment><USER8> This, along with Siacoin ([http://sia.tech/](http://sia.tech/)), will likely be the future of decentralised cloud storage.

They have active developments and have (somewhat) clear roadmap. I'm a bit concerned about their strategy to overcome the big ones right now such as Google Drive, Dropbox, etc. Seems like a really long way to go.

</comment><comment><USER5> The "25GB free" thing bothers me. Nothing wrong with free stuff, but this sounds like something rather undecentralized.

</comment><comment><USER9> Let me be the first to say this is probably a terrible idea.

Some reasons:

1\. Decentralized storage? What the hell for?

2\. Blockchain hype anyone?

3\. "Where only you have access to your data" It's called NextCloud, or OwnCloud. This app should be called WheelReinvented.</comment>


Computer scientist Viral Shah helped build Julia from Bengaluru, India - <AUTHOR> https://qz.com/963225/julia-an-indian-computer-scientist-built-a-new-programming-language-from-bengaluru/ <comment><USER2> I don't have an opinion on the 'built in Bangalore' part, but I know Viral (live in the same apartment block and run across him occasionally and talk tech) and can confirm he is quite the agent of change. In addition to being immensely talented of course.

In a world of people trying to puff themselves up, Viral Shah is very understated and humble and flies somewhat under the radar. Good to see his contributions are getting more publicity.

<comment><USER3> Ravi, you are very kind as always!

</comment></comment><comment><USER4> As an Indian working in India, the 'in Bengaluru' part actually makes me a bit sad. While Shah may have physically been in India when he was working on Julia, the project might not have happened without his having gone to the US for graduate studies. I can't think of any comparable project started entirely by people working in India.

The reason is not lack of ability: my guess is that at any time India's best institutions like the IITs or IISc have at least a hundred people who could build something like Julia. Instead it is perhaps India's poverty that prevents people from starting something that does not have an immediate academic or business payoff.

<comment><USER3> While it is true that my PhD experience at UC Santa Barbara was crucial to my network of collaborators and work on Julia, there are several contributors to Julia from India who have not gone for graduate studies abroad. There are many contributions from undergraduates as well. I am just glad that Julia is a living breathing open source project with significant contributions and work from around the world, including India.

I don't think that IITs or IISc or institutions are deciding factors. I feel that people who do interesting stuff (not just in software, but in all aspects of life) do it despite their backgrounds, conditions, and other labels. Of course there is poverty, and lack of basic education for millions of people, but there are also millions of people in India, and the rest of the world, who have the skill to do very interesting things.

<comment> happy-go-lucky "While it is true that my PhD experience at UC Santa Barbara was crucial to my network of collaborators and work on Julia, there are several contributors to Julia from India who have not gone for graduate studies abroad. There are many contributions from undergraduates as well. I am just glad that Julia is a living breathing open source project with significant contributions and work from around the world, including India."

It reminds me of the collaboration between Hardy and Ramanujan. To me, the significant contributions from around the world, including India, represent Ramanujan working on his own in India while your PhD experience at UC Santa Barbara could be comparable to that of Ramanujan working in collaboration with Hardy at Cambridge. Hope that is not an overstatement. Your work should serve as an inspiration to many in India. Thank you for giving us a language to code in.

</comment></comment></comment><comment><USER5> Why every Indian programmer has a burdon to prove that he/she is not an IT coolie ? How's is that less racist than some redneck assuming all black people are thugs ?

<comment><USER6> I am from Hyderabad, India and felt racist reading that title - "IT coolies". I believe the author just want to make headline.

A lot of innovation happens that gets unnoticed in Silicon Valley or around the world. People here do not have access to mentorship & peer network so easily (openly available in California), still being self-motivated and connected by reading tech blogs, advises, etc. Just top of my head are following two examples.

* Little Eye Labs => Facebook * HackerRank ==> YC

<comment><USER7> Perhaps OP is just trying to point out that talents aren't just in Silicon Valley.

It can also be seen as a counter example against those other articles about how Indian programmers aren't qualify.

I don't see it as a negative race baiting whatever but more as the above sentence (goto: sentence above).

</comment></comment></comment><comment><AUTHOR> The original HN post in 2012. [0]

[0] [https://news.ycombinator.com/item?id=3606380](https://news.ycombinator.com/item?id=3606380) / [https://julialang.org/blog/2012/02/why-we-created- julia](https://julialang.org/blog/2012/02/why-we-created-julia)

</comment><comment><USER8> It's a comparison between apples and oranges. If he is creating a language he is at least 10x guy and they are in high demand and don't have to bother about the things 1x devs have to worry about like... getting a job or getting financial stability. The problem is always with 1x guys taking jobs of another 1x guys from another nation.

<comment><USER8> Simple economics when supply is in shortage demand increases. The shortage for 10x devs are so much that the worst a 10x guy from another country can do is reduce the salary of another 10x guy. But with the 1x guy it's a live or die situation, their job is in danger because 1x guys are too much in supply.

<comment><USER9> That's sad

</comment></comment></comment><comment><USER10> Viral, how far away are we from a deep learning framework in Julia ? It looks like there are many streams, from GSoC mentoring proposal for what seemed like a CuDNN layer, to a LLVM backend for CUDA/SPIR-V. I figure it'll take atleast 2 years for things to stabilize.

<comment><USER3> I assume you are not speaking about MXNet.jl and TensorFlow.jl, which are really good, and in some ways, even better than the Python wrappers for those libraries.

There are several pure Julia DL frameworks - Mocha.jl, KNet.jl and Merlin.jl. What will make things really exciting is native GPU codegen, which has recently landed in Julia, but will take some time to stabilize and make its way into these packages.

We do have a deep learning workshop and a GPU workshop in this year's JuliaCon covering the state of the art and the way forward.

</comment></comment><comment><USER11> I know at least one YC fellowship co that was using Julia in production.

There was like 1 evangelist at CMU who used it for their work. The deep learning wave kinda took things in a rather different direction.

</comment><comment><USER12> What kind of introduction is this?

"In India, the formula for success is drilled into children from a very young age: get good grades, go to an elite Indian Institute of Technology (IIT), move to America, and never come back."

So parents will drill this into their kids? Never come back?

</comment><comment><USER13> Alternative is the new black. Or maybe it is the new indie?

</comment><comment><USER14> Why can't people just settle on Python? Julia is going to die anyways in the near future.

<comment><USER15> The same reasons people migrated to Python in the late 90s from matlab, Fortran and C++: Python addressed pain points in those languages and ecosystems. Julia addresses deficiencies in Python the language. Unfortunately, it lacks the ecosystem: until it gets the rich library set that Python has, we're stuck with Python since ecosystem is arguably more important for practical traction than the base language.

<comment><USER16> I don't know how well it works, but Julia does let you use python libraries.

        @pyimport matplotlib.pyplot as plt
        x = linspace(0,2*pi,1000); y = sin(3*x + 4*cos(2*x));
        plt.plot(x, y, color="red", linewidth=2.0, linestyle="--")
        plt.show()

[https://github.com/JuliaPy/PyCall.jl](https://github.com/JuliaPy/PyCall.jl)

<comment><USER3> Works really well and a lot of python scientific software is part of a typical julia install.

<comment><USER16> Glad to hear that. Will it be possible to overload . in the future?

<comment><USER17> There's been extensive discussion of that and yes, it may well happen [1]. The biggest concern at this point is how to avoid having people abuse such a feature by trying to fake out class-based o.o. code in Julia in a way that will look right but work poorly.

[1] [https://github.com/JuliaLang/julia/issues/1974](https://github.com/JuliaLang/julia/issues/1974)

</comment></comment></comment></comment></comment></comment><comment><USER18> Reading this post it seems that Julia is the silver bullet for every programming problems (or at least the ones addressable in python). I guess it's not. It is just another (very useful, admittedly) tool for the job. I really can't stand when every new language or framework or technology or whatever tries to be _THE_ silver bullet. How long it will be before we learn that such a thing doesn't exist and we are always going to learn new tools to solve problems in a hopefully more efficient way? Maybe in the future we'll have such a thing, but I bet my 2 pennies that it won't be either Julia or any of the existing languages.

Edit: speaking specifically about the language I think that it's a very good effort and it can be only a good thing for the current languages ecosystem, even if I don't appreciate some of its idiosyncrasies.</comment>


Ask HN: Bloggers who talk about AWS? - <AUTHOR>

I'm growing increasingly interested in becoming well versed in AWS, and I would like to spend some serious time reading and learning more.  Other than official documentation, what are some recommended readings about AWS? <comment><USER2> I would like to know other areas besides documentation as well. Mostly, I've only found piece meal blog posts, etc regarding how to solve specific AWS issues, etc.

AWS engineering blog is good - [http://aws.amazon.com/blogs/aws/](http://aws.amazon.com/blogs/aws/)

AWS forum has helped me solve issues I've run into - [https://forums.aws.amazon.com/](https://forums.aws.amazon.com/)

I can sometimes see new methods for approaching things with AWS tags on StackOverflow

High Scalability occassionally covers robust architecture learnings eg: [http://highscalability.com/blog/2013/11/5/10-things-you- shou...](http://highscalability.com/blog/2013/11/5/10-things-you-should-know- about-aws.html) [http://highscalability.com/blog/2010/8/16/scaling-an-aws- inf...](http://highscalability.com/blog/2010/8/16/scaling-an-aws- infrastructure-tools-and-patterns.html) [http://highscalability.com/blog/2014/3/11/building-a- social-...](http://highscalability.com/blog/2014/3/11/building-a-social-music- service-using-aws-scala-akka-play-mo.html)</comment>


TensorFlow, Keras and deep learning, without a PhD - <AUTHOR> https://codelabs.developers.google.com/codelabs/cloud-tensorflow-mnist/#0 <comment><USER3> As a researcher in the field I am not quite sure how I feel about these kind of resources. I am all for making research accessible to a wider audience and I believe that you don't need a PhD, or any degree, to do meaningful work.

At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result on arXiv so they can put AI research on their resume. This has resulted in so much noise and low quality work that it really hurts the field.

You don't need a degree, but I think you do need to spend some time to get a deep enough understanding of what's going on under the hood, which often includes some math and takes time. This can be made accessible and there are plenty of good resources for that. But all these "become an AI pro by looking at some visualizations and copying this code" is maybe hurting more than it helps because it gives the illusion of understanding when it's actually not there. I wouldn't want people learning (solely) from this touching my production systems, writing blogs, or putting papers on arXiv.

<comment><USER4> Well, maybe I got the wrong impression but after reading the (very accessible) Yolo V3 paper [1], it seems to me that even the experts do little real math and lots of guesswork, kicking a model until it starts giving results.

[1] [https://pjreddie.com/media/files/papers/YOLOv3.pdf](https://pjreddie.com/media/files/papers/YOLOv3.pdf)

<comment><USER3> Oh yes, I don't think advanced math is required in any way. However, there is a difference in that researchers who have worked with these models for many years (often including having done some of the math) have a very good intuitive understanding of these models. Once you have that, it's fine to be driven by gut feeling. Just like many engineers are driven by gut feeling that come from tacit knowledge through experience.

What is dangerous is reading a 30 minute blog post and getting the illusion of having some kind of understanding, when in reality it can take years to develop that. It's like cloning the postgres Github repository, compiling it, running a few queries, and then saying you've built databases and being hired to become the "database expert" at some company, spreading wrong knowledge left and right.

That's why the popularization of these quick immediate reward tutorials is dangerous. It takes time and effort to become knowledgable at something. Of course, many people are smart enough to know that these tutorials (or cloning the postgres repo) is just the first step on a longer learning journey, and in that case it's totally fine, but there are also many people who start thinking they are experts ready to work on research or production models after going through such things, not being aware of the many things they don't know [0]

[0] [https://upload.wikimedia.org/wikipedia/commons/thumb/4/46/Du...](https://upload.wikimedia.org/wikipedia/commons/thumb/4/46/Dunning%E2%80%93Kruger_Effect_01.svg/1024px- Dunning%E2%80%93Kruger_Effect_01.svg.png)

<comment><USER5> It sounds like what you're complaining about is some people who put papers on arxiv and deceivingly claim to be experts. And kind of implying that these blog posts are to blame, so the blog posts should be retracted due to those dishonest academic people? You're making a very confusing point.

<comment><USER6> Not confusing. He's saying that watching Judge Judy for a week doesn't make you a lawyer. And when hiring, be careful because lots of people who claim to be experts are far from it.

<comment><USER5> That's valid, but he's implying that Judge Judy should be taken off air because some people watch it and then pretend to be lawyers. Squelching information seems like a terrible way to eliminate a few impostors downstream.

<comment><USER3> Yes, but that's a bad example because pretending to be a lawyer is hard. A better example would be gurus spreading nutrition recommendations that are not wrong per-se, but extremely simplified. Nutrition is a complex topic and individual differences make it hard to generalize. Let's say the information are so simplified that they are likely to hurt people who blindly follow them without doing further research. So, should this information be taken off air or not? I would say yes, and perhaps you would say no. In either case, I don't think the answer is quite as clear-cut.

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> I'm doing deep learning without a data science background.

Some of my current results are:

* [https://vo.codes](https://vo.codes)

* [https://trumped.com](https://trumped.com)

The voices need better data curation and longer training, but some speakers such as David Attenborough are quite good.

I've also built a real time streaming voice conversion system. I want to generalize it better so that it can be an actual product. I think it could be a killer app for Discord. Imagine talking to your friends as Ben Stein or Ninja.

I've been watching TTS and VC evolve over the last few years, and it's incredible pace at which things are coming along. There are now singing neural networks that sound better than Vocaloid. If you follow researchers on Github (seriously, their social features are a killer app!), you'll see model after model get uploaded - complete with citations, results. It's super exciting, and it's the future I hoped research would become.

If you're diving into this, I would recommend using PyTorch, _not_ TensorFlow. PyTorch is much easier to use and has better library/language support. TorchScript / JIT is really fantastic, too. I even mean this if just you're poking around with someone else's model - find a PyTorch alternative if you can. It's much easier to wrap your head around. TensorFlow is just too obtuse for no good reason.

<comment><USER8> Hi Brandon, nice work! Some questions to learn more if you don't mind - <USER2> you using Tacoton2 for the voice generation? If it's Tacotron2, are you using a base model before you train up new speakers, or is each speaker model trained from scratch? How long do you run the training for normally (for both cases), and what hardware are you running?

You mentioned elsewhere you're renting the V100s, what services have you used, and would you recommend them?

By the way, your Trumped.com is throwing some errors in the console so the site isn't working for me.

Keep up the good work!

<comment><USER7> > are you using Tacoton2 for the voice generation?

Nope. glowtts. Tacotron2 has higher fidelity (it's a denser network), but it's really slow and expensive to run.

> are you using a base model before you train up new speakers

Absolutely! Transfer learning is essential to training on sparse or limited data, and it's incredibly effective.

> How long do you run the training for normally

> and what hardware are you running?

I think I explained this in my other answer? But if not, it's typically awhile. The best guiding light though is to frequently listen to the inference results. Are they improving? Watch the Tensorboard graphs to see what loss and attention look like and make sure you're actually learning.

> By the way, your Trumped.com is throwing some errors in the console so the > site isn't working for me.

Oh no. I realize there are some bugs I left for iPhone (simply because I don't have one), and I really need to get those fixed. I'm not sure if this is your case or not. Perhaps the pods with the Trump model are also experiencing duress -- I'll need to investigate that too. I have yet to hook up monitoring (yikes).

</comment></comment></comment><comment><USER9> Tangentially related (and also using the ubiquitous MNIST dataset), Sebastian Lague started a brilliant, but unfortunately unfinished video series on building neural networks from scratch.

This video was an absolute eye-opener me [1] on what classification is, how it works and why a non-linear activation function is required. I probably learned more in the 5 minutes watching this than doing multiple Coursera courses on the subject.

[1] [https://www.youtube.com/watch?v=bVQUSndDllU](https://www.youtube.com/watch?v=bVQUSndDllU)

<comment><USER10> One "ah ha" about that required non-linear function I had was the fact that if you were just passing numbers through a series of linear functions they could by definition just be combined into one equation.

</comment></comment><comment><USER11> The whole AI/ML stuff has become so hyped up that its probably time to find another topic of interest in software engineering for me. Its a weird melange nowadays where frameworks and "academic credentials" are fused together by major tech companies and leaves me - who has deployed a dozen of classical ML models into production that are still running after couple of years - wondering what this is all about.

Overall, working with people with different backgrounds, a ML-related PhD is usually not correlated nor anti-correlated to these people having a good understanding of the relationship between models and and their applications.

I wish we could leave the framework and name-dropping behind and talk more about what it takes to evaluate predictions, how to cope with biases, etc.

<comment><USER12> Instead of saying anti-correlated is better to say "inversely correlated" (or if you mean lack of correlation then "uncorrelated")

<comment><USER13> You mean "negatively correlated"?

<comment><USER12> Yes, "negatively correlated" is more widely used (the iron law of nitpicking strikes again).

</comment></comment></comment></comment><comment><USER14> It's a pity that most Tensorflow tutorials out there seem to deal with images. We tried to use it for real-time data classification (data -> [yes | no]). Every tuturial out there seems to assume you're using Python (which is probably not an invalid assumption). Here's my 2c when trying to use Tensorflow with C++:

a) Loading SavedModels is a pain. I has to trawl the Tensorflow repo and Python wrappers to see how it worked.

b) It's incredibly slow. It added ~250ms to our latency. We had to drop it.

c) It has a C++ framework that doesn't work out-of-the-box, you have to use the C lib that wraps an old version of the C++ framework (confused? me too).

d) It's locked to C++03.

Tensorflow-Lite looked to fit the bill for us, but our model weren't convertible to it. We no longer use Tensorflow.

<comment><USER15> I don't understand why you are getting downvoted. TensorFlow 1.x barely worked and people stuck with it because the alternatives were worse. I moved to PyTorch as soon as I could because it is better than TensorFlow 1.x, TensorFlow 2.x, or Keras w/TensorFlow backend.

TensorFlow is designed by committee and is more of a brand now than a machine learning framework. Did you know there is TensorFlow Quantum?

<comment><USER16> There were alternatives. IMO, Microsoft Cntk being one which was better than tensorflow. Tensorflow 'won' because of herd mentality and the perception that everything google does is cool.

</comment></comment></comment><comment><USER17> This is a nice introduction, even though as most of the tutorials on ML it goes from 0 to 100 in 2 lessons.

A couple years ago I started studying ML, and I have a design background, so I needed to digest all the math and concepts slowly in order to understand them properly.

Now I think I understand most of the fundamental concepts, and I've been using it quite a lot for creative applications and teaching, and I have to say the best resource I've found for beginners, by far, is "Make Your Own Neural Network" by Tariq Rashid.

It starts really from the beginning and it takes you through all the steps of building a NN from zero, with no previous knowledge. really good.

</comment><comment><USER18> Since everyone is talking about hype in ML, I wish there was some hype for good ole' conversional scientific computing. Yes, it's not so sexy, you have to build your own model yourself, and then the hard work is in finding and verifying a suitable numerical method and finally devising a solid implementation. It requires a vast number of different skills, anything from pure math to low level programming and it is definitely not trivial work, but it does not seem like it pays that well.

<comment><USER19> So excited to start my Masters in Numerical/Scientific Computing for these reasons!

<comment><USER20> What are some good universities for something like this?

</comment></comment></comment><comment><USER21> I am constantly puzzled by people saying that AI is overhyped and fresh grads won't have enough jobs for them. Almost every real life industry: retail, logistics, construction, farming, heavy industries, mining, medicine have just recently started to try AI. The amount of manual and suboptimal tasks that have to be automated and optimized is enormous. I am pretty sure there is more the enough work for applied DSs with domain knowledge in mentioned industries.

<comment><USER22> Agreed, I feel we are just barely scratching the surface. I'm fairly involved in the AI/ML world (research at FAANG) and the amount of hype definitely scares me, but ML/AI isn't going anywhere soon imo.

</comment></comment><comment><USER23> This is very well done, hitting on some pain points and explaining how to work around them.

I have devoted close to 100% of my paid working time on deep learning for the last six years (most recently managing a deep learning team) and not only has the technology advanced rapidly but the online learning resources have kept pace.

A personal issue: after seven years of not updating my Java AI book, I am taking advantage of free time at home to do a major update. New material on deep learning was the most difficult change because there are so many great resources, and there is only so much you can do in one long chapter. I ended up deciding to do just two DL4J examples and then spending most of the chapter just on advice.

The field of deep learning is getting saturated. Recently I did a free mentoring session with someone with a very good educational background (PhD from MIT) and we were talking about needing specializations and specific skills, just using things like DL, cloud dev ops, etc. as necessary tools, but not always enough to base a career on.

Definitely it can help peoples' careers working through great online DL material, but great careers are usually made by having good expertise in two or three areas. Learn DL but combine that with other skills and domain knowledge.

</comment><comment><USER24> I attended a conference talk of a FB AI-engineer talking about her paper with backprop equations so obviously wrong my eyes hurt, and incorrect definitions of objects. It did not stop her from participating (btw. this is always unclear -- who did what) in state-of-the art research in object detection.

PhD is overrated in the deep learning context. It is more about forging the intellectual resilience and ability to pursue ideas for months/years than learning useful things/tricks/theorems.

<comment><USER25> > It is more about forging the intellectual resilience and ability to pursue > ideas for months/years ...

Isn't this what a PhD generically signals?

</comment></comment><comment><USER26> Twenty five years ago, this would have been "LINUX, UNIX, and serving, without a PhD" and Matt Welsh's Linux Installation And Getting Started was the intro ([https://www.mdw.la/papers/linux-getting- started.pdf](https://www.mdw.la/papers/linux-getting-started.pdf)). I was one of many who adopted Linux early, using this book (later I read the BSD Unix Design and Implementation, which I would describe as senior undergrad/junior grad student material).

Having those sorts of resources to introduce junior folks to advanced concepts are really great to me- my experience is that I learn a lot more by reading a good tutorial than a theory book, up until I need to do advanced work (this is particular to my style of learning; I can read code that implements math, but struggle to parse math symbology).

</comment><comment><USER27> Mandatory plug: do consider using PyTorch instead. It's far easier to pick up and work with. Easy things are easy, hard things are possible.

<comment><USER28> Does it train just as fast on gpus?

<comment><USER29> In my experience, not quite as fast for fully-tuned code, but the difference is small - and given the same project deadline, the PyTorch version will probably be faster.

</comment></comment></comment><comment><USER30> The video version [1] is also pretty awesome, though its code itself is a bit outdated now. Explains a lot of very practical issues that you might not find in most academic textbooks, but you encounter every day in practice.

[1] [https://www.youtube.com/watch?v=vq2nnJ4g6N0](https://www.youtube.com/watch?v=vq2nnJ4g6N0)

</comment><comment><USER31> Have a simple rant here. All these BIG $ companies every now and then come out with statements and what not, that doing AI ML is very easy and every one including their cats should do AI, ML courses and training(preferably on their platform). Once that is done the job market is yours. Reality is far from this.

\- Today AI|ML does not have the capability marketed by these big companies. Incidentally marketing is targeted at governments, big non tech companies and gullible undergraduates.

\- Undergrads many a times take these training courses wherein they acquire the skill set to call these APIs and flood the job market wherein a data-entry or data-analyst job is tagged as AI|ML job.

\- High paying jobs in AI|ML still require a Masters or PhD or a mathematical background.

In conclusion, the current hype around AI|ML is misguiding gullible undergrads and governments(I dont mind the government being cheated THOUGH).

<comment><USER32> Another side of this that I've seen is people with PhDs in applied mathematics on a research team where a PhD is a hard requirement, spending all of their time passing dataframes around and convincing the sales team that the bar charts in the product are in fact correct.

<comment><USER33> Have we been working on the same team? :P The worst part (or best part of the joke) is that with a mathematical PhD you're very ill equiped for a discussion with your typical sales rep.

</comment></comment></comment><comment><USER34> The title is obviously clickbait-y, but it's fine: they're trying to sell a product (Google Colab).

IMO if you're interested in AI research or ML engineering, you already know that — in order to avoid getting people killed - you have to understand how it works under-the-hood. You're doing yourself, your employer and your fellow humans a favour.

Just keep up the good work, and ignore the bullshit. If an AI winter comes, you'll be well prepared to migrate to another engineering role.

</comment><comment><USER35> I wonder if Google is using this resource to train their own staff without PhD and after that allow them to work as ML engineers? That would provide credibility to such program - instead it is more aimed to sell more ML computing power to the masses (who won't really understand how to use it to get meaningful results).

<comment><USER36> not bloody likely...

</comment></comment><comment><USER37> To me it seems the real goal of hyping these things so heavily is to increase their cloud revenues.

</comment><comment><USER38> As usual with tools, it's the use instead of the instrument. Domain knowledge still has advantages against generalism, in applied and critical fields. I mean, you can't do medicine or materials yet with AI/ML without understanding the domain?

</comment><comment><USER39> This is something I did with fast.ai - [https://deeplearningmantra.com/](https://deeplearningmantra.com/)

</comment><comment><USER40> Is this tutorial aiming at entry level? Those graphs are quite difficult. I guess it will take a lot of time to do homework on some fundamental curriculums.

</comment><comment><USER41> It is totally possible and easy to use tensorflow/torch if you havn't skipped linear algebra classes. Ph.D is needed if you are going for a job where you design a sophisticated model (not just adding layers, but experimenting with activation, attention etc).

<comment><USER32> A PhD isn't even a requirement for doing the more advanced stuff. Obviously you need a lot of math and ML specific knowledge but there's no reason why you can't have that knowledge with an undergraduate math degree (for example). Spending 3-6 years doing research in a very narrow and possibly unrelated branch of mathematics will give you a PhD, but the linear algebra and multivariable calculus that you actually need for the ML stuff are covered in a bunch of undergrad/masters courses in mathematics, computer science, engineering, physics etc.

<comment><USER42> I second this. I have worked with a bunch of undergrads (I am pursuing a masters degree in CS) and they had a thorough grasp of the math and could really contribute to the research agenda of the group. When I did my undergrad (in 2011-15), I ended up taking a lot of electronics/hardware courses. Turns out undergrads these days just swap them with math/machine learning courses. Good for them.

</comment></comment></comment><comment><USER43> Does anyone know what is this site generated with?</comment>


Ask HN: Why you use python most? - <AUTHOR> <comment><USER2> Python used to be my "main" language, but that was only due to familiarity. For most IO intensive tasks (especially networking), Python is not the best choice [0]. Recently I've been using nodejs with promises and async/await for heavy IO. I really like the productivity gains of Javascript's functional programming style, and the event loop lends itself well to network/IO intensive tasks. (I would prefer to use golang for this, but haven't gotten around to learning it yet)

Now I use python when it's the best tool for the job, where the "job" is a part of a project. My projects usually involve a combination of three languages: python, javascript (node and/or frontend), and bash scripting. The bulk of the code is node, but for scripts I use a combination of bash and python.

I use bash scripts for anything that involves piping data between processes, simple file I/O, backup scripts, etc. I prototype almost every script in bash. When it gets too complicated, or requires parsing a lot of data, I switch to Python (sometimes I even inline the Python in bash with heredocs).

In my experience, bash is usually the best tool for the job if the "job" involves invoking lots of shell commands and piping their outputs somewhere. Using Python to call shell commands is of course possible, but I find it to be much less readable than bash for that specific use case.

Nothing beats Python for parsing and manipulating data, which is why it can be so powerful when combined with bash. Here [1] is an example bash script I wrote in ~30 minutes that uses inline Python. It deploys schema from a JSON file to parse-server. Bash handles the raw IO of reading files, network requests, etc, but inline Python does the simple manipulation of the JSON files.

[0] I haven't worked much with python3 or async/await... but that looks promising (hah) for increased throughput. I've even seen some benchmarks inline with node.

[1] [https://gist.github.com/milesrichardson/b30acef8827c53aae088...](https://gist.github.com/milesrichardson/b30acef8827c53aae0885186cc6078b3)

</comment><comment><USER3> The _biggest_ reason I use Python? Honestly, because it's great for interview- type problems, comfortable for writing on a whiteboard, and suits one-man projects where I don't find myself wishing for static typing or interfaces as a way to coordinate code.

P.S.: I'm sure you _can_ make big multi-team projects in Python with the right discipline and tooling, but that's just not the spot it occupies in my language-toolkit.

<comment><USER4> And when you do find yourself wishing for static typing, there is MyPy. It's not perfect but it's much better than nothing.

</comment></comment><comment><USER5> Python used to be my go to language. For me it was clear, easy to read, had a lot of great packages. Trying ideas out in the REPL was handy. (I've since mostly abandoned it for golang, and wonder how any of my Python ever worked considering how many silly mistakes the compiler catches).

<comment><USER6> Yeah, both Golang and Swift are direct descendants of Python, and it's sort of hard to justify a niche for Python now that Google and Apple have performant in-house versions.

I have to wonder if Go and Swift would've just been Python if Guido had put more attention on the Python VM (and been willing to make some compromises to accommodate improvements).

Now Python is stuck playing catch up. Discussion around removing the GIL is finally beginning to be taken seriously instead of considered hypothetical, optional type annotations are being added, etc., but it may be too little too late.

<comment><USER7> if python is the language of ML and ML is the future, i dont think the snakes are going extinct.

<comment><USER6> 1\. There's no reason that Python has to be "the language of ML", especially not with so many new Python-inspired languages, like Go. ML is a performance- sensitive sector. Python's strong/established C FFI is its selling point here, since Python gives such nice icing on the cake buried in a C module like NumPy, but this will only matter as long as the newer languages are not fast enough for the performance-sensitive bits. Once native Go is fast enough, it seems logical that everything will move to Go instead of C+Python.

2\. "ML is the future" is vague and meaningless. ML is certainly not the future for many things, and it's certainly being misapplied on many other things in the cargo-cult belief that ML must be used everywhere.

<comment><USER8> Actually there are several reasons that Python has to be the language of Machine Learning and Data Science. It begins with two decades of being promoted as the most accessible language for scientists to get the job done. Then the widespread use of Python led to an explosion of libraries creating a rich ecosystem.

Nowadays there are sophisticated tools like Orange3 and Jupyter (not to mention Beaker Notebook) that make it easy to share code within teams of scientists. None of this is going away and even R users are discovering that it is easier to use R for the stats part of the problem and integrate with Python for everything else. R used to have the best presentation graphics but now Python has caught up with improved Matplotlib and Bokeh. Also some people prefer to use d3.js in the browser moving from just presenting their data to enable exploration of it.

In the ML/DS area, CPUs will never be fast enought with Go. You need to exploit the GPU which Python has good support for, or move to distributed computing which Python also does well.

No other language will dislodge Python. Instead we are making integration more prominent and using multiple languages working together. Beaker Notebook is particularly good at this and is driving Jupyter in that direction too.

</comment></comment></comment></comment></comment><comment><USER9> Jupyter Notebooks + Theano + Lasagne + Matplotlib + [the pip package ecosystem]. For my weird little niche (Machine Learning / Computational Intelligence), it is the perfect tool. It's also the perfect language for basic management scripts - a handful of lines of Python made setting up a shared machine learning server with Slurm/Jupyter- Hub/IPFS/Theano+Lasagne+Tensorflow (the last of which I've not personally used) easy. I work with a tmux session with scripts running training nets, and a Jupyter notebook open to evaluate and explore parameters and behavior at save points each epoch. Python/Theano/Lasagne allow for concise easily readable code that documents and describes the concept at hand such that my code cells wind up containing few, if any, comments - instead writing anything that needs to be written in markdown with any inline graphic needed in cells between code cells. For real-time work like mine, scripts are likely to be run once (or rather several times, changing meta-parameters along the way), and most code written will be notebook cells either soon to be deleted or as part of a record that should ideally be in clear readable language.

</comment><comment><USER10> Python is my go-to language and I write big batch jobs in it and profile them and use PyPy. I do it because it's so clean and concise.

Recent C++ with lambda and auto is much improved but still not nearly as productive as Python.

Likewise lambda has really made Java better, but lack of type inference really makes it clunky to write.

I really want a statically typed Python. But because I don't have it, I choose Python for the syntax over c++/Java even for performance critical code.

<comment><USER4> > I really want a statically typed Python.

You need to give type annotations a shot! [https://www.python.org/dev/peps/pep-0484/#abstract](https://www.python.org/dev/peps/pep-0484/#abstract)

        pip install mypy
        mypy script.py

<comment><USER10> (I am the author of [https://pypi.python.org/pypi/obiwan](https://pypi.python.org/pypi/obiwan) ;) )

</comment></comment></comment><comment><USER11> I use it most for numpy, pandas, scikit-learn, and a bit of graphing. I use it because there is so much data processing, formatting, piping, munging, and so forth involved in analytical work in data analysis, and I think python brings in the ideal combination of general programming and stats/numerical libraries.

I don't use it for web programming. If I had my choice, I'd still use an integrated system like rails or Django, and bring in javascript sparingly, but unfortunately, that's not my choice, so I end up using a lot of Javascript.

</comment><comment><USER12> Automation. I'm building a dev environment that does various bits and pieces transparently - DNS, databases, reverse proxying, backups/restores, secrets, stacking/merging/relating containers etc.

I kinda just stumbled into it as the logical way to move forward once the shell scripts I started out with became too complex, but I've fallen in love with it. I'm sure it will be my go-to language for a lot of coding tasks in the future.

</comment><comment><USER13> I work on a GUI application written with PyQt. It's been very productive for my small team. The downside is that Python's permissiveness has bitten us over and over again. We've become focused on managing mutability --- it's the central problem we have to solve.

</comment><comment><USER14> I have been using Python to interact with APIs, download data and files, then manipulate the data and put it into a database. Then I set up my own APIs (Python cgi-bin scripts) to access the data. I also use the PIL library for image thumbnailing.

Years ago I would have done all of this with Perl 5. I started using Python because so many people were talking about it. What really got me using it was that a programmer told me he thought Python pip libraries were as good as Perl CPAN libraries. He was right, although in some instances one particular library can be superior to the other's equivalent.

</comment><comment><USER15> It was easy to learn, has great libraries to do virtually anything and I haven't yet found a need to require strict static typing (I haven't explored MyPy yet though). More importantly, I know the internals of the language pretty well and I invested a lot of time in learning them. So I'm kind of reluctant to switch to another language, although I want to explore Go.

</comment><comment><USER16> I'm in ops and python is pre-installed on debian and fedora so besides writing stuff in bash, python is my next best thing without installing other languages. I would like to do more advanced things in python but I don't have any big projects / automation in my head yet.

<comment><USER17> ops is a great place for Python. How about backups? Backup monitoring?

Don't roll your own monitoring solution, but tie Python scripts into Nagios, etc..

<comment><USER16> Yea i'm tying python into that stuff where I can. But i'm hoping to do something not ops related with Python eventually. Maybe implement my own monitoring dashboard with django and some other front end framework.

<comment><USER17> Write a React app with Flask. You can write something in <1000 LOC that actually does something.

</comment></comment></comment></comment><comment><USER18> Python is great for prototyping. I work a lot with spatio-temporal textual datasets (think geo-tagged tweets), and most of the time, I need to verify my idea actually works before working on performance improvement using C++.

</comment><comment><USER19> To attract potential job offers.

I don't care about the language nor its ecosystem the slightest. If I need a dynamic language I stick to Perl or Chicken Scheme.

</comment><comment><USER20> Django. A great web framework with great third party libraries such as Django Rest Framework, which will help you to quickly get your app up and running.

</comment><comment><USER21> Mainly I got into it because I hated debugging code for missing semicolons and curly braces. I'm a neat freak, and Python is, too.</comment>


Activate God Mode in Windows 10 - <AUTHOR> http://www.lifehacker.co.in/technology/Activate-God-Mode-in-Windows-10-for-Quick-Access-to-Windows-Commands/articleshow/48349948.cms

<comment><USER2> This has worked on Windows since at least Vista. You can use any folder name with the suffix .{ed7ba470-8e54-465e-825c-99712043e01c}. Explanation of why it works here: [https://en.wikipedia.org/wiki/Windows_Master_Control_Panel_s...](https://en.wikipedia.org/wiki/Windows_Master_Control_Panel_shortcut)

It's just another view of the control panel. It's maybe a little more useful in Windows 10 (and also Windows 8.x) since there are two settings apps, although I'd suggest that the best way get things done in Windows is to use search - just hit the Windows key (or click the Windows logo) and type "Bluetooth" or "printer" or "uac" or "update" or whatever. And of course, with Windows 10 you can boss Cortana around, too.

</comment><comment><USER3> Interestingly, this also works in Windows 7, so it must be a pretty old feature. But it doesn't give you any god-like powers, just a lot of functions from various control panels assembled in one place. And any function that requires admin privileges (like "create an account") brings up the usual prompt for the admin password.

<comment><USER4> Still really convenient, though.

</comment></comment><comment><USER5> TL;DR it's the same "godmode" hack that works in Win7. So if you just kept the modified shortcut in a random folder somewhere, it should still work.

</comment><comment><USER6> In Windows 10 Microsoft is the god.

<comment><AUTHOR> It is true 100 %</comment></comment>


Artist peels his painting of an orange with brushstrokes - <AUTHOR> http://gizmodo.com/#!5783805/an-artist-peels-his-painting-of-an-orange-with-brushstrokes

<comment><USER2> The link doesn't work when using French language as default (redirects to <http://www.gizmodo.fr/>). It is possible to access it after manually removing the hashbang, though, but I don't like hashbang URLs. The video is also viewable at <http://www.youtube.com/watch?v=kDnlSn_CNNM>. Nice video, though.</comment>


Hackerrank.com sends passwords in the clear - <AUTHOR> https://gist.github.com/3020091/ So I saw the recently launcher hackerrank.com and noticed on sign up that the page wasn't HTTPS.  I was a little bit worried and checked a packet dump.
Sure enough, passwords are sent plaintext to their servers (I hope not stored plaintext, although that is harder to confirm).
If you want to try them out, use a throwaway password! <comment><USER2> And you already told them it's a bad idea. So what are you trying to achieve by posting this on HN? Public humiliation?

<comment><AUTHOR> I'm trying to prevent other people from using passwords that matter because there is no warning at all.

</comment></comment><comment><USER3> We'll add https in a day. Sorry.

<comment><AUTHOR> Hey, it's no big deal. I fully understand product launches :-)

I just wanted a warning out there for people so they don't use real passwords.

I am still of the opinion that you should put a red warning on the web site, or disable signups/logins.

<comment><USER4> sorry dude but what? You do know more than half of the internet doesn't use https for login? Hackernews (as pointed out above) doesn't. Are you seriously suggesting a company put "we're not secure!" on their login page because they don't use https?

<comment><AUTHOR> Yes. Bad practices are bad practices. I don't think they should be maintained.

Just a simple warning to users that other people on the Internet could see their password might help in identity theft.

People would at least be cognizant that others can see their password.

</comment></comment></comment></comment><comment><USER5> If you don't have time to do it right yourself, why not have someone else do it for you. For ex: <https://www.dailycred.com>

This is my startup. Getting SSL working is a pain. Building proper account UI is a pain. We think startups should stop burning time reinventing the wheel when they're trying to launch.

</comment><comment><USER6> Every step you take to improve security has costs associated with it. At some point, you just have to say "this is secure enough for all reasonable purposes," because to do otherwise is to risk getting bogged down.

Hashing + salting passwords in storage and using HTTPS is better than a shocking number of large-scale sites. I fail to see a problem.

<comment><AUTHOR> The problem is that HTTPS isn't in place today.

And people are signing up with passwords being sent plaintext.

I just wanted a warning out there---if you want to use the site today, please don't use a password that matters to you.

</comment></comment><comment><USER7> Seems a bit unreasonable, the site is not even available yet. Looks like they protect vs CSRF and salt + hash passwords in the DB and have HTTPS on the way. Why keep nagging? Client side hashing in login pages is very rare, banks like wells fargo dont even do it.

<comment><AUTHOR> The problem is that people are signing up and may be using passwords which they do use on a bank website etc.

Man-in-the-middle ARP attacks or even just sniffing the wire will reveal your password to attached machines on the path from you to HackerRank.com.

<comment><USER7> Right away they said they are adding HTTPS yet you continue pounding and then broadcast it to the world in HN, seems unreasonable.

"just sniffing the wire" will only work if your connected to the same dumb hub, modern switches don't broadcast packets on the subnet. ARP poisoning could force their browser to forward the traffic to you and you can forward it on to the gateway. Even that would require you to be on the same subnet. Once they have HTTPS they are protecting you just as much as any other service. If they refused to support HTTPS I would see this being a problem - but they are not.

<comment><USER8> Wi-Fi is common enough that Apple has been dropping Ethernet adapters from products.

Sounds like the perfect 'dumb hub' to me!

</comment></comment></comment></comment><comment><USER9> This appears to be a good response.

On the other hand, my local government had a website that let you book and pay for driving tests online, and, presumably due to a configuration error, it was taking credit card numbers over plain HTTP.

I contacted them to let them know that they were accepting credit card numbers over an unencrypted connection, and they just sent me an email saying "Thanks for your inquiry. I have been assured by our IT team that the site is secure." I sent a few more emails trying to prove to them that it wasn't secure, but then they stopped replying.

Eventually it got fixed, but I've just checked again and it's now insecure -- again.

<comment><AUTHOR> Yes i know.

I think the excuses like "many sites dont do this" or "its up to the users to secure passwords" are just that---excuses for poor security practices.

<comment><USER10> Hashing on the client-side wouldn't make a difference if a MITM already has control of the SSL connection. They could just spoof the login form, swap out HackerRank's hashing code, and take your password anyway.

If you don't trust the connection don't trust any part of it. Especially not the code your browser is executing.

</comment></comment></comment><comment><USER11> The problem is that it is a site for Hackers, so they have to understand their target market is going to be way more critical than the average user. Welcome to developer marketing

</comment><comment><USER12> I beat the candies challenge with 601 candies and it's still telling me to beat it with more!

How do we input a program to beat the challenges?

<comment><USER3> challenge <any-integer>

<comment><USER12> What I mean is I can beat it with any integer (except those divisible by [REDACTED]), as long as I have the patience and typing accuracy to follow my algorithm for that many turns. Beating it on 601 candies took 2 tries purely due to a mistype on the 70th or 80th iteration.

But the site must let us type our solution algorithm in to a program of some sort and let the computer handle the grunt work, right?

<comment><USER3> There are 3 guys who've written an automated script. Can you figure? ;)

</comment></comment></comment></comment><comment><USER13> Even more fun are the sites which don't let you change your password (!!!).

</comment><comment><USER14> HN doesn't even force https for login. (Heh, based on the direction of the whole thread, I should clarify) I'm not defending using non-HTTPS logins, just pointing out that lots of sites haven't decided to force HTTPS.

<comment><AUTHOR> It depends on your login process. For signing up to a site, you generally will want/need HTTPS.

For login, what is transported on the wire can be a hash or some form of the password which is no longer plaintext.

Thus, _logins_, may not need HTTPS protection.

It depends on implementation.

As a quick path for HackerRank.com to fix, I proposed HTTPS which is already implemented, rather than modifying web app logic.

<comment><USER15> No, encrypting client-side is still vulnerable to a MITM attack. The MITM simply removes the code to encrypt the password from the website.

</comment></comment></comment><comment><USER16> I don't know how these guys stay afloat and continue to get companies to use them. I don't like any of the challenges on there, and I don't see a need to `centralize` the whole process.

EDIT: Not to mention how poorly this site is set up already. The only "challenge" I see on there right now is to type candies <N> to start playing a game where you and a CPU take turns taking between 1-5 candies (you go first). The person who takes the last piece of candy wins.

So, type candies 5. Then choose to take 5 candies. You win.

<comment><USER3> This is a completely new product. It's not linked to jobs/companies.

<comment><AUTHOR> I agree, the top comment confused me...

It's new and could be pretty nice especially for younger people.</comment></comment></comment>


Hypermiling Causes Road Rage? Lets keep things in perspective... - <AUTHOR> http://www.treehugger.com/files/2008/07/hypermiling-road-rage-fad-fuel-efficiency-gas-mileage-money.php

<comment><USER2> I can tell you that I am pretty tired of Priuses jumping into the HOV lane in front of me and doing 45MPH in a 65 zone.

Congratulations on getting 100mpg or whatever, but the train of cars backed up behind you aren't exactly ready to throw you a party.</comment>


Nintendo's weird and wonderful CIC lockout chip - <AUTHOR> http://hackmii.com/2010/01/the-weird-and-wonderful-cic/ <comment><USER2> In the eighties and nineties in China, Russia and other countries it was possible to buy NES clone that would play pirated NES games. It was called Dendy. Obviously it didn't have the CIC chip.

Typical game cartridge contained multiple games (i.e., for example, Mario, Contra and Ninja Turtles could be on the same cartridge) and was quite inexpensive. I didn't realize for years that in US one game would cost 10 times more than 10 games one cartridge for Dendy. Ah, golden days of piracy.

<comment><USER3> I remember that I think. It may have been a different pirate NES, but I remember spending a few hours playing through multicarts on a yellow NES clone when I spent some time in Russia.

</comment></comment><comment><USER4> The Japanese Famicom didn't have a CIC (also known as "10NES") chip, and consequently there were tons of low-quality games cheaply available for it. When Nintendo came to America, they were the first videogame console to arrive after the Great Atari Crash of 1984 - most retailers wouldn't even look at a videogame, let alone sell one (hence the VCR-styling and "Entertainment System"). The CIC was Nintendo's effort to enforce quality control on their console (and heaping piles of money never hurt, either).

Naturally, this wasn't a popular move. Many things were tried to force Nintendo into making their console open, from the legislative (anti-trust suits) to the technical (in early NES revisions, a high-voltage spike to the CIC chip at startup would disable it until the console was reset... Nintendo stopped that soon enough). Eventually Atari, embittered that Nintendo had succeeded where they had failed, decided to devote their engineering resources to reverse-engineering the CIC chip. After many months of work, they finally presented their own equivalent to the CIC, which they called the Rabbit, and used it in a few games including a version of Tetris.

Nintendo was highly skeptical that Atari could have decoded their CIC chip, and sued them[1]. It came out that despite their best efforts, Atari had failed to reverse-engineer the CIC, and quietly requested a copy of the CIC program from the Copyright office, messed it around a bit and called it Rabbit. Things did Not Go Well for Atari after that. [2]

Fast forward to 2006, where the NES homebrew community (who so far had been making their NESs homebrew compatible by simply snipping the RESET pin off the CIC chip) decided that since the CIC patents had expired, and since so much reverse-engineering hardware was available to hobbyists, they should have a crack at the Greatest Unsolved Mystery of NES Programming, and began to poke around. If you've got a few hours, I recommend reading the forum thread[3] as an excellent example of clever, creative, knowledgeable people working on an interesting problem.

For the less patient, even twenty years later, the CIC still resisted the best efforts of dedicated researchers... until one of them decided to abandon the CIC, and start poking at a Rabbit chip instead. After poring over a decapped Rabbit and trying to decipher its workings, somebody noticed a few circuit traces with no obvious purpose snaking across the silicon. It turns out the Atari engineers must have wanted to save others the stress and heartache they'd suffered through themselves: when Rabbit is hooked up like CIC, it behaves identically - but if a voltage is applied to a seemingly unused pin, another pin suddenly begins broadcasting the internal state of the chip after every stage of the algorithm. Once they had that bitstream, it was simple to figure out the instructions Rabbit was executing, and from there to put together the entire CIC algorithm.

These days, CIC is a Solved Problem as the original post shows... but it's impressive how long it lasted, considering how short-lived some content protection systems are these days. The only other system I know that's been remarkably long-lived is the original Playstation's - it's well understood by now, but there's still no way for ordinary consumers to burn a disc that will play on an unmodified PS1.

[1] <http://digital-law-online.info/cases/24PQ2D1015.htm>

[2] For more information on the early history of Nintendo up to the late SNES era, see the excellent book "Game Over" by David Sheff: <http://www.amazon.com/dp/0679736220/>

[3] <http://www.nesdev.com/bbs/viewtopic.php?t=1219>

<comment><USER5> very impressive write-up, sir!

i'm not super-proud of it, but i got started in programming by breaking the copy protection of dos games. i ran the game on my color monitor and borland's turbo debugger on the monochrome one, and single-stepped my way through the code until i had figured it out. then i'd use the norton utilities to find a particular code string in the executable on disk, and patch it with my NOP'ed out version. twenty years later and i still have the byte patterns of many 8086 machine language instructions memorized.

this would have just been harmless fun, except i posted my cracks on the BBSes of the day. well, i was young.

<comment><USER6> Wow. Nostalgia so strong I got vertigo. I now humbly submit to HN a picture of myself circa 1989, doing exactly this to the game "Spacewar".

<http://www.jonandkarrie.com/images/prints_028-2.jpg>

I know why you might want to post the fruits of your labors to BBS's. When I tried to boast of my accomplishments to friends of the day, I couldn't get them past "you can have _two_ monitors on the same computer and _they both go_!?"

<comment><USER5> heh! i had that exact same (cracked) game! it didn't hold my interest for very long, though.

i could always tell the difference between the companies that rolled their own weak-ass protection schemes in-house, and the ones who ponied up for the good stuff, like softguard.

the simplest game i ever broke, before i even owned turbo debugger, was the dos version of rampage. all i had to do was NOP out three calls to INT 10h, the bios direct disk interrupt, and it wasn't able to 'see' the corrupted sectors on its copy-protected floppy anymore.

in the mildly difficult range was this flight simulator i cracked for a friend. i found the protection function that i wanted to NOP out and wrote down the sequence of bytes that needed to be changed. but when i searched the executable with norton, those bytes weren't found! further exploration revealed that the programmers had applied some weak XOR-style "encryption" to the copy protection routine i was trying to get rid of. i had to write a little utility in C to break that one.

i tried to break several of the sierra games, like king's quest. but they were using their own in-house interpreter, so trying to crack it with an 8086 assembler was approaching the problem at the wrong level of abstraction. i couldn't do it.

i never did manage to get any of the softguard games, either. they were doing really wicked things like using the INT 1 and INT 3 interrupt vectors as scratch space. they knew perfectly well that people like me needed those interrupts to be intact for our debuggers to work. i tried redirecting the first 100 calls to a different area of memory, but they were really fiendishly thorough. i gave up.

bet you've got some good stories, too. hint hint.

<comment><USER6> Most of my pc exploits involved searching for days to find just the right place to set 1 equal to 1, doing it and declaring victory.

I spent a _huge_ amount of time on my Amiga tweaking disk copy algos to foil those rotten floppies that had tiny holes physically burned or punched into the medium. After you got the damn thing to copy, then you had to hunt through the executable to find out what behavior the program expected from the disk drive when it encountered the hole and put in a jump to skip this.

I remember my 13 year old self exclaiming "they use the drive error _as the key_?! That's just diabolical!"</comment></comment></comment></comment></comment>


Building a Flask Single Page Application – Part 3 - <AUTHOR>

Video: http://youtu.be/C88eNjVUDCM
Part 3 of a 5 Part Series on building a Flask Single Page Application.
Purchase the Real Python at http://realpython.com courses to be part of the next session!
We're building a Tinder for finding pair programmers! :) <comment><AUTHOR> clickable links:

[http://youtu.be/C88eNjVUDCM](http://youtu.be/C88eNjVUDCM)

and

[http://realpython.com](http://realpython.com)</comment>