How ACH works: A developer perspective - <AUTHOR> http://engineering.zenpayroll.com/how-ach-works-a-developer-perspective-part-1/ <comment><USER3> This made me cringe:

> _Your [bank] will set you up with a secure FTP server onto which you'll > upload ACH files._

Automated financial transactions, with lots of money involved. Secure. _FTP server._

Of course I know that FTP is still considered "secure" for some business applications but this is absurd. With all the security (mis)features the banks have put forward with card payments in order to shift liability to account holders, the very core of inter-bank transactions then runs on something as insecure as FTP.

The shoemaker's children have no feet.

<comment><USER4> > This made me cringe

As well it should. But it gets worse. Much, much worse:

[http://blog.rongarret.info/2010/11/personal-banking- nightmar...](http://blog.rongarret.info/2010/11/personal-banking- nightmare.html)

It often amazes me that the U.S. financial system works at all.

<comment><USER5> Maybe. Here's a slightly different perspective. First take the case of ftp'ing/sftp'ing files to a bank.

There are multiple layers of security in sftp - not just the username and password. Source IP is checkec in a stateful way so spoofing is hard even if one somehow manages to get the credentials. Furthermore the files will not be processed unless there is an _out-of-band_ message (typically an email, could be a phone call too ) with certain info also required in very specific format which must match the contents of the file. So there you go. 3 layers of security with one layer being completely out-of-band with the other two is reasonable enough for the risk involved. (There are other measures hard coded e.g. a dollar limit which basically "cap" any fraud but I won't go into those)

Now for the missing (or atleast MIA) transaction you allude to on your blog: Just because _you_ were not told how the money ended up where it did doesn't mean people didn't know. Bankers are steeped in "social engineering" and trained from day one to only give out information required by law and really nothing else. Whether you use Fedwire or Swift (or really any other RTGS = real-time gross settlement network ) there is always a trail. Always. In your case it was probably Fedwire - and the bank I'm guessing was BofA or maybe Chase. Regardless if the sender's account is an individual account (not business) they are still protected by Reg E (as well as FDIC insurance). Reg E as you may know basically puts the onus of returning the funds to the sender's account on the bank. The problem is the timelines or the lack of it. Since banks are generally allowed upto 30 days (I don't know if 30 days is hardcoded ) in law they take their sweet time in tracing. And when they find out - <USER2> almost always do - they deliberately will not share details with you to protect themselves from liability.

The bottom line is ACH/FedWire moves trillions of dollars - hundreds of billions a day - and barring a very small percentage they all work.

<comment><USER4> > Just because you were not told how the money ended up where it did doesn't > mean people didn't know.

That's true, but there was a lot of additional evidence that they really didn't know:

1\. They said they didn't know.

2\. Two weeks passed during which they could not find the money.

3\. The money was only found after the person in whose account it ended up contact me (not the bank!)

And, BTW, I actually found out later how the mistake had been made: the original wire form had been filled out wrong. It was supposed to be a for- benefit-of wire, but the form was filled out as if for an intermediate- institution wire. And the bank employee who entered the data was apparently completely clueless because they entered an account number as an ABA number (or maybe it was the other way around, it was a long time ago). But it was without a doubt a total clusterfuck from beginning to end.

> barring a very small percentage they all work

Yes. As I said, this never ceases to amaze me.

</comment></comment></comment></comment><comment><USER6> If you want to get into the nitty gritty of the format of ACH/NACHA files you can buy the spec here:

[https://www.nacha.org/nacha-estore-rules](https://www.nacha.org/nacha-estore- rules)

But you can find parts of the book online at various places. Many financial institutions support NACHA files, but you'll also find many that violate the spec. I happen to have implemented a C# NACHA implementation recently and it wasn't a lot of fun. And there is a huge difference between ACH receipt and ACH origination.

</comment><comment><USER7> If you don't want to wait for part 2, NPR had a decent podcast about how the current American ACH system works, and why it hasn't moved out of the 1960s:

[http://www.npr.org/blogs/money/2013/10/04/229224964/episode-...](http://www.npr.org/blogs/money/2013/10/04/229224964/episode-489-the- invisible-plumbing-of-our-economy)

<comment><USER8> The main reason it hasn't moved out of the 1960s is it _works pretty damn well_. It's a bit slow, but money goes where it needs to go, and there's a good audit trail to trace in cases of fraud.

<comment><USER9> It's crazy slow. Most ACH moves through a modern stack and then into COBOL that runs nightly batch jobs that then update the stuff in the modern stack and in on its way. Nevertheless, it takes the nightly batch job.

<comment><USER8> Honestly though, they processed 22 billion payments totaling 38.7 _trillion_ dollars in 2013.[0] When you're dealing with numbers like that, there's an argument to made in favor of battle-hardened COBOL batch processing over some real-time rewrite.

[0] [http://www.prweb.com/releases/2013/ACH_Volume/prweb11740323....](http://www.prweb.com/releases/2013/ACH_Volume/prweb11740323.htm)

</comment></comment></comment></comment><comment><USER10> Just curious. How hard is it for a bank to agree to be your "Originating Depository Financial Institution"? We currently use Balanced for ACH payouts, but it sounds like we could actually do this ourselves and significantly speed up the process as well as cut down on fees.

<comment><USER11> It varies depending on your risk profile and how much the bank wants to do your transactions (dollar volume of deposits matter, and fees). If you're coming in as a customer, it may just be a matter of being a commercial customer and paying enough fees. If you're trying to come in as a third party processor, it's more involved. Relationships help there.

You might need significant volume (hundreds or more per day) to make it worth switching from Balanced to something else for outgoing payments. With enough volume (10k+/month), you might get under .10/transaction. Balanced doesn't seem like they're particularly slow about outgoing payments. (from a quick scan of their site) The slowness comes from the ACH network where nothing is ever confirmed settled, it just hasn't come back returned yet.

<comment><USER10> My main issue with Balanced is that since I don't use them to collect payments, it takes forever to get the money moved to them. And it is a manual process.

So every Monday morning I get a file with payments. I have to move that amount over to Balanced. This can take 3 to 5 business days, but almost always takes 5 or even 6. Meaning I cannot pay my customers until the Friday (5 days) or the Monday (8 days) later.

Also, Balanced cannot process ACH for payments from clients. Having to verify an account with micro deposits is simply not workable for our clientele, many of which do not want to take credit cards because of the high fees.

While our volume right now is too low, it is interesting info to have, and something I'll look into further.

Thanks!

<comment><USER12> Hi theflyingkiwi42. I'm a co-founder of Balanced. Happy to answer whatever questions you have about our product and hear how we can make Balanced's ACH products better, but don't want to derail this thread. Would you mind commenting on one, or more, of our ACH related GitHub issues [1]? Or you can email firstname.lastname@example.com to expand on your thoughts here. Cheers!

-jkw

[1] [https://github.com/balanced/balanced- api/search?q=ach&ref=cm...](https://github.com/balanced/balanced- api/search?q=ach&ref=cmdform&type=Issues)

</comment></comment></comment></comment><comment><USER13> So, an entity that's set up with an ODFI can initiate arbitrary debits to arbitrary accounts, with no action by the owner of the debited account. What happens if that privilege is abused? Is it like chargebacks with credit cards? What if the debit is so small the victim doesn't notice?

<comment><USER14> There are all sorts of _policy-based_ resolution methods in the ACH world. But there is nothing technologically restricting arbitrary debits/credits to arbitrary accounts in arbitrary amounts.

One policy meant to deal with this is called "Velocity limits" which is basically a rate-limit of how much the OFDI will underwrite your account for. i.e. You can debit up to $50,000/day and then your OFDI will cut off further ACH transactions until some of your outstanding ones have settled.

Another policy is that (for Check21 checks processed via ACH anyway) the checking account owner has up to 60 days _from the statement date_ to dispute the charge, meaning the ACH transaction can be reversed up to 90 days after it was initiated.

It's straight out of the 60's. And on one hand, it's cool that everybody uses this system and nobody notices how old it is. On the other hand, it's freaky when you realize that every time you swipe a debit card or write a check, you've basically handed over all the credentials needed for that person to debit your account as much as they like (until they're discovered).

Also, remember that automatic deposit implies automatic withdrawal.

ACH systems are by far the craziest API integrations I've ever had to do.

</comment></comment><comment><USER15> I'm an ACH product manager who's worked the user side of ACH for years, and I've never seen a conversation like this. It's amazing. Even though most of it went off on a file-transfer tangent, I've picked up some really interesting tips. Like, I never imagined there would be stuff about ACH on GitHub. BTW, ZenPayroll is an amazing product and I want to work there.

</comment><comment><USER16> Great intro to a near-ubiquitous payment format (in the US - outside the US, it looks like ISO 20022 [1] is taking over).

I'm looking forward to seeing details about payment confirmation/rejection, and possibly stuff about automated wire transfers (so-called "high value payments" in payment-geek parlance).

For those of you who are just coming into the Finance space, I found this site to be very helpful in getting your feet wet [2].

Anyone have other resources they've found useful/informative?

[1] [http://en.wikipedia.org/wiki/ISO_20022](http://en.wikipedia.org/wiki/ISO_20022)

[2] [http://www.accountingcoach.com](http://www.accountingcoach.com)

<comment><USER17> If you want to learn about accounting from fundamentals up, I recommend Frank Wood's Business Accounting 1: [http://www.amazon.co.uk/gp/aw/d/0273712128?pc_redir=13981155...](http://www.amazon.co.uk/gp/aw/d/0273712128?pc_redir=1398115582&robot_redir=1)

For more advanced topics, the 3rd party study guides for the major accounting qualifications are pretty good. Try searching Amazon for 'BPP CIMA'.

Note: these are resources focused on accounting. If you are interested in finance, then you could do worse than starting with Brealey & Myers: [http://www.amazon.co.uk/dp/B00DDML9U2](http://www.amazon.co.uk/dp/B00DDML9U2)

</comment></comment><comment><USER18> ACH libraries have been around for 20 years... Hope you didnt recreate the wheel...

<comment><USER16> Any good ones you'd recommend for major languages?

<comment><USER18> Technically... You cant initiate an ACH transaction. You request it from a member institution, who authorize and initiate it. Many large institutions have their own APIs... But cybersource [http://www.cybersource.com/developers/](http://www.cybersource.com/developers/) has most of what you'd need

<comment><USER16> Interesting - as I understand the process is generally to send an ACH file to a bank's file transfer point once agreements are signed and credentials/keys are shared.

Are there any libraries to create the ACH file itself from basic information like ODFI, etc?

<comment><USER19> There are a few good ones on GitHub: [https://github.com/jm81/ach](https://github.com/jm81/ach) At my old company, we manually had someone use ACH Tools to build out these files, upload to the bank's interface, etc.: [http://www.achtools.com/HomePage.aspx#!/](http://www.achtools.com/HomePage.aspx#!/)

</comment></comment></comment></comment></comment><comment><USER20> Good start. Would love to see what an actual ACH file looks like. Googling returns some specs and the like, but not sure if any contain the exact doc that gets FTP'd to the Fed.

<comment><USER6> If you google for an example "nacha file" instead of an "ach file" you should be able to find what you're looking for. Here is one example of what one looks like:

[http://www.treasurysoftware.com/ach.html](http://www.treasurysoftware.com/ach.html)

</comment></comment><comment><USER21> "moves electronically through the banking system today"

I assume only in United States? I wonder if one can automate SWIFT transfers the way one does ACH, that would be epic.

</comment><comment><USER22> Seems very logical, not much to learn in part 1! Does the ACH still hit the federal reserve if the two accounts are at the same bank?

<comment><USER11> Maybe, depends on the bank. It certainly doesn't have to if the bank is optimally processing the file.

<comment><USER23> there is a lot of discussions and actions being made for intra-day settlements.

<comment><USER6> Yep, here are the current details:

[https://www.nacha.org/news/nacha-leads-industry-toward- ubiqu...](https://www.nacha.org/news/nacha-leads-industry-toward-ubiquitous- same-day-ach-settlement)

<comment><USER23> IMO - this is the only way ACH can remain competitive with Card transactions. Intra-day settlements and returns

<comment><USER6> The whole payment space is getting really interesting in the last couple of years. Banks and retailers trying to work around credit card processing fees as they became larger part of their balance sheets.

</comment></comment></comment></comment></comment></comment><comment><USER24> So they use FTP and files as an RPC mechanism. What happens when you accidentally overwrite files? How does sequencing happen? Do they use timestamps on the files?

It sorta, kinda makes sense and I can't really deny the simplicity of the whole scheme.

<comment><USER11> Generally, there's a confirmation step for transaction counts and file totals.

Also, there's something of a convention to never send the same file name twice.

</comment></comment><comment><USER25> I too was in finance. Had to write ACH processing for loan payments and collection. No fun. However, there are plenty of docs from the Fed. No test systems tho :(. I think I'll go look for some of my old ACH code just for fun.

</comment><comment><USER26> Is there no getting around debiting party A into your own account, then crediting party B from your own account?

Is it not possible to initiate a direct payment from A to B?

</comment><comment><USER27> I hope you guys continue with this series and make the next one more in-depth. I am very curious how everything work, especially the fees.

</comment><comment><USER28> I've integrated with several domestic ACH APIs, and none of them are FTP based. Most are SOAP services.

</comment><comment><USER29> What are the fees?

Random question: is this how Dwolla does its transactions--through ACH?

</comment><comment><USER30> Obligatory "Bitcoin will disrupt this" post.

<comment><USER31> Not in our lifetime. Cryptocurrencies will just be another bolt on and thing we need to integrate. The upshot is that the banks probably won't manage this. The downside is that the banks probably won't manage this.

</comment></comment><comment><USER32> THIS is why Bitcoin. THIS.

</comment><comment><USER33> seems logical

</comment><comment><USER34> "documentation explaining the ACH system is targeted towards bankers, not software developers."

The ACH system itself is designed for the benefit of bankers, not developers, or even depositors.

Bitcoin alleviates the need for the excesses of the described system.

<comment><USER35> Bitcoin doesn't let me withdraw US $ from one US bank account and deposit it in another US bank account in bulk.

Bitcoin lets me transfer bitcoin around - which is fantastic and wonderful - but it doesn't address the problem at hand.

<comment><USER32> You don't understand. Once you're in the Bitcoin ecosystem, you don't necessarily need to backtrack out into US $ anymore. I buy almost everything with Bitcoin, and shop at Bitcoin-friendly stores. Except for grocery stores. I don't know of any grocery stores yet that accept Bitcoin in my area, but they will surely arrive someday.</comment></comment></comment>


Why do people click Google ads? - <AUTHOR> http://blog.cleartrip.com/journal/2008/9/19/why-do-people-click-google-ads.html <comment><USER2> i think the lesson is this: the more tech-oriented your target crowd becomes, the more you need to base your revenue model on sponsorships. the masses will click. the web veteran won't click but will view PPM ads. the tech-savvy will adblock all standard ads.

<comment><USER3> It's an interesting insight into the business models of many Web 2.0 startups- your early beta users will most likely be tech savvy, and thus the least likely to click on your ads. In other words, don't expect to make money from AdSense anytime soon.

<comment><USER2> or seek alternative revenue models.</comment></comment></comment>


AppLP: A Dialogue on Applications of Logic Programming - <AUTHOR> https://arxiv.org/abs/1704.02375 <comment><USER2> A recommended read voicing opinions of renowned practitioners and theoreticians in logic programming.

According to one speaker/panelist, "Prolog is almost unknown in the U.S., outside of academic circles". Is this really the case?

Also of practical interest (to me at least):

> "Warren asks, "In a multi-language, multi-software ecosystem, what are the > roles of logic programming languages?"

I will admit I'm _not_ a fan of using Prolog for general-purpose programming. But Jan Wielemaker (of SWI Prolog fame) argues otherwise, and I think it shows in SWI Prolog's somewhat procedural library collection. For example, SWI Prolog has primitives for explicit multi-threading.

<comment><USER3> I've worked mostly in enterprise software the past 24 years, and current and prior job have been in security companies. This is far from academia, and I've never seen Prolog or other logic-based approaches mentioned, or even LISP-ey or other AI-ish languages / approaches for the most part. There are places, though, where it would have been useful ... example ...

In my immediately prior job I had to deal with some convoluted XML standards called OVAL and XCCDF (from the same general folks that bring you the term "CVE" for computer vulnerabilities) that express "rules" for gathering data from systems and analyzing the data to determine whether the remote system conformed to security best practices. I wish those developing the standards had dealt before with Prolog because the rules and engine could so much more easily have been split into (a) a Prolog-based rule engine; and (b) a low- level telemetry component to gather data from the remote system(s). The XML- based "reproduction" of a subset of Prolog was very limited, not very expressive, and artificially constrained a rule writer to not be able to deal effectively with many conditions you'd naturally want to check for. Also, to extend this standard to look beyond the scope of a single machine will be very hard -- people these days naturally want to evaluate conditions across an entire group of related machines, cloud accounts, containers, VMs, etc. There's no way XCCDF or OVAL will deal effectively with that any time soon, whereas Prolog + telemetry would have addressed that very easily.

We licensed a very good engine for dealing with the XCCDF/OVAL execution (look up "jOVAL"). However I think wistfully back to the first days at that startup and think that with a good computing foundation our lives could have been so much easier. The company was a ... mess ... in other ways and will visit Mr. Joyboy soon [https://www.youtube.com/watch?v=-edybFkBiO4](https://www.youtube.com/watch?v=-edybFkBiO4) .

(Hmmm, Prolog + XCCDF turned up something interesting since I last checked a long time ago: [http://www.academia.edu/20069871/SCAP_based_configuration_an...](http://www.academia.edu/20069871/SCAP_based_configuration_analytics_for_comprehensive_compliance_checking))

<comment><USER2> Interesting. More often than as the primary product or deliverable of a project, I've personally used Prolog for development support, such as for combinatorial test case generation, parsers and DSLs, metadata stores and code generators, and also as a theorem prover of sorts/decision procedure for properties of process models. I always thought Prolog really shines in these roles.

I also often wondered why folks spend big time on "rules engines" (glorified RETE forward-chaining engines) but don't consider Prolog, it being an ISO- standard language with multiple mature implementations.

<comment><USER4> I think the focus on RETE is due to the implicit efficiency advantages of the more restricted execution model. Unification and backtracking aren't always friendly or easily optimized.

</comment></comment></comment></comment><comment><USER5> I totally believe distributed systems should be modeled using declarative circuit metaphors.

</comment><comment><USER6> That was a good read. I especially liked the mention near the end of hybrid login and deep networks.

EDIT: I looked on YouTube for videos of the panel discussions and couldn't find any. Any links to panel videos!</comment>


Openssl: uses only 32 bytes (256 bit) for key generation - <AUTHOR> http://bugs.debian.org/742145 <comment><USER2> An RSA key isn't 4096 random bytes.

OpenSSL doesn't always retrieve all its random bytes from urandom; it seeds its own local PRNG with urandom, and feeds raw random bytes from its own state.

There are no simple answers in public key crypto: the idea that you only need 256 bits of random data can blow your head off if you use something other than RSA. For instance, if you fill only some of the bits of a DSA k value, attackers can extract your private key from a series of public signatures.

Some of the comments on that bug thread are batty, though. The OpenSSL CLI tool doesn't implement its own randomness or key generation; it's a CLI wrapper around the core OpenSSL functions. The CLI might be for "debugging purposes" (but obviously not really, since most instructions for generating SSL certificates involve using that CLI), but the core routines surely aren't.

<comment><USER3> There's also a possibly-not-completly-wrong table on that _other_ TLS library's website ([http://www.gnutls.org/manual/gnutls.html#Selecting- cryptogra...](http://www.gnutls.org/manual/gnutls.html#Selecting- cryptographic-key-sizes)) that takes a stab at estimating the relative security levels of (correctly-used) public key algorithms.

<comment><USER4> [http://www.keylength.com](http://www.keylength.com) also estimates the relative security from various sources.

</comment></comment></comment><comment><USER5> RSA keys aren't just random strings (unlike symmetric keys, which are short random strings, generally 128 or 256 bits now).

You start with two random primes. Generally to generate these what you do is pick a range of numbers in the right size, then test the odd ones with a prime sieve (vs small primes), then do probabilistic primality test on the winners. This doesn't need 4096 bits of randomness.

Once you have the primes, you don't need any more random numbers to generate the key; it's purely deterministic calculation at that point.

<comment><USER6> To add: not only you test for primality, but also for common attacks. Not all primes are equally strong [http://www.uow.edu.au/~jennie/WEB/WEB99/1999_07.pdf](http://www.uow.edu.au/~jennie/WEB/WEB99/1999_07.pdf)

<comment><USER7> For primes of current acceptable size (>= 1024 bits), you might as well drop those extra checks. The elliptic curve method gives you around 2^512 attempts at doing the exact same thing, each one with the same probability of having a smooth p-1 or p+1. The paper you link to reaches essentially the same conclusion.

</comment></comment></comment><comment><USER8> This thread seems to be generating a lot of fuss, so let me weigh in quickly.

This seems to be expected behaviour for generating RSA keys. An RSA key of _length_ 4096b does not provide you with a security level of 4096b. That is, you don't have to straight-up guess the keys, you work at it via factorisation.

The issue is arising from the fact that entopy and key length are both commonly measured in bits. The entropy of an RSA key is much lower than the key length. 256b of entropy seems to be more than reasonable at first glance.

Normally, I would not defend openSSL, but I will do so here, having not even looked at the responsible code (nor has the submitter of this bug), and not even looked at the mailing list link.

Seriously, do not panic, let the cryptographers look at this and decide if it's really an issue. It's probably good that they're getting 256b of entropy from urandom, it's likely that they're seeding a CSPRNG for prime generation & testing.

I would wager some small sum on this being closed by the end of the week, and us looking back on this and shaking our heads at the uninformed knee-jerk reaction of people in here:

"If I ask for a 4096-bit key, I should get one, or an error message. I shouldn't get a 256-bit key that looks like a 4096-bit key." \-- taejo, 15 minutes ago

</comment><comment><USER9> What is really troubling is not the amount of entropy, 256 is fine for everything up to paranoia. What is troubling and reflects the sad status and detachment from reality of the project is this:

>Historically, the OpenSSL command line tools have been intended for debugging only.

<comment><USER10> Apart from genrsa, what exactly have you used the openssl cli for in the past?

My answer would be the exact same as you quoted, the openssl cli tools are quite horrendous to use and you certainly wouldn't use them if you were a CA. If you are a CA or deal with certificates, openssl does provide sweet inspection dumping tools for asn1 and certs.

If you have an application which generates keys or certificates, why would you system exec openssl cli when your language of choice has a Crypto implementation, or glue to OpenSSL.

So I ask openly, run `history | grep openssl` and see. Even running `openssl help` is daunting unless you're familiar with most symmetric block modes.

<comment><USER11> I use the openssl cli every time I generate a CSR or a self-signed certificate. If you Google "generate csr" you'll find lots of sites, including companies that issue SSL certificates, instructing you to use the openssl cli.

</comment></comment></comment><comment><USER12> So what?

The 32 bytes it gets goes back into a similar CSPRNG as the CSPRNG it came out of.

So instead of requesting 4096 bits from a CSPRNG, it requests 256 bits from a CSPRNG and uses that to initialize another CSPRNG which it reads 4096 bits from. Cryptographically speaking it's the exact same thing.

The question to be asking is whether the output from either CSPRNG is predictable.

<comment><USER6> Cryptographically speaking you are completely wrong :) You need both: good algorithm and enough entropy. Ad absurdum if you read 1 one byte and feed it into your CSPRNG, you may get up to 256 random streams of 4096 bits, which is easily enumerable and hardly secure.

<comment><USER2> I don't know what you're trying to say here, but feeding 256 bits to a CSPRNG and then pulling 4096 bits of output from that CSPRNG is not "cryptographically completely wrong".

</comment></comment></comment><comment><USER13> > Historically, the OpenSSL command line tools have been intended for

> debugging only.

That very much surprises me. Can someone explain, elaborate, or source this idea?

<comment><USER10> Have you used the openssl cli tools before?

<comment><USER13> Haha, yes, extensively. Should I not have?

<comment><USER10> I wouldn't say so, you can probably use the tool to your hearts content.

Obviously the tools are useful for debugging purposes, testing tls connections, dumping cert information and asn1.

I've now been made aware that tools wrap the openssl cli instead of using it's programmatic API.

Once I tried to use the ca functions of the tool, found the whole tool entirely too cumbersome and wrote my own using libopenssl.

What do you use it for extensively if I might ask?

<comment><USER14> I personally use it for generating CSRs and private keys.

</comment></comment></comment></comment></comment><comment><USER14> >Florian Weimer dixit:

>Historically, the OpenSSL command line tools have been intended for debugging only.

This seems rather out-of-touch with reality.

</comment><comment><USER15> It seems that this is intentional: [https://groups.google.com/forum/#!topic/mailing.openssl.dev/...](https://groups.google.com/forum/#!topic/mailing.openssl.dev/fpnieXRtEYo)

</comment><comment><USER16> Also - isn't 256 bits (32 bytes) enough entropy? Is the post about a lack of bytes or bits? Can anyone weigh in?

<comment><USER17> If I ask for a 4096-bit key, I should get one, or an error message. I shouldn't get a 256-bit key that looks like a 4096-bit key.

<comment><USER18> You don't know what you are saying. I suppose you speak about RSA-4096, which is about prime numbers. The 256 bits are just used as a seed to find large prime numbers. The 256bit entropy is more than enough to be secure, and 4096 bit prime numbers are never as secure as 256 bit entropy.

<comment><USER19> Even 64 bits would be sufficient as long as the entropy is good. It just needs to be large enough that bruteforcing 2^N isn't feasible. 256 seems like a perfectly safe amount to me.

<comment> ryan-c 64 bits isn't enough to provide brute force protection, you need at least 80 bits, preferably 128 bits, and 256 bits if you're paranoid.

<comment><USER19> Yeah I should have elaborated a little more. I was in a rush. I wasn't trying to endorse 64 bits, but think for a second about what it would take to attack weak entropy here. For every possible entropy state:

        1. Put the random number in that state
        2. Use that randomness to build a RSA key pair
        3. Do an RSA decryption of the captured key exchange using this key
        4. See if the symmetric key we get decrypts into sensible plaintext

The thing to note is that steps 2 and 3 are both expensive -- especially step 2. This isn't nearly as easy as scanning a 64-bit key on a symmetric cipher! So suppose for a second that you've got a huge data center full of machines at your disposal and you can do a billion tests per second -- a full sweep of the 64-bit space would still take 584 years. So you better hope you'd get lucky.

Now would that be enough protection? Of course not -- computers get faster, and a determined enough adversary could build special hardware. (Although, again, it would be orders of magnitude more complicated than just SHA-512 bruteforcing)

The point I was trying to make is that even _if_ they were somehow only using 64 bits of entropy, a practical attack would still be difficult to mount. I'd say that each test would be at least 2^16 times more computation than a typical symmetric cipher check. Therefore I think it would be about as hard as bruteforcing a 80-bit symmetric key.

In other words, 256 is _way_ more than plenty.

</comment></comment></comment></comment></comment></comment><comment><AUTHOR> I don't feel this belongs on the front page of HN. I was pretty worried when I posted it though.

The remaining questions are

1\. Is openssl(1) really only intended for "debugging purposes"?

2\. [http://www.philandstuff.com/2013/03/14/why-does-gpg-need- so-...](http://www.philandstuff.com/2013/03/14/why-does-gpg-need-so-much- entropy.html)

</comment><comment><USER16> I see that several people (even in this thread, which you'd expect better of) still expressing the prevalence of incoherent and inconsistent-with-reality beliefs about /dev/random and /dev/urandom.

This quote in particular struck me as very strange.

> From: Florian Weimer <firstname.lastname@example.com>

> To: Thorsten Glaser <firstname.lastname@example.com>

> Cc: firstname.lastname@example.com

> Subject: Re: openssl: uses only 32 bytes (256 bit) for key generation

> Date: Wed, 19 Mar 2014 21:33:10 +0100

>

> * Thorsten Glaser:

>

> >>Historically, the OpenSSL command line tools have been intended for

> >>debugging only.

> >

> > I disagree,

>

> It's what I was told by the OpenSSL developers.

>

> > Also, what do other tools (that do not invoke openssl(1)

> > unlike most of these I saw, which were shell wrappers

> > around it) do, entropy-wise?

>

> There are different choices. Some use more bits from /dev/urandom,

> some even block on /dev/random. The latter is quite problematic for

> non-interactive key generation during package isntallation.

1\. I would doubt most actually block on /dev/random, but why? /dev/urandom should be Good Enough For Everyone, except in very narrow circumstances. But why isn't someone with a firm grasp of crypto setting a safe default? Why are implementers and consumers of these components making these choices of entropy size ("some use more bits") and underlying sources ("some even block..."). This is ridiculous, I don't trust the average developer to make safe choices in this respect. Why is it okay? This is incoherent. Either OpenSSL has safe defaults, or it doesn't. Leaving it up to consumers to do it correctly is giving up - and leads me to believe it's done unsafely by default.

2\. The inconsistency alluded to by referring to non-interactive key generation and "some even block on" startles me. I don't know why developers think blocking on key generation is generally safer, but if they do, why are other (perhaps the same?) developers sacrificing that safety when generating keys on package installation for the sake of user experience? If /dev/random is safer, it should be used, and if not, it shouldn't be (because of the blocking issue.) The lack of guidance from crypto-savvy developers is deeply concerning.

<comment><USER12> /dev/random is not safer. It just blocks.

Numbers can't be more or less random, either both are unsafe, or both work. Both use exactly the same algorithms (on Linux) to generate numbers, and the numbers come from the same pool.

More in depth analysis: [http://www.2uo.de/myths-about- urandom/](http://www.2uo.de/myths-about-urandom/)

<comment><USER20> This is probably a stupid question, as I don't know much about /dev/random and /dev/urandom (and that article doesn't seems to address it fully), but I'll ask anyway: Doesn't /dev/random block only when it estimates that the entropy pool is empty, aka when it would create unsafe numbers? The article says that the estimate is likely wrong and there is still enough entropy left, but what if it's actually empty?

If this is true, then /dev/urandom might split out unsafe bits (as unlikely it is), which to me sounds like a terrible idea when creating a key or similar very importants secrets.

<comment><USER21> You should read the article to answer your own question in depth. The short version is: on Linux, both /dev/random and /dev/urandom are fed from the same CSPRNG. A CSPRNG only needs to be _seeded_ with a few bits of entropy to generate a whole stream of unpredictable numbers. After the CSPRNG is seeded (at system boot) it no longer matters that you run out of entropy.

Disclaimer: I am not a cryptographer.

<comment><USER20> I have (actually, twice. Once when it was posted sometimes ago on HN, and once now before writing the comment).

What threw me off was that part: "Still, if you insist on never handing out random numbers that are not "backed" by sufficient entropy, you might be nervous here. I'm sleeping sound because I don't care about the entropy estimate."

I was missing the fact that the CSPRNG only needs to be seeded once to be safe, and reseeding is only a "nice thing" that's not really needed. To be fair, the article cover this, I guess I just didn't understand that part really well.

I've got it now, and it actually make sense. Thanks to you, and everyone else that used some time to educate me.

</comment></comment></comment></comment></comment><comment><USER22> Wow! Are we going to need a second set of "debian ssl" blacklists?!

<comment><USER23> Last time was SSH, not SSL. Still crypto, but different beasts.

<comment><USER24> He's referring to the 2008 OpenSSL fiasco:

[https://www.schneier.com/blog/archives/2008/05/random_number...](https://www.schneier.com/blog/archives/2008/05/random_number_b.html)

Edit: oh right, it led to the SSH blacklist. Sorry.</comment></comment></comment>


How could public libraries disrupt themselves? - <AUTHOR> https://www.quora.com/How-could-public-libraries-disrupt-themselves/answer/Alan-Kay-11?share=1 <comment><USER2> If you have ever tried to do research you know how vital libraries are. Once you leave the University environment you lose a lot of access to fundamental results. Industry needs to keep up with the leading edge but is denied access.

Hopefully people will stop locking up research results under paid subscriptions, which would make a public library equal to University libraries. That would make them a place where new knowledge could build on prior work.

At the present time my local library is a book exchange program for read-once detective novels. That said, the ability to read books without fee is important for people who cannot afford to spend a lot of money for entertainment.

<comment><USER3> If you're disappointed in what's on the shelves at your local library, ask the librarian about inter-library loan. Depending on what networks they participate in, they may be able to get nearly any book you ask for, with some delay to request and ship it from another branch or organization.</comment></comment>


Rate My Startup: gotProject.com - <AUTHOR>

Hi everyone, I've spent the last several months developing gotProject.com and I am very curious to hear your opinions about it.
In short, gotProject is a simple way to save and organize online information like urls, images, videos, and text. The web service is based on a drag drop interface that is fully supported on both Computers and the iPad. I tried to bring a new approach by merging search and spatial organization of items.
There are several competitors in this space, so I'd love to know how you think gotProject compares. Thanks!
Here is a link to short 1 minute intro video that introduces the concept http://www.gotproject.com/?concept <comment><USER2> If you've read any of my review posts before I'm going to sound like a broken record, but...

If you're handling anything which could be considered personal data, you need a privacy policy, a TOS page, and a way to export that data. From looking at your site and watching your video, I can't tell whether you intend to sell my list of clips to advertisers, connect it to my name and make it available in a data dump, or any one of 100 other things I have no interest in you doing.

As for your web page, why make such a user-unfriendly thing as a log-in screen your main page? Why not include the video on the main page so I don't have to click 3 or 4 times to start finding out what the heck your service even is.

That said, I think you get it, and that's refreshing. For me, Google Notebook and Evernote are too linear. Zoho Notebook is too complicated. No one else has really sparked my interest.

The two main features I want in an online notebook are the ability to grab anything from anywhere with minimal effort and the ability to organize things freely. These you deliver quite well.

There are a few features I would definitely consider.

1\. Inline notes. If a note is really short, it would nice to be able to just stick it on the page as plain text, instead of having to double click to open it. This would add a lot to the ability to freely organize things because you could use notes as headers, separators, whatever.

2\. Inline pictures. Since there's a little preview of the picture anyway, why not let me clear away the title and URL and just show the picture. Then, if you're doing that, why not let me resize it. Again, this goes to letting me freely drag stuff around however I want it. It would also let me use a picture as a background or header or just about anything else.

3\. Sharing at least read access to a project with someone else. This is pretty big. The reason I started looking into online notebooks in the first place was to plan vacations with my fiancee. I can share login details, and that's fine, but eventually, I'm going to want to share a view of a project with someone I don't trust with my login.

And one little minor thing. Double-clicking an open folder should close it.

I also think it might be cool if you could arrange things inside folders just like you can on the main project page. At the very least, reordering would be nice.

Just so we're clear after all that: My overall impression is very, very positive.

<comment><USER3> I mostly agree but wanted to re-iterate that the login as the first page is a little off-putting. If I landed there from somewhere other than an "Rate My Startup" post, I just would have left. In fact, I wanted to read a few reviews BEFORE signing up.

\- The font on the inside page with more info is way too small. Again, I would have just left.

\- Clipping something should auto-clip and not make me hit "Clip" again to actually do it.

\- I clipped a different Ask HN post that I am interested in tracking and in the sidebar all it shows me is "Hacker News | Ask HN" and "news.ycombinator.com" with no option to expand the sidebar. So if I did this often (which I would), they would all look alike. I understand that I can modify each one, but that's more clicking/typing that I don't want to do.

But I agree with DanielStraight1 that you "get" the organization I would like more than Evernote or some other sites. And that seems like the harder problem to solve. So it seems like you could clean up some of these usability items and have a hit...

<comment><AUTHOR> Thanks for the suggestions. My hope with the front page was to be minimalist and just a login. New visitors would hopefully land on the concept page, not the home. As for auto clipping I felt it would be confusing for a first time users. You can always hit enter after clicking the clipper if you want a speedier experience!

</comment></comment></comment><comment><USER4> I think this is a wonderful idea. Well done. However, I think that the name is extremely misleading. I was expecting some sort of project management tool ...

Best of luck with this.

<comment><AUTHOR> I believe gotProject is a project management tool for regular people (not companies or large project teams) but I understand your confusion.

</comment></comment><comment><AUTHOR> Here are the direct links :)

Video: <http://www.gotproject.com/?concept>

Site: <http://www.gotproject.com>

</comment><comment><USER5> This looks like it has some potential, but the 10 character limit on usernames is a bit short. That means I can't sign up with my Hacker News username ("markchristian") or my everywhere else username ("shinyplasticbag").

<comment><AUTHOR> Yes, I agree with you. I implemented this some time ago then forgot about it. I will increase the limit to something reasonable.

</comment></comment><comment><AUTHOR> Thanks to everyone who left a comment, I really appreciate the feedback!

</comment><comment><USER6> Minor nitpick on the video, at the end it says "Create and Account"

</comment><comment><USER7> i like it and i would totally use it! good job

<comment><AUTHOR> Thanks! feel free to keep using it, as is up and running now.</comment></comment>


Stolon v0.5.0 released. Cloud native postgres high availability - <AUTHOR> https://github.com/sorintlab/stolon/releases/tag/v0.5.0 <comment><AUTHOR> Just a little note, since some people, that don't know how it works ended with wrong conclusions:

Stolon has a very different architetture than governor or patroni. It has much higher consistency guarantees like avaoiding that a partitioned master is reelected (losing transactions), very strict sync replication to avoid losing any transaction, point in time recovery and many more advanced features.</comment>


With DaaS Windows coming, say goodbye to your PC as you know it - <AUTHOR> https://www.computerworld.com/article/3293429/microsoft-windows/with-daas-windows-coming-say-goodbye-to-your-pc-as-you-know-it.html <comment><USER2> Let me just pick a quote the article, as it sums the idea well:

"So, with this track record, do you want to pay good money to let Microsoft maintain your desktops for you? Yeah, that's what I thought."</comment>


FAA temporarily halts flights to NY airports amid virus-related staffing issues - <AUTHOR> https://www.nbcnewyork.com/news/local/all-flights-to-new-york-area-airports-halted/2337853/ <comment><USER2> It's due to staffing issues but it would make sense to cut travel to the state somewhat anyway, considering that it's the leader in infections among all the states right now. California, I think, is second and it has nowhere near New York's number, which is currently at around 8,500.

<comment><USER3> NY is also the leader in testing, so that isn't surprising.

<comment><USER2> That's part of it but it's also highly popular and densely populated, both contributing factors.

</comment></comment></comment><comment><USER4> Why isn't all domestic air travel on hold?</comment>


Ask HN: Best source for awesome (above average) talent? - <AUTHOR>

Where are you finding the best talent these days? Networking referrals? Linkedin? TalentBin? Headhunters? <comment><USER2> Your question is well-timed.

[https://news.ycombinator.com/item?id=7685170](https://news.ycombinator.com/item?id=7685170)

<comment><AUTHOR> Thanks for the link!</comment></comment>


Wall Street Calls Traders Back to Office - <AUTHOR> https://www.bloomberg.com/news/articles/2020-04-07/greed-and-fear-collide-wall-street-calls-traders-back-to-office <comment><USER2> In this epoch I think that you can trade with high profit margins but also with high losses due to the volatile nature of the markets. [http://agenjudiqq8.eklablog.com/the-sport-of- dominoes-a12786...](http://agenjudiqq8.eklablog.com/the-sport-of- dominoes-a127865636)</comment>


USPTO rejects Oracle's patent claim on Google Android - <AUTHOR> http://9to5google.com/2011/12/26/uspto-delivers-final-rejection-to-google-oracle-suffers-patent-claim-rejection-companys-response-due-feb-20/ <comment><USER2> This is a terrible summary.

The USPTO office rejected 17 of the 21 claims in one of the patents Oracle is suing Google using.

It's (very) helpful to Google, but saying _USPTO rejects Oracle's patent claim on Google Android_ (the title of the story) is a huge, huge overstatement.

<comment><USER3> The original Groklaw post says:

> "The USPTO has delivered one final (rejection) holiday package to Google, > this one at Oracle's expense."

The 9to5google post says:

> "The United States Patent and Trademark office delivered a final rejection > to Google at the expense of Oracle."

Almost exactly the same sentence but with completely different inplications.</comment></comment>


How to Write a Simple Interpreter in JavaScript - <AUTHOR> http://www.codeproject.com/Articles/345888/How-to-write-a-simple-interpreter-in-JavaScript

<comment><USER2> Nice article - covers pretty much everything in a reasonable amount of detail.

As for the author, it should be noted that he's a 17 year old who has been programming for 2 years. Good job - we need more people like that!

</comment><comment><USER3> I'm a bit disappointed, that the author didn't built a turing-complete interpreter. Frankly I've seen several of such articles which tell you how to build this kind of deterministic evaluator, but they all seem to stop before loops and branching.

And then there are articles about building compilers which do support loops etc, but the trick is that these constructs are implemented by some other already existing machine.

So I feel like there's some gap in my education. Pointers, anybody?

<comment><USER4> I'm not sure I'd be so quick to say it's not turing complete--the language includes building functions (not first-class functions though), but it might be possible that they can be used to emulate conditionals and branching.

Anyway, I'm pretty sure it wouldn't be that hard to extend the language to add some looping and branching constructs to the language, there's nothing really special there to add, it's just another simple type of construct to evaluate.

In my understanding, loops are implemented at the lowest level with jump instructions, but I've never really done much low-lever programming so maybe someone else can explain that better.

<comment><USER5> There is no conditional, and without a conditional you can't get a conditional.

However modifying it to support conditionals is easy. Just add a built-in function if(condition, expr1, expr2). Now you can build if's. Since the language already properly supports recursive function calls, now it is possible to implement loops through recursion, and you're off and running.

<comment><USER6> If you have function application, then you don't truly need a primitive conditional. You can represent Boolean values and conditional functions with the Church encoding [1]. This is how conditionals are represented in, for instance, the lambda calculus.

[1] <http://en.wikipedia.org/wiki/Church_encoding#Church_booleans>

<comment><USER5> Good point. But you need to have first class functions for that. This toy language does not. And converting from "this native number equals that native number" to your encoding of "true" is going to be an interesting trick.

</comment></comment></comment></comment></comment><comment><USER7> This is interesting, but I feel as though the author shouldn't be instructing one on writing a parser by hand, because this is highly impractical and there are parser generators out there that target JavaScript. One's time is better spent studying the abstract math behind how parsing works. Context free grammars are complex, and it's far too easy to do something wrong.

That being said, writing an interpreter is fun and anyone with even a mild interest in parsing or compilers should do it.

<comment><USER8> This article made me want to try making a compiler or interpreter when I get a block of free time. I actually stopped reading because I want to try to figure out as much as I can on my own.

<comment><USER7> It's a very fun thing to do! Especially if you construct your own virtual machine--function calls aren't as simple as they appear. Also fun is targeting the Java Virtual Machine using a bytecode library such as ASM.

I personally enjoy using ANTLR for parsing, so that might give you a place to start.

</comment></comment></comment><comment><USER9> If you want to see an interesting way to parse using Javascript check out Douglas Crockfords Top Down Operator Precedence paper here:

<http://javascript.crockford.com/tdop/tdop.html>

<comment><USER4> I think the parsing method in this article was borrowed from Crockford's TDOP article, except it parses a simpler language than Javascript.</comment></comment>


Register Allocation by Graph Coloring (2003) - <AUTHOR> http://www.lighterra.com/papers/graphcoloring/ <comment><USER2> I didn't realize it was that old. Also surprised that intraprocedural techniques havent had as much adoption. There's potential there for improvement in real-world compilers. Just gotta integrate it with Clang or GCC which Im pretty sure academics already have in prototypes. Then make it fast.

<comment><USER3> Chaitin published his stuff in the early 80s. Presumably the implementation is a bit older. But the idea of doing register allocation via graph coloring goes back to the late 60s and the idea of using graph coloring to minimize storage requirements generally goes back to around '63.

The interprocedural problem is mostly engineering - how to manage libraries, make files, etc., in a way that programmer will use it.

<comment><USER2> So I'll add graph coloring to the list of great things that happened in the 1960's that still kick arse today. Burroughs architecture, graph coloring, Dijkstra's methods... quite a few.

</comment></comment></comment><comment><USER4> We did a lot of this in our compilers class. A lot of fun.

<comment><USER5> We spent lots of time talking about parsing in my compiler class :-/

<comment><USER6> Unfortunately, a lot of compiler classes are like that! Fortunately, the availability of gcc / clang / path64 means that it doesn't have to be that way.

</comment></comment></comment><comment><USER7> Doesn't libfirm utilize this?

<comment><USER8> libfirm ([http://pp.ipd.kit.edu/firm/](http://pp.ipd.kit.edu/firm/)) uses SSA- based register allocation ([http://d-nb.info/986273813/34](http://d-nb.info/986273813/34)), which allows a decoupled spilling phase. Moreover, the register assignment becomes polynomial, while register coalescing remains NP-complete.</comment></comment>


Quine (computing) - <AUTHOR> http://en.wikipedia.org/wiki/Quine_%28computing%29

<comment><USER2> See <http://www.madore.org/~david/computers/quine.html> and <http://www.nyx.net/~gthompso/quine.htm>

<comment><USER3> Also, I think this was on HN last year, but it's very cool: [http://research.swtch.com/2010/03/zip-files-all-way- down.htm...](http://research.swtch.com/2010/03/zip-files-all-way-down.html)

</comment></comment><comment><USER4> I once wrote a gimp script which would create a black and white image (I think 120x120), which produced the original source code when read as a binary string.</comment>


Rebinding attacks in browsers - <AUTHOR> http://www.securityfocus.com/columnists/455 <comment><USER2> couldn't this be ameliorated by forcing dns responses with short ttls to be persistent for longer periods? how much of an impact would that have on load balancing strategies? would anything else be affected?</comment>


[pdf] Networks, Crowds, and Markets (textbook to be published by Cambridge) - <AUTHOR> http://www.cs.cornell.edu/home/kleinber/networks-book/networks-book.pdf

<comment><AUTHOR> Related course website: <http://www.infosci.cornell.edu/courses/info2040/2010sp/></comment>


Google Voice is getting an upgrade - <AUTHOR> https://techcrunch.com/2017/01/10/google-voice-is-getting-an-upgrade/ <comment><AUTHOR> It's about time! Been waiting on this forever!</comment>