A Russian App That Will Find You in the Internet Crowd - <AUTHOR> https://www.bloomberg.com/news/articles/2016-12-06/the-russian-app-that-has-destroyed-privacy-forever <comment><USER2> Clickbait about FindFace from earlier this year. A photo-to-id tool that simply utilizes the US American Facebook which has destroyed privacy forever.

<comment><USER3> But the spin is that it is the Russian tool that is based, not the American Facebook.

Welcome to propaganda.

</comment></comment><comment><USER4> The main problem with FindFace: literally _anyone_ can use it, not just government but everyone, including criminals.

</comment><comment><USER5> The American equivalent being Facebook...

</comment><comment><USER6> The actual TFA's title just turned a tad more dramatic, quote: "The Russian App That Has Destroyed Privacy Forever."

So we finally find out who really destroyed our Privacy. Forever!</comment>


How the JVM compares strings on x86 using pcmpestri - <AUTHOR> http://jcdav.is/2016/09/01/How-the-JVM-compares-your-strings/ <comment><USER2> The JVM has a number of cool features that enable you to efficiently drop down into C (FFI) yourself: these tricks aren't just for built-ins.

A quick overview:

\- First there was JNI. JNI means that you write method stubs with the "native" keyword. Then you run javah, which gives you some C glue code that you eventually need to compile. This is very fast, but it's annoying because now you need a tool chain everywhere. The Python equivalent of this is roughly writing CPython extensions.

\- People thought JNI was annoying, so Sun developed JNA. JNA lets you bind a library directly: all the magic comes with the JVM, and you can just dlopen something and call some syms. This works fine, but it's very slow. The Python equivalent of this is roughly ctypes.

\- Most recently, there's JNR and jnr-ffi. They do a very clever trick: you use JNI to get to libffi, and then you use libffi to call everything else performantly. You get roughly the performance of JNI, with the convenience of JNR. The Python equivalent of this is roughly cffi.

JNR is way more usable than I thought it would be. I develop caesium[0], a Clojure libsodium binding, using jnr-ffi and a pile of macros. I gave a talk about this at Clojure Conj (recording [1], slides [2]) if you're interested.

To be fair: this code uses intrinsics, which means that it's implemented differently than the three methods shown above, so it's still slightly different. It's just not different in a way that's meaningful to you unless you're working on the JVM itself :)

[0]: [https://github.com/lvh/caesium](https://github.com/lvh/caesium) [1]: [https://dev-videos.com/videos/Lf-M1ZH6KME/Using-Clojure-with...](https://dev- videos.com/videos/Lf-M1ZH6KME/Using-Clojure-with-C-APIs-for-crypto-and-more-- lvh) [2]: [https://www.lvh.io/CCryptoClojure/#/sec-title- slide](https://www.lvh.io/CCryptoClojure/#/sec-title-slide)

<comment><USER3> JNR (and JNA and cffi) strike me as unsafe due to the lack of type enforcement. Systems like this let you call any random pointer as if it were a C function of any type. That usually works, but sometimes doesn't. If you're lucky, mistakes make your program blow up right away. If you're unlucky, you get impossible to diagnose memory corruption.

I'd much rather write conventional bridges and have the system check that I'm right.

<comment><USER4> To be fair, when you're writing the C side of your JNI bindings, it'll also let you call any random pointer as if it were a C function of any type. You'll have non-type-safe code _somewhere_, it's just a choice of where.

<comment><USER3> Ah, not true! You'll have as much type-checking as C allows for, which is actually quite substantial. Moreso in C++. The opportunities for accidental type mismatch are much reduced: basically, to the JNI->function bindings (eliminated if you codegen the glue).

<comment><USER2> I see what you're saying, but that piece of code doesn't exist at all in JNR (so it can't have bugs) and you can codegen JNR/FFI too to eliminate the same hole, right?

I feel like saying "don't mess up this signature" is not only plausible but it's a lot better than having to deal with writing and compiling the stub for every platform.

This is why pycparser exists, for example. [https://github.com/eliben/pycparser](https://github.com/eliben/pycparser)

</comment></comment></comment></comment></comment><comment><USER5> One thing I learned about pcmpxstrx is that it's surprisingly slow: latency of 10-11 cycles and reciprocal throughput of 3-5 cycles on Haswell according to Agner's tables, depending on the precise instruction variant. The instructions are also limited in the ALU ports they can use. Since AVX2 has made SIMD on x86 fairly flexible, it can sometimes not be worth using the string comparison instructions if simpler instructions suffice: even a slightly longer sequence of simpler SIMD instructions sometimes beats a single string compare.

The SSE 4.2 string comparison instructions still have their uses, but it's always worth testing alternate instruction sequences when optimizing code that might use them.

<comment><USER6> I have the same experience. I've tried using pcmpestr in substring search a few times, and it had always turned out to not be worth it. I have however never tried it in during comparison functions, so I can't speak to that, but I wouldn't be surprised if the latency of the instruction mad at impact there too.

</comment></comment><comment><USER7> Go uses the same technique[1], or a Duff's device[2] when AVX2 is unsupported.

I am convinced that similar tricks are employed by almost every language runtime or standard library (GNU libc also does this, etc.)

[1] [https://github.com/golang/go/blob/master/src/runtime/asm_amd...](https://github.com/golang/go/blob/master/src/runtime/asm_amd64.s#L1530) [2] [https://en.wikipedia.org/wiki/Duff%27s_device](https://en.wikipedia.org/wiki/Duff%27s_device)

</comment><comment><USER8> This guy's blog is awesome. A lot of in-depth information, original research, nice and funny writing style. I hope he finds some time to write / speak more soon.

</comment><comment><USER9> I find it a bit sad that, instead of optimising the existing REP CMPS instruction to do vectorised compares like they did with REP MOVS/STOS and block copies/writes, Intel introduced another even more complex instruction that itself requires a bunch of additional support code to use. I certainly don't think it's a "good use of CISC".

<comment><USER5> pcmpxstrx is a lot more powerful than rep cmps.

I'm lukewarm on pcmpxstrx too, but for a different reason: I'd prefer the effort to go into more general purpose, highly flexible SIMD instructions (which is thankfully happening now with AVX 512).

<comment><USER10> What do you think about Cray-style vectors, which are coming back in the form of ARM SVE and the RISC-V V extension?

At least the latter claims code compiled once is compatible with _all_ possible hardware configurations, from the start (by way of giving the CPU a "remaining iterations count" and having it reply with how many it can do for the chosen vector lane shapes).

IMO, if it does end up working that well in practice, it does put all of the various incompatible versions of packed SIMD extensions in a pretty awkward spot - could we have skipped all of MMX, SSE, AVX, NEON, etc. versions with technology that has been around for almost half a century?

<comment><USER5> Sounds like a good idea to me. Seems similar to how GPUs work, which is generally a good path to follow.

</comment></comment></comment></comment><comment><USER11> Another excellent series on JVM perf/internals [https://shipilev.net/jvm- anatomy-park/](https://shipilev.net/jvm-anatomy-park/)

</comment><comment><USER12> > _But did you know there is also a secret second implementation? > String.compareTo is one of a few methods that is important enough to also > get a special hand-rolled assembly version._

oooooh.

Is there a list somewhere of all the functions that have had such special treatment?

<comment><USER13> The intrinsics for the HotSpot JVM are declared here: [http://hg.openjdk.java.net/jdk10/jdk10/hotspot/file/5ab7a67b...](http://hg.openjdk.java.net/jdk10/jdk10/hotspot/file/5ab7a67bc155/src/share/vm/classfile/vmSymbols.hpp#l679)

(look for "java_lang_Math" for instance)

<comment><USER14> Are you telling us it's not a secret then?

<comment><USER15> Don't know if this is sarcasm, but no, doubtful it's a secret. Or might depend on the JVM license. OpenJDK obviously has no secrets.

But otherwise, assembly overrides for hot paths in execution targets that support them aren't actually a big secret, just that they're rarely visible. Think Go has a lot of processed architecture specific assembly for some functions, especially crypto iirc.

<comment><USER14> Yeah, it was a sarcasm. The article (quoted in parent comment) called it a secret implementation.

</comment></comment></comment></comment></comment><comment><USER16> I think it could be fairly profitable performance wise to write a specialized version of compareTo (simple memcmp) for cases where the result is directly compared to 0, equal or not equal case.

Or to have compareEqualityTo() version.

Current version supports greater and less as well, which is of course useful in many data structures and sorting, but might not represent bulk of the call sites. This feature does not come for free.

This alternative version (memcmp alias) could be written to run faster.

Only benchmarking could tell whether or not it's ultimately worth it.

<comment><USER17> That version exists - it's called equals(), and it generates different code that doesn't need to distinguish the less than/greater than cases.

<comment><USER16> Of course, silly me, haven't touched Java for a while.

</comment></comment></comment><comment><USER18> > The code that generates this, MacroAssembler::string_compare in > macroAssembler_x86.cpp is well-documented for the curious.

I was expecting that this would be some C++ code, or maybe inline assembly, but it turns out it's just a bunch of macros that are a thin layer over assembly instructions. Is this done for portability, to abstract over differing instruction names on different architectures?

</comment><comment><USER19> What a lovely ready; I enjoy learning about the intricate details of the JVM. Thank you for sharing!

</comment><comment><USER20> tl;dr: The string comparison intrinsic in the JVM uses a vectorised string comparison instruction.

(vpcmpestri (of the pcmpxstrx family) isn't an especially crazy instruction to use for string comparison. That's what it's designed for.)

<comment><USER21> Thanks for the tl;dr. I think the "crazy" adjective refers to the instruction, and not to the use of it. As the article explains, the instruction is quite complicated and has a large number of parameters.

<comment><USER22> Is there any place to get a proper manual of all these uses?

</comment></comment></comment><comment><USER23> Click here to discover that crazy instruction! C++ programmers HATE IT!!

(I am just ironizing about the title, the content is actually great!)

<comment><USER24> Yes indeed. We've un-clickbaited the title. Nothing is safe, it seems.

</comment></comment><comment><USER25> UTF-16 is one of these ill-fated developments that curse some languages & platforms (WinNT incl Win10, WinAPI32, Java, Flash, JS, Python 3) to his day.

      compareTo uses 0x19, which means doing the "equal each"
      (aka string comparison) operation across 8 unsigned words
      (thanks UTF-16!) with a negated result. This monster of an
      instruction takes in 4 registers of input:

<comment><USER26> Java uses UTF8 in latest release for strings without special characters.

[http://www.baeldung.com/java-9-compact- string](http://www.baeldung.com/java-9-compact-string)

UTF16 is not really a curse for languages that require it. String operations in non-English languages are very fast because of it, and most software these days has to deal with localization.

<comment><USER27> Well, not necessarily a curse, but a suboptimal solution. Are there any situations where UTF16 is a clear upgrade over UTF8?

<comment> Const-me Any non-Latin string operations.

While technically UTF16 is variable length, 99.99% cases use single word per character. I.e. on modern hardware with branch prediction and speculative execution, these branches don't affect speed. With UTF8, CPU mispredicts branches all the time because spaces, punctuations and newlines are single bytes even in non Latin-1 text.

<comment><USER16> I think the most common operations are comparisons for equality and copying anyways. UTF-8 is faster for those.

I tried out how fast I could make UTF-8 strlen, with an assumption of a valid UTF-8 string. The routine ran at 18 GB/s on a single core using SSE.

> With UTF8, CPU mispredicts branches all the time because spaces, > punctuations and newlines are single bytes

I don't understand this sentence. Why would there be any more mispredictions because of those being single bytes? These days code is so often bandwidth limited if anything, so smaller data helps.

<comment> Const-me > most common operations are comparisons for equality and copying anyways

Indexing & substrings are common, too.

> These days code is so often bandwidth limited if anything

Right, and for 1 billion Chinese speaking people UTF16 is 2 bytes/character, UTF8 is 3 bytes/character.

<comment><USER16> > Indexing & substrings are common, too.

Indexing code points in both UTF-8 and UTF-16 requires reading the whole string up to index location. Substrings are the same as well.

> Right, and for 1 billion Chinese speaking people UTF16 is 2 bytes/character, > UTF8 is 3 bytes/character.

That's true for a text file without markup. But most text is not like that in 2017. HTML is probably the most common text format nowadays.

So let's see how a popular Chinese language website does.

      curl http://language.chinadaily.com.cn/ --silent | wc -c
      52678

      curl http://language.chinadaily.com.cn/ --silent | iconv -f utf8 -t utf-16le | wc -c
      93368

So UTF-8 seems to be quite a bit more efficient in this case, 52678 bytes. When converted to UTF-16, same page was 93368 bytes.

<comment><USER28> > Indexing code points in both UTF-8 and UTF-16 requires reading the whole > string up to index location. Substrings are the same as well.

Java's String functions don't index by Unicode code points, though. Java strings are encoded in UCS-2, or at least the API needs to pretend that they are.

<comment> Const-me Right. Same in C#, C++ STL, and in Apple's obj-c/swift.

<comment><USER29> Swift takes a different approach than Objective C[0].

[0] [https://www.mikeash.com/pyblog/friday-qa-2015-11-06-why- is-s...](https://www.mikeash.com/pyblog/friday-qa-2015-11-06-why-is-swifts- string-api-so-hard.html)</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment>


Shoot down the stars - <AUTHOR> http://blog.allotrop.com/post/45661089251/shoot-down-the-stars

<comment><USER2> Interesting perspective of ratings. I always wondered if the UI interactions are tangible enough for rating an item as web is often a whimsical activity. This somehow justifies it well .

<comment><USER3> One of the things that I believe leads to fewer people rating things is the amount of mental work involved in giving a rating other than 1/5 star. This is where the like/dislike, up/down systems fare much better.</comment></comment>


Uber Strikes Deal To Lower The Cost Of Car Ownership For Drivers - <AUTHOR> http://techcrunch.com/2013/11/24/uber-driver-car-financing/

<comment><USER2> Great! Let's encourage drivers who can't afford a car to take out (in effect) a huge loan on an expensive car that they couldn't otherwise obtain, then make their more-affordable repayments conditional on working as a reduced-rights contractor for us. So this is basically indentured service then?

<comment><USER3> I think we've already seen something like this in practice. One missing piece: let's make money by betting that these drivers won't be able to pay the loans.

<comment><USER4> The difference between this and a mortgage for a house is that the value of the collateral _will_ decrease over time, even in the best case.

</comment></comment></comment><comment><USER5> Can anyone explain to me what is so attractive about Uber?

To me, Uber is just taxi with smart phones. My opinion is that is that Uber is avoiding a lot of the bureaucracy and fees that real taxi companies are paying for their rights and things such as insurance. I always figured that once the taxi companies figure out the convenience of smart phones they will start to integrate that technology more aggressively and be able to fight Uber back.

But there must be something I'm not seeing.

> "The $258 million is an 86 percent chunk of Google Ventures' $300 million > dollar a year fund..."

That's a lot of money, and that's a large part of Google Ventures fund. What are they seeing that I'm not seeing? Is there a future vision of Uber that's going over my head?

<comment><USER6> Convenience and reliability. Lots of people have stories of cabs never showing up, or taking forever, or barking and grumbling when offered a credit card as payment. Particularly if you're in a hurry, paying with a credit card also seems to take a lifetime – at least with cabs that use the paper carbon copy method (which they do here in Seattle).

Uber experience: you open the app and see on a map where all the cars are. It tells you _exactly_ how many minutes the closest cab will take to arrive. When you arrive at your destination, you just hop out. The app knows what credit cards your account has on file and will just charge the one marked as the default or the one you've selected. There's no tipping. If you're in a hurry, not having to wait around to sort out the payment is a big deal.

I've never had an Uber not arrive or arrive exceptionally late; when I've had a less than stellar experience, customer service has been extremely responsive and offers refunds. On a recent trip, the driver made a poor route choice just because there ended up being a bunch of construction downtown, so it ended up taking longer than it could have. Customer service adjusted my fare to what it would have been had we taken the optimal route.

</comment></comment><comment><USER7> _" Kalanick said drivers could expect to save anywhere from $100 to $200 on monthly payments, depending on the make and model of the car they're buying."_

I'm not from the US, so don't have an idea of car ownership costs, but is this that big a deal?

<comment><USER2> It's a little difficult to say for sure, there's not enough information in the article to judge (e.g. No info on lease length) Since it is in effect a structured loan, any advantage is better expressed in terms of interest rate improvements over a non-Uber lease.

</comment></comment><comment><USER8> Yes, because that what we need - more cars, more traffic, more pollution and more health problems.

<comment><USER9> While adding more Uber drivers does directly increase the number of cars on the road, that growth could be offsetted (not sure about the actual numbers) by people deciding they can rely on mass transit + Uber instead of driving a car.

Or it just means that there are more Uber drivers and less taxi drivers.

</comment></comment><comment><USER10> When it includes Palm Springs I'll apply.

</comment><comment><USER11> Sounds a lot like Share-Cropping.</comment>


Who gets to be creditworthy? (Audio) - <AUTHOR> https://creative.gimletmedia.com/episode/who-gets-to-be-creditworthy/ <comment><AUTHOR> One of our farmers was quoted at the opening of this podcast, and I thought it was a pretty interesting insight into some of the new ways credit scores are changing in the developing world.</comment>


GoDaddy Uses ICANN Domain Verification for (Dirty) Marketing Purposes - <AUTHOR> http://fvrit.com/archive/2008/03/13/godaddy-uses-icann-domain-verification-for-marketing-purposes.aspx

<comment><USER2> I have all of my domains under GoDaddy and this company's making me nervous. GoDaddy's become a liability.

Anyone suggest a different registrar? One that will not steal your domain or shut your down because someone complains?

<comment><USER3> I just transfered all of my GoDaddy domains to <http://www.enomcentral.com/>. A good friend of mine has had a merchant account with them for years with no problems. Their interface is is pretty good too.

Transferring out of GoDaddy is a challenge though (but it feels great once you finally rip your domains out of their cold, dead hands). They make the process convoluted and confusing, hoping you'll give up. But don't give up (never give up!). Here's what you have to do:

\- log in to GD and go to the domain manager. Go to each domain individually and check to see if the domain privacy proxy is turned on. If it is, it means that your whois info is being proxied by <http://domainsbyproxy.com/> (it seems to be a sister site to GD). You need to find your account number for DBP which they would have sent to you when you reg'd the domain. You need to login to DBP and disable the proxy for all of your domains.

\- Go back to GD and you'll notice the proxy is now off (magic!). You now need to unlock each domain and get an Authorization Code for each one by clicking the send Auth Code By Email link on the domain screen. Most (all?) registrars will request this when you initiate the transfer.

\- Go to your new registrar and initiate the transfer (this is where you'll need the auth code).

\- The new registrar will send an email to the administrative contact for the domain(s) confirming the start of the transfer. You may have to wait up to a day or so to get this.

\- GD will send an email confirming the transfer is started. You can also check GD periodically because eventually they will display your domains as Transfer/Pending. Go to the My Domains page, mouse over the Domains tab and click pending transfers. Select all of your domains and click the "Accept/Decline" icon (it's a very confusing icon) and choose accept (obviously) and be sure to click all the Okays.

And you're done! If you use GoDaddy's DNS servers you may have to reconfigure your DNS and mail settings (maybe someone else has experience with that?). If you already use a 3rd party for DNS and mail then the transfer shouldn't affect your site/email.

<comment><USER2> enom seems to have some issues as well: [http://yro.slashdot.org/article.pl?sid=08/03/09/025222&f...](http://yro.slashdot.org/article.pl?sid=08/03/09/025222&from=rss)

<comment><USER4> When you get a domain through enom the spam starts automatically but they do have a great api at least.

</comment></comment></comment></comment><comment><USER5> Not to mention that there isn't even an "ICANN Domain Confirmation" link on the &*&^ site!

They're hoping you'll get bored looking for it and drop some money.

<comment><USER6> There is. But it might take a bit of time to notice it.

<comment><USER5> Ah it's an image. That was way to hard to find.

</comment></comment></comment><comment><USER7> pretty bad, but they aren't a netsol :)</comment>


Ask HN: OSX vs Ubuntu for Ruby/Rails Development - <AUTHOR>

i use OSX since 5 years. Now i think about mac mini or PC. But i don't know which one OS to chose . Any pros for leave OSX for Ubuntu? HELP ME <comment><USER2> I never has a mac and i'm using linux-based distros for 8 years now. I love rails and i'm happy with the following setup:

Elementary OS Luna beta (based on Ubuntu LTS ([http://elementaryos.org](http://elementaryos.org)), "rbenv" as ruby version manager ([https://github.com/sstephenson/rbenv](https://github.com/sstephenson/rbenv)), "bundler" as gem packages manager ([http://bundler.io](http://bundler.io)), Sublime Text and vim as code editor ([http://www.sublimetext.com](http://www.sublimetext.com)) and SQlite as developement database (or simply install PostgreSQL or MySQL throught apt-get package manager)

</comment><comment><USER3> I use Ubuntu. In a past life I was strictly a windows user. The thought has crossed my mind to switch to a mac because of GUI nav speed and peer pressure. It's not that trendy to haul around a 17" Dell XPS with a big power cable to a coffee shop or ruby meetup. Luckily I realize that those good with macs could be equally fast on Ubuntu with practice and being trendy is overrated. It's all about getting things done and I like having a big monitor + power.

</comment><comment><USER4> Ubuntu is a far better option for RoR development. \- apt-get is far better than homebrew \- many things just work better in linux land (especially if you are into emacs and the like). \- development on a system more closely resembling production

the only thing OSX does better (for RoR) is development with GUI programs and fonts... which most ubuntu-friendly developers are willing to do without.

</comment><comment><USER5> Both are fine. If you use an OS X-only editor like TextMate, that may be an issue.

</comment><comment><USER6> I use vagrant on my mac that has Ubuntu loaded on my VM.</comment>


Confession as an AI researcher; seeking advice - <AUTHOR> https://www.reddit.com/r/MachineLearning/comments/73n9pm/d_confession_as_an_ai_researcher_seeking_advice/ <comment><USER2> Being a grad student, this person is at a perfect place to build up their math background. Any school almost certainly offers the following:

1\. Convex Optimization -- not all problems are convex, but solutions for nonconvex problems end up primarily using convex methods with slight adaptations.

2\. Stochastic Optimization -- ML is pretty much all stochastic optimization. No surprise there.

3\. Statistical/Theoretical Machine Learning -- courses built around concentration bounds, PAC learnability, and the Valiant/Vapnik school of thought. This gives you what you need to talk about generalizability and sample complexity.

4\. Numerical Linear Algebra -- being smart about linear algebra is most of efficient machine learning. Knowing which kinds of factorizations help you solve problems efficiently. Can you do a Gram-Schmidt factorization? Cholesky decomposition? LU factorization? When do these things fail? When do you benefit from sparse representations?

5\. Graphical Models -- Markov chains, Markov fields, causal relationships, HMMs, factor graphs, forward-backward algorithm, sum-product algorithms.

If you're in school, _take advantage of the fact that you 're in school_.

Once you have a grasp on these things (and you'll have to catch up on real analysis, matrix calculus, and a few other fields of math), you'll be able to start reasoning about ways to improve existing methods or come up with your own. I think a lot of it is just developing mathematical maturity to give you a vocabulary to think about things with.

<comment><USER3> What courses (starting from pre-calculus) should one take to do what you listed above? I want to match your recommendations to course titles starting with pre-calculus. List book recommendations as well if you would. Thanks!

<comment><USER4> I guess baby Rudin (or Hubbard & Hubbard for something simpler) in the analysis department; and Halmos (or Axler) in the linear algebra department.

This is, essentially, Math 55. All 4 books have been used at different stages in this famous course.

<comment><USER5> Halmos seems to discuss the same things as Hoffman and Kunze, which is the more "standard" and recommended book. Nevertheless after these you will still have to read up on multilinear algebra (tensors and determinant-like functions) as well as stuff on the numerical side of linear algebra.

</comment></comment></comment></comment><comment><USER6> Learning all the advanced math within a few years is a hopeless endeavour. It would take decades of hard work because there is so much out there, and we need to know all of it if you want to make progress (e.g. pull out a fancy- name theorem out of nowhere to solve some practical problem).

I find a better approach is to focus on a few basic ideas you need _specifically_ for your work and digging deep in there. Nobody can be expert- level in everything, but you can be expert level in your specific domain of research.

Also for ML stuff, it's hard to overemphasize the importance of understanding linear algebra really well. Here is a excerpt of a book I wrote on LA which should get you started on learning this fascinating swiss-army-knife-like subject: [https://minireference.com/static/excerpts/noBSguide2LA_previ...](https://minireference.com/static/excerpts/noBSguide2LA_preview.pdf)

<comment><USER7> "Learning all the advanced math within a few years is a hopeless endeavour."

Not in my experience. It's possible to get the equivalent of a bachelors and masters in math within two years (which is enough to overcome the issues listed in the post), but it's all you'll be doing for that period of time. Well worth it imo.

<comment><USER8> > Not in my experience. It's possible to get the equivalent of a bachelors and > masters in math within two year

Seriously.

In fact, _most_ people who learn this math learn it in a period of 2-3 years. Far from being impossible, learning this math in a few years is normal. It's not even a full time job. Most people learn all this math while also doing other classes and school stuff. Even a very dedicated math major probably only spends 20-25 hours a week actually studying math. I'm not sure much more than that is sustainable for most people anyway.

Now, I'll grant, this is going to be a lot harder to do without the structure of well thought out syllabi and lectures, but it's certainly manageable.

<comment><USER9> If you're already a grad student you can usually take any undergrad course at your institution for free. It will slow down your progress on your graduate degree, but you might as well do it right if it's what you really want.

</comment></comment></comment></comment><comment><USER10> You don't need to understand everything in every field and I doubt very many people do.

When Einstein was working on general relativity, he had a lot of help from friends and colleagues who pointed him towards the math he needed. He didn't learn differential geometry until he was already deep into general relativity.

Find a level of abstraction that you're comfortable with and learn to be okay with black boxes at the lower level, and only dig into those boxes when what's inside them actually matters.

<comment><USER11> I think an important lesson for any grad student is to learn to read through the bullshit in papers and try and understand what the authors actually did.

It helps a lot that in CS you can often see the code that the authors published along with the paper. Just staring at formulae doesn't mean much, because for all you know the author just hammed up the equations to get their paper into a top conference. That's not to say that the equations are excessive, or the authors are being misleading, but I think there is definitely an expectation in some fields that putting equations in makes your paper look clever even if they're broadly unecessary.

It's also wildly different depending on the field. If you look at variational methods in computer vision, images are [continuous] mappings from some domain onto the reals (I : Ω->R3 for colour). Does that change the fact that an image in memory is just a bunch of numbers in a grid? Not really, but it's bloody confusing the first time you see it.

This doesn't help with understanding the maths, but at some point you have to give up and say "This guy proved it, and someone else peer reviewed it, so I can use it to solve my problem". It's perfectly OK to stand on other people's work and still make creative contributions to your field, that's the point of research.

<comment><USER12> >I think an important lesson for any grad student is to learn to read through the bullshit in papers and try and understand what the authors actually did.

We actively work to make our writing hard to understand in this field. I do this all the time myself. I don't really need this complex looking equation to make my point. But if I don't have it in there a reviewer will think my writing is not academic enough. So there you have it. Once you go in realizing this is the case everywhere, it becomes a lot easier to understand academic papers.

<comment><USER13> I get your point but I wish this wasn't the case with most research. I, like the author, am not a math guy but have been reading tons of ML papers recently. I usually skip the formal definition parts and get to the 'juicy' implementation parts.

I wish there was a ELI5 section in each paper.

<comment><USER14> What have been your favorite papers so far?

<comment><USER13> That's hard as I haven't read too many. The recent deepmind papers (the ones about imagination) were good. The papers were pretty standard but they came along with explanatory blogpost[1] and some videos covered them too[2][3][4]. This supplementary content is what made them accessible for me.

[1] [https://deepmind.com/blog/agents-imagine-and- plan/](https://deepmind.com/blog/agents-imagine-and-plan/) [2] [https://www.youtube.com/watch?v=xp- YOPcjkFw](https://www.youtube.com/watch?v=xp-YOPcjkFw) [3] [https://www.youtube.com/watch?v=agXIYMCICcc](https://www.youtube.com/watch?v=agXIYMCICcc) [4] [https://www.youtube.com/watch?v=56GW1IlWgMg](https://www.youtube.com/watch?v=56GW1IlWgMg)

</comment></comment></comment></comment></comment></comment><comment><USER15> How does one expect to do 'AI research' without much of a background in math? Machine learning is pretty much all math.

Researchers are generally expected to be experts in their field. The people writing the papers on arxiv likely spent most of their lives learning about machine learning and mathematics.

Unfortunately, there's not an easy path to become an expert. One just has to dig in and learn from the ground up.

Edit: The good news is that it's never been as easy to learn math than it is now. When I was an undergrad in math, there were almost no resources available to learn the intuitions behind the math. One just has to keep doing proofs and exercises over and hope that it would 'click' at some point. But, sometimes that wouldn't happen until many years later. Nowadays, one can watch YouTube videos where experts describe the intuition behind the math. It's awesome.

<comment><USER11> Absolutely you can do 'AI Research' without a degree in maths. Sure you need a grounding in linear algebra, stats, probability and calculus, but not much more than a CS or physics degree will teach you. That stuff is indeed learned easily, and it's not what the Reddit user is worried about.

That also ignores applications of machine learning, which is also a massive (and lucrative) field. But because it's a trendy field, I think there is an obsession with people _needing_ to understand everything theoretical that comes out for fear of missing the boat.

Some of the really interesting papers that have come out over the last few years - for example artistic style transfer and Faster R-CNN - have hardly any maths in. You can count the equations on one hand in both those papers. No doubt the authors know their stuff, but how readable are those papers compared to e.g. a 100-page proof? Which did I learn more from?

They're a combination of two things: intuitive network architecture and a clever loss function. The first thing is a combination of intuition and programming, the second involves a little maths, but nothing outrageous.

[1] [https://arxiv.org/abs/1508.06576](https://arxiv.org/abs/1508.06576) [2] [https://arxiv.org/abs/1506.01497](https://arxiv.org/abs/1506.01497)

<comment><USER15> You're right - I see now when he says 'AI Research' he really means doing applied ML. And, that can certainly be done without knowing everything.

If the goal is to make a lot of money doing applied ML, then become a consultant and aim to know 10% more than the customers. If the goal is to create models that are relatively effective, then read tutorials, play with data, experiment, and iterate. But, if the goal is to create very effective models and be able to actually explain why they work (which I think is what many companies want), then one has to understand the math.

That is, showing some interesting relationships, trends, predictions, or inferences on a data analysis portal or consumer web site is one thing. But, using ML to dispense medication, regulate a medical device, drive a power plant, or identify criminal suspects - those may require different skills.

(BTW I don't mean to disparage the middle group, as that's largely what I do. But, luckily I have people in the latter group who can validate what I'm doing.)

</comment></comment></comment><comment><USER16> ""I understood nothing, but it was really fascinating," he said. So Scholze worked backward, figuring out what he needed to learn to make sense of the proof. "To this day, that's to a large extent how I learn," he said. "I never really learned the basic things like linear algebra, actually — I only assimilated it through learning some other stuff.""

[https://www.quantamagazine.org/peter-scholze-and-the- future-...](https://www.quantamagazine.org/peter-scholze-and-the-future-of- arithmetic-geometry-20160628)

<comment><USER17> I've long wanted a series of interactive math ebooks that work that way. Each would take one interesting theorem, such as the prime number theorem, and work backward.

When you start the book, it would give the theorem and proof at a level that would be used in a research journal. For each step of the proof, you would have two options for getting more detail.

The first option would be at the same level, but less terse. E.g., if the proof said something like "A implies B", asking for more detail might change that to "A implies B by the Soandso theorem". Asking for more detail there might elaborate on how you use the Soandso theorem with A".

The second expansion options gives you the background to understand what is going on. In the above example, doing this kind of expansion on the Soandso theorem would explain that theorem and how to prove it.

Both types of expansion can be applied to the results of either type of expansion. In particular, you can use the second type to go all the way down to high school mathematics.

If you started with just high school math, and used one of these books, you would get the basics...but only those parts of the basics you need to understand the starting theorem.

Pick a different starting theorem, and you get a different subset of the basics. It should be possible to pick a set of theorems to treat this way that together end up covering most of the basics.

That might be a more engaging way to teach mathematics, because you are always working directly toward some interesting theorem.

<comment><USER18> Yes, you and absolutely everyone else in the world that loves math, didn't have time to get a phd and isn't elitist wants this.

Sadly, the monetization of this is tricky. Probably has to be an open source effort. Need some visionary like wales or khan, but they are very very rare.

</comment></comment></comment><comment><USER19> I think this is fairly common first-year grad student emotional response. And quite frankly, it is the job of your mentor and department to ensure you receive sufficient training for an academic or research career.

Modern AI is evolving rapidly but there is a foundation upon which everyone draws upon. The Sutton and Barto book is one such foundational text.

Find a collaborator in the Math department to work with. And participate daily in stack overflow forums for math and stats, such as Cross Validated.

I can also recommend CASI by Efron and Hastie. Deep historical understanding of where we are today in probabilistic inference.

[https://web.stanford.edu/%7Ehastie/CASI/](https://web.stanford.edu/%7Ehastie/CASI/)

</comment><comment><USER20> The explanation by TillWinter about systematic task assessment is quite impressive

[https://www.reddit.com/r/MachineLearning/comments/73n9pm/com...](https://www.reddit.com/r/MachineLearning/comments/73n9pm/comment/dnrsmh9)

</comment><comment><USER21> I always had the same feeling. I'm not bad at math in general (I'm not well versed in high level maths either), but as a developer, trying to jump in the ML field seems really impossible. One would think that he could teach himself ML algorithms, but you ALWAYS end up reading math notation instead of pseudo code.

To be honest the 3blue1brown videos seems really wonderful at explaining what is going on without going too deep, as the math in ML lecture seems to be trying to prove everything, and is always trying to teach using math notation all the time.

I guess this is happening because most of ML is mostly coming from research since it's all new, so it's being taught mostly by people who can grok the math, meaning mathematicians, it's not taught by people who are programmers. This really shows how much math should keep being math, and not leak into fields where practice matters more. Programming languages and pseudo code are not for nothing. Computers don't talk math.

So as years go by, ML will be taught more as a practice subject rather than a theory one, and things will get better. I think it's just a matter of how it's being taught, because reading code will always make more sense than reading high level math. Videos and oral explanations also help a lot.

<comment><USER22> You might appreciate the Deep Learning for Coders courses from Fast.ai. It's basically ML as a practice subject rather than theory as you suggested.

I felt similar to you when I first started learning ML but their code first approach really helped it click for me on an intuitive level. Then you can go back and dig into the maths behind it.

</comment></comment><comment><USER23> Currently getting my Masters in AI. I'll be honest, I can understand the concepts when presented to me, but the mathematical proofs are beyond me. I've learned to be ok with that. There's just not enough time in a 2 year program to teach myself the underlying vagaries of everything I encounter.

</comment><comment><USER24> Sound like a need for a "imposters handbook" for Mathematics, just like there is for Computer Science: [https://bigmachine.io/products/the-imposters- handbook](https://bigmachine.io/products/the-imposters-handbook)

</comment><comment><USER25> A little off-topic, but hope this would be an another reminder that despite having huge success/hype these days, deep learning/machine learning are just another tools for solving problems, and you can't go really far if you just treat it like a magic framework (i.e. import tensorflow as tf) and fail to understand its underlying principles.

</comment><comment><USER26> When I left school, I found myself in a similar boat, and decided to set a goal of getting myself the knowledge equivalent of an undergrad degree in math. I already had a physics degree under my belt, so it wasn't as long of a path as it might be for others, but over a bunch of years of self study it paid off. When it comes to your career a few years is nothing. The strong foundation pays regular dividends because learning things that use it comes so much more quickly.

It's a huge, slow, painful investment, but totally doable and with tremendous ROI if you want to work with stats/ML/optimmization/really any numerical computing for a living.

The reason I recommend this route is that most of the more advanced math books you will encounter will assume this stuff as the readers' common knowledge. Having that foundation, the majority of the literature is already tailored to you!

</comment><comment><USER18> A technique I have found that works well is read _a lot_ of paper and their citations, but don't dive deep. Each paper usually provides some easy to grasp insight (far too little per paper, but that is elitism for you) that you can use to get a good picture of the field. Reread papers to grasp more insight. Once you have a good overall picture, find some area/problem that really interests you, you like the math, and isn't covered well and bone up on the math techniques. Do your research and present.

Showing up at thesis defenses is good too. Learn a lot from the back and forth with advisors.

The key is to understand at a high level why the different math techniques are being used without actually understanding all the details. This won't be sufficient for your own work,but at least you'll have a good idea how your part fits in the scheme of things.

</comment><comment><USER27> That person approaches the issue as his personal problem. However, there are likely many other students around him with the same problem. It is a problem of the whole field recently.

One solution would be simply to a arrange a local seminar, and understand a couple of papers _in full detail_. It would help to invite a couple of mathematically aware students, from mathematics, physics, or the part of cs faculty where they prove stuff. They should be able to explain and answer questions immediately, which is way more effective than reading whole books or taking courses. Those can be read for details later.

If the papers for the seminar are deep learning papers, part of the outcome of the seminar is likely to be an understanding that the authors of these papers do not necessarily understand the mathematics themselves.

</comment><comment><USER18> This all does bring up a good point. Why not create a publicly accessible 'math tree' that people can use to learn about any kind of math. If there is a symbol or step they don't understand they should be able to follow it all the way down to basic counting.

</comment><comment><USER7> Based on your post, I highly recommend taking a year or two off to focus on math only. You can get the equivalent of a bachelors and masters in pure math in just two years (if that's all you're doing with your time), and it would be enough to fix all the issues you're experiencing. Just take the pure math courses instead of computational math, as abstract and difficult as possible, it will generalize much better :).

I got into math for exactly the same reasons while doing research in computer vision as an undergrad, and taking the requisite time off to learn advanced math (actually going overboard on it) has been an incredible boon to my AI career.

</comment><comment><USER28> I think the real issue is that the OP has a mistaken expectation that they should understand everything. For instance, the group that wrote the Wasserstein GAN paper are surely those that think night and day about distance metrics. And they might be totally lost reading a paper about some energy based method that relies on concepts from physics.

The point is that researchers have their little niche and they try to make contributions in areas adjacent to it. It's unrealistic to think everyone publishing papers understand all the other papers, particularly in such a cross-disciplinary field like ML. There's also a big gap between a researcher deep in their career and a student fresh out of a masters program.

It's also hard to transition from someone who's used to reading and understanding textbooks to someone who's often reading really technical research and understanding very little of it at first. You just have to push through and have confidence that you'll eventually learn enough to make a contribution. That's what it means to "become an expert"\--you start off as not being an expert and then beat your head against the wall for a few years until you bootstrap your way out of it. And if you want to do it in a reasonable amount of time, you should probably choose something you have some of the fundamentals for.

</comment><comment><USER29> From one of the comments:

>"Professional heavy math people are those who said in the 60s that the perceptron's limitations proved all AI was impossible. And in the 90s that one hidden layer was all you needed, deep learning was useless."

Can anyone provide the citations for this? I was aware of the latter but not the first one. You can find people still repeating the one layer stuff up to a few years ago just by reading stackexchange.

<comment><USER30> I wonder if the author is referring to the oft-repeated claim that Minsky's and Papert's proof that a perceptron cannot learn the Xor function had a chilling effect on research into neural networks generally, even though Minsky and Papert themselves had shown that multi-layer networks were capable of doing so [1][2].

I realize that even this alleged misunderstanding is not the same as a claim that AI is impossible. The closest attempt of a mathematical proof of the impossibility of AI that I am aware of is the Lucas-Penrose argument from Gödel's first incompleteness theorem [3].

[1] [https://en.wikipedia.org/wiki/Perceptrons_(book)](https://en.wikipedia.org/wiki/Perceptrons_\(book\))

[2] Minsky M. L. and Papert S. A. 1969. Perceptrons. Cambridge, MA: MIT Press.

[3] [http://www.iep.utm.edu/lp-argue/](http://www.iep.utm.edu/lp-argue/)

<comment><USER29> Thanks, the first ref looks like it may be the one.

</comment></comment></comment><comment><USER31> Do you really not need familiarity with the relevant math to be admitted to AI doctoral programs? I wouldn't have thought that was the case.

</comment><comment><USER32> How does one get into an ML PhD like this? I was under the impression it was impossible if you're not a math double major.

<comment><USER33> I am a math double major doing a research masters in ML/CV right now, and I know plenty of pure CS majors who are doing just fine. The math that 95% of ML scientists use is not that hard to grasp. Sure, when they encounter functional analysis stuff, they start to cry inside a little, but that doesn't happen very often.

<comment><USER32> Interesting. I'm interested in a related MS/PhD with a similar math background as OP and assumed I was disqualified.

</comment></comment></comment><comment><USER34> As Alan Kay noted, the right point of view can add 80 IQ. I was in a quantitatively heavy field and always felt out classed by those with strong physics and maths backgrounds. Nevertheless I published to papers in Nature journals and overturned about 10 years of high profile research, not because I was smarter but because I spent more time trying to find the right perspective and when I found anomalies instead of brushing over it, confident in my own intelligence, I instead drilled down until I found the root of the problem — something that everyone else had overlooked. You don't need to be a classical genius to make a contribution but you probably do have to be tenacious.

</comment><comment><USER35> One thing that hasn't been mentioned: learning mathematics from talking to another human can be 10-100 times faster than getting it from books. Another thing: mathematics is huge and seems to accommodate all personality types. Pick something that turns you on and grow outwards from there. The folks on reddit seem to be obsessed with Rudin, and that's good stuff, but there's so many other roads to follow.

And I'm so impressed by how much better the comments are here than on reddit! Good job HN, you rock.

</comment><comment><USER36> > the "utility density" of reading those 1000-page textbooks is very low. A > lot of pages are not relevant, but I don't have an efficient way to sift > them out. I understand that some knowledge might be useful some day, but the > reward is too sparse to justify my attention budget. The vicious cycle kicks > in again.

That is their main problem. All those useless pages are what becomes useful later.

And we find the same kind of attitude everywhere in tech: why read a full RFC when you can assume shit and get done with a 2 paragraphs tutorial?

</comment><comment><USER37> I'll try to reply to the frustrations of the author of the OP.

I'll give some high level views and also outline the math topics mentioned, high school through much of a Ph.D. in parts of applied math.

I respond in three parts:

Part I

I have a good pure/applied math Ph.D. and work in applied math and computing; while I call my work applied math and not artificial intelligence (AI), machine learning (ML), or computer science, it appears from the OP that there is significant overlap between my background and work and what the OP is concerned about.

The Reddit post by the guy in Germany was terrific although easy to parody as a big feature of old German _culture_! :-)! That post is maybe a bit over organized.

I've posted too often that the best future for computer science was pure/applied math, e.g., that someone seriously interested in the future of computing should as an undergraduate just major in pure math with some applied math and, essentially f'get about anything specifically about computer science.

Or, for the _essential_ computer science, write some code in some common procedural programming language for some simple exercises, check out at the library D. Knuth's _The Art of Computer Programming, Volume 3, Sorting and Searching_ , learn about big-O notation, the heap data structure and heap sort, as an exercise program, say, a priority queue based on the heap data structure, learn about the Gleason bound and how heap sort achieves it so is in an important sense the fastest possible sort algorithm, as a side exercise look at AVL trees, and call computer science done for an undergraduate! This is partly a joke but not entirely.

Well, it appears that the OP has started to discover some of why I've said such things about math.

This role for math is just a special case of the old standard situation that, in nearly all fields, the best work _mathematizes_ the field as in, e.g., mathematical physics. Indeed, there is the old joke that good coverage of the math needed for theoretical physics is so much about just the math that can do the physics just in the footnotes.

It is a standard situation that nearly everyone in the STEM fields is convinced that they need to know more math. As I read papers by computer science professors, I tend to agree!

Here I'll try to help the person with their lament in the OP:

(1) Start at about 50,000 feet up and begin to identify in what fields, broadly on what problems, you want to work. Remember: One of the keys to success is good, early work in problem selection.

(2) If you want to work in AI, I suggest you try to regard the current headline topics in AL/ML as nearly irrelevant. Sure, for political reasons, might have to keep up, and maybe there are some current, hot applications, but I don't see that work -- first-cut, ballpark, basically empirical curve fitting to huge quantities of data -- as much of a start on AI.

E.g., at one time I was hired to work in an AI project, specifically expert systems. My first reaction was that expert systems -- rule based programming, working memory and the RETE algorithm, rule firing conflict resolution -- were junk and, in particular, nothing like a good start on AI, programming style, or anything else. After 20 years my opinion has changed: Expert systems were worse than junk, as a style of programming, for much of anything, and in particular on anything like AI.

Maybe I got some revenge: We were trying to use expert systems for the monitoring and management of large server farms and networks. For the monitoring, for essentially _health and wellness_ , they were using essentially just thresholds set by hand on single variables one at a time. My view was, wrap that data and processing in all the rules possible, and still the results won't be very good.

Sure, for monitoring there are two ways to be wrong, (A) a false alarm and (B) a missed detection. So, right, we're forced into statistical hypothesis testing with (i) a _null hypothesis_ that the system is healthy, (ii) a false alarm is Type I error, (iii) a missed detection is Type II error. Quickly decide that need some statistical hypothesis tests, with the null hypothesis that the system is healthy, that are both multidimensional (treat several variables jointly) distribution-free (make no assumptions about probability distributions) and find that apparently there were none such in the literature. So, I invented a large class of such tests that, really, totally knocked the socks off expert systems for much of monitoring. And what I did was just some applied math and applied probability -- some group theory, some probability theory based on measure theory, some measure preserving as in ergodic theory, etc.

So, yes, I could say that my monitoring took in _training_ data, did some _machine learning_ , was better at its monitoring than humans so was some _artificial intelligence_ , was _computer science_ , etc., but I didn't: I just called my work some applied math. E.g., I got real hypothesis tests where false alarm rate was adjustable and known in advance and some useful best possible results on detection rate. So, as I wrote in the paper, take away the mathematical assumptions, derivations, theorems and proofs, forget about false alarm rate and detection rate, and just do the specified data manipulations and call the work AI.

So, such real problems and applied things are one approach to computer science, AI, ML, etc. Then, sure, for such work as in what I did for detection, need, really, the Ph.D. coursework in pure and applied math and some ability to do publishable theorems and proofs in math -- for both, net, need a pure/applied math Ph.D.

So, back to AI and mostly setting aside the current headlines: For AI, I'd suggest that from 50,000 feet up start by watching

[https://www.youtube.com/watch?v=vc8MddDFRw4](https://www.youtube.com/watch?v=vc8MddDFRw4)

with a mommy kitty cat (domestic short hair tabby) and 6 or so of her kittens, maybe only 2 days old, and how they learn. Their learning is astoundingly fast. E.g., for how they learn to use their hind legs, can see fairly directly some of just how they do that. In just that one video clip, in real time of likely just a week or so, those kittens go from nearly helpless balls of fur to young kitty cats. Easy to guess that in two months they will be safely 40 feet up a tree catching something or other, effortlessly doing gymnastic feats that would shame Olympic athletes, etc. Astounding.

Okay for AI from 50,000 feet up, start by trying to guess how those kittens learn. Maybe there will be some math in that, maybe not. Then if you can implement your guess in software, if the software appears from good tests in practice to learn some things well, and if your guess is fairly general, then maybe you have some progress on AI. Here f'get about my view and confirm with some AI profs who would have to review your work, hire you for a research slot, give you a research grant, etc.

So, for some research, (i) could do some math as I did for that monitoring or (ii) try to do some real AI, e.g., by starting by watching those kittens learn where maybe, eventually there will be some math.

For both those research directions, notice that need (i) some overview of the real problem, (ii) some intuitive insights, (iii) some good, new ideas. Maybe the ideas will be mathematical or use math and maybe not. E.g., might go a long way on what those kittens are doing before use much math.

(3) But for the math as mentioned by the OP, I'll try to give an outline:

(i) Start with the real numbers, e.g., as learned likely well enough by the 9th grade. Then, sure, learn about the complex numbers. So, net, have a high school major in math, e.g., everything short of calculus.

(ii) Learn college calculus well. At least in part, you can do well alone: Due to some circumstances beyond my control, for freshman calculus I just got a good book, studied, worked the exercises, and learned. Then at a college with a good math department, I started on sophomore calculus. You can do such stuff yourself.

(iii) It would be good to take a course in abstract algebra, especially one where nearly all the exercises are proofs. So, learn about sets, functions, groups, rings, fields, more about the real and complex numbers, maybe some basic number theory, the greatest common divisor and least common multiple algorithms, and some about vector spaces. Might touch on cryptography and coding theory.

Really the more important, maybe main, value of the course is just learning how to write proofs. The math there is nearly all just so childishly simple that it's easy to learn to write very highly precise proofs -- crucial stuff if later want to publish theorems and proofs.

Blunt fact of life, politically incorrect observation: Without such training in writing theorems and proofs, and, really, just in math notation and how to do math derivations, tough ever to learn how. So, can find chaired professors of computer science at top computer science departments in top US research universities who, however, fumble terribly with just standard math notation and, especially, with how to write theorems and proofs.

<comment><USER37> Part II

Super simple view: In math, there are sets with elements. That's the logical foundation of essentially all of current pure/applied math. The details are in Zermalo-Fraenkel axiomatic set theory assuming the axiom of choice. E.g., can construct from sets a set that looks like the real numbers we knew about in the 9th grade. Soon define ordered pairs and, then, functions. After that, a huge fraction of everything is functions. The proofs -- as actually written but without the crucial intuitive ideas that permitted finding the proof -- are all essentially just symbol substitution as in basic logic and Whitehead and Russell. Warning: Any math you write for publication should be easily translated back to just sets and symbol substitution; that and nothing else is the criterion. If you also want the proof to be readable by humans, there is more. E.g., in the proof might mention one by one each theorem assumption and where it gets used.

(iv) Learn linear algebra. Really the subject grew out of Gauss elimination for solving systems of linear equations -- with some additional attention to numerical stability (e.g., partial pivoting, double precision inner product accumulation, and iterative improvement) \-- that's nearly always still the way to do it. It's fun to program a good Gauss elimination routine, e.g., just in C.

There see clearly that any such system of equations has none, one, or infinitely many solutions. Later will discover that the set of all the solutions is an affine subspace, that is, a vector space plus some one vector; that is, a plane that does not pass through the origin. And will discover that the left side with the unknowns is a linear function -- big time stuff.

So, that start was a small thing for a huge future.

Next up, we consider n-tuples of real (or complex, here and always in linear algebra) numbers. Then we see how to make the n-tuples a vector space. They are the most important of the vector spaces.

But should also see the abstract (with no mention of n-tuples) definition of a vector space because (a) the n-tuples are the most important example, (b) even when working with just n-tuples often need the more abstract definition (especially for subspaces, which are often the real interest, e.g., hyperplanes in curve fitting and multivariate statistics), and (c) there are other important vector spaces that are not just n-tuples (in signal processing, sets of random variables, wave functions in quantum mechanics, solutions to some differential equations, and much more).

So, learn about linear independence and bases (essentially coordinate systems).

Learn about inner products, distance, angles, and orthogonality. See generalizations of cosines, e.g., the Schwarz inequality, and the Pythagorean theorem.

Learn about eigenvalues and eigen vectors \-- those eigen vectors are often the most important ones in applications, e.g., your favorite coordinate axes.

Then for the crown jewel, learn about the polar decomposition and, thus, singular values, principal components (e.g., data compression), the core of the normal equations in statistics, etc.

There is a remark in G. Simmons that the two pillars of mathematical analysis are linearity and continuity. The _superposition_ in physics is essentially linearity. In applied math, linearity is the main tool, the key to the land of milk and honey. Well, a good course in linear algebra is a good start on linearity. In particular, those linear equations solved with Gauss elimination are _linear_ as in linear transformations in, right, linear algebra.

Then, sure, that version of linearity takes one through much of all of applied math, e.g., Fourier analysis, the fast Fourier transform, X-ray diffraction, Banach space, Hilbert space, the classic Dunford and Schwartz, _Linear Operators_ , etc.

So, a Banach space is just a vector space where the scalars are the real or complex numbers, there is a _norm_ , that is, a definition of distance, and the space is _complete_ in that norm. _Complete_ is what the rational numbers are not but the real numbers are. Or, _complete_ means that a sequence that appears to converge, that is, converges in the weaker sense of Cauchy, actually does have something to converge to and does. E.g., in the rationals, the approximations to more and more decimal places of pi have nothing to converge to but in the reals do.

In theorem proving, nice to have completeness. But, sure, computing knows next to nothing about completeness because we compute essentially only with rational numbers. So, can do a lot of work without completeness. Indeed, in applications, often we are just approximating, and the rationals can get as close as we please to pi!

Banach spaces are not trivial or useless: E.g., based on the Hahn-Banach theorem, there is the grand applied math dessert buffet

David G. Luenberger, _Optimization by Vector Space Methods_ , John Wiley and Sons.

A Hilbert space is a Banach space where the norm comes from an inner product.

E.g., the set of all real valued random variables X such that E[X^2] is finite form a Hilbert space. That the space is complete for those random variables is nearly mind blowing; actually the proof is short.

(v) There remains _Baby Rudin, Principles of Mathematical Analysis_.

There see calculus done with essentially full care, as theorems and proofs. So, again, get lessons in how to write theorems and proofs on the way to being a good mathematician.

The main content of the book is just showing that a real valued continuous function defined on a closed interval of the reals has a Riemann integral.

The key is that that interval is _compact_. So, learn about compactness, which is of quite general usefulness.

Then with compactness and continuity, have uniform continuity. Now the doors to grandeur start to open: That the Riemann integral works is a short proof. And, later in the book, get the _three epsilon_ proof that the uniform limit of continuous functions is a continuous function (was a question on one of my Ph.D. qualifying exams -- from baby Rudin, I got it!).

Compactness is so powerful that it is nearly the same as just finiteness -- there's a famous, old paper on that.

Well for a positive integer n and the set of real number R, in R^n a set is compact if and only if it is closed and bounded. Now we are cleaning up the Riemann integral and a lot of associated stuff.

At the back, Rudin gives a nice, short definition of a set of the real numbers that has _measure zero_ (without really getting deep into measure theory) and, then, shows the Riemann integral exists if and only if the function is continuous everywhere except on a set of measure zero. Nice. Now an exercise is to find a function that is differentiable but whose derivative is not Riemann integrable. Might look in Gelbaum and Olmstead, _Counterexamples in Analysis_.

Also the later editions of baby Rudin cover the exterior algebra of differential forms. That material is of interest in differential geometry, some applications, and general relativity.

That's an overview of what baby Rudin is about.

Go through that book carefully and will come out with (a) some good knowledge of the "principles" of the analysis part of pure math and (b) much better skills at doing math derivations, definitions, theorems, and proofs. If you want to write and publish new proofs for, say, AI, baby Rudin is one of your best mentors, maybe your Fairy Godmother?

For being an applied mathematician, sure, you already guessed, and you were correct, that essentially always in practice the Riemann integral exists; so, why sweat the details? Okay, then just use baby Rudin to learn about compactness, continuity, uniform continuity, measure zero, and more on how to write proofs. And, really, focusing just on compactness, continuity, etc., can pull that off in a few, nice weekends, maybe just one. Then look at that result near the back on the uniform limit of continuous functions is continuous to see how to do such work.

Sure, Rudin discusses compactness, etc. on metric spaces. Well, easily enough, the set of real numbers R is also a metric space! And for positive integer n, so is R^n.

So, why say metric space instead of, say, just R^n? Well, first, the theory is _cleaner_ because a metric space has so many fewer assumptions than R^n so that can see more clearly just what assumptions make the results true. Second, maybe some fine day all you will have is just a metric space and, then, can still use the results -- don't hold your breath while waiting for a significant application, either pure or applied, with a non-trivial metric space that isn't also much more! Or, proving the stuff in a metric space is no more difficult, more general, and maybe, and actually, more useful.

Or, maybe math had the results in R^n and then invented a metric space just to have a place to have just enough to make the results true! So, a metric space was invented to have the least assumptions needed for proving the results; so, what came first were the results, and the metric space definition came later. Maybe!

<comment><USER37> Part III

(vi) Continuing on, there is the subject of measure theory. That was from H. Lebesgue, a student of E. Borel, right, in France near 1900.

They were correct: They improved on the Riemann integral. Don't worry: Whenever the Riemann integral exists, the integral from Lebesgue's measure theory gives the same numerical answer.

So, why a new (way of defining the) integral? Two biggie reasons:

(a) For a lot of theorem proving about integrals, e.g., differentiation under the integral sign, clean treatment of what physics needed from the Dirac delta function (right, there can be no such function, but measure theory has a good answer), definition of an integral of several variables, interchange of order of integration in iterated integrals, tying off some old, loose ends in Fourier theory, the deep connections between integration and linear operators, and more, Lebesgue's work is crucial and terrific stuff.

(b) Lebesgue's integral is much more general than the Riemann integral, and that generality is crucial, especially as the foundation for probability theory.

Now, for what Lebesgue did:

First, he developed _measure_ theory. That's essentially just a grown up theory of area like you have known about since grade school. E.g., given a set, in the real numbers, R^n, something more complicated, or something fully abstract, the _measure_ of that set is essentially just its area. With the generalization, sure, can have some sets with measure infinity, negative measure, complex valued measure, etc. But on the real line, with the usual measure, _Lebesgue_ measure, the measure of an interval is just its length, and you already know about that.

But Lebesgue measure is darned general: It's tricky to show that there is a set of the reals that is not Lebesgue measurable, and the usual proof uses the axiom of choice.

So, for measure theory, there is a _measure space_ with three things: There is a _space_ , just some non-empty set, say, M; there is a collection of _measurable_ sets, subsets of M called, say, S; and there is a measure m. Then the collection of measurable sets S satisfies the simple, essentially obvious axioms we would want for area and, thus, is a _sigma algebra_ of sets. Then for each set A in S, its _measure_ is the real (or complex) number m(A). We ask that m have the properties we want for a good theory of area. It's just a grown up version of area. The theory is not trivial; it was tricky to get all the details just right so that have a good theory of area, that is, so that area works like we want it to.

E.g., for the space, can have the set of real numbers R. For the measurable sets, have the intervals and, then, all the other sets needed to have a sigma algebra (a short proof shows that this definition is well defined). Then the Lebesgue measure of an interval is just its length, and the measure of all the other measurable sets are what they, then, have to be (need some theorems and proofs here).

So, for probability theory, a _probability_ space is just a measure space (each point in that space is one experimental _trial_ ); a _probability_ , P, is just a positive measure with maximum value 1; an _event_ A is just one of the measurable sets; and the _probability_ of A is just its measure P(A). What we want for probability is already so close to a theory of area that, really, we have little choice but just to follow what Lebesgue did. That's what A. Kolmogorov observed in his 1933 paper.

Second, with the foundation of measure theory, Lebesgue defined the Lebesgue integral.

So, what is being integrated is (usually) a function taking real or complex values. The domain of the function is a measure space.

Then for the integral, say, in the case of a real valued function, we partition on the Y axis, that is, in the range of the function instead of in its domain. So, we don't have to do Riemann-like partitions of the domain of the function and, thus, the domain can be much more general.

As the first step, we only integrate functions that are >= 0, and we do that by approximating, right, again with essentially rectangles, only from below the function, not both above and below as for Riemann. Here the domain of the function can be the whole space, e.g., the whole real line. We don't care about either continuity or compactness.

For a function that is both positive and negative, we multiply the negative part by -1 and integrate the two parts separately. If at least one of the two results is not infinity, then we subtract, and that's the integral.

A _random variable_ is just such a function, and its _expectation_ is just its integral.

Summary

Don't feel like the Lone Ranger; not everyone knows this stuff. E.g., from all I've been able to see from quantum mechanics, the wave functions are differentiable. Then I'm told that the wave functions, wondrous, are also continuous. From baby Rudin, of course they are continuous; every differentiable function is continuous! Then I'm told that the wave functions form a Hilbert space. Well, I can see that they can be points in a Hilbert space, but they can't form a Hilbert space because the continuous functions won't be complete.

In elementary probability, it is common, e.g., for finding the expectation of a Gaussian random variable, to integrate over the whole real line. Tilt: Baby Rudin defines the Riemann integral only over compact sets. So, can use an improper integral, and then they want to differentiate under the integral sign. Tilt again -- the needed theorems are not so easy to find. So, really, this common stuff, in probability, physics, etc. of integrating over the whole real line or over all of R^n is using the Lebesgue theory where have clean theorems for such things.

That's your overview and your work for the weekend. You are now permitted two beers and half a pizza but only if you have someone you like a lot for the rest of the pizza and some more beer. Can substitute Chianti for the beer. But no more math permitted; maybe a movie, but not math!

</comment></comment></comment><comment><USER38> Perhaps you can focus on improving the usability of AI tool-sets to a wider market rather than focus on finding the Next Big Magic Equation. Example: [https://github.com/RowColz/AI](https://github.com/RowColz/AI) An AI expert(s) may do the initial setup, but factor tables allow more "typical" office workers to tune and prune the results.

</comment><comment><USER39> I like the comment that likens this sort of deeply-linked knowledge to a DAG. In my own (limited) experience, once I've mentally found the DAG where every node either references some other node or baseline knowledge, the learning task almost immediately switches from daunting to routine. Just work on understanding each node in the dependency chain until you get to the one you seek!

</comment><comment><USER40> From TillWinter's response: > Also: doing the master is to understand that you don't know anything, and doing your doctorate is to learn the others know nothing as well.

I had always heard variations on the first part -- that going to a good school was supposed to humble you by showing you how much you don't actually know.

Never heard the second part. That's great.

</comment><comment><USER41> I am an AI researcher and faculty member at a large and famous university. I probably know less math than that Reddit poster. Math is important if you are specifically interested in the math of AI. If you are interested in inventing algorithms and solutions you mostly don't need the math.

</comment><comment><USER3> Starting from pre-calculus what areas of mathematics (with book recommendations) should one study rigorously to have the foundations to pursue a PhD in Machine Learning?

</comment><comment><USER42> Remember:

"Complex models are rarely useful (unless for those writing their dissertations)." (V.I.Arnold)

</comment><comment><USER43> Perhaps the author should write a ML tool to help sift through all the material ;)

</comment><comment><USER44> I think he should just go on with his research and don't bother with understanding every obscure reference in papers. One can grasp the core ideas surprisingly well even when skipping over proofs and formulas.

And when the time comes to write his own papers, he should remember to intentionally make it harder to read for outsiders. E.g. instead of writing "I calculated the total error by summing the per-neuron errors", one should write "the loss function utilized an integral over the output lattice using a discretized method by Newton et. al.", or some other bullshit.

<comment><USER45> As an amateur who has jumped in and out of learning basic ML over the years, it has been interesting to see the web of terminology expand to the point where your post is no longer satire. Writing a dictionary or annotating papers to decipher ML-speak to basic-math speak would be a pretty worthwhile endeavor for someone (I see glimmers of this in the work being done by the folks at fast.ai) and in general would probably not remove much real information.

</comment></comment><comment><USER46> Most-upvoted reply is excellent.

<comment><USER47> you should link to the reply instead, because the most upvoted reply might change.

<comment><USER48> [https://www.reddit.com/r/MachineLearning/comments/73n9pm/d_c...](https://www.reddit.com/r/MachineLearning/comments/73n9pm/d_confession_as_an_ai_researcher_seeking_advice/)

</comment></comment></comment><comment><USER49> The depth and illegibility of the field he is describing make me believe that we are much further away from general purpose ai than I previously thought</comment>


Pomera: Pocket Typewriter with E Ink - <AUTHOR> https://www.kickstarter.com/projects/2132003782/pomera-pocket-typewriter-with-e-ink <comment><USER2> Cool idea, but pretty expensive. Might be worth the price if you could run linux on it, but I suppose that would drain the battery pretty quickly; I'm not sure how well the Linux TTY is optimized for e-ink. Either way, only being able to type on Qwerty and Japanese Kana input is pretty limiting considering that the main feature is being able to type stuff.

</comment><comment><USER3> Too bad it doesn't seem to have any kind of encryption option.</comment>


How to split large Python Functions across multiple files - <AUTHOR> https://www.openfaas.com/blog/multifile-python-functions/ <comment><USER2> Was this article really just pointing out that functions can be put into submodules, imported into other modules, and used by other fucntions?

I was hoping for something about, say how to decompose a 10,000 line function with cross-cutting concerns into multiple files.

BTW, I felt unease with:

      STOPWORDS = set(
          map(str.lower,
              map(str.strip,
                  open(os.path.join(FILE, 'stopwords'), 'r').readlines()
                  )
              )
      )

I feel better with:

      STOPWORDS = set(line.strip().lower() for line in
                         open(os.path.join(FILE, 'stopwords'))

While there aren't enough stopwords that this will be a performance issue, according to my timings using /usr/share/dict/words on a Mac for 100 reads, it may have made performance sense for Python 2.7 to use this approach:

        == Python 2.7 ==
      map 20.7530391216
      me  24.4680659771
      ratio: 1.17901122017

but under Python 3.7 the more concise (and I argue more readable) way is faster:

        == Python 3.7 ==
      map 19.93948793411255
      me  19.127523183822632
      ratio: 0.9592785555490216

</comment><comment><USER3> The title should be changed to indicate that this is "for OpenFaaS services"; otherwise it seems kind of trivial...</comment>


Show HN: Crossroad – the easiest way to (really) gather your friends' pictures - <AUTHOR> http://crossroad.io

<comment><AUTHOR> Creator here :)

Why Crossroad ? All our friends take pictures during shared event, but we never manage to get the pictures taken by the others once back home…We downloaded other apps aiming to do it…not working for friends and us. That's why we created Crossroad which allows you to share 1, 10, 50 or even 100 pictures with your friends in just one tap and in background (no need to keep your iPhone awake during the pictures uploading / downloading process).

Hopefully it will help other people to finally see the pictures they never had the chance to see until now.

We would love to have your feedbacks on the UX / UI.</comment>


HPE unveils The Machine, a single-memory computer with 160 terabytes of memory - <AUTHOR> https://venturebeat.com/2017/05/16/hp-enterprise-unveils-single-memory-160-terabyte-computer-the-machine/ <comment><USER2> I am not sure if I should be impressed until they reveal more details.

160 TB over 40 nodes is 4 TB per node, which assuming 512 GB dimms requires only 8 dimms (4 per socket) or with 256 GB dimms 16 dimms (8 per socket).

How fast is the interconnect? 100Gb/s?

So far it sounds like an common high memory HPC cluster unit with really unknown technical parameters.

<comment><USER3> Isn't the idea that a single process can address all 160TB of memory as if it were local?

Of course you can cobble together that much memory when you're programming a distributed system of communicating processes. The interesting part would be programming it as if for one computer.

<comment><USER2> As far I understand, this memory would be not local to any of the processors - all of them access it through shared bus (fabric as they call it) and it appears to be handled rather as a weird (their words) disk device than as a memory.

I did not find information about the bandwidth or latency of such architecture.

</comment></comment></comment><comment><USER4> Wasn't the Machine supposed to be a memristor computer? Did they ditch the project?

<comment><USER5> AFAIK there was licencing issues with memristors - in usual forward thinking HP way, they exclusively licenced the tech to someone else, who did no work on it, so it was not ready for the machine.

The IP is also a joint asset of HPE and HP Inc, so that sounds like something I would avoid baking into my product if I was HPE

</comment></comment><comment><USER6> Neat press release. Bombastic, eye-catching, has people asking what it is.

But the product is just a high memory HPC cluster unit, 4TB per node. It's not "revolutionary", it won't "change everything", it's not a "technological breakthrough".

 _TL;DR:_ _It 's press-bait._

</comment><comment><USER7> > The new prototype has 160 TB of shared memory spread across 40 physical > nodes, interconnected using a high-performance fabric protocol.

So basically something similar to RDMA scaled to 40 nodes, 4 TB RAM per node.

<comment><USER8> the article is pretty weak, people have built rdma machines that size before, and there have been architectures that allow for direct memory addressability of that size before.

so i have to assume the latter. that seems to be borne out by the little information i can find...addressable persistent memory is clearly a theme. but i haven't found any discussion of what kind of latency hiding mechanisms might be at play and what kind of consistency model is being used. will keep looking for anything detailed an authoritative

(edit - this seems to be pretty relevant [https://www.labs.hpe.com/publications...but](https://www.labs.hpe.com/publications...but) i don't know how much of it was speculative and how much was built...after all the security papers, there are some about concurrency control)

<comment><USER9> The Next Platform has had many articles with bits of detail on HPs Machine. Unfortunately, they don't really have any tagging mechanism, but you can search the phrase "the machine" or look at related articles there.

[https://www.nextplatform.com/2017/01/09/hpe-powers- machine-a...](https://www.nextplatform.com/2017/01/09/hpe-powers-machine- architecture/)

<comment><USER10> Great resource. Based on that, it looks like the fabric has a bandwidth of 600Gb/s (or 1.2Tb/s full duplex).

The interconnect seems to be the real innovation here.

</comment></comment></comment></comment><comment><USER11> The Machine should be fine, but if Samaritan comes online, we are doomed!

</comment><comment><USER12> How does the Machine recover from software errors? If it has only persistent RAM, then when some important program goes bonkers (it always goes), there is no luxury of pressing the reset button to get back to a pristine state.

<comment><USER13> My guess is that there would be a microkernel service that is the core OS, like the BIOS, and it could be instructed to actively wipe the memory and reload the boot files when the machine freezes

<comment><USER12> There can be bug in a microkernel service (and as it is distributed, it is actually guaranteed it will crash unexpectedly at some point)

<comment><USER13> Well I guess it's a disposable computer then. Use until it crashes.

</comment></comment></comment></comment><comment><USER14> This falls into a scale up type (vertical) architecture, in what scenarios would you need tin like this? Versus A more scale out/distributed (horizontal) architecture

<comment><USER15> It's a 40 node mesh. Isn't that scale-out?

<comment><USER14> Let me rephrase rather. a scale out architecture with commodity machines (using Hadoop, kube, mesos etc.) versus an engineered system in hardware. A client purchased a an HP super dome (another highly engineered system) the cost per CPU core, gb ram, tb of storage was horrendous. More than 10x comparable commodity hardware solutions. 8 months later and they hit performance problems, space issues and adding capacity is a serious chunk of change.

What I am trying to understand in what use cases would this make sense?

<comment><USER15> This is a scale-out solution, it's just higher performance / lower effort than building your own solution from commodity harwdare. Enterprises often want to buy canned, supported solutions rather than expend the R&D to build their own.

What workload does this configuration make sense for? I don't know, especially with those ARM cores. My guess would be the cluster performs well on workloads similar to Apache Spark. But I don't know why you would choose one over the other.

</comment></comment></comment></comment><comment><USER16> Interesting to see ARM starting to become datacenter worthy now. It's cheap and there is a version geared towards any application you can think of. Now let's see HPE use some FPGAs with embedded ARM CPUs. That would really be the killer for all the smaller shops that are building similar exabyte systems.

<comment><USER17> A lot of it is the big customers can go to a chip company and say "I want this SOC, with X, Y, and Z" \- and if they want enough of them, will actually be able to get them at a reasonable price.

I read this is why Qualcomm did their data center chip - some of the big Chinese companies wanted it.

</comment></comment><comment><USER18> Nice for Redis.

</comment><comment><USER19> This is funny, because in 100 years, computers with 100 yottabytes of memory will be standard, and they'd be laughing at a headline like this one.

<comment><USER20> Or they will let out a nostalgic sigh as they think about those simpler days where you could get useful work done on 4 GB of RAM... ;-)

</comment></comment><comment><USER21> "and also allows accelerators to get direct access to a massive memory-storage footprint" \-- ehh?

</comment><comment><USER22> I'm sure those SGIers now at HPE will be delighted to see so much being made of the UV.

Oh.

</comment><comment><USER23> Not enough.

Because, you know, there happens to be something else that is driven by memory. I'll give you a hint as to what it is: it's comprised of approximately a hundred billion units. (And no, it's not a galaxy.)

Divide 160 terabytes by the number I just stated, and you'll find that that you only get 1.6 kilobytes per unit. Not really enough (or at least, too close for comfort).

Step up your game, HP! :) But, this is a VERY good start.

-

Edit: I got downvoted to -4 but I have a right to state my requirements. If your requirements aren't that high, use whatever equipment you want. (the guesses below are correct, if you want "the answer to the puzzle.")

<comment><USER24> Maybe just say what you mean, instead of riddles, as part of contributing to the discussion?

<comment><USER25> ~100 billions neurons in a human brain I guess. Not sure what 1.6KB per neuron is too close to

<comment><USER23> This comment currently has a reply ( [http://i.imgur.com/N7BZJfu.png](http://i.imgur.com/N7BZJfu.png) ) which shows why I wanted to leave my original post as a "riddle" rather than coming out and saying it.

The following "crap" is not relevant to anything, only read it if you want the answer to why 1.6 KB is insufficient for a certain science fiction fantasy which will never, not in ten, a hundred, or a thousand years, become relevant any more than magicians will start flying around on brooms. It has nothing to do with anything. Don't get me wrong and think that I think it has relevance to something.

Original version of this comment:

\--

Well, it's clear that you could not come even close to perfectly describing the state of a cell and all its connections in 1.6 KB. As just one example, every cell has a different genome[1] which if you encoded it in full would be 700 MB right off the bat. I'm not saying the fact that every neuron has a different genome is _relevant_ to its computational functions, but 1.6 KB is cutting it very close. Let's explore why this is so.

Suppose we simplified and said that each neuron may be connected to n other neurons (we will calculate what n fits in our memory). Addressing 100 billion takes 36 bits. (You need approx. 36 bits to name a number between 1 and 100 billion, i.e. if all you're doing is naming which other neuron it's connected to.). So let's see how many "synapses" the 1.6 KB might be enough to address. -- we are looking at how many connections could be encoded into 1.6 KB.

So 1.6 KB if you did nothing else but as a simplification say that you were referring to another neuron, then 1.6 KB = 1.6 * 1024 * 8 = 13107 bits, divide that by 36 and you get 364 addresses. Suppose you were to then encode each of those addresses with a single byte (an 8-bit brain) representing the strength of that connection, and you are down to only encoding 13107/44 = 297 neural connections with an 8-bit connection value. [EDIT!! As pointed out in a reply, original version of this paragraph contained a math error - I originally divided by 8 after dividing by 36 -- as though each bit of the address needed a connection strength, rather than adding 8 to 36 to get the number of bits needed for each connection and only afterward dividing that into the 1.6 KB]

Okay, so let's see how "297 neural connections" stacks up.

In fact "Each neuron may be connected to up to 10,000 other neurons", but "Each of the neurons has on average 7,000 synaptic connections to other neurons." This means we do not have 8 bits - we do not have even a single bit. (Because 7,000 is more than 297 or even 364).

So you cannot even store the full _address_ of each synaptic connection (the full address of the other neuron it's connected to) let alone a value of the strength of it, or anything else computationally interesting that may be going on chemically. Of course, obviously one neuron will not be directly connected with another far away, for example on the other hemisphere of the brain (meaning you can almost certainly shave nearly 1 bit of addressing off of the 36 bits from the get-go, and likely you need a far smaller address space, if you really look at it.)

But as you can see, however you slice it, you're just cutting it extremely slow. If synaptic connections had even 10 bits of value of some kind than you're way over your budget here.

But as you can see we're close - the numbers just barely don't work.

HP are on the right track - they simply need to step their game up.

[1] recently in the news - [https://www.scientificamerican.com/article/scientists- surpri...](https://www.scientificamerican.com/article/scientists-surprised-to- find-no-two-neurons-are-genetically-alike/)

EDIT* - thanks for the correction, yes, there was a math error.

<comment><USER26> You made a math error:

You can encode about 300 addresses, at 36+8 = 44 bits per item, not 45 addresses, at 36*8 = 288 bits per address.

The bits are additive, because we're storing an address and data, not a byte of data per bit of address.

<comment><USER23> thanks, corrected.</comment></comment></comment></comment></comment></comment>