The problem with self-driving cars: who controls the code? - <AUTHOR> http://www.theguardian.com/technology/2015/dec/23/the-problem-with-self-driving-cars-who-controls-the-code <comment><USER2> Not sure I follow, how would these scenarios be any different if a human was making the decision? Some humans will act with self interest, some with a utilitarianism, and some just simply will freeze. I'm not so sure it would make much a difference how the car chooses.

So, philosophical question aside, the concern for autonomous vehicle fleets having backdoor and being abused is more concerning..

<comment><USER3> Not sure you got beyond the headline. Doctorow rejects the philosophical question; any car that can be programmed to kill it's driver, a driver can reprogram.

<comment><USER2> Which makes the point: it is indistinguishable from what happens now..

</comment></comment></comment><comment><USER4> I love the tee-up: "Should autonomous vehicles be programmed to choose who they kill when they crash?!"

...because the answer provided is approximately, "that's a stupid question."

It's like guardian didn't even read Doctorow's editorial before publishing.

<comment><USER5> I think it's more pernicious than that, they aren't ignorant of the content when they write headlines like that, they just don't care about the mismatch.

</comment></comment><comment><USER6> The main assumption made by Cory Doctorow is that the autonomous cars will be fully automated and under the control of somebody other than the driver. That scenario - with a Uber-like owner of a point-to-point fleet of self-driving cars - is indeed a bit concerning, considering that you, the passenger, give up control of the vehicle and you have to trust the machine with your life in order to get you from point A to point B.

But this is exactly why I think that the consumer version of the self-driving car will always have a manual drive setting that will completely disengage the auto-pilot.

The insurance companies will never insure at an acceptable cost point a car that cannot yield control to its passenger that is the designated driver.

Also, passenger will want the reassurance that at any point they can take over the car and switch it off the auto pilot.

iPhones and any other mobile devices are a bad analogy with a self-driving car. The more apt comparison is with the current modern airplanes: even if these planes can drive themselves, the pilot is able at any time to take over.

This option is needed even more on the roads, where you don't have the dedicated flight path a plane has in the air.

<comment> irq-1 > The insurance companies will never insure at an acceptable cost point a car > that cannot yield control to its passenger that is the designated driver.

We insure medical devices, satellites, soldiers lives and many other more complicated and important things. The insurance industry will figure out how to collect payment for automated cars as well.

> Also, passenger will want the reassurance that at any point they can take > over the car and switch it off the auto pilot.

Bus, train, plane, escalator, elevator, moving walkway and wider analogies say otherwise.

</comment></comment><comment><USER7> Aren't these two separate questions?

Should autonomous vehicles be programmed to kill their owners in some circumstances?

and

Should owners of autonomous vehicles be legally allowed to change the vehicle's code?

Both have their own separate legal and moral quagmires.

<comment><USER8> Exactly. There's an awkward, obvious shift in the article where it turns an age-old riddle that goes back way farther than 1967 into a rant that covers everything from DRM and jailbreaks and chain of trust to "morally ambiguous" and "exploitative overseas labour arrangements." It's not a tech article, it's a manifesto.

Write the same article over and over for 15 years at a Zine and I guess they'll eventually let you spew it at The Guardian provided you refer to the subway as "tube" and spell labor with a "u."

These are important issues that deserve far better presentation. Sourcing quotes from people actually doing the work and manufacturing the vehicles. Getting statements of policy from regulators on the record. Strengthening the ethical argument by having someone besides the author say it.

</comment></comment><comment><USER9> Can someone explain to me how Cory Doctorow became a spokesperson for privacy and technology policy?

<comment><USER10> Basically because of the EFF.

He worked full time at the EFF 2002-2006 and was named an EFF Fellow when he left. That combined with his blogging and fiction writing left him a reasonably high profile privacy and technology spokesperson.

<comment><USER9> That's all I could find. I could find nothing in his background or what he does now that would indicate he had any direct knowledge about what he gives an opinion on.

<comment><USER11> He's a writer with experience and deep interest in tech issues. What kind of credentials he needs to have for his opinion on those issues to be "valid"?

<comment><USER9> > writer with experience

Ok, what experience? That's what I'm asking. A deep interest is not experience.

> What kind of credentials he needs to have for his opinion on those issues to > be "valid"?

Usually actual experience in the matter. The EFF does good work because it is an organization of lawyers, not Science Fiction writers.

<comment><USER11> Have you read other articles by him, or seen his talk? Like this one I personally consider very insightful and important: [http://boingboing.net/2012/08/23/civilwar.html](http://boingboing.net/2012/08/23/civilwar.html).

But since you insist - he has a honorary doctorate in computer science from one of the top UK universities[0]. He also co-founded a software company[1]. I think it's enough credentials to give him right to be a tech advocate, with the fact that he's also a journalist backing up the "advocate" part.

[0] - [http://craphound.com/bio/](http://craphound.com/bio/)

[1] - [https://en.wikipedia.org/wiki/Opencola_%28company%29](https://en.wikipedia.org/wiki/Opencola_%28company%29)

</comment></comment></comment></comment></comment></comment><comment><USER12> Cory is of course completely and utterly wrong here on a technical level. Indeed, what he's saying is the exact opposite of the truth. Locks that prevent you from modifying the code running on your hardware don't make it easier for an attacker to break in, it's the exact opposite - any feature that lets you install modified code can also be used by an attacker to install malicious code that will steer your car into a bridge on command. This is why some experts recommend the use of locked-down devices like iPhones if your hardware might be targeted by an attcaker.

Likewise, there is no requirement that "A digital lock creates a zone in a computer's programmer that even its owner can't enter. For it to work, the lock's associated files must be invisible to the owner." You need that for DRM because the purpose of DRM is to protect access to files, but a properly- locked-down system will not let you modify the code even if you can look at every single piece of code and data on the system.

<comment><USER13> Where did he say that locks make it easier?

> but a properly-locked-down system will not let you modify the code even if > you can look at every single piece of code and data on the system

I think in the quote you used Doctorow is simplifying things for a general audience. Please give an example of a "properly-locked-down system" that can be protected from someone with physical possession of it. It's not the iPhone, as he said iPhones can be "jailbroken," (I don't know if I would call it "easy" as he does).

</comment></comment><comment><USER14> Another reason why self-driving cars should have manual controls:

There is a road near my house that is closed by the neighboring school at random. Google treats this road as a normal road when choosing routes via Maps.

If a self-driving car drives up to the gate, that's at least a lot of wasted time, and if the car can't detect the chain-link gate...

</comment><comment><USER15> What I wonder about most is how we are going to do the certification of the code. Given that the code is probably far too difficult to analyze manually, I assume that certification will mostly be a matter of letting the car drive for X miles and seeing if it doesn't crash.

The problem that arises then is that after every tiny update, do we have to go through the complete certification process again? It is not possible to simply record and replay sensor information, since every change in behavior may result in a change in the environment, and you cannot predict the environment.

<comment><USER16> There is a great deal of academic research on formal methods of code validation, most of which is completely ignored by industry.

<comment><USER17> The problem with code validation in the context of a self driving care is that the space of inputs to the software is essentially the entire world. The code is _necessarily_ complex and the invariants nearly impossible to adequately specify. A general statement like "Does not collide with deer in road" is in fact insufficient.

This isn't like an airplane autopilot or your typical other sorts of highly vetted software. It's many orders of magnitude more complex.

<comment><USER11> And that's why I think questions like "whether the car will be programmed to kill you or someone else" are missing the point. The car _won 't have an explicit decision point like that_! Evaluating pros and cons is what humans do because it's hard for us to do any better. A car can be continuously observing, updating and navigating the solution space, and if it ends up killing someone, it's either because of not having sufficient data, or not being able to stay within its envelope on the solution space (i.e. a situation when it's physically impossible for it to avoid a collision). I don't see why a car would ever have an explicit if(something()) { killDriver(); } else { killTarget(); }. "Killing" won't even be a concept in code.

</comment></comment></comment></comment><comment><USER16> What I see as inevitable is a transition to software engineering being a licensed profession. And additionally the application of formal methods of system validation.

</comment><comment><USER18> Title changed from "Cory Doctorow on Software Security and the Internet of Things". Submitters: please use the original title unless it is misleading or linkbait, as the site guidelines ask.

[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)</comment>


Show HN: A less-than 100 SLOC PHP Framework - <AUTHOR> http://roevhat.github.io/AsterixVC/

<comment><USER2> I like this. Thank you.

<comment><AUTHOR> You are welcome :-)</comment></comment>


Amazon and Google Discover Erlang (IMDB is switching from Perl to Erlang) - <AUTHOR> http://perfdynamics.blogspot.com/2008/07/amazon-and-google-discover-erlang.html

<comment><USER2> > Somewhat ironically, Erlang was released by Ericsson as open source in 1998 > because an internal executive decision was made to go with "standards" > instead.

Nice technical trivial pursuit factoid

</comment><comment><USER3> I'm not so sure if SDB is implemented in Erlang.

Check out [http://www.allthingsdistributed.com/2007/10/amazons_dynamo.h...](http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html)

It says:

"In Dynamo, each storage node has three main software components: request coordination, membership and failure detection, and a local persistence engine. All these components are implemented in Java."

"Engines that are in use are Berkeley Database (BDB) Transactional Data Store, BDB Java Edition, MySQL, and an in-memory buffer with persistent backing store."

"All communications are implemented using Java NIO channels. "

I doubt they wrote Dynamo in Java and then converted the whole thing to Erlang.

Also, Erlang does not fit in the 'eventual consistency' and 'failure is an option' model that Amazon like to promote.

<comment><USER4> Amazon has stated that simpledb does not run on dynamo though so this is all moot.

</comment></comment><comment><USER5> Link to original blogpost on this subject:

[http://vertonghen.wordpress.com/2008/07/05/erlang-or- utility...](http://vertonghen.wordpress.com/2008/07/05/erlang-or-utility- computing-vs-appliance-computing/)

</comment><comment><USER6> According to the post the extent of Google discovering erlang is erlang style concurrency in Google Gears. To me erlang the language and erlang style concurrency are two very different things.

<comment><USER7> Looks like blatant misinformation to me. As to sources:

* Amazon SimpleDB on Erlang (friend of a developer): <http://www.satine.org/archives/2007/12/13/amazon-simpledb/>

* IMDb on Java/Erlang (a job posting): <http://www.computerjobs.com/job_display.aspx?jobid=2052172>

* Gears on Erlang: N/A

<comment><USER8> I can confirm that SimpleDB is Erlang (with Berkeley DB as the persistence engine), or at least I can say that one of the lead developers of SimpleDB has told me that this is the case.

</comment></comment></comment><comment><USER9> Is there any other sourcing about IMDB moving to Erlang? This is the first (and last) that I've heard that.</comment>


How the Dropbox Datastore API Handles Conflicts – Part Two - <AUTHOR> https://www.dropbox.com/developers/blog/56/how-the-dropbox-datastore-api-handles-conflicts-part-two-resolving-collisions <comment><USER2> The problem with this approach is that it requires you to resolve conflicts when you first see them. So you can't do workflows that accumulate more than two versions of what happened. Nor can you resolve the conflict asynchronously.

For an approach that doesn't ever force you to throw away or merge data, see the data structures in my OSCON talk. [https://speakerdeck.com/jchris/couchbase-lite- oscon](https://speakerdeck.com/jchris/couchbase-lite-oscon)

I owe the world a write-up explaining these slides. Or come see me talk at StrangeLoop in St. Louis or Couchbase SF in September. [http://www.couchbase.com/couchbase- sf-2013](http://www.couchbase.com/couchbase-sf-2013)

<comment><USER3> I could interprete it as a multiple-branch model with Git. But it's still rather confusing for me what use it would have for anybody outside of source code repos (meaning: I don't have the knowledge, not that there wouldn't be a usecase).

For some end users it's already confusing to have different states in the same linear history on different devices because of upload times and devices being offline. I think giving most users multi-branch capabilities would overpower them and they might just ignore all branches but one, manually merging parts of older branches into the newer ones.

So I guess you assume usecases for application writers? I'm also not deep enough into this topic that I could imagine any usecases. Would you mind giving some examples?

<comment><USER2> As a database, the #1 rule is don't lose data. So a model that allows you to defer conflict merges, and to do the merge on any peer, keeps the database doing what it does best, and gives maximum flexibility to the application.

For instance on a customer record you might have three sales folks all qualify a lead as a 4, one adds a fax number, one adds an email address, and one adds the same fax but a different email, and a note about the best time to call. If the database forced the application to merge on encountering each update, there'd never be a chance to set all three versions side by side and make an informed comparison. Instead it would be luck that A and B merge, and then the result merges with C. Or some other order.

Where it gets really useful is if you have different actors that have ownership of the merge on different fields. So for instance you could partially resolve the conflict by merging some fields automatically but present a human with the full set of options for a bigger octopus merge on, say, the notes field.

<comment><USER4> If a client app is processing the cust record, how would cust.email look like ? Typically I would assume a single value, but in case of unresolved conflicts will it be a iterator/array ?

<comment><USER2> The client will have N copies of the customer record, so the record schema doesn't change. Instead all the potentially valid versions of it are preserved until a peer (any client or server) marks the conflict as resolved.

</comment></comment></comment></comment></comment><comment><USER4> Trying to wrap my head around this. Seems difficult without clear usecases.

Let's say I have 10 devices d1,d2....d10 making updates to "a" on the server and went offline. a==20 and last update was by d5 before everyone went offline.

When the devices come back up, the fate of "a" depends on the rulesets. Following are 3 possible high-level combinations.

i) All devices have "remote" rule. On reconnection, everyone rollback "a" to 20. They are essentially back to the time before going offline. Even the device which did the last update(d5) before going offline is rolled back too, which seems bit odd. Still simple to reason with..

ii) All devices have "local" rule. On reconnection, the last device to reconnect updates "a". It is then broadcasted to all other devices. Note that it is _not_ the last device to update "a". Rather it is the last to reconnect (Now, even if all of them reconnect at same time, depending on the queueing at server, the one at the tail wins). Not really simple..

iii) Mix of "remote" and "local" Let's say d1 had "local" rule and all others had "remote". On reconnection, d1's "a" will be propagated to everyone. This is irrespective of the order of reconnection (I am assuming that between reconnections "a" is not modified). This is pretty simple and perfectly predictable. Now, if we have more than one "local", we start getting non- deterministic, and at the extreme move to case ii)

<comment><USER5> Note that every device submits its change with an expected "parent revision." The server checks the change against the server's current revision, and the change is accepted if and only if the server revision matches the parent revision of the submitted change.

So when devices d1 through d10 make a (simultaneous, I assume) change, they all submit their change with the same parent revision. Assuming they were up- to-date before they submitted that change, exactly one of the device's changes will succeed (whichever reaches the server first). For example, if the previous revision was 100, they'll all submit changes with the parent revision 100, and the first one to reach the server will succeed, at which point the server revision will be increased to 101. When the other changes come in, they'll all fail because their parent revision doesn't match the server.

I have to slightly revise your scenario and say all these devices went offline and then made a local change. This may be what you meant, but I want to clarify that the changes were queued up locally but not yet sent to the server.

So in (i), where the conflict resolution strategy is "remote," what will happen is that one device's change will win (whichever reaches the server first), and all other devices will throw out their change in favor of the change that made it to the server. It's not the case that _everybody 's_ changes are rolled back.

In (ii), the first device to connect submits its change and is accepted by the server. Subsequent devices submit their change (with parent revision of 100), see that they're out of date (server revision is now 101 or higher), and resubmit their change with the new parent revision, effectively clobbering any changes that have been made on the server. So each device in turn clobbers the value on the server, and the ultimate value is whichever change was submitted last.

For (iii), really don't do that. Having different devices use different conflict resolution strategies is a bad idea, and I can't really think of a valid scenario fro that. Can you?

So to sum up, (i) would mean "first change to reach the server wins," and (ii) would mean "last change to reach the server wins."

<comment><USER4> Thanks for the comprehensive clarification. Yes you are right that the use of revision-id would allow "first change to reach the server wins" for case (i). It would result in a simple and fair outcome. I did not see the mention of revision usage in the parent link. But it makes absolute sense.

Think we agree on case ii). On case iii) I still think that having 1 local and N-1 remote might be useful when we want to prioritize a particular writer over others. Borrowing the sales example from commenter jchrisa, consider a new user sales-head (local rule). He syncs the data uploaded by his sales folks (remote rule) and then goes offline. When he is done editing and comes online, he wants to make sure his delta takes precedence irrespective of any previous changes by sales folks during his being offline. Since he has local rule, his update will just win. Further, he does not want sales guys who were offline and come online after him to overwrite his last update immediately. I am assuming that the sales folks with "remote" rule will see the data with newer server version and accept it.

<comment><USER5> I suppose your scenario for (iii) is reasonable in the case of sharing a datastore among multiple users. In that case, you'd probably still want all of _that user 's_ devices to use the same conflict resolution strategy.

BTW, (for now) the Datastore API doesn't support sharing among multiple users.

Given the relationship the Datastore API has between the client and the server, it's also possible for a client to use a custom resolution rule (though this isn't yet exposed in any of our clients), which would let you do smarter things like look at the changes that were made and decide based on app-specific logic which one should win. I personally think that's a big advantage, but as I said, it's not one that's yet been fully realized.

</comment></comment></comment></comment><comment><USER6> Awesome, this is one of the best explanations of version conflict resolution in distributed storage.

The best engineers not only needs to write great code, they need to write the best documentations too :)

</comment><comment><USER7> I might not understand what the Datastore API is for, but if I wanted to make a simple app that could sync to Word Docs between to clients, would these .doc files get represented in this database structure (tables, records and values)? Is that data structure just for example purposes?

How would this algorithm handle changes to the same text file? Is the file a record, and each row considered a value in that record?

EDIT: Looks like this API is only for "structured data like contacts, to-do items, and game state." [https://www.dropbox.com/developers/datastore](https://www.dropbox.com/developers/datastore)

<comment><USER5> Your edit is correct. To sync files, you would use the Core API or Sync API.

([https://www.dropbox.com/developers/core](https://www.dropbox.com/developers/core) and [https://www.dropbox.com/developers/sync](https://www.dropbox.com/developers/sync))

<comment><USER7> I'd be interested to hear how conflicts are handled with those APIs.

<comment><USER5> The API uses (optional) optimistic concurrency. So when you write a file, you can specify the expected parent revision, and if the revisions don't match, the operation will fail. If you want, you can have Dropbox instead just overwrite whatever file is present on the server or write the new file but rename it in the process (e.g. "myfile (1).txt").

This is pretty much all that's possible to do generally for files, since their structure is opaque. One of the ideas behind the Datastore API is to support developers structuring their data in such a way that changes can be merged automatically.

</comment></comment></comment></comment><comment><USER8> Why does chrome prompt me for keychain access on this page?

I'm not logged in, and it doesn't log me in.

<comment><USER9> I've heard around (I don't use Chrome personally) that Chrome just prompts for keychain access until you give up and grant it access all the time.

<comment><USER8> Yes, but even then it only needs to prompt when it wants information from the keychain to fill in forms and such. The question is why would this blog page trigger such an event?

<comment><USER10> There is a sign in dropdown form in the upper right.</comment></comment></comment></comment>


How the Carmakers Trumped Themselves - brian-armstrong https://www.theatlantic.com/science/archive/2018/06/how-the-carmakers-trumped-themselves/562400/?single_page=true <comment><AUTHOR> This is an interesting problem. I believe that the migration of demand from sedans to mini-SUVs is largely the result of decades of marketing, which itself was an attempt to increase profit margins by creating a new class of vehicles (SUVs) to escape CAFE regulations. In the late 1990s (when the first SUVs hit the mainstream market), advertisements were screaming from every speaker about SUVs, and the market responded.

The Volvo V70 XC/XC70 is the Volvo V70 with a taller suspension to escape CAFE standards and increase profits. The only difference is height. The V70 had to be special ordered for the US market.

People also like to be able to see from a higher vantage point while driving, another advantage of a taller car. This problem has compounded itself; since more people buy SUVs, you need to be progressively taller to see around them.

When you look at an SUV, especially a mini-SUV like the Honda HRV or the Mazda Cx-5, realize what you're actually seeing is a modern station wagon that's been raised a couple inches. The modern consumer wants a wagon that's tall, because wagons are the ultimate in utility per dollar, and being tall is reassuring.

There are ways to deter such thinking. Imagine a tax on all non-commercial vehicles with hoods more than, say, 3.5' from the ground where they meet the windshield, or taller than 5.5' over the driver seat. If that tax were, say, 10-15% of the cost of the vehicle, the market would eventually adjust to wagons rather than tall wagons. The road would be safer due to better visibility for all, and the air would be cleaner due to better aerodynamics.

P.S. If you'd like an interesting exercise in the power of marketing, go to a dealer and look under the hood of a new full-sized truck. Note how much space there is between the top of the engine and the hood when it's closed. Then imagine the fuel economy gains if the hood was lower, and the visibility gains of both the driver and all the cars around the truck. The tall hood only exists to make the truck look more butch, and makes the truck worse in every way.

<comment><USER2> >If you'd like an interesting exercise in the power of marketing, go to a dealer and look under the hood of a new full-sized truck. Note how much space there is between the top of the engine and the hood when it's closed. Then imagine the fuel economy gains if the hood was lower, and the visibility gains of both the driver and all the cars around the truck. The tall hood only exists to make the truck look more butch, and makes the truck worse in every way.

That's actually because of the increasing safety standards regarding pedestrian safety more than anything; regulations require a minium amount of space between the hood and the engine to act as a crumble zone. And automakers aren't exactly a fan of this either, it screws with both aerodynamics and aesthetics.

Unfortunately safety and eco standards are at odds with each other. You can either have a light, small car that crumples into a pointy tin can, or a reinforced behemoth with more air bags than an air mattress depot that gets 10 mpg. And at least with the former you can also get a fun car to drive, the latter is just an ugly barge with blindspots to match

</comment></comment><comment><USER3> You could solve all these problems by gradually shifting taxes away from income and towards carbon. The market would respond accordingly.</comment>


New Recommendation System = 40 Percent More Diggs - <AUTHOR> http://www.techcrunch.com/2008/07/31/digg-credits-new-recomendation-system-with-40-percent-surge-in-activity/ <comment><USER2> Can anybody who still uses Digg comment on if this makes Digg worthwhile again (or at least not a waste of time)?

<comment><USER3> I use Digg but I found the recommendation engine pretty poor. The problem I had with it is it's very simplistic. If you and user X have both digged up a story, and then user X diggs up another story, it concludes that you probably like that story too because you both agreed on the first story. This tends to be not true all the time. Ideally they should try for more of a netflix-like system that evaluates why certain people like certain types of stories, but that would be much more effort to implement

</comment></comment><comment><USER4> just changes the scale of the filtering 'now takes more diggs for the front page' .... so if they havent adjusted yet .. ( although im sure its automatic) it would be easier to hit front page</comment>


Google AI Challenge 2011 - <AUTHOR> http://aichallenge.org/ <comment><USER2> From the rules:

> Any attempt to disrupt the normal operation of the contest software or the > contest servers will result in the immediate involvement of law enforcement > officials. Our policy is to always prosecute.

I get that people shouldn't be intentionally disrupting the servers, but that sounds like an awful policy.

<comment><USER3> Why? If they don't want people trying to break into their servers. I'm honestly asking, I don't understand your argument

<comment><USER4> For one, it exposes a heavy-hand predisposition makes me worry that a bug in my program might result in a visit from police.

</comment></comment></comment><comment><USER5> See also this previous thread from 212 (!) days ago: <http://news.ycombinator.com/item?id=2349826>

There are two things worth pointing out:

1) This competition has previously been sponsored by Google but isn't run by them. It's hosted by the University of Waterloo Computer Science Club.

2) Some people have been working on bots for this challenge for months now, so they are likely to have a bit of a head start.

<comment><USER6> That sucks (some people having a head start). What a silly way to run a competition.

<comment><USER7> I think the OP just meant that some people had seen this page before, not that there is any conspiracy behind that.

<comment><USER6> I didn't say it is a conspiracy, just silly.

</comment></comment></comment></comment><comment><USER8> The one thing that bugs me about the Google AI Challenges is that they do not really encourage using modern AI techniques. It's all about intelligent _developers_ rather than writing intelligent _software_. I really would like to see a challenge which is all about data: identifying patterns and learning to make predictions – rather than developing yet another heuristic for a minimax algorithm...

<comment><USER9> You are free to try to identify patterns in enemys ants behaviour, make predictions and win the game this way.

<comment><USER10> The challenge does not offer disk space for your bot to save data so it has to start from scratch every match (unless you supply it with a lot of offline training data).

There has been talk among the developers of the challenge of offering disk space in on of the future challenges.

<comment><USER11> An alternative would be to allow bots to connect to an external server maintained by the bot author to get the latest strategy, and dump match history for analysis.

<comment><USER12> Even just getting a second or two during startup to connect to external servers and again a second or two at the end would be enough.

</comment></comment></comment></comment></comment><comment><USER13> Very cool game spec. One thought: your ants are basically telepathic, so they are able to share information instantly--e.g., the squares they see flow back to your "master bot" which can instantly use this information in the current turn for every other ant. I wonder how much more challenging it would be if you had to build not a "master bot" that controls all ants, but a "bot" that runs for _each ant_ , and they have their own individual inputs and state and can perhaps transmit messages to other ants within a 3 block radius. Maybe they can even lay down pheromones to mark territory... (too much SimAnt as a kid, can you tell?)

Certainly it would be more realistic, and I think even more fun--but maybe a little taxing on the server. But this is run by Google, right? Maybe next year.

<comment><USER12> This is a hive mind ant game!

<comment><USER14> I wrote one in Python a while back. <https://github.com/phreeza/cells>

Contributions are always welcome.

</comment></comment></comment><comment><USER15> From the Reddit post:

> The contest is not ready till tomorrow. Everything is still beta today, all > accounts so far will be purged tomorrow. ~amstan Contest Organizer

Source: [http://www.reddit.com/r/programming/comments/lhlt9/googles_a...](http://www.reddit.com/r/programming/comments/lhlt9/googles_ai_challenge/c2sr1fa)

</comment><comment><USER16> When is the contest over? Last time I tried to do one of these things I only had about a week which was not enough time for me to come up with a quality program. Otherwise I would like to compete.

<comment><USER17> The last one started September 10th and ended November 27th.

</comment></comment><comment><USER18> Great to see a game similar to the one I developed some years ago. My game was a multiplayer realtime programming game. The ants were controlled by a uploadable lua code. If you are interessted, feel free to visit and download the sourcecode on <http://infon.dividuum.de/>

</comment><comment><USER19> Just so it is clear, we haven't actually launched yet. Although we should be live within the next 48 hours.

</comment><comment><USER20> I still think a webservice-based version of these contests would be much more fun. The idea would be to only give the simulator a URL of where your AI is hosted. There would be an expected set of endpoints for the simulator to call and would invoke your service when it was your turn. Perhaps the AI could call services on the simulator as well.

So, instead of having to write your AI in the simulator's language you could choose whatever you want. Another advantage is that you could run your own database to store and query information so that your AI could become more intelligent.

I've been looking for a new side project. Perhaps I've found it.

<comment><USER17> Stdin/out is used to transfer information from/to the simulator, so any language is possible. The list of languages they support is here: <http://aichallenge.org/starter_packages.php>

You can also ask the contest organizers to add a compiler/interpreter if anything is missing.

<comment><USER20> Ah, thanks. I was glad to see a C# package there ;)

</comment></comment></comment><comment><USER11> It looks like you can only upload code, and you aren't allowed to write any files, which means that your bot can't be self-improving between matches. Which IMHO is half the fun of an AI contest.

Or am I missing something?

<comment><USER21> You could run the contest locally and have the bot incrementally improve itself.

<comment><USER22> But then you're adapting against yourself, or some limited set of demo bots. It's not the same as out in the wild.

</comment></comment></comment><comment><USER23> After watching a game or two, I can tell that ants should always travel two by two for warmonging. When two ants (walking together on the same front) encounter a single ant, the one ant is always destroyed.

<comment><USER24> :) what happens when your pair encounters three? Or if your pair are walking horizontally next to each other in : formation and are attacked from above by a pair in .. formation? I'm very interested to see the strategies that emerge from this challenge.

</comment></comment><comment><USER25> Has anyone found the contest end date? I have a project I can't put down (unfortunately), but would really like to have time to enter.

</comment><comment><USER26> From a quick look the success of a strategy is very much dependent on the characteristics of a map. It seems that there are maps where you cannot even reach your opponent (maze_2, if I'm not mistaken.) Such maps punish defensive strategies. On the other hand, open maps with anthills not far from each other probably punish greedy strategies.

<comment><USER27> The maps are toroidal, i.e. the left and right sides are connected as are the top and bottom. So it is possible to reach the enemy in maze_2.

<comment><USER26> I stand corrected, I missed the connection wrapping around the top/bottom edge.

</comment></comment></comment><comment><USER28> Hey Andrew(tectonic), this is right up your alley ;)

<http://andrewcantino.com/ants.html>

</comment><comment><USER29> What? No Clojure starter package yet?! C'mon, Google!

<comment><USER11> Google isn't actually running this. They were only a sponsor in years past.

It's actually run by the University of Waterloo computer science club as an open-source project.

A Clojure package is in the works, I think.

<comment><USER4> A poster above makes the opposite claim, that the contest graduated from Waterloo to Google.

</comment></comment></comment><comment><USER30> Reminds me of <http://en.wikipedia.org/wiki/Core_War> and <http://en.wikipedia.org/wiki/Conway%27s_Game_of_Life>

<comment><USER31> Back in the early 1980s I used to play a programming game on the Sharp MZ-80K which involved two programmers writing assembly language programs that would move through the operating system memory in an attempt to reach low memory (where the operating system was not stored) without crashing the machine.

Moving through the operating system meant actually overwriting portions of the operating system as it was running with the running code and the relocating the code to a new location lower down in memory (I've forgotten now but we had some restriction on how far we could jump). Clearly this often meant keeping a cached copy of the memory we were overwriting to put it back, but we could only do that in the operating system itself.

<comment><USER32> This seems like a true hacker's version of the Tron lightcycle game!

</comment></comment></comment><comment><USER33> The ICFP 2004 contest was also about "ants" - [https://alliance.seas.upenn.edu/~plclub/cgi- bin/contest/task...](https://alliance.seas.upenn.edu/~plclub/cgi- bin/contest/task.php)

</comment><comment><USER34> There are similar Java games for long <http://robocode.sourceforge.net/>

You write a robot tank which battles with other tanks.

</comment><comment><USER35> If you can reproduce faster than anyone else you have a big edge. But I'm curious on what the rules of reproduction are.

<comment><USER36> A bit simplified: when an ant moves to a food square, a new ant is produced from a hill.

<comment><USER35> But if you look at the games, some teams create new ants at a faster rate than others, regardless of food consumption. The red ants seem to have an edge on reproduction than others, right from the very beginning of the game.

</comment></comment></comment><comment><USER37> Account creation is closed.

<comment><USER38> The contest starts from tomorrow : AI Challenge Fall 2011 - Ants Opens Oct 20 [http://ai-contest.com/forum/viewtopic.php?f=3&t=1499](http://ai- contest.com/forum/viewtopic.php?f=3&t=1499)

</comment></comment><comment><USER39> Third time for the challenge, nice going Waterloo!

</comment><comment><USER40> <http://antme.codeplex.com/>

Not new, did a similar thing in C# in high school. Definitely a good way to get into programming

</comment><comment><USER41> Soooooo Cool! Never participated in one of these before. Looks like tons of fun.

</comment><comment><USER42> ok well i cant it says account creation closed

</comment><comment><USER43> My challenge back to Google:

The setup:

* A certain person named Prasand is in my Google contacts, along with his phone number.

* Prasand has recently sent email to me.

* Prasand is a reasonably common name in a certain English-speaking country of well over a billion people.

* A common phrase when calling someone is to say "Hi this is <name>" or "Hey this is <name>"

* Usually this comes near the beginning of the call.

* Prasand probably also has a Google account, and Google probably associates his phone number with his account, and thus can look up interesting things about him, such as his name and various words he is likely to use, by using caller ID when he calls into a Google voice number.

* Google knows my first name. Let's call me Natch.

* Speech recognition can be made more accurate if large quantities of data are available with which to build models of how language is used in context.

* Large data sets are available on the internets. Something tells me Google may even have access to large quantities of data already.

* Google even has the capability, if it wants to, to build user-specific language models.

Google, here is your challenge:

Hire an engineering director for your Google Voice team who can manage to figure out how to do the correct transcription of the following five words at the beginning of a phone call: "Hi Natch, this is Prasand."

Hint 1: you should fire whoever did the one you have right now.

Hint 2: less AI, more common sense.

<comment><USER44> > Large data sets are available on the internets. Something tells me Google > may even have access to large quantities of data already.

You don't just need audio data, you also need the correct transcriptions to learn anything from it. This reduces the amount of available data significantly. And producing correct transcriptions is time consuming and expensive.</comment></comment>


GUN: a realtime, decentralized, offline-first, graph database engine - <AUTHOR> https://gun.eco <comment><USER2> Flagged as a dupe. Is that right? Struggling to find a previous one and I was curious to read the comments.

<comment><AUTHOR> I struggled to find it before posting too. It's not a very search friendly name!

I finally found [https://news.ycombinator.com/item?id=14533546](https://news.ycombinator.com/item?id=14533546) using "Graph database" as search keyword

The author ( [https://news.ycombinator.com/submitted?id=marknadal](https://news.ycombinator.com/submitted?id=marknadal) ) posted it a few more times after that. Might explain the [dupe]...

<comment><USER3> Man... He's really struggling to get this Database noticed, might be forced to do that name-change.

</comment></comment></comment><comment><USER4> What are graph databases? It seems it belongs to the NoSQL movement. Is it related to graphQL? What is so different?</comment>


Any decent Intel/AMD-based competition to Macbook Air coming up? - <AUTHOR>

Doesn't have to be equally small, but preferrably cheaper with 7h battery life, reasonable weight, no DVD, non-Atom based with a decent size trackpad? I just can't stand buying a Macbook Air to install Linux on it (yes, I tried OSX but it just does not cut it for my usage). <comment><USER2> This doesn't answer your question but I thought I would weight in regarding your opposition to buying the Macbook Air. I bought a 13" Air when the latest revision came out and it's by far the best laptop I've ever used. Even though I use OSX on it most of the time, because it suits my work, the form factor is what sold me on the device. I wouldn't hesitate to buy one to run Linux or even Windows full time on it. My only concern would be the trackpad behavior, but the machine is so good I'd write my own driver for Linux if I had to.

Apple does a fantastic job of putting hardware of moderate specs in a better package than others. The device just feels better in your hands. It's almost intangible but no matter what OS you're using its a joy to use. Perhaps you don't feel the same way, but I think its worth keeping an open mind, especially if money is not an object.

This post probably comes off as yet another fan boy, but I hope it doesn't. A few years ago I wouldn't even consider Apple, but I finally had enough of the cheap plastic crap that vendors insist on shoving down our throats.

</comment><comment><USER3> I believe system76 has an ultra-thin laptop that meets your specs. I do not have one, but I have been eyeballing one for a while and think it might be what you're looking for.

[http://www.system76.com/product_info.php?cPath=28&produc...](http://www.system76.com/product_info.php?cPath=28&products_id=106)

<comment><AUTHOR> Looks like a nice machine, if you're in the US, Canada or UK. I'm in Norway. Maybe it's worth the hassle of importing one indirectly...

<comment><USER4> It's probably just an ASUS; you should be able to find a local reseller that can get one.

<comment><AUTHOR> Good tip. I found the Asus U31F which looks like a good machine, or Asus U36Jc with Nvidia graphics. From my local vendors, the U31F has a better battery, and the way things are going with linux graphics drivers, the Intel "embedded" graphics on U31F might be a smarter choice (not from pure performance, but support of chipset features, like "good enough performance" and "powersave modes" etc). YMMV.

</comment></comment></comment></comment><comment><USER5> The soon-to-be released Lenovo x220 fits your list very well.

</comment><comment><USER6> Vaio X is a good device

any reason non-atom is on the list? Xmonad, Chromium and emacs are snappy

</comment><comment><USER7> MSI X-Slim, Samsung 9 series.</comment>


Forget passwords - <AUTHOR> http://www.scientificamerican.com/article.cfm?id=computer-game-for-security

<comment><USER2> Okay, let's get the first part out of the way really quickly: this would be an amazing piece of technology, if it were to be implemented successfully. Having a password that even _you_ don't consciously know would be a huge leap in security, and would presumably make password guessing all but impossible-- after all, if you don't need to _remember_ your password in the traditional sense, it could be significantly more complex than traditional "Pa55w0rd1!"-style passphrases.

I, personally, don't know enough about the science behind this to comment on its feasibility.

I do, however, know a lot about security in general. Every time there is a leap in "password technology," meaning public key authentication, or password- keepers, or xkcd's "let's use several words instead of random characters," people seem to think that it would protect all their online assets. That if the password is secure, so is their data that's stored somewhere on the Internet.

On one hand, yes, this technology would easily protect you from brute-force and dictionary-based attacks. After all, if there's no dictionary (because no words are used in the secret "pattern,"), how could a dictionary be used to attack the protected account?

What isn't factored into this example is the service you're protecting _itself_. If you look at high profile breaches over the last year, you'll notice that few, if any of them are straightforward password-guess breaches. Sure, some (like the Wired journalist) could have been prevented with strong and unique passphrases. Most of the attacks could not.

If there is, for example, a buffer overflow in a service that is listening on the Internet, remote code execution may be possible. If you can execute arbitrary code on a remote service, you are able to effectively take over that machine. Combine this with privilege escalation, and it's pretty much game over for most data stored there. Proper database security like unique salts per user account can minimize the damage, but the fact is that the compromise still happened and the data--like your personal photos, or documents uploaded to cloud storage--are probably in the hands of an attacker.

It's always great to see cutting edge research in any field related to computer security, but don't think that "passwords even _you_ don't know!" are going to protect anything behind them. Passwords are just the front door to the mansion, and there's a whole lot of other ways to break in.

The short version is that there will always be a difference between _account_ security and _application_ security.

<comment> B-Con > On one hand, yes, this technology would easily protect you from brute-force > and dictionary-based attacks.

I highly doubt that.

There will be a finite number of possible patterns. So brute force will be possible. There will likely be some types of patterns or some pattern components that do not follow an even distribution, so dictionary attacks will still exist. (Really, a dictionary attack is simply a brute force attack that takes advantage of the non-uniform choices of passwords.)

My guess is that such games would be chosen based on techniques that produce the best (aka, most uniform) distributions of responses from users, but I doubt that it will be perfect. A brute-force attacker would probably be able to do noticeably better than blind brute-force.

</comment></comment><comment> B-Con This would be awesome, but I feel skeptical of how practical it would be.

First, the biggest problem today isn't password quality, it's protecting them.

1) Hack the password verification/storage database. If you can do that and recover the user-provided input, who cares what type of data it was?

2) Be the man in the middle. Convince the user to give you their password. No password is immune from being entered into, for example, a false bank website.

That said, this type of password input opens up a new set of side-channel password attacks:

* What about people who can record the audio of you pressing keys? If you share an office with a co-worker, could he set his cell-phone to record when you enter your "password" a few different times and learn to copy your rhythm?

* What about extenuating circumstances? Stress, unfamiliar "password entry" layouts, or personal injuries may cause your "password" entry rhythm to get goofed up (I don't know in detail how resilient these schemes are, but I'm assuming that something as drastic as switching hands would mess it up), so you need a fall-back. If you have a traditional password backup authentication, this scheme simply offers an alternative to passwords and not a replacement of them, so what else do you do? More potential "I can't input my password" problems means that it will be easier for an attacker to con his way through the password work-around process. It's already a weak-point of security infrastructures.

It sounds awesome, but might be better suited for covert agents who need to be able to not be capable of giving up a password. Not sure how well it would work as a replacement for passwords in an enterprise or for GMail.

(Also, for the curious, at least one of the authors of the original paper is a high-profile cryptography researcher from Stanford. So this paper isn't just another "hey, this might have security uses" after-thought from researchers outside the field.)</comment>


A neighborhood garbageman explains modern Egypt - <AUTHOR> http://www.newyorker.com/magazine/2014/10/13/tales-trash <comment><USER2> "In 2009, during the worldwide epidemic of H1N1 swine flu, the Ministry of Agriculture decreed that all Egyptian pigs had to be killed. There was no evidence that pigs were spreading the disease, but the government went ahead and slaughtered as many as three hundred thousand animals. Some Egyptians believe that the decision was driven by a desire to appease Islamists, who had become outspoken critics of the regime, and supposedly hated pigs even more than they hated Mubarak."

wow.

<comment><USER3> Also see [http://www.reuters.com/article/2009/05/05/us-afghanistan- pig...](http://www.reuters.com/article/2009/05/05/us-afghanistan-pig- idUSTRE5444XQ20090505)

</comment></comment><comment><USER4> For those debating about reading the article:

I read it in its entirety. It's quite long, but it has some fascinating vignettes into life in Egypt. Topics covered include garbage collection and recycling, politics, marriage and divorce, and the roles of males and females in Egypt.

<comment><USER5> Very interesting read indeed, a very thorough insight into life in Egypt. I especially liked the relationship part, the female oppression and dominance at the same type. Seems almost like a paradox through the eyes of a westerner.

</comment></comment><comment><USER6> What a great little vignette on a part of our modern world. The clash of tradition, politics, love, and garbage. My family owns an auto repair shop and I could just feel the clothes that Sayyid wears on my skin again (grease and salt, similar, but not the same as his). My father also deals with most strata of society and Sayyid's observations ring true to me as my father's words: 'Assholes have all manner of 0's on their bank slips'.

</comment><comment><USER7> > The government played no role in establishing this system, which worked > remarkably well. Social scientists often cite it as a success story among > developing-world megacities... Brilliant!

</comment><comment><USER8> _One year, we couldn't buy bottled water for months, because the plant that produced the water somehow caught fire._

Ouch. A way to _know_ one lives in the developing world.

<comment><USER9> please come and visit michigan.

<comment><USER8> Please confirm, is bottled water unavailable for months at a time in this benighted place? What do they do about Wal-Mart, which claims to be able to supply bottled water at all times in 118 different locations in a place they're calling "michigan"?

</comment></comment></comment><comment><USER10> no one suspects the social engineer that is illiterate and wears shoes 2 sizes too large

</comment><comment><USER11> Anyone else reminded of the garbageman from Dilbert?

[http://dilbert.wikia.com/wiki/The_Garbageman](http://dilbert.wikia.com/wiki/The_Garbageman)

</comment><comment><USER12> I scroll down past the opening photo so the 1st line of text is at the top of my browser window and settle in to begin reading and then the header scrolls down to cover the top 3 lines of text.

Hate.

<comment><USER13> Side project idea, chrome or firefox extension that takes text and pictures from popular news sites and removes all the distracting bells and whistles. Now that I think about it, something like that probably already exists.

<comment><USER14> Readability?

[https://www.readability.com/](https://www.readability.com/)

<comment><USER15> I would rather have something local, not send a list of everything I read to some server somewhere.

<comment><USER16> Ah, then you want Readable.

[http://readable.tastefulwords.com/](http://readable.tastefulwords.com/)

<comment><USER15>
         Evernote has acquired Readable!

Hmmm, let's check Firebug's Net tab

        http://readable.tastefulwords.com/log.js?rand=0.8736547357568928&url=http%3A%2F%2Farstechnica.com%2Finformation-technology%2F2014%2F10%2Fborked-belkin-routers-leave-many-unable-to-get-online%2F&options=%5B%5B%3Dtf%3A%20quote(Palatino%20Linotype)%2C%20Palatino%2C%20quote(Book%20Antigua)%2C%20Georgia%2C%20serif%5D%5D%5B%5B%3Dtfm%3A%20quote(Courier%20New)%2C%20Courier%2C%20monospace%5D%5D%5B%5B%3Dtfh%3A%20quote(Times%20New%20Roman)%2C%20Times%2C%20serif%5D%5D%5B%5B%3Dts%3A%2018px%5D%5D%5B%5B%3Dtlh%3A%201.5%5D%5D%5B%5B%3Dbw%3A%2030em%5D%5D%5B%5B%3Dcb%3A%20%23F5F5F5%5D%5D%5B%5B%3Dct%3A%20%23282828%5D%5D%5B%5B%3Dcl%3A%20%230000FF%5D%5D%5B%5B%3Dta%3A%20normal%5D%5D%5B%5B%3Db%3A%20blueprint%5D%5D&customCSS=&_=1412717055539

I'll stick to Color Toggle [1] then.

1: [https://addons.mozilla.org/en-US/firefox/addon/color- toggle/...](https://addons.mozilla.org/en-US/firefox/addon/color- toggle/?src=search)

<comment><USER13> Good stuff on firebug research. I am however, not sure how switching between color modes helps with messy layout.

<comment><USER15> It doesn't, but if you uncheck "Let pages override this settings", all the backgrounds will be the same and the effect is quite similar.</comment></comment></comment></comment></comment></comment></comment></comment>


Mosfet the cat 2001 – 2018 - <AUTHOR> https://blog.adafruit.com/2018/02/28/in-loving-memory-mosfet-the-cat-2001-2018/ <comment><USER2> RIP Mosfet, sorry to read that. Cats are wonderful life (and lab) companions. I also have a couple and had others I miss a lot. All of them had their personal character, habits and quirks, but invariably all of them have this fantastic common trait in all cats: curiosity! I just love when they stare at us when we're doing something; they don't interfere, just sit there and watch. One of them apparently didn't like circuit boards; one day I was looking at a circuit board I just finished soldering, then that cat came closer, looked briefly at the pcb and started hitting it with his paws... that was hilarious, just like telling me i sucked at soldering stuff.:) Probably there were light reflections on the solder joints he didn't like, but whatever... He and another one were also attracted by ultrasound, I discovered this when testing a 40KHz transducer connected to the signal generator: as soon as I hit the transducer resonance the cats rushed in the lab from 2 rooms away meowing... apparently they can hear 40KHz from distance but also totally ignore you when you call them a meter away. I just love those animals!:)

Get a new one soon, there are countless of them in need of someone taking care!

<comment><USER3> Apparently they can't hear male voices. Their hearing is sensitive to higher frequencies than most male voices and the usual human range. I usually chirp or make other high pinched noises to get their attention-though still get ignored much of the time, of course.

Also apparently they'll attack whats in our hands as they think they're helping us hunt.

They really are great. I was lucky to have a cat part of my life for 2 months shy of 20 years. I learned a lot from him and still miss him absolutely every day. He knew many human words - would come running and even let us hold him up to swat at whatever bug was flying around, if needed.

<comment><USER4> Scientific resources online seem to disagree about sound range, which is both wider and more sensitive than human hearing or voices.

[http://www.lsu.edu/deafness/HearingRange.html](http://www.lsu.edu/deafness/HearingRange.html)

Cat sensitivity is a good 6 dB more sensitive so even if they're peakier in treble than humans, they still hear human males from further away than humans can hear them. Somewhat intuitively this makes sense, lots of human powered evolution to make dogs work well with males would force excellent hearing of male voices.

Also you can run a reasonably intelligible for all voices telephone service off 300 to 4000 hz for a century or so, so no point debating 50 hz cutoff vs 75 hz or whatever, its vastly wider band than telco standards either way. Something kids under 30 will never experience is before digital vocoders and high bit error rate cell phones, telco voice service was somewhat hi-fidelity almost AM radio sound quality. Maybe someday voice communication will be that high quality again.

Cats of course are very intelligent and can train humans to do all kinds of stuff that is likely very amusing to cats, as per the anecdotes of them training their humans to perform in singsong voices and whatever.

<comment><USER5> As my gf said - cat fascinated by laser pointer => human fascinated by cat.

<comment><USER6> More than an echo of Montaigne's famous quote, "when I am playing with my cat, how do I know she is not playing with me?"[1]

1: [https://en.wikiquote.org/wiki/Michel_de_Montaigne#Book_II](https://en.wikiquote.org/wiki/Michel_de_Montaigne#Book_II)

</comment></comment></comment></comment></comment><comment><USER7> Sorry for your loss, adafruit.

I have not developed a single line of code without a cat by my side for 11 years now. While they occasionally walk all over my keyboard, they are more effective than the Apple Watch at getting me to stand up, walk around, and interact with the world. Cats are truly great; their antics definitely spur engineering creativity.

</comment><comment><USER8> Would it be inappropriate to ask for the black banner, in recognition of Mosfet?

Just reading the article, you can tell what a pillar of love he was for everyone at Adafruit. I'm sitting here crying for a cat I never met, because I can feel the love for him bleeding through my screen. He was a hell of a guy, and a hell of a friend to everyone he knew.

If we can give a Britsh navy cat (Simon) a medal in recognition for his service, then I say we can also respect and give tribute to Mosfet as a great developer and hacker, in his own cat way.

</comment><comment><USER9> He is visually indistinguishable from several blacks I've had over the years. They have wonderful personalities, and I can't imagine what led people to think them ill omens. Lovely story, and am sure he will live on in the hearts and minds of everyone who knew him. :-)

<comment><USER10> I adopted my two nearly all black tuxedo cats after learning that black cats get adopted less often because people think they're bad luck. If they won't then I'll just have to adopt all the black cats.

<comment><USER11> I've heard this before, but when I've adopted my cats I didn't see a particularly large number of black cats at the shelters. Last time there was maybe 1 of 15, and he was in a brother/sister pair that wanted to be adopted together (otherwise I might have taken him!). Maybe it's a regional thing? Or maybe my experience is unusual.

Similarly unconfirmed reports from twitter that the Black Panther movie has emptied shelters of all of their black cats. I can believe that pretty easily.

</comment></comment></comment><comment><USER12> Back in the days of CRTs, I had a cat named Trouble. She would sleep on top of my 15" Monitor. Friends and family would invariably remind me how I shouldn't let this happen. This monitor I got for free lasted for years being Trouble's favorite spot.

Trouble disappeared never to be seen again. And within a month the monitor stopped working.

</comment><comment><USER13> If it wasn't for the cats we had when I was a kid, I'd be a different person today :')

<comment><USER14> same 4 myself :3

my 1st cat was killed by neighbor's dog when I was 4 years old

life never was the same since then

<comment><USER13> I'll never forget the day my mother called to tell me that their cat had died in an accident, suffering. I was (technically) grown up then, and it was many years ago, but it still hurts. Writing this hurts :(

</comment></comment></comment><comment><USER15> I am the tiniest bit disappointed that Mosfet didn't have three legs.

<comment><USER16> Mosfets are 4-terminal devices!

<comment><USER17> But one of the terminals is called "body"...

</comment></comment></comment><comment><USER18> Suddenly makes me think of Heinlein's "The Door into Summer".</comment>


How to build your own feature store for ML - <AUTHOR> https://www.logicalclocks.com/blog/how-to-build-your-own-feature-store <comment><USER2> This is a good idea - every ML operation should have something like this, to store, organize, version data, check for drift, do time-travel, backups/replication et cetera.

But to borrow from Steve Jobs, I think this is a feature, not a product. If you've already done the hard work of setting up a data lake or data warehouse in a cloud provider, the cloud provider can give you backups and replication, and even some time-travel. Using something like Delta Lake or even just the standard Kimball DW audit columns will get point-in-time queries. Feature versioning is just query versioning in source control, and if you have schema, you can schema version with views if you need to. If you don't have a data lake, data warehouse ... well, you'll still need to gather and clean all your data before you put it into a feature store, and that's where 90% of the work is.

I'd love to learn more, I'm sure I'm missing something, but it seems that they're re-solving the solved part - data storage and versioning. Checking for drift and data integrity is a nice bonus, but again, lots of libraries for that. I guess I could see it being beneficial for ML shops that don't have modern development practices, but if you don't have that, you have bigger problems anyways.

<comment><USER3> All your points are valid points. However, operational models (models used by online applications, for example) typically need access to lots of historical features that are not available in the application. In that case, you need to go to a low-latency database/store to get your feature values (build your feature vectors). If you want to reuse those features in different models, you will need join support for building the feature vectors, so a key-value DB won't help there. Now, your features are duplicated between this online/serving layer and the data warehouse. How do you sync them up? The other thing you're missing is time-travel queries (temporal logic for SQL in data warehouse speak). Yes, Delta Lake gives you this, but you will need to wrap that data in APIs so that your data scientists will be able to use it. For data drift, a library alone won't cut it. You need to compare descriptive statistics/distributions of the data used to train the model and the live data coming in. Where do you get those statistics from - the feature store, in our case (with the help of versioning+metadata). Then, there is end-to-end governance of ML models - what training dataset was used to train this model, can i reproduce that training dataset if it hasn't been archived? You need metadata to manage all that. So, yes you can do it - but you have to build something (as the article describes) or buy it.

</comment></comment><comment><AUTHOR> As this topic will inevitably become more trendy find some some additional interesting resources on the subject as well:

\- [https://www.quora.com/What-are-the-implementation- challenges...](https://www.quora.com/What-are-the-implementation-challenges- of-a-machine-learning-feature-store)

\- [http://featurestore.org/](http://featurestore.org/) (a list of -some of- the available feature stores)

</comment><comment><USER4> A great collection of real-world case studies and various implementations can be found here: [http://featurestore.org/](http://featurestore.org/)

</comment><comment><USER5> Similar to generating feature vectors for dataset augmentation here [https://vectorspace.ai/covid19.html](https://vectorspace.ai/covid19.html)

</comment><comment><USER3> I'm the author. Let me know if you have any questions.

<comment><USER6> Just to add a counter-anecdote, as I see lots of (good/valid) questions about "why do I need this?", here's an anecdote about "yes we definitely benefited from this":

\- Years and years ago, we already had a data warehouse (DWH)

\- In the data warehouse, you would store data like, each and every order that all customers have made (i.e. up to and including full-fledged facts and dimensions about each)

\- Now, let's say axiomatically/hypothetically, a very useful and highly predictive feature for ML is "# of orders made in past 7 days" for each customer

\- Can this be computed from the data already in the DWH? Yes, absolutely, but it's a new computation and not an existing column/attribute in the dimensional model.

\- What if you need to recompute this feature daily, for millions of customers and orders? Well, we could always just add it to the dimensional model, compute it once, and let people just use it/share it... but why? Most internal users of the DWH probably don't care about something like "# of orders past 7 days" as something to be added to a customer dimension or per-customer grain (too specific or whatever), and moreover, the DS/ML folks want the same feature but for "every 1/3/7/30/90/180/365/730/etc." days breakdown, as well as a bunch of variations about orders and things other than orders (e.g. "average time between new orders, over past 7/30/90 days" or "average $ spent over past 7/30/90 days", as features that serves as a proxy for frequency of activity and level of engagement)

\- Hence, it makes sense to keep the "golden copy" of data in a canonical form in the classic/standard DWH as a baseline, and to separately/independently compute features out of that data and to store them in a different system (which can also be optimized for the different query/access patterns that DS/ML have, vs traditional BI). Over time, it also made sense in certain cases to go upstream of the DWH to source data from and process it more directly (for performance/efficiency reasons), though generally deriving features out of the transformed dimensional models was still very useful.

\- It took our teams ~1-2 years to really go through this evolution and reach a mature-ish state, but for the past 2-3 years, we've benefited tremendously from having an independent feature repository/store, that is separate from the classic DWH. Benefits came in all the obvious and some non-obvious ways, i.e. in faster iteration/cycle time, in better quality/repeatability, and in being able to automatically discover interesting relationships that no human could have anticipated - simply by having a very broad/large repository of features and running automatic feature selection over it.

<comment><USER7> In your use case is a feature store roughly a collection of incrementally updated feature tables + time/version meta?

We've been storing our features in RDBMS/normalized with each feature table having a run_id column (run_id is unique to pipeline version + timestamp of execution if batch processed; if streaming there is run_id for pipeline version but the date comes from interaction date, which is part of the raw data already]) and I'm curious what we're potentially missing..

In this sense you can query features for given users generated by particular version and/or date but it does involve potentially lots of joins (to get a collection of features).

<comment><USER6> Conceptually I think that's about right (meaning, similar concepts as us, but who's to say whether this is objectively the best approach...). Practically, we're heavy on AWS, so we've found that for our size/scope/breadth, it was more performant/efficient to store the data as parquet files in S3 and cataloged in AWS Glue, and yes there are a lotttttt of joins needed, so it's worth taking time to invest in some deep thinking about, how best to partition your data and how best to optimize the type/variety/complexity/number of joins you'll have to do.

I can see RDBMS being reasonable for small-to-medium size of data, but beyond a certain threshold, I think it starts to breakdown (at the few 10s/100s of TB level, maybe?).

<comment><USER7> Yeah, we are in the 1-10 TB range. We are bound to on-prem Oracle Exadata and so far it's ok.</comment></comment></comment></comment></comment>


JavaScript Errors: An Exceptional History - <AUTHOR> https://blog.appsignal.com/2019/10/17/javascript-errors-an-exceptional-history.html <comment><USER2> I remember that in IE all you got was a yellow alert sign in the status bar that there was a JavaScript error _somewhere_ and you added alert boxes on every 100:ed line eg. alert("line 200") alert("line 220") until you had located the error. This is still how you do it on some mobile/tv browsers. Some browsers will not even tell you there is an error - and fail silently.

<comment><USER3> Wait, so when did console.log start to appear?

I remember alert without console.log as well but I was too young to understand the context.

For ios mobile dev I use Safari as a debugger. If I have to use Chrome, I basically use a scrappy console that I made on the HTML page itself.

<comment><USER4> Sometimes dumping data to a pre tag is the simplest thing, especially on mobile. I did it just yesterday :)

Be careful about using JSON.stringify on objects as a debugging technique though, because you may not be able to see properties on the prototype chain ('inherited' properties), and some objects customize their serialization to JSON (supply a custom toJSON method) which in some cases can be misleading. Things like that have contributed to a number of my grey hairs. Yes, I'm talking about you, Geolocation Coordinates! :)

Chrome has really nice support for remote debugging on Android.

If you can't get physical access to the device to configure them for remote debugging, there are some other nice remote debugging tools. One example: [https://jsconsole.com/remote-debugging.html](https://jsconsole.com/remote- debugging.html)

</comment></comment></comment><comment><USER5> I had no idea JavaScript shipped without Arrays. Too bad they didn't just let objects take anything as a key, like in Lua. Then you wouldn't have all the edge cases Arrays cause in JS. Oh well!

<comment><USER6> This (anything can be a key) was eventually added with the Map class. They're an iterable type too, so they're much more useful than Objects imo.

[https://developer.mozilla.org/en- US/docs/Web/JavaScript/Refe...](https://developer.mozilla.org/en- US/docs/Web/JavaScript/Reference/Global_Objects/Map)

<comment><USER5> I agree! They are more useful! That's why I wish objects behaved that way to begin with!</comment></comment></comment>


Building a pure CSS 3D City - <AUTHOR> http://cubiq.org/building-a-pure-css-3d-city

<comment><USER2> A lot of these CSS posts reference the performance gains realized from using CSS transforms as opposed to canvas because the transforms are hardware- accelerated.

How long will it be before canvas gets hardware-accelerated? Is there a technical hurdle that needs to be overcome in getting (for example) V8 to talk directly to GPUs?

<comment><USER3> Canvas is still 2D though, so unless they develop a 3D drawingContext for canvas, you won't be able to do what's in this article with the same kind of hardware acceleration. With canvas you'd still need to run all your own poly drawing routines, while I imagine Webkit is able to take 3D CSS properties and map <div>'s straight to polys that render with hardware T&L pipelines. (Which makes me wonder, how the heck are they dealing with Z-order?)

<comment><USER4> There already is a hardware accelerated 3D context that uses the canvas element... WebGL!

<comment><USER3> Whoa, didn't realize WebGL used canvas. Cool.

</comment></comment></comment></comment><comment><USER5> Tried cranking up the number of buildings...

<http://dl.dropbox.com/u/18782/durp.html>

</comment><comment><USER6> For the record, even though all of his live demos are marked Safari only, they work just fine in Chrome (for me anyways)

</comment><comment><USER7> If anyone likes this kind of stuff, you might find my weekend project to be interesting: <https://zetabee.com/weave/>

You can make pretty 2D designs using CSS:

<https://zetabee.com/weave/share/5e6a5811>

<https://zetabee.com/weave/share/c6593101>

It is similar in concept to Structure Synth ( <http://structuresynth.sourceforge.net/> ) but in 2D and using CSS. I was thinking of making a browser-based 3D Structure Synth but not many people seem to like making art with code.

</comment><comment><USER8> Very very creative! A browser based SimCity clone should theoretically be possible with enough processing power.

<comment><USER7> Minecraft demo in Chrome/Safari: [http://mrdoob.github.com/three.js/examples/webgl_geometry_mi...](http://mrdoob.github.com/three.js/examples/webgl_geometry_minecraft_ao.html)

</comment></comment><comment><USER9> You could skip all the CSS transformation stuff and just serve the browser the images in the orientation you want them. In this projection and buildings-of- cubes, there's only 2 perspectives for a wall and 1 for ground/roofs.

<comment><USER10> And how does real time spinning and panning work with your suggestion?

You need the transformations to make the buildings spin in your browser window. Didn't you see the demo?

<comment><USER9> Panning would work fine since the objects in this projection don't change anything but position with panning. Spinning would only work in 90 degree increments, but that's fine for many, maybe most cases you'd be doing this. Most isometric-ish projection systems I recall seeing don't spin / change perspective. I'd imagine that's because it's disturbing to look at since things don't get smaller as they get farther away. For whatever reason the static version doesn't induce the same reaction.</comment></comment></comment>


Hikikomori - <AUTHOR> https://en.m.wikipedia.org/wiki/Hikikomori

<comment><USER2> Anyone interested in this phenomenon should check out the anime "Welcome to the NHK."</comment>


JQuery-style CSS selectors in Python, with lxml - <AUTHOR> http://codespeak.net/lxml/dev/cssselect.html

<comment><USER2> Ditto for ruby: <http://nokogiri.org>

Both a little bit of a pain to build on OSX since it ships without the required version of libxml. Not too bad though.</comment>


A fast 100% spec compliant, self-hosted JavaScript parser - <AUTHOR> https://github.com/KFlash/seafox <comment><USER2> Interesting project. Wonder if this would be useful for web based programs that could allow third party add-ons, etc... I was thinking WASM with some bindings would be prefect, but being in JS itself sounds cool as this acts as a sort of sandbox I'd imagined (Well I assume someone could try to make an infinite loops and lock it up?). Be neat if could some how try to limit/monitor memory and CPU usage.

<comment><USER3> > web based programs that could allow third party add-ons

The library in the current post, Seafox - cute name! - is "just" the parser, turning JavaScript code into an abstract syntax tree. One would need a sandboxed interpreter/evaluator/virtual machine to run it.

You might be aware of Figma's efforts, where they compiled Fabrice Bellard's QuickJS engine (in C) to WASM, to use for their plugin system.

[https://www.figma.com/blog/how-we-built-the-figma-plugin- sys...](https://www.figma.com/blog/how-we-built-the-figma-plugin-system/)

[https://www.figma.com/blog/an-update-on-plugin- security/](https://www.figma.com/blog/an-update-on-plugin-security/)

[https://bellard.org/quickjs/](https://bellard.org/quickjs/)

The first article describes an earlier system using Realms, a shim/polyfill for an ECMAScript proposal.

[https://github.com/agoric/realms-shim/](https://github.com/agoric/realms- shim/)

[https://github.com/tc39/proposal-realms](https://github.com/tc39/proposal- realms)

</comment></comment><comment><USER4> These are pretty surprising claims... As far as I know, V8 itself is written in C++, is this self-hosting parser faster than Node's parser?

<comment><USER5> Fair point, though I read it like this, which doesn't make the claim _so_ bold :)

> Twice as fast as other Javascript parsers (sic) written in JavaScript.

<comment><USER6> Yes, this was my interpretation too (should probably be made more clear). Most JS dev tools are written in JS (Webpack, Babel, VSCode) so there's a large use-case for a fast JS-native parser. It could also, theoretically, be run in- browser for a macro system or something.

</comment></comment></comment><comment><USER7> Is it possible to build an AST with v8 in JS?</comment>


Ask HN: What are some key ingredients for a compelling Ask HN? - <AUTHOR> I've noticed that some users respond with suggestions to refine an Ask HN question to make it more compelling for the wider audience, so I know multiple users will have an opinion on this.
Theoretically there is a decent mixture of factors that activates more users desire to respond based on the prevailing themes and culture of HN, yeah?
I'll start it off with something I don't think works too well:
A blatant attempt to generate a business idea or conduct market research.
Something that does work:
A question that taps into personal experience and gives users a chance to express their philosophy on a subject.
I notice I'm more likely to respond to questions about jobs and participation in industry.
Thoughts? <comment><USER2> As dumb as this might sound, I think timing is important.

I am going to make a broad generalization that most of our readers are located between Eastern and Pacific time. I would expect most activity to happen during common down times in the work day (eg. after lunch, post scrum). I think this is essential for generating meaningful and well thought responses.

Perhaps you can do some time analysis on some of the most successful Ask HN threads and tell us your findings :)

Also popular threads like "Who's hiring/firing etc" always happen at the beginning of the month, am now programmed to expect those.

Best of luck

<comment><AUTHOR> I don't think it's dumb at all, I bet it does count as an ingredient to an overall compelling post.

Ignoring a fact like posting when a large number of users are potentially active does appear detrimental to the goal.

</comment></comment><comment><USER3> >A question that taps into personal experience and gives users a chance to express their philosophy on a subject.

This works to some degree; if you ask "what is your experience with (Node.js|Electron)?" I feel like that won't do well due to the polarization and broadness attached to the subject. Some people will just avoid it because they don't want to read through pages of conflict (and they already know the popular quips that get copy pasted on every thread like that) and the people that are flocked to it will ultimately turn it into a flame war that gets shut down.

I think it has to be a question people can relate to, but at the same time, not have an obvious or universal answer. Ultimately your question is "what makes you click on an Ask HN thread?" and, apart from obvious clickbait, the simplistic answer is "an Ask HN where I'm interested in reading the answers." To extrapolate from that, _who_ is I? Well, _I_ is shaped by "the prevailing themes and culture of HN," so I figure anything that taps into tech or entrepreneurship is fair game. Is there more to it? Without clickbait..I don't think there's an objective answer.

<comment><AUTHOR> That's an interesting position, I would expect clickbait to be more important on the normal posting format, because users are being asked to go to some external resource for reading.

</comment></comment><comment><USER4> Going as meta as possible is probably the most effective way for your Ask HN to make it to the front page.

<comment><AUTHOR> Fair point, since this question is getting engagement, but I don't think breaking the fourth wall equates with an attempt to ask a compelling question about a real subject, if that makes sense.

</comment></comment><comment><USER5>
      RecursionError: maximum recursion depth exceeded

<comment><AUTHOR> Must not have been tail call recursion.

</comment></comment><comment><USER6> An attempt to generate a business idea actually fits extremely well here. I am not sure where you get that feeling from but I am interested to hear the rationale at least, as a conversation starter perhaps.

<comment><USER3> I tried this Ask HN before: [https://news.ycombinator.com/item?id=19846911](https://news.ycombinator.com/item?id=19846911)

It didn't do really well, maybe due to the timing and not the subject matter, but it's worth noting that some people in the thread did take an accusatory stance and assume malicious motives from my question. I tried my best to cover my bases and make sure the intent _didn 't_ come across as personal gain, but alas, you can't convince everyone.

My goal was to get responses from people who work in "exotic" or otherwise "not techy" industries who could benefit from technology that they may know about but are not experts in (alternatively, they are capable, but do not have the time nor incentive to care). I'm sure that, especially among the people who frequent HN, we've all had experiences with processes in our lives where we've thought, "man, I could script this in a few hours."

<comment><AUTHOR> I don't find the activity inherently malicious at all, it's a certain type of brain picking that people don't respond to positively. Thanks for the example!

</comment></comment></comment><comment><AUTHOR> I think gao8a is probably right about data analysis providing a more comprehensive answer to the question in lieu of a greater quantity of subjective responses.

Two sides of the same coin.

</comment><comment><USER7> Something contentious - i.e. Doesn't have a simple "correct" answer.

<comment><AUTHOR> I get a feeling that contentiousness works better with the normal post format that usually links to someones opinion on a subject that is obviously contentious here.

</comment></comment><comment><USER8> metaness

<comment><AUTHOR> Indeed, as I responded to the other mention of this, I think engagement with organic subject matter is different than the engagement that comes from asking introspective questions like this one.

That's my personal opinion right now, I could be wrong about it philosophically.</comment></comment>


Ask HN: 'Crash Courses' for Mathematics Related to DL, ML and Data Analysis - <AUTHOR> I am specifically looking for free pdf or online materials for mathematics needed in ML, DL and Data analysis which doesn't necessarily go in depth. My primary aim to have a good top level view and if possible get hold of the most basics stuffs as soon as possible. <comment><USER2> First, please read and internalize dsacco's comment.

In the meantime, the following might help. But from experience, if your math is already shaky, you would certainly need to look for resources covering everything listed in a more in depth manner.

Basic calculus refresher - [http://www.stat.wisc.edu/~ifischer/calculus.pdf](http://www.stat.wisc.edu/~ifischer/calculus.pdf)

Learning from Data - [http://www.inf.ed.ac.uk/teaching/courses/fmcs1/readings/matl...](http://www.inf.ed.ac.uk/teaching/courses/fmcs1/readings/matlab- barber.pdf)

Basic probability theory - [http://homepages.inf.ed.ac.uk/sgwater/teaching/general/proba...](http://homepages.inf.ed.ac.uk/sgwater/teaching/general/probability.pdf)

Mathematics for Machine Learning - [https://pdfs.semanticscholar.org/910e/3118b50f426e5e840561e1...](https://pdfs.semanticscholar.org/910e/3118b50f426e5e840561e1f9f1fdd67e5724.pdf)

Math for Machine Learning - [https://www.umiacs.umd.edu/~hal/courses/2013S_ML/math4ml.pdf](https://www.umiacs.umd.edu/~hal/courses/2013S_ML/math4ml.pdf)

<comment><AUTHOR> Yes I have read his comment and added a reply there. To reiterate my point, I asked for crash courses not because I wanted to make quick progress but because I wanted to have an idea of what kind of mathematics is needed. In a way I was looking for some syllabus with some description added.

Thank you for posting these links. I'll definitely go through these.

</comment></comment><comment><USER3> I'm sorry to say this because I observe that we get these questions somewhat frequently, but I don't think what you're asking for - in the format, goals and speed you'd like it - is a thing.

That's a pretty discouraging statement, so let me try to be more helpful. The mathematics comprising contemporary machine learning and data analysis primarily consists of linear algebra, calculus, mathematical statistics and, at the high end, probability theory. There are boutique efforts to use things like topology but that's non-standard.

As it stands, your question is underspecified, which is what I repeat for most of these questions. The answers you receive here are going to variously interpret what you're actually looking for and prescribe based on the author's interpretation and intuition. It would be more useful if you explained exactly what your goals are. Here's precisely the problem: you apparently don't know these mathematics already, but you're asking for something that "doesn't necessarily go into depth." How much depth is too much, and how would you know that exactly given your present unknown unknowns? If you explain what you actually want to achieve, we might be able to

1\. Tell you that you don't actually need that "top level view" to achieve what you'd like,

2\. Tell you that such a top level view is not nearly enough for what you'd like to do, or

3\. Tell you that a top level view is coherent, and optimize the best materials for you to learn from based on what you want to do.

It would also help us make recommendations if you explained what your current level of mathematical background looks like. Have you taken linear algebra at least once? Exactly how basic do our resources have to be? Different textbooks written for different audiences can variously explain the same concept in two pages or 10 pages, and they can emphasize different things.

I'd like to help, and I probably can, but it would go a long way if you could tell us what your end goal is instead of what you interpret as the next step towards that goal. Then we can provide resources based on your mathematical maturity.

<comment><USER4> Topology for ML sounds amazing. Got any papers/links?

<comment><USER3> Sure, try this: [http://math.ucr.edu/home/baez/information/](http://math.ucr.edu/home/baez/information/).

In a nutshell, you take probability distributions and project them onto topological manifolds, such that the distribution consists of points on the manifold. You can find more by searching for key words like "information geometry" or "differential geometry machine learning".

<comment><USER5> there's also the stuff that ayasdi and carlsson were/are doing but i never really saw the point of that (e.g. just compute connected components instead persistent homology).

</comment></comment></comment></comment><comment><USER6> Any PreCalculus text -> Stewart Calculus -> Strang + Axler Linear Algebra -> a Calculus based Statistics & Probability book. Do the problems in the books. Read each chapter. If you do that you'll know Calculus and Linear Algebra better than most. If you want to then move up from there study Real Analysis and higher level math like Measure theory (but this may not be necessary for your purposes).

</comment><comment><USER7> Take a look at the Coursera specialization: Mathematics for Machine Learning [1]. The specialization isn't free but you can certainly apply for financial aid.

[1]: [https://www.coursera.org/specializations/mathematics- machine...](https://www.coursera.org/specializations/mathematics-machine- learning)

<comment><USER8> Yes! It's a new course that will be open for enrolment soon. I think it's exactly what most people here are looking for.

There is the financial aid and normally there is the option to watch the lectures and see the assignment for free. On the other hand, the cost of the courses rarely exceeds $50.

</comment></comment><comment><USER9> Check out my books on CALCULUS+MECHANICS, and LINEAR ALGEBRA: [https://minireference.com/](https://minireference.com/) They are not free, but not expensive. It's like 2 years worth of undergraduate maths packed into two small books.

Previews are free though, and might be useful if all you need is an overview: [https://minireference.com/static/excerpts/noBSguide_v5_previ...](https://minireference.com/static/excerpts/noBSguide_v5_preview.pdf) [https://minireference.com/static/excerpts/noBSguide2LA_previ...](https://minireference.com/static/excerpts/noBSguide2LA_preview.pdf)

This is also good: [http://ml-cheatsheet.readthedocs.io/en/latest/](http://ml- cheatsheet.readthedocs.io/en/latest/)

</comment><comment><USER10> [http://www.cs.cornell.edu/jeh/book.pdf](http://www.cs.cornell.edu/jeh/book.pdf)

<comment><USER11> Foundations of Data Science

Avrim Blum, John Hopcroft, and Ravindran Kannan

Copyright 2015

</comment></comment><comment><USER12> Undergrad lin algebra and calculus ... start there! Although there is no skimmable view of those worth a damn. You need 2-3 months @ 8 hours a day.

Stats thereafter is pie, ML low hanging fruit will come easily.

Good inference and learning from data on the other hand is experience: expect years, no shortcuts. In fact, suffer. It'll make the journey as informative as possible.

</comment><comment><USER13> Similar quesion, aimed at math majors: [https://www.reddit.com/r/math/comments/81px9v/what_are_some_...](https://www.reddit.com/r/math/comments/81px9v/what_are_some_good_mathematical_books_should_i/)

Really, if you're not going to go to e.g. community college where you can do a lot of this (LA, prob/stats, calc) your best shot is to find study groups in Data science meetups, etc which are plentiful in major cities since many people have the same impulse. Truthfully, self study rarely gets far if people never went beyond Calc 2.

Here's a blog about one guy's adventure, read the part where he talks about Shores' LA text(not a bad book by the way but yes, lots of typos): [https://news.ycombinator.com/item?id=8996024](https://news.ycombinator.com/item?id=8996024)

</comment><comment><USER14> Week 1 Linear Algebra [https://ocw.mit.edu/courses/mathematics/18-06-linear- algebra...](https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra- spring-2010/)

Week 2 Calculus [https://www.youtube.com/playlist?list=PLZHQObOWTQDMsr9K-rj53...](https://www.youtube.com/playlist?list=PLZHQObOWTQDMsr9K-rj53DwVRMYO3t5Yr)

Week 3 Probability [https://www.edx.org/course/introduction-probability- science-...](https://www.edx.org/course/introduction-probability-science- mitx-6-041x-2)

Week 4 Algorithms [https://www.coursera.org/courses?languages=en&query=Algorith...](https://www.coursera.org/courses?languages=en&query=Algorithm%20design%20and%20analysis)

</comment><comment><USER15> Read these books!

* [http://www-bcf.usc.edu/~gareth/ISL/ISLR%20First%20Printing.p...](http://www-bcf.usc.edu/~gareth/ISL/ISLR%20First%20Printing.pdf) * [http://web4.cs.ucl.ac.uk/staff/s.prince/book/book.pdf](http://web4.cs.ucl.ac.uk/staff/s.prince/book/book.pdf) * [http://users.isr.ist.utl.pt/~wurmd/Livros/school/Bishop%20-%...](http://users.isr.ist.utl.pt/~wurmd/Livros/school/Bishop%20-%20Pattern%20Recognition%20And%20Machine%20Learning%20-%20Springer%20%202006.pdf)

I am not an expert in ML. I have heard experts in ML call the above books introductory.

</comment><comment><USER16> This got trending on Github today. It might help:

[https://github.com/llSourcell/Learn_Machine_Learning_in_3_Mo...](https://github.com/llSourcell/Learn_Machine_Learning_in_3_Months)

First month covers the basics (calculus, algebra, probability, algorithms)

Second month is focused around coding (Data science in Python) and putting the things learned in the first month in a ML context (Siraj's Math of Intelligence course on YouTube)

Third month is all around Deep Learning.

</comment><comment><USER17> This article on matrix calculus seems pretty good for someone familiar with ML and DL concepts but not on the maths:

[http://parrt.cs.usfca.edu/doc/matrix- calculus/index.html#sec...](http://parrt.cs.usfca.edu/doc/matrix- calculus/index.html#sec3)

</comment><comment><USER18> [https://developers.google.com/machine-learning/crash- course/...](https://developers.google.com/machine-learning/crash-course/ml- intro)

</comment><comment><USER19> the strang lectures on LA, any decent calculus book (khan academy or something maybe too) and then i would jump into ISLR and the karpathy Stanford lectures on DL, the rest you learn as you go adhoc.

<comment><USER12> One can't over emphasise: LA and Calc. Do that!

The rest will be a cake walk.

But you need LA from matrices to spectral decomposition and Calc from differentiation to Laplace transforms; so no easy feat. You'll need a fair amount of "give a damn" to put yourself through it.</comment></comment>