Using SimCity to diagnose my home town's traffic problem - <AUTHOR> http://gamasutra.com/blogs/MikeRose/20130219/186896/Using_SimCity_to_diagnose_my_home_towns_traffic_problem.php

<comment><USER3> I love simcity, but recently played it and noticed it is really paternalistic.

The attitude towards development it has is very much like China - <USER2> is controlled and all land is owned by the government, and the government's job is to manage zoning, roads, schools, education, in order to make people happy. Nobody actually has any property rights - even trees don't grow without government planting efforts. Farmers can't choose their own crops.

This is actually really different than what happens in the real world. Really, if a community doesn't have a school, people get together and start one. Small civic organizations evolve into permanent buildings, training schools, etc. Pickup sports leagues evolve into organizations that commission their own stadiums.

If there is no fire department, a volunteer one starts. People build churches, or start them at home, whenever they feel the need. But in simcity world, everything good comes from the government.

I love simcity, but I do think it's weird that there isn't a mode where you can just let people build whatever they want. I love the built-in, realistic needs people in it have - but where are the entrepreneurs who will actually start a society, or borrow money to build a building on their own? Where are the rebels who start a revolution if the government doesn't meet their needs?

<comment><USER4> WTF. Libertarianism is like some grumpy religion, people are always trying to bring it up even when it doesn't fit.

And it's so wrong about how the real world works. If a community doesn't have a school the first thing that normally happens is they form a government and elect a school board to figure how to build and fund it. Same with a fire department, even with volunteers there are a lot of expenses that need to be paid by the town. Even the sports team is normally brought to a town via the work of politicians.

What these government-haters don't understand is that government IS the community. It's how a community comes together to get things done.

<comment><USER5> "What these government-haters"

Hi. libertarian here. I don't hate government. Government is probably a necessary evil.

I hate obnoxious, inefficient, bug-ridden, corrupt, expensive, government. I hate government that says one thing to get elected and then promptly does another. I hate government that ostensibly is an extension of the will of the people and then acts against their expressed wishes.

But I'm not grumpy about it.

<comment><USER6> Hi. Anarchist here, I do hate government. Government in my opinion is an unnecessary tragedy that has befallen humanity and murdered close to 1 billion people.

This is by far the most intelligent community I have come across in all my internet travels, and if there is any group of people who can think of a way for society to run peacefully, it is this community.

Any problem we have as a society that people currently point to government as a solution, can be solved without resorting to threatening everyone with police action.

<comment><USER7> It worked in that utopia called Somalia after all. :-)

<comment><USER6> The non-sharia controlled parts have been better off than they were with their previous government. Their air travel industry is even profitable, more than we can say.

I'd say that's a lot more than you could ask for considering the exploitation of the region.

<comment><USER8> Kinda sucks if you do live a sharia controlled part then. Here in the UK there are groups who would love to annex of chunks of the country as sharia zones but thankfully the law prevents this.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> In the original Sim City my brothers and I eventually figured out the solution to ALL traffic problems was to simply never build roads and only use rail lines. Apparently sims were much happier waiting for hours or even days for a train to finally pass their out of the way residential area.

<comment><USER10> Subways can be even better, since they leave the surface clean for other things.

<comment><USER11> Original Sim City didn't have subways, just railways with very sparse commuter trains.

I tried the all-rail town, too. There was a persistent alert to "build more roads".

<comment><USER12> You could have a few sparse roads to get rid of that alert, and all things are good. Also, bulldozing congested roads was a net positive.

</comment></comment></comment></comment><comment><USER13> He should have built the town in mirror image so as to account for driving on the right. It may be that the interaction of various turns across the road have an impact.

<comment><USER14> That was my first thought too. It surely would make a difference as in one case a busy road moving to another via a left turn can simply filter in and in the other it has to wait for the opposite flow of traffic to clear.

Is there a widely available traffic profiling tool like Sim City without the extraneous "fun" parts?? I assume there's some sort of gui based analysis prog that city planners use?

</comment></comment><comment><USER15> Sim City should be a recommended game for those taking Geography, especially in HS.

When I first started playing, I was reacting to situations - fires, disasters, as they happened, which was already too late.

Or I'd build all the fire stations in a nice row, or police stations all in a ordered block of land, then get frustrated when they all jammed each other trying to get out and answer calls. My tactics from Red Alert isn't going to work here.

Can't wait for the latest SC!

<comment><USER16> Just so you're aware about the new SimCity, it requires a constant internet connection to play, and there's no way to save/load previous versions of a city. You can't mess around and experiment without doing irreversible damage, which was a big part of the fun.

<comment><USER17> I was worried about DRM, but the beta convinced me it was worth it. Server- side simulation processing makes the game run lightning fast.

<comment><USER18> The simulation was a huge resource drain in SC4. So I can see how offloading it would be a speed up. That said, if it's such a drain on today's insanely powerful desktop computers, they must have an insane backing server infrastructure in place to support it. I've got 50W worth of processing power available for the sim engine (investing half of my CPUs; still far more power than I had for SC4), are they going to put up 50W of processing power while I'm playing?

I can't imagine they've really got that. Either GlassBox is simplified, or made more efficient, or important parts of it are run locally. Either way, using it as a reason to have it online-only is a sham. It's just DRM, the same kind as Assassins Creed's online-only protection.

I'll buy it anyway. I wouldn't even care about the DRM, it's _everything else_ that I'm hearing that scares me.

<comment><USER17> I don't know the details about which parts of the simulation are run locally and server-side, but the fact that your cities are "always on" (i.e. other players' cities can trade and visit your city even if you're not playing) tells me that a good portion of the resources/traffic simulation is run on Maxis servers.

It may be not be a valid reason for you, but Sim City ran significantly faster than Civilization V on my machine. To me, they both perform a massive number of simultaneous simulations, and Sim City's performance is near instantaneous.

That said, there's one huge factor none of the beta players can take into account when judging Sim City's performance: no one has been able to build massive, multiple-city metropolises, as play time was limited to hour-long sessions.

<comment><USER19> The game can actually go on for a few minutes when you lose connection. So I don't think there is much of the simulation happening server-side.

<comment><USER20> Yep, one of the developers mentioned that everything is verified server side, but the temporary offline mode indicates that its not that important.

</comment></comment></comment></comment></comment></comment></comment><comment><USER21> FTA: "...including the fact that once I'd built East Didsbury, the strip of shops in Northenden stopped making as much money as they once were, and some were even beginning to close down as my time ran out. Walk along Northenden high street, and you'll know that feeling."

This is interesting. Urban planners out there, do these types of simulations occur before an area is zoned?

<comment><USER22> The problem with a lot of small to medium sized towns is that the people in charge of making zoning decisions are just local good ol' boys (or girls) who got voted in office because people knew who they were. They are in no way qualified to make these kinds of decisions.

And so you get what you get.

<comment><USER23> s/voted/appointed/ in most cities.

<comment><USER24> My uneducated opinion is that appointed/career people tend to do better. You do not want to get politics involved, that tends to screw things up.

<comment><USER25> Who do you think appoints those people? It's all about politics, either directly or indirectly.

<comment><USER24> There are ways of shielding those decisions from politics. Relying on appointed people is one such way. No shielding is perfect – but it's better than nothing.

</comment></comment></comment></comment></comment></comment><comment><USER26> I had the opportunity to play through both betas, and I'm fairly certain we haven't even scratched the surface of the interesting "experiments" we can do with it.

I'm still purchasing this game, but I do wish EA would consider not forcing you to connect to their servers at all times.

</comment><comment><USER27> Anyone have info about the beta program? Simcity 2000 was my favorite game on my 486. 3000 and classic were flops... this looks really really good.

As for the article, really interesting. Sounds like proof of the power of SimCity's engine. Would be more interesting to see him fix his issue.

<comment><USER28> I got into the final wave of the beta over the weekend.

Absolutely buying this on release day - gorgeous game and IMO a large improvement over previous installments. The only things that bother me is the limited size of the map (unsure if it's a beta thing or not) - if I can't build a megacity I'm going to be sorely disappointed.

Traffic model, economic model, new zoning model, and new utilities model are all IMO simplifications that make the game more fun, but does not remove depth. For those who haven't followed the game: power, sewage, and water no longer require power lines/pipes/etc, and travel along roads. They propagate along roads outwards from their generation source (water towers, power plants, sewage plants, etc) until their capacity is reached. There's a great animated overlay view that lets you monitor the state of each of these pretty easily.

The biggest pleasant surprise was how well it _ran_. I pushed all settings up to max and it didn't even flinch. SimCity 4 in contrast got insanely slow when your city became even slightly large - even on today's hardware.

The ability to monitor cashflow is also a welcome change. Instead of having to dive into your balance sheet to figure out your fiscal position it now shows a very clear flow rate (negative or positive) next to your balance.

<comment><USER29> What video card are you using? Mine was apparently too old, so I'm looking for a new one.

<comment><USER28> Two GTX570's in SLI - I'm surprised all the newfangled overlays and postprocessed effects ran so well, though the real surprise is that the CPU- side simulation didn't eat my computer for lunch. SimCity 4 for mid-sized cities and above _still_ kills a i7.

<comment><USER30> Sim City 4 is single threaded and only uses one core, which is why it is so slow on new hardware.

The other major drawback to Sim City 4 is that there is a bug which prevents rich ($$$) Sims from working in the High Tech industry. The only way to fix this is to use a 3rd party mod.

<comment><USER28> There are _many_ problems with SC4. The most egregious one (for me anyways, someone who likes to build insanely dense cities) is that pathfinding "bug".

The short version is, your sims must find a reasonable commute from home to work, otherwise your economy stagnants and unemployment runs rampant. SC4's pathfinding heuristic _disallows_ a sim from going backwards to go forwards.

This creates _many_ transportation deadzones and a lot of unintuitive bottlenecks. You will literally see sims live _directly next to_ a subway station that takes them directly to work, but they will drive instead.

There is a mod to fix this, but the performance impact is expectedly awful.

<comment><USER30> Ugh. It's been a while since I played, so I had forgotten about this.

It's not as bad as you described it. It's even worse.

The hack the creators of SC4 used to get around the problem of traveling the same path twice in one trip is responsible for a large waste of CPU cycles. Essentially, the hack allows the Sim to "forget" how he got to a certain location and start a new trip to the final destination. The downside is that this is that all of the information from the previous leg of the trip is lost, and the algorithm must run again. There isn't a 1:1 ratio between the amount of citizens and CPU cycles wasted, because as your city grows the path each citizen must take becomes more complex and consumes more cycles.

You can read more about it _here_ , at a third party fan site. [1]

[1] [http://sc4devotion.com/forums/index.php?PHPSESSID=fcff28ec82...](http://sc4devotion.com/forums/index.php?PHPSESSID=fcff28ec82b1624d842dbd7309a161d6&topic=2763.0)

</comment></comment></comment></comment></comment></comment></comment><comment><USER31> Too bad it has horrible DRM.

</comment><comment><USER32> Microscopic traffic simulation is a difficult problem to solve, it's not car movement, it's driver's behaviour. The model has to be good enough to reproduce the current situation and, what is the most important part, be able to test realistically future scenarios.

In any case the new SimCity looks fantastic, maybe we can exchange our simulation model (Aimsun) for their graphic engine :)

</comment><comment><USER33> I haven't been this excited for a game to come out in probably more than a decade.

<comment><USER34> Me too. I had the chance to play the beta last weekend and even with reduced functionality the game was quite a lot of fun.

I rarely if ever play games and I'm certain this will be a terrible time suck.

</comment></comment><comment><USER35> Here is the first of 3 videos explaining the simulation engine: <http://www.youtube.com/watch?v=vS0qURl_JJY>

</comment><comment><USER36> I wonder if adding toll booths on either end of the city would be a viable- enough source of income to make up for the terrible traffic in the city.

</comment><comment><USER37> I think the solution, at least where you create everything, is buses that go non-stop between major hubs. And with traffic lights synchronized to allow them to hit their top speed. And, where possible, set the speeds high.

If every spoke is timed so it takes the same amount of time, you can minimize how long people have to wait. Make everybody get on at one end of the bus and exit at the other and hub stops can be short.

You'd have to walk to the nearest hub and from the one closest to your destination. But you need the exercise anyway.

And charge tolls for passenger cars that vary with demand, something that would work without recreating the roads.

</comment><comment><USER38> Unless you can build cycle tracks there's no point.

<comment><USER39> You can't. It's a super car-centric game. You can't even build pedestrian-only streets or lanes. Light rail is married to roads. There's no way to set bus routes, buses will visit every single stop in town. Oh and subways don't exist either.

Hopefully there's a 'rush hour' style expansion that lets us simulate cities other than Los Angeles.

<comment><USER20> There is a pedestrian only road, but at the moment its tied to the university as a plopable. But a developer mentioned that he would have a look at it.

</comment></comment></comment><comment><USER40> It seems to me that they blundered when it comes to marketing. If they had just called it SimCity Online and promoted it as an MMO, no one would get upset about the persistent connection requirement and no save/load functionality.

<comment><USER41> That may do more damage than good. [Franchise] Online titles tend to be met with suspicion and disinterest from gamers who traditionally enjoyed the franchise.

<comment><USER42> It's already SimCity Online in everything but name, and many gamers who traditionally enjoyed the franchise are exceedingly sceptical about the changes. Might as well call a spade a spade.

</comment></comment></comment><comment><USER43> I would love to see someone do this with the Holland Tunnel traffic on both sides.</comment>


MSFT Office Live Workspace is in Beta - <AUTHOR> http://office.microsoft.com/en-us/office_live

<comment><AUTHOR> First impressions:

You can use firefox.

Worked on 2007 but not 2003. I assume this means it requires Office 2007 or better.

Requires an additional download (update to Office, which I believe also authenticates your copy) to create or edit files. (Can view files without)

No OneNote Live (yet?)

Overall, I must say it is pretty janky.</comment>


Ars Technica plagiarizes space history articles - <AUTHOR> http://thespacereview.com/article/2394/1

<comment><AUTHOR> At least one of the Ars Technica posts was discussed on HN: [https://news.ycombinator.com/item?id=5711784](https://news.ycombinator.com/item?id=5711784)</comment>


Why Are Some Non-Technical Founders Unwilling to Learn About Technical Concepts? - <AUTHOR>

Founders who describe themselves as "non-technical" that I meet or consider themselves the business side of the equation are unwilling/flat-out refuse to learn basic technical concepts relating to their startup. Even if it's just the bare minimum - some JavaScript, basic understanding of the relevant server requirements for their company, etc. It's confusing to me, especially when they talk about finding an awesome engineer to be their technical co-founder not realizing that awesome engineers want to work with those who think like engineers and are educated about the technical side of things. <comment><USER3> > not realizing that awesome engineers want to work with those who think like > engineers and are educated about the technical side of things.

Really? I have the opposite feeling. Good engineers do not want to work with a CEO who will dictate how to write the code. The ideal is obviously a CEO that has enough understanding of the technical aspect to respect your decisions as technical expert.

<comment><USER4> I mostly agree...but when I was a junior technical person I liked having a very technical manager. But once I'd learned my skills it was much easier to work with people who would let me do the things that I do well. So I'm glad to let them make the decisions for the sales, marketing, and business side - <USER2> that I don't have to.</comment></comment>


For startups in emerging fields, patents are critical - <AUTHOR> http://pandodaily.com/2012/10/12/for-startups-in-emerging-fields-patents-are-critical/

<comment><USER2> Their idea is obvious ("existing X applied to existing Y"). Their business value should be in implementation, execution and first mover advantage, not in some imaginary "claims" to an obvious idea that anyone can come up with without breaking a sweat.

The entire article boils down to "patents are bad except when they're ours", and that's disgusting.

<comment><AUTHOR> Patents definitely have problems - I agree.

But they are here, now, and I hear patents are crushing startups. This article just presents that sobering reality and how an innovator might address the patent threat until this mess is fixed - which looks to be taking a while.

<comment><USER2> > how an innovator might address the patent threat until this mess is fixed

I have no issue with anyone claiming patents defensively, but he's trying to justify having a claim and an exclusive right to an obvious idea. THAT is trolling no matter how much he calls himself an innovator, a grassroots developer or whatever. He's part of the problem, not the solution.

<comment><AUTHOR> I totally see your point.

I gather you would not accept that when others are claiming obvious patents one has to as well until some higher power changes the rules lest one be trolled into nonexistence?

I think the guy is being pragmatic - it isn't great - but it is pragmatic. He may well be using the exclusive power of patents in ways that are anti- competitive. He doesn't say - you are right about that.

Patents are very likely to be a waste of money unless the patent laws are 'fixed' somehow to preserve the value of the existing patents - at least until they expire. And for a lot of people that wouldn't be a 'fix' at all.</comment></comment></comment></comment>


Advanced Data Structures (2017) - <AUTHOR> https://courses.csail.mit.edu/6.851/fall17/ <comment><USER2> Hey! I'm a coauthor with this guy :^). Incredibly fascinating (and extremely smart) dude. His interest in origami is what drew me to MIT in the first place, since I was interested in it as well from a young age. A few years ago I took his class on the computability/complexity theory of folding, and we ended up solving an open problem in the field and published a paper out of it [0]. If you had told highschool me something like that would happen, I never would have believed it :)

[0]: [https://arxiv.org/abs/1901.08564](https://arxiv.org/abs/1901.08564)

<comment><USER3> Hi there. What would you suggest to a person who loves these kind of stuff (data structures and algorithms) but finds it hard to develop the underlying intuitions for coming up with these data-structure and algorithm? I personally just hammer down on deliberate practice to recognise patterns and underlying concepts to solve as many problems as I can on my own. I was wondering apart from general DS and Algo what topics in Mathematics might supplement this process?

<comment><USER4> I recently had someone ask me about a problem in machine learning. The algorithm for it turned out to be something I had read about 30 years earlier, where it was being used to accelerate the simulation of the motion of stars in galaxies. I had never expected to use that algorithm, but its existence, if not details, was there in my long term memory, waiting to surface.

So, I suggest you just read everything you can, getting the gist of things so if you encounter something similar you know where to look.

I also suggest looking for little tricks you can apply again and again. For example, some of the data structures in that course use a trick where you break the structure into a top part, using one algorithm, and a bunch of leaf parts, each of size maybe O(log n), that are handled using another algorithm. Often this combination does better than either algorithm by itself.

<comment><USER3> Few questions:

1\. Where can I find that algorithm you are talking about?

2\. The algorithm which you stated seems a bit complex from the algos taught in the undergrad level, so what are the pre-requisites for understanding it that I must be aware of?

3\. What books or papers you would suggest for reading?

4\. How the hell you remember something you read decades ago :)?

<comment><USER4> The ML problem involved, for a set of points on a line, computing the sum (over all the pairs of distinct points) of a function d(x_i,x_j) that is, in some sense, well behaved. The purpose was to compute the similarity of two sets of real numbers.

It turns out this can be done (to sufficient accuracy) in nearly linear time using something called the fast multipole method.

[http://www.umiacs.umd.edu/labs/cvl/pirl/vikas/publications/F...](http://www.umiacs.umd.edu/labs/cvl/pirl/vikas/publications/FMM_tutorial.pdf)

I wouldn't consider the "O(log n) chunks" trick all that complex, btw, although Tarjan did have to point it out to me when I should have used it.

<comment><USER3> > Tarjan did have to point it out to me when I should have used it.

Wow, you worked with Tarjan. How cool!!!.

<comment><USER4> Not directly, but he did point out the improvement.

</comment></comment></comment></comment></comment></comment></comment><comment><USER5> Got confused by the broken link too. Those aren't separate links. It's a single link, and given the name of the file is "poster-design.pdf", I don't think the PDF is very important. It's just an image that gives examples of what is probably covered. The real content is described below as the video lectures and notes of the class[1] and the github account[2]. There are also problem sets[3], and it recommends a couple of books[4][5], both of which are apparently free.

[1] [https://courses.csail.mit.edu/6.851/fall17/lectures/](https://courses.csail.mit.edu/6.851/fall17/lectures/)

[2] [https://github.com/6851-2017](https://github.com/6851-2017)

[3] [https://courses.csail.mit.edu/6.851/fall17/psets/](https://courses.csail.mit.edu/6.851/fall17/psets/)

[4] [http://epubs.siam.org/doi/book/10.1137/1.9781611970265](http://epubs.siam.org/doi/book/10.1137/1.9781611970265)

[5] [http://opendatastructures.org/](http://opendatastructures.org/)

<comment><USER6> It's too bad that the links are broken. When the links at the top of the page didn't work, I immediately assumed that this was for an old course and none of the material would still be available.

I only kept looking because I didn't think a useless site would make it to the top of HN. People who find the course site another way might not have a reason to keep looking.

<comment><USER5> What links are broken? I only know of a single link to a seemingly unimportant file that is broken.

<comment><USER6> I thought that the single image was a series of links to the different parts of the courses. It's at the top of the page and has 9 distinct icons representing different things that will be taught. It looks a lot like it's there for navigation. When clicking on each of them didn't work, I assumed I couldn't get to the course content.

<comment><USER5> Yes, this is what I tried to explain in my top-level comment.

<comment><USER6> Ah, I thought you were saying that each link was actually to the same URL. I see now what you actually meant.

</comment></comment></comment></comment></comment></comment><comment><USER7> As a busy self-learner, when is it time to learn data structures in depth? I feel like there are always 10 other technologies I need to know more urgently (eg, more bash, Linux, testing frameworks, deep learning, c++ libraries, linear algebra, common security mistakes/attacks, OpenGL, etc)

<comment><USER2> Given the bulk of the things you've listed here, which seem to serve very practical purposes, maybe never. I wouldn't call data structures a "technology" so much as I would label them as a theoretical underpinning to much of computer science. If your goal is to learn practical technologies (which is a good goal!) you don't really _need_ to learn data structures deeply -- they're just not a skill you need in day-to-day software life (for most people). I'm a software engineer, and the idea of implementing so much as a binary heap (which is nearly one of the first data structures you learn) at work is a preposterous idea. That said, if you want to learn about data structures, go for it! They're incredibly interesting and will give you a way deeper appreciation for computer science and the work that goes into developing complex algorithms. There's just not necessarily a "pedagogical" place to stick in learning data structures with the sorts of technologies you've listed.

<comment><USER8> > _" they're just not a skill you need in day-to-day software life (for most > people)."_

I mostly disagree. You won't need to implement them, but a working knowledge of the basics is the basis of any performance analysis and can make a huge difference in system performance in day to day programming. At some point all the small inefficiencies also add up.

I would go so far as to say that data structures and basic algorithms (e.g. Dijkstra) are more important to become a competent engineer than any of the practical tools he listed. It's like the difference between a mechanic and a mechanical engineer.

There is also a reason they are taught very early in a computer science degree.

<comment><USER9> I think we should remember that the vast majority of computer developers and engineers will not, and probably never, work on stuff that requires deep understanding of things like data structures and algorithms, even those with advanced degrees. Most are just, how to put this, IT janitors.

To use your mechanic and mechanical engineer analogy: They're not even mechanics, they simply change the oil and light bulbs from time to time, along with cleaning the interior.

Not saying that it's not useful - you'd get a much better understanding of things...but this obsession with learning everything under the sun just to be "ready" for the job is overkill

<comment><USER8> I guess it depends what you want to work on and what your career goals are. I think my mechanic vs mechanical engineer example is actually a pretty good analogy of actual conditions. You have (many) people learning how to use tools and fix known problems (like mechanics, or e.g. CRUD developers) versus the others that actually need to solve problems with unknown solutions or optimize existing systems (the engineers). Maybe also with known solutions, but with a far deeper theoretical knowledge as a base for doing that.

Yes, the majority of work is on solved problems and we need far more mechanics than engineers. But it's not such a clear cut and a lot of actual software engineers work on solved problems most of their time and only spend a small amount of time on the engineering part (which is probably similar to other engineering disciplines). But when you spend the small amount of time on engineering work these basics are important and I realized multiple times that I could make better decisions because of it. And I'm kind of certain that's the case for most engineers with an academic background or self-taught basics though they might not even realize it.

</comment></comment></comment></comment></comment><comment><USER10> the professor is a fascinating guy:

[http://erikdemaine.org/](http://erikdemaine.org/)

[https://en.wikipedia.org/wiki/Erik_Demaine](https://en.wikipedia.org/wiki/Erik_Demaine)

EDIT: so is the coauthor! [http://joebergeron.io/posts/post_four.html](http://joebergeron.io/posts/post_four.html)

<comment><USER11> He's obviously incredibly smart to have achieved what he did at such a young age, but he's actually a joy to listen to as well.

<comment><USER12> I was trying to remember why his name sounded familiar. 12 years ago, when I was an 18 year old with mediocre math grades trying to figure out what to do with my life, I went to a guest lecture he was doing down the street at Dalhousie University. It was the first time I remember being excited about math, which became my major.

</comment></comment></comment><comment><USER13> FYI: lecture notes are here [https://courses.csail.mit.edu/6.851/fall17/lectures/](https://courses.csail.mit.edu/6.851/fall17/lectures/)

<comment><USER14> Updated notes from 2017 are cool, though I was hoping the videos would be updated too. They are good, but I was hoping for newer higher resolution recordings.

</comment></comment><comment><USER15> Years ago, I watched all those video preparing for a Google interview. They are really great. I was told advanced data structures would be a big part of it. Not a single thing came up in the process :/

<comment><USER16> Tech interviews never go beyond the most basic data structures. I'm willing to bet that most of the interviewers would not be familiar with these ones either.

</comment></comment><comment><USER17> 22 recorded classes

[https://ocw.mit.edu/courses/electrical-engineering-and- compu...](https://ocw.mit.edu/courses/electrical-engineering-and-computer- science/6-851-advanced-data-structures-spring-2012/lecture-videos/)

</comment><comment><USER18> I was pleasantly surprised to see that improving Wikipedia articles is an acceptable final project for this course. Is this an MIT thing?

<comment><USER19> I saw that too! What a brilliant idea, given that extremely technical articles tend to have inconsistent/incomplete coverage, and could benefit greatly from this kind of initiative.

</comment></comment><comment><USER20> I watched these lectures a few years ago and they were really good.

Since this is meant to be a graduate level research course, I wonder if any cutting edge topics have been added since? Were there any major advances or new ideas in fundamental data structures in the past few years?

</comment><comment><USER21> "Grading...There are four requirements:"

And then it lists _five_ requirements...do I get the MIT course description equivalent of a Knuth check now?

<comment><USER22> There are two hard problems in computer science...

</comment></comment><comment><USER23> I happened to interact with Erik Demaine via an open source project(KaTex). It was my first OSS contribution and he was very welcoming and friendly. Of course, I was also humbled to read about his work.

</comment><comment><USER24> I am getting 404 when I click on linked images.

<comment><USER25> me too!

</comment></comment><comment><USER26> The data structures look really cool, but I couldn't image example to make use of them in "everyday" software. Would someone give me some example of problems that would be easier to solve with special data structures?

<comment><USER27> I haven't looked at the site properly yet, bear that in mind, but to answer your question I probably never have used anything 'interesting' in a couple of decades of working in industry, despite giving a lot of time learning about such things - I do find them interesting.

Nearest I might have got was trying to use a bloom filter in a vehicle tracker, but nope.

What learning about stuff generally has given me is indirect, the ability to dig down and optimise because I understand systems. As I read up on eg. sorting I understand that quicksort is very much not the quickest sort and fixed bad performance bugs from its misuse.

Or understanding Btrees so I can get very good performance out of DBs, even if I've never written a Btree and frankly never want to.

Stuff like that. Maybe one day I'll find the need for dynamic programming and learn it properly, until then it's just a waste of time.

But use them directly, no. Surely sux.

</comment></comment><comment><USER28> At my college, this class was the primary weeding-out class in junior year. Started with a full class, maybe 50? At the end of the semester there were 7 of us.

</comment><comment><USER29> These are some of my favorite lectures, although often goes over my head. Whenever I think I've done something clever, I watch these to ground me.

</comment><comment><USER30> Bookmarked for when I master basic data structures.

</comment><comment><AUTHOR> This works: [https://courses.csail.mit.edu/6.851/fall17/lectures/](https://courses.csail.mit.edu/6.851/fall17/lectures/)

</comment><comment><USER31> Links are broken :/

</comment><comment> 1-6 Honest question, I'm very new to this but I really want to learn. Does putting the video on repeat help with grasping the concepts?

<comment><USER32> I haven't watched any of it, didn't pay much attention in college and get freaked out when I don't understand things the first time, so take this all with a grain of salt, but I would say try watching parts and then find a reason/ way to use the individual concepts.

</comment></comment><comment> 1-6 Is the instructor using Hagoromo Fulltouch Chalk?

<comment><USER33> Ha, I remember seeing that post recently. Looks like it goes back to a 2015 gizmodo article, and was in a recent youtube clip.

[https://news.ycombinator.com/item?id=19718287](https://news.ycombinator.com/item?id=19718287)

[https://news.ycombinator.com/item?id=9723202](https://news.ycombinator.com/item?id=9723202)

[https://www.youtube.com/watch?v=PhNUjg9X4g8](https://www.youtube.com/watch?v=PhNUjg9X4g8)

</comment></comment><comment><USER34> Having been a double major at MIT I still get anxiety spikes whenever I look at one of these course webpages.

Looks like a great course though!

</comment><comment><USER35> Guess this might help,

[https://pastebin.com/m7q1WRcw](https://pastebin.com/m7q1WRcw)

</comment><comment><USER36> Love that they recommend going to Tarjan -- nothing like drinking from the original well!

</comment><comment><USER37> link to pdfs 404

</comment><comment><USER38> Will this be good to study for a FAANG interview?

<comment><USER39> Having just spent several months preparing for FAANG interviews myself, I would say not - these go far beyond the data structures I'd expect an interview to cover at most tech companies.

<comment><USER38> Thanks. What study guides did you use?</comment></comment></comment>


Rails 5: New upcase_first Method - <AUTHOR> http://glaucocustodio.com/2016/05/19/rails-5-new-upcase-first-method/ <comment><USER2> Pretty cool!

In the comment for the code,

# 'what a Lovely Day'.upcase_first # => "What a Lovely Day"

How does it know that "a" should not be capitalized? Does it work with words like "of", "the", for" etc?</comment>


Reimplementing "git clone" in Haskell from the bottom up (2013) - <AUTHOR> http://stefan.saasen.me/articles/git-clone-in-haskell-from-the-bottom-up/ <comment><USER2> As somebody who has only recently started looking into functional programming, this was absolutely fascinating.

Even though I'm not very familiar with Haskell, I find this code much more concise and easy to follow compared to the original git-clone implementation in C. Though perhaps that's more down to the author's coding style?

<comment><USER3> In my experience, pure functional programming, combined with Haskell's relatively permissive syntax, allows for very natural-language-like and well- structured programs.

Languages like Agda can do this even more impressively. You can do stuff like write

        if_then_else_ : Bool -> a -> a -> a

And you can split up the "if", "then", and "else" and put the arguments where the underscores are.

<comment><USER4> The mixfix notation in Agda seems really amazing. I am always wishing I could write code like "add _ to _" or "sort _ by _", etc...

<comment><USER5> It's almost exactly smalltalk (circa 1980 or so) notation.

<comment><USER6> Agda mixfix supports things like:

        [ foo ]
        bar !
        ! baz

etc.

It also supports precedence parsing.

I don't Smalltalk can do all of these.

Smalltalk, unlike Agda, can probably omit the spacing around operators, though, while Agda can't :)

<comment><USER5> I probably should look more closely at Agda.

But the if_then_else_ example is basically smalltalk equivalant - although I suspect Agda will be able to do without the angle brackets that Smalltalk often needs in these cases.

</comment></comment></comment></comment></comment></comment><comment><USER7> Interesting.

It would also be a nice exercise to implement all of the internet RFCs, such as an email server, etcetera in Haskell. That's still a bit lacking. And it would be very useful if it existed, IMHO.

<comment><USER8> I wrote an NFS server in Haskell, using Literal Haskell to implement all related RFC's 'inside' the RFC documents, interleaving code (lots of serdes stuff...) with the specification.

<comment><USER9> link please

<comment><USER8> Closed source (for now) I'm afraid...

</comment></comment></comment></comment><comment><USER10> Don't suppose anyone knows where those nice diagrams are from? (Hoping they're generated from a tool)

<comment><USER11> You're correct that they're generated from a tool, and it's diagrammix [1].

[1] [http://www.reddit.com/r/programming/comments/1cgi2x/reimplem...](http://www.reddit.com/r/programming/comments/1cgi2x/reimplementing_git_clone_in_haskell_from_the/)

<comment><USER10> OSX :-(</comment></comment></comment>


Efficiency of Our Healthcare Systems - <AUTHOR> https://kyso.io/KyleOS/healthcare_life_exp <comment><USER2> I think the opioid crisis is more the problem than "inefficient healthcare"[1]

Inefficient healthcare sounds like it's just that ambulance transit times are too long or that there's too much paperwork. It doesn't bring to mind "pharma companies getting consumers addicted to opioids later triggering a massive addiction crisis that nobody is equipped or prepared to tackle"

[1] [http://thenationshealth.aphapublications.org/content/49/1/1....](http://thenationshealth.aphapublications.org/content/49/1/1.2)

<comment><AUTHOR> I think the opioid crisis and the structural inefficiencies of the healthcare system are 2 separate (but related) problems that both need solving.

<comment><USER2> They definitely are, but I'm replying to the idea that the drop in life expectancy is because of inefficiencies - the us system has always been pretty inefficient and nothing there has changed in a way that would trigger a decrease in expectancy as far as I know. This current decrease in life expectancy seems to be linked to the ongoing opiod crisis, not a sudden decrease in health delivery efficiency

<comment><USER3> The number of uninsured people has been climbing the past few years and the expense of healthcare has continued to grow in relation to inflation.

Those are two factors that drive down life expectancy

<comment><USER4> Life expectancy started to drop in 2015, right as the uninsured rate reached its all-time low. They numbers don't correlate. It might track to the increased cost of coverage and care, but not the uninsured rate.

</comment></comment></comment></comment></comment><comment><USER5> Honestly, I'm happier to see an article talking about efficiency problems rather than just how to pay for it. I'd much rather our efforts on healthcare reform revolve around fixing the problems than paying for them.

<comment><USER6> They're related. The way healthcare is paid causes inefficiencies. If you first need to check whether a patient is insured, and how they're insured, before you can start saving their life, that can cost valuable time and reduce life expectancy.

But there's no single problem hampering US healthcare, and therefore there's no single silver bullet that will fix it. Many healthcare systems have some problems, but US health care has a lot, and many of them need to be fixed in order to improve the situation.

But if I could point to a single underlying problem that causes many of the other problems, it's that the system is primarily designed to make profit, rather than to heal people.

<comment><USER7> Are you saying the people who manufacture/engineer drugs should work for free?

<comment><USER8> Looks like someone's allergic to commie talk, eh? Don't worry, that's not what they're saying. All of the countries in that article do pay for healthcare. It's just that the US pays an imperial shit-ton, and doesn't get all that much for it. So no, not for for free. And the people _working_ in the sector should probably make _more_ if anything. Just investors and CEOs might have to make less to make this system resemble anything like the meritocracy that people like to pretend it is.

</comment></comment></comment></comment><comment><USER9> [https://www.medicalnewstoday.com/articles/282929.php](https://www.medicalnewstoday.com/articles/282929.php)

Our number one reason for dying is heart disease. This tracks pretty well with the obesity epidemic. Likely is one of the primary factors.

Drug epidemics, medical efficiency and others are much smaller factors.

We really need to sin tax food that hits our pleasure receptors. But mess with a persons favorite food and watch what happens. Doubt there is the will to move the needle in the right direction.

</comment><comment><USER10> Americans on average engage in riskier and unhealthier behaviors than people in similarly developed countries. It is unfair to place the full blame on the medical system for lower life spans when they are not causing them.

The people with the highest life expectancy in the world are Asian-Americans living in NJ.

<comment><USER11> > Americans on average engage in riskier and unhealthier behaviors than people > in similarly developed countries. It is unfair to place the full blame on > the medical system for lower life spans when they are not causing them.

Do you have numbers for these claims? What risk taking behaviour is the entire US doing that other countries are not?

>The people with the highest life expectancy in the world are Asian-Americans living in NJ.

And does that group represent the socio-economic gamut? While you're correlating race, I wonder if it ends up being reducible to something like "upper-middle class individuals who don't smoke or drink".

<comment><USER10> Americans are far more obese than people in comparable countries. We also drive a lot more than our counterparts. (Driving is likely the riskiest activity you do every day)

That group represents about a million people but I don't have detailed enough information to claim that it is not skewed.

[https://bigthink.com/strange-maps/two-maps-and-one-graph- com...](https://bigthink.com/strange-maps/two-maps-and-one-graph-comparing- obesity-in-america-and-europe)</comment></comment></comment>


The Age-Old Question: Is Pokemon Go AR? - <AUTHOR> https://arinsider.co/2019/04/10/the-age-old-question-is-pokemon-go-ar/ <comment><USER2> Maybe it's not AR, but it's certainly XR.</comment>


Random Write Considered Harmful in SSDs (2012) [pdf] - <AUTHOR> https://www.usenix.org/legacy/event/fast12/tech/full_papers/Min.pdf <comment><USER2> > _SFS drastically reduces the block erase count inside the SSD by up to 7.5 > times._

Did Apple implement algorithms with similar reductions in APFS?

<comment><USER2> > _What APFS does, however, is simply write in patterns known to be more > easily handled by NAND. It 's a file system with flash-aware > characteristics_

from [https://arstechnica.com/gadgets/2016/06/a-zfs-developers- ana...](https://arstechnica.com/gadgets/2016/06/a-zfs-developers-analysis-of- the-good-and-bad-in-apples-new-apfs-file-system/2/#h3)

</comment></comment><comment><USER3> > However, there remain two serious problems limiting wider deployment of > SSDs: limited lifespan and relatively poor random write performance.

This is a 5 year old paper which is an eternity in SSD engineering. Anyone know if ssd firmwares today handle this better? Last gen consumer SSD was hitting nearly 100K IOPS on random writes which is 3-4x better than a spinning disk. On random reads you're looking at least 50-100x difference.

As far as limited lifespan goes, a fairly popular study from 2014 showed that even a consumer Samsung SSD could hit 2 petabytes without fault.

A 2016 study by Google showed that the write endurance of its drive isn't correlated with failure, but its age. So these things are aging out moreso than exhausting its writable NAND. That's on top of a lower replacement rate than spinning disks.

[http://www.techradar.com/news/computing- components/storage/t...](http://www.techradar.com/news/computing- components/storage/think-your-ssd-will-last-forever-google-has-some-answers- on-that-1316031)

Not sure what current gen is doing, but I imagine its even better. It seems like there's been a lot of work towards write leveling and extending endurance lately and in our own data center we're seeing a lower fail rate of SSD's than we did with spinning disks. Not sure of all the black magic involved here, but from both a desktop and server perspective we never experienced the SSD doomsday scenarios often listed out in the early 2010's when SSDs began to dominate. Curious to know if modern SSDs have worked around the worst issues with flash-based media. It seems they have.

<comment><USER4> I've worked on ssd based systems and while we over-engineered our system due to such fears we never saw any device getting anywhere close to its limits and only once did I see an SSD have a media problem (i.e. failure to read data previously written, after a rewrite of the data from raid no more issues came from that ssd).

This was on enterprise SSDs but I generally believe that the fear from SSD media is somewhat overblown. You do need to care about the wear you will put on the SSD and engineer things so that you are within bounds but the SSD makers and the firmware essentially assume and spec for these random writes.

In fact, I truly doubt that unless you can as an app know in advance what data is going to have the same lifetime and is better written together and what data needs to be placed in different places there is very little that you can do to make things far better.

When you know the lifetimes you can use new features like stream support that is going into newer SSDs in order to help both sides work better. You will get longer endurance and also better performance (less random erase cycles slowing your reads and writes down).

<comment><USER4> An issue with their work for current SSDs is that the write groups they use are about 16MB to 32MB, current SSDs, need far larger write sizes in order to get the optimization they used to get (essentially they tried to fill an entire erase group at once to get control of the locality). This will no longer work at the same level it worked before.

Their estimate of improving endurance is based on using a disk simulator with FTL and not by checking the actual wear level change before and after their test which to me would be fairly suspect. They are not testing against a real FTL and full world behavior of the SSD and are rather "inventing" a test which is very limited and doesn't say much. Admittedly, testing the way I suggest would require a far longer test and take a lot longer but this is the only thing that really tells you if you are making things work in real life.

SSDs are also affected by the workload and especially by any down times that you give it. If you let the SSD sit with no queued commands it will clear buffers and do maintenance work that will improve its behavior which will also affect the result so to get a real "real-world" behavior you need to factor that as well which will increase your test time.

</comment></comment></comment><comment><USER5> I believe this research fed into the design of the F2FS filesystem, which is in the upstream Linux kernel (also from Samsung).

<comment><USER6> No, this issue isn't something the filesystem can work around.

The issue is that random writes leave you with all your erase units partially overwritten, and none of them fully overwritten. If the writes really are random then there's no data layout that will avoid this issue - this is a fundamental issue when write size < erase unit size.

<comment><USER7> That problem is actually simple to solve. You _append_ new versions until you hit some threshold, at which point you actually compact/GC. You don't have a 1:1 mapping. Getting the best performance requires careful tuning, but reducing erase count by a large factor is quite straightforward.

<comment><USER8> FYI, the guy you're replying to is the author of bcache and bcachefs. He's pretty familiar with the tradeoffs of the "simple" solution you refer to.

Fragmenting when you have a partial block invalidation lets you avoid an immediate and costly read-modify-write cycle (both in terms of performance and write endurance). But storing metadata to keep track of those fragments does provide some write amplification of its own, and the eventual garbage collection is a pretty big source of further write amplification. And this requires a non-trivial spare area to keep write amplification under control and performance high. And doing all this on top of an existing flash translation layer that's already doing the same thing makes it even more complicated, and it means you might not be saving anywhere near as many media writes as a naive analysis would suggest.

New SSDs are starting to allow for application-layer hints about data lifetime, in order to allow for the SSD's flash translation layer to do a sort of generational garbage collection. This actually does provide a big reduction to write amplification (and improve performance) beyond the baseline that you get with a traditional SSD with or without a filesystem that tries to be flash-friendly.

<comment><USER7> > FYI, the guy you're replying to is the author of bcache and bcachefs. He's > pretty familiar with the tradeoffs of the "simple" solution you refer to.

Then I don't know why he oversimplified so much. It's not a silver bullet, but it's _absolutely_ a workaround the filesystem can use to mitigate the issue.

> And doing all this on top of an existing flash translation layer that's > already doing the same thing makes it even more complicated, and it means > you might not be saving anywhere near as many media writes as a naive > analysis would suggest.

Sure, but "some drives already have that mitigation" is a far cry from "filesystems can't apply that mitigation". And the end result is still a reduction in write amplification, no matter who does it.

<comment><USER8> > Sure, but "some drives already have that mitigation" is a far cry from > "filesystems can't apply that mitigation".

 _ALL_ flash-based SSDs have FTLs that work more or less as described above. They wouldn't survive even short-term light usage if they did a full erase block R/M/W cycle on every 512B sector modification. If you don't even understand that most basic principle of what SSDs are doing under the hood, you're in no position to judge what a filesystem might be able to do to help. It's as naive as suggesting zipping a file twice.

<comment><USER7> But it's equally naive to state that it's a 'fundamental issue' that writing a bunch of text to disk requires a lot of I/O, and that compression doesn't help.

He didn't just say filesystem smarts were redundant with controller smarts, he said _no data layout_ could avoid the problem. That's a big misstatement of the problem.

<comment><USER8> > But it's equally naive to state that it's a 'fundamental issue' that writing > a bunch of text to disk requires a lot of I/O, and that compression doesn't > help.

That wasn't stated anywhere in this thread.

> He didn't just say filesystem smarts were redundant with controller smarts, > he said no data layout could avoid the problem. That's a big misstatement of > the problem.

You're imagining the conversation to be about a different problem than the one that was explicitly referred to. The fundamental problem under discussion is that random writes will thrash any data structure. You might be able to convert a stream of random writes to sequential writes on the first pass of filling the disk, but once you have a data set on disk and are _updating_ it randomly (which is what almost all real-world random writes are), nothing can prevent the fragmentation and need for garbage collection. Doing it in the filesystem layer won't magically be far more effective than what's already being done at the drive level.

<comment><USER7> > That wasn't stated anywhere in this thread.

It's an extension to your "double zipping" analogy.

> nothing can prevent the fragmentation and need for garbage collection

Nothing prevents fragmentation entirely but these tactics can decrease the write amplification by an enormous amount. If the initial comment was only saying that filesystems can't improve on what drives already do, it was accidentally worded in a very misleading way.

And are you sure filesystems can't arrange writes to make it more effective?

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> Maybe filesystems do not have enough metadata to perform sensible optimisation. Do you want to treat the same way a video file, a shared library, a temporary file and a log file ?

<comment><USER10> > Do you want to treat the same way a video file, a shared library, a > temporary file and a log file ?

But none of those require random writes.

</comment></comment><comment><USER11> I have skimmed the paper. It seems that they assume that the writes are skewed (e.g. some parts of the memory gets written to more often). This seems to be a reasonable assumption for real-life situations, but I am not sure if 'random writes' is an appropriate name (since usually people assume random to be uniformly random).

(Not 100% sure if I understood this correctly, though)</comment>


Stock Market Prices Do Not Follow Random Walks - <AUTHOR> http://www.turingfinance.com/stock-market-prices-do-not-follow-random-walks/ <comment><USER2> This post is quite interesting, and I will have to re-read and ponder it some more, but there is one obvious flaw in the analysis. By analyzing the past returns of current S&P500 companies, the author is allowing for survivorship bias; companies which have done consistently well (in terms of market capitalization) over the analysis period are likely to be over-represented in current indices. To correct for this, the author should re-run the analysis using the S&P500 companies from the beginning of the period instead of the end.

This is the same problem that a study ran into some time ago when it demonstrated that the portfolio managers with the worst returns were the best investments; it analysed the returns of a number of managers over a period and found that the ones with the worst returns at the beginning had the best returns at the end. The problem is that all the consistently mediocre or bad managers were discarded, as they did not survive until the end of the period.

<comment><USER3> I didn't quite understand your last paragraph, but it sounds interesting. Do you have a link?

<comment><USER4> Toy model: A fund can do well or badly at the start of the period, and it can do well or badly at the end. Both happen completely at random. A fund that does badly at both ends is closed and never heard from again.

If you analyse funds in this situation, you will find that _every_ fund that does badly at the start of the period does well at the end. (Because the ones that do badly at the end too are all gone.) You might be tempted to think up clever explanations about how fund managers with bad initial results make extra effort, or how stocks that do badly tend to rebound later as investors recognize their true value, or something -- but that would be a mistake, because in this situation the _only_ thing leading to the relationship between early and late performance is the fact that the "bad at both ends" funds aren't represented in the analysis.

</comment></comment></comment><comment><USER5> It is quite a leap to go from 'not gaussian' to 'not random' as done here. All that has been falsified, as far as I can tell, is a very simple model of a random walk with normally distributed disturbances.

It would be interesting how much better it becomes if higher moments, in particular kurtosis ('fat tails') are included.

<comment><AUTHOR> The article clearly states that the test extends to many forms of randomness beyond Gaussian:

"Nevertheless, the desired effect of stochastic volatility namely, fatter tailed distributions ..."

"... we want a test for the random walk hypothesis which passes (it concludes the market is random) even if the returns demonstrate heteroskedastic increments and large drifts. Why? Because both of these properties are widely observed in most historical asset price data (just ask Nassim Taleb) and neither invalidate the fundamental principle underpinning the random walk hypothesis, namely the Markov property (unforecastibility of future asset prices given past asset prices)"

<comment><USER6> His model is not heteroskedastic. The log-error terms are i.i.d, so they have actually all the same variance. The distribution they are sampled from is a normal variance mixture.

<comment><AUTHOR> Interesting. I'm no statistician but the blog post states that the authors of the original paper (Lo and Mckinlay) claim the test is heteroskedasticity- consistent. So what are you saying? Are you saying the original paper is wrong? That his example was wrong? (This seems more likely) And if the example is wrong, is it wrong to say it is heteroskedastic AND wrong to say it is a stochastic volatility model? Or just that it is heteroskedastic? As far as I can tell stochastic volatility simply means the variance itself is randomly distributed which looks consistent with his example? Just trying to clarift. Thanks.

<comment><USER6> I haven't read the original paper, I can't comment on that.

The model in the webpage however is not heteroskedastic (literally "unequal variance") because all the log-increments are iid. It could be legitimately considered a geometric Brownian motion with stochastic volatility, because the log-error is indeed normally distributed with variance picked from some stochastic distribution, in this case a normal distribution. This term however is normally used for models in which the volatility has more structure (e.g. the ARCH or GARCH models which are mentioned in the page).

</comment></comment></comment></comment></comment><comment><USER7> Mandelbrot and NN Taleb's claim requires non-Markovianness of the walks. Fractional brownian motion, for example, exhibits long-range dependence.

Also "randomness" is measured using Kolmogorov complexity, a quantity which is incomputable.

</comment><comment><USER8> " weak form of the efficient market hypothesis which states that: future prices cannot be predicted by analyzing prices from the past ..."

No. That is the _consequence_ of the efficient market hypothesis being true. The hypothesis itself is subtly different. Though most people miss it:

All information from past prices is in current prices.

That is a better formulation.

I spent five years studying it and I think it is robust if you remember that it takes time for information to be consumed. That is why HFT works, because it acts before the information can be processed.

<comment><USER8> "All information from in past prices is in current prices."

All information from past prices is in current prices.

Duh!

</comment></comment><comment><USER9> I did not read the link but Benoit Mandelbrot basically showed this 30 years ago. So what's the news?

<comment><USER9> Not sure why I get downvoted

[http://www.amazon.com/The-Misbehavior-Markets-Financial- Turb...](http://www.amazon.com/The-Misbehavior-Markets-Financial- Turbulence/dp/0465043577)

<comment><USER10> No doubt because you reacted dismissively to the title alone instead of engaging with the specifics of the article.</comment></comment></comment>


Tourist map of SSD drives - <AUTHOR> http://things.gnod.com/ssd_drives/

<comment><USER2> It would be good to get some sort of standardised score for performance on there too (as a sorting factor). And (in that vein) some PCIE SSDs added to the list.

<comment><USER3> If not performance, perhaps Amazon star ratings?

<comment><USER4> I trust arbitrary rating less than raw, testable numbers.

<comment><USER3> If a certain SSD gets a number of 1-star ratings that's way out of the standard deviation, I think that would be a pretty good sign the drive is a piece of garbage.

<comment><USER5> Or as often the case with Amazon reviews nobody has a clue what they are talking about

</comment></comment></comment></comment></comment><comment><USER6> This is interesting, but I wouldn't use it to guide buying decisions. Just buy an EVO or MX100; all the second-tier brands are just adding noise: [http://www.storagereview.com/it_s_game_over_for_most_consume...](http://www.storagereview.com/it_s_game_over_for_most_consumer_ssd_companies)

</comment><comment><USER7> See also PCPartPicker's graphs: [http://pcpartpicker.com/parts/internal-hard- drive/price-per-...](http://pcpartpicker.com/parts/internal-hard-drive/price- per-gb-vs-capacity/#)

</comment><comment><USER8> Cool, I'll probably make use of this later. Would be nice to have maybe a color gradient or alpha layer on the items to signify average review scores on Newegg or whatnot though. I don't buy a drive without reading those.

<comment><AUTHOR> Amazon does not provide the scores via their api. And they probably don't like it when you scrape them. So no easy fix for this one. But if there is interest in this project, I will look up other options like Newegg.

</comment></comment><comment><USER9> I was just looking for the flash drive graph from this site yesterday, and could not find it. What a great coincidence! I like the data-centric approach to shopping.

P.S. there should be a graph for ATM machines

<comment><AUTHOR> Happy to hear you remembered the flash drive map!

ATM Machines? Like this thing?

www.amazon.com/dp/B000UJHCSE/

<comment><USER9> That was just a lame attempt at a joke.

SSD Drive = Solid-State Drive Drive

ATM Machine = Automated Teller Machine Machine

</comment></comment></comment><comment><AUTHOR> Mapped the 200 most popular SSD-Drives by size and price today. Some interesting patterns emerged. Im not sure yet if they are artefacts or have some deeper meaning.

<comment><USER10> very cool way to present price comparison

</comment></comment><comment><USER11> I find the following to be more effective. [http://forre.st/storage#ssd](http://forre.st/storage#ssd)

<comment><USER12> Yeah plotting price/GB vs price seemed weird to me. Seems like GB vs price (or price vs GB) would make more sense, but price/GB vs GB is also better since you can find the size you're interested in on the x-axis, then look up to find the best value.

</comment></comment><comment><USER13> This is spectacular! Not only are consumers winners with this, retailer buyers would love it too!!!

</comment><comment><USER14> Wish there was a find feature to hilite matches. Hard to see where a particular model lives.

<comment><AUTHOR> Matches? You mean add a text search?

<comment><USER14> Something like that, so you can find where your current SSD lives.

<comment><AUTHOR> That's a good idea.

</comment></comment></comment></comment><comment><USER15> This is awesome. What are the chances you could expand this so that one could use amazon search terms and custom values to change the graph? That is, could it be altered so I can search for "top 50 laptops with at least 4GB of RAM?"

<comment><AUTHOR> I think that would only be possible when entering the data manually. Amazon does not really have standardized data of product attributes.

</comment></comment><comment><USER16> Any way you could add filters? e.g.: I want only 1TB drives that are SATA 2.5".

Also performance details (maybe read/write iops?)

</comment><comment><USER17> The results for amazon.de include one spinning-platter hard disk. Looks like it's miscategorized on their site.

<comment><AUTHOR> Thanks, thrown that one of the map!

</comment></comment><comment><USER18> Now I only wish there was an option to filter based on ratings and size of drive.

</comment><comment><USER19> Any chance you can add amazon.ca to the list? ^^;</comment>


Subscription Based Crowdfunding for SaaS - <AUTHOR> https://saascriber.io/ <comment><USER2> Hold on. Why does the "continue to website" after you've entered your email address go to maildb.io?

"MailDB is a new and easy way to find the email address of anyone on the web with just a domain name."

Is this just some sort of lead scraping vapourware to get people to submit their contact details to maildb?

<comment><AUTHOR> I'm sorry, this is a mistake!

I'm using MailChimp for building the email list, and I'm using the same MailChimp account as I do for my main SaaS, MailDB ([https://maildb.io](https://maildb.io)).

This is completely separate and has nothing to do with that site - I thought the settings were different for this specific list.

I'll double check now, thank you! And no worries, you will never be emailed regarding that site - this list is only for SaaScriber and this idea alone.

 _Edit_ : fixed, thank you again!

</comment></comment><comment><USER3> The investors page kills your credibility both with developers and with investors.

If this product does anything of value, it needs to proove it by using subscription based crowdfunding itself.

<comment><AUTHOR> The way I was looking at it, this is something we want to form a real team around and build in 4-6 months time.

While we could try to leverage our own idea, without having this site known and heavily trafficked, the subscriptions will be minuscule and probably not enough to achieve our goals - look at early Kickstarter. An influx of cash would help get the word out.

Plus, we're hoping to work with SaaS companies who sign up and see which products/ideas want to launch with us.

I do see your point though. Maybe this is something where it needs to start small, using the idea itself as proof.

<comment><USER4> It would let you better test the pain points for this approach by dogfooding the process yourself.</comment></comment></comment>


What do you use for CI/CD ? - <AUTHOR> At the moment we are using Jenkins however we had some debates that there are better alternatives (circleCi, codeship ...).<p>It would be great if you could share your set up and what works best for you. <comment><USER2> Gitlab CI with the gitlab multirunner container all on AWS ECS.

In this way you can run multiple concurrent test suites each in an insolate container.

<comment><AUTHOR> I'm assuming you can't use gitlab CI with github/bitbucket ?

<comment><USER3> No, but you can use GitLab.com or EE to mirror the repository on GitLab to run the tests.

<comment><USER2> Yep, this is the right way! Thanks,</comment></comment></comment></comment>


Democratic Republic of Congo Blocks the Internet - <AUTHOR> https://www.iafrikan.com/2018/01/22/internet-accesss-blocked-in-the-democratic-republic-of-congo-drc/ <comment><USER2> BBC reports the Internet being cut only in the capital [1]. Having trouble finding a reliable source for the broader claim made in this article.

[1] [http://www.bbc.com/news/world- africa-42766151](http://www.bbc.com/news/world-africa-42766151)

</comment><comment><USER3> What Cloudflare is seeing: [https://twitter.com/jgrahamc/status/955391830899789824](https://twitter.com/jgrahamc/status/955391830899789824)</comment>


Volunteering helps with anxiety and stress - <AUTHOR> https://motherboard.vice.com/amp/en_us/article/a37nvk/volunteering-is-the-best-kept-secret-for-mental-health-stressweek2017 <comment><USER2> I volunteer because the line in the front of the soup kitchen is a lot longer than the line for the volunteer entrance in the back. I may not get a huge feeling of accomplishment, because I'm doing quite menial tasks. But it's humbling, and its been said "never think flipping burgers is beneath you".

I also volunteer to play music for a religious service, which is a ton of fun and enriches a lot of people's lives. It's challenging and definitely provides an outlet.

Finally I give a charitable donation to a charity that I know is making a local impact and they manage their finances wisely. They give free food away to anyone no questions asked.

It's the combination of these things (giving my time away giving, using my talents, sharing some money) that bring me a lot of satisfaction about my life.

The people in Hacker News are in a special position because we're typically well-off and there are many talented people on here. I'll never chastise anyone about giving, but I wish everyone would.

</comment><comment><USER3> The key is to find a cause to help in which you truly recognize the blessings/fortune/privilege/opportunities you have.

I've volunteered with animal shelters, geriatric hospitals, youth sports, and with veterans. Animal shelters were fun because I love animals, but they didn't humble me like working with veterans and the elderly. Every time I work with those groups, I learn more about life, and get insight into ways I could change how I'm living to be a better influence on society. I think that's the key - at least, that was the key for me.

<comment><USER4> My wife an I rescue dogs (and have cats from past rescues). We tend to specialize in French bulldogs, but may get an English soon. The ones we get are usually puppy mill throwaways, and have a lot of initial health problems. The vet bills definitely eat into our disposable income, but nothing else I have ever spent money on has made me happier.

If anyone gets a chance read up on puppy mills and how horrible they are. Go to a shelter and rescue an animal today.

<comment><USER5> And stop buying purebreds, especially trendy or short-snouted ones:

[https://www.washingtonpost.com/news/animalia/wp/2016/08/02/w...](https://www.washingtonpost.com/news/animalia/wp/2016/08/02/why- breeding-bulldogs-is-borderline-inhumane/)

<comment><USER4> Completely agree. We have always rescued animals either from the pound or street (our 2 cats were found as kittens outside when my wife was capturing feral cats to get them spay/neutered). Once my wife found out more about puppy mills, and how the dogs were treated she decided to do whatever she could to combat mills. Until the laws are changed though, there isn't that much that can be done to stop them. The best an individual can do is get pets from the pound or other legit rescue, and never from a pet store (some stores partner with pounds which is fine).

</comment></comment></comment></comment><comment><USER6> Seeing as we're all giving anecdotal evidence, I personally didn't feel much of a benefit. I spent two years helping a disabled sports charity. It was enjoyable being there and I think the people organising it are wonderful.

However it didn't make me feel much better about myself or life in general. I would occasionally simply not want to go.

So I think at the very least it's no perfect solution.

<comment><USER7> Volunteering to teach non-violent communication in a prison _increased_ my anxiety and stress.

It was worth it, though.

</comment></comment><comment><USER8> This is so true. One of the benefits I found most surprising about volunteering is how much easier it is to make quality connections with people while working on a volunteer project. I suppose the simple act of signing up to volunteer probably biases the population towards extroverts but it seems like people feel more comfortable opening up while being altruistic. I thoroughly enjoy connecting with people in this way.

<comment><USER9> Keep in mind it is highly likely those connections won't survive once you are out of volunteering. Source: my experience with various volunteer activities (sometimes as organiser, sometimes as participant) and discussions about it with various people involved in such activities.

</comment></comment><comment><USER10> Volunteering is one of those things that is wonderful idealistically speaking but not necessarily in reality.

All too often these organizations are sought by horrible people to socialize with fellow horrible people.

My point is not that volunteering is bad (it isn't!) but that it's pretty much like any other interpersonal activity. Something magical can result from it but all too often (if you're not desperate to meet people) you waste your time with a bunch of ruthlessly selfish people.

</comment><comment><USER11> Does anyone have advice on how to do skills based volunteering as a software engineer? Something where my software skills help a good cause. I'm currently volunteering teaching kids and teens to code every few weekends and it's okay, but looking for something higher impact if possible.

<comment><USER12> Are you volunteering with schools? I've been thinking about doing something similar, but haven't figured out the best way to get started.

<comment><USER11> Currently an organization called "Canada Learning Code".

</comment></comment></comment><comment><USER13> It's a nice thought, but the mental health angle seems pretty thin on the ground here. The article eschews any actual mention of illness, clinical psychology, or psychiatry, and the volunteering-specific evidence it cites is preliminary at best.

One interesting thing is the mention/citation that helping other people being built into a job doesn't seem to have similar benefits. Another study found a similar difference for the psychological benefits of physical activity [1]. I imagine there's a significant cultural bias in the Protestant(ish) Anglosphere against "leisure is good for you", though.

[1] [https://www.ncbi.nlm.nih.gov/pubmed/21037212](https://www.ncbi.nlm.nih.gov/pubmed/21037212)

</comment><comment><USER14> Absolutely! This was my experience too. I'd like to add that it helps you to get out more and focus less on yourself, as often we over-analyse and self- doubt too much.

And no real commitment. No performance trackers, no material gains, just your best shot. Children are really awesome too - it's amazing how they're so small and naive, yet super curious and active. Really inspiring. If you're interested, I recommend checking with your local school, it's very likely that they're short of hand. A few hours a week is enough!

</comment><comment><USER15> Now, if only volunteering in the open source space worked this way consistently.

Surely lots of open source voluteers have good experiences and forge lasting connections. The more in-person events, videochats and phone calls you participate in, the better.

However, communication through low bandwidth text-only channels tends to lack empathy. People who are not terrible in person are terrible online. And since a disproportionate amount of open source communication occurs online, many volunteers have bad experiences.

</comment><comment><USER16> on a related note, i recall Peter Drucker writing about how many people, executives in particular, often talk about volunteering or "giving back to the community" once they retire.

but Drucker noted that in his observation, executives who did not volunteer earlier in life were just fooling themselves, they rarely followed through with that "promise" once they retired, they failed to contribute much of anything after their career.

</comment><comment><USER17> I don't do much volunteering aside from helping take care of cats at a cat shelter every two weeks, but I do find that it helps relieve stress. I started doing it after my own cat was very sick and almost died. The fact that he's still with us is nothing short of a miracle and it feels satisfying to be able to help care for other cats after getting so lucky (knocking wood) with my own.

</comment><comment><USER18> Several years ago I was in need of something to do, the only problem being how to find volunteer opportunities besides mindless spaghetti-on-the-wall searches. Luckily enough I found volunteermatch.org and within a month I was in action and with a website I could return to for finding more.

After I started, I experienced similar results as stated in the article.

</comment><comment><USER19> I am somewhat saddened that we have descended to a point of moral decay where the notion of the positive results of generosity is now breaking news.

<comment><USER20> > we have descended to a point of moral decay

Really? What is the time in the past where things were better? When you were younger, I presume?

<comment><USER19> Actually I was feeling pretty good about two years ago it is much worse now.</comment></comment></comment>


Ask HN: Web apps, Mobile Apps or Desktop App Developers - What's your Dev Setup? - <AUTHOR>

What kind of development infrastructure do you (and your team) use to create, test, build and deploy your product? <comment><USER2> Most do web app development (full stack) but once a while have built desktop (Ubuntu) and mobile (Android) apps as well.

Technologies I work with: Node.js, Scala, Java, Python, Android

OS: Ubuntu (Xubuntu) 12.04, Windows 7

IDE: IntelliJ IDEA - Consistent across various technologies and perfect for large code bases

Text Editor: Sublime Text 2 - Pretty damn awesome, even with large files

Terminal: Terminator

</comment><comment><USER3> I'm a web and mobile (Android) app developer.

Hardware: Last gen MacBook Pro (15" with matte screen), iPhone 4.

Operating systems: Mountain Lion, iOS 5.

Development tools for web apps: Terminal.app, bash, vim (for Sinatra apps), TextMate (for Rails apps), git, GitHub, Ruby 1.9.3, Sinatra (with DataMapper and Haml), Rails, jQuery, Google Chrome. In my experience, vim is better for projects with fewer or larger files, and TextMate is better for projects with more or smaller files.

Production environment: Heroku Cedar stack, Heroku Postgres, Amazon RDS, Amazon S3, Amazon CloudFront, MemCachier (through Heroku), New Relic (through Heroku for profiling/performance info).

Development tools for Android apps: Eclipse, git, GitHub. I hate Eclipse, but I haven't found a better way to develop Android apps.

</comment><comment><USER4> WP7 (mobile) app developer:

Windows 7 with VS2010, solarized theme.

Calcium SDK for MVVM development.

Unit tests: Jeff Wilcox's Silverlight Unit Test Framework (WP7 SDK doesn't have unit tests built-in).

Test device: a Lumia 800. The emulator doesn't work well on my machine.

Source control: git (command line, no VS plugins) and GitHub.

I occasionally also use my Macbook 13" with TextWrangler for writing some code when I can't access my main PC or when I want to make some fast fixes.

</comment><comment><USER5> I'm a web app developer, mostly. I've recently wrote about my setup & toolbox (<http://goshakkk.name/my-toolbox/>). Shortly:

Macbook Air 13" (OS X ML) + Thunderbolt Display 27".

Terminal: Terminal.app, zsh, oh-my-zsh. Solarized theme.

Editor: Vim, Janus, bunch of plugins. With Solarized theme too.

Technologies: Ruby, Rails. MongoDB, Redis, Node.js, Backbone, CoffeeScript.

Deployment: Heroku.

</comment><comment><USER6> Mobile app developer.

Hardware: Lenovo T400, 8GB RAM, Core 2 Duo (due upgrade) + 50" Samsung Plasma display.

OS: W7 x64, OS X ML.

Platforms: Android, iOS, Blackberry Playbook Android emulator.

Technologies: Java, Objective-C, PHP, JSON, Google App Engine.

Unit testing on Android: Robolectric.

IDE: Eclipse and Xcode.

Deployment: Google Play Store, App Store.

</comment><comment><USER7> Web and mobile developer

MacBook 17'' Editor: SubEthaEdit, XCode for iPhone, Eclipse for Android Languages: Rails, Redis, Objective C, Python Infrastructure: EngineYard

</comment><comment><AUTHOR> A clarification - Web app developers, Mobile Developers _or_ desktop developers, what's your dev setup ?</comment>


A first step toward more global email - <AUTHOR> http://googleblog.blogspot.com/2014/08/a-first-step-toward-more-global-email.html <comment><USER2> What are the security implications?

Does anyone know how canonicalization is handled? Does every mail program need to know how to precompose/decompose etc? How do you protect against impersonation using look-alike letters?

This is, as far as I know, not yet a solved problem even at the domain name level[0], and it's likely to open a whole new can of worms at the account level.

[0] [http://en.wikipedia.org/wiki/IDN_homograph_attack](http://en.wikipedia.org/wiki/IDN_homograph_attack)

<comment><USER3> One small fix would be to mark non-latin characters in an email address.

<comment><USER4> I have a "non-latin" letter in my surname, and I find highlighting it as somehow wrong or suspicious offensive.

</comment></comment></comment><comment><USER5> This is very useful for non native english speakers that uses other than latin characters. I have to repeat word by word and check what people wrote when I need to tell my email address.

If my address was a word in hebrew, or something people can pronounce and write it would actually help, saving time and avoiding misspellings.

But this should be an alias, or it should be very easy to create one, I already have my email for some years, I don't want to create a new one and handle two accounts unnecessarily.

Also lets say someone creates his email address in his native language, so it is easier to give his address to his friends at school, later in life he wants to give his address to a VC from US he met. even if he gives a business card with his email printed in clear letters, this western person can't even spell or even type, unless he could copy/paste the address he won't be able to send him an email.

<comment><USER6> Creating a second Gmail account and setting it up to forward to your primary account is already a pretty straightforward procedure, so that probably won't be much of a problem. (There _would_ presumably be a need for some user education on how to do that, though. And I agree that it might be nice to have a built-in "aliases" feature in Gmail, regardless of internationalization.)

</comment></comment><comment><USER7> Quick question; when characters appears as blocks on my screen, they can still be copy-pasted, right? I would imagine so since the required font is missing but the data, which is the important part, is still available.

<comment><USER8> Fist of all, I am quite aware that funny remarks are frowned upon on HN and I agree to this policy. That said, I think that [http://www.bash.org/?244321](http://www.bash.org/?244321) is still the correct and informative answer.

<comment><USER9> That's not relevant at all to that person's question. Did you link the right thing? That is the generic password "joke."

</comment></comment></comment><comment><USER10> Maybe this is a good thing but I wouldn't have an email address with accented characters (and my name contains two of them). It would be quite awkward if I gave someone my address and he/she couldn't simply type it with his/her default keyboard configuration.

<comment><USER11> Right, that's a legit concern. How about an app that quickly loads up all the language keyboards in the OS and lets you pop into one temporarily for the purposes of typing one thing in notepad or just straight into the clipboard?

<comment><USER12> > How about an app that quickly loads up all the language

That sounds convenient.

/s

<comment><USER11> Not all at once... just retrieves them for searching and really quick usage. launch the app, type in "m....a....n.." and hit the mandarin option. Your new Chinese friend types in his email address and you pop it into a contact. No settings need to be changed or anything. Is that really such a weird idea?

<comment><USER12> > Not all at once... just retrieves them for searching and really quick usage. > launch the app, type in "m....a....n.." and hit the mandarin option

And now you have two problems. Why would you be searching for mandarin which is the anglicised-ASCII translation for Guānhuà. More correctly, you should be searching for ㄍㄨㄢ ㄏㄨㄚ.

<comment><USER11> One would assume that if the OS is set to English, the keyboard options will be listed in English, and if the OS is set to ㄍㄨㄢ ㄏㄨㄚ, the keyboard options will be listed in ㄍㄨㄢ ㄏㄨㄚ. This isn't a problem at all if you know the name in your own language for the language you are seeking.

</comment></comment></comment></comment></comment></comment><comment><USER13> For people who are wondering, the example Japanese email address translates to "email@example.com" (Takeshi is a male, Japanese first name).

</comment><comment><USER14> I just tried to create a gmail account with non-latin characters and received this message from Google:

        Please use only letters (a-z), numbers, and periods.

EDIT: I guess I missed this crucial sentence

        Of course, this is just a first step and there's still a ways to go. In the future, we want to make it possible for you to use them to create Gmail accounts.

</comment><comment><USER15> I am a Dane. We use the latin alphabet plus æ, ø and å. Some names, like Søren or Åse, can't be written in pure ascii, but the people who have these names tend to just have addresses like email@example.com or firstname.lastname@example.com. It seems preposterous for me to risk breaking the web over something so relatively trivial.

Heck my name is ascii compatible but it isn't available.

<comment><USER16> A lot of cases of "accented characters" are much simpler, compared to e.g. Chinese or Japanese, where the mapping is not 1-1 (a given kanji could have multiple readings, or for Chinese, pinyin is not 1-1). There's also a number of people who don't know the ASCII mapping for their language. Chinese can be written with bopomofo or 5-stroke input, for example. There are programs for input of indic languages that use visual keyboards.

Let's say the Internet had been invented in Japan instead of the US. How would you feel if people told you that you had to write your name in katakana everywhere? As another commenter mentioned, internationalization is here to stay, and if we want to expand to the next few billion users it's even more important. FWIW internationalized usernames are already available on a number of non-email platforms (Weibo as a prime example). For email to remain competitive, it's important to keep up in the internationalization space.

<comment><USER17> Well, internationalized usernames are "available" on weibo in the sense that your displayed "name" can be anything you want. But you don't log in with your displayed name; it's an arbitrary bit of account data, and is changeable whenever you want. You log in with an email address, which is how the system identifies you.

(checking now just to make sure, I see that weibo allows three options for logging in: an email address (not internationalized), an account number (not internationalized), and a phone number (not internationalized))

<comment><USER17> I admit I don't understand the downvote. Email already has internationalization in the same sense as weibo does. You might receive email from me as 'From: Michael Watts <firstname.lastname@example.com>'; the email address doesn't support arbitrary characters, but the name does (I've received email from '"=?gb18030?B?w8DIy7nY?=" <firstname.lastname@example.com>', which worked out to a displayed name of 美人关). Similarly, if I wanted to display 美人关 as my handle on weibo, I could do that, but I wouldn't be able to use it to identify my account.

</comment></comment></comment></comment><comment><USER18> Interesting.

Question: What do you guys think it'd take to get the other N% of email providers, clients, servers, and whatnot onboard?

Is this an ipV6-like situation?

<comment><USER19> Hardly.

Almost all email moves between only a handful of companies. Google, Microsoft, Yahoo, Facebook, Apple. Between them they dominate the landscape. It only takes a handful of engineers and product managers at these companies to decide "let's do this" and pretty quickly such email addresses can become a reality for at least person to person mail.

Of course for them to become usable for signing up to websites, mailing lists etc, will take much longer. But people may not mind having two email addresses if they can put one on their business card.

<comment><USER12> > Almost all email moves between only a handful of companies. Google, > Microsoft, Yahoo, Facebook, Apple

That's pretty naive. You need to step outside your comfy bubble.

Out of all the small businesses on the internet, how many of them are still running an email server under someone's desk?

And not to mention, what about all the client-side javascript out there that parses email addresses on web forms? Think about all those throw-away regexes to parse email forms on websites. I've seen a lot of them break just on these new TLDs - and they are ASCII!

</comment></comment></comment><comment><USER20> Out of curiosity, how would I send an email to somebody who's address is in a language I can't type?

<comment><USER13> In addition to copying the address, you can also use a virtual keyboard to map your existing keyboard to the keyboard of another language. I know that OS X has this built in, and I am assuming Windows does too.

<comment><USER20> I have a business card and the email address is in a language I don't know how to type (having a virtual keyboard doesn't mean I know how to reproduce the actual characters, which is true for many languages).

Or worse, somebody is trying to give me their address over the phone.

Here's an example: Try to figure out how to type:

宮本茂@任天堂株式会社.com

Here's another I'm pretty certain I can't figure out with a virtual keyboard.

প্রিয়াংশু.চ্যাটার্জী@बॉलीवुड.com

How about mixed languages? Like the case of a foreign worker assigned to another foreign department in yet a third country.

김기덕@बॉलीवुड.ประเทศไทย.com

I'm not going to tell you what languages these are in. Just pretend you were handed these on business cards or saw them on a slide deck at a conference and can't get a digital copy. Let me know how it goes.

</comment></comment></comment><comment><USER21> I wonder how gmail will handle the type-ahead for addy's in diff languages. Especially since I only type in US English.

</comment><comment><USER22> Good luck typing in those characters, or even knowing how to pronounce them if you had a "sounds-like" index.

</comment><comment><USER23> So, about those email validation regexes...

<comment><USER24> A refresher, for people who might not have come across this before: parsing even a limited subset of all possible email addresses with a regex is hard

[http://www.ex-parrot.com/pdw/Mail-RFC822-Address.html](http://www.ex- parrot.com/pdw/Mail-RFC822-Address.html).

</comment></comment><comment><USER25> Does it mean Gmail now support creating non-latin email addresses for Google account? I tried creating one now (in US) and got: "Please use only letters (a-z), numbers, and periods."

</comment><comment><USER26> Does it support emoji?

<comment><USER13> If it supports all characters defined by the Unicode standard, then it should. However, it wouldn't surprise me if Google blacklists the non-language symbols (like the emoji, shapes, etc.) because it is not within the scope of their goal. You can't read emoji like a word, so it wouldn't work well with the goal of making email addresses readable.</comment></comment>


Announcing VM Import for Amazon EC2 - <AUTHOR> http://aws.amazon.com/about-aws/whats-new/2010/12/15/announcing-vm-import/ <comment><USER2> Amazon continues to amaze me with what they are setting up.

I've had my problems with some of their policies, but overall they are really doing a bang-up job. They are doing what any good business does: they are making it really hard to pick any other alternative.

I really like the idea of moving VMs in and out of the cloud. Can't emphasize enough how neat that is. If other vendors start implementing this, we could have true infrastructure portability. Very cool.

<comment><USER3> One problem I've had at every place employment I've worked at in the last few years is getting a VM which matches production to use as a sandbox. With this kind of thing it would make that a snap. "What's the difference between this and production?" "Nothing. It's the same image."

<comment><USER2> Yep

Not only that, but many enterprise systems are not just a server, or even a couple of servers, but a cluster of servers of various configurations.

With good bandwidth, you could pull the entire bunch down locally to use in dev and testing. That's unheard of as far as I know (although it has been the holy grail for some time) Businesses that start and grow in an environment like this are going to have significant advantages over older businesses.

</comment></comment></comment><comment><USER4> Hopefully VirtualBox support will be added at some point. It's amazing what Amazon is doing with AWS.

</comment><comment><USER5> They announce it when only support Windows Server 2008 SP2? Seems rather underwhelming when that is all you can import.

Building an AMI or cooking with chef/knife is easy enough, but I know a lot of people who'd like that kind of simplicity. Just too bad it is only one OS.

<comment><USER6> It's just a starting point.

And I bet they focused on Windows because there aren't as many great provisioning tools as there are for Linux.

</comment></comment><comment><USER7> Out of curiosity, how do people currently build a custom new EC2 image? Spin up a base instance and then run a script?

<comment><USER8> We start with a base image and then use puppet to manage anything that needs to be installed/setup.

<comment><USER9> I don't understand why more people don't use bcfg2. Its awesome for this purpose (and I think better than puppet or chef.)

<comment><USER10> Then educate people! Don't just say it's better, say why!

</comment></comment></comment></comment><comment><USER11> I'd like to see export.

</comment><comment><USER12> Interesting that they support importing VMware images - I was under the impression that EC2 used Xen.

<comment><USER13> Converting between image types isn't too big of a deal; qemu-img can convert between most of your common on-disk formats (including VMDK), and wouldn't be hard to extend to whatever they're using behind the scenes.

<comment><USER14> I wonder if I can convert my old dusty Win XP Parallels image and VM-import it into EC2... That would be pretty cool!

<comment><USER15> We support specific versions of Windows 2008 SP2 at present, with plans to add other versions and other OS's over time, with a focus on server OSs (as opposed to desktop).

<comment><USER16> This is an amazing change that hugely reduces the cost of transitioning to AWS. We're deploying VMs on Xen at our university and have talked about AWS but there's a high 'coefficient of static friction'. This could change that whole discussion. Well played.

</comment></comment></comment></comment></comment><comment><USER17> Beautiful</comment>