Python dicts are now ordered - <AUTHOR> https://softwaremaniacs.org/blog/2020/02/05/dicts-ordered/en/ <comment><USER2> I live in the Python bubble, so I haven't realized until this post that so few people knew about that.

This post is massively popular despite talking about a feature we had since Python 3.6, in 2016, that was posted on HN at the time and that is featured in most popular tutorials.

A good reminder that most of the world doesn't revolve about my favorite language. And that information is not that fast to spread.

<comment><USER3> > about a feature we had since Python 3.6, in 2016

No, you had that feature in one implementation. Now it's in the language specification. That's vastly different because only now you can rely on it without fearing it can go away with the next release.

<comment><USER2> First, cPython is 99% of Python deployments. So much that if a script works on it but not on another implementation, people often consider the later broken.

As this post proves that most people don't even know about this feature, you can be pretty sure the vast majority of people don't know about pypy, micropython, etc.

Secondly, even if you want to nit pick, Python 3.7 made it official more than one year ago. We are currently in the 3.9 alpha.

In fact, 3.7 didn't touch the implementation, just merely declared "yep, good idea, let's keep it that way".

<comment><USER4> There's a great talk by David Beazley[0], which I don't remember its title right now, where he bashes all previous python versions except the latest at the time[1], which was 3.6. And he says "since I'm not a core developer, I can tell you this: rely on dicts being ordered so much that eventually they'll make it official". Guess he "won" in the end ;)

[0] Well, he usually gives great talks anyway

[1] Alright, he literally started many talks doing that

<comment><USER5> Raymond Hettinger does or did a few talks on dictionaries too. He covers the history of dictionaries in python and how they evolved. It was mentioned in the article's comment section.

[https://www.youtube.com/watch?v=p33CVV29OG8](https://www.youtube.com/watch?v=p33CVV29OG8)

</comment></comment></comment></comment></comment><comment><USER6> This is awesome, because the ordered map is the best data structure out there for easy & predictable programming, possibly only barring the array.

There's so many cases where it's a benefit for map entries to retain order (and none where it's a problem). PHP really got this one right (and immediately messed it up by mixing ordered maps with arrays into a big soup, but hey, PHP). And so did, JS, sorta-kinda-by-accident.¹

When I went from PHP to Ruby back in the day, Hashes not being ordered definitely was my biggest gotcha. Everything about Ruby (except the deployment) was nicer, but the hashes... I've spent serious amounts of extra thinking just to make code work that would've worked out of the box in PHP.

Yay for ordered maps! Every language should have them, they're the best! There's just so much ergonomics packed in such a simple API.

1) JS objects are ordered maps, _iff_ the keys are strings that cannot be parsed as an integer (really) (yeah it's nuts) (but still better than unordered maps!!)

<comment><USER7> Could you explain some examples of what this is useful for? What sort of algorithms or operations do you have in mind where you both want insertion order, and also key-based lookup in the same data structure?

I've made heavy use of all kinds of maps, and of queues and channels and arrays, but I don't recall ever noticing a situation where I wanted the properties of both mixed into the same data structure.

I'd love to learn more about useful tools to add to my toolbox!

<comment><USER8> For "algorithms", I mean, some really do care about insertion order. Like idk, if you have a priority queue, then you generally want FIFO ordering when the priority is the same? It like a pretty obvious desire in most cases... imagine a thread scheduler or a packet scheduler or what have you.

But generally it's about determinism and avoiding loss of information, not just whether a particular algorithm needs it. For example, you'd want serialize(obj) and serialize(deserialize(serialize(obj))) to produce the same output, otherwise you e.g. might not be able to cache stuf. But for a data structure like a hashtable, it's pretty tough (not logically impossible, but rather pointlessly difficult) to make that happen without preserving insertion order.

As another example, it's incredibly handy for a user to see items in the order in which they were inserted. Like say you're parsing command-line arguments, and the command is ./foo --x=y --w. If the user sees {x: y, w: None} then that tells them w was passed after x. That can be extremely useful for debugging; e.g. maybe you expected the caller to specify w earlier, in a different context and for an entirely different reason. Seeing that it came afterward immediately tells you something is wrong. But when such information is lost it's harder to debug code.

<comment><USER9> If --w can be specified in two places for "an entirely different reason", then an unordered data structure is simply inappropriate, full stop. That's not a question of debugging, that's a question of correctness.

</comment></comment></comment></comment><comment><USER10> Am I the only one that thinks this is a stupid decision? This will silently break code that starts to rely on this behaviour that gets executed on Python3.5 and lower. I would consider changing how a builtin works to be a major breaking change. It would have been fine if this was a change between 2 and 3 but on a minor version? Thats insane.

<comment><USER11> I'm not clear on how this break existing code.

Code that assumed it was arbitrary, would expect to handle any arbitrary order, including a happens-to-be sorted order.

Code that assumed it was random, like actually inserted by random(), was already broken, because that simply isn't the case.

Code that assumed the order would stay constant was relying on implementation- specific behavior, and could potentially break on any version update; as with any reliance on implementation-specific behavior, you'd break if the dictionary code ever got touched -- even if it were for a bugfix.

Code that ordered the dictionary keys before iterating are now slightly innefficient due to extra work of sorting a sorted list.

<comment><USER12> It doesn't break existing code. Code written for Python 3.7 might break on older versions of Python

<comment><USER11> You're right, I read the gp too quickly.

But in the case of downgrading, I'm fairly sure there's a number of other breaking changes that can't trivially downgrade minor versions. Like f-strings were only introduced in python3.6 as I recall. Async keyword only exists as of 3.4 as well I think?

<comment><USER13> Introducing things is different than changing things.

<comment><USER11> Sure, but you can't safely take everything from a higher version to a lower version in any case; if insertion order became gauranteed due to a bugfix, and wasn't backported, you'd be in the same boat.

The only way to consistently code cross-version is to start with the lowest you plan to support (assuming the higher versions are actually backwards- compatible).

Does any language gaurantee that code is both backwards and forwards compatible?

<comment><USER14> Issue seems to be silent incorrect behavior, what happens if you attempt to run python code containing f-strings using an older python version. Does it raise an exception? That's good! What happens now if you write code for 3.7 which takes advantage of the new ordering and someone grabs it from your repo and runs it using 3.2, it would happily give incorrect results and noone is the wiser.

<comment><USER15> If you expect this situation you can assert the language version.

<comment><USER16> But the whole point is that some developer won't expect that someone would run their code on an older Python, isn't it?

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER17> This is an amazing contribution to the language. A mixture of speed and convenience, probably made by volunteers. As for people criticizing a change to what use to be a non-deterministic ordering of a dict iteration; I don't know what to say to them, other than, are you serious? There are people out there who are working for us, they work for free and they did some heavy lifting to give us this. They might read what you wrote and think, "Why bother? Maybe I should spend my weekends playing with my kids instead."

<comment><USER18> Other languages don't generally have special order guarantees about standard maps. This seems very idiosyncratic.

<comment><USER19> Golang map iteration is returned in random order specifically to make sure that people don't rely on the order.

I think this feature says a lot about the philosophy of Python vs Go.

<comment><USER20> Sounds like a fast and idiomatic way to shuffle a deck of cards is then to convert to a map and back.

<comment><USER21> Convert a deck of cards ([]Card?) to a map (map[Card]bool?) and back just to shuffle? That's unlikely to be faster or more idiomatic than a straightforward implementation of the Fisher-Yates shuffle[1]. Try writing the code to do it both ways and compare.

[1]: [https://en.wikipedia.org/wiki/Fisher%E2%80%93Yates_shuffle](https://en.wikipedia.org/wiki/Fisher%E2%80%93Yates_shuffle)

<comment><USER22> I think "idiomatic" would be using rand.Perm, which implements the Fisher%E2%80%93Yates shuffle. But aside from whether it's idiomatic, converting to a map isn't random enough.

<comment><USER22> With Golang 1.11, I get orderings like this. S is spades, H is hearts, D is diamonds, and C is clubs, because HN eats the Unicode characters I was actually using.

         S9  SJ  SK  HQ  D9  S4  S5 S10  HJ  S8  H6  DA  D2
         D4  DQ  C6  C8  CJ  H3  H9  DK  C3  CQ  SA  S2  S3
         S6  SQ  H4 H10  D8 D10  C4  CK  S7  H7  D5  D6  D7
         C7  H2  HK  D3  DJ  C2  C5  C9  HA  H5  H8  CA C10

On average you would expect about one card in a shuffled deck to be (cyclically) followed in the deck by its succeeding card, and on about one out of 50 shuffles, that card would be followed by its successor. Here we have S4 S5, DA D2, SA S2 S3, and D5 D6 D7.

This is very strong evidence of bias.

I'm interested to hear if my Golang can be made more idiomatic, or if there are bugs in it: [http://canonical.org/~kragen/sw/dev3/mapshuffle.go](http://canonical.org/~kragen/sw/dev3/mapshuffle.go)

In particular it seems like there ought to be a less verbose way to express the equivalent of the Python list({card: True for card in deck}) in Golang.

</comment></comment></comment></comment></comment></comment></comment><comment><USER23> Great decision IMO. I remember maintaining a bunch of Python code that we supported on both OSX and Windows. Out of all the platform-specific bugs we had (where it worked on one OS and not the other), one of the most common causes was code that relied on a certain key order. And we knew that relying on key order was bad, we're supposed to use things like OrderedDict, blah blah blah. It was still a really easy mistake to make, just like it's really easy to have memory safety bugs in C.

At some point, if a human error is common enough, it makes pragmatic sense to change the design so the error is impossible.

<comment><USER24> The Go designers went the other way (as they often do):

> When iterating over a map with a range loop, the iteration order is not > specified and is not guaranteed to be the same from one iteration to the > next.

Actually it is not only "not guaranteed to be the same", the runtime actively makes sure that the iteration order is actually different so you don't even start to rely on it...

<comment><USER25> That seems like it would be a performance hit to actively make it different? That seems like a very weird and strange design decision if true

<comment><USER26> It's a very small performance hit because it doesn't do the work to be _uniformly_ random, just "not always the same". If you think about how you have to iterate through a hash table or similar data structure anyhow, it's either O(1) or O(log n) paid once per "range" on the data structure, which is dwarfed by the actual act of ranging on the data structure.

Go's philosophy is also definitely willing to pay that price to avoid a large class of known bugs that has hit all kinds of code bases. It is not about being the fastest language. As compiled languages go, it's solidly middle- tier, and not likely to go up much from there. (Among the C-style compiled languages, it's low-tier on performance, at around half the speed of C in general. However there's enough compiled languages like Haskell that are still generally relatively slow so that Go is mid-tier for compiled languages over all.)

</comment></comment></comment></comment><comment><USER27> I'm glad to see other languages finally catching up to PHP. I'm joking (kind of) but after a lot of years of doing this, I've begun de-prioritizing pure abstractions and favoring the way that humans tend to do things on their own.

Technically this is along the lines of the worse-is-better philosophy. The single biggest cost in software development is friction. Performance, size, etc are all less important, because they become less important with each passing year as computers grow more powerful. And along those lines, I think that silly concepts like assigning letter names to drives in Windows, or having a paltry few registers in x86, made those architectures more accessible to the masses than the more generalized Unix and Mac platforms. Not that I completely agree with that, just, it's all I can come up with (other than cost and familiarity) for why people are so attached to worse-is-better paradigms.

Preserving insertion order has some small cost associated with it, but that's going to be dwarfed by the cost of bugs introduced when junior devs are surprised by unordered dictionaries. And the pedantic people can just use an unordered dictionary manually since they are already converting pure abstractions to whatever imperfect implementations are provided by languages anyway.

<comment><USER28> >Performance, size, etc are all less important, because they become less important with each passing year as computers grow more powerful.

That isn't as true as it once was. We are running up against fundamental limits and people are even starting to talk about the environmental impact of computing.

<comment><USER29> It should be noted that the ordering of python dict occurred as a side-effect of a new implementation with better memory profile (and as good or better performances).

</comment></comment></comment><comment><USER30> "Ordered dict" is ambiguous, and from the title I thought that key order was meant. Reading the article, I see that it's actually insertion order. Which makes much more sense. Key order would have been a much more significant change.

<comment><USER31> Not to mention there's often no meaningful way to order by key value since keys can be any hashable values -- so things like this break this idea:

        _dict = {}
        _dict.update({MyObject: 3})
        _dict.update({'MyObject': 3})

There's no ordering over _most_ hashable values since they span multiple types, so insertion ordering is the only sane way to do it.

<comment><USER29> A sorted map would be tree-based and require its keys to be orderable but not hashable. For instance Rust's HashMap has its key bound on Hash + Eq, while BTreeMap's is bound on Ord.

They're different data structures, with different use cases and different requirements.

> There's no ordering over _most_ hashable values since they span multiple > types, so insertion ordering is the only sane way to do it.

Python 2 actually had total ordering of all values. The result was usually stupid but it was there.

<comment><USER31> >The result was usually stupid but it was there

This is what I meant by "sane" in my comment. It's _way_ more meaningful to the user if data's insertion-ordered.

</comment></comment></comment></comment><comment><USER32> Where "now" dates back to 2018? [https://docs.python.org/3/whatsnew/3.7.html](https://docs.python.org/3/whatsnew/3.7.html)

<comment><USER33> Came to say the same.. insertion order of dict has been here for years for 90+% of Python users (CPython implementation) and a part of the language spec also for almost as many years.

I guess it's good to spread awareness to HN readers who apparently were unaware, but the headline is very misleading.

<comment><USER34> 90% seems optimistic. I don't think that many Python projects are running the most recent version in production. From personal experience, upgrading to 3.7 was especially a pain because of packages that used "async" for variable/kwarg names.

<comment><USER33> That's a very good point, in that case even 50% would probably be optimistic.

We can probably attribute this article's novelty to the slow adoption of Python 3. In that way it's probably a good thing that it's such a popular topic, even if it is old news.

</comment></comment></comment></comment><comment><USER35> I still see dict ordering as an implementation detail; not a technical one but a descriptive one. If you want to rely on insertion order, use collections.OrderedDict. It communicates your intention far better, and there should be no overhead.

<comment><USER36> OrderedDict is less efficient. It's best to replace usage with basic dict where possible to improve efficiency. There is one obscure feature of OrderedDict that isn't in the basic, so it can't always be swapped.

<comment><USER35> Really? I would have figured they'd just do something like:

        class OrderedDict(dict):
            pass

(Plus a little bit of API shimming.)

<comment><USER37> No, OrderedDict has it's own C implementation which was created just before it was decided that dict would preserve order across iteration.

Further there is a big difference, regular dict preserves order across iteration but OrderedDict treats order up to equality.

I.e. this returns True:

        {1: 1, 2: 2} == {2: 2, 1: 1}

Where as this returns False:

        OrderedDict({1: 1, 2: 2}) == OrderedDict({2: 2, 1: 1})

To make that difference speedy it needs to be done on the C level.

<comment><USER29> Also ordereddicts provide methods to move items to the start or end, and remove items specifically at the start or end, not so for regular dicts.

<comment><USER36> dict has popitem for removing at the end. That used to be arbitrary, but now it (de facto) means last-inserted.

</comment></comment></comment></comment></comment></comment><comment><USER38> Cool, but, this just encourages people to rely on hashmaps being ordered by insertion. The nature of a hashmap is not one where you rely on the ordering. This is going to trip up newbies who then move to other languages.

Some comments on the post don't seem to understand the changes, for example, the C++ comment saying that std::map is ordered, so this isn't a strange change. The difference is that std::map is a red-black tree implementation, which has a traversal order, not a hashmap with insertion order. So there is already confusion.

</comment><comment><USER39> Brandon Rhodes gave a great talk at PyCon 2017 about the evolution of dictionaries in Python.

I really appreciate the people who dive deep into these implementation details, and continue to optimize the languages we all use.

[https://www.youtube.com/watch?v=66P5FMkWoVU](https://www.youtube.com/watch?v=66P5FMkWoVU)

<comment><USER40> I usually feel like a reasonably bright person, and then I go to a talk like this and suddenly remember how terribly much I don't (and will never) know. There are some very sharp people at play here.

</comment></comment><comment><USER41> This is a solid argument for why you should have more datatypes available to you. If your program relies on an insertion order, say so. If anyone ever moves to another implementation, they will thank you for it.

<comment><USER42> Ordered dictionaries already existed as a separate type. Much of the gain was in the use of dicts for core language features.

In particular, Python 3.6 guarantees that the order in which class attributes are defined is preserved, and that functions which take variadic keyword arguments receive them in the order in which they were passed. It permits other implementations to use types other than dict to accomplish that.

Dataclasses make use of ordering of dictionaries in a subtle way. If you write:

      class Foo:
          bar: int
          baz: str

Then Foo.__annotations__ is a dictionary {'bar': int, 'baz': str}. The @dataclass decorator transforms that into standard boilerplate, and it needs to know the order to do that.

<comment><USER41> And this is all the more reason using a type would make sense. Curious why things use ordered? Look at all the places that declare they need ordered. :)

<comment><USER42> You don't know in advance when ordering might be useful, though.

Class __annotations__ was new in 3.6. For its original use there's no need for ordering. They were ordered, because dicts were ordered, but only as an implementation detail.

Dataclasses were added to 3.7, after 3.6's features made a nice syntax possible. If __annotations__ hadn't happened to be ordered already then I would guess it wouldn't have been made ordered just for dataclasses - dataclasses just wouldn't have existed.

Making everything ordered in one go opens up possibilities you haven't even thought of yet.

<comment><USER41> That argument cuts both ways. You don't know when the benefits of random will be there...

As soon as you do something that cares about order, state how it is derived. Sometimes, insertion order is right. Sometimes, not.

Don't get me wrong alists are nice. And order def matters to those. But it is part of their definition. And reinsertion changes the order in obvious ways. Not even clear what it does to just "dict".

Now, I will concede this is overblown. Life will easily go on.

</comment></comment></comment></comment></comment><comment><USER43> For some definition of "now". Python 3.7 was released in June 27, 2018. So "now" is "for the past year and a half". Python 3.6, which implemented the change originally, was released in December 23, 2016. By that measure "now" actually means "for the past 3 years".

<comment><USER44> The whole Python 3 affair made me realise that a lot of people upgrade their tools like once every 5-10 years. Which is shocking, in an industry as fast as this. I'm not saying you should churn your whole build setup every 6 months (I think release speed for Python is a bit excessive at the moment, for example), but complaining in 2020 for a feature that was released almost two years ago, was in development for a year or so before that, _and was widely publicised_... I mean, c'mon.

</comment></comment><comment><USER45> This is a refreshing update for those of us that like having code which will always act in the same way across multiple invocations.

Now, if only Lua could follow the same path with their "tables" ("tables" is what Lua programmers call their form of Python's "dictionaries" and Perl's "hashes").

I just spent eight hours earlier this week debugging Lua code which would run differently on different invocations of the same code.

The standard way to iterate in a table with Lua is like this:

      for key, value in pairs(foo) do

One problem: The order we get elements from the table "foo" is undefined, and _it can change between different invocations of the same Lua code, even if the elements were put in the table in the same order_. In order to fix things so that we can iterate a table in a consistent manner, this is my fix (public domain [1], if those who want to copy and paste it):

      function sorted_table_keys(t)
        local a = {}
        local b = 1
        for k,_ in pairs(t) do -- pairs() use OK; will sort
          a[b] = k
          b = b + 1
        end
        table.sort(a, function(y,z) return tostring(y) < tostring(z) end)
        return a
      end

Then we iterate the table like this:

      for _, key in ipairs(sorted_table_keys(foo)) do
        local value = foo[key]

(In Lua, two dashes indicates a comment.)

Note that this code will not always sort in the same order all tables. If we have a table with the keys 1 (as a number) and "1" (as a string), iteration order is still undefined.

The code is open source, and is a procedural ("random") map generator for Doom written mainly by Andrew Apted which I have added some features and fixed some bugs with. It's here: [https://github.com/samboy/ObHack](https://github.com/samboy/ObHack) and the issue is here: [https://github.com/samboy/ObHack/issues/4](https://github.com/samboy/ObHack/issues/4)

[1] The project I added this code to is GPL, but this function, which I wrote entirely by myself, is one I am donating to the public domain.

<comment><USER46> You can write an iterator yourself, no need to leave ?pairs idiom:

      function sortpairs(t)
        local keys = { }
        for key in pairs(t) do
          table.insert(keys, key)
        end
        table.sort(keys, function (a, b)
          return tostring(a) < tostring(b)
        end)
        local i = 1
        return function (t)
          local key = keys[i]
          i = i + 1
          if key ~= nil then
            return key, t[key], i-1
          end
        end, t
      end

      t = {a=10, c=20, d=30, b=40, 50}
      for k,v,i in sortpairs(t) do
        print(k, v, i)
      end

<comment><USER45> What Lua is lacking here (and why the above iterator function needs 17 lines) is the ability to have "for" go through a list ( _without_ converting the list in to values returned by an iterator function), which would let us quickly and easily sort lists that "for" can use. Something like:

      d = {"foo": 2, "bar": 1, "zoo": 4}
      for k in sorted(d.keys()):
        print k

(I'm not advocating Python here, since Perl has a similar way of using "for" to go through lists which can _also_ be easily sorted)

However, with Lua, "for" only accepts a numeric range, or an iterator function, so customizing "for" requires understanding function closures: Understanding how a function, when called multiple times, stores variables altered in previous invocations of the function, and understanding how to give those variables initial values (usually in the "function factory" function which creates the function we use).

In other words, "for", in most modern high-level languages, can be one of:

1\. for variable in [something that specifies a numeric range]

2\. for variable in [iterator function]

3\. for variable in [list]

But Lua only has "something that specifies a numeric range" and "iterator function"; it can not natively go through a list.

<comment><USER46> You can convert a list first and then feed it to a simple iterator. I don't fully understand what your exact real-code issues can be, but hope this snippet may help:

      function vs(t)
        local i = 0
        return function (t)
          i = i + 1
          return t[i]
        end, t
      end

      function sorted(t, cmp)
        table.sort(t, cmp or function (a, b)
          return tostring(a) < tostring(b)
        end)
        return t
      end

      function keys(t)
        local keys = { }
        for key in pairs(t) do
          table.insert(keys, key)
        end
        return keys
      end

      t = {a=10, c=20, d=30, b=40, 50}
      for k in vs(sorted(keys(t))) do
        print(k, t[k])
      end

I.e. if "natively" means strictly "for in t" that generates values, then no, Lua can't do that. But if "for in vs(t)" is okay, then that vs() is the solution.

<comment><USER45> That looks good, and I think putting these in a prominent place of the Lua documentation (along with a notice that the code is public domain) would help us who are used to the AWK/Perl/Python/PHP way of having "for" natively traverse a list without needing a complicated list-to-iterator function that uses function closure (i.e. the iterator function remembers the value "i" -- I'm writing this for the lurkers because code like this can be difficult to follow).

One honest question: Is there any reason why the function factory (i.e. a function which returns a function) which converts a list (Actually, table with ascending integer indexes) in to an iterator Lua can use with "for" returns both the element and the entire table here? Here is the code I am asking about:

        return function (t)
          i = i + 1
          return t[i]
        end, t

I'm curious why we're returning both the table element for the iterator and the entire table.

</comment></comment></comment></comment></comment><comment><USER47> 1) I am personally annoyed as someone who has had to unwind knotty dict soup python codebases.

2) This will make life easier for thousands of programmers and prevent a massive number of extremely difficult bugs from hurting users.

</comment><comment><USER48> It seems people dont realize that you can have your cake and eat it too. You can have both orderered and unordered maps in the same language:

[https://yaml.org/type/map](https://yaml.org/type/map)

[https://yaml.org/type/omap](https://yaml.org/type/omap)

No reason to argue about which one to make "dict". In fact it would be better to have both because youre taking a performance hit (a significant one) by ordering the entries.

<comment><USER49> The new dict does not come with a performance hit, let alone a significant one. It's much more memory efficient and generally slightly faster.

<comment><USER48> pretty strong evidence to the contrary

[https://apps.dtic.mil/dtic/tr/fulltext/u2/a627127.pdf](https://apps.dtic.mil/dtic/tr/fulltext/u2/a627127.pdf)

do you have any references to back your claim that its actually faster?

<comment><USER50> [https://morepypy.blogspot.com/2015/01/faster-more-memory- eff...](https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient- and-more.html)

The new python dict implementation is benchmarked as faster in both microbenchmarks and in practice for almost all real-world workloads.

Note that this isn't a sorted map, like C++'s "Ordered Map", but an Ordered map. C++ get's the name wrong. Items aren't ordered by key comparison, but ordered by insertion time.

<comment><USER48> Good link, thanks

</comment></comment></comment></comment></comment><comment><USER51> Not sure why this is posted and upvoted to front page now. After all this is a major bullet point in py37 What's New, and even py38 has been out for a while.

Anyway, I'll keep using collections.OrderedDict (except for personal scripts) until py35 EOL.

<comment><USER52> Probably because it was posted to lobste.rs and got good traction there:

[https://lobste.rs/s/htcz5f](https://lobste.rs/s/htcz5f)

</comment></comment><comment><USER53> I think this change is great, but this really only becomes news again once all major LTS are shipping with at least Python 3.7, right? No one can really use it in code they plan to distribute at the moment. Maybe I am underestimating the amount of Python code that is meant for internal or personal use only.

<comment><USER54> We've been running FROM python:3.7 in production for months. Works just fine. ;)

</comment></comment><comment><USER55> This 37 minute Pycon video covers the essential details of how Python dictionaries are implemented: [https://www.youtube.com/watch?v=npw4s1QTmPg&t=3s](https://www.youtube.com/watch?v=npw4s1QTmPg&t=3s)

</comment><comment><USER56> Wait, if it is held in a separate dense array, is removal of a key from a dictionary O(N)?

<comment><USER57> You can use memcpy to remove items from the middle of a memory segment. It's probably not a single memory segment either; I imagine it works more like a Golang slice.

<comment><USER56> Memcpy is certainly O(N). But yeah, if they implement it with multiple slices (e.g. a B tree) it could be fast.

<comment><USER57> Hrm, I'm pretty sure I meant memmove: [https://golang.org/src/runtime/slice.go](https://golang.org/src/runtime/slice.go)

EDIT: I could have sworn they were shifting entire pages around without using a cycle per byte but I can't find any reference to that now haha.

</comment></comment></comment></comment><comment><USER58> Java has many types of Map, Set, and List implementations. And that's just in the core library. Here are some Map implementations that come with Java: TreeMap, a HashMap, A LinkedHashMap, ConcurrentHashMap, EnumMap. There are many more and they each have their own uses, features, and performance/memory characteristics.

I always wondered why languages like ruby, javascript, and python don't include a bit more choice on this front. It seems like sets are still somewhat of a novelty for javascript developers and people just wing it with half-assed solutions involving stupid O(N) contains operations.

</comment><comment><USER59> The historical background for those interested…

The ordered dict for Python was originally an implementation detail, part of a more compact dict representation that was first proposed for Python in 2012 [1], implemented for PyPy in 2015 [2], and merged into CPython 3.6 in 2016 [3] [4].

[1]: [https://mail.python.org/pipermail/python- dev/2012-December/1...](https://mail.python.org/pipermail/python- dev/2012-December/123028.html)

[2]: [https://morepypy.blogspot.com/2015/01/faster-more-memory- eff...](https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient- and-more.html)

[3]: [https://mail.python.org/pipermail/python- dev/2016-September/...](https://mail.python.org/pipermail/python- dev/2016-September/146327.html)

[4]: [https://news.ycombinator.com/item?id=12460936](https://news.ycombinator.com/item?id=12460936)

The decision to add it officially to the language spec wasn't made until Python 3.7 in 2017. [5]

[5]: [https://mail.python.org/pipermail/python- dev/2017-December/1...](https://mail.python.org/pipermail/python- dev/2017-December/151283.html) "Guido says 'Make it so.'"

</comment><comment><USER60> It sure is annoying seeing people complain about a well-received feature implemented 3 years ago that has already transitioned smoothly.

</comment><comment><USER61> question: does the following expression still evaluate to True with ordered dicts?

        {'a':1, 'b':2}  ==  {'b':2, 'a':1}

<comment><USER43> Yes. Dict comparison doesn't rely on iteration order.

<comment><USER61> thanks :-)

i mean, i would hope not if it's an implementation detail and not a change in the abstraction itself... feels a bit like it is breaking the abstraction, though.

<comment><USER43> The definition of a dict hasn't changed. It's still defined as a set of key:value pairs. It just happens to also have a particular guarantee now that iterating over the keys will return them in insertion order.

</comment></comment></comment></comment><comment><USER62> Funny, I've never expected or needed dict to be ordered. I've known about collections.OrderedDict but never really found much use for it. A dictionary is a hashtable and the order generally isn't important for typical uses of a hashtable. So, what is it that people are putting in dicts where the order of insertion actually matters?

</comment><comment><USER63> Pretty important to note that it's "insertion order". I immediately thought, ordered by what comparator function?

</comment><comment><USER64> A great talk behind the redesign which achieved this can be found here:

BayPIGgies at LinkedIn June 2017: "Techniques for Design Reviews" by Raymond Hettinger [https://www.youtube.com/watch?v=cNqJDRsefg8](https://www.youtube.com/watch?v=cNqJDRsefg8)

There's a lengthy lead in, but its worth listening to.

</comment><comment><USER65> I've always thought this was a bit annoying in Python, good to know a (very) small itch is now scratched.

<comment><USER66> It's been this way since python3.6 in 2016, so you really only have to worry about very old versions

</comment></comment><comment> c-smile Same thing as in JavaScript : [https://www.stefanjudis.com/today-i- learned/property-order-i...](https://www.stefanjudis.com/today-i- learned/property-order-is-predictable-in-javascript-objects-since-es2015/)

</comment><comment><USER67> Unfortunately ordered dict doesn't provide index access. In many cases, it's still required to maintain parallel list + dict approach, even if dictionary is ordered. List for index access and dict for fast look ups and containing the data.

</comment><comment><USER68> But how does ordered map(self-binary search trees) handle hash-only key? I mean you can compare lexicographical order with character strings or numbers, but not a class instance right?

</comment><comment><USER69> It'd be nice if using collections.OrderedDict could elicit a warning. Is there a tool like flake8/autoflake for flagging things that used to be Pythonic but no longer are?

</comment><comment><USER70> I'd love to know more about where an ordered dict comes in handy. Anyone have use cases? Otherwise, guaranteeing this behavior just seems ‾\\_(ツ)_/‾

If this is useful, I wonder if an ordered set is useful.

<comment><USER29> > If this is useful, I wonder if an ordered set is useful.

An ordered set is a unique list. It's useful. However it's not present in Python, dicts and sets have separate implementations and sets were _not_ moved over to the ordered implementation (because the ordering was initially a side- effect of a change in implementation which was not considered useful or advantageous for sets).

<comment><USER70> An ordered multiset is a list, not an ordered set.

</comment></comment></comment><comment><USER71> How does one have insertion order and maintain O(1) deletions?

</comment><comment><USER72> From TFA "... it brings Python on par with PHP"

Faint praise indeed

</comment><comment><USER73> I had firsthand experience with this otherwise useful feature masking an obvious bug. In one codebase there was function along the lines of (for calling into MSSQL with it's "peculiar" stored procedures):

        def call_stored_procedure(name, **args):
            cursor.call_proc(name, *args.values())

Of course this is obviously wrong and on Python 3.6 this will break horribly on first use, while in Python 3.7 this worked as long as the keyword arguments were in the correct order.

<comment><USER74> On 3.6 it will work, it's just relying on an implementation detail (dicts are ordered in 3.6, it's just that it is considered an implementation detail, not a guarantee)

</comment></comment><comment><USER75> A bit of a side topic, but when Judge Alsup (himself a descendent of Boole and with a middle name of Haskell, and a programmer to boot) said "It is so ordered" ([https://casetext.com/case/oracle-america-inc-v- google-8](https://casetext.com/case/oracle-america-inc-v-google-8)) in the Oracle Google case, I could only think that's what a computer scientist would say when quicksort was done.

</comment><comment><USER76> Must say I am glad about that. It confused me when I first discovered this and was told it was not a bug

</comment><comment><USER77> I've written a lot of Python, but more Java. This is where I have a gripe with "batteries included." In Java, I'd have to think slightly about this, then use a LinkedHashMap. It's been in Java since *2002. It also has a Set flavor. Python just doesn't have as rich of a collection of included data structures, and the APIs are more limited.

<comment><USER29> What java calls linkedhashmap, Python calls ordereddict.

It's not quite as old as java's linkedhashmap but is no spring chicken either (it's a bit above 10 years old).

<comment><USER77> Where's orderedset, though?

<comment><USER29> Nowhere because so far the core team has remained unconvinced. People have brought up replacing standard set implementation by a variant of dict's (even expressing surprise that that wasn't already the case) but no dice yet.

There are discussions on the subject on python-dev once in a while e.g. [https://mail.python.org/pipermail/python- dev/2019-February/1...](https://mail.python.org/pipermail/python- dev/2019-February/156466.html)

</comment></comment></comment></comment><comment><USER78> Title should say what "ordered" means. Is it by key, or by insertion-order?

</comment><comment><USER79> This has been true since 3.7 why is this such a seemingly major revelation on HN?

</comment><comment><USER80> So isn't that a backward-incompatible change for versions prior to 3.7?

</comment><comment><USER81> Ruby introduced similar change in 1.9 back in 2007, and we're fine.

</comment><comment><USER82> Unpopular opinion: I'd rather have classes that have precise meanings such as in Java. ArrayList, HashMap, TreeMap.

Having classes with ambiguous names makes it seem like a toy language.

</comment><comment><USER83> You ordered snake dicks?!

</comment><comment><USER84> Javascript taught me to never assume any ordering in a dict/map/hashmap (whatever the term)

<comment><USER85> Is the ordering there at least fixed, or can it vary from run to run even on the same data, or between implementations or versions of the same implementation?

EDIT: to clarify, I was asking about JavaScript there.

<comment><USER86> It is deterministic, but it's an odd rule-set. For example: `var o = { 3: 'foo', 1: 'bar', b: 'baz', a: 'quux' }` Will yield: `{1: "bar", 3: "foo", b: "baz", a: "quux"}`.

Numeric keys get sorted. String keys are insertion-order. If key order is a priority, a Map should be used instead. Often when JavaScript's objects are used and a particular sort order is required, an accompanying (sorted) array is used.

<comment><USER87> (deleted--missed context)

<comment><USER86> Go do `var o = { 3: 'abc' }` and then `o[1] = 'def';` Now tell me the output of `o`. (Doing this in your console should suffice). Is that insertion order?</comment></comment></comment></comment></comment>


Teaching Yourself to Make Music Software: Steve Duda in Conversation [video] - <AUTHOR> https://www.youtube.com/watch?v=Cp0rtLaXBio <comment><AUTHOR> Context: You probably don't know who Steve Duda is unless you're into electronic music production, but you've likely heard music made with his software.</comment>


I will mail you WireGuard stickers - <AUTHOR> https://lists.zx2c4.com/pipermail/wireguard/2017-May/001338.html <comment><AUTHOR> Yesterday's envelope-writing frenzy was a success, so I'm throwing caution into wind and posting this on HN today...</comment>


I almost feel obligated to pirate Office for Mac - <AUTHOR> http://rys.sommefeldt.com/2012/10/01/i-almost-feel-obligated-to-pirate-office-for-mac.html <comment><USER2> I was fairly supportive of his thoughts until I read at the bottom that he works for Imagination Technologies, makers of the PowerVR chipset and quite possibly the worst company in the world when it it comes to giving any support at all to open source drivers.

After reading that I was struck by a couple of things -- one is that it is hard to feel bad for someone complaining about a bad actor in the software space when they work for a bad actor in the hardware space; two is that even if I have a bad opinion of an employer already, I still think it is bad form to post an article like this with your employer's name linked right at the bottom.

Even though he ultimately concludes that he won't pirate it, it is fairly easy for bad internet "journalists" to twist what he said and involve his employer since he names and links right to them at the bottom of his rant-post. Rant- posts where you invoke even the idea that you may break the law are best made on blogs that do not mention your employer.

<comment><USER3> Rys is cool though. He gives out of office hours support for people learning 3d graphics.

Check out his recent talk at the London Graphics Hackerspace: <http://www.youtube.com/watch?v=uWzcEKHbBv4>

<comment><USER4> That's no excuse. He may be a great guy but the OP's point stil stands. In general we're all great people until we're not. I'm a nice guy but I've been known to be a total asshole like anybody else. Cool guys act like dicks sometimes and a record of being a good guy doesn't excuse one for doing something uncool.

Bill Clinton left us with a surplus. He also cheated on his wife. Still a good president but cheating is a dick move. See what I'm saying (please don't read into it and focus on the wrong thing now)?

<comment><USER5> There are lots of things about Bill Clinton that I liked and just a few things about him that I didn't, but seriously, I don't think even the most incompetent person in the world could have left us without a surplus in the era that the internet went mainstream.

I do agree with the point you were making.

<comment><USER6> The internet bubble didn't create the surplus, the "peace dividend" did. After the former Soviet republics became our BFFs, we promptly eviscerated our Cold War military. Combine that with some good growth in the economy (which obviously includes the tech sector) and some welfare reform that Clinton and Gingrich worked on together, and you got a nice surplus.

Throw in Bush-era tax cuts and two wars, and you can say bye-bye to that.

<comment><USER5> How much did our defense spending decrease? I know that we must have saved a ton of money when Clinton cut the Army nearly in half and when we decommissioned so many of our battleships. I've heard it said that we still spend more on defense than the next 10 countries combined, but I'm not sure what the numbers looked like before the cutbacks.

Edit: I did a little research and I was astonished to find out that welfare programs cost hundreds of billions of dollars annually. I didn't think that something like welfare would make up a significant percentage of the budget. It's over 10% of the federal budget if you include all of the programs that fall under "welfare."

Looking at the statistics, the amount of money spent on welfare didn't significantly increase or decrease during or after Bill Clinton was in office (until our current financial crisis.) I know that he was trying to find smarter ways to spend the money, such as by getting people back to work with education and employment assistance, but how could the reform have added to the surplus if the amount of spending wasn't reduced?

After looking at this site a bit longer, it looks like every category of spending increased each year he was in office. Even though spending increased, there were some areas where he slowed down the rate of increase, which is admirable. But if we increased spending in every area before, during and after his presidency, yet we produced a surplus during his presidency, doesn't that mean the surplus was created by the thriving economy, which was in turn thriving because of the internet boom? I'm not asking to be argumentative, I just don't understand how we can say that our reduced need for defense spending combined with welfare reform caused a surplus when we didn't reduce our spending in either category from 1993-2001.

[http://www.usgovernmentspending.com/breakdown_1994USbt_13bs5...](http://www.usgovernmentspending.com/breakdown_1994USbt_13bs5n)

Edit2: Since I asked a couple of questions. Wouldn't it be more productive to answer them instead of downvoting me? I wouldn't have asked them if I didn't genuinely want someone to explain things to me from their perspective. Why let someone continue to be wrong when given the opportunity to correct it?

<comment><USER7> Inflation + population growth will result in increases unless you make a conscious decision to cut. So if you leave something alone in the budget, you'd expect it to increase naturally over time. Even if there was no internet boom, and the budget continued to rise, tax receipts would still go up b/c the economy must grow to accommodate all those kids entering the workforce each year. If you hold the line on spending increases, not eliminate the increases but put them below the rate of growth, you'll have a surplus.

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> I just wanted to give a counter-point here, to show they're not all bad. Microsoft saved me recently, when (due to my own stupidity) I was in a foreign country without my main computer and I'd forgotten to install Office for Mac, which I needed urgently to finish a presentation. Luckily, I had my license key so I entered it in on their website, and they gave me a link to download the full version of Office. No questions asked. I hadn't even bought it off Microsoft directly in the first place; I'd bought a student version off a third party.

This might seem like a no-brainer feature, but when I've bought software online in the past it's often come with a 30-day download link, and when that link expires you're out of luck. You have to have the forethought to retain the install image to be able to install it in future. So I was actually impressed and grateful to Microsoft for that.

</comment><comment><USER9> I currently live in Belarus and dealt with the same crap a few days ago. Win 8 pro upgrade offer? Sure, but not for you, pal. Try looking for it at local resellers... Same situation is here with paypal, which makes us register it for different countries. So there I was, buying a VPN to buy Windows. How stupid is that? I've always been confident that internet is one single area. It can not be separated by countries. It is one network with no geography, no boundaries. Itunes music purchase? Yeah, very funny. Google Play books or movies or music? Amazon mp3 store? Nope and nope on every point. So nope, I'll not give them a single cent until the day they fix their behaviour. And yes, I'll pirate as much as I can, even being a software developer myself, knowing all the efforts it takes.

</comment><comment><USER10> Is Office even necessary anymore? Where I work, most people are using LibreOffice or Google Docs and not a single complaint about document exchange or document formats. There's a few Office holdouts, but even they haven't had an issue exchanging documents with the rest of us. Even I behave like a normal human being and use LibreOffice instead of Emacs when I wanna play nice at work.

What exactly is the killer feature of Office to people not obsessed with Outlook? Mac doesn't even have that, having to settle for EntouRAGE. (EDIT: My mistake, not used Mac Office in a long while -- Outlook now exists!)

<comment><USER11> Outlook on Mac certainly exists - and is much, much better than the thoroughly aneurysm-inducing Entourage.

But yeah, at my last two jobs we've exclusively used Google Docs, and it's been a dream. No more hunting your inbox for a (probably outdated) copy of the Office doc. Sharing is a dream.

It's not quite as usable as a native app. I would be rather happy if there were a native client to GDocs, but as it stands, the ability to collaborate, share, and disseminate far, far trumps whatever usability gaps between it and Office.

<comment><USER8> Yep, Outlook (Office 2011) replaced Entourage (Office 2008) on Mac.

</comment></comment></comment><comment><USER12> At least they only _made a bad job_ of trying to take his money. Commonly when I want to stream a movie rental, the studios have seen to it that there is absolutely no way I can legitimately pay for that service.

</comment><comment><USER4> I am annoyed and frustrated by so,e software therefor it's okay for me to not pay for it? Give me a break. This is such a lame justification for piracy. If the author truly _wants_ to pay for it then he will. Saying you want to pay for it in an attempt to seem on the up and up and really _wanting_ to pay for something are different. I can understand the frustration and I can empathize and I can get on board with the idea that the process they put you through to buy Office is ridiculous but I can't buy the justification for piracy. Just because you sell software yourself for a living doesmt put you in a position to decide when it's okay to pirate software. The author may consider the way in which he sells his software to be superior and therefore it's not okay to pirate what he sells but when it comes to someone else's well that's a different story. But its a slippery slope. Microsoft's process certainly is ludicrous but it's not long before people start saying "you want me to enter an email address to buy software? Oh, that's too much, I think I'll just pirate it, thanks".

Calling it one's and zeros doesn't justify it either. I hate the ones and zeros argument. You're not paying for one's and zeros. You're paying for the value the software creates for you. It may only need to be coded and compiled once but does that mean that after the first person gets value from it that the software has been paid for? Of course not.

If someone creates software and you value what it can do then you are obligated to pay for it if that's what the creator asks. Argue about the definition of stealing till your blue in the face but it doesn't change the fact that you are depriving the author of revenue they would otherwise have gotten. It is stealing and to say otherwise is to do some impressive mental gymnastics.

The system may suck but piracy isn't going to solve the problem. It's just going to create more problems. Using Office without a license doesn't hurt Microsoft. What hurts them is a more popular alternative. The only thing that will change what the author complains about is someone else creating some half decent office replacement in a way that shows Microsoft how to do it right. If such a program existed and were widely adopted you'd better believe they'd take notice and shape up. Piracy hurts people. It doesn't hurt the big guys though and we know it so we all tend to justify it. But that same attitude that gets people thinking its okay to pirate Office allows people to justify pirating software from smaller companies and that's who it hurts.

I see no justification for piracy.

<comment><USER10> How about when you pay USD$10,000 for a niche software package with a crappy USB hardware dongle with drivers that only work on Windows XP, despite the application itself working just fine under Windows 7 64-bit?

When I asked the vendor, I was told I could pay another $10,000 so I could buy a new version of the software that just so happened to have a newer DRM dongle.

At that point you really have four options:

1.) Hope someone else pirated it. 2.) Kill a few evenings cracking the application with IDA Pro. 3.) Forever remain on Windows XP on a graphics workstation, unable to use the entirety of RAM installed, or some horrible dual-boot situation isolating you from the rest of your tools. 4.) Pay ANOTHER $10,000.

Seriously, you have no position to tell anyone that there's 'no justification'. It's obvious you're missing real world experience here. I picked #2, myself, but I'd have rather saved time and picked #1 if it had already been done.

<comment><USER4> I have the real world experience and I stand by my position. It does suck and I can empathize and of course I can understand why you'd pirate the software but are you really going to try to tell me that it's not wrong? Can you honestly say that not paying is okay because the price is too high and you don't agree with the reasons why you can't run your dongle on a new OS? That's what it comes down to in the end. It's about not wanting to pay and disagreeing with the price. The fact that you know the dongle should work on your machine doesn't put you in a position to _know_ why they want you to upgrade. Anyone in that situation would have a knee-jerk reaction and jump to thinking its a corporate conspiracy and they're just gouging people. That's plausible. Or maybe there's a legit reason that you don't know. But in either case, no matter how ridiculous the experience is (and I grant you that the example you provide is very unreasonable) it still doesn't change the fact that someone created something of value to you, you want to use it above all alternatives, and you choose to pirate it which is wrong.

Suppose I go to the Apple store and I want a new MacBook Air. I have to wait in line for an hour. Then when I get into the store they've upped the price by $10k. Then, even after swallowing my frustration, I look to pay but all the checkout guys are busy. So I decide this is fucked up and just walk out with it. You'd agree that was a messed up situation but it's not okay for me to walk out of a store without paying.

The point is something I can't fathom anyone disagreeing with. There's a product. It has a price. If you want it you have to buy it. If you take it without paying _that is wrong_. How can you disagree with that?

<comment><USER10> Yes, I can clearly and with good conscious say that it is 100% a-ok to break the lock or take a cracked copy of software you already paid $10,000 for to operate with a $180,000 system if the vendor is going to gouge you over a trivial issue.

I'm really sorry that you live in a fantasy world sponsored by the MPAA and RIAA. I've already paid my pound of flesh to get my work done, intentionally not backporting a dongle driver to force upgrades at $10k a whack is the stuff slime is made of. They do this because they know very well there are not many/any alternative vendors for this equipment, which happens to be an exotic motion capture rig absolutely critical to my work. Fuck that and fuck anyone who tells me I have to grin and bear it or otherwise just wave my hands and go without all because otherwise I might offend their sensibilities over how I might see _their_ software. Personally, I don't care about your software. It hasn't done anything slimy to me yet. You're safe.

<comment><USER4> I'm sorry, I misunderstood part of what you said. I'm actually cool with people cracking software they've already paid for. That's fine. To me that's like modifying your car in ways that stop it from being street legal. You should be free to do so but at the same time I won't feel bad for you if you were ever stopped for speeding. Now that I understand your example better I don't think you're comparing equivalent things. It's one thing to just take something because you don't like the price and it's another thing to break contract and modify something you've already paid for. Both are still wrong but I can see the justification for the latter.

</comment></comment></comment></comment></comment><comment><USER13> The original author uses Office on Windows at work. My employer has a licensing scheme that means purchasing a licence for my own use at home is very cheap (something like £10 for Office 2010).

I'm thinking could the oa use something like crossover office

[http://www.codeweavers.com/compatibility/browse/name?app_id=...](http://www.codeweavers.com/compatibility/browse/name?app_id=6132)

or could they use a Windows 7 VM? Better than pirating in my personal world.

<comment><USER14> I'll ask and see if we have a scheme like that at work. I don't own a license of Office for Windows that I can use in a VM or with Crossover, but I'd be willing to entertain that if the price was right.

</comment></comment><comment><USER15> I can completely sympathise with your experience! I tried buying window 7 a while ago, as a digital download, from Canada. Guess what? The on-line store wouldn't sell to Canadians? We're on the same frigging continent, it's the internet! WTF?! Frankly I don't even know why they are still in business. The whole thing is run by monkeys. I ended up purchasing a licence, and then pirating the actual software, thus I can say that I paid for it, and frankly pirating MS software is SUPERIOR EXPERIENCE compares to purchasing it. Monkeys.

</comment><comment><USER16> In the grand scheme of things, it's better for Microsoft that you pirate Office than use something else - so really you are not punishing them for their mistake by doing that.

Your continued use of Office, will mean people can continue sending Office documents and you can send them too, if you used something else that whole system gets disrupted and people you work with consider new tools too. If you share your doc with Google docs for example, their copy of Office for that task is useless and after a while everyone might be using it, in my view it's a much better workflow than email documents around.

Just my 2 cents [I don't think anyone should pirate software by the way].

</comment><comment><USER17> Why do these companies think it's a good idea to refuse to sell to a customer depending on where they live?

I can understand in the case of Netflix etc where they legally cannot provide access outside of the areas where they have the licences, but there is no such issue here - Microsoft is fully able to distribute their own product, and there isn't even a shipping issue since it's just online.

Even Google has this problem, where they won't ship their devices from the play store to anywhere outside of a few countries, even though they would be fully able to.

<comment><USER18> A) People in different regions will pay different amounts of money. By restricting regions in which a person can use a thing (such as DVDs) they can sell at the price each market will bear. B) Sometimes, the separate regions are managed by separate corporations (wholly owned by the parent) which have their own pricing policy. C) Often there are local laws or agreements dictating that the corporation not undercut local distributors.

</comment></comment><comment><USER19> I have a saying I like to think I coined, "Don't make it hard for your customers to give you money". I think that applies here.

When you are working to optimize your business, take a step back. If you're focused on adding new features or exploring some marketing campaign, consider how you can optimize the most important part instead: getting paid.

There have been more times than I can remember where I would have bought or paid for something if only the retailer hadn't made it so difficult.

</comment><comment><USER20> If your experience with Office sucks, just don't buy it. There is not justification for pirating. It's a free market, and Microsoft's monopoly is long gone.

I haven't used Office in years and I've done just fine. The few time people send me `docx` documents, I can either open them up with Preview, Pages, or simply not open it. Most of the time, I don't care enough about the contents. When it's a resumé, sending anything other than a PDF or a website is an instant fail.

<comment><USER21> >When it's a resumé, sending anything other than a PDF or a website is an instant fail.

What sucks is when you make your resume in latex and you apply to a company that only takes word or raw text files. I've actually skipped out on companies because of this.

</comment></comment><comment><USER22> So? Set up with a VPN that has stateside IPs, or build your own on AWS.

Whether you pirate Mac Office or not though: I've been using it for awhile now and it's pretty lackluster. Most of the programs are stripped down versions. It usually gets the job done, but you'll probably run into a situation at some point where you want to do something that isn't included with Mac Office 2011.

<comment><USER23> >> So? Set up with a VPN that has stateside IPs, or build your own on AWS.

Is that a serious suggestion...?!

<comment><USER22> Well it's sort of a half-assed suggestion since I don't like the program. But there are many free VPNs that could solve this problem easily and rolling your own is fun and informative.

Pretty weak blog piece to be honest. Proclaiming that it's somehow a moral duty to steal something is trumped up nihilism at best.

</comment></comment></comment><comment><USER24> While I appreciate the frustration (and lord knows Microsoft have frustrated me plenty given I spend half my time developing for their platform), no matter how difficult they make it for you to buy, it's never an excuse not to.

</comment><comment><USER25> In my arrogant opinion, if a company _will literally not allow you to give them your money_ , they shouldn't whine later about how piracy is killing their sales, making angels and puppies cry, etc.

</comment><comment><USER26> It sounds like the problem is that a sales & marketing email took him to the wrong site.

To pirate the software he'd at the very least need to do a search on Google to find out how. A simple search[1] shows how easy it is. The first result is a warez site called "Official Apple Store - UK" where he can download it immediately with very little effort.

[1] <http://www.google.co.uk/search?q=office+mac+uk>

</comment><comment><USER27> The best way to run MS Office on a Mac? I use VMware and just run Office within Windows. What's funny is that this is actually faster than launching Office for the Mac while showing me what a Windows user is seeing. By the way I'm doing this on an iMac so I'm not using anything too exotic in terms of hardware.

</comment><comment><USER28> If you have Office at work you may be elegible to get office for just $10.

[http://www.microsofthup.com/hupus/otherproducts.aspx?culture...](http://www.microsofthup.com/hupus/otherproducts.aspx?culture=en- US)

I got it through my University for just $10.

</comment><comment><USER29> If you use Microsoft Office you are part of the problem.

Please don't be part of the problem.

</comment><comment><USER30> [http://www.amazon.co.uk/s/ref=nb_sb_noss?url=search- alias%3D...](http://www.amazon.co.uk/s/ref=nb_sb_noss?url=search- alias%3Daps&field-keywords=mac+office)

I'm pretty sure it was a lot easier for me to come up with that link than for him to write that narcissistic blog entry.

</comment><comment><USER31> BizSpark.

</comment><comment><USER32> rys, what are using for your blog's backend? octopress? jekyll?

</comment><comment><USER33> I'm not sure why Microsoft doesn't put it up on the App Store. They will probably get a lot less customer information, but things would be so convenient. Apple will stand to make a cut, but then there are several companies which sell both on the web as well as the App Store - why not do that.

Adobe was hesitant at first too. They brought several smaller apps to the App Store at first, then followed by some big ones.

<comment><USER34> Apple getting a cut is the reason why MS doesn't put it on that service. There's no point in paying 30% if you don't have to, and MS doesn't have to.

</comment></comment><comment><USER35> You could just go to an apple store and buy it. Pretty lazy to pirate something thats pretty cheap to start with. If you dont have 149 bucks, maybe you have bigger problems than word processing?

</comment><comment><USER36> Microsoft.Office.2011.With.SP1.MAC.OSX-CRBS is what you're looking for.

Also, guide from torrent comments (don't personally know if they work, but many people reported that they did):

\- unrar the download and install the .dmg

\- download the crack from [Edit: Google "2011fixTL.rar" to find a link to a "www-mediafire.com" site, around the 6th result. Change the "www-" in the URL to "www." to get the MediaFire download link] (this one still worked for me) and unrar it

\- now open spotlight and type terminal

\- click Terminal to run it

\- type "cd /" and hit enter

\- type "open /Library/Preferences" and it will open up the correct folder

\- drag the files from the crack folder in to this preferences map

\- now try open Word and it should ask you for your name, this means the hack worked. If it ask to validate your licence-key it failed.

\- click no on all the support questions and keep continueing

\- word should now open, same for the other office programs

<comment><USER37> I suggest removing the direct link to the crack if you want to avoid being flagged for deletion.</comment></comment>


Aereo Founder's New Startup Wants to Bring You Wi-Fi–And Cut Out the Providers - <AUTHOR> http://www.wired.com/2016/01/aereo-founder-prepares-for-battle-with-new-wireless-startup/ <comment><USER2> Quoting from a slightly older Fierce Wireless article that delves into some of the details of Starry's fixed wireless millimeter spectrum play as outlined by CEO Chet Kanojia:

 _" [Cellular carriers] have more money than God, so I'm not sure that's a good area for us anyways." Then you start looking at spectrum at 24 GHz and above, and there are plenty of challenges, but the 37 GHz range might be the sweet spot for Starry.

To be clear, "we're really smart people, but we're not that good," he said. "We took a tremendous short cut, which we think is the right strategic short cut." The company didn't develop its own radios; it's using 802.11ac MIMO radios off the shelf, having figured out a way to convert them to a higher frequency band without causing distortion, he said._

[1] [http://www.fiercewireless.com/tech/story/att-verizon- other-c...](http://www.fiercewireless.com/tech/story/att-verizon-other- cellular-carriers-dont-need-worry-about-starrys-disruptiv/2016-01-28)

</comment><comment><USER3> Is sounds like they are developing along the similar lines I have.

My first product was free television (Free.Tv) and I recently open sourced my patent on free wireless at [http://Free-Fi.org](http://Free-Fi.org)

</comment><comment><USER4> Previous discussions:

[https://news.ycombinator.com/item?id=10984755](https://news.ycombinator.com/item?id=10984755)

[https://news.ycombinator.com/item?id=10981220](https://news.ycombinator.com/item?id=10981220)

tl;dr: consensus is, barring some wild, unknown innovation, the technology claimed does not seem to be a good fit with the mission stated.

my option: investor bait

<comment><USER5> Yet, Google has announced they're playing with millimeter-wave networking from aerial drones and that was much better received for some reason....

[https://news.ycombinator.com/item?id=11009109](https://news.ycombinator.com/item?id=11009109)

<comment><USER4> Google is not promising gigabit speeds that will displace landline providers, and the article linked acknowledges the challenges with the technology that are yet to be overcome.

It doesn't blow sunshine in my face and tell me everything is golden.

</comment></comment></comment><comment><USER6> I like the way this guy thinks. Takes on the fat cats who are abusing their unfairly advantaged business positions to squeeze more money than they deserve out of us.

If he keeps doing this and getting squashed, it could lead to a successful career in politics :)

<comment><USER7> It's deeply ironic to paint Aereo as taking on those with "unfairly advantaged business positions." Aereo's whole business model was getting other peoples' content _for free_ because of a loophole in the broadcasting laws. It's great to build a business delivering content other people create that you don't have to pay for, but _that 's_ the definition of "unfairly advantaged business position."

<comment><USER8> I never felt that situation was clearly resolved in my mind. The essentially ran a subscription dvr service. It's hard for me to wrap my mind around how it's illegal for a business to do that, but just fine for me to do it myself.

<comment><USER7> I'm not talking about the law so much as the moralizing. Aereo was a middle- man. The _only_ reason anyone ever paid them money was to get access to content that Aereo didn't create or even pay for. When that content became unavailable, Aereo collapsed because the _entire value of their service_ was derived from other peoples' content.

Whether or not their "subscription DVR service" fit within the legal loophole they were trying to take advantage of doesn't change the fact that it's pretty rich to build a service like that and then go around saying that the people whose content you're distributing -- the people who create the content that is the sole basis for the existence of your business -- are "fat cats" with "unfairly advantaged business positions."

<comment><USER6> Nah. They were providing "dvr as a service" at a reasonable price. The "fat cats" are the cable monopolies that lobby to protect their unfair positions while drastically overcharging and providing poor customer value.

Aereo made it easy to take advantage of something that is already free (and free by law). That was going to cut into the fat cats margins, so the fat cats got them shut down.

</comment></comment></comment></comment></comment><comment><USER9> Remembering the heated discussion in my neighborhood about installing some extra mobile towers, I can already see the protests when they hear "high- frequency millimeter waves","Starry Beams".

</comment><comment><USER10> How is another provider "cutting out the providers"?

<comment><USER11> I think that the value is "adding another provider" to the market, where in many places there is only one or two choices for internet, from entrenched providers with low levels of service.

</comment></comment><comment><USER12> > Aereo, now Chet Kanojia is taking on the country's biggest Internet > providers with a new start-up called Starry.

Excuse me while I piss myself laughing at the thought that the major ISPs would be worried about a wireless broadband startup. I tried Clearwire for six months. Even when it works well, wireless is no substitute for even copper wire, much less fiber. Bandwidth is lower, they have to throttle because of it. The physics just doesn't work to give anywhere close to the same experience as copper without doing something drastic like mesh, and that's got its own problems.

<comment><USER13> Is there a fundamental problem with mesh? Why couldn't that work in combination with something like maidsafe?

Just thinking out loud here.

<comment><USER12> The main issue with mesh is the property rights. It's too much of a headache to put mesh devices everywhere. So your market is going to be extremely limited.</comment></comment></comment>


The Internet Worm of 1988 - <AUTHOR> http://www.cs.unc.edu/~jeffay/courses/nidsS05/attacks/seely-RTMworm-89.html <comment><USER2> That was a scary time, the first ever large scale network attack. I found myself a couple days later flying down to DC with some other folks from MIT and Harvard to brief a bunch of senior DoD and agency types on what happened, but the thing I remember most vividly was getting home late at night after spending the day repeatedly trying to disinfect and protect our machines, only to log in via a 4800 baud modem and see our machines were somehow infected yet again, with the realization we'd changed root passwords so many times I had no idea how to get in and fix it, nor any way to reach our sysadmin who was even more exhausted than I was.

So I called a friend, who is now a physics professor at MIT, and said "Our machines are infected, could you please break in, go root, clean the infection, and send an email to our sysadmin explaining to him you did this at my request?" All he said was "Ok, get some sleep" and yes even though we'd just spent almost 24 hours locking down every possible attack vector into our machines and network we woke up to clean machines with a polite email in the sysadmin's inbox. I never have figured out whether that was more a measure of the state of network security in the late 1980's or of the kind of mad skills it takes to become tenure track at a place like MIT.

There is a good telling of the worm story in the final chapters of Cliff Stoll's amazing book on discovering a case of internet-hacking meets East- German-spies meets 2400-baud-modems and three-letter-agencies back in the mid 1980's (which spent 42 weeks on the NYTimes bestseller list and is a ton of fun to read)[0]

[0][https://www.amazon.com/Cuckoos-Egg-Tracking-Computer- Espiona...](https://www.amazon.com/Cuckoos-Egg-Tracking-Computer- Espionage/dp/1416507787)

<comment><USER3> I thought this article sounded familiar! :-) The Cuckoo's egg is an amazing read.

<comment><USER4> I can't believe that I found a copy for 50c in a second-hand store. I call it my "most precious item" because it has the highest price:enjoyment ratio of anything I own.

<comment><USER5> Just purchased, thanks for the heads up everyone, I'm looking forward to this.

</comment></comment></comment></comment><comment><USER6> Fun connections if you're into that kind of thing: Robert Morris, who wrote the work, was one of the founders of YCombinator, and by extensions Hacker News.

One of the classic books on the history of computers is Clifford Stoll's "The Cuckoo's Egg". It's a great book in general, and it mentions Robert Morris and Paul Graham, which I was surprised to find when I read it (it's not about the worm, but about a hacking incident).

And lastly, Clifford Stoll is apparently pretty active on YouTube, which I discovered when seeing a video of his for Numberphile, and finding out that it is, indeed, _that_ Clifford Stoll.

<comment><USER7> [removed]

<comment><USER8> Rtm was pretty involved in YC back then.

</comment></comment></comment><comment><USER9> Here's Cliff Stoll's post to the RISKS digest.

[http://catless.ncl.ac.uk/Risks/7.69.html#subj1](http://catless.ncl.ac.uk/Risks/7.69.html#subj1)

 _It 's now 3:45 AM on Wednesday 3 November 1988. I'm tired, so don't believe everything that follows... Apparently, there is a massive attack on Unix systems going on right now._

<comment><USER2> Risks was a pretty amazing list back in those days. It was very much the Hacker News of an earlier era.

<comment><USER9> Indeed. 32 years and still kicking. And there's an amazing amount of history in those archives...

I did a tribute to RISKS on it's 30th over at Metafilter

[http://www.metafilter.com/151727/Complex-Systems-Break-in- Co...](http://www.metafilter.com/151727/Complex-Systems-Break-in-Complex-Ways)

</comment></comment></comment><comment><USER10> Here are some older threads with more info: [https://news.ycombinator.com/item?id=5302924](https://news.ycombinator.com/item?id=5302924)

which includes links to

[http://docs.lib.purdue.edu/cgi/viewcontent.cgi?article=1701&...](http://docs.lib.purdue.edu/cgi/viewcontent.cgi?article=1701&context=cstech)

[http://www.mit.edu/~eichin/virus/main.html](http://www.mit.edu/~eichin/virus/main.html)

</comment><comment><USER11> For those who don't know, the Robert T. Morris quoted in the text turned out to be the author of the worm, and the R. H. Morris also quoted was his father (who also did a lot of work with the NSA).

[https://en.m.wikipedia.org/wiki/Robert_Morris_(cryptographer...](https://en.m.wikipedia.org/wiki/Robert_Morris_\(cryptographer\))

[https://en.m.wikipedia.org/wiki/Robert_Tappan_Morris](https://en.m.wikipedia.org/wiki/Robert_Tappan_Morris)

<comment><USER12> Robert T Morris also started Viaweb and YC with P Graham. He has chosen to (understandably) maintain a low profile.

<comment><USER13> I also seem to remember pg saying something somewhat tongue-in-cheek to the effect of other PhD students being jealous of rtm because, by being arrested, he managed to find a way out of the PhD program without the shame of quitting or flunking out...

<comment><USER14> _" The danger with grad school is that you don't see the scary part upfront. PhD programs start out as college part 2, with several years of classes. So by the time you face the horror of writing a dissertation, you're already several years in. If you quit now, you'll be a grad-school dropout, and you probably won't like that idea. When Robert got kicked out of grad school for writing the Internet worm of 1988, I envied him enormously for finding a way out without the stigma of failure."_

[http://www.paulgraham.com/college.html](http://www.paulgraham.com/college.html)

</comment></comment></comment></comment><comment><USER15> Another very good document (with some deeper technical detail) re: the Internet Worm is: [http://www.mit.edu/people/eichin/virus/main.html](http://www.mit.edu/people/eichin/virus/main.html)

I doubt RTM ever wrote any commentary about the worm's creation but I think that would be really interesting to read. No doubt it's something he'd probably rather be forgotten, but it would be interesting to hear him talk about it.

<comment><USER2> Mark Eichin and Jon Rochlis at MIT and Gene Spafford at Berkeley did some pretty amazing work, reverse engineering the attack payload on the fly in real-time to regenerate the source C code responsible for the attack while the attack was under way and at a time when no one was doing that sort of work yet.

My favorite part of those efforts was when Spafford posted a tongue in cheek bugfix patch to the attack code midway through the attack (at a time when the only people in the world with the source code were his team, Mark and Jon, and of course RTM since he wrote it). I couldn't find the patch but it's somewhere in the RISKS archives from that era.

<comment><USER16> I believe the worm traveled with it's code, which it compiled since it attacked both Vax and Sun systems, so if you caught it in the act. You could grab a copy of the C code.

<comment><USER2> Some of the code, yes, but not all

</comment></comment></comment></comment><comment><USER17> I remember the day it hit, quite well .. it was a huge deal. Nobody thought the Internet (in those days) was that safe .. but nobody expected the extent to which the worm brought systems to its knees - it really woke a lot of us up to just how exposed we were.

I wonder what the equivalent would be in todays terms .. some sort of Facebook-crippling JS injection, or so? I think its hard to come up with an example in the modern era- the scales are much higher, so the chances of making such a huge impact a lot lower.

<comment><USER18> > some sort of Facebook-crippling JS injection, or so?

You mean like the Samy worm ([https://samy.pl/popular/](https://samy.pl/popular/))?

<comment><USER19> The thing is, while Sammy was absolutely _huge_ (and landed the author is a lot of legal trouble) it was totally harmless. The only thing is did was added a friend and a sentence to your MySpace profile, nothing that brings down sites or anything like that. Though technically MySpace did bring the site down to fix the worm though.

Plus 12 years ago can hardly be considered "modern day" in computer terms. ;)

</comment></comment></comment><comment><USER20> PDF Version of the embedded analysis here:

[http://securitydigest.org/phage/resource/seely.pdf](http://securitydigest.org/phage/resource/seely.pdf)

<comment><USER2> The really amazing thing was the speed of the response, as reflected in the chronology section. Nothing like this had ever happened before, the entire net was getting shutdown by the attacking worm, and within 48 hours the community had come together on its own, reverse-engineered the payload, and shut it down, problem solved. A truly amazing thing to have experienced and played a small part in.

</comment></comment><comment><USER21> My favorite is how MS started development of OS/2 2.0 and NT OS/2 around that time, and there was x86 segmentation back then but they decided not to use it for 32-bit.

<comment><USER22> Thanfully, segmentation was a royal pain with near and far pointers.

Flat adressing was surely the feature I mostly envied from the Motorola 68000.

<comment><USER21> But with 32-bit segments you can just have one code and one data segment and maybe one stack segment.

<comment><USER22> Ah, from that point of view yes.

However it doesn't bring much over MMU configuration regarding read and write accesses, right?

<comment><USER21> You can't even create a single segment selector that is both writable and executable at all in x86.

<comment><USER22> Wasn't it possible on 16 bit protected mode segments?

My Windows 3.x days have long faded away.

<comment><USER21> Using separate selectors.

</comment></comment></comment></comment></comment></comment></comment><comment><USER23> Found the code:

[https://github.com/arialdomartini/morris- worm](https://github.com/arialdomartini/morris-worm)

</comment><comment><USER24> This is such a cool breakdown of the worm. Is there some sort of repo with these types of stories?</comment>


Ask HN: Are slides really necessary during a speech? - <AUTHOR> <comment><USER2> Visuals help people retain info. At the very least, slides that contain 1-2 sentences that state your main idea for that section would help. Having lots of text is a great way to get people to ignore what you're saying and read the slides so don't do that. But best of all is simple, clear graphics that help communicate your main points.

</comment><comment><USER3> I recently attended a talk [1] by Robert C. Martin where he spoke eloquently and astutely for over an hour on the subjects contained in the diagram on a single slide. Afterwards, I was impressed by the quality and content of the _talk_ , and didn't feel like I'd been hypnotised by yet another slide-by- slide "presentation".

A well-given talk should not require its contents to be summarized as it is being given. Pictures are worth a thousand words, so technical diagrams are an obvious exception (there are others) to that.

[1] [https://skillsmatter.com/meetups/11368-an-evening-with- uncle...](https://skillsmatter.com/meetups/11368-an-evening-with-uncle-bob)

</comment><comment><USER4> No, but they definitely help. My rule on this is that slides should make it easier to follow a talk and get back on track if your attention wanders. If the slides require extensive explanation then the information is probably better summarized in the talk and laid out in greater detail in a paper.

Obviously a lot depends on your audience. Slides at an academic symposium can be dense enough that you can get the gist of the talk from just looking at the slide deck, slides for an introductory talk on a subject should be more like the list of contents in a book.

</comment><comment><USER5> The question is too broad to answer with yes or no.

If you can come up with slides that make it easier for your audience to follow you, go for it.</comment>


Show HN: Compare Trends on Hacker News - <AUTHOR> https://hnprofile.com/compare?search=AWS,GCP%20|%20Google%20Cloud,Azure <comment><USER2> Did you use any library for the visualization?. Is it open source?

<comment><AUTHOR> Not open source, used highcharts</comment></comment>


Apple Watch: Faulty Taptic Engine Slows Rollout - <AUTHOR> http://www.wsj.com/articles/apple-watch-faulty-taptic-engine-slows-roll-out-1430339460

<comment><USER2> For what it's worth, I've voted for your particular article since you've got a title that actually communicates it's content. I like that....a lot.

Edit: turns out you posted both articles...</comment>


Startup Sheep Vs. Non-Startup Goats (Or Transitioning From Coder to Founder) - <AUTHOR> http://grokcode.com/741/startup-sheep-vs-non-startup-goats-or-transitioning-from-coder-to-founder/

<comment><USER2> That's a pretty good post. Despite this part about "natural-born" programmers. But I hope it was just a simplification.</comment>


Show HN: Indie Solver – Black-Box Optimization as a Service - <AUTHOR> https://indiesolver.com <comment><USER2> Seems similar to the YC backed company SigOpt:

[https://sigopt.com/](https://sigopt.com/)

<comment><AUTHOR> Yes, it is.</comment></comment>


A Cautionary Tale to Startups - sam-mueller https://twitter.com/samuel_mueller/status/1291571146861629441 <comment> sam-mueller In the wake of Facebook copying TikTok with Reels, here's a story in 5 tweets of how my startup got xeroxed. Happy to answer questions, ama.</comment>


Wayfair Should Have Seen This Coming - <AUTHOR> https://www.theatlantic.com/technology/archive/2019/06/wayfair-walkout-and-corporate-responsibility/592597/ <comment><USER2> i cant really agree with this, unless youre a decision maker in the company and you disagree with how they do business just leave, its that simple.

do we really want to live in a world where retailers will asses your moral stance before selling you things? "no problem sir, we just need to check your social media accounts to make sure you fit our moral/ethical code and that sofa is all yours"

lets just totally give up and go with that chinese social credit thing eh?</comment>