Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset - <AUTHOR> https://github.com/amznlabs/ion-java <comment><USER3> To think about the difference between serialization formats, here's an analogy I hope will help.

Protocol Buffers (and I think Thrift, and maybe Avro) are sort of like C or C++: you declare your types ahead of time, and then you take some binary payload and "cast" it (parse it actually) into your predefined type. If those bytes weren't actually serialized as that type, you'll get garbage. On the plus side, the fact that you declared your types statically means that you get lots of useful compile-time checking and everything is really efficient. It's also nice because you can use the schema file (ie. .proto files) to declare your schema formally and document everything.

JSON and Ion are more like a Python/Javascript object/dict. Objects are just attribute-value bags. If you say it has field fooBar at runtime, now it does! When you parse, you don't have to know what message type you are expecting, because the key names are all encoded on the wire. On the downside, if you misspell a key name, nothing is going to warn you about it. And things aren't quite as efficient because the general representation has to be a hash map where every value is dynamically typed. On the plus side, you never have to worry about losing your schema file.

I think this is a case where "strongly typed" isn't the clearest way to think about it. It's "statically typed" vs. "dynamically typed" that is the useful distinction.

<comment><USER4> That's a great analogy! However, I do think strongly typed vs. weakly typed has a role in thinking about this, just a different dimension than the one you're describing. Let's say we come across a JSON structure that looks like this:

      {"start": "2007-03-01"}

Is that a timestamp? Maybe! Does it support a time within the day? Perhaps I can write "2007-03-01T13:00:00" in ISO 8601 format if we're lucky. Can I supply a time zone? Who knows for sure? It's weakly typed data. The actual specification of that type of that field lives in a layer on top of JSON, if it's even specified at all. It might be "specified" only in terms of what the applications that handle it can parse and generate. I could drop that value into Excel and treat it as all sorts of different things.

Ion by comparison has a specific data type for timestamps defined in the spec [1]. The timestamp has a canonical representation in both text and binary form. For this reason, I know that "2007-02-23T20:14:33.Z" and "2007-02-23T12:14:33.079-08:00" are valid Ion timestamp text values. In this instance I would describe Ion as strongly typed and JSON as weakly typed. Or, as the Ion documentation puts it, "richly typed".

To make an analogy, weakly typed is the Excel cell that can store whatever value you put in it, or the PHP integer 1 which is considered equal to "1" (loose equality). Strongly typed is the relational database row with a column described precisely by the table schema. Weakly typed is the CSV file; strongly typed is the Ion document.

[1] [http://amznlabs.github.io/ion- docs/spec.html](http://amznlabs.github.io/ion-docs/spec.html)

<comment><USER3> Ion has more data types than JSON, it's true. Ion has a timestamp type and JSON does not, so you could say it's "richer" if you want, but that just means "it has more types."

However I don't think it's accurate to say that the typing of Ion is any "stronger." Both Ion and JSON are fully dynamically typed, which means that types are attached to every value on the wire. It's just that without an actual timestamp type in JSON, you have to encode timestamp data into a more generic type.

<comment><USER4> The notions of "strong" and "weak" typing have never been particularly well- defined, but I think my usage is in line with their usual meaning: [https://en.wikipedia.org/wiki/Strong_and_weak_typing](https://en.wikipedia.org/wiki/Strong_and_weak_typing)

> Some programming languages make it easy to use a value of one type as if it > were a value of another type. This is sometimes described as "weak typing".

Strong typing makes it difficult to use a value of one type as if it were another. In PHP, you can compare the integer value 1 to the string value "1" and the equality test returns boolean true. Conflating integer 1 and string "1" is weak typing. A data format that expresses the concept of the timestamp 1999-12-31T23:14:33.079-08:00 using the same fundamental type as the string "Party like it's 1999!" is what I would call weakly typed.

Ion does not make it easy to use a string as if it were a timestamp or vice versa. It has types like arbitrary precision decimals, or binary blobs, that can't easily be represented in a strongly-typed way in JSON. You can certainly invent a representation, like specifying strings as ISO 8601 for timestamps, or an array of numbers for binary -- actually, wait, how about a base64-encoded string instead? Where there's choice there's ambiguity. These concepts of "type" live in the application layer in JSON, instead of in the data layer like they do in Ion.

Note as well that stronger is my term. The Ion documentation says "richly- typed". Certainly Ion does not include every type in the world. Perhaps a future serialization framework might capture "length" with a unit of "meters", or provide a currency type with unit "dollars", and if that existed I'd call it stronger-(ly?)-typed or more richly typed than Ion. In that case, the data layer would prevent you from accidentally converting "3 inches" to "3 centimeters" by accident, since those would be different types. That would be stronger typing than an example where you simply have the integer 3, and it's the application's job to track which integers represent inches, and which represent centimeters. So perhaps "strong" and "weak" are not the best terms, so much as "stronger" and "weaker".

<comment><USER3> By your definition, any language with strings is weakly typed, since you can always interpret a string as being something else. Strongly/weakly typed has never been a particularly useful description (as the page you linked notes), and I think it's particularly unhelpful here.

<comment><USER4> > By your definition, any language with strings is weakly typed, since you can > always interpret a string as being something else

No, I wouldn't say that's the case. For example, in PHP you can literally write:

      if (1 == "1") { ...

... and the condition evaluates to true. You can do similar things in Excel; Excel doesn't even really differentiate between those two values in the first place. (At least that's how it seems as a casual user.)

This is not the case in strongly typed programming languages that have strings such as C++ or Java. You can _convert_ from one type to another, sure, by _explicitly_ invoking a function like atoi() or Integer.toString(), but the conversion is deliberate and so it is strongly typed. A variable containing a string (java.lang.String) cannot be compared against one containing a timestamp (java.util.Date) by accident. An Ion timestamp is a timestamp and can't be conflated with a string, although it can be _converted_ to one.

Edit: The set of types that are built in, in conjunction with how those types are expressed in programming languages (e.g. timestamp as java.util.Date, decimal as java.math.BigDecimal, blob as byte[]), is why I'd call Ion strongly typed or richly typed in comparison to JSON. Specifically, scalar values that frequently appear in common programs can be expressed with distinctly typed scalar values in Ion. I don't know if there's a good formal definition. You could probably define a preorder on programming languages or data formats based simply on the number of distinct scalar or composite types (so in that sense, yes, it's the fact that Ion has more). However it goes beyond that subjectively. Subjectively it's about how often you have to, in practice, convert from one type to another in common tasks. There is no clear way to represent an arbitrary-precision decimal in JSON, or a byte array, or a timestamp -- so you must "compress" those types down into a single JSON type like string-of-some-format or array-of-number; and _several_ different scalar types must _all_ map to that same JSON type, which creates the risk of conflating values of different _logical_ types but the same _physical_ JSON type with each other. There's no obvious or built-in way to reconstruct the original type with fidelity. There's no self-describing path back from "1999-12-31T23:14:33.079-08:00" and "DEADBEEFBASE64" back to those original types.

I subjectively call JSON weakly typed because its types are not adequately to uniquely store common scalar data types that I work with in programs that I write. I call Ion strongly typed because it typically can. I acknowledged earlier that a data format would be _even more strongly typed_ if it was capable of representing not just the type "integer", but "integer length meters". Ion does not have this kind of type built in, though its annotations feature could be used to describe that a particular integer value represents a length in meters.

<comment><USER3> > You can't misuse any kind of Ion value that is a string as if it were a > timestamp without performing an explicit conversion.

The same is true of JSON. There is no difference, except that Ion has a timestamp type and JSON does not.

If you disagree, please identify what characteristic of Ion's design makes it more strongly typed than JSON, other than the set of types that is built in.

<comment><USER5> You are choosing a definition of strong typing that supports your argument, but the argument is over the meaning of strong typing to begin with. It's not as if there's some universally accepted definition of strong typing. Like functional programming, functional purity, object oriented, etc.—none of these terms are universally defined.

<comment><USER3> The fact that "strong typing" has no universal definition is exactly why I think it's not useful.

<comment><USER4> I hate feeling like I'm nitpicking, but I don't think that's true. I think they do have a well-accepted definition, which appears in Wikipedia, in assorted articles online, and in computer science publications. Here are some examples of CS publications that describe a research contribution in terms of strong typing:

> Strong typing of object-oriented languages revisited. This paper is > concerned with the relation between subtyping and subclassing and their > influence on programming language design. [...] The type system of a > language can be characterized as strong or weak and the type checking > mechanism as static or dynamic. > [http://dl.acm.org/citation.cfm?id=97964](http://dl.acm.org/citation.cfm?id=97964)

> GALILEO: a strongly-typed, interactive conceptual language. Galileo, a > programming language for database applications, is presented. Galileo is a > strongly-typed, interactive programming language designed specifically to > support semantic data model features (classification, aggregation, and > specialization), as well as the abstraction mechanisms of modern programming > languages (types, abstract types, and modularization). > [http://dl.acm.org/citation.cfm?id=3859](http://dl.acm.org/citation.cfm?id=3859)

> Design and implementation of an object-oriented strongly typed language for > distributed applications. > [http://dl.acm.org/citation.cfm?id=99813](http://dl.acm.org/citation.cfm?id=99813)

> Strongly typed heterogeneous collections. (Oleg Kiselyov et al.) > [http://dl.acm.org/citation.cfm?id=1017488](http://dl.acm.org/citation.cfm?id=1017488)

> Strongly typed genetic programming. Genetic programming is a powerful method > for automatically generating computer programs via the process of natural > selection [but] there is no way to restrict the programs it generates to > those where the functions operate on appropriate data types. [When] programs > manipulate multiple data types and contain functions designed to operate on > particular data types, this can lead to unnecessarily large search times > and/or unnecessarily poor generalization performance. Strongly typed genetic > programming (STGP) is an enhanced version of genetic programming that > enforces data-type constraints and whose use of generic functions and > generic data types makes it more powerful than other approaches to type- > constraint enforcement > [http://dl.acm.org/citation.cfm?id=1326695](http://dl.acm.org/citation.cfm?id=1326695)

The argument that the terms have no universal definition cannot be sound in light of their widespread use in computer science publications, even in the title and abstract. Perhaps what you mean to say is that the terms don't have a _completely unambiguous_ or formal definition. That's probably true, but not all CS terms do. The words are contextual and exist on a spectrum, in the sense that a _strongly-typed thing_ is typically in comparison to a _more- weakly-typed thing_ [1]. However, the fact that they're widely used by CS researchers is why I think we should reject the argument that they don't have a universal definition or are not useful. CS researchers like Oleg Kiselyov use the term when describing their papers and characterizing their contributions.

[1] This is true for static and dynamic typing as well: they exist in degrees. Rust can verify type proofs that other languages can't regarding memory safety. Some languages can verify that integer indexes into an array won't go out of bounds. Thus it's not the case that a given language is either statically typed or dynamically typed; rather, each aspect of how it works can be characterized on a spectrum from statically verified to dynamically verified.

<comment><USER3> > I think they do have a well-accepted definition [...] [You] shouldn't > confuse your dislike for them for the absence of a well-accepted definition > that's widely used in computer science literature.

Just upthread, you said:

> The notions of "strong" and "weak" typing have never been particularly well- > defined

And the Wikipedia article you cited ([https://en.wikipedia.org/wiki/Strong_and_weak_typing](https://en.wikipedia.org/wiki/Strong_and_weak_typing)) says:

> These terms do not have a precise definition

The Wikipedia article also says:

> A number of different language design decisions have been referred to as > evidence of "strong" or "weak" typing. In fact, many of these are more > accurately understood as the presence or absence of type safety, memory > safety, static type-checking, or dynamic type-checking.

Also on Wikipedia ([https://en.wikipedia.org/wiki/Type_system](https://en.wikipedia.org/wiki/Type_system)):

> Languages are often colloquially referred to as "strongly typed" or "weakly > typed". In fact, there is no universally accepted definition of what these > terms mean. In general, there are more precise terms to represent the > differences between type systems that lead people to call them "strong" or > "weak".

...which is exactly what I'm saying in this entire thread.

It's very strange to me how you seem really seem to want other people to be on board with your particular interpretation of what everybody (even you, 13 hours ago) agrees is not a very well-defined concept.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER6> Finally! I've had to live the JSON nightmare since I left Amazon.

Some of the benefits over JSON:

* Real date type

* Real binary type - no need to base64 encode

* Real decimal type - invaluable when working with currency

* Annotations - You can tag an Ion field in a map with an annotation that says, e.g. its compression ("csv", "snappy") or its serialized type ('com.example.Foo').

* Text and binary format

* Symbol tables - this is like automated jsonpack.

* It's self-describing - meaning, unlike Avro, you don't need the schema ahead of time to read or write the data.

<comment><USER7> Okay, but they did a really poor job marketing it in this release. Plus, if it's used within Amazon, why it's Java-only so far?

<comment><USER8> Amazon's mainly a Java shop, not sure if that helps you.

<comment><USER7> Not really the case - a lot of major projects (like boto and AWS CLI) are in Python.

<comment><USER9> These are only client-side interfaces. The server-side is usually much larger.

<comment><USER7> True, but my point was that there's enough talent at Amazon, working on SDKs, and others, and there are precedents where even more complex projects such as JMESPath have wide support [0].

[0]: [http://jmespath.org/libraries.html](http://jmespath.org/libraries.html)

</comment></comment></comment></comment></comment></comment><comment><USER10> I Consider this Harmful (TM) and will oppose the adoption in every organization where I have an opportunity to voice such. (In its present form, to be clear!)

There is no need to have a null which is fragmented into null.timestamp, null.string and whatever. It will complicate processing. Just because you know the type of some element is timestamp, you must worry whether or not it is null and what that means.

There should be just one null value, which is its own type. A given datum is either permitted to be null OR something else like a string. Or it isn't; it is expected to be a string, which is distinct from the null value; no string is a null value.

It's good to have a read notation for a timestamp, but it's not an elementary type; a timestamp is clearly an aggregate and should be understood as corresponding to some structure type. A timestamp should be expressible using that structure, not only as a special token.

This monstrosity is not exhibiting good typing; it is not good static typing, and not good dynamic typing either. Under static typing we can have some "maybe" type instead of null.string: in some representations we definitely have a string. In some other places we have a "maybe string", a derived type which gives us the possibility that a string is there, or isn't. Under dynamic typing, we can superimpose objects of different type in the same places; we don't need a null version of string since we can have "the" one and only null object there.

This looks like it was invented by people who live and breathe Java and do not know any other way of structuring data. Java uses statically typed references to dynamic objects, and each such reference type has a null in its domain so that "object not there" can be represented. But just because you're working on a reference implementation in such a language doesn't mean you cannot _transcend_ the semantics of the implementation language. If you want to propose some broad interoperability standard, you practically _must_.

<comment><USER11> > It will complicate processing.

In practice, it doesn't. If you want to know if an IonValue is null, ask it with #isNull. If you don't care about the null's type, ignore it. On the other hand, the type is an additional form of metadata which allows overloading the meaning of a value.

nulls can also be annotated, so Ion doesn't really have the concept of a singular shared null sentinel.

More so than JSON, Ion often uses nulls to differentiate presence from value (that is, the lack of a field in a struct has a different meaning the presence of that field with a null value). Since nulls are objects, they can be tested separately from the lack of a field definition.

> a timestamp is clearly an aggregate and should be understood as > corresponding to some structure type.

Timestamps are structured types with a literal representation that is explicitly modeled in the specification. You're free to ignore it and use a custom schema for representing time, but you've moved any validation into your application at that point and are no better off than JSON.

<comment><USER12> I think the concern is that if you take an IonValue and cast it to IonText, then call stringValue(), you'll get an exception somewhere if the document contained a null value.

It recalls the nullability arguments between the ML family and the C/Java family.

kazinator is asking for safer document semantics and a type-safe API.

</comment></comment></comment><comment><USER13> This reminds me a lot of Avro:

[https://avro.apache.org/docs/current/](https://avro.apache.org/docs/current/)

They both have self-describing schemas, support for binary values, JSON- interoperability, basic type systems (Ion seems to support a few more field types), field annotations, support for schema evolution, code generation not necessary, etc.

I think Avro has the additional advantages of being production-tested in many different companies, a fully-JSON schema, support for many languages, RPC baked into the spec, and solid performance numbers found across the web.

I can't really see why I'd prefer Ion. It looks like an excellent piece of software with plenty of tests, no doubt, but I think I could do without "clobs", "sexprs", and "symbols" at this level of representation, and it might actually be better if I do. Am I missing something?

<comment><USER14> Amazon invented Ion because yaml, Avro, etc. didn't exist at the time. Ion is actually pretty old.

The timing of open-sourcing it mystifies me a bit. Maybe Amazon is trying to become more open-source friendly, like Microsoft did?

Perhaps more likely: they're planning on making some internal APIs that use ION heavily public?

<comment><USER15> Are we talking about a different YAML, or has Ion existed for ~15 years?

<comment><USER14> Woah, you're right. I thought YAML was much younger. I stand corrected.

Ion is definitely from last decade, at least. So most of the speculation in my post stands.

<comment><USER11> The JSON RFC was published right around the time Ion was being created making Ion about 10 years old this year. It was not clear at the time that JSON would become so popular ;-) and at the same time Ion fixed a lot of its weak areas (numeric types, dates, struct/object syntax, etc.)

</comment></comment></comment></comment></comment><comment><USER11> Big congrats to Todd, Almann, Chris, Henry, and everyone else who made this happen.

Several years ago, I wouldn't have imagined this possible and I'm a little bummed that I left before it happened.

Like leef said above, I'm glad to have Ion as an option again.

<comment><USER16> I am curious why you "wouldn't have imagined this possible", is the reason technical or political?

<comment><USER17> Political.

Amazon doesn't open source things, as a general rule. It can be done but it is a lot of jumping through hoops and they generally need good reasons to do it (as opposed to a lack of good reasons not to).

</comment></comment></comment><comment><USER18> Interestingly enough a JSON alternative named "ION" was just posted as a Show HN[0] about three months ago.

So now not only do we have the problem of redundant and mutually incompatible protocols (cue obligatory xkcd), but that we have _so many_ such protocols that name collision is becoming an extra problem.

[0] [https://news.ycombinator.com/item?id=11027319](https://news.ycombinator.com/item?id=11027319)

<comment><USER14> I mentioned this Ion in that thread, if anyone is interested in the ensuing discussion: [https://news.ycombinator.com/item?id=11028205](https://news.ycombinator.com/item?id=11028205)

</comment></comment><comment><USER19> Binary values can be stored as base64 in regular old JSON as well. Yes that is bigger but same as email/MIME binary chunks are converted to base64. Email messages and attachments are handled this way, we do this everyday. Base64 does bloat by 40%ish, so the larger content could be compressed/decompressed prior to base64 encoding it and vice versa or even encrypted/decrypted on either end in software/app layer.

No need for a new protocol when doing it that way for basic things, if you need more binary (busy messaging/real-time) there are plenty of alternatives to JSON.

I love the simplicity of JSON, so do others and it is successful so many try to attach on to that success. The success part was that it was so damn simple though, most attachments just complicate and add verbosity, echoes back to XML and SOAP wars which spawned the plain and simple JSON. Adding complexity is easy and anyone can do it, good engineers take complexity and make it simple, that is damn difficult.

<comment><USER11> > Binary values can be stored as base64 in regular old JSON as well

But in JSON you'd encode that Base64 as a string and the application must know that the data isn't really a string but a blob of some type of encoding. That probably means wrapping in another struct to provide that metadata. Ion provide a terse method of doing the same while maintaining data integrity:

        'image/gif'::{{ R0lGODlhAQABAIABAP8AAP///yH5BAEAAAEALAAAAAABAAEAAAICRAEAOw== }}

The 'image/gif' annotation is application specific, but all consumers know that the contents of that value are binary. In the binary Ion representation, those 43-bytes are encoded as a 45 byte value (one byte for the type marker and a second for the length in this case; as little as 47 with the annotation and a shared symbol table), making the binary representation very efficient for transferring binary data.

Since Ion is a superset of JSON, it's by definition more complex, but the complexity isn't unapproachable. Most of the engineers I worked with assumed it was JSON until coming across timestamps, annotations, or bare word symbols.

</comment></comment><comment><USER20> I can't decide if "JSON-superset" is technically accurate or not.

JSON's string literals come from JavaScript, and JavaScript only sortof has a Unicode string type. So the \u escape in both languages encodes a UTF-16 code _unit_ , not a code _point_. That means in JSON, the single code point U+1f4a9 "Pile of Poo" is encoded thusly:

        "\ud83d\udca9"

JSON specifically says this, too,

       Any character may be escaped.  If the character is in the Basic
       Multilingual Plane (U+0000 through U+FFFF), then it may be
       represented as a six-character sequence: a reverse solidus, followed
       by the lowercase letter u, followed by four hexadecimal digits that
       encode the character's code point.  The hexadecimal letters A though
       F can be upper or lowercase.  So, for example, a string containing
       only a single reverse solidus character may be represented as
       "\u005C".

       [… snip …]

       To escape an extended character that is not in the Basic Multilingual
       Plane, the character is represented as a twelve-character sequence,
       encoding the UTF-16 surrogate pair.  So, for example, a string
       containing only the G clef character (U+1D11E) may be represented as
       "\uD834\uDD1E".

Now, Ion's spec says only:

       U+HHHH	\uHHHH	4-digit hexadecimal Unicode code point

But if we take it to mean code _point_ , then if the value is a surrogate… what should happen?

Looking at the code, it _looks_ like the above JSON will parse:

      1. Main parsing of \u here:
         https://github.com/amznlabs/ion-java/blob/1ca3cbe249848517fc6d91394bb493383d69eb61/src/software/amazon/ion/impl/IonReaderTextRawTokensX.java#L2429-L2434

      2. which is called from here, and just appended to a StringBuilder:
         https://github.com/amznlabs/ion-java/blob/1ca3cbe249848517fc6d91394bb493383d69eb61/src/software/amazon/ion/impl/IonReaderTextRawTokensX.java#L1975

My Java isn't that great though, so I'm speculating. But I'm not sure what _should_ happen.

This is just one of those things that the first time I saw it in JSON/JS… a part of my brain melted. This is all a technicality, of course, and most JSON values should work just fine.

<comment><USER21> > But if we take it to mean code point, then if the value is a surrogate… what > should happen?

Surrogates are code points. The spec does not say what should happen if the surrogate is invalid (for example, if only the first surrogate of a surrogate pair is present), but neither does the JSON spec.

Java internally also represents non-BMP code points using surrogates. So, simply appending the surrogates to the string should yield a valid Java string if the surrogates in the input are valid.

</comment></comment><comment><USER22> Is there a source for benchmarks/reviews for the various ways to represent data? As far as I see it, there are a lot of them that I'd like to hear pros/cons for: json, edn + transit (my fave), yaml, google protobufs, thrift (?), as well as Ion.

And where does Ion fit here?

<comment><USER4> Ion's advantage is that it's both strongly-typed with a rich type system, as well as self-describing.

Data formats like JSON and XML can be somewhat self-describing, but they aren't always completely. Both tend to need to embed more complex data types as either strings with implied formats, or nested structures. (Consider: How would you represent a timestamp in JSON such that an application could unambiguously read it? An arbitrary-precision decimal? A byte array?) I'm not familiar with EDN, but it appears to be in a similar position as JSON in this regard. ProtocolBuffers, Thrift, and Avro require a schema to be defined in advance, and only work with schema-described data as serialization layers. Ion is designed to work with self-describing data that might be fairly complex, and have no compiled-ahead-of-time schema.

Ion makes it easy to pass data around with high fidelity even if intermediate systems through which the data passes understand only part of the data but not all of it. A classic weakness of traditional RPC systems is that, during an upgrade where an existing structure gains an additional field, that structure might pass through an application that doesn't know about the field yet. Thus when the structure gets deserialized and serialized again, the field is missing. The Ion structure by comparison can be passed from the wire to the application and back without that kind of loss. (Some serialization-based frameworks have solutions to this problem too.)

One downside is that its performance tends to be worse than schema-based serialization frameworks like Thrift/ProtoBuf/Avro where the payload is generally known in advance, and code can be generated that will read and deserialize it. Another downside is that it's difficult to isolate Ion-aware code from the more general purpose "business logic" in an application, due to the absence of a serialization layer producing/consuming POJOs; instead it's common to read an Ion structure from the wire and access it directly from application logic.

<comment><USER23> EDN supports dates, etc, too.

However, it doesn't support blobs. I'm conflicted about this point. On one hand, small blobs can occasionally be useful to send within a larger payload. On the other hand, small blobs almost always become large blobs, and so I'd rather plan for out-of-band (preferably even content addressable) representations of blobs.

</comment></comment></comment><comment><USER24> Surprised nobody mentioned CBOR ([http://cbor.io](http://cbor.io)) yet. Aka RFC 7049 ([http://tools.ietf.org/html/rfc7049](http://tools.ietf.org/html/rfc7049)).

<comment><USER25> It is referenced in the Ion docs: [http://amznlabs.github.io/ion- docs/index.html](http://amznlabs.github.io/ion-docs/index.html)

<comment><USER26> They complain about how CBOR is a superset of JSON data types and so some CBOR values (like bignum) might not down-convert to JSON cleanly, and then in _the next paragraph_ they talk about how Ion is a superset of JSON data types including 'arbitrary sized integers'. Bad doubletalk. Boo. (I have implemented CBOR in a couple languages and like it. Every few months we get to say, "oh look, _another_ binary JSON.")

</comment></comment></comment><comment><USER27> Wasn't this solved already by the BSON specification - [http://bsonspec.org](http://bsonspec.org) ? Sure this allows you a definition of types, but this could easily be done using standard JSON meta data for each field. I find BSON simpler and more elegant.

<comment><USER28> BSON is awful.

* It doesn't have "true" types in the sense that Ion does. It's basically just a binary serialization of JSON, with extra stuff.

* Despite being a binary format, it's actually bulkier than JSON in most situations.

* It removes any semblance of canonicity from many representations. A number, for instance, can potentially be represented by any of at least 3 types (double, int32, and int64).

* It has signed 32-bit length limits all over the place. Not that I'd _want_ to be storing 2GB of data in a single JSON document either, but it's not even _possible_ to do so with BSON!

* It requires redundant null bytes in unpredictable places. For instance, all strings must be stored with a trailing null byte, which is included in their length. There's also a trailing null byte at the end of a document for no reason at all.

* It is unabashedly Javascript-specific, containing types like "JavaScript code with scope" which are meaningless to other languages.

* It also contains some MongoDB-specific cruft, such as the "ObjectID" and "timestamp" types (the latter of which, despite its name, cannot actually be used to store time values).

* It contains numerous "deprecated" and "old" features (in version 1.0!) with no guidance as to how implementations should handle them.

<comment><USER29> Most of this comes from BSON also being the internal storage format for a database server. For example, at least the redundant string NULs make it possible to use C library functions without copying, the unpacked ints allow direct dereferencing, etc.

I've no clue about the trailing NUL on the record itself, perhaps a safety feature?

<comment><USER28> > I've no clue about the trailing NUL on the record itself, perhaps a safety > feature?

Could be. Or perhaps there's enough code paths in common between string parsing and document parsing that they decided to put a trailing null byte on both.

Stepping back a bit, though, the fact that BSON is optimized for "direct" use in C code is really scary. That suggests that any failure to completely validate BSON data could open up vulnerabilities in C code manipulating it.

</comment></comment></comment></comment><comment><USER30> > Decimal maintains precision: -0. != -0.0

What? This means their "arbitrary-precision decimals" are actually isomorphic to (Rational x Natural).

<comment><USER31> The use of != there is very confusing but what they mean is stores a precision along each number, not that -0 != -0.0

e.g. in Python:

        >>> from decimal import Decimal as D
        >>> 2 * D("1.0")
        Decimal('2.0')

        >>> 2 * D("1.000")
        Decimal('2.000')

        >>> D("1.0") == D("1.000")
        True

<comment><USER30> That just means == is a "lossy" equivalence relation. I rather the precision be truely observable----every number is "infinite precision". Once can always include natural as extra field if one cares about empirical precision.

<comment><USER32> I'm having a bit of trouble parsing this, but Ion decimal values are not "infinite precision". Every decimal has a very specific, finite precision. It's a standard "coefficient and exponent" model, with no specification- enforced limit on either.

<comment><USER30> The idea is each number is adequately precise in that additional precision would mean additional useless trailing zeros.

</comment></comment></comment></comment></comment><comment><USER33> Do any of the popular message serialization formats have first class support for algebraic data types? It seems like every one I've researched has to be hacked in some way to provide for sum types.

<comment><USER34> Protocol buffers support oneof, which is a union type. [https://developers.google.com/protocol- buffers/docs/proto#on...](https://developers.google.com/protocol- buffers/docs/proto#oneof)

(Insert joke here about Google engineers just copying around protobufs.)

</comment></comment><comment><USER35> Would like to see a comparison to EDN. [https://github.com/edn- format/edn](https://github.com/edn-format/edn)

</comment><comment><USER36> Can anyone share links to some examples, showcasing the differentiating features vs. json? I couldn't easily find any via the main link

</comment><comment><USER37> Almost every time I see yet another structured data format I'm surprised at the number of people who haven't ever heard of ASN.1, despite it forming the basis of _many_ protocols in widespread use.

<comment><USER29> Usual ASN.1 caveat: parsing its specifications requires money and a lot of time, implementing many of its encodings (e.g. unaligned PER) is a lifetime's work, and even the simpler ones thousands of eyes haven't managed to get right despite years of effort (see OpenSSL, NSPR, etc)

ASN.1 also has a million baroque types (VideotexString, anyone?) where most people just need "string", "small int", "big int", etc.

Some more on BER parsing hell here: [https://mirage.io/blog/introducing- asn1](https://mirage.io/blog/introducing-asn1)

<comment><USER37> _Usual ASN.1 caveat: parsing its specifications requires money and a lot of time, implementing many of its encodings (e.g. unaligned PER) is a lifetime 's work_

...unless you're Fabrice Bellard, who apparently wrote one just because it was one of the minor obstacles on the way to writing a full LTE base station:

[http://www.bellard.org/ffasn1/](http://www.bellard.org/ffasn1/)

</comment></comment></comment><comment><USER38> A question for frontend devs: Will H2 being binary on the wire inspire more use of binary data representations as well, with conversion to JSON only on the client? Passing around JSON or XML across a big SOA (or micro-services) architecture is a waste of cycles and doesn't have types attached for reliability and security.

<comment><USER39> Do you mean passing around binary between backend services and then having a binary->JSON "proxy" behind whatever is receiving AJAX requests from the client?

<comment><USER38> My idea was that the client (HTML+JS) will transform the binary data into JSON or skip the conversion and process it directly. Seeing how fast JS engines have become and the amount of typed binary arrays processed in JavaScript, I believe it's a viable approach. But I'm not a frontend dev, so I can't be certain.

<comment><USER35> Sounds a lot like the ActionScript compiler, which compiled their EcmaScript- style scripting language into bytecode for the Flash runtime.

</comment></comment></comment></comment><comment><USER40> How does Ion help with schema evolution? I see it mentioned, but not described.

<comment><USER11> In practice there are three properties that help with schema evolution:

        1) open types - typically applications consuming Ion data
           do  not restrict the fields included (that is, they
           gracefully ignore, and often even pass along additional
           fields). Schemas may grow while being backwards
           compatible with existing software.
        2) Type annotations allow embedding schema information into
           a datagram without the need for agreeing on special
           fields. Datagrams may have multiple values at the top
           level, so its possible to provide multiple
           representations without introducing a new top-level
           container.
        3) The only data might need to be shared between a producer
           and consumer is a SymbolTable which may be applicable to
           several schemas and may be shared inline if necessary.
           Otherwise, objects in a datagram are always inspectable
           and discoverable without additional metadata.

</comment></comment><comment><USER41> This appears to be something in between of JSON and Protocol buffers. I wonder under what conditions Ion makes more sense than either of the JSON/PBuff.

<comment><USER11> One significant advantage is you can opt-in to sharing schemas - <USER2> requiring all consumers to have your schema. Like a lot of Amazon's internal data formats, Ion designed to support backwards compatible schemas as well (that is, adding additional fields does not break existing consumers).

It has isomorphic text and binary representations as part of the standard making debugging or optimized transport a config option.

The type system is significantly richer than JSON and maps well to several languages (internally Amazon uses it with C, C++, Perl, Java, Ruby, etc.).

S-Expressions.

<comment><USER42> > without requiring all consumers to have your schema

Then how is the client supposed to handle the data? Guessing?

> backwards compatible schemas

> text and binary representations

> type system

> maps well to several languages

Protos have all these.

> S-Expressions

Okay? Is that useful?

<comment><USER43> > Okay? Is that useful?

I bet it's super useful if you have sexprs in your data.

<comment><USER11> It's interesting to have languages written for and in the same specification as your data.

</comment></comment></comment></comment></comment><comment><USER44> So far, most of the interesting bits I see in Ion are covered in YAML (which is also JSON-superset). Most of the rest are extra types, which YAML allows you to implement. The only really missing bit is the binary encoding... but that seems unrelated to the text format itself.

This really looks like a NIH specification.

<comment><USER32> Ion's equivalently-expressive text and binary formats is absolutely central to its design, and IMO one of its most compelling features. You don't have to choose between "human readable" or "compact and fast", you can switch between them at will. This helps Ion meet the requirements of a broader set of applications, eliminating the cost and complexity and impedance-mismatch problems you get by transforming between multiple serialization formats.

<comment><USER44> I get that binary format is nice, but I just don't get why instead of adding binary format to an existing good text format Amazon decided to first extend a poor text format and then add binary to that.

Basically: Ion == JSON + extra features + binary format spec. But Ion ~= YAML + binary format spec. You're going to write a new serializer/deserializer in both cases anyway, but in the second one, at least you get the text part for free in almost any language available.

</comment></comment></comment><comment><USER45> Is there any other implementation besides Java? I would be using it from iOS.

</comment><comment><USER39> Open question to anyone reading this: Would you use Ion if you were designing a new house-wide message queue? (e.g. broadcast messages to /Home/Lounge/Lights/ to turn on/off)

<comment><USER42> No that's overkill, just use JSON.

<comment><USER37> Even JSON is overkill for an application like that. Pure binary would be my choice.

</comment></comment></comment><comment><USER46> I use this [http://dataprotocols.org/tabular-data- package/](http://dataprotocols.org/tabular-data-package/)

</comment><comment><USER47> Things I dislike about Ion, having used it while at Amazon:

\- IonValues are mutable by default. I saw bugs where cached IonValues were accidentally changed, which is easy to do: IonSequence.extract clears the sequence [1], adding an IonValue to a container mutates the value (!) [2], etc.

\- IonValues are not thread-safe [3]. You can call makeReadOnly() to make them immutable, but then you'll be calling clone since doing anything useful (like adding it to a list) will need to mutate the value. While it says IonValues are not even thread-safe for reading, I believe this is not strictly true. There was an internal implementation that would lazily materialize values on read, but it doesn't look like it's included in the open source version.

\- IonStruct can have multiple fields with the same name, which means it can't implement Map. I've never seen anyone use this (mis)feature in practice, and I don't know where it would be useful.

\- Since IonStruct can't implement Map, you don't get the Java 8 default methods like forEach, getOrDefault, etc.

\- IonStruct doesn't implement keySet, values, spliterator, or stream, and thus doesn't play well with the Java 8 Stream API.

\- Calling get(fieldName) on an IonStruct returns null if the field isn't present. But the value might also be there and be null, so you end up having to do a null check AND call isNullValue(). I'm not convinced it's a worthwhile distinction, and would have preferred a single way of doing it. You can already call containsKey to check for the presence of a field.

\- In practice most code that dealt with Ion was nearly as tedious and verbose as pulling values out of an old-school JSONObject. Every project seemed to have a slightly different IonUtils class for doing mundane things like pulling values out of structs, doing all the null checks, casting, etc. There was some kind of adapter for Jackson that would allow you to deserialize to a POJO, but it didn't seem like it was widely used.

[1] [https://github.com/amznlabs/ion- java/blob/master/src/softwar...](https://github.com/amznlabs/ion- java/blob/master/src/software/amazon/ion/IonSequence.java#L457)

[2] [https://github.com/amznlabs/ion- java/blob/master/src/softwar...](https://github.com/amznlabs/ion- java/blob/master/src/software/amazon/ion/IonValue.java#L103-L112)

[3] [https://github.com/amznlabs/ion- java/blob/master/src/softwar...](https://github.com/amznlabs/ion- java/blob/master/src/software/amazon/ion/IonValue.java#L119-L140)

<comment><USER48> This is a good critique. Have you found anything better?

</comment></comment><comment><USER49> > <groupId>software.amazon.ion</groupId>

Why not "com.amazon.ion", like thousands of other existing packages?

<comment><USER50> They just want to use their shiny new-gTLD domain: [http://amazon.software](http://amazon.software)

</comment></comment><comment><USER51> Are there any object marshalling/serialization solution for Ion? (Like GSON, Jackson)

<comment><AUTHOR> It _is_ possible to adapt Jackson (with minimal effort) to use Ion, since it's very similar to Jackson's native JSON format.

</comment></comment><comment><USER39> I wonder how difficult this would be to port to C#?

</comment><comment><USER52> Why this instead of clojures "transit"?</comment>


Why Facebook is doomed to an eventual irrelevance - <AUTHOR> https://stacksavings.com/detail-post/Why-Facebook-is-doomed-to-an-eventual-irrelevance-BgkE7/en <comment><USER2> Interesting, but IMHO the main problem that Facebook faces today is not that people discover that it uses personnal data to get richer, but that Facebook is a leaking boat (should try a new logo, what about this one ? [https://i.pinimg.com/564x/3c/42/87/3c42873bfdd5fec3cb163fdc4...](https://i.pinimg.com/564x/3c/42/87/3c42873bfdd5fec3cb163fdc4292550c.jpg)). Even if its business model relies on an abusive use of personnal data and that people may be surprised to learn about his, people are more surely suprised to learn that Facebook did not secured the way it keeps those data. There is still some room for a firm offering services to people in exchange of personnal data, as long as this firm shows that it will use those data carefully. Obviously, Facebook won't be that firm, because this business model will be anti-Facebook by definition.</comment>


Ask HN: Is there a relation between age and learning style? - <AUTHOR> Has your learning style changed over the years? Or do you continue the same way? What works and what doesn't? <comment><USER2> Doing, not watching.

College tuition is essentially payment for someone grading my problem sets.

The lectures themselves were worthless.

</comment><comment><USER3> I'd say now I rarely 'learn' anything. I just Google it.</comment>


Ask HN: What do with if your experience is 1 year 8 times - <AUTHOR> Hello HN friends. I stayed in a company for 8 years and feel severily bad-skilled (I know it was the worst mistake in my life). The point is that I feel like I have 1 year of experience repeated 8 times. Last years I worked in a startup (defunct) and in a medium-sized company with success (project with multinational company as client), and from I know this project is now the most important one in the company and is only in maintenance mode (I keep contact with CEO and other guys there).
However, I recently started in a new place and I feel that am struggling to keep at pace and all my peers are younger than me.
So, fellow HNers, I need your wisdom and I wanted to ask you this questions:
- Have you been in a similar situation? What would you do to increase your skills?
- Is it imposter syndrome amplified to the 10x?
- Could I make up for the lost time?
- Is it possible to change my mindset and don't feel like I wasted my time?
- Could it be that making a Masters programme or some kind of education to feel more adequate?
- Is this a subjective "ego thing" and should to man-up and shut up? <comment><USER3> Yep. You probably wasted your time. Been There Done That.

I stayed at one company for 9 years between 2000-2008 between the time I was 25 and 34. My skills atrophied, I was woefully behind technologically - <USER2> were still using VB6 in 2008.

Suck it up, try to sell your maturity and soft skills, be prepared to check your ego at the door and learn modern technology and processes from people who are a lot younger than you.

You've got a lot of grinding to do, but you won't permanently be behind. There is a point where you get "get enough" for your local market and where any increase in experience has diminishing returns as an individual contributor.

More formal education is not the answer. "There is no compression algorithm for experience". You're going to have to work longer hours to stay caught up and to produce at the level you should until you catch up.</comment>


Derby – The simple way to cater - <AUTHOR> http://getderby.co <comment><AUTHOR> Hey guys, founder of Derby here. I mainly just wanted to see what the HN community thought of our web app. Feedback is very important to us, and we would love an outsider's perspective on what we've built. We will launch next week with mobile apps for iOS/Android. We'll be launching Derby to make corporate catering simple in Atlanta, GA.</comment>


HN: Looking to set up zipcar-like non-profit service for the poor. - <AUTHOR>

I'm working with an post exit software entrepreneur(who's backing financially) to try to figure out a way to get transportation solution for the poor.  Rather than just giving them a car, we're considering a free or significantly discounted zipcar like service.  However to figure out feasibility we need to figure out what type of costs are involved.
Anyone on HN have any ideas on how something like this could be insured/infrastructure set up, or pitfalls to look out for?
We've got some donated cars, which have been fixed and "guaranteed" by the local Toyota dealership for free (great people).  Now we just have to figure out the best possible way to expand such a program <comment><USER2> I think one of your big issues is going to be optimizing utilization. How can you get a single car to help as many people as possible? As notahacker mentioned, some people may need a car simply to get to and from a 8 - 9 hour shift. You don't want one person putting a car out of circulation for 10 hours. ZipCar works because it charges by the hour and keeps the cars idle in high-population, easy-to-reach areas.

Something to look into is how many people will have a smartphone. If not many (or for the ones who don't) I'd give them a cheap Android phone that does nothing but run your app. The phones can run on a group data plan (and like Republic Wireless' phones, WiFi whenever possible), and just be a more powerful kind of key fob that lets your members find, request, schedule, etc. cars. You could also use it as a key, requiring a password in the app to send a bluetooth signal that unlocks the car. I might also use these phones to connect to the cars' onboard computers whenever the car's in use, so that you can check fuel levels, engine status, etc. Some of that may be a bit further down the road -- you may want to go with more lowtech solutions while you get this up an running.

re: insurance and infrastructure, the best thing you can do is talk to people at ZipCar. You're not a threat to them, and they have the exact expertise you need.

<comment><AUTHOR> I was hoping maybe someone from ZipCar or Lyft would see the posting. I will take your advice and try to reach out to them directly. Believe it or not, the phone thing is a serious issue too. We had one family spending $169/mo on phone bill w/ 1200/mo in income. Maybe the ultimate solution would be somewhat of a combination of zipcar and lyft, since we don't anticipate charging for the car. (i.e. if you have one of our cars, you are somewhat obligated to give others a lift on your way). Thanks for the suggestions!

</comment></comment><comment><USER3> Or perhaps ZipCar isn't the business to emulate/disrupt but rather Lyft. I could imagine the poor getting a ride to their work much faster than the bus, and if it's faster that means more time with their children, more time for (or even the logistical ability to get) a second job, or hopefully more time to attend school. I could envision both outcomes: a service like Lyft that targets lower wage workers, or maybe even just subsidizing a Lyft membership.

The hazard I envision with ZipCar-lite is not only the monopoly the other commenter raised but also incidental costs (gas, accidents, etc) unless your plan is to subsidize those items, too.

Best wishes for your project, it could be great!

<comment><USER4> Reminds me of American Cancer Society's "Road To Recovery" program: [http://www.cancer.org/treatment/supportprogramsservices/road...](http://www.cancer.org/treatment/supportprogramsservices/road- to-recovery)

 _The American Cancer Society Road to Recovery program provides transportation to and from treatment for people who have cancer who do not have a ride or are unable to drive themselves. Volunteer drivers donate their time and the use of their cars so that patients can receive the life-saving treatments they need._

<comment><AUTHOR> I wonder how well something like that would work. Lyft is pretty popular in big cities but unheard of where I am. Reaching volunteer drivers sounds like a good idea, I guess we can experiment and see how much it costs to sign up a volunteer thru marketing the cause. Thanks for the info, I'll see if I can reach out to someone who is part of that program and see if I can get some pointers.

</comment></comment></comment><comment><USER5> One of your biggest issues to iron out is who qualifies for access to the scheme (anyone poor enough to consider renting a donated car for a very low fee? That's probably a lot of people, even in the US). Then you've got to figure out how much you want to spend promoting the scheme to those people - assuming the economics of the scheme are such that marketing doesn't generate a positive ROI.

Another issue might be that the genuine poor who most need cars are those that use them - all day every day - to go to and from their low paid jobs. If you intend the cars to actually be shared you'll have to choose a way of addressing that.

<comment><AUTHOR> We've already figured out the qualification part. This non profit is under the umbrella of another non profit which already does case management for poor families who are at risk of homelessness, usually due to loss of job. One of the big burdens on some of these families is a car payment from a buy here pay here scammer.

We're working on emergency housing, in conjunction with some other non- profits, however, the transportation situation is still unresolved.

We also noticed that many of the these families are in a similar area and often work in close proximity to one another.

Ride sharing or car sharing would not only be a greener solution, but I personally feel, it would be a far more effective solution. It may help lift some of these families out of poverty as they can actually get to a job/school and what is generally a usurious car payment and full insurance can go into a fund to get some training or actually save up to get a car.</comment></comment>


Cayzu Help desk any good? - <AUTHOR> http://www.cayzu.com <comment><AUTHOR> I am currently trialing them, and they seem to work really well. Just wondering if anybody out here has used them and what they think.</comment>


Ask HN: Anyone in Manhattan this Friday want to meet up? - <AUTHOR>

I'll be free in Manhattan (near 24th & 5th ave) from 11.30am Friday while my wife is on interviews.  Just wondering if anyone on here fancies meeting up, talking tech, start-ups and whatever else?  I guess also I'll be starting to look for positions in the area soon as well so anyone wants to chat about that let me know.
Shoot me an email at jamie [at] jamieos [dot] com :) <comment><USER2> I won't be in Manhattan this Friday but I live on Long Island. It would be nice to coordinate a HN meetup though!

</comment><comment><USER3> I'm in Manhattan but I'm only free after 5:30pm

</comment><comment><AUTHOR> I'm still looking to meet up if anyone is around just email me :)

</comment><comment><USER4> I could do it.</comment>


Great Things Take Time - <AUTHOR> http://mdubakov.com/post/44532952261/great-things-take-time

<comment><USER2> Sorry, but I don't agree. Really great software never ages. Like math: if it's correct, it never ages. Period. If it does age, it means it was not well engineered from the very beginning.

        «10 y.o. technologies are history. 10 y.o. software without major upgrades is dead. While 10 y.o. cars are quite good and decent.»

What about the Internet? What about LISP? What about UNIX? I could name a thousand pieces of software that date back from the 70s and are today as powerful and "new" as they were back then.

If the web does age, and it does, that should give us some pointers on what's wrong with it.

<comment><AUTHOR> Internet is improving. UNIX is improving -> Linux, etc. LISP? I'm not sure whether it still alive.

<comment><USER2> You're not sure whether LISP is still alive? O_O

As for the Internet improving... well, during its 40 years of age we reviewed some protocols, added some others, but I'm not sure that's what the author implies. The Internet (not the web, the Internet: <http://en.wikipedia.org/wiki/Internet_protocol_suite>) of today is quite the Internet of twenty years ago.

But still, if you prefer to, we can go down to algorithmic level, where things never age if they're done right.

</comment></comment></comment><comment><USER3> I can't decide if this post is asking people to move fast and break things, or iterate gracefully.

I think iteration is very powerful, but we don't seem to have fixed the problem of when something needs to iterate beyond its old ecosystem.

I haven't said that coherently, but I mean things like WordStar or WordPerfect moving from DOS into Windows - these two huge word processing systems failed to make the transition and are now niche products used by a tiny number of geek enthusiasts. They reached a point when iteration wasn't enough, and they needed some other thing.

<comment><AUTHOR> You define iterations like something small. In general, iteration can mean something huge, like full re-work.

I mean, improvements can be gradual or revolutionary. Still it is iterative process.

</comment></comment><comment><USER4> Would Google fall in the "fox" category ? They created significant stuff in various domains ...

<comment><AUTHOR> Huge companies are interesting. I think Google is closer to fox now. It closed down quite many products and focused on several (search, android, docs).

Maybe Fox/Hedgehog caregorization is not applicable to such huge companies. Most likely there are departments and products where foxes are in charge and vise versa.</comment></comment>


For YC Companies Raising Seed Rounds - <AUTHOR> http://blog.eladgil.com/2019/03/for-companies-raising-seed-rounds.html?m=1 <comment><USER2> The advise I give to all the YC companies is: prepare for the investor day as much as you prepare to the demo day.

The investor day can save you a lot of time fundraising later if you close few people on the spot, so make sure to be ready for a 20 minutes session with a longer coherent pitch and answers to the common questions.

In my batch (W17) the investor say was completely deemphasized for some reason, and many companies came unprepared, myself included</comment>


What Rutan Did - <AUTHOR> http://www.flyingmag.com/aircraft/turboprops/what-rutan-did

<comment><USER2> Ah, Peter Garrison, not only a great aviation writer, but a great writer, period.

I still remember his article about the original BD-5, and about his self- mantra ("RPM, RPM, RPM") and death grip on the collective during his first flights as he got his helicopter rating.

He is the kind of a guy who, in an article about flying gyrocopters at El Mirage dry lake, could turn it into poetry and write, "Only the soul survives loss of rotor at altitude."</comment>


I Broke Rust's Package Manager for Windows Users - <AUTHOR> http://sasheldon.com/blog/2017/05/07/how-i-broke-cargo-for-windows/ <comment><USER2> This is a great example of something else about software. As software grows in usage and use cases, it starts bumping up against edge conditions which need to be handled for various reasons.

Cargo now is becoming stronger and more stable because of bugs like this being discovered. All software goes through this growth cycle. It's great to see these things worked out in the various projects that support Rust.

There is another point here though; anytime the question comes up to just rewrite a piece of software, throw out all the technical debt, it's not as straightforward as it seems. Remember, together with that technical debt lies a lot of valuable learnings written into the code. I haven't worked on Windows directly in years, but I never knew that NUL was a reserved word as a file. I would, and probably still will make this mistake in the future.

Which makes me wonder, has anyone written a file name validation crate that guarantees that you're not writing to any reserved words on a filesystem of the host OS? A quick search of crate.io doesn't turn anything up.

<comment><USER3> It also shows how necessary it is to have some sort of deprecation process. Maintaining nonsensical landmine features for compatibility with an operating system released 36 years ago is putting the interests of MS's lazy long-term users ahead of the interests of its current users. Even if MS maintained a policy of only removing functionality after a 10-year deprecation period, this "feature" would have been gone long ago. Transitions must be orderly, but they should still happen.

It's nice that Rust's toolchain is better able to live Windows crazy ecosystem, but that doesn't make Windows any less crazy.

<comment><USER4> Joel Spolsky famously praised this policy of backward-compatibility at all costs which he called "The Raymond Chen Camp"[1]. Many agreed with him, but I always thought that Microsoft compatibility ideals were too radical to be real wisdom. At some point the list of features you try to keep compatibility with grows large enough that the Raymond Chen Way becomes unmaintainable.

The received wisdom of the 90s is wrong. Most users don't care about compatibility, as Apple's success has clearly shown, and most companies are now out following the Apple road. Large enterprise care about compatibility, and they pay a lot, but this is not a forward-looking market. They'll keep using buying new versions of your software because of the compatibility, but if compatibility is the only story you have to offer, you'll slowly lose that market.

I completely agree with you that Microsoft should have had a strategy for deprecating these features _back in the 90s_ , when they were already old.

In this specific case of outdated filename restrictions, you could start with what they already did: Windows NT 3.5 - Allow accessing all filenames with a special prefix (which they already did). Windows NT 4.0 - Make it easy to migrate to sane filenames by providing an opt-in per-process flag that makes all APIs use them by default. At this point they can easily dogfood and migrate all Microsoft software to the new APIs, so you would be able to delete these pesky files in explorer. Windows 2000 - Make the new API flag default for all versions compiled with the latest version of the Windows SDK. Windows XP - Make the new API default for any app without a special entry in the compatibility database.

Somewhere along the road, batch files (which is the only place where compatibility with the old filenames was necessary) could be easily made compatible by modifying the batch parser to replace redirections to NUL with redirections to \\\?\devices\null or something akin. You may see some breakage in scripts which use NUL and CON in non-standard way (e.g. as an argument), but the migration pain won't be huge, and you could still save an old script with a compatibility flag.

Microsoft obviously didn't take that way, and yeah, all the batch files written back in 1981 may still work without hitch, but newer things keep breaking in strange ways.

[1] [https://www.joelonsoftware.com/2004/06/13/how-microsoft- lost...](https://www.joelonsoftware.com/2004/06/13/how-microsoft-lost-the-api- war/)

<comment><USER5> Newer things only break in strange ways because they're broken. So rather than break the old stuff, why not fix the new stuff?? - because after all, approximately the only criticism you _can 't_ level at the Windows NUL/PRN/COMx/etc. special names is that they're some kind of surprise that appeared suddenly out of nowhere! It's been this way for a very long time.

(I wonder if part of this is the rage of Unix fans discovering that portable means actually, you know, making an effort... and that there's more too it than just checking it builds on x86 Debian as well as x64 Ubuntu...)

<comment><USER6> You can't just say it's been that way for a long time so it's acceptable, because the industry (and for that matter the Internet) is getting fresh new people every day. You can't expect them not to be surprised, and you can't just arbitrarily require them to know something they haven't stumbled upon until after it caused problems.

</comment></comment></comment></comment></comment><comment><USER7> Other magic aliases include CON, PRN, AUX, COM1-9 and LPT1-9. They are aliased to respective devices in Win32 namespace "\\\\.\". COMs and LPTs above 9 don't have aliases in global namespace and must be accessed explictly in Win32 namespace, eg. "\\\\.\COM10" (which itself is symlink to NT native "\Device\Serial9")

In fact, it is possible to create files named NUL, COM1, etc. using \\\?\ (eg. "\\\?\C:\NUL" is valid path) prefix which disables parsing arcane Win32 magic files. Unfortunately these files are causing strange behaviour in applications that don't use that prefix, Explorer included.

source: [https://msdn.microsoft.com/en- us/library/windows/desktop/aa3...](https://msdn.microsoft.com/en- us/library/windows/desktop/aa365247\(v=vs.85\).aspx#namespaces)

<comment><USER8> I still remember using "copy con foo.txt" and ending with ctrl-z to quickly create a file. It was years before I understood how that actually worked.

</comment></comment><comment><USER9> As the blog post mentioned, we solve the issue by deleting the crate from the package repository and reserving these problematic names. The incident lasted about 2 and a half hours.

Crate names have to be one or more valid idents connected by hyphens, so no other clever names like `/home` would be possible to upload. We already had some crate names reserved and we just needed to add these to the list.

<comment><USER10> Reserving just the crate names won't cover your bases, though, no? I'm not clear on what exists as part of a crate—but if there's any user control over the filenames of the _contents_ of the crate (e.g. if the crate's source code is in there) then any crate might contain a _file_ named e.g. "nul.rs", triggering the same problem.

<comment><USER11> I think you're misunderstanding the problem described in the OP. When you build a project via cargo using the default settings, it fetches the git repository at [https://github.com/rust-lang/crates.io- index](https://github.com/rust-lang/crates.io-index) to enable it to resolve dependencies locally. This git repository contains metadata for each library on crates.io, where the metadata for a given library is located in a file with the same name as the name of that library. When the OP uploaded a library whose name was an illegal filename on Windows, git unexpectedly choked when updating the local crate index repo, impacting all Windows users.

It sounds like the concern you're describing is a different matter. It's likely true that if the source of a crate contains a file named "nul.rs", cargo on Windows will fail if it attempts to git-fetch the source (unless you're using Linux Subsystem for Windows, anyway). While this would indeed be a problem, it would only affect users who elect to use specific libraries, rather than serving as a denial-of-service for every Rust user on Windows.

<comment><USER12> I was going to ask how a remote pckg could do that. Not knowing how rust works (or package managers apparently) I didn't understand how it could be widespread. Makes sense, damn; that's substantial.

<comment><USER11> I'm not sure how other package managers do it (it should be noted that this approach was designed to avoid some problems that other package managers have encountered), but there is still room for improvement here: ideally, I think we'd be hashing crate names rather than storing them verbatim on the filesystem, to enforce more uniform distribution in the trie.

<comment><USER12> Interesting, hashing them makes sense but it was a corner case; huge outage but it was definitely something that was easy to overlook for sure

</comment></comment></comment></comment></comment></comment><comment><USER13> There was a bug in Windows 95 (98 too?) where if you tried to open 'nul\nul' or 'con\con' etc, it would BSOD instantly. Provided lots of drive-by fun in computer labs... (got really good at typing win+r con\con)

<comment><USER14> For more fun, you could also target other machines with SMB shares. \\\thevictim\foo\nul\nul would BSOD _that_ machine. Good times.

<comment><USER15> IIRC, you could also reference it in a HTML page, so the whole computer would crash when that page was viewed.

</comment></comment></comment><comment><USER16> For those who don't use Windows and might need this info: [https://msdn.microsoft.com/en- us/library/windows/desktop/aa3...](https://msdn.microsoft.com/en- us/library/windows/desktop/aa365247%28v=vs.85%29.aspx)

<comment><USER17> That is a great page. Is there also such a page for Linux and Mac OS?

<comment><USER18> There's path_resolution(7) in the Linux man-pages set plus of course the relevant parts of the POSIX standard.

</comment></comment></comment><comment><USER19> What I don't understand is why cargo fetches the entire crate list and create a directory for every crate (even if you never install it). Why not just have a single file with the entire list? The issue mentions they use a trie, but why use the filesystem as the trie store? Why not have a single file?

<comment><USER11> The original authors of cargo, wycats and carllerche, aren't around today to ask (it's a weekend!) though IRC attempted to answer regardless:

      <foo> to keep the number of files in a single directory down
      <foo> tools become unhappy with hundreds of thousands+ of things in a single dir
      <foo> as do filesytems
      <bar> why not just a flat file
      <bar> or sqlite or whatever
      <qux> right now it uses git's deduplication feature
      <qux> aka, when downloading updates you only download the objects that changed
      <qux> but it mostly works on a per file basis
      <qux> so git hashes each file and if the hash didnt change, it doesnt download an update
      <qux> but if it did, it treats it as completely new file, even if its just a little change

<comment><USER11> Update:

      <wycats> Because of this: https://github.com/CocoaPods/CocoaPods/issues/4989#issuecomment-193772935
      <wycats> I ran some scenarios against huge repos when I first worked on cargo
      <wycats> Trying to minimize the cost of operations
      <wycats> I landed on the current strategy, and GitHub in the above thread more or less endorsed what we were already doing at that time
      <wycats> Also see https://github.com/rust-lang/cargo/issues/2452

<comment><USER20> It's still fundamentally a waste of disk space. On my system, as of a minute ago, ~/.cargo/registry/index took up about 200MB for three different checkouts (for some reason). After deleting that and running `cargo update`, only one of them is recreated, 104MB. Out of that, 57MB is the JSON files and 47MB is git history. But if I just concatenate all the JSON files, the result is only 33MB, and after gzipping, 3MB. Hypothetically, a non-GitHub-based Cargo could store only those 3MB (using binary deltas to avoid resending it on every update), or even 0MB if it just relied on the server to resolve dependencies.

<comment><USER21> Once you've gzipped to achieve that 3MB storage, binary deltas are useless. Perhaps it the data could be (almost certainly is) transferred gzipped, then expanded to the full 33MB size so binary diffs could be applied to it later, but setting up a system to do binary diffs is a lot of incidental complexity: xdelta is a surprisingly complex format, and bsdiff is really tuned for executables, not arbitrary content (and is pretty complex too).

It sounds like the biggest win would be for cargo to keep using git, but clone the crates.io index as a bare repository rather than checking out the plaintext content. Then it would only take 47MB by your count, which is pretty close to 33MB, and you could still get out the plain content with `git cat- file` and friends.

<comment><USER20> Technically, the Cargo /already/ bundles a full copy of libxdelta as part of libgit2 (in addition to the separate Git binary delta algorithm); I just checked using nm that it's actually included in the binary. It could probably be removed, but, well, it probably adds a lot less than 44MB to the binary size :)

Alternately, since JSON is text, I suppose you could just ensure that whatever emits this hypothetical merged JSON file puts newlines between different packages' entries, and then use a regular text diff (on the uncompressed version, of course). But reading 44MB of JSON isn't instant; it would probably be better to switch to either a binary format, or even something silly like a sorted list of JSON strings separated by newlines.

There would be some incidental complexity around generating and applying the diffs… you'd probably want to precalculate them on the server side, but it could be rather expensive to, on every change, calculate a diff between the current version and every previous change. Instead, you could have daily checkpoints: each day the server would make a checkpoint and calculate a diff to the last N checkpoints; on every update the server would recalculate the diff between the latest checkpoint and HEAD. The client would store both HEAD and a reverse diff to the latest checkpoint (or just store the checkpoint separately and waste a few MB), so when it updates, it could revert to that checkpoint and request the diff from there to the new latest checkpoint; it would also request the diff from the checkpoint to the new HEAD. If its checkpoint is too old then it would just redownload from scratch.

Overall, not a trivial change, but probably not too hard either.

apt-get does something vaguely similar with its pdiff files.

</comment></comment></comment></comment></comment></comment><comment><USER22> Sounds more like a problem with stupid Windows design choices than with anything you did.

<comment><USER23> Windows is, for better or worse, fiercely proud of its backwards compatibility. So it's not so much a stupid Windows design choice as a 'stupid' DOS 1.0 design choice (and not even so much a choice as simply a quirk of how the DOS 1.0 file system worked) that Windows doesn't want to break backwards comparability with.

<comment><USER12> I agree with parent that it's a bit crazy; but I wouldn't be as critical. to your point; presumably even if they dropped DOS support something between DOS and now likely relies on that. It's a fine line.

</comment></comment></comment><comment><USER24> In the Mac System 7-ish days, people used to earnestly warn each other not to name a file '.Sony' (a special name reserved for the floppy driver) as it supposedly trashed your HD. Although I've never heard of anyone reproducing it.

<comment><USER25> Now's your chance: [https://archive.org/details/mac_MacOS_7.0.1_compilation](https://archive.org/details/mac_MacOS_7.0.1_compilation)

I'd try it myself, but I've only got my phone with me.

<comment><USER24> 7.0.1 might be a little late, at least, for the supposed catastrophic results. It doesn't like the file at all but nothing dreadful seems to happen.

<comment><USER25> I tried it in that and System 6 ([https://archive.org/details/mac_MacOS_6.0.8);](https://archive.org/details/mac_MacOS_6.0.8\);) System 6 actually didn't care at all. An interesting bug.

I haven't actually done so, but earlier versions are available if you know where to look ([https://archive.org/details/mac_Paint_2](https://archive.org/details/mac_Paint_2)).

</comment></comment></comment></comment><comment><USER26> Wouldn't it make sense for Cargo not to use crate names in file names, and use hexadecimally encoded hashes instead?

<comment><USER27> Yes, or some other identifier that's unique to that crate. Assuming that the crate name is also a valid file name seems risky.

</comment></comment><comment><USER28> What did you end up calling the new crate?

Edit: I suggest "terminated"

<comment><USER29> The .toml file in the master branch on Github seem to still call it "nul": [https://github.com/SSheldon/nul/blob/master/Cargo.toml](https://github.com/SSheldon/nul/blob/master/Cargo.toml)

I can't find it on crates.io though.

<comment><USER28> I guess we can say, its To be Determined or entirely scrapped.

</comment></comment></comment><comment><USER30> Urgh, this "nul" filename / reserved filename bug is probably in a _lot_ of software.

<comment><USER31> Every MS-DOS programmer of old knows about nul, con, and the other reserved names. Those might come from CP/M actually (so are even older), and Atari TOS had them as well I believe.

</comment></comment><comment><USER32> I was working on a video project for a local comics convention, and named the project file "con.proj". That file hung around until I upgraded my hard drive because no file manager could delete it.

<comment><USER33> You can remove such files using Bash on Windows. `rm con.proj` works just fine.

<comment><USER32> This was years ago, I'm sure there were ways to do it but I didn't know about them.

</comment></comment></comment><comment><USER34> It's very tricky to do cross platform file handling stuff, and only the most mature projects have ironed out this. Just look at your pet project and see if it handles

\- Windows and unix line breaks in text files

\- Windows and unix path separators

\- BOM and non-BOM marked files if parsing UTF

\- Forbidden filenames such as in this article

By "handling" I mean it should accept or fail nicely on unexpected input - e.g. say that line breaks should be unix style, or paths should be backslashes etc. Very few projects actually do this well. Even fewer will do even more complex things like handling too long paths with nice error messages etc.

<comment><USER31> Since Windows 10 now comes with an official Linux subsystem, why not just use POSIX APIs and conventions everywhere, and not bother with Windows-specific code if possible?

<comment><USER35> For one, because the Linux subsystem is an optional install. If you're making anything user-facing you can't rely on it being there - it's really a tool for developers, not end-users.

<comment><USER36> I'm pretty sure there are already programs out there whose install instructions include activating developer mode and installing WSL Ubuntu.

</comment></comment></comment></comment><comment><USER37> I recommend ':?', as it will work in POSIX, but not Windows.

<comment><USER38> This sounds like end of Rust for next month for me...

</comment></comment><comment><USER39> While I know nothing of Rust, Diesel, or CrateDB, I do know that Windows uses a case-insensitive file system and this fix doesn't seem to take that into consideration. However, the author of the fix does note:

> I believe crates.io's namespace is case insensitive let me know if that's > wrong

Someone should probably validate that.

<comment><USER40> Not quite. Windows uses a case insensitive API on top of a case sensitive file system.

FUN FACT: As of 2017, Windows 10 is (partially) binary-compatible to Ubuntu Linux. Any application that was originally compiled for Linux will still be case sensitive when running on Windows 10.

<comment><USER39> Right. I forgot NTFS is in fact case sensitive. Thanks.

</comment></comment></comment><comment><USER41> I tried `npm install nul` on my win7 VM and it created a folder called nul which I can't get rid of ¬_¬

<comment><USER41> Well, notably `npm uninstall nul` gets rid of it. But in the Explorer you can't do anything with it.

</comment></comment><comment><USER42> Makes me wonder if I can make a crate called "../.." and have it overwrite some user files.

<comment><USER11> Crate identifiers are required to be [a-zA-Z] for the first character and [a-zA-Z0-9_-] for the rest. So, no. :P

<comment><USER43> More succinctly: [A-z]+[A-z0-9_-] _

<comment><USER11> If you look closely at your comment, you'll realize the hilarious HN bug that prevents me from writing it as you suggest. :P

<comment><USER36>
      [a-zA-Z][a-zA-Z0-9_-]*

FTFY

</comment></comment></comment></comment></comment><comment><USER44> Hmm... I feel like someone should stick the reserved names into a json somewhere for easy reference for the next package manager.

</comment><comment><USER45> Remember not to name anything pr#6 for either...

</comment><comment><USER46> Don't they have a continuous integration system where they run the unit tests on all platforms for all checkins to master?

<comment><USER47> Yes. Why do you ask?

<comment><USER38> Obviously, he wanted to know.</comment></comment></comment>


How Your Bank is Tracking Your Phone - <AUTHOR> http://www.wordloosed.com/how-your-bank-is-tracking-your-phone <comment><USER2> This is paranoia.

Much more likely that your card was flagged as bing possibly compromised by some other factor than your spending behavior.

For example If a a bank sees that many cards used at a particular retailer end up having fraudulent charges associated with them, they may suspect a problem with that retailer. They may then re-issue all of the other cards used at that retailer to be safe.

Also, people assume that because they sometimes get a call/email asking them if a purchase they made was valid, that they always know about how their credit card is used. This is not the case. Obvious fraud may be declined on the spot (think brute-forcing a CVV code, or a brand new retailer in Vietnam processing 100 cards while your last charge was in Kentucky). Some charges may go through but be flagged, reviewed, found to be fraud, removed from your account, and your card re-issued. Both cases are issues between banks and card processors, and may not necessarily show up on your transaction overview.

The likelihood of this happening is so much higher than the bank tracking your phone, that I would need a lot more evidence before I'd believe it.

</comment><comment><USER3> I work at a bank and I can tell you that they will often allow one or two instances of actual fraudulent card use before issuing a new card. It is actually very expensive for them to issue a new card and very inconvenient for the client so most won't do it willy-nilly.

I would also be remiss if I didn't mention the completely clickbait headline of the article that in no way reflects the actual content of the article. A more accurate headline might be, "Weird Coincidence Elicits Paranoia About My Bank".

<comment><USER4> I was just thinking this; banks take their time to issue a new card.

What're the odds that a new card was ready to go and posted (and that Royal Mail delivered it next day?) due to one questionable use?

</comment></comment><comment><USER5> I find this hard to believe, if this were so - every time I leave my phone on my desk and go to lunch across town I'd get a new card.

More likely is that a batch of details went missing from the issuer itself or a store that the owner had shopped at legitimately any time in the past reported a loss of data, and the timing was completely coincidental.

<comment><USER6> Then you're the sort of person who is not necessarily close to their phone. If banks are in fact doing this sort of thing, they surely take into account past behaviour to compute the probability that unexpected behaviour is related to credit card theft.

Maybe they don't even know that they are doing it. Maybe they just trained some neural network to attain certain levels of precision/recall. Maybe they feed the network with all the signals they can put their hands on. Things are not this simple anymore. Which also means that assuming phone tracking is a huge jump to conclusions, of course.

</comment></comment><comment><USER7> No, it is not

It may not have been the tire purchase that triggered it, but something else.

</comment><comment><USER8> I don't think that's what's going on.

I think it's more likely a standard multi-factor risk threshold that's been triggered. A vendor you've never used (or haven't used recently), a vendor category (auto parts) which you rarely purchase within, a higher risk payment processing method (perhaps they put through a CNP (cardholder not present) transaction) in a location (geo or vendor) where use of stolen cards is above average.

However, that said, many mobile banking applications require coarse (mobile network) and fine (GPS) location permissions ([https://play.google.com/store/apps/details?id=com.grppl.andr...](https://play.google.com/store/apps/details?id=com.grppl.android.shell.BOS) and [https://play.google.com/store/apps/details?id=com.barclays.b...](https://play.google.com/store/apps/details?id=com.barclays.bca) for example). I highly suspect that this data, along with other information gathered from your device such as IMEI, is used to assess login risk. If you look at the markup on a lot of internet banking login pages you'll often find Javascript and 1px images loaded from unusual subdomains at the bank (sometimes with "risk" or "security" in the name). A couple of the banks I use also embed hidden Flash objects, only on the login page, which I suspect are used for the same purpose.

My understanding is that they pull together data from a number of sources/signals to calculate a login risk score, in the same way virtually every bank calculates a transaction risk score when you use your card.

I doubt that this information is tied with physical card transactions, however.

Edit: I'll add that I'm the most surveillance/tracking conscious person I know (most just don't care) but this is a little paranoid even for me.

</comment><comment><USER9> This is a heck of a leap of faith and a very clickbaity headline.

</comment><comment><USER10> You may find it interesting that BillGuard is developing a new anti-fraud feature that (I believe) compares your phone's location to where your cards are used.

</comment><comment><USER11> what about testing this theory?

</comment><comment><USER12> just as easily they could compare the pace/rhythm at which you usually hammer you pin in or something alike

<comment><USER8> I don't think the EMV PED (PIN entry device) standard supports such a thing. You could certainly verify that by looking up the EMV standards.</comment></comment>


How Steve Jobs makes money from the iPhone - <AUTHOR> http://www.vator.tv/news/show/2008-10-23-jobs-gets-at-t-to-show-him-the-money-up-front

<comment><USER2> This reminds me of the article that i read on 'Common Sense in Business'. While most of the times, people say that marketing is all about common sense, time and again Apple has been sailing in the direction opposite to the wind, doing things that defy common sense, and creating newer and newer success models! Be it, launching better iPod to kill the existing ones or opting for a no subsidy model with AT&T.</comment>


Ask HN: Why doesn't Google update Gmail anymore? - <AUTHOR> I'm increasingly using 3rd party services to enhance Gmail (eg. Mixmax, 3rd party task managers). Gmail went from iterating rapidly to no major upgrades in several years. Why? <comment><USER2> Gmail is the legacy interface, Inbox is the modern interface. Gmail still exists specifically for people who don't want any major change, so it makes sense that it's not seeing major change.

<comment><USER3> I very much like Inbox. Shame about the speed, hopefully they get that sorted but haven't seen much in the last few months.

</comment></comment><comment><USER4> Very likely because Google is acutely aware of how many people will leave if they do. Gmail is an incredibly powerful, versatile mail client which has a ton of custom options like the density of the lists, that you'll find missing in all of Google's newer apps.

Inbox by Gmail was intended to be the new version of Gmail, and they decided to make it a separate site after the internal backlash against the Gmail team by other Google employees for "ruining" Gmail.

If you want the newest, Googliest version of Gmail today, use Inbox. And then there's Gmail, for those who want a dependable, stable email client that has all the features that people have come to depend on over the past decade.

</comment><comment><USER5> Gmail works well. Sometimes, it is good to follow "don't fix what is not broken" rule.

But out of curiosity, what would you like to see in gmail as an update ?

</comment><comment><USER6> There are also a lot of businesses that do not want the GMail interface to change.

For example, Yesware, Hubspot For Sales, and ToutApp are all built on top of GMail probably using the gmail.js library - and I'm sure there are a lot more.

I'm not sure they have the clout to keep Google from changing, but it's something for them to consider.

</comment><comment><USER7> inbox.google.com</comment>


Ask HN: Why the "snooze" alarm button is on focus? - <AUTHOR> <comment><USER2> Safer to snooze than to clear the alarm.</comment>


Confession: I Have No Idea How to Use Git - <AUTHOR> https://kevq.uk/confession-i-have-no-idea-how-to-use-git/ <comment><USER2> > Are there any resources available for people who are NOT seasoned software > developers to learn Git?

Is Google broken or something? There's literally thousands of resources out there, you just have to look for them. This list took me 30 seconds to put together:

[1]: [https://anitacheng.com/git-for-non- developers](https://anitacheng.com/git-for-non-developers)

[2]: [https://unito.io/blog/guide-to-github-for-project- managers/](https://unito.io/blog/guide-to-github-for-project-managers/)

[3]: [https://premium.wpmudev.org/blog/the-non-developers-guide- to...](https://premium.wpmudev.org/blog/the-non-developers-guide-to-git-and- github/)

[4]: [https://www.youtube.com/watch?v=SWYqp7iY_Tc](https://www.youtube.com/watch?v=SWYqp7iY_Tc)

[5]: [https://medium.com/@williamschinonso11/understanding-git- for...](https://medium.com/@williamschinonso11/understanding-git-for-absolute- beginners-479518d76532)

[6]: [https://www.freecodecamp.org/news/an-introduction-to-git- for...](https://www.freecodecamp.org/news/an-introduction-to-git-for-absolute- beginners-86fa1d32ff71/)

</comment><comment><USER3> For a very introductory perspective, the following sites are really useful:

[1]: [http://web.mit.edu/6.031/www/sp20/classes/05-version- control...](http://web.mit.edu/6.031/www/sp20/classes/05-version-control/)

[2]: [https://try.github.io/](https://try.github.io/)</comment>


Austria is the next country working to impose a tech tax on companies like Apple - sahin-boydas https://9to5mac.com/2018/12/30/austria-tech-tax-apple/ <comment><AUTHOR> I am worried that this tax is partially based on jealousy and envy. I'm not so sure we should create a very complex tax system to specifically target companies and industries we feel deserve to pay more. Eventually this will end up a very complex tax system where nobody feels they are treated fairly.

If tax avoidance is legal, is it really tax avoidance? We've had leftist and rightist governments for 30 years saying they want to close tax loop holes, but they seem to be unable to create a system that works and successfully makes large corporations pay tax on profits.

I agree that it's messed up that Facebook and Apple make all their profits on tax havens, but I'm concerned that we are a bit too narcissistic by believing we can create a complex tax system that will solve this.

Humans have a dark side and seem completely unaware that their believes are driven by self-interest, envy and hate rather than making systems that work.

<comment><USER2> Taxes were always paid by those who could bear them. Nobility and landowners in the past, AmaGooBookSoft today.

It could well be motivated by envy. But it can't be any other way because you can only effectively tax people who have the money.

<comment><USER3> Actually the peasants were taxed to pay the nobility, not the other way around.

<comment><USER2> Peasants were obligated to provide some services to the nobility, and sometimes taxed in nature, but real taxes were either from monopolies or the landowners.

It's like saying that really consumers are being taxed here because they are the ultimate source of revenue for Apple. True but you can shift the distribution of this burden between consumers and Apple's owners.

</comment></comment></comment></comment><comment><USER4> To put this into perspective: The tax revenue in France for 2016 was 1,013,134 million Euro (€1 trillion) [1].

The additional €500 million constitutes less than 0.05% increase.

[1] [http://www.oecd.org/tax/revenue-statistics- france.pdf](http://www.oecd.org/tax/revenue-statistics-france.pdf)

</comment><comment><USER5> > According to dw.com, the European Commission estimates that tech firms pay > an estimated 9% on profits, as compared to 23% for traditional companies.

Why not figure out why that is and close whatever loopholes currently exist instead of introducing a targeted tax meant to close that gap?

<comment><USER6> Because the loopholes exist EU wide, but EU legislators failed to collaborate to close them. That's why individual nations try to find ways to level the field.

(As an Austrian I'm fairly certain that this has nothing to do with fairness: Everybody hates FB and Google at the moment, so the populists scream for taxation. At the same time Austrian companies who are active internationally get tax breaks.)

<comment><USER7> >Everybody hates FB and Google at the moment

Serious question for OP - do you think this is because people feel these companies are truly doing something wrong or a perception that these companies (and their employees in California/Silicon Valley) are making so much money that it's not fair?

<comment><USER6> My perspective: Tax for the middle class is insanely high. Seeing companies like FB, Google, Amazon, Apple, etc. pull off insane stunts to "minimize" tax payments (because of their responsibility to the shareholder). They make (and hoard) insane amounts of money without giving back.

You might say, "if you don't like them boycott them". For Amazon and Apple it's easy. Google and FB are inescapable if you need to advertise.

<comment><USER7> Thanks for your perspective! Is there any local push back on the high middle class tax rates?

<comment><USER8> Another Austrian here. The problem is not the rate per se, but the fact that those tax evasion practices of international conglomerates put local small and medium businesses at an unfair and unjustified disadvantage.

</comment></comment></comment></comment></comment></comment><comment><USER9> If we project out even a few decades, it's looking like corporate automation is going to put a permanent ceiling on wages for anyone working outside of corporations. Products and services will get cheaper and cheaper (approaching free) but nobody will have any money to buy them with. If we argue for everyone joining corporations then we get fascism. If the majority want to live outside corporations then we end up with Luddism and vast underemployment. Other than global climate change and loss of biodiversity, this will likely be the great problem of our time.

If we look at the endgame around 2040 or 2050 (assuming there's no strong AGI singularity yet but that we have widespread weak AI) then a lot of people are going to be out of work or doing menial work that wastes all human potential. We're seeing that today as large swaths of the economy get eaten by non- productive work like finance, management, bureaucracy, etc. Also roughly 5 billion people in the world work in farming, mining, timber, manufacturing, etc but are underpaid by roughly 100:1 because they have little education or leverage to demand more for their families.

So honestly, the next economy after unregulated capitalism is probably going to look like highly-automated corporations that are taxed at a high rate (perhaps 90% eventually) where the tax revenue is redistributed as universal basic income (UBI). One reason why this is probably inevitable is that robots don't pay into social security or other safety nets. If we don't want people starving in the streets then that revenue has to come from somewhere, and the most proven revenue stream we have is corporate profits. So Austria appears to be one of the trend setters.

I don't see many other viable alternatives at this point, so am open to suggestions!</comment>


Walmart – OneOps Now Available as Open Source - <AUTHOR> http://www.walmartlabs.com/2016/01/oneops-now-available/ <comment><AUTHOR> [https://github.com/oneops](https://github.com/oneops)

</comment><comment><USER2> Discussed at [https://news.ycombinator.com/item?id=10975838](https://news.ycombinator.com/item?id=10975838).</comment>


MenderCon 2020 – a virtual conference focused on software modernization - <AUTHOR> https://corgibytes.com/blog/2020/04/20/announcing-mendercon-2020/ <comment><USER2> At first glance I thought this was about the Mender IoT project (mender.io). Is is not.</comment>


Gmail Overhaul Might Be On the Way - <AUTHOR> http://mashable.com/2013/05/25/gmail-overhaul-rumor/

<comment><USER2> I'm not the only one who feels frustrated by these "news" articles based on rumors that a popular service might be changing, of all things, their UI?

If it changes, we'll see it when it changes. No need to talk about it before hand.</comment>