RedHat and SUSE Withdrawing Support for OpenLDAP - <AUTHOR> https://www.ostechnix.com/redhat-and-suse-announced-to-withdraw-support-for-openldap/ <comment><USER2> This is a real shame, Howard is an absolute asset not just for his LDAP work, but for the LMDB database engine borne from it and the ton of supporting technical materials surrounding it. Symas (Chu's consulting company) have a far clearer take on the matter: [https://symas.com/message-president- regarding-red-hat-suse-r...](https://symas.com/message-president-regarding- red-hat-suse-removing-openldap-linux-distributions/) , seems this is entirely motivated by upsell.

Red Hat's move here is a bit like taking a trusted well-supported tool like bash and replacing it with some shit branch of ksh from 1984, and forcing people to pay for it.

<comment><USER3> That article is _incredibly_ biased. Precisely because it's run by Chu's consulting company, I cannot trust them when they describe 389DS as "a less- capable LDAP server". It is simply one company attacking a competitor's product.

And of course the article contains direct links to both a page where you can buy support packages for OpenLDAP and something called "OpenLDAP Gold".

<comment><USER2> That is not an opinion based on any marketing material - 389DS is an unreliable dog, and even today it's still built on top of an Oracle-owned database engine.

There is no need to attack Symas for selling a supported variant of OpenLDAP, it's valid under the license after all, they have been the exclusive maintainers for OpenLDAP for many, many years, and it is of course the exact same model used for 389 DS.

Meanwhile everyone is free to share links to counterpoints in support of 389DS, assuming you can find any.

<comment><USER4> Let's break this down as an external party that uses neither products:

> "That is not an opinion based on any marketing material - 389DS is an > unreliable dog"

The press statement you linked to reads as an advertisement. You posit a statement as fact without any information to back it up. Why is it so unreliable?

> "and even today it's still built on top of an Oracle-owned database engine."

Whether the product is built on an Oracle-owned database engine or not, is not a reflection of the stability of a product. It's clear that you mention this as a negative to reinforce your position, but while technically correct, looking through the history of the product, it was developed by Sun and acquired by Red Hat in 2005, years before Sun was acquired by Oracle.

> "There is no need to attack Symas"

@amyjess's comment simply declared your linked article as biased, which it clearly is. This is not an attack.

> "Meanwhile everyone is free to share links to counterpoints in support of > 389DS, assuming you can find any."

Why do we have to assume the product is broken and find proof that it is not? You have posited an opinion without any sources to back it up; the burden of proof lies with you.

(Edited for formatting changes)

<comment><USER2> You make a fair point, it is hard to read as a bystander. However I disagree the burden of proof lies with me. I'm not here to convince you of anything, if you want to be sure of something, you must put the effort in like everyone else. Despite that..

Just focusing on storage backends - if you have ever deployed anything that uses BDB (Subversion, Apache, 389, Exim, god knows how many more), one thing you'll become quickly familiar with is it's amazing ability to randomly get into an unusable state, despite an otherwise healthy and unrebooted machine, despite no administrative interference, despite apparently no users or no load at all (e.g. on a Monday morning in a ~10 user office).

My opinions are based on having worked in infrastructure since ~2002, having deployed BDB many times, and having spent many hours trying to figure out how to get an app unbricked without losing its underlying database.

I spent time as an administrator at several companies where Subversion would go down monthly or worse because BDB locking would get messed up like this, and the result is always yet another tarball of the previous database just in case poking it completely corrupts the install. I've had to handle emergency calls because Exim was spewing permanent errors to customers due to a BDB lookup randomly starting to fail.

In other words over the years, like many before me, and for very good reasons, I've come to consider BDB a red flag, and where it goes trouble goes. Put in simpler terms, on a scale of software quality running from 10 ("excellent") to 0 ("crapware"), BDB sits somewhere around -7 ("you had one job"). But don't take my word for it, look at the list of common hazards listed here: [https://web.stanford.edu/class/cs276a/projects/docs/berkeley...](https://web.stanford.edu/class/cs276a/projects/docs/berkeleydb/ref/debug/common.html)

Despite that, having deployed OpenLDAP and suffered through its.. suffering.. documentation on several occasions going back ~15 years, including during periods when BDB was the default, post installation I have never once seen OpenLDAP going down. It's very solid software, in spite of prior database engine hindrances.

LMDB is of course no panacea, but having spent years with it as a developer (and maintainer of py-lmdb), I can say categorically that I would not pick BDB over it in any situation, not just due to performance concerns (which are great), but because the design is so damned simple (a single write lock, a single lock file) that I have never once seen it self-corrupt or get wedged in ways that I would have struggled to deal with in my time served an administrator.

<comment><USER5> Hmm...I've spent decades using BDB and honestly this doesn't jive with my experience at all. I remember the Subversion debacle, but wasn't that resolved before 2005?

<comment><USER2> As I recall, it was only finally resolved by Subversion moving to a home-built database backend, the issues with BDB were never 100% solved.

</comment></comment></comment></comment></comment></comment></comment><comment><USER6> For a large enterprise deployment of LDAP I used OpenLDAP over 389DS/RHDS/FreeIPA because it was straight-forward and solved the problems of a multi-master and slave-to-each-master architecture well.

The initial deployment used Sun Directory Server Enterprise Edition (Sun DSEE, now Oracle DSEE) but when Oracle bought Sun we had conversations with Oracle's product team where they explicitly stated that going forward Sun DSEE would be discontinued and replaced by Oracle's competing (but from our perspective, inferior) product.

Contrary to most of the posters here I had no significant trouble figuring out OpenLDAP and found the documentation complete. The main pain points I ran into were: \- OpenLDAP was trying to move configuration inside the DIT, which made system administration harder \- I ran into some bugs with MDB on Solaris/UltraSPARC before it was officially announced \- I ran into some bugs, which the OpenLDAP team fixed

In a later job, I was supporting a customer who purcashed RedHat Directory Services (RHDS) and therefore they wanted to use this in preference to OpenLDAP, but it conveyed no advantages, and had worse characteristics and licensing requirements with regards to multi-master replication. The reason the customer purchased RHDS was because their "Solution Architect" team thought they could use it to integrate with Active Directory authentication. However, RHDS's integration mechanism required that something be installed on all Active Directory servers (of which there were around 200, managed by various other companies) this plan was not feasible. Instead, I created a PAM module and used RHDS's "pam_tally" to pass through authentication attempts. This was not as good as could be done on OpenLDAP, however, because pam_tally creates a lock in RHDS and no authentication could happen while it was authenticating -- leading to a few nation-wide lockouts while I figured this undocumented feature out.

</comment><comment><USER7> I recently installed OpenLDAP on a server because I wanted an LDAP Server.

Sadly, LDAP isn't the most accessible system in the world and I only managed to set it up using FreeIPA, which is based no the 389 system mentioned in the article.

Most advice online around LDAP seems to amount to "ask your sysadmin", which isn't terribly useful either when I'm the sysadmin.

I hope 389DS will make LDAP a bit easier for newcomers like me.

<comment><USER8> I built my company's LDAP infrastructure using OpenLDAP from scratch. It's redefined my pain threshold in the process!

OpenLDAP is a solid piece of software, no doubt, but there's one critical problem - most of the guides are third-party because the OpenLDAP documentation has been pretty lacking for a long time, and those guides are usually from the era before the entire thing was redesigned around OLC and LDIF! Adapting the guides is beyond irritating. I'm now at the stage where I have the schema defined and I no longer want to touch it, /ever/!

<comment><USER9> My guess is that it's consultingware. Open Source projects with crap documentation usually are. You're meant to hire the devs as consultants.

I could be wrong, but it's a common pattern.

</comment></comment></comment><comment><USER10> Source was down for me, here's the archive link: [https://web.archive.org/web/20180828133125/https://www.ostec...](https://web.archive.org/web/20180828133125/https://www.ostechnix.com/redhat- and-suse-announced-to-withdraw-support-for-openldap/)

</comment><comment><USER11> If I'm reading this correctly, they are withdrawing _commercial_ support. I don't see the problem here; if you want 'normal' FOSS support you'll get whatever you are used to, and if you want paid support from your distro vendor it matters just a little bit what implementation you use since you're paying someone else to make it 'just work'.

</comment><comment><USER3> According to the article (and articles I've read on this subject before), they are not ditching LDAP entirely but rather they are switching which implementation they'll be supporting from now on: instead of supporting OpenLDAP, they will be supporting the 389 Directory Server, which does everything OpenLDAP does.

<comment><USER12> 389 is based on Netscape Directory Server which forked UMN's slapd way back when. Red Hat sells a branded version of it called "Red Hat Directory Server", although last I checked everything was released under reasonably free software licenses. Frankly, as long as I can install current OpenLDAP packages on CentOS/Fedora from base, an SCL, or EPEL, I'm OK with this change.

</comment></comment><comment><USER4> Reading the article, it doesn't look like RedHat announced anything recently. In the SUSE release notes, "389 Directory Server replaces OpenLDAP to provide a sustainable directory service." Is there really any news here?

</comment><comment><USER13> If they linked to the Redhat article, it would have removed some confusion I believe. [1] Or maybe they did and I missed it?

[1] - [https://access.redhat.com/solutions/2440481](https://access.redhat.com/solutions/2440481)

>Important: Starting with Red Hat Enterprise Linux 7.4, the openldap-servers package has been deprecated and will not be included in a future major release of Red Hat Enterprise Linux.

<comment><USER14> > If they linked to the Redhat article, it would have removed some confusion I > believe.

Given the source of the article, they're not looking to _remove_ confusion, they're looking to _benefit_ from it.

</comment></comment><comment><USER14> > Since 2003 Univention supports OpenLDAP for enterprise use. It is one of the > central components of their product Univention Corporate Server (UCS).

...

> This is a guest post from Univention.

</comment><comment><USER15> I'm in a small (< 10 users) shop having OpenLDAP to store users that integrates with Apache authentication, postfix/dovecot authentication and manages users with (patched) phpLdapAdmin but hearing that OpenLDAP may not have a bright future, what should I replace it with for central user management for Linux?

<comment><USER16> OpenLDAP will not go away, see [https://lwn.net/Articles/755207/](https://lwn.net/Articles/755207/) for a current road map. Only RedHat doesn't support it any longer, that's all, no worries.

<comment><USER15> Seeing how I need a patched version of phpLdapAdmin maintained by a third party instead of the one provided by Ubuntu 18.04 and a lack of easy to use GUI app for LDAP administration, let alone any third party client interface for users to login and change their password, I'm starting to think no one likes LDAP and should move over, perhaps even to MySQL as user management database.

</comment></comment></comment><comment><USER17> Shouldn't the title in some way allude to that the article was written by a RedHat/Suse competitor?

</comment><comment><USER18> As a long time OpenLDAP user (ie. probably going on 20+ years at this point), and as somebody who will be rolling out LDAP again on a long-term project on RHEL 7.5 shortly I have a few questions about 389DS. I've considered migrating to it for this project, but I just have too many open questions regarding setting it up right now.

1) I need support for both RFC2307 and RFC2307bis. I haven't found any good documentation on how to set these up from scratch with 389DS where it's fairly trivial to make them work together with OpenLDAP and the memberof overlay.

2) I also have a need for an equivalent of the Tacacs schema that works with the mavis-Tacacs fork that Google maintains. Will this schema work with 389DS?

3) Same goes for the sudo.schema that is provided with sudo - does this work with 389DS?

4) Same goes for the ppolicy.schema (password policy) schema (ppolicy overlay) and memberof overlay. What are the equivalents of these for 389DS?

5) multi-master replication - OpenLDAP makes this a breeze, but not so sure with 389DS.

Finding information about how to implement these things from scratch seems extremely hard for 389DS. Granted, it was also hard for OpenLDAP, but that work is behind me now. I have serious misgivings about trying to reimplement all of this on 389DS within my given timeframe.

Other things that irk me about this decision:

a) I really dislike the Java console - doesn't feel like it changed much since the 1990's. It's still slow, ugly and doesn't feel like it integrates well.

b) feels like a weird choice to drop the openldap-server packages but not the openldap-clients package. Are they going to provide a replacement for the OpenLDAP client libraries eventually as well? Is there even a good alternative (ie. API compatible and replacements for all the command line utilities) to openldap-clients?

While it seems like an easy decision to move on the surface, most OpenLDAP deployments have a massive amount of customisations to make all the third- party applications work, and 389DS making those kind of migrations easy doesn't currently seem like one of its strong selling points.

<comment><USER19> >b) feels like a weird choice to drop the openldap-server packages but not the openldap-clients package. Are they going to provide a replacement for the OpenLDAP client libraries eventually as well? Is there even a good alternative (ie. API compatible and replacements for all the command line utilities) to openldap-clients?

They have no choice here, they use OpenLDAP's libraries. Mozilla/Netscape's own LDAP SDK has been unmaintained for more than a decade; it's abandonware.

<comment><USER18> If they have the resources to maintain their own directory server (389DS, Netscape or iPlanet, whatever it's called this year) surely they can manage to ship some client libraries of their own as well.

<comment><USER19> You are correct, and that should tell you all you need to know.

<comment><USER18> So my impression is that the goal for Redhat here is to sell RHDS licenses, rather than rid itself of having to support OpenLDAP on the server side. But developing client libraries, which Netscape/Sun/389DS/Mozilla etc. all failed to do, is a bridge too far. Oh well.

</comment></comment></comment></comment></comment><comment><USER20> There is an interesting fork of OpenLDAP called ReOpenLDAP - [https://github.com/leo-yuriev/ReOpenLDAP](https://github.com/leo- yuriev/ReOpenLDAP)

</comment><comment><USER21> I wish 389ds would get rid of the java requirement. I'm sticking with Samba4 right now, both for AD and as an LDAP store for apps; replication handles itself, sites etc, makes life so much easier.

<comment><USER5> There is no Java requirement.

<comment><USER21> As "vetinari", you need it for the console.

<comment><USER5> And you don't need the console.

</comment></comment></comment></comment><comment><USER22> ahh, OpenLDAP, I've a weird Stockholm syndrome with that piece of software.

It's probably one of the most reliable nosql DB out there. And you can actually implement schemas to ensure data formatting \o/ (provided you like OIDs...).

It can support quite rich ACLs, but those are a little hard to debug and fix. But it's nice it is there.

You also have an interesting query language using polish notations (okay, I prefer to slapcat the complete directory and do '|less' or '|grep' if I've direct access to the server and if it's not too big, I could never remember the options names in ldapsearch).

Lastly the ldif syntax is so great, I always have to google it even for the most basic edits...

More seriously, ldap is somewhat painful to deploy:

* Indeed the documentation is a bit lacking, the zytrax one is great but somewhat dated (specially with the cn=config paradigm).

* If you are doing complex setups like n-way multi-masters replication the documentation is somewhat verbose and really hard to understand ([https://www.openldap.org/doc/admin24/replication.html](https://www.openldap.org/doc/admin24/replication.html)). Generally it involves a lot of trial and errors in a lab to figure it out.

* There can be traps if you are not aware of the internals. For example, in one of our deployments we were maintaining large groups, and batch updates were taking a really long time. After digging, I finally understood why: each time you add or remove an entry in a group, it rewrites the complete group entry (at least with the BDB backend), rewritting 100000 entries each time you update one is somewhat costly... Not sure if it's still the case with the lmdb backend however.

* The source code is not really easy to read in my opinion. I did try it once (debugging the issue mentioned above), but was not successful at roughly understanding how it worked from reading the source code, at least in the few hours I devoted to it (I know, it's not necessarily a lot).

* Usually, with replication, schema tweaking, backups, monitoring and some testing, deploying OpenLDAP can easily be a one or two months project.

* deploying all the integrations correctly is also quite hard (ldap auth for ssh or web authentication on each web applications),

That being said, 389 Directory/RedHat Directory Server is not much better, the setup is done through a weird perl script. Setup ssl/tls is a pain (I learn to hate certutil, and its certs/ca/private keys in DBs, deploying flat pem files is so much easier and convenient). The "rich" client looks like it has not changed in years (or removed all together), and it looks so much like the Old Sun One I'm wondering if it has changed since the Netscape days apart from code maintenance.

Also, I'm not sure if it's still the case, but IIRC RDS is a separate subscription from RedHat, it doesn't come with a basic RHEL OS subscription (I recall having to recompile the srpm my self when 6.0 went out).

RHEL was also not a really great maintainer of their OpenLDAP packages, I remember for example a breaking change because they went from 2.4.38 to 2.4.41 (I'm expecting a stable distro like RHEL to not introduce breaking changes, and to not update software like a rolling release, Debian does a far better job at it for example).

After all this 'praise' of the ldap world, you must think I actually hate it. No, I actually kind of like it (to the point of maintaining an ldap web UI). Having a centralized user DB makes things so much easier for everybody, the user only has one password, the security officer can trustfully enable/disable accesses, the sysadmin as one source of truth. If you maintain local accounts for each of your applications you are in for a lot of troubles down the line.

It's also an authentication method that is widely supported by most applications, and if it's not, there is probably a ticket/PR/patch somewhere to add it.

Lastly once it's setup and configured, it's generally quite reliable even with complex replication setups, this makes it relatively low maintenance. And at least for reads, it has quite good performances.

OpenLdap: I love to hate you, but I love you.

</comment><comment><USER23> The link has an internal server error :(

</comment><comment><USER24> Will the LDAP server merge into systemd or will it merely exclusively product tie or will it just be marketing?

Just curious how easy it'll be to continue using what works as opposed to what marketing wants to sell.</comment>


Adam Cohen: Is the cryptocurrency Bitcoin a good idea? - <AUTHOR> http://www.quora.com/Bitcoin/Is-the-cryptocurrency-Bitcoin-a-good-idea/answer/Adam-Cohen-2

<comment><USER2> i have no clue who adam cohen is, but he clearly didn't even read the FAQ, let alone the fine manual, on bitcoin.

and to top it off, his 'analysis' starts with a diatribe BEFORE he make any point.

He may have a point on item 2 and 3. but reading point 1 was just whoooooosh.

and i'm not even into bitcoin!

<comment><USER2> and on point two, he also compares bitcoin to a currency that has a government printing money.

the fact that you want a currency to look like a number of twitters graph he show is because you are assuming inflaction by government action. Where people gamble with the need of said government to print money!

if you remove government action, of course people will hold to money --if they do not have the need to spend. that's the whole point of it!

I hold down to my money in the hopes that i will have means when i need it. economists juggle money to trick me out of my wealth while i sleep!</comment></comment>


Show HN: Palindrome Haiku poem generator - <AUTHOR> https://github.com/khamoud/haikupal

<comment><AUTHOR> It is not perfect by any means and it is a very basic script but I thought it would be a fun thing to write. I got the idea from a combination of Demitri Martin writing a very long palindrome poem and the rap pad guy with auto-fill lyrics.</comment>


Apple releases iphone-cf - <AUTHOR> http://apple-cf.com

<comment><USER2> yes, it is a prank by the yes men <http://en.wikipedia.org/wiki/The_Yes_Men>

</comment><comment><USER3> It looks like an elaborate prank!

<comment><USER4> Looks likely. Whois information on the domain doesn't match apple.com.

edit: Plus, their search box redirects to Google. Amusing :)

</comment></comment><comment><USER5> More info <http://goo.gl/pxcep></comment>


Fixing JSON - <AUTHOR> https://www.tbray.org/ongoing/When/201x/2016/08/20/Fixing-JSON <comment><USER2> Shameful confession: when I was first introduced to JSON, I was convinced it would go nowhere. "XML already does everything JSON does! And there's no way to differentiate between nodes and attributes! And there are no namespaces! And no schemas! What's the point of JSON?" And a couple of years later I looked up from the tangled mess of XSLT I was working on to discover that the rest of the world had moved on.

JSON is just javascript, and it leaves out everything else. That's its entire reason for existing, and it caught on because that's all that 99.44% of anyone needed.

Timestamps you can add today, without changing the protocol; just put them in your data if you need them. So I'm not sure what he's even proposing there.

Schemas: OK, he doesn't like JSON Schema's anyOf. Fair enough. There's no real proposal here for how to fix it, so not much to say here.

Replacing commas with whitespace sounds to me like replacing a very minor irritant with a constant full-body rash. Stuff like his example of "IDs": [ 116 943 234 38793 ] would lead to _far_ more confusion and errors than the occasional stray trailing comma.

So I guess I pretty much vote no on this one thanks for asking

<comment><USER3> I was thinking along very similar lines when reading the article by Tim Bray. JSON is designed for JavaScript compatibility, I almost stopped reading when he said:

> They're inessential to the grammar, just there for JavaScript > compatibility.

The entire article goes on to suggest extremely terrible solutions to the problems he is pointing out.

In order:

Commas: If comma's are an annoyance, is that to mean there is no annoyance with XML markup? You can forget a closing tag just as easily, probably more easily. Just get a JavaScript or JSON linter. Fixed. If you are really annoyed by commas, move to YAML or another format.

Datetimes: Just store your dates in a standardized timezone and format, how about [ISO 8601]([https://en.wikipedia.org/wiki/ISO_8601](https://en.wikipedia.org/wiki/ISO_8601)). You can also use a [JSON-Schema]([http://json-schema.org/](http://json- schema.org/)) if you really want.

Schemas: There's lots of potentials here. However, again, if you're going to change JSON so much, why not just pick another data format that has better schema support.

<comment><USER4> > JSON is designed for JavaScript compatibility,

Javascript compatibility is the least important part of json, lots of people use json with python or ruby or rust or java or c for things that will never interact with javascript, and anyways you don't want people to parse json with eval or confusing json with things that include string concatenation or function calls adding non javascript syntax helps make that clear.

There is a good reason to not change JSON(backwards compatibility, updating parsers, confusing formats, ect.), since you're not planning on parsing javascript with eval there is no good reason why any changes made in JSON 2.0 have to conform to a subset of javascript.

<comment><USER3> I think if breaking changes are made, there is no point, it would not be called JSON 2.0; it is an entirely new data format. When there are hundreds to thousands of alternatives, trying to "fix JSON" isn't the solution. JSON works because it is simple and easy to follow, it is robust and flexible, and has no restrictions. There is no good way to store dates in XML or many other formats, without the use of Schemas, and the syntax changes are minimally beneficial. It wouldn't save a lot of bandwidth with gzip, and the text data likely doesn't add up to a lot of disk space when compared to the data being stored; so removing commas which make parsing JSON easy and increases its adoption and usage, just so some people can have less linting errors doesn't seem like a large benefit.

</comment></comment></comment></comment><comment><USER5> While I respect tbray we need to fix engineers that try to 'fix JSON', absolutely nothing is broken with JSON.

It is nearly a perfect spec for the role it plays. It has taken off because it is so simple. I am constantly amazed that something so simple keeps being jabbed at with more complexity by engineers.

The job of an engineer is to make things that are complex simple, JSON did just that. Make a new standard for your fixes and watch them spiral out of control like XML. Making a simple format that works is damn hard and JSON is a nearly flawless attempt. It is baked into the ecosystem so let's not break it. It was a response to XML complexity/verbosity for use in javascript first, it is so damn useful that it is used everywhere now, why? _simplicity_.

Everything you need can be done with current JSON. Dates in quotes or ticks (another key for offset if need be), comments in separate files (do you really want © Oracle or Microsoft in all your data files?), strong typing GTFO (use a schema if verbosity is needed -- it supports all needed base types string, numeric, list, object), and it is Javascript Object Notation so it's purpose was to tie to Javascript, don't even attempt to break it from Javascript. If you need more use YAML or XML or a new standard.

If you are trying to change JSON, back away slowly, go add your complexity for minimal or worse benefit elsewhere...

</comment><comment><USER6> In my opinion there are five problems with JSON in practice:

1\. Trailing commas should be allowed. This would fix the entire comma ordeal the author is complaining about without introducing awkward whitespace logic.

2\. Date time values should be a distinct type using the format of JS's Date.prototype.toISOString method. Had this been in the language when JSON was originally defined I'm sure it'd already be in JSON.

3\. There should be a way to represent Infinity (for IEEE 754 compatibility) and possibly NaN.

4\. Comments would be nice because JSON is often used for configuration and documenting JSON snippets can get awkward without them.

5\. There's no real way to represent binary data other than base64 encoded strings. It would be nice if there was at least some hex format (e.g. `"blob": <dead beef cafe babe>` with optional spaces for readability) but I can see why some people might be opposed to the idea of adding this to the language

<comment><USER7> I'm a bit wary of the second and fifth one. You could add an arbitrary amount of datatypes. Also I think the common argument against comments makes much sense: they could get abused easily for parsing directives etc. JSON isn't that nice for configuration files anyway IMO.

I would actually take away a feature of JSON: Unicode escaping. It's not needed anyway and astral plane characters being encoded as surrogate pairs is an added complexity for encoders/decoders. The selection of escape codes for control characters also is rather arbitrary.

<comment><USER8> The argument about abusing comments with parser directives is totally bogus. You can create parser directives with extra field names too! Worse, people create comments anyway using fields like "//" \-- but they're much crappier than real comments, since they have to be a valid string value. We gave up a solution to a real problem in favor of a bogus solution to a theoretical problem.

</comment></comment></comment><comment><USER9> Two more issues:

\- No support for comments.

\- No support for NaN and Infinities (i.e. lack of full mapping to IEEE 754).

Also at least on Chrome, JSON.stringify(-0) gives 0, though JSON.parse("-0") does give -0. Weird..

As far as schemas are concerned, I find it nicer to not use schemas but access the JSON through an interface that checks things on the fly (types, existence), effectively verifying the JSON while you're examining it. E.g. things like x.get_int('name'), which throws exception if it doesn't exist or isn't an integer. Possibly even with an error message explaining where exactly and what is wrong (like: foo.bars[4].bazes["abc"]: attribute "qux" does not exist).

<comment><USER10> The original JSON spec included comments. They were removed because people were using them as processing directives.

</comment></comment><comment><USER11> With the exception of trailing commas, all these things would break the fact that JSON is a subset of JavaScript. Breaking that would cause a lot more problems than it solves. It's just that the minor problems you have to live with seem worse than the major problems you don't.

<comment><USER12> What value is there in having JSON be a subset of JavaScript? In fact, it would be a good idea to make it not be a subset of JS, because that would prevent people from evaling it.

<comment><USER13> It's also (almost) a subset of Python, and possibly other languages. The benefit is that it is trivial to remember, because you already know the syntax. And you can not just eval it (which you if course shouldn't do if you didn't write the JSON yourself), but more important you can just copy it into your code. Or start with a object literal in code, and when it grows, or you want to use real-world data, switch to an external data source.

There are a couple of projects which extend JSON. I believe an ideal JSON superset would be compatible with ECMAScript 6 object literals, and have comments and trailing commas.

<comment><USER14> JSON is pretty close to being a subset of Lua, as well.

<comment><USER15> Well = vs : is a bit different. And requiring quotes around keys.

</comment></comment></comment></comment></comment><comment><USER16> At this point, I'd settle for JSON5 ([http://json5.org/](http://json5.org/)), which says any JSON that's a valid ECMAScript 5 object is valid JSON (including trailing commas, unquoted keys, single quote strings, and comments). I don't think it's very popular at this point though, but I guess I've never tried to use it.

<comment><USER17> Ugh no. Adding all that crap complicates writing a correct parser. JSON's value is in that it is easy to parse.

If editing JSON is your problem, use a better editor. If you really want to use JSON as a config file format, use something like YAML instead.

<comment><USER18> Or TOML, if you find YAML's syntax too messy.

</comment></comment></comment><comment><USER19> JSON is not optimized to be written by humans (I'd argue even JS is not well optimized for that </joke>).

There are other formats for that, like YAML and TOML[1].

When JSON is needed for human-to-computer stuff, I'd say TOML is pretty much always a better choice. A common use case for TOML is config files. And I see a lot of config in JSON lately; this makes me sad. (Though not as sad as the config-in-XML thing the Java ecosystem suffered from.)

1: [https://github.com/toml-lang/toml](https://github.com/toml-lang/toml)

<comment><USER20> TOML is kinda ugly compared to YAML.

<comment><USER21> YAML is a pain to parse. And as a human, having to remember quoting string values like "yes" and "on" to prevent them from being interpreted as booleans in YAML really grinds my gears. I don't care if it's prettier; YAML can't disappear quick enough.

I wrote a haiku some years back to demonstrate how broken the Ruby implementation was.

        YAML:
          takes a no for a 'no': no
          spring,summer,fall,winter: 3,6,9,12
          the game is: on

Running it through ruby 2.3.1 yields..

        {"YAML"=>
          {"takes a no for a 'no'"=>false,
           "spring,summer,fall,winter"=>36912,
           "the game is"=>true}}

<comment><USER20> I agree. That's partly why I wrote this: [https://github.com/crdoconnor/strictyaml](https://github.com/crdoconnor/strictyaml)

e.g.

        >>> load(x, Map({"YAML": MapPattern(Str(), Str())}))

        {'YAML': {'spring,summer,fall,winter': '3,6,9,12',
          "takes a no for a 'no'": 'no',
          'the game is': 'on'}}

        >>> load(x, Map({"YAML": Map({"spring,summer,fall,winter": Str(), "takes a no for a 'no'": Bool(), "the game is": Bool()})}))

        {'YAML': {'spring,summer,fall,winter': '3,6,9,12',
          "takes a no for a 'no'": False,
          'the game is': True}}

</comment></comment></comment></comment><comment><USER22> I'm not sure how dropping comma separators would work in practice, because carriage returns aren't required between array or hash elements in JSON.

So this would also have to be legal:

       { "foo": "bar" "baz": "quux" }

I'm not sure that's better, personally. It's also syntactically invalid in Javascript:

       > "bar" "baz"
       SyntaxError: Unexpected string

<comment><USER23> Let's just remove periods from sentences to They clearly aren't needed since there's a capital letter there

If anything I'd rather trailing commas in lists be defined as valid, and separation there is useful like you say

</comment></comment><comment><USER24> I know this is from Tim Bray, one of the primary authors and/or maintainers of XML, RDF, Atom, and JSON, but while I too find some annoyances with JSON, I accept them because:

\- JSON was never designed to be hand-editable. It's a lucky accident that it's more palatable to edit than XML, but there are other formats like YAML, TOML, and even .ini that fulfill that purpose.

\- While JSON was originally envisioned as a subset of Javascript and intended to be materializable by JS eval(), this very quickly turned out to be a bad idea. Once we all switched to using parsers, there is no more compelling need to remain compatible with Javascript, other than human recognizability of syntax, and, y'know, compatibility with 'old' JSON; and

\- JSON's greatest strength is that it has such a good impedance match to native 'abstract' data structures supported in just about every contemporary programming language: strings, numbers, lists, maps. When people say 'simple', I don't disagree, but I'm convinced they're actually referring to this feature. Adding additional types -- yes, even timestamps -- would break this property, as datetime handling in most languages leaves a lot to be desired.

<comment><USER25> The first language that comes to mind when I hear "datetime handling in most languages leaves a lot to be desired" is, slightly ironic considering JSON's origins, javascript. If you ever touch a date in JS you HAVE to pull in something like moment, full stop. JS Date is a steaming pile of shit.

<comment> nevi-me To go on a tangent, Java8 dates are "awesome" if you've used Joda-Time before, for the rest of us it's a pain in the backside. I found myself spewing lots of code to just do Moment one-liners. That made me appreciate Moment much more, to the extent that I sometimes just eval a Moment script with Nashorn at times.

On the JSON part, I agree with everyone, my go-to language for most things is JavaScript, and I'm okay with dates in JSON. I think it's RFC3336 as everyone says (2016-08-20T21:12:26.231Z).

Date parsing could be better when moving across languages, but I mostly prefer sending around millis since Epoch. Considering that people mainly look at their JSON for debug purposes, anyone seeing 1471727663538 should recognise that it's a date.

Other commenters put it well that the author should rather propose a new language, JSON is fine the way it is. Typed successors like Protobufs don't have native date parsing, but I guess being binary, they take away the commas too. I'm comfortable hand-writing JSON or editing it when needed. Most text editors help out by highlighting errors and keeping track of parentheses.

<comment><USER24> To continue the tangent, perhaps you read my mind, because I believe that Java8 and Joda (and their ports) are the only two libraries that do datetimes properly, and every other one confuses and/or conflates human calendar and clock concepts with absolute point-on-a-timeline timestamps, and the like.

But this illustrates the point: datetime is hard and we don't agree, so even going so far as retrofitting a native RFC 3339 datatype into 'official' JSON is, in my opinion, too far.

<comment> nevi-me I'm fairly new to date manipulation in Java, between LocalDateTime, Instants, TemporalAdjusters etc. I get confused, but I'll catch up.

The comment I replied to mentions Moment for JavaScript, which I would add to your list. It's beyond excellent, great documentation as with most good quality libs, and very intuitive.

I agree with your point and opinion. Someone mentioned on another thread that engineers are there to simplify things, 'official' JSON has done a darn good job at that, no surprises across languages there.

</comment></comment></comment></comment></comment><comment><USER26> Mandatory double-quotes for keys just to get rid of required commas does not make it better.

There are better relaxed JSON's available than this one:

* [https://hjson.org/](https://hjson.org/) does not need those double quotes and can handle without commas just fine.

* [https://github.com/phadej/relaxed-json](https://github.com/phadej/relaxed-json) also allows comments, trailing commas, simplier strings as in YAML.

* RJSON [http://search.cpan.org/~miko/JSON-Relaxed-0.03/lib/JSON/Rela...](http://search.cpan.org/~miko/JSON-Relaxed-0.03/lib/JSON/Relaxed.pm) is similar.

</comment><comment><USER27> The proposal doesn't fix JSON it introduces an entire new format. One which I would not be able to use without a 3rd party library that parses it. If I'm already using JavaScript that is one of the many benefits it has in the first place.

Timestamps are easily included when I want them, I tend to prefer UNIX time as it's just so much more impact proof. With this proposal can I use UNIX time if I want?

I don't have any problem with JSON. If I did I would have a problem with JavaScript itself, and while there's plenty to complain about I wouldn't propose mixing more formats into it as a solution. There isn't enough care being taken not to introduce additional complexity into the JavaScript world.

</comment><comment><USER28> Dave Winer's remark from 2006 offers some historical perspective:

"I've been hearing, off in the distance, about something called JSON, that proposes to solve a problem that was neatly solved by XML-RPC in 1998, the encoding of arrays and structs in a format that could easily be processed by all programming languages. The advantage being that you could easily support the protocol in any language that supported XML and HTTP, which, at the time, was quickly becoming all languages."

[http://scripting.com/2006/12/20.html](http://scripting.com/2006/12/20.html)

Then there was someone, much younger than Dave Winer, who wrote a response to this (during 2006), and who said (paraphrase) "When I'm older, I look forward to my XML moment; my horror at something that the young kids are doing."

I wish I could find that second quote, but I worry it is lost in the mists of time.

</comment><comment><USER29> If you're going to drop the commas in the dictionary definitions, why not drop the colons too?

        { "key" "value" 42 37 }

It's a rhetorical question; I think removing the commas is a bad idea. Troubles while hand editing json shouldn't be an overriding use case that drives the syntax.

All of those complaints about json and nothing about how you can't have comments?? I feel like Tim Bray and I aren't even on the same planet.

</comment><comment><USER30> One way to get a comma-free JSON-like format is to write the document in hjson, which is a superset of JSON.

[http://hjson.org/](http://hjson.org/)

var config = HJSON.parse(fs.fileReadSync('config.hjson'))

Another more obscure, and more powerful serialization format that does away with commas (commas are treated as whitespace!), has a date type, and much more is Rich Hickey's (of Clojure fame) EDN. [https://github.com/edn- format/edn](https://github.com/edn-format/edn)

{:a 1, "foo" :bar, [1 2 3] four}

</comment><comment><USER31> _Major irritant: Commas_

I thought the major irritant was having to add double-quotes around keys? Commas never bothered me personally, but having to use double quotes is actually irritating.

<comment><USER32> It comes from the flow of edition: remove the last element of a list (which typically is on its own line), forget about the fact that there is a comma at the end of the previous line.

But both optional commas and optional double-quotes are provided by Hjson <[https://hjson.org/>](https://hjson.org/>). They do not advocate its use in protocols, however.

</comment></comment><comment><USER33> JSON was fixed in 2014:

[http://blog.cognitect.com/blog/2014/7/22/transit](http://blog.cognitect.com/blog/2014/7/22/transit)

[https://github.com/cognitect/transit- format](https://github.com/cognitect/transit-format)

<comment><USER34> Sounds nice, is anyone using it? Is it somewhat readable by humans?

</comment></comment><comment><USER35> Two quick observations:

\- Getting standards passed is hard (especially for popularly used standards), but can be supported by latent usage in the community. You can create a plugin for a few languages that automatically allows the timestamps and doesn't require commas. Get a bunch of people using it, and you'll be on your way to actually getting it in the standard in the future (still debatable in this specific example, but broadly, usage never hurts for getting something into a standard)

\- If your concern is your own frustrations (rather than benefits of the vast majority of people) - I highly recommend a simple editor-level mapping that translates from your preferred DSL to the underlying standard

</comment><comment><USER36> I agree with those who say JSON should never change, but should one day be replaced. It behooves us to think about what that replacement should be.

My own peeves:

* Numbers are stored in decimal but usually interpreted as binary with no guidance about rounding.

* The concatenation of valid JSON blobs is never valid JSON.

* The empty string is not valid JSON.

The last two can be "fixed" by a format called "arraySON" where the global blob represents not a JSON object (dictionary), but the _contents_ of an array plus an optional trailing comma.

Or given some of the ideas on this thread, perhaps a manadatory _leading_ comma.

<comment><USER37> > * The concatenation of valid JSON blobs is never valid JSON.

You might want to check out JSON Lines:

[http://jsonlines.org/](http://jsonlines.org/)

<comment><USER36> Is there are formal defintion of the format anywhere?

</comment></comment></comment><comment><USER38> If you did all of that you'd still fall way short of what you have been able to do with XML for probably 10 years. XSD defines structure in terms of inheritance, composition, strong types, numerical ranges, enumerations, referential integrity, namespaces.

<comment><USER39> Raise your hand if you miss hand-editing or reading XML.

<comment><USER40> Requiring that a markup language (that must be sent across the wire between multiple buggy parser implementations) be hand editable is a recipe for disaster, and will never be as flexible as building/modifying an appropriate AST in a scripting language.

There's not even a good reason to read HTML anymore. You hit Ctrl+Shift+C in Firefox (or I think it's J in Chrome) and you view the actual machine-parsed DOM structure. The only reason human-editable/readable formats were ever necessary is a lack of appropriate dev tooling.

<comment><USER41> I have to agree, I've been working with JSON for 10 years and nearly never touched JSON by hand.

</comment></comment></comment></comment><comment><USER42> Personally I would rather it require trailing commas like Go does, any optional syntax is a step backwards IMHO

<comment><USER9> The best way is to require one comma between elements but allow one trailing comma after the last element. This way you can both do multi-line and one-line lists nicely.

</comment></comment><comment><USER43> I never understood why the double quotes around property names are mandatory. For example, in JavaScript, I can write

        { foo: 1, bar: 2 }

but JSON syntax insists on

        { "foo": 1, "bar": 2 }

This makes JSON less easily readable by humans, and harder to write or edit by hand.

Anyone know why JSON was designed this way?

<comment><USER44> Because using certain keys could break your JavaScript (and JSON was designed to be a subset), like

        { delete: "me" }

If you don't want to keep a list of known keywords around, just insisting on quotes is the easier way.

[EDIT: See also this YouTube video, where Douglas Crockford explains it himself: [https://youtu.be/-C-JoyNuQJs?t=5m](https://youtu.be/-C-JoyNuQJs?t=5m)]

<comment><USER45> Unquoted keywords are valid ES5.1 object properties.

        alert(JSON.stringify({delete: "me", for: 1, var: 2}));

results in:

        {"delete":"me","for":1,"var":2}

The only browser it does not work in is IE8 and earlier.

<comment><USER46> JSON significantly predates ES 5.1

</comment></comment></comment></comment><comment><USER47> I don't know who the OP is but I'll assume he's an authority in JSON since he talks about how he wrote some RFCs and stuff.

Anyway this is why I think there should be a better way to move tech standards forward than leaving it up to some small number of people who are way too ahead of the curve that they think everyone else has the same problem they do.

In my opinion there is nothing wrong with commas. I am sure the OP had his own issues while working too much with JSON, but for the 90% of the population it's not a problem and even a better way to represent data.

As an example, [116, 943, 234, 38793] is 100% clear and intuitive. Anyone who has seen any array representation will get what this means. But change it to [116 943 234 38793]? There are so many ways you can interpret this. I guess the only people who would think this is better than the 100% intuitive [116, 943, 234, 38793] option are JSON nerds who think too much about this (and when I say JSON nerds I don't mean JSON "users", but people who try to come up with the standard)

As someone who really hates what's going on with javascript (with all the new syntax that needlessly sacrifices simplicity and intuitive nature for the sake of making things shorter), I can't really think nicely about these opinions.

I think these standards that are used by a huge population should move forward in a more democratic way. Or at least let more "normal" people make the decision.

<comment><USER48> The author, Tim Bray, is one of the co-authors of the XML specification; [https://en.wikipedia.org/wiki/Tim_Bray](https://en.wikipedia.org/wiki/Tim_Bray)

</comment></comment><comment><USER49> Anecdotally, I've really enjoyed using ION[0] at Amazon. It has all of the features the OP wants, besides semantic whitespace which I'm not a fan of.

[0] [http://amznlabs.github.io/ion- docs/index.html](http://amznlabs.github.io/ion-docs/index.html)

</comment><comment><USER50> Commas irritate me too. But I think removing them from The Serialization Format Known as JSON is a mistake.

My workaround is to use CSON, since I'm already using CoffeeScript. It add the syntactic sugar where I'm hand-editing JSON, yet it doesn't impose my tastes on the rest of the world.

</comment><comment><USER51> Dude really? Commas are not a big deal and you're talking about a major change to get rid of them; making white space matter. Currently white space doesn't matter. In order to get rid of commas it has to. That should not be glossed over or treated lightly.

</comment><comment><USER52> I don't agree with the "timestamps" part, for several reasons:

1) the author does not mention in his example the use of timezone offsets (which are a part of RFC3339), that's a bit irritating

2) Timestamp parsing and emitting sucks, no matter the language, because the format specifiers are wildly inconsistent across platforms. I prefer the good old UNIX timestamp, because with it there's no complex massaging needed in order to operate with the timestamp.

Also, the author is missing comments entirely - no matter if shorthand // or long-form /* */ gets included, JSON desperately needs commenting ability; but on the other hand I'm worried about compatibility with older systems...

<comment><USER6> 1) The author does. Z means UTC.

2) The JS Date type has a "toISOString" function that emits an ISO date time string like the ones used in the examples. Since everything else is already based on what JS does, why not just move that behaviour into the JSON spec as well?

</comment></comment><comment><USER53> Just use yaml?

<comment><USER12> Or if you still want something easy to parse, s-expressions.

<comment><USER54> Could you please elaborate? I'm vaguely familiar with S-expressions from my cursory inquiry into lisp (planning to take the plunge in November), but I don't see how they can be applied in this context.

<comment> nv-vn S-expresions are really just a way of representing a tree of data (in Lisp code, the S-expression literally represents the abstract syntax tree that is used for the code).

Take the JSON example:

        {
          "IDs": [116, 943, 234, 38793],
          "Settings": { "Aperture": 2.8, "Shutter speed": "1/250" }
        }

You could rewrite this as an S-expression as:

        ((IDs (116 943 234 3879))
         (Settings
           (Aperture 2.8)
           ("Shutter Speed" "1/250")))

<comment><USER54> Thanks! I figured it would be simple, and indeed it was :)

</comment></comment></comment></comment></comment><comment><USER55> Commas aren't a huge deal for me personally but if there were to be a "json2", dealing with them and removing the double-quoting requirement for keys would make authoring a little easier. Ultimately these are issues that are easily solved with a linter though.

A standard date type would be huge however, and I'll add to that built-in support for comments. These two things would expand the scope of what you could do with JSON. JSON is already used in so many places for config files but without comments it can be a poor choice.

</comment><comment><USER56> JSON is fine for what it was intended to be: a way to serialize fundamental data structures to disk or over the network.

If you want something optimized for human editing, such as for a configuration file or a DSL, we already have YAML for that. Hell, JSON is a valid subset of YAML, so you don't even need two parsers.

<comment><USER57> YAML including the entirety of JSON is not a positive argument in favor of YAML.

</comment></comment><comment><USER58> I think JSON is great, because it's simple. Who really needs something more oriented for humans, take a look at HOCON.

[https://github.com/typesafehub/config](https://github.com/typesafehub/config)

</comment><comment><USER59> I'd argue that getting rid of commas would lead to an error-free experience. Say I use UUIDs:

        { "884fbf28-b285-49c2-8335-e42aa3fc896d" "f08bc223-2b49-4d49-a308-7bb93691cef8" }

vs

        { "884fbf28-b285-49c2-8335-e42aa3fc896d", "f08bc223-2b49-4d49-a308-7bb93691cef8" }

Another example is long ids:

        { 214748364721474 89234789 7248374293843 23874827 }

vs

        { 214748364721474, 89234789, 7248374293843, 23874827 }

Should we take into account text format and things like font-spacing? I say yes, because user experience is what's valuable here.

Edit: formatting

</comment><comment><USER60> Json is flawed in any way possible, imo. It was intended to be edited and read by human; it can not comments, trailing commas, non-quoted keys. It was planned for universal exchange; it doesn't contain dates, full-floats, byte arrays in readable form. It can't have backreferences for graph exchange or per-packet string interning.

Even "obvious syntax" argument is wrong. "Key": value is not what comes to mind unless your parents spoke to you on js.

Why it is still here: legacy that was easier to read-write than sgml-flavoured monster. Other arguments are nonsense.

</comment><comment><USER61> _Hand-editing JSON may not be the most important way of interacting with it..._

In that case, the arguments in favor of breaking long-established compatibility for the sake of hand-editing should be particularly strong.

If personal forgetfulness about editing around commas or date strings are strong arguments, then they should surely also apply to every programming language that uses commas or lacks the concept of a date literal. After all, the code of programming languages exists to be edited, and JSON is merely a means of serialization.

</comment><comment><USER62> Regarding timestamps, the at-sign is already used by GNU tools, as a prefix to indicate actual UNIX timestamps (seconds since epoch): [https://www.gnu.org/software/coreutils/manual/html_node/Seco...](https://www.gnu.org/software/coreutils/manual/html_node/Seconds- since-the-Epoch.html#Seconds-since-the-Epoch)

</comment><comment><USER63> I would like timestamps introduced as a first class citizen, and can be done relatively cheaply. As the OP said, they are very common in protocols, and it would be nice to have 1 way of encoding them. Especially 1 that doesn't involve an arbitrary offset...

The other suggestions are interesting, but rather distracting IMHO.

</comment><comment><USER64> JSON is a beast to deal with for anyone who's manually editing it. All the comments here about special formatting strategies to avoid comma problems, or how to workaround the lack of a timestamp type just prove the point that JSON needs to fix these things.

</comment><comment><USER65> We are using [https://hjson.org/](https://hjson.org/) to describe metadata of xml/json API and how to parse the result. It solves all the problems mentioned in the post.

</comment><comment><USER66> Too much effort for too small a payoff.

Would trailing commas and timestamps be nice. Yes. Would they be so nice that dealing with the json to json 1.1 transition would be worth it. Not really.

On removal of commas I'd just disagree.

</comment><comment><USER67> JSON isn't broken. There isn't anything to fix. :/

</comment><comment><USER68> I miss AS3.

</comment><comment><USER69> Or you know, use something else like YAML.

</comment><comment><USER70> Removing commas makes minification less useful.

</comment><comment><USER71> Just turn it into a Python dict already. ;)

</comment><comment><USER72> Timestamps should include timezone offset.

<comment><USER73> His example uses RFC3339 which does include timestamps.</comment></comment>


Cassandra 0.8.0 has been released - <AUTHOR> http://cassandra.apache.org/download/

<comment><USER2> They fixed the driver issue by adding "CQL". The new interface should be simpler for people to pick up and understand since it is similar to SQL. Some slides on CQL: [http://www.slideshare.net/shotaz/cql-cassandra-query- languag...](http://www.slideshare.net/shotaz/cql-cassandra-query-language)

</comment><comment><USER3> Would be nice if their release notes were somewhere easily found. Nothing on the download page. Google searching doesn't even find it easily.

<comment><AUTHOR> The email announcing the release has some high level information:

[http://mail-archives.apache.org/mod_mbox/cassandra-user/2011...](http://mail- archives.apache.org/mod_mbox/cassandra- email@example.com%3E)

The complete list of changes:

[https://svn.apache.org/repos/asf/cassandra/tags/cassandra-0....](https://svn.apache.org/repos/asf/cassandra/tags/cassandra-0.8.0/CHANGES.txt)

</comment></comment><comment><USER4> Thumbs up for the counter feature:

"Cassandra also has distributed counters now. With counters, you can count stuff, and counting stuff rocks."</comment>