CockroachDB Skitters into Beta - <AUTHOR> https://www.cockroachlabs.com/blog/cockroachdb-skitters-beta/ <comment><USER2> I don't really get all the whining about the name, I even kind of like it. It fits well with their narrative.

Besides, they could've called it PoopDB and I still would use it if it survived a nuclear blast and had joins. I couldn't care less about the name. But apparently I'm in a minority...?

<comment><USER3> I think it's more about the senior management and people like marketing, who aren't technical. That could go well: "Yeah so we're ditching MS SQL Server Pro and instead will go with Cockroach from now on"

<comment><USER4> Can you just call it something else internally? It's not like any of those people know what databases do and do not exist. Maybe CR, CRDB or like someone here suggested, RoachDB.

</comment></comment></comment><comment><USER5> Been watching this DB for a while. Glad to see it enter beta. I recommend including [https://github.com/mauricio/postgresql- async](https://github.com/mauricio/postgresql-async) on [https://www.cockroachlabs.com/docs/install-client- drivers.ht...](https://www.cockroachlabs.com/docs/install-client-drivers.html) (it's my favorite Postgres client, it's very fast).

<comment><USER6> Hey kodablah, just opened an issue for CockroachDB devs to investigate this driver and possibly add it to that page. Thanks!

</comment></comment><comment><USER7> I've been watching this project for a while. This is very cool. I think they've made a lot of very good design choices.†

Oh, and the client drivers page should probably also list pgx ([https://github.com/jackc/pgx](https://github.com/jackc/pgx)), which is a nice, fast PostgreSQL client that doesn't use the not-as-nice database/sql package.

___

† The name being the only exception; I don't have bug phobia at all, but cockroaches are just icky.

<comment><USER6> Opened an issue for CockroachDB devs to investigate this driver and possibly add it to that page. Thanks!

</comment></comment><comment><USER8> Looks neat!

I noticed this: CockroachDB is not suitable for use cases involving joins (yet)

Well guys, that's the kicker. If you've got joins, you've got SQL. The selection language isn't so important if you can't do set relations.

I'm sure you're going to rock this. Can't wait to see how you do it!

<comment><USER9> Agreed - a lack of joins means it's not real SQL.

</comment></comment><comment><USER10> This is one of the most exciting DB's coming into the world, and I've had a lot of fun watching progress from the sidelines! Keep kicking ass!

<comment><USER11> > This is one of the most exciting DB's coming into the world, and I've had a > lot of fun watching progress from the sidelines!

I'm going to have to disagree. The German HyPer DBMS has a lot more interesting ideas. Cockroach is just distributed MVCC from the 1980s on top of a key/value store.

MemSQL also has a new LLVM architecture written by the HipHop VM inventor that they poached from Facebook.

<comment><USER12> HyPer and MemSQL are in-memory databases. CockroachDB is not. This is a big difference.

And according to HyPer home page, HyPer doesn't look distributed, which is another big difference.

<comment><USER9> MemSQL used to be just in-memory. It added a second storage engine designed to hold data on disk using a column store.

</comment></comment></comment></comment><comment><USER13> Great news! I still don't understand why HyperDex (which has similarities in design and goals) has existed for years and nobody seems to know about it. Is this because it doesn't support SQL?

[http://hyperdex.org/](http://hyperdex.org/)

<comment><USER7> HyperDex is interesting. It has a lot of pluses — fast, written in C, easy to run, seems robust.

On the other hand, the author (or authors) seem completely unresponsive about some things. Their Homebrew package's Ruby support is still broken a year after reporting it [1]. The Homebrew package is also ancient (1.4.4 instead of 1.8.1); I did a PR [2], which requires that the authors to roll a release of their fixes to HEAD of libpo6 [3]. All of this has been ignored for some time.

I'm just worried that the project, which seems to be maintained by just one developer, is languishing. Very little commit activity, lots of open issues, lack of attention, etc.

[1] [https://github.com/HyperDex/homebrew- hyperdex/issues/10](https://github.com/HyperDex/homebrew-hyperdex/issues/10)

[2] [https://github.com/HyperDex/homebrew- hyperdex/pull/15](https://github.com/HyperDex/homebrew-hyperdex/pull/15)

[3] [https://github.com/rescrv/po6/issues/9](https://github.com/rescrv/po6/issues/9)

</comment></comment><comment><USER14> I still don't get it why people get so excited with every new even-more- scalable DB yet at the same time completely ignore what's happening with things like _graph DBs_ or _hybrid-document-graph DBs_ that seem so f cool!

Also... why would you want to live WITHOUT ANY JOINS nowadays?! Unless the scale requirements actually prohibit it, and let's face it, 99% of what people are doing is not even close to "big data" on modern hardware, and no, your 1TB DB is not big data and you can handle it with Postgres or a powerful graph db just fine on a pretty cheap VDS if you stay way from AWS/Azure etc. that get nowhere near the performance/price of renting "dedicated metal" on mid-term deals. I'm getting closer tho thinking along the lines of: _not even plain JOINs are enough, I want at least Postgress-like recursive joins if not full- fledged infinite-depth graph traversals! I know the budget /hardware allows is, so give it to me, stop the whole "scalability" bullshit!_

<comment><USER15> For me it's because there are very few databases (you can count them on one hand) that check the following boxes:

* Document or SQL Storage * Integrated Full Text Search * Can transparently survive a node going down (load balancer or client driver support) * Highly consistent by default, or allow operation level quorum settings * DR options with minimal lost writes * Transaction support * Hosted * Affordable for small businesses with low average load, but including the occassional peaky or seasonal issues (large bulk imports, auto/consumer/product/etc-show traffic spikes for a few days, etc)

Cloudant meets everything but Transactions. But that's a big one.

Most databases you might think of don't satisfy the Hosted option. And considering how easy it is to loose other items on the list through misconfiguration or maintenance issues, that's a lot more of a core feature than might be obvious for a small shop. It's great if node failure is transparent in theory, but if it doesn't work in practice it may as well not exist.

Most of the rest don't integrate full-text-search. That's huge. It means a ton of concerns move from the database vendor, to you, the application developer. Any database that doesn't (at least plan to) integrate full-text-search is one with a much higher development and operations cost.

And the number of databases that promise to (eventually) deliver distributed SQL? There was 1 a few years ago (AFAIK) that met most of those: FoundationDB. Never to be heard from again. :-)

I don't need a GraphDB for my CMS-like sites. That's just not a very good fit. And working without JOINs (though it sounds like they're on the roadmap here) is not as onerous as you'd imagine for many sites, and it often brings along the side-effect of forcing you to write a much more efficient/fast product against denormalized data.

So any database that promises to check most of those boxes is worth keeping an eye on (IMO). Though the things that kill most solutions candidacy for me are:

      - lack of integrated Search
      - lack of Hosted/Managed option

Everything else is pretty flexible.

<comment><USER7> We're building something that satisfies all of those criteria.

We call it a data layer, not a database, because it's a higher-level document data model layered on top of Postgres (with pluggable data stores that can be added at runtime; we hope to support other backends such as Redis and Cassandra and maybe CockroachDB if it's a good fit) and Elasticsearch (also intended to be pluggable), similar to how TitanDB is implemented.

It supports transactions — atomic multi-document updates within a single data store that supports this — fine-grained CRDT-style document patching, update streaming, versioned schemas and a fine-grained permissions system. The query language isn't SQL, but our own relationship-oriented, vaguely GraphQL-like query language that allows querying on anything that Elasticsearch can express, as well as complex joins and aggregations. Since Elasticsearch is eventually consistent and all queries go through it, we aim to provide some consistency-tolerance semantics when a client does need strict consistency in combination with queries.

We're not quite ready to publish the code, but drop me line (email in profile) if you want to be notified when it becomes available.

</comment></comment></comment><comment><USER16> Honestly as a new yorker the name of your company and product and the title of this blog post make me cringe!

<comment><USER17> The "mouse" (computer hardware) has an equally bad name, and yet...

<comment><USER18> You think mice and cockroaches evoke the same emotions in the average human?

<comment><USER17> No, but many humans will jump up on a chair, or run from the room if a mouse runs across the floor.

Calling a product a "mouse" must have seemed like a strange decision at the time. Now we don't think twice about it. My point is that people get accustomed to things over time.

</comment></comment></comment></comment><comment><USER19> Like both idea and traction, but I'm taking bets how long it takes for the investors to "convince" the team of a rebranding. I mean imagine anyone suggesting "CocktoachDB" in an enterprise setting... Guess we'll see the rIse of the all new and shiny HydraDB soon, mythological creatures always go well :)

<comment><USER20> I'm an investor in CDB. I love the name and there's always an acronym for those with Acarophobia.

</comment></comment><comment><USER21> Any chance we'll see something like functional indices or Pg-like UPSERT?

<comment><USER9> Check out MemSQL Community Edition which has these features

<comment><USER21> Thanks Eric, MemSQL is pretty awesome but I can't really begin to consider it when:

1\. Backing up your data is somehow an Enterprise feature?

2\. No pricing anywhere on the site, which usually means:

2.1. Sales calls. Nothing I hate more in this world than sales calls.

2.2. $$$$$$$ (which is fine, but I like to know roughly how much $$$$$ before 2.1.)

As far as functional indices, I don't see that here:

[http://docs.memsql.com/docs/create-index](http://docs.memsql.com/docs/create- index)

What I mean is specifying a WHERE clause on the index.

<comment><USER22> If you are referring to indices on expressions, MemSQL definitely supports them. In MemSQL you do it by creating a computed column ([http://docs.memsql.com/v5.0/docs/persistent-computed- columns](http://docs.memsql.com/v5.0/docs/persistent-computed-columns)) and creating an index over one of those. Computed columns turn out to give you a significant perf boost since you only have to run the expression once (at INSERT time), and never while executing a SELECT query or seeking in the index.

</comment></comment></comment></comment><comment><USER23> I am just beginning to realize how many delicious puns are hiding in the name. "There is a bug in your database" \- "No, the database _is_ the bug." ;-)

Seriously, though, this looks like a very interesting project. Until now, I had not been aware it supports SQL. I think I have to give CockroachDB a try in my next toy project.

It sounds like they try do build something very ambitious. To my knowledge none of the commercial database vendors (except maybe Tandem back in the day?) have succeeded in building a distributed/replicated SQL-based database engine with strong consistency and high availability. I know MSSQL supports hot- standby/failover, but AFAIK that does not cover all the scenarios CockroachDB seems to want to address.

</comment><comment><USER24> What are the current performance characteristics in terms of number of select and insert per second per node ? Does anyone have synthetic benchmarks published ?

<comment><AUTHOR> CockroachDB here. We haven't published benchmarks yet and performance work is ongoing. As this is a beta product, the current performance is not indicative of where it will eventually be.

</comment></comment><comment><USER25> Is this an implementation of Google's Spanner? I looked the projects github page but didn't see much in the way of project's origins.

<comment><USER26> The designed is compared against Spanner's several time in the interesting design doc: [https://github.com/cockroachdb/cockroach/blob/master/docs/de...](https://github.com/cockroachdb/cockroach/blob/master/docs/design.md)

</comment></comment><comment><USER27> Aside from everything else, it's nice to see what appears to be sane support for using TLS right there in the command line, and in the beta:

[https://www.cockroachlabs.com/docs/secure-a- cluster.html](https://www.cockroachlabs.com/docs/secure-a-cluster.html)

(Much thanks to Go's extensive standard library: [https://github.com/cockroachdb/cockroach/blob/master/securit...](https://github.com/cockroachdb/cockroach/blob/master/security/certs.go) )

I've lost count on the number of projects that leaves "securing the server" as an exercise to the reader (about as useful as those error prompts that helpfully ask you to "contact the server administrator" (I AM THE ADMINISTRATOR!)).

I understand why not using TLS doesn't give an error, although I'd prefer it if it was more work to set up an insecure instance (eg: --force-no-tls --force-no-auth) -- but what cockroach does here is pretty good, and AFAIK best-in-class from those which it is natural to compare to (in all fairness, projects like postgresql a) doesn't do too bad, and b) have a lot of legacy cruft -- the old (current) assumption is of course that if you want TLS, you'll use your secure in-house CA for everything. Which even when running Microsoft AD is in my experience way too complex for most to bother with).

Another (bad) example here is openssh, that has had decent support for certificates rather than keys for a long while now, and yet I've yet to see anyone that appear to use ssh certs in anger (Myself included, it's high up on the infinite todo-list).

One other great example of making the best of what awful tools are available for securing communications is the Caddy web server, that comes with built-in support for letsencrypt: [https://caddyserver.com/](https://caddyserver.com/)

That said, while I think the docs are pretty good here, a note that leaving an unecrypted, ca-key lying around is a bad idea, might be worth a mention even in the quick-setup. At least as it stands, it's reasonably easy for someone with a working knowledge of TLS/certificates to guess which parts should be secret (all the keys) and which parts should be kept air-gapped (the ca-key).

Maybe a link to how cockroach parses/verifies certs would be nice too, for those that _do have_ a working internal CA -- along with a little info on how/if certs integrate with authenticating nodes (eg: will my printer.example.com x509-cert allow my hacked printer to join the cluster, if it is signed by the same CA?).

Despite all the comments, I'm really liking what I see so far -- I'll be keeping an eye on this project!

<comment><USER27> On a tangential note, I used to think the openssh developers was a little crazy for not using a (a subset of) x509 certs, but now, seeing how ssh is a part of go, I'm not sure if using ssh certs[1] for things like intra-server auth/authz might not be a good idea.

TLS is ok if you need to secure something that speaks HTTP (or IMAP etc) -- but I wonder if ssh certs might not be the lesser evil for something like what cockroachdb needs. Perhaps esepcially now as self-signed/"home grown" x509 certs are being relegated to the background by modern browsers -- so you'd end up needing a "real" cert for the status-server to present securely to a web- client, which might not match the wish for every node to have its own keypair.

[1] [https://godoc.org/golang.org/x/crypto/ssh#Certificate](https://godoc.org/golang.org/x/crypto/ssh#Certificate)

</comment></comment><comment><USER28> But why the name?

<comment><USER29> Cockroaches have the ability to survive some of the most difficult situations... they've been around for the past 320 million years.

That's what they expect from their db... the ability to survive huge failures whilst maintaining data integrity.

</comment></comment><comment><USER30> Nice seeing a new SQL database but who comes with with names like this?

<comment><USER31> Ahm it is a good choice, since the name implies its survivability. They say "cockroaches can survive a nuclear explosion".

<comment><USER32> Why not just RoachDB? Same meaning but rolls off the tongue a lot easier.

<comment><USER33> Plus it has the happy alternative connotation of something you might actually be _happy_ to find when tidying up the kitchen the morning after a party...

</comment></comment></comment></comment><comment><USER34> I have serious problems with the name of this, even opening this post is giving me real trouble. I can't believe someone thought this was a good idea for a name. No matter what the tech is, or how good it is, I will NEVER be able to use this.

<comment><USER35> This is a serious question even if it seems like it isn't; how do you handle the real world with such a strong adverse reaction to a mere word?

</comment></comment><comment><USER36> Full disclosure: I work for a non-competing (EC vs SC) database - which also has an ugly name "gunDB".

Super excited to hear about CockroachDB's progress - and their decision to add SQL support I think was smart.

However, I can't go without critiquing their marketing of "Scalable" and "Strongly Consistent", especially when they reference horizontal scaling. Strongly Consistent systems bottleneck around a centralized master, or around complex consensus algorithms like PAXOS or Raft. CockroachDB has previously written this off with "we're using Google's Spanner algorithm" but this makes them vulnerable to a Split-Brain failure (which contradicts their other goal of "Survivability"), for anybody interested in this line of research I recommend googling Kyle Kingsbury "Aphyr" "Call Me Maybe" Jepsen Tests. With that said, you can't just toss horizontally "Scalable" and "Strongly Consistent" into the same blog because they are fundamentally at odds with each other - the physics just don't work. To achieve this you only have two options... either (A) vertically scale the Master, or (B) make incredibly technical sharding decisions (where no two pieces of data that depend upon each other can be in separate shards). The first (A) is not the definition of being scalable, because a single machine can only scale so much (limits of the physical system). And (B) cannot be generalized, which would require every company building on top of CockroachDB to either have the expert knowledge of predicting the exact sharding schema in advance (maybe why they are moving to SQL?), or have to depend upon expensive consulting options (perhaps provided by CockroachDB, after a startup has already built itself around using the free open source product?). The third option is that they are over-generalizing their database trade-offs, and that you cannot actually get horizontal "Scalability" and "Strong Consistency".

That said, I know the CockroachDB guys are smart, so I'd love to hear the more nuanced/detailed explanation. I'm just trying to keep your guys' marketing and promises in check with the reality.

<comment><USER37> CockroachDB's replication protocol is based on Raft, which is not susceptible to split-brain. (Neither is Spanner, but Spanner requires specialized hardware for clock synchronization.) And contrary to your assertion, it allows consistent snapshots and transactions that cross shard boundaries.

The rest of your comment ignores a couple decades or so of _very_ extensive research into scalable distributed transactions.

Design documentation is here if you care to read it: [https://github.com/cockroachdb/cockroach/blob/master/docs/de...](https://github.com/cockroachdb/cockroach/blob/master/docs/design.md)

<comment><USER36> Raft isn't, but that is extremely different from whether an implementation of Raft is or isn't. Kyle Kingsbury's work shows this to be the case quite often - see [http://jepsen.io/](http://jepsen.io/) .

I have read the design document, and know how Spanner works. This isn't the first time I've asked about this, I had the 2nd highest upvoted comment on the initial CockroachDB announcement ( [https://news.ycombinator.com/item?id=9660339](https://news.ycombinator.com/item?id=9660339) ) which means it is many many more people than just me that are concerned about these claims.

Unless you can show me exactly what research contradicts my critique, your response has no merit and doesn't address my concerns. You can't handwave magic appeals to authorities without citing them. So my questions still stand.

<comment><USER38> If an implementation of raft is susceptible to split brain then it is not raft, it's something else. Please read the literature rather than dismissing it, no one owes you any further explanations and no one is handwaving or appealing to authority.</comment></comment></comment></comment>


How I Moved Away from Mac After Leaving Apple - <AUTHOR> http://readwrite.com/2014/03/20/apple-mac-windows-pc-balance#awesm=~oAm6T7jrBgv94D

<comment><USER2> That article could be interesting in a number of different ways like -What software did he choose when switching from osx to windows. (I use both OSs, and there is always some cryptic obscure tool that I didn't know about).

Instead, it reads like a bunch of diary entries from a guy who worked for Apple 10 years ago (this is why he claims to know Apple "inside out"), about some of his past jobs and what OS was used.

Sorry for the negativity, maybe someone else can extract something useful from the article.</comment>


Images of Los Angeles when it was covered in wetlands - <AUTHOR> http://www.businessinsider.com/history-of-los-angeles-ballona-creek-wetlands-watershed-2018-2 <comment><USER2> Looking at that original wetland map makes me better understand the liquefaction zones south of Santa Monica. I'm very happy that my apartment is on the original surface bedrock, but it's amazing how much has been built on relatively baseless infill. That's going to be huge trouble during the next big earthquake.

[http://geohub.lacity.org/datasets/4842ad85584c43048124685228...](http://geohub.lacity.org/datasets/4842ad85584c430481246852280257c2_9?geometry=-119.23%2C33.789%2C-117.364%2C34.131)

<comment><USER3> For anyone from LA visiting Christchurch in New Zealand, go to the some of the empty areas of the eastern suburbs of Christchurch: large areas where homes used to be - all removed due to liquifacation.

</comment></comment><comment><USER4> Water in the American West is one of those topics that sounds boring on the surface, but gets more interesting the further down you go.

A good introduction is the "Cadillac Desert" documentary:

[https://www.youtube.com/playlist?list=PL0444F9186975498D](https://www.youtube.com/playlist?list=PL0444F9186975498D)

<comment><USER5> Great documentary! Too bad its quality is so poor!

</comment></comment><comment><USER6> One thing to keep in mind that a natural wetland area in the LA basin in 1800's is going to be full of mosquitoes that carry malaria. People were highly motivated to eliminate all standing water that mosquito larva could breed in. Thus, fill in the marshes/wet meadows to create dry land and/or dredge to make lakes/ponds. Most current First World inhabitants greatly underestimate the scourge of malaria, especially before modern medicine allowed for a cure. The herd immunity like situation of modern California to endemic malaria is due to a wide assortment of factors and is maintained with relatively small cost today, but I imagine malaria could tip back to being widespread pretty quickly.

</comment><comment><USER7> The history of US fascinates me as a foreigner. How it went from a totally unknown land to a super power within few centuries.

<comment><USER8> In places like LA it only took a few decades from the late 1800s on. Almost like China now.

</comment></comment><comment><USER9> I lived in Playa del Rey for years just a few blocks from the house on the hill in the 4th pic.

A few of the local coffee shops and restaurants have some of these old photos and I always found it amazing that you could still see the resemblance despite all the development.

The building being built along the road in the 4th pic is a karate school now. The lagoon is a few ugly condos, a baseball field and mostly mud now.

<comment><USER10> As far as I can tell, that building is next to the karate school. My office last year was the two upper left windows. Crazy seeing how different it was!

<comment><USER9> Do you know what all of that piping and industrial looking stuff is east of Playa on Culver?

<comment><USER10> There are methane storage wells as wells natural methane pockets under playa vista and that area or playa del rey. Those pipes are part of the wells.

</comment></comment></comment></comment><comment><USER11> Almost like we're a virus that invades something beautiful and balanced, replicates at breakneck speed with no limiter— except when it kills / ruins its host entirely....

<comment><USER12> Was it ever beautiful and balanced if it gave birth to such a virus? We didn't come to be outside of nature.

</comment></comment><comment><USER13> So it wasn't a desert after all, and all those people claiming 'all' of LA's problems are due to idiots building a city in the desert are just... false?

<comment><USER8> From what I have heard LA could support around one million people naturally along a very thin strip along the coast. Certainly not the number of people there now.

It's also not a desert climate but "arid mediterranean climate".

<comment><USER14> > From what I have heard LA could support around one million people naturally > along a very thin strip along the coast. Certainly not the number of people > there now.

One million people at what level of resource efficiency? The efficiency of the 1970s, when washing machines used ~200L of water per load, and people watered their lawns with spray sprinkers, which would evaporate half the water before it touched the ground?

Or one million people using modern, water-conserving appliances, and, heaven forbid, not having evergreen lawns?

<comment><USER8> This statement was made at a workshop I attended a while ago. I don't think it was meant as a precise statement but to illustrate that the LA area is vastly overpopulated in comparison to local resources.

</comment></comment></comment></comment><comment><USER15> A lot of it was "tamed" with major engineering around the river.

[http://lariver.org/](http://lariver.org/) for pictures

</comment><comment><USER16> What will be left of earth in 100 years? We can't seem to understand, or be disciplined with what we understand, that we're destroying what ultimately sustains us.

These are beautifully sad pictures. If there were any pictures of the vast Icelandic forests before they were razed, I'd feel the same.

Nearly every space of earth is trending towards pavement and buildings. I'm asking myself, what can be done, where is there hope...

<comment><USER17> What can be done is to build up areas which are already built up, increase density, and bring people out of sprawl. Preserve areas and leave them untouched, and meanwhile make the most of the areas that we do use.

This is good for humans (far higher economic activity) and good for the environment. We must fight height restrictions, and other sprawl inducing laws. We must have more protected areas. And we need to provide transit options that scale well in dense areas.

<comment><USER18> I'd wager that Agriculture has eliminated more natural habitats than any population sprawl. I'm reminded of this every time I drive out of Colorado and into Kansas.

<comment><USER17> Yes, agriculture is a massive impact, but as we get better at farming we require less land (at least in the US, I'm not very familiar with other countries.)

From the 1920s to the 1970s, forests on the Eastern seaboard had actually been rebounding due to less intense agricultural use:

[https://www.scientificamerican.com/article/us-eastern- forest...](https://www.scientificamerican.com/article/us-eastern-forests- resume-decline/)

But then since the 1970s they have begun shrinking again as suburban sprawl took over.

</comment></comment></comment></comment><comment><USER19> Warning: This site breaks your navigation and spews modals at you even with uBlock Origin.

</comment><comment><USER20> Cool! I had no idea.

<comment><USER16> What is cool about this? I understand the beauty and also the amazement about not knowing this before, but just to say it's neat is so disconnected from what followed. I.e. "wow, that's so cool the area used to be vast wetlands and now it's all paved over and suffering from droughts etc etc!"

<comment><USER21> The user you're replying to read a cool article, with some cool photos, learning a cool new fact, and wanted to express that it was "cool" in the comment section of the link.

Don't be weird with people, it makes people less likely to hear your message.

</comment></comment></comment><comment><USER22> That URL...</comment>


Software Engineers Think They're The Best - <AUTHOR> http://www.cnet.com/news/software-engineers-think-theyre-amazingly-great-study-says/

<comment><USER2> I see no reason not to you the original title of the article, which is "Software engineers think they're amazingly great, study says.."

However, the article is a bunch of BS. They only interviewed 1000 people, and they didn't say where that 1000 people came from. If they spoke to 1000 engineers in SF, I suspect, yes, they are likely taking people who work at software companies where the company wouldn't exist without them. Talk to 1000 engineers in NYC, and you would probably get a different result.

In the end, we can't deny the fact that a huge volume of business today is reliant on technology, and that technology, even if it is hardware, runs software. I don't think this is a case of the farmer taking credit for the success of a great chef, this is more an example of a chef having great influence over a restaurateur.</comment>


This is how Doom generates random numbers (1993) - <AUTHOR> https://github.com/id-Software/DOOM/blob/master/linuxdoom-1.10/m_random.c <comment><AUTHOR> This is the random number generator used in the original release of Doom (the 1993 video game).

I was leafing through the source code and stumbled upon "random.c"

I found it amusing to see how they generated random integers; basically, they have a hardcoded array of "random" values, and whenever the game requires a new random int, it will return the element at the array index and increment the index by 1.</comment>


Show HN: Journal Your Life as Data with Thyself.io - <AUTHOR> https://thyself.io/ <comment><USER3> I really like the idea of tracking myself (e.g. books read, weight, steps taken etc.), but as a hacker I'd never let anyone else hold this data. I mean, Facebook is one thing, but highly personal details about my personality is another thing.

That said - it would be cool to see something like this open sources. Currently I'm using excel to track most of the stuff I'm interested in, but I know that there must be a better alternative.

<comment><AUTHOR> I totally understand. I'm the kind of person who keeps a diary inside an encrypted true-crypt folder, so security and privacy have been my primary concern from day one. The main reason I'm deploying this as a web-service rather than a downloadable app is so that you can track things from anywhere (without having to be technical enough to manage your own servers). It's much harder to do that with excel or true-crypt, even if you use it with things like google-docs or dropbox.

<comment><USER4> Still, and I don't mean to bash your work, I think this is a disservice to the internet because it perpetuates cloud tools and keeps the state-of-the-art even for something as personal as journaling in the cloud. That's a very, very sorry situation. Paraphrasing the old adage, if you haven't something to contribute that's in the best interests of people, better not to contribute at all. I keep my private stuff local as well, and you yourself admit to doing the same. Why not focus on making it easier for people to do the same, instead of riding the popularity contest of the quick-start cloud?

<comment><AUTHOR> Because I believe that privacy and the cloud can co-exist in a mutually beneficial fashion. Just because the app is online doesn't mean that it has to be "social" (there is intentionally no social-sharing options for the log or journal entries). But by being online, the application itself can improve over time by analyzing aggregate data to detect anomalies (like Mint.com), give predictions, refine the natural language parsing etc.

If this was just a journal, I would agree with you and make it self-contained. But it's not. In the next 5-10 years we're going to be seeing a massive number of wearable sensors embedded in our clothes, appliances and tools. A managed, online API will be better able to be always available, consuming this tsunami of data and turning it into useful insights for you.

</comment></comment></comment></comment><comment><AUTHOR> By the way, the site's running entirely on a single ec2 micro instance (nginx, redis, posgres and go). I'm curious to see if my optimizations pay off and if it can continue to handle the HackerNews traffic.

<comment><USER5> It appears you've done a pretty good job :)

</comment></comment><comment><USER6> That's crazy, I've been developing something very similar for personal use: [http://rcrd.org/](http://rcrd.org/)

<comment><AUTHOR> That looks pretty good! You have an interesting model of abstraction with the records, cats and magnitudes. For me, each data point is an Entry. Entries have a primary Metric/Action and many details, which may have a numerical amount associated with them. Free-form representations like yours and mine make logging data easy, but analyzing it can get pretty tricky.

<comment><USER6> Awesome, if you're open to talk some time I'd love to learn more about your way of doing things.

</comment></comment></comment><comment><USER7> This is executed very well. I love the natural language processing for diary entries.

However I think you need to change your pitch if you want more incoming traffic. At the moment this has an aura of creepiness and vainness to me and no chance of SEO (who googles for "track every moment of my glorious life in detail"?). You should configure this for whitelabelling and create a version for different markets.

This is just one idea because I knoe my wife would love it:

Dieters to track calories eaten each day and exercise done. Would be awesome to see energy counts of food eaten vs energy consumed by exercise done - <USER2> like a weekly comparison of energy in and out.

<comment><AUTHOR> Thanks, do share it with your wife!

I definitely need to work on my pitch more. As for search engines, I tried to target the words Online Diary/Journal (which seem to get searched a lot (1M+ queries per month)) without coming off as spammy.

</comment></comment><comment><USER8> I'm confused by this, because there doesn't seem to be a clear "do this, to achieve this" message anywhere on the page.

</comment><comment><USER9> For privacy concerns you could store the keys (i.e. sleep, running) locally with identifiers to retrieve the values (i.e. hours, kilos) from your database and display/handle them in your app. This way the user keeps track of what exactly is being journaled. Of course, this brings along several other problems like data preservation (local storage).

</comment><comment><USER10> Pretty solid running pace for 2.5 miles.

</comment><comment><USER11> This is pretty impressive in that it uses NLP to allow you to track most (any?) verb.

Any chance in making it open source?

<comment><AUTHOR> Sure! I can release it as a library if there's interest. It's basically a go- client querying a redis instance which has been pre-fed with the NLTK categorization for most common words (about 8000 or so).

<comment><USER11> I think there would be some interest in both the NLP as well as the tracking in general (many say they want to track themselves but privacy is an issue)

<comment><USER12> I for one am massively interested in Go NLP, and this project as a whole. I'd really love to see this project opened up.

<comment><USER13> So do I, but mostly because i'd like to deploy it on my own private server. I'm a bit paranoid when it comes to my data.</comment></comment></comment></comment></comment>


Show HN: WhenLambo – new crypto trading platform demo - <AUTHOR> https://whenlambo.trade/demo <comment><AUTHOR> Long story short, it has a bunch of awesome unique features that'll make your trading life much easier.

1\. Color coded market overview, so you can see coins price movements without clicking through all charts.

2\. Automagical calculation of all your portfolio numbers, profits and loses __, __average buy prices (even cross-currency!), etc. No more Excel, no more manual calculations — forget it!

3\. Best charting solution from TradingView. And moreover, you can trade right on the chart!

4\. Carefully crafted notification system will notify you via Telegram about major coins price movement, significant changes of your portfolio, and manual alerts that you've set. You can stop watching every tick of your assets' prices. We'll take care of it and inform you upon important moves.

5\. Evaluate your trading performance just by looking on automagically calculated gains from trades. In percents, in coins, in USD. (And it's even bidirectional! If you sell 1 BTC for $8000 and bought back for $7000 — it will show you that gain of ~0.14 BTC.)

6\. And it's mobile ready. You can trade on the go, anytime, anywhere.

7\. And it's well-secured! All the sensitive data is stored highly encrypted. And 2FA.

8\. Currently, Binance only. More exchanges to come soon.

ABOUT SECURITY: [https://news.ycombinator.com/item?id=17252443](https://news.ycombinator.com/item?id=17252443)</comment>


Holocaust 'hero' Sir Nicholas Winton dies aged 106 - <AUTHOR> http://www.bbc.com/news/uk-england-33350880

<comment><USER2> Not sure why the BBC placed "hero" in quotes. Sounds like the very definition to me.

Almost as amazing as his actions which saved 669 (!) children is the fact that _he said nothing about it for decades_ and it wasn't until his wife found an old notebook that the story came to light.</comment>


Django REST framework 3.1 released - <AUTHOR> http://www.django-rest-framework.org/topics/3.1-announcement/

<comment><USER2> Super useful tool, thanks to Tom and everybody who contributes. When we began moving some of our APIs from Django to Flask at my company, djangorestframework was the single tool/package that I missed most. It's lovely.

What are the plans (if any) for continued development on Flask-API?

<comment><USER3> I'm curious why you decided to move APIs from Django to Flask? We've built our APIs on Flask, and, though I like Flask a lot, the major complaint is that there's no real story around user authentication and authorization, endpoint versioning, and a handful of other things that aren't directly serving the purpose of the API, but are important nonetheless.

There are a hodge-podge of plugins to do various things, but they don't necessarily share the same design principles or work well together. Some are so abstracted that by the time you've got a concrete implementation you could have just rolled your own.

EDIT: Grammar, added a sentence.

<comment><USER4> I am writing an API right now in Flask, and I chose it over Django never having used Django for a REST service before.

When you say "there's no story" around auth, endpoint versioning, and other things does that just mean that you're not forced to follow those conventions with Flask / that Django offers utilities to help? I am a bit confused/curious what you meant by that since I've seen some wonderful Flask auth work, and endpoint versioning is pretty standard on most APIs, but seems like Django is a bit more strict with what they want you to do.

<comment><USER5> strict in what way? I've built many api's using django-rest-framework and it is truly awesome to work with, completely extensible and flexible and never seems to get in my way

<comment><USER4> I just meant that they offer more of a base + guidelines for you to work with when compared to Flask's "roll it yourself" approach to REST.

</comment></comment></comment></comment></comment><comment><USER6> DRF brought me back to Django after a foray into Flask-land. It's a pleasure to work with. Thanks for your work, and to the backers who funded the latest development!

</comment><comment><USER7> @OP:

What is your long term vision for the project?

Reason I ask is due to how django keeps (IMO) gaining territory on the corporate and/or business stack. I get requests for support and new development consistently from companies in different markets/regions.

<comment><AUTHOR> Broadly, we aiming for it to be a mature, sustainable and well-supported general purpose Web API framework.

We tend to be fairly aggressive about keeping the scope manageable in order to keep to that. For example 3.1 moves some functionality out into third party packages, so that we can concentrate on the quality of the core framework.

Some of the longer term feature work:

* Admin style interface - See [https://www.kickstarter.com/projects/tomchristie/django-rest...](https://www.kickstarter.com/projects/tomchristie/django-rest-framework-3)

* Fuller support for alternative storage backends. Eg. Easier to integrate with SQLAlchemy, as an alternative to the Django ORM.

* Hypermedia support & client library. (Too involved to go into detail here.)

<comment><USER7> Thank you.

</comment></comment></comment><comment><USER8> Just wanted to say that this is one of the most beautifully-documented projects I've ever used. Bravo!

<comment><USER9> Agreed, although I wish there was more documentation on the benefits of each feature, rather than just how to implement it.

E.g. there are dozens of pages on how to implement serializers, which is great, but the section on the benefits of serializers is only a paragraph or two long. So after reading all the documentation I'm still unclear as to the best practices on when vs when not to use them, and what the best practices are for endpoints that don't take advantage of them. I'm sure this kind of stuff is obvious to people who have been using it for a while, but after just going through the tutorial and then reading the relevant API pages I was still unclear on a bunch of issues.

<comment><AUTHOR> Thanks Alex, I think that's a great point.

Please do feel free to follow this up by raising an issue on the repository - it'd be good to talk through some specifics about areas to improve.

</comment></comment></comment><comment><USER10> Anyone have any thoughts on DRF vs Tastypie?

I've been using tastypie for the past few months in a project and I'm kindof regretting it - I'm finding some of the design decisions hard to understand and not super well documented. But I've gotten to the point where I'm using it in production and it feels like it might be too big of a pain to switch at this point.

<comment><USER11> I'm not an expert, but from what I've read of the docs of both projects I understanding DRF is more modular and easier to customize with its Serializer, Parser, View, Viewset, etc. classes than Tastypie with its monolithic Resource class.

</comment></comment><comment><USER12> Really loved this tool when I was writing Python.

</comment><comment><USER13> My friend and I just started getting into Django the other day. Coming from Node.js it is a welcome change.

<comment><USER14> what do you like about it vs node?

</comment></comment><comment><USER15> Has anybody used both Django REST and restless[0]? My company's starting to migrate a Django-based system towards something more REST-full.

[0] [http://restless.readthedocs.org/en/latest/](http://restless.readthedocs.org/en/latest/)

<comment><USER16> I've got a little experience with both. I went full bore into DRF, and found myself wrestling with the framework too much. I wanted to use the framework (ModelSerializer), but found it difficult to find the hooks to customize some behavior. Then going down a layer of abstraction felt like I had to do too much. This is probably the reality of any framework, not necessarily the fault of DRF.

Right now I'm using django-restless. I'm sticking with it for now. It uses Django forms for validation, which works ok even though they weren't meant for validating json input. Now I'm wondering if it would be best to just use Django, and be restless in style. Perhaps DRF, restless or some other libraries would have some pieces that I could use in a more library fashion if I wanted them.

</comment></comment><comment><USER17> I've started using DRF last week and I don't know why I did'nt use it before.

The authentication framework and the Model serializer are really good points to start, but I was missing some versioning scheme and 1 week later here it comes. Great work, Tom.

</comment><comment><USER18> Just wanted to let you know we're a huge fan of Django Rest Framework! Thx from getstream.io

</comment><comment><USER19> Is the upgrade path from django 1.4.18 / drf 2.4.2 relatively easy?

<comment><AUTHOR> The 3.0 release notes are the place to start for looking at upgrading. They go into quite a bit of detail about this.

</comment></comment><comment><USER20> Still no M2M for ModelSerializer..? Guess I'll have to just keep using a ton of shitty hacks.

<comment><AUTHOR> Of course you can use many to many fields with ModelSerializer.

I'm not sure what you mean here, but I'd suggest opening an issue if you believe there's a problem - we're pretty comprehensive about dealing with incoming issues.

<comment><USER20> An issue that I've consistently run into is being able to write to related models. I can serialize them just fine, but without being able to write, it forces users to come up with nasty work arounds. To be fair, I hate having to write resources that handle writing nested objects, but there are some many front end use cases where multiple writes is not ideal.

<comment><AUTHOR> Oh right, writable nested serializations, sure. I won't rehash the reasons that we don't support that in core (ill defined what behavior the user should expect) but yeah it'd still be good to have a third party package that does a decent "best effort" job of that.

My personal take on this is forget about "shitty hacks" or trying to handle this automatically, and just write the serializer create and update methods explicitly.

<comment><USER20> I know why it's not supported and I agree that there should be a third party package for it that is more or less officially supported/recommended like the jwt package.

Most recently I created a new handler object that runs some introspection of the model. I then override my model view sets' create and update methods to implement the handler. Maybe I should get off my ass and open source it (currently buried in a project).</comment></comment></comment></comment></comment>


What Clojure Spec is and what you can do with it - <AUTHOR> https://www.pixelated-noise.com/blog/2020/09/10/what-spec-is/ <comment><USER2> I think the instrumenting and generator stuff gets disproportionate attention. For me by far the biggest win from spec has been with parsing. This completely changes how you'd write a library that takes a data structure and parses it into something meaningful (for example, what hiccup does for html or what honeysql does for sql).

In the past, this required a lot of very ugly parsing code and manual error- checking. With spec, you write specs and call s/conform. If it failed, you get a nice error, especially if you pair it with expound. If it succeeded, you get a destructured value that is really easy to pull data out of. I've done this in a half dozen different libraries and i'm pretty sure i wouldn't have even written them without spec.

<comment><USER3> and if you then combine s/conform with core.match, you can build really elegant code to traverse these structures.

<comment><USER4> Why not just pattern match immediately if you are going to bother with using core.match?

<comment><USER3> How would you pattern match with Clojure without using core.match? Do you mean using another library?

Edit: oh I understand what you mean, you were thinking of skipping the s/conform part, and use core.match directly. Personally, I consider spec an excellent library to describe the shape of data, and core.match allows for better describing the actions you want to take based upon that.

For example, with spec you can define a bunch of alternatives using s/or, and then use core.match to then easily traverse the results.

It's more a matter of separation of concerns to me. I don't use core.match for validation or describing shapes of data.

<comment><USER5> Do you have to adapt spec definition to core.match somehow? Is there a resource to read more how to use them together?

<comment><USER3> No you don't have to do anything.

Let's say I have something like

      (s/def :thespec (s/or :foo ::foo
                            :bar ::bar))

I can then use it in conjunction with core.match like

      (match [(s/conform :thespec val)]
        [:foo x] (do-something-with x)
        [:bar y] (do-something-else-with y))

Which imho is an easier way to navigate these things.

</comment></comment></comment></comment></comment></comment><comment><USER6> As a very heavy user of spec, I've since switched to using Malli [0], which is similar but uses plain data to model specs (and doesn't use Clojure Spec at all).

Also, Malli is being funded / supported by Clojurists Together [1], which is a wonderful initiative that's also worth a look.

[0]: [https://github.com/metosin/malli](https://github.com/metosin/malli)

[1]: [https://www.clojuriststogether.org/news/q3-2020-funding- anno...](https://www.clojuriststogether.org/news/q3-2020-funding- announcement/)

</comment><comment><USER7> I found spec very useful and use it more and more. I'm looking forward to newer revisions, with support for optionality, it's been a big problem area in my case.

Here's a quick list of gotchas (well, they got me, so perhaps other people will find this list useful):

* `s/valid?` does not actually tell you that the data is valid

The naming of `s/valid?` suggests that you can call it on your data and find out if the data is valid according to the spec. This isn't true. What it actually tells you is if the data, _when conformed_ , will be valid according to the spec. If you pass the data as-is to your functions (without passing it through `s/conform`), you might find that they will be surprised at what they get.

* Conformers are likely not what you think. They are not intended for coercion and have many pitfalls (for example, multi-specs dispatch on unconformed value).

* s/merge doesn't necessarily do what you wanted if you're using conformers, only the last spec passed to merge will be used for s/conform (but you're not using conformers, right?)

* specs are checked eagerly. If you think that (s/valid? ::my-spec x) will only check ::my-spec, that is not the case. It will check any key of x found in the spec registry.

I settled on a subset of spec, because of the pitfalls.

<comment><USER8> I think all of those problems are only true when you try to do coercion with custom conformers. Which is not the intended use of conformers.

Conformers are meant to parse the data when there are multiple possibility of what something can validate against, the conformer will disambiguate and return a result that tells you which path was chosen.

Coercion is not supported as part of Spec, you're expected to do that seperatly either before or after validating/conforming.

<comment><USER7> Yes, that was largely my point. Many people (me included) assume that s/conform is a kind of coercion, which it is not.

Not all of the above problems are due to coercion, but the majority are.

To be clear: I'm not complaining here, I find spec to be very useful and I like it, just pointing out traps for the unwary.

<comment><USER8> Ya fair enough, I've definitely seen a lot of people think conforming is meant for coercion. But it's not, it's only meant for disambiguating the chosen path (when multiple are possible) for validation.

</comment></comment></comment></comment><comment><USER9> We use spec A LOT to validate a huge config file/DSL thing for our internal ETL application.

My general feelings are:

\- spec is great, you should use it, the composeability and flexibility are awesome features.

\- I've never once used conformers - maybe I just don't "get it" (which if so, speaks badly of them I think since I've been heavily using spec for years), but the use cases for them seem strange to me and I feel they cause more confusion than they're worth. I wish they were separated out into more separate/optional functionality.

\- It's SO MUCH more powerful than things like JSON schema, but that comes at the cost of portability - there's no way we could send our spec over the wire and have someone else in a different environment use it. But also, there's no way we could implement some of our features in a tool like JSON Schema [and have it be portable] ("Is the graph represented by this data-structure free of cycles?" "When parsed by the spark SQL query parser, is this string a syntactically valid SQL query?").

\- Being able to spec the whole data input up front has saved hundreds of lines of error-checking code and allows us to give much better errors up-front to our users and devs

\- Spec has a lot of really cool features for generative testing, but we rarely use them since we've implemented lots of complex specs where it's not really practical to implement a generator (i.e. "strings which are valid sql queries" or "maps of maps of maps which when loaded in a particular way meet all the other requirements and are also valid DAGs"). I feel torn about this because the test features are great, but the extreme extensibility of spec is what I love most about it. I haven't often found a scenario where I actually have a use for the generative features (either the data is so simple I don't need them, or so complex that they don't work).

<comment><USER10> Your feeling on there being no way to transport this over the wire is puzzling to me but I admit I don't have all the details. My feeling is why not? Surely if we have wire formats for self describing binary objects that can then be serialized into an in memory structure, transporting a spec shouldn't be harder than that?

<comment><USER11> Specs in the general case require code execution, so you'd essentially need to execute that (untrusted) clojure on the other end of the wire.

<comment><USER10> Again, apologies if this sounds ignorant, but we have pretty standard practices now for sandbox execution of untrusted code. A LISP seems especially suitable for this type of task.

<comment><USER11> I don't have a clue how you would implement this. The difference in portability between spec and something like json-schema/protobuf/avro is that you can serialize the schema in these and then clojure and (say) python, go, java, C#, JavaScript applications can talk to one an other.

How would you propose to serialize clojure spec's and use them from a python app? Port the clojure compiler to python?

<comment><USER9> I second this emotion - "How would you check a spec from anything other than clj/cljs" is (IMO) the critical question here. Sure you could check out my git repo in a safe VM and execute it there, but that's a WHOLE lot more hassle than an XML or JSON schema. It's not just a language barrier thing.

There's nothing stopping spec predicates from making network calls, looping forever, etc. If I wanted to be able to call my spec from other apps I'm writing, I could package it as a library easily, but a workflow like rest call->get spec->validate data (which I've implemented many times for JSON schema for simpler things) wouldn't really be practical with spec (without at least setting some really tight restrictions on what features of spec you're allowed to use)

Again, not really a failing of spec, it's just not designed for that kind of workflow.

<comment><USER12> Yeah spec is quite a general thing, there are libraries which convert specs into database schemas, JSON schema, swagger etc

If communicating constraints to another environment is required they should help

</comment></comment></comment></comment></comment></comment></comment><comment><USER13> I strongly recommend that anyone using spec for validation should check out Orchestra, to instrument the functions and have them automatically validated on each and every call: [https://github.com/jeaye/orchestra](https://github.com/jeaye/orchestra)

For my team, generators and parsing are basically useless with spec. We just don't use them. But describing the shape of data and instrumenting our functions, using defn-spec, to ensure that the data is correct as it flows through the system is exactly what we want and nothing I've seen in Clojure land does it like spec + Orchestra can.

I think part of this may boil down to different types of testing. We primarily use functional testing, especially for our back-end, so we're starting from HTTP and hitting each endpoint as the client would. Then we ensure that the response is correct and any effects we wanted to happen did happen. This is much closer to running production code, but we do it with full instrumentation. Being able to see an error describing exactly how the data is malformed, which function was called, and what the callstack was is such a relief in Clojure.

        Call to #'com.okletsplay.back-end.challenge.lol.util/provider-url did not conform to spec.

        -- Spec failed </comment><comment></comment><comment></comment><comment>--

        Function arguments

          (nil)
           ^^^

        should satisfy

          (->
           com.okletsplay.common.transit.game.lol/region->info
           keys
           set)

        -- Relevant specs </comment><comment>-

        :com.okletsplay.common.transit.game.lol/region:
          (clojure.core/->
           com.okletsplay.common.transit.game.lol/region->info
           clojure.core/keys
           clojure.core/set)

        </comment><comment></comment><comment></comment><comment></comment><comment>-
        Detected 1 error

<comment><USER8> I'm onboard with orchestra because I tend to be lazy. But I also want to explain the rational for why Spec's instrumentation only instrument the input.

This has to do with the philosophy. If you want to write bug free programs, and I mean, if you care A LOT about software correctness.

The idea in that case will be that all your functions will have a set of unit tests and generative tests over them that asserts that for most possible inputs they return the correct output.

Once you know that, you know that if provided valid input, your functions are going to return valid output. Because you know your function works without any bugs.

Thus, you no longer need to validate the output, only the input. Because as I just said, you now know that any valid input will result in your code returning valid output as well. So re-validating the output would be redundant.

And this goes one further. After you've thoroughly tested each functions, now you want to test the integration of your functions together. So you'd instrument your app, and now you'd write a bunch of integration tests (some maybe even using generative testing), to make sure that all possible input from the user (or external systems if intended for machine use) will result in correct program behavior and an arrangement of functions that all call each other with valid input.

Once you've tested that, you now also know that the interaction/integration of all your functions work.

At this point you are confident that given any valid user input, your program will behave as expected in output and side-effect.

You can thus now disable instrumentation.

But before you go to prod, you need one more thing, you have to protect yourself against invalid user input, because you haven't tested that and don't know how your program would behave for it. Thus with Spec, you add explicit validation over your user input which reject at the boundary the input from the user of invalid.

You now know there things:

1\. All your individual functions given valid input behave as expected in output and side-effect.

2\. Your integration of those functions into a program works for all given valid user input.

3\. Your program rejects all invalid user input, and will only process valid user input.

Thus you can go to prod with high confidence that everything will work without any defect.

\---

Now back to orchestra. Orchestra assumes that you weren't as vigilant as I just described, and that you might have not tested each and every function, or that you only wrote a small amount of tests for them which only tested a small range of inputs. Thus it assumed because of that, probably when you go towards running functional/integ tests, you want to continue to assert the output of each function is still valid, as you anticipate those will probably create inputs to functions that your tests over that function did not test.

<comment><USER13> Something like Haskell, or even Rust, requires similar vigilance in order to get the program even into a working state. With thorough, strong, static type checkers, novel borrow checkers, and more, a lot of development time is spent up front, dealing with compiler/type errors. Thus you can go to prod with high confidence that everything will work without any defect.

Now, back to Clojure. Clojure assumes that you weren't as vigilant as I just described, and that you don't have static type checking for each function, or that you don't have a fixed domain for all of your enums. Thus it is assumed because of that, probably when you go running toward testing (unit, functional, or otherwise), you want to assert the validity of all of this data.

My point in re-painting your words is that we all trade certain guarantees in correctness for ease of development, maintainability, or whatever other reasons. Developers may choose Clojure over Haskell, for example, because maintaining all of that extra vigilance is undesirable overhead. Similarly, developers may reasonably choose not to unit test every single function in the code base, but instead functionally test the public endpoints and unit test only certain systems (such as the one which validates input for the public endpoints), because maintaining all of that extra vigilance is undesirable overhead.

<comment><USER14> Also note that if you try thinking with types, you may start seeing them as tools rather than overhead.

A good blog post about this is:

[https://lexi-lambda.github.io/blog/2020/08/13/types-as- axiom...](https://lexi-lambda.github.io/blog/2020/08/13/types-as-axioms-or- playing-god-with-static-types/)

</comment></comment></comment></comment><comment><USER15> My pet project is a partial evaluator for Clojure code that uses data generated from spec to fuzz code and optimize it. The coverage is accepted as complete, so there are no guards and deoptimizations like you'd have in a JIT, the programs are just wrong.

It seems like a fairly powerful technique, although you couldn't ever rely on it with production code. After several years of tinkering I managed to get a Forth interpreter written in Clojure executing a specific input string partially evaluating down to OpenGL shader code, to hardware accelerate my friend's Stackie experiment (link to his version below).

(nth (sort [0 n 5]) 1) where sort is a merge sort also successfully compiles down to just the branches you'd hand optimize it to, which is Graal's party trick. Although they're solving the problem in a bulletproof general way, so the difficulty is incomparable.

The eventual goal is to write Clojure in Clojure without it being horrendously inefficient.

[http://blag.fingswotidun.com/2015_09_16_archive.html](http://blag.fingswotidun.com/2015_09_16_archive.html)

</comment><comment><USER16> At Hyperfiddle we are using spec to specify UIs, so e.g. (s/fdef foo :args (s/cat :search-needle string? :selection keyword?) :ret (s/coll-of (s/keys ...))) describes exactly what you need to render a masterlist table with some query parameters. It's being used with pilot customers, now in production. This stuff works!

</comment><comment><USER17> Malli[1] and specmonstah[2] are my favourite Clojure(Script) libraries built on top of Clojure Spec.

[1] [https://github.com/metosin/malli](https://github.com/metosin/malli)

[2] [https://github.com/reifyhealth/specmonstah](https://github.com/reifyhealth/specmonstah)

<comment><USER18> +1 for Specmonstah; been very useful for us at Reify Health.

</comment></comment><comment><USER19> I find spec really interesting but struggle to find a good fit for it in my projects. I end up getting bogged down writing generators and give up, or end up breaking down functions into such small pieces that speccing every one of them results in an unreasonable proliferation of specs and generators. This was an interesting and helpful overview of some different ideas about how to take advantage of spec, so thank you!

Also I definitely share your feelings about clojure massively increasing my job satisfaction and teaching me how to think about programming in a new (and better tbh) way.

<comment><USER20> We use spec at work for validating data passed to our HTTP routes in our luminus web application. When we get data as JSON we use spec to validate that there are the required fields being passed in and if it isn't valid we send back an error HTTP status code. It is nice to be able to easily see what fields are required without having to dig deep into code.

</comment></comment><comment> john-shaffer If you're considering using spec, you should know that spec alpha2 is significantly different internally, and is much easier to write tooling for, but has not been ported to ClojureScript. I would highly recommend trying alpha2 if you're not using CLJS, even if it's a little less stable.

[https://github.com/clojure/spec-alpha2](https://github.com/clojure/spec- alpha2)

</comment><comment><USER21> > Still alpha: spec is still alpha and it looks like it will remain like that > since the intention is for spec2 to completely replace it in the future.

Out of curiosity for the folks in the know: why is this the state of affairs? Where we have the library in alpha for a long time, and being replaced already?

<comment><USER22> I remember Rich mentioning his in a talk. There's one problem he considers unresolved and he wouldn't finalize the API until he had a good solution. Unfortunately, I can't remember what it was. Also, this must have been more than a year ago, whatever goes on in spec2 might have evolved since then.

</comment></comment><comment><USER23> for simple typing, does this seem more complex than plumatic/schema?

<comment><USER24> I have used both Spec and Schema professionally.

It's not exactly a secret that Spec is not _intended_ to be a type-like system. But it turns out, it's perfectly possible to use Spec as a building block to do so!

As for complexity, I think instrumentation (Spec's approach) _is_ complex. People routinely struggle with it, in fact you must use an external lib (Orchestra) to fully enable it. It also slows down execution, sometimes disproportionally.

Whereas Schema simply has a global toggle that also can be overriden in a fine-grained manner. It's explicitly made to be fast and have a type-like use.

As for type definitions, honestly both are elegant and powerful; you can quite easily use arbitrary predicates as "types" with both, and compose those predicates. Ultimately Spec is better designed because it fosters namespaced- qualifed keywords which compose better.

For addressing the instrumentation problem, I created [https://github.com/nedap/speced.def](https://github.com/nedap/speced.def) which uses Clojure's :pre system. I have used :pre for over a decade so I know it has an extreme simplicity. Spec can take :pre's usefulness to the next level.

</comment></comment><comment><USER25> Very interesting</comment>


Computers don't dumb down math education - <AUTHOR> http://blog.wolfram.com/2010/09/27/do-computers-dumb-down-math-education/

<comment><USER2> Is this an article or an advertisement for Mathematica?</comment>


Motorola to release phone with no bezel by end of year. - <AUTHOR> http://www.androidpolice.com/2012/08/31/bloomberg-motorola-will-release-a-phone-with-no-bezel-by-the-end-of-the-year/

<comment><USER2> One should note this is the first phone developed by Google under control of Motorola. When this is released, it will finally answer many questions about how much Google will influence them. This news about the lack of bezel is the first sign that Google might be willing to use Motorola as a test lab for weird innovation.

<comment><USER3> > This news about the lack of bezel is the first sign that Google might be > willing to use Motorola as a test lab for weird innovation.

I think this is more about avoiding Apple's design patents. I'm pretty sure that a rectangular bezel was one of them.

</comment></comment><comment><USER4> While the microphone and buttons (if any) fit well on device's edges, I wonder how do they provide earpiece? Do they drill a hole through the display, or is it some sort of flat panel speaker? The later probably would be bad for privacy.

<comment><USER5> I can think of two ways to do it.

First, you could put the piezo film (commonly used in speakers for cellphones, more efficient for smaller transducers but requires a special amplifier) directly below (but in contact with) the display and use the display as the 'diaphragm'. Makes the entire display a speaker, but would require some interesting sound cancelling tricks to avoid feedback with the microphone.

Second, you could use a piezo below the display with an airgap and use something akin to the laser used in the power LEDs on the Macbooks to poke enough airholes to direct the sound toward the ear.

A third method, which I'm not sure would work, is to suspend the piezo material between two ITO layers (like that used in the capacitive touchscreen) and put it _above_ the display. Depends on if someone has found a transparent piezoelectric material yet; I haven't been keeping up with the materials science side of things.

</comment></comment><comment><USER6> Remember the Nokia Aeon concept phone?

<http://www.nokiaaeon.com>

PS: Also picked this (<https://www.youtube.com/watch?v=HU2nG9qy6vs>) up from <http://forums.whirlpool.net.au/archive/1921701>

</comment><comment><USER7> I'm curious to see the usability. If I look at the way I usually hold my phone, about 5-7mm of the front are obscured. I'm not saying this is bad, but I'd have to see it in action.

</comment><comment><USER8> The one big question is not answered by the article. Will they get a design patent for it?

<comment><USER9> Most probably, and they will sue Apple if they ever use a similar design. Motorola is under Google's management now. So, they must do whatever it takes to protect the Android ecosystem and thattheir partners. Sadly, that mostly means to sue somebody these days.

<comment><USER10> Well, that's the difference between deterrence and Thermonuclear War. Once the despots strike, there's no going back.

</comment></comment></comment><comment><USER11> Whatever next? Flat loudspeakers!?! Telegraphic communication!?!?!

</comment><comment><USER12> Where did the photo in this article come from?

<comment><USER13> Iron Man probably.

<comment><USER14> yup, Iron man 2.

</comment></comment></comment><comment><USER15> What about accidental touches?

<comment><USER16> Surely you can make a UI to deal with this.

<comment><USER17> Sure, just setup the UI so that no elements are within 1/4 inch of the edge of the screen so that your fingers don't activate something. Simple.

/s

<comment><USER18> Or just ignore the fingers that comes in all the way from the edge of the screen.

<comment><USER3> Wait even better, we could put some kind of border around the screen that isn't responsive!

<comment><USER19> And send movie playback, which doesn't need touch, back to postage stamp territory!</comment></comment></comment></comment></comment></comment>


Steak: because Cucumber is for vegetarians - <AUTHOR> http://jeffkreeftmeijer.com/2010/steak-because-cucumber-is-for-vegetarians/ <comment><USER2> Somehow these "real language" extensions for Ruby remind me of the XML craze for Java. To feel right, everything had to be configured in an XML file for Javaists. In the same way, Rubyists apparently feel uneasy if something doesn't read like nice natural language.

I suspect both things are just cleverly masked procrastination....

<comment><USER3> I agree in most cases, but Cucumber is one of the few times when it really felt beneficial to me. In most cases, I'd rather have straightforward Ruby code like we all understand or a simple YAML config file. But for a declarative spec framework, Cucumber works nicely. Try looking at a Cucumber spec vs. a Steak one and I think you'll see what I mean.

Then again, I am a vegetarian.

<comment><USER4> Why is writing "oh hai, i want my blog to be totally teh awesome. it must have no posts initially, and then should have one post after i add a post with the text OH HAI." better than writing "my $blog = Blog->new; is $blog->post_count, 0; $blog->post("OH HAI"); is $blog->post_count, 1;"?

<comment><USER3> Since I was speaking about this specific case and not similar DSLs in general, all I can say is that neither of those is valid Ruby with either of these frameworks.

<comment><USER4> I am speaking in general. I don't know Ruby, so I am not going to use that in any code examples.

<comment><USER3> OK, so given that I said I generally don't think it's beneficial but it is in Cucumber's particular case, why would you ask me how it's beneficial in general?

</comment></comment></comment></comment></comment></comment><comment><USER5> This is similar to Citrusbyte's Stories library (disclaimer, I work for Citrusbyte). <http://github.com/citrusbyte/stories>

It's a fairly lightweight layer on top of Test::Unit that lets you define nested stories and scenarios, integrating with Webrat (soon Capybara when we get to it).

One differentiating feature of Stories is that you can print out a readable PDF doc with each story, and the specific scenarios that make it up, leaving off the Test::Unit implementation of the scenarios. Because really, clients don't want to read even cucumber's specs. The nasty secret of Cucumber is that it's still programming. You have to be very explicit in what you setup, and what you expect. Even if your explicitness is hidden behind english, it's still there, and many client's just don't think like that. Heck, that's why they hire programmers.

</comment><comment><USER6> " ... because Cucumber is for vegetarians!"

Is this supposed to be some sort of slight, or a put-down?

Is there something lesser about being a vegetarian?

I'm curious, being, you know, a vegetarian and all.

</comment><comment><USER7> What advantage does this give over the built in integration tests?

<comment><USER8> Webrat and/or Capybara. Other than that, it is pretty much rspec on top of integration tests.

<comment><USER9> It also gives you the nice syntax "keywords", feature, background, and scenario.

</comment></comment></comment><comment><USER10> Great, so all we need now is a library named 'Strippers'. (see: [http://blog.cusec.net/2009/01/05/zed-shaw-the-acl-is-dead- cu...](http://blog.cusec.net/2009/01/05/zed-shaw-the-acl-is-dead-cusec-2008/) )

</comment><comment><USER11> For me the disconnect of translating what I want into natural language is part of the benefit of Cucumber.

</comment><comment><USER12> I like Cucumber better. Plain language means you could use it without being a rubyist.

I'd like to see a repository of Cucumber tests for basic HTTP and DOM interaction, with implementations in multiple languages.

<comment><USER13> do you mean webrat steps?

<comment><USER12> No. I mean a plain english API to webrat (or an equivalent to webrat). Right now you have to write your own wrapper steps around webrat (unless I've missed something). No wonder people get tempted to just write webrat tests directly.

</comment></comment></comment><comment><USER14> How many times do we need to go through this cycle?

JBehave: text

RBehave (based on JBehave): code

RSpec Story Runner (based on RBehave): code, later also text

Cucumber (based on RSpec Story Runner): text

Steak (based on Cucumber): code

<comment><USER8> Steak is an extension of rspec and sortof inspired by Cucumber. I do remember Obie Fernandez had a blog post about a dsl that looks very similar a few years ago, however I cannot find it.

Maybe I'm going senile.

</comment></comment><comment><USER9> We just started using steak and it is slowly replacing cucumber in our application.

We found the translation between English -> Step Definitions -> Ruby to be too onerous, especially after our stakeholders showed no interest in helping write the scenarios...

</comment><comment><USER8> I'm happy with Rspec. It accomplishes the same thing with slightly different wording. Actually, I can have Steak in rspec.

      class Spec::ExampleGroup
        class << self
          alias_method :context, :feature
          alias_method :example, :scenario
        end
      end

<comment><USER8> Well, duh, Steak is an extension of rspec. Very nice. :-)

</comment></comment><comment><USER15> I know this is hacker news... but I was honestly disappointed when the article had nothing to do with Steak.

<comment><USER16> I was disappointed that I couldn't read the README in the git repo without a scrolling past a picture that has nothing to do with BDD.</comment></comment>