SQL is a better API language than GraphQL – Convince me otherwise - <AUTHOR> https://twitter.com/simonw/status/1250803209871847426 <comment><USER2> Wasn't expecting this to show up here!

I've been playing around with SQL as an API querying language for a couple of years now as part of my Datasette project: [https://datasette.readthedocs.io/en/stable/](https://datasette.readthedocs.io/en/stable/)

I started out thinking that SQL for API queries was genuinely a terrible idea. Datasette supported it because it's designed as an ad-hoc querying tool (which happens to be able to export resuls as JSON), but obviously that was an awful idea for production applications.

Then I started building a few SQL-in-client-side-JavaScript applications, just as a rapid prototyping tool. And they worked amazingly well. Here's an article I wrote about that in January 2018: [https://24ways.org/2018/fast- autocomplete-search-for-your-we...](https://24ways.org/2018/fast-autocomplete- search-for-your-website/) (really simple demo here: [https://media.24ways.org/2018/willison/](https://media.24ways.org/2018/willison/) )

I keep on waiting for the obvious downsides to show up, and they keep stubbornly refusing to appear. And meanwhile the rest of the software world is going all-in on GraphQL which seems MUCH harder to work with - and less performant.

<comment><USER3> I've tried this a couple times in the past. I think the biggest issue I've ever run into is from database modifications. In those cases, instead of merely needing to update my queries in my api server, then I needed to update my client code, and this gets much harder with multiple clients

Of course, this can also be the case when changing the API, but deprecated fields can be faked in that portion of the application to allow existing applications to continue to work.

I think it comes down to where you're managing your database interactions. I don't care about most features in an orm, so I maintain my own that's barely anything more than a SQL generator that won't break every time I change the database.

Overall, I think SQL on the client is excellent - even ideal. And as a matter of fact, I try to develop apis that simply generate a user-specific excerpt of our database, where SQL would be perfect for querying the data locally on the client. Last year I tried running sqlite in the browser for this exact reason, but it was far too heavy.

Anyway, back to the issue - the idea of having to immediately adjust all our client software every time I make a database adjustment is a bit anxiety- inducing. Especially when building fast In a drive toward market fit .

That said, with some great logging that tracks the actual queries and where they come from, it probably wouldn't be so bad. Especially if there's a means to maintain backwards compatibility.

<comment><USER4> What if the tables your API queried were actually views? You could do the same faking of data during migrations of the underlying data (structure) and the client should not be effected.

<comment><USER5> What if the "views" your API queries didn't _have_ to be tables? You could just keep "tables" as a backend implementation detail, and 'fake' a different model on top of them that's closer to what the client needs.

<comment><USER3> I think what's becoming clear in this discussion is that there are some developers (myself definitely included) who would like to use SQL as an interface to plain programming structures outside of actual databases.

I understand postgres provides something like this, but it would be interesting as a library rather than a separate application. I'd love to be able to query hashmaps or arrays of data using SQL.

Maybe this already exists somewhere? My web-search-fu is failing me but I recall reading somewhere that facebook had something like this for querying systems, so maybe such a library exists extracted from that usecase?

Edit: Here's one. [https://github.com/mozilla/moz-sql- parser](https://github.com/mozilla/moz-sql-parser)

<comment><USER6> SQLite has "virtual tables" feature [0] which was designed exactly for this use-case.

Edit: and that Facebook project is called OsQuery [1]. It was built on top of SQLite virtual tables and is listed in their implementations list.

[0] [https://www.sqlite.org/vtab.html](https://www.sqlite.org/vtab.html)

[1] [https://osquery.readthedocs.io/en/stable/](https://osquery.readthedocs.io/en/stable/)

</comment></comment></comment></comment></comment></comment><comment><USER7> I feel like this thread is tongue-in-cheek to make people realize that "run any query on the client side" is a terrible idea, and that's exactly what most GraphQL installations offer.

As a penetration tester though I like GraphQL, it's a very handy data exfiltration engine. By default, there's no authorization, no authentication, and it even comes with introspection. One query and I get your entire schema! This trendy movement will be far worse for data security than SQL injection, unsecured S3 buckets, or misconfigured NoSQL instances.

<comment><USER8> Yeah my first encounter with GraphQL was a client going "let's save development time by using graphql!" so I looked into it and it's _more fucking work_ unless your stack has one of those auto-GraphQL-to-and-from-DB generators (they'd made a poor and somewhat obscure choice of DB for the project, over our advice, so there wasn't such a thing in this case anyway) and it happens to suit your needs very well—and even with the benefit of one of those I'd _still_ lose sleep over security or DoS bugs lurking in that 3rd party code. Or in mine.

<comment><USER9> The cases where GraphQL make sense are the ones where the front end is making tons of intricate, complicated calls. If it's a simple REST api you're interacting with it probably doesn't need GraphQL and might actually be harmed by it.

<comment><USER10> GraphQL is only interesting for flexibility to fast prototyping/changes on the client side, to me. And it should stay there. Meaning your client JS should have a data layer that can understand GraphQL and translate that to a REST API call and cache the result.

</comment></comment></comment></comment><comment><USER11> I responded in a twitter thread[1], unrolled here here:

As all good things in life, and programming, this is a tradeoff. GraphQL is better when what you are requesting is best expressed as a tree (or a "graph", though only the DAG variety). This is not always the case, but it very often is when building API:s for production use.

Of course, you _can_ express tree structures in table form, but it is not very convenient for clients to consume. In particular if your client is rendering nested component views, what you want is very often something hierarchical.

Another aspect of GraphQL that is better for us production people is that the performance is more predictable, exactly because the language is more restricted. You can't just join in all the things, or select billions of rows by accident. The schema dictates what is allowed.

Of course, again, it is possible to restrict this in SQL, just configure your schemas, limits etc appropriately, but SQL is anything-goes by default, whereas GraphQL is nothing is allowed by default. Whitelist vs Blacklist.

This said, as a language, SQL is clearly superior. It is the most (only?) successful 4GL (declarative) language. I wish more languages were this well- designed, and that there would be more language innovation in this direction.

The way I see it, GraphQL is a DSL for flexibly requesting hierarchical data from API:s in JSON format, optimized for complex evolving API:s. SQL is a full-fledged generic language for relational data transformation. They have different niches, but SQL has a much bigger one.

[1] [https://twitter.com/joakimlundborg/status/125091692202945740...](https://twitter.com/joakimlundborg/status/1250916922029457408)

<comment><USER5> > a DSL for flexibly requesting hierarchical data from API:s in JSON format, > optimized for complex evolving API:s

Perhaps the same goal could be achieved with more flexibility by issuing CONSTRUCT queries to a SPARQL endpoint (CONSTRUCT requests a custom RDF graph as opposed to a set of variable bindings ala SELECT) and obtaining results in JSON-LD format.

<comment><USER12> Exactly. I was always interested in all this semantic web stuff and as I have time now, I'm experimenting with Apache Jena. It's super neat! SPARQL endpoints use HTTP per default. "API for free" so to say. Plus you can even use a binary format for receiving the data. Querying the database is three lines of code in Kotlin. Fuseki (the SPARQL Server in Jena) also supports crazy stuff like "recursively traverse this graph by following the specified type of link until you can't go any further". I'm pretty sure this is not possible in GraphQL.

</comment></comment></comment><comment><USER13> The author makes a case for just sending sql queries from the frontend. For public APIs you can probably make a case. But for a normal application it doesn't make much sense. You only have a fixed amount of queries to run. So why would you implement the hassle and huge exploit surface of validating queries from the frontend. So once you realize this, you'll just give every query some identifier and send the identifier + some parameters instead of the query. Et voila, you have created RPC

<comment><USER14> Because the "frontend" might be a native application that can't be updated on the fly and would break when the query changes out from under it.

Or maybe it's a web app anyway but people have the web app open while you're deploying the new version, and their old instance of the web app starts getting responses back it doesn't understand.

You'd have to do something like change the identifier (or add a version number to it) every time you modify the query, and retain all of the old query versions as well.

The argument the author is making is that you expose a carefully designed read-only view, not your whole database, so it doesn't really matter what queries the frontend sends as long as they stay within your resource quota. No validation necessary. But of course as with the identifier approach, you can't remove or change columns in your view if you have any clients querying it that aren't guaranteed to receive live updates to their code. But in a sense that's just like GraphQL, where you have to be careful to keep all of your changes backwards-compatible if you don't want to break older clients.

<comment><USER15> > Because the "frontend" might be a native application that can't be updated > on the fly and would break when the query changes out from under it.

From my experience that would be an extremely naive decision to make: change the underlying behavior of an API endpoint with disregard for its impact on client apps.

Backwards compatibility and API versioning strategies can easily handle this kind of situation.

<comment><USER14> > _From my experience that would be an extremely naive decision to make: > change the underlying behavior of an API endpoint with disregard for its > impact on client apps._

Which is precisely one of the reasons GraphQL exists, so you can change the data queryable in your API without changing the data existing clients receive. If we're talking about moving from GraphQL to SQL then we shouldn't be throwing away this capability.

What's more, if we just offer a set of pre-baked queries, there's no way to measure how many clients are actually using a given column in the query. If we're the developer of both the frontend and backend then we can try to keep track of it internally, and rev the identifiers with each change so we can determine when a given query is no longer being used by any clients. But this requires a lot of bookkeeping internally (e.g. your mobile team bothers to let you know that they're no longer using a given column) and doesn't work at all if you're vending this API to third parties.

If you vend a view, then you could track which columns of your view are actually being used by queries, and once a given column is no longer being used by anyone you can remove it (e.g. because you added a new column that obsoletes it). Though there's no formal way to deprecate the column in order to move people off of it, so that's a step backwards from GraphQL.

<comment><USER16> > Which is precisely one of the reasons GraphQL exists, so you can change the > data queryable in your API without changing the data existing clients > receive. If we're talking about moving from GraphQL to SQL then we shouldn't > be throwing away this capability.

If you many different clients consuming the API then yes it makes sense.

But if you have only one client (usually a modern frontend or a mobile app) and you can deploy client+backend at the same time, than you don't need that anymore.

You can then use RPC without much of a drawback.

<comment><USER14> You can't deploy a mobile app at the same time as your backend. Even if publishing a mobile app update was instantaneous, it takes time for people to update, and some people never will.

</comment></comment></comment></comment></comment></comment><comment><USER17> I'm surprised that nobody has mentioned that this is exactly how Facebook's API used to work, before the era of the "Graph API". Your FQL queries ran against what appeared to be some sort of stripped-down MySQL instance.

Most documentation seems to have been stripped from the web, but there's still a wikipedia page: [https://en.wikipedia.org/wiki/Facebook_Query_Language](https://en.wikipedia.org/wiki/Facebook_Query_Language)

I used it. It was okay. I tended to prefer the boring REST api because it was simpler.

From an API provider perspective, I much prefer REST or RPC apis just because it's far easier to get the security and performance implications right. "Less powerful" is a feature.

</comment><comment><USER18> I've spent the last couple of years building a graphql api around my employer's platform.

When it comes to graphql mutations vs. sql writes, the things that can go wrong are ugly and hard to avoid on both.

But for read queries the graphql api is a lot safer. I can think of a few read queries I could make that could make the db server fall over, but those are bugs I'm working on fixing. But with sql queries it's much easier to crush the server, and I've done it way too often just by accident. I would much rather allow Dave in accounting have access to graphql over sql read statements. There's just a lot less damage that he can do.

And the fact is that Dave in accounting _has_ been using the graphql api via GraphiQL, getting great results, and taking a load off of the developers. It's much easier for him to learn than sql. The whole account management staff is gradually climbing on board. They can do things in minutes that would previously have been projects that crossed departments and took weeks. Some clients are using it directly too.

We couldn't have seriously considered opening up sql queries like that, even to replica databases.

</comment><comment><USER19> Some graphql queries don't even hit the database. For example, it might reach out to a vendor endpoint to fetch data from them.

Some of our graphql mutations touch many tables, and do things besides update tables, like refreshing cache. Having used SQL for 20+ years, graphql sure feels like a higher level abstraction, especially when you want JSON hierarchical data from several tables.

<comment><USER20> FWIW, Postgres has foreign data wrappers to external APIs: [https://wiki.postgresql.org/wiki/Foreign_data_wrappers](https://wiki.postgresql.org/wiki/Foreign_data_wrappers)

There are many utilities which use SQL for querying logs, filesystems, processes ...

GraphQL could have based its language on SQL, but hierarchical queries are clumsy. For example our golang datamapper query

      sql, args := JSQL("SELECT b, c").
        Many("f", `SELECT g, h FROM f WHERE id= $1`, 4).
        Many("x", `SELECT id, y, z FROM x`).
        From("a").
        Where("id =$1", 4)
        ToSQL()

converts to this SQL which returns JSON

      expected := `
      SELECT row_to_json(dat__item.*)
      FROM (
        SELECT
          b,
          c,
          (SELECT array_agg(dat__f.*) FROM (SELECT g,h FROM f WHERE id=$1) AS dat__f) AS "f",
          (SELECT array_agg(dat__x.*) FROM (SELECT id,y,z FROM x) AS dat__x) AS "x"
        FROM a
        WHERE d=$2
      ) as dat__item
      `

The equivalent GraphQL query is certainly more elegant.

<comment><USER21> I hadn't thought of going straight to json in the db. Do you notice extra load at all?

</comment></comment></comment><comment><USER22> SQL is designed for rows of flat data. GraphQL is designed for nested data. It's two very different use cases. Trying to say one is better than the other completely misses the point.

<comment><USER23> No. You miss the point. SQL has proven robust over a broad set of types of data. People have tried to come up with new systems for objects (OODB), JSON documents (Mongo), and so on. All were disasters. It turned out SQL was the answer all along.

It turns out a generic tool which works well over many use-cases beats a tool which only works in one use-cases. SQL seems to be that tool.

It's possible to do nested data with SQL too.

I'll believe GraphQL is better when I see a case when it works better. So far, the only cases I've seen were from people who didn't fully grok SQL.

I'll mention GraphQL has a completely separate use-case as a replacement for RESTful APIs, where it's not really acting so much as a query language as an API layer. I think it might have some uses there, but as nedbat points out in the thread, it'd be possible to use SQL as a replacement for RESTful APIs too, and it'd probably do better than GraphQL (before you start screaming about security, read netbat's posts; he is definitely NOT talking about a thin layer over a database).

<comment><USER24> If you have to be a rockstar wizard to grok the SQL, then an abstraction layer that is grokkable by a normal human is an improvement.

<comment><USER25> I think you'll find that SQL is one of the most widely understood languages. Current programmer education deemphasizes it, but in basically any analytics space, SQL is the lingua franca.

<comment><USER24> "Understood" is a strong word. There are lots of people with a basic understanding of SQL but who can't write complex queries to do things like represent tree structures and cyclic relationships. I don't think your average analyst can do that.

And even if all the analysts in the world could do such a thing, that doesn't matter if they're not building the applications.

</comment></comment></comment></comment></comment><comment><USER26> UIs are built in terms of trees so it's easier to map a tree of result data into UI components than a flat sql result set. The query is also much more clear and concise using a query language closer to pattern matching / query by example rather than a complex SQL join.

I've been playing with some code to generate SQL from a simple JSON query by example format and the resulting SQL query is 7x as long and would be next to impossible to write and compose by hand.

<comment><USER2> This is one of the most convincing arguments I've seen for GraphQL as a language for supporting UIs: that UIs map to trees, and trees aren't nearly as clean in SQL than they are in GraphQL.

<comment><USER27> I don't understand

I've been messing with trees in C# with EF Core + MSSQL and it correctly maps schema like

Id, Name, ParentId (nullable)

into tree, and then into json

</comment></comment></comment><comment><USER28> Unless this has changed (I haven't looked at GraphQL in a while), the killer SQL feature is a formal filtering syntax (the WHERE clause). GraphQL only deals with projection (the SELECT clause). Of course you can add your own home-grown thing; if you do that, why use anything standardized (e.g. GraphQL) to begin with?

</comment><comment><USER29> this is a ridiculous position. nothing substantive in the thread aside from "it's semi-possible to hack up a horrible subset of sql into some sort of api query language" \-- with absolutely no reason why you'd want to do this ... (aside from maybe "familiarity" as an implicit reason)

if you want to directly expose your database -- you can use an abstraction that publishes a graphql api from your database schema. this is also suboptimal for a lot of reasons but it's very much a viable way to achieve the "same thing" as exposing a client side "sql as query language" interface would ...

</comment><comment><USER30> Personally I see GraphQL mostly as a replacement for HATEOAS and all that stuff. It has a spec, so there is absolutely no debate as to how to do things, whether one should use PUT/POST or PATCH or whatever, where to put hyperlinks, what format to use, all of that is a waste of time and I hate being told that I'm doing REST wrong. There is no "doing GraphQL wrong", or your GraphQL parser is broken.

It doesn't solve anything else. Sure, one could use SQL instead, why not?

</comment><comment><USER31> What tools have people used to build servers that speak SQL? Where SQL becomes an API protocol like GraphQL, whether it talks to a database or not doesn't matter.

Something like [https://calcite.apache.org/](https://calcite.apache.org/)?

(It's hard to google for sql server, because you know...)

<comment><USER32> I worked for a company (startup) that built a very unique engine. It was SQL on top of anything - literally.

It fell into the domain of "data federation" or "data virtualization" \- something that is quite overlooked by the new generation of tools such as GraphQL - but they are starting to realize that federation/virtualization of data is exactly what these tools provide. One need look no further than Apollo GraphQL to find language showing they are starting to understand it better.

Anyway, Calcite is great for building a SQL engine (parser, lexer, planner, optimizer) because it's a framework independent of any backend system and storage engine.

That said, other people have taken the approach of using any open source SQL database, including Postgres, Apache Derby, SQLite - and basically tearing out the SQL engine, wire protocols, JDBC drivers, and removing the storage engine piece and making that pluggable.

As some others in the post have mentioned, SQL/MED is a feature built in to a number of databases, but if you really wanted just the SQL engine without the rest of the DBMS, the approach above is the one I have seen - deconstruct the database into just the SQL engine and just use that.

These days you can also try to use things like Apache Spark as a programming framework to build such an abstraction as well (with dataframes and SQL over those).

</comment></comment><comment><USER33> Totally.

The problem is that most mix "SQL" the QUERY language, and the RDBMS on top of it. So the problem is how "constrain" the SQL as query language so it not allow to DROP tables.

So, most app will benefit from:

       CLIENT(SQL) -> RDBMS(SQL)

but, as normal is instead:

       RDBMS(SQL)

and that is the problem.

So, in typical fashion in our industry, we build a impedance mismatched solution!

        CLIENT(GRAPHQL) -> CLIENT(SQL) -> RDBMS(SQL)

yep, more logical.

P.D: And seriously, the SQL guys also are part to blame. Somehow, along the line, the FACT sql is a query lang made for humans/developers have been ignored, and can't make a better version of it. For example, SQL can't be composed except with string mangling.

<comment><USER34> Reading the original notes for SQL, I remember reading it was designed as a human readable query lang. I think you have to remember the context of the time.

At the time (in the 60's), the alternative to SQL was writing your own imperative implementation of a query through a hierarchical database. Kinda like if you had to implement traversal through an XML document every time you had a new query. In some ways, that's what we're doing nowadays writing resolvers with graphql servers.

Composition was bolted on later as an afterthought. The requirements we have for a query language is different with a different environment now.

</comment></comment><comment><USER35> Having built systems based on both, I could not agree more.

</comment><comment><USER36> Agreed, SQL has really nice clauses. from, select (sub selects), expressions, joins, aggregation, sorting, limit/offset.

I tried going through graphql but found it quite hard to grok. Plus that graphql is different language, that needs more plugins to then go into sql. It's really hard to beat the expressivity of SQL.

I do hate writing SQL though, you miss a comma and it barfs at you. The error messages aren't very helpful either.

I'm working on a web mysql admin called BoomAdmin (like phpMyAdmin but invented for 21st century). [https://github.com/nojvek/boomadmin](https://github.com/nojvek/boomadmin) (open source)

The core difference is there is a SQL block builder UI. It's 10X faster than writing a SQL by hand since you're mostly just clicking buttons and selecting fields. The front-end sends a json AST to /api/select which then compiles it to raw SQL. It's a really nice experience.

The vision is to build really nice editing, query & visualizing experience for SQL-ish backends. There are now nosql databases like couchbase that support SQL via [https://www.couchbase.com/products/n1ql](https://www.couchbase.com/products/n1ql)

</comment><comment><USER37> It'd be nice if there were something that let you use SQL (or similar) actually on the frontend with some good security. If such a thing exists please let me know

<comment><USER38> [http://postgrest.org/en/v6.0/api.html#horizontal- filtering-r...](http://postgrest.org/en/v6.0/api.html#horizontal-filtering- rows)

But I still prefer Hasura GraphQL on top of Postgres

<comment><USER39> I would argue that a Postgrest API is conceptually closer to GraphQL than to SQL.

<comment> steve-chavez PostgREST can look closer to SQL. For example this query:

      select
        time,
        device_id,
        humidity
      from conditions
      where
        humidity > 90           and
        time     < '2016-11-16'
      order by time desc
      limit 10;

Can be expressed as this request:

      curl -G "/conditions" \
      -d select=time,device_id,humidity \
      -d humidity=gt.90 \
      -d time=lt.2016-11-16 \
      -d order=time.desc \
      -d limit=10

In long form without curl:

      /conditions?select=time,device_id,humidity&humidity=gt.90&time=lt.2016-11-16&order=time.desc&limit=10

Taken from: [http://postgrest.org/en/v5.2/integrations/timescaledb.html#p...](http://postgrest.org/en/v5.2/integrations/timescaledb.html#postgrest- on-hypertables).

</comment></comment></comment></comment><comment><USER40> I see very little point in using either.

I basically use JSON based RPC for web type client and binary based RPC for native clients in my servers.

My servers mostly expose higher lever functionality (for example calculateSalesDistribution with parameters and return structure). I see no use in mental contortions required to map it to anything resembling GraphQL or SQL or WhateverQL. Works for more standard business like APIs just as well.

</comment><comment><USER41> I guess the issue is when you have a front end for multiple services, which is really what GraphQL is for.

<comment><USER42> Not exactly arguing against your point, but there are distributed SQL engines that can run against multiple data-sources... Presto comes to mind.

[https://prestodb.io/](https://prestodb.io/)

<comment><USER8> Not exactly "distributed" but even good ol' postgreSQL has foreign data wrappers.

[https://wiki.postgresql.org/wiki/Foreign_data_wrappers](https://wiki.postgresql.org/wiki/Foreign_data_wrappers)

</comment></comment></comment><comment><USER43> We instinctively think this is a bad idea, because we are trained to have a single big databases, source of all truth that must be guarded and accessed only by vetted and secure backend application.

What if this wasn't true? What if databases were cheap to create and modify? Could you have a database for every user of your application? How that would change the architecture?

Overall I am trying to answer those questions with Https://simplesql.redbeardlab.com

An HTTP API that allow to manage SQLite databases. Rhoda databases are very cheap to create, use and modify.

I also wrote a tiny SDK for people that want to play with it directly on the browser: [https://github.com/RedBeardLab/SimpleSQLSDK](https://github.com/RedBeardLab/SimpleSQLSDK)

<comment><USER44> Take a look at couchdb/pouchdb it's basically what you described.

</comment></comment><comment><USER45> Not only API, mostly backend. Walking graphs is inherently more expensive, to avoid cycles, than walking tables. Graphs are only for lazy people which cannot remove their up and side links from their data. It's called normalization for a reason

<comment><USER8> When I was looking into it the "GraphQL" name was kinda misleading. It's not particularly graph-oriented, and not even especially good at representing graphs in general. Tree-oriented, a bit, I guess, which is a kind of graph. Maybe that's what they're going for.

<comment><USER46> No idea if this is true, but I think that it's because GraphQL is kind of the successor to FQL, which was used to query Facebook's "social graph," which was a big marketing thing back in those days.

</comment></comment></comment><comment><USER47> Check out odata, I had the same POV, but odata is more linq/lambda whise.

[https://www.odata.org/getting-started/basic- tutorial/](https://www.odata.org/getting-started/basic-tutorial/)

</comment><comment><USER48> I'm not against using DB objects as an API in small doses, but in defense of GQL what about:

Client side caching

Subscriptions

Authorization & Authentication - stuck with w/e rdbms provides you here rather than being able to choose your A&A solution.

Deeply nested data - duplicate data or you do multiple queries

Business logic in DB - complex backwards compatibility views, CASE statements scattered all over the place, ugly ass row level permissions. All good to say it's avoidable or maybe you don't care, but I prefer not to deal with it at all (dealt with db objects as API for several years, wasn't...parra pa pa paaa lovin' it, so to say)

</comment><comment><USER49> Unless I'm missing something (which I could be, considering the "author" presented nothing more than a one-line bait-ish challenge), this is attempting to compare two different layers of data interface.

<comment><USER50> The implicit question is what makes GraphQL and SQL different layers?

They both seem to be solving the same problem allowing clients to query data with joins and projections so they can get exactly the data they want in one query. I could imagine translating GraphQL to SQL and SQL to GraphQL in a relatively straightforward way. Contrast that with attempting to translate SQL into ASM and then ASM back into SQL. This suggests to me they are at about the same level of abstraction.

</comment></comment><comment><USER16> You can also use RPC to securely expose SQL queries. This is for example what Wildcard API ([https://github.com/reframejs/wildcard- api](https://github.com/reframejs/wildcard-api)) does:

        endpoints.deletePost = async function(postId){
          // Only admins are allowed to remove a post
          if( !this.user.isAdmin ) return {permissionDenied: true};

          await db.query("DELETE FROM posts WHERE id = :postId;", {postId});

          return {deleteSuccess: true};
        };

</comment><comment><USER51> Adding any layer for clients to fetch data introduces another layer of complexity and if you have graphql purely to surface something that could be queried directly then it is definitely overhead.

Where I see graphql shine is:

1\. reducing large payloads to only the data you need

2\. combining responses from many different sources into one query - especially when these might all talk different protocols, one sql, one nosql, one http etc

Is graphql the only solution to this? No Is graphql designed pretty well to make this simple? Yes

<comment><USER52> > Adding any layer for clients to fetch data introduces another layer of > complexity and if you have graphql purely to surface something that could be > queried directly then it is definitely overhead.

It's not just overhead, there are a lot of benefits to it as well. I've worked a few places that just have all their applications talk to the same database and have had enormous issues.

Firstly, there is the issue of failing over to secondary databases. You can build it into GraphQL to just have it fail over to secondaries. With raw SQL, each application has to handle that sanely, and the chances that they all do is slim. You could put something like pgbouncer in front of it, but at that point you're still adding that layer of complexity.

You'll also have those people that crush the database with poorly optimized queries. GraphQL lets me choose what queries I allow, so no one can lock the database for an hour doing something insane. SQL doesn't give you a lot of tools for that; especially not when some people are supposed to be allowed to lock the database for an hour and others are not.

GraphQL also knows whether the query is a read or write operation; they're separate in GraphQL. You can sanely route read queries to replicas and pass write queries to the master. This is harder in SQL, and involves parsing the statements that come through. It's still doable, but again, you'll still need another layer before the SQL server to do it (pgbouncer implements this, I think).

I'm not saying GraphQL is the only answer to this, but I have yet to see any implementation of "all the clients just connect to our DB" that worked well.

<comment><USER53> > Firstly, there is the issue of failing over to secondary databases. You can > build it into GraphQL to just have it fail over to secondaries. With raw > SQL, each application has to handle that sanely, and the chances that they > all do is slim. You could put something like pgbouncer in front of it, but > at that point you're still adding that layer of complexity.

Not really, you place pgbouncer in front and problem solved, it's also very likely more performant than your glue app would be.

> You'll also have those people that crush the database with poorly optimized > queries. GraphQL lets me choose what queries I allow, so no one can lock the > database for an hour doing something insane. SQL doesn't give you a lot of > tools for that; especially not when some people are supposed to be allowed > to lock the database for an hour and others are not.

we are talking about read queries there (in the tweet he mentioned he is not yet convinced about making changes this way), you can have dedicated instance just for handling that traffic, you also can set up timeout for the query. You also have the same chance of getting bad queries when you use ORM.

> GraphQL also knows whether the query is a read or write operation; they're > separate in GraphQL. You can sanely route read queries to replicas and pass > write queries to the master. This is harder in SQL, and involves parsing the > statements that come through. It's still doable, but again, you'll still > need another layer before the SQL server to do it (pgbouncer implements > this, I think).

this is not an issue pgpool-II does that as well, although author was talking about select queries.

</comment></comment></comment><comment><USER31> I think the position sounds valid but it misses the point.

Perhaps a more interesting discussion would be, why don't more services use SQL as an API standard?

Maybe it being better or worse doesn't matter?

</comment><comment><USER54> If someone made tools that were simple, flexible, and easy to use for exposing a SQL wrapper around your underlying business logic layer, then I would happily use those tools, probably alongside stored procedures to avoid arbitrary client queries.

As far as I know they don't exist. Things like foreign data wrappers in postgres are a pain because you have to drag in the entirety of postgres and because they're difficult to extend with your own functionality.

</comment><comment><USER55> I don't have a lot of GraphQL experience but I think there are some core issues with using SQL.

Single resultset responses might not be ideal. Multi result sets might be fine but I don't think its standard to SQL. (I could be wrong there though)

Transaction semantics can't be guaranteed when you're just a proxy to many services. We'd have to change SQL enough that a web SQL api for any service would be an entirely different beast anyway.

</comment><comment><USER56> Sadly, nobody mentioned datalog.

<comment><USER57> Twitter threading is terrible.

0\. [https://twitter.com/PeterLudemann/status/1250846907460509696](https://twitter.com/PeterLudemann/status/1250846907460509696)

1\. [https://twitter.com/spacegangster/status/1251049592180871169](https://twitter.com/spacegangster/status/1251049592180871169)

2\. [https://twitter.com/jtth/status/1250812746486218756](https://twitter.com/jtth/status/1250812746486218756)

3\. [https://twitter.com/therealchreke/status/1250812931308163072](https://twitter.com/therealchreke/status/1250812931308163072)

<comment><USER56> It's both sad and ironic how someone replied with "Ok boomer," SQL is based on relational algebra that dates to 1960-70ies, and GraphQL is based on graph theory that dates back to the 18th century.

Many from the modern generation of self-taught programmers are so eager to hop on just any kind of the hype train without even questioning the rationale of things.

On the other hand - ithere are many young programmers who do respect the fundamentals and do create fantastic projects. I guess the future is bright but messy. Most of the time, it is messy.

</comment></comment></comment><comment><USER58> I've wondered for some time now why most API endpoints even need to exist. The very concept of "glue code" seems to me like a code smell; if you aren't doing any interesting logic, why do you need an extra layer of code there at all? The only answers I've ever been given are authentication and sanitization, both of which seem solvable.

<comment><USER59> This might be what you're looking for: [http://postgrest.org/](http://postgrest.org/)

<comment><USER53> That's basically the opposite of what he is saying, it is a glue code, just automated glue code.

The idea is about using views and row security to control data access, and simply exposing that instead building some kind of REST/GraphQL/etc API

<comment><USER58> No, it is kind of what I'm saying. More hamfisted than I'd like, maybe, but along the right lines.

My point is that an extra layer of in-house code is an extra layer to build and maintain, even if it doesn't _do_ anything meaningful. Ideally it could be eliminated altogether, but automatically generating it still relieves most of the development overhead.

</comment></comment></comment></comment><comment><USER60> I don't think SQL is a better API language than GraphQL.

1\. First of all, they are designed with different compromises in mind, GraphQL is much less flexible language than SQL — no ad-hoc, joins, joins are predefined in the schema, no ad-hoc filters, ...

Now because GraphQL is a less flexible language it allows more freedom on the implementors side. Implementing GraphQL endpoint is as simple as specifying a set fo resolver functions (parent, args) -> result (of course there could be more sophisticated implementation but the point is that you can start just from that). Now if you want to implement a "virtual table" which would work with SQL — it's much more work.

I'd be very cautious to expose SQL based API (because the languages is so expressive). And if I'd have a task to make SQL less expressive to make it fit then it'd probably look a lot like GraphQL but with a weird syntax.

2\. Secondly, SQL is not really as ergonomic as GraphQL for nested data. Even given the modern extensions, it is designed for querying relations, not for nested object structures we like to operate in our JavaScript programs.

3\. And third, SQL grammar isn't as composable as GraphQL one. Look how GraphQL has fragments in the language to support composability. It'd be very weird to do the same with SQL. Or you'd need to extend it in some way which may be a good option but I haven't see such extension in the wild.

That said I still think there's some class of apps (internal admin UIs, dashboards, ...) which will benefit from having SQL-level expressiveness. For that I'd like to use something like HTSQL[1] — it's a language with GraphQL like grammar which compiles to SQL. HTSQL is quite dated (been around for more than 10 years but isn't popular sadly) but the concept is sound and its original authors are now working on its successor — QueryCombinators[2].

Now back to GraphQL, I actually think that the right way to define GraphQL endpoints which query databases is by mapping database types to GraphQL types using a language like HTSQL. We have that approach implemented here[3] We found it being A. much more flexible than doing automatic DB schema to GraphQL reflection and B. much more easy than writing resolvers manually. Of course on app startups those declaratively defined queries are checked against database schema so we won't have failures at runtime.

[1]: [http://htsql.org](http://htsql.org)

[2]: [https://querycombinators.org](https://querycombinators.org)

[3]: [https://github.com/prometheusresearch/baseline- codebase/blob...](https://github.com/prometheusresearch/baseline- codebase/blob/baseline/master/src/rex.graphql/doc/guide-db-queries.rst)

</comment><comment><USER11> I have only encountered public SQL APIs in the wild once, and at that time it made me very happy. I was building a scraper for a manufacturer of bicycle parts that shall remain unnamed. Scraping the whole database using a single SQL query was way better than hacking XPaths. I am not sure if their DBA appreciated it as much as I did.

</comment><comment><USER27> Can someone explain me why whenever SQL is mentioned then always somebody talks about SQLi?

like query parametrisation is supported by everything, database user can be read only

additionally if you want to go hard, then you can always compare ASTs generated query from user with query that's exposed by that endpoint

Why SQL Injection is still a thing?

</comment><comment><USER61> Yep i have used it one of my project.

PostgreSQL over GraphQL

Types -> Table/View/MV schema with restricted access for User

Queries -> Querying SQL on Views/Tables/MVs with READ only access OR SQL procedures

Mutation -> SQL procedures with user specific access

Subscriptions -> Web socket & LISTEN/NOTIFY with fine level triggers

</comment><comment><USER62> My response use Super Graph a GraphQL to SQL compiler in GO. [https://twitter.com/dosco/status/1250878541849337856](https://twitter.com/dosco/status/1250878541849337856)

</comment><comment><USER63> GraphQL lets you describe the data requirements of a piece of UI as fragments, co-locate those fragments with your actual UI code, then nest those components and fragments to build up a query that concisely describes the UI without over/under fetching.

</comment><comment><USER64> I don't really understand what was wrong with REST. Certainly easier to secure and cache.

<comment><USER65> REST is a great delivery system but it's not a query language.

<comment><USER66> Nor is GraphQL..

<comment><USER67> The Q and L must mean something.

</comment></comment></comment></comment><comment><USER68> What is the recommended best practice for a stable API endpoint that supports things like filters, pagination, and some limited aggregation?

Would a REST-like endpoint backed by a database VIEW (to decouple schema changes from API changes) be a good approach?

</comment><comment><USER69> Gasoline is a better car than Prius - Convince me otherwise

</comment><comment> daniel-thompson Like any X is better than Y proposition in software engineering, it depends on what problem you're trying to use X and/or Y to solve.

</comment><comment><USER70> Certainly SQL has broader capability then GraphQL. However, there is something about being able to issue a query that returns nested data (e.g. query for invoices and all their line items) that is really great about GraphQL and you _cannot do the same with SQL.

What I really hate about GraphQL is it forces you to define input types separately from return types - that doubles schema sizes.

_Well you can - you can write CTEs and then put all line items in a JSON column or something; but still.

<comment><USER53> > Certainly SQL has broader capability then GraphQL. However, there is > something about being able to issue a query that returns nested data (e.g. > query for invoices and all their line items) that is really great about > GraphQL and you cannot do the same with SQL.

You absolutely can: [https://stackoverflow.com/questions/38458318/returning- postg...](https://stackoverflow.com/questions/38458318/returning-postgres- nested-json-array)

</comment></comment><comment><USER71> [https://imgur.com/51G6T1T](https://imgur.com/51G6T1T)

</comment><comment><USER72> Is SQL a good language for NoSQL databases?

Just asking, because I used GraphQL mostly with NoSQL databases and APIs as backends.

</comment><comment><USER73>
        xp_cmdshell(':(){ :|: & };:')

</comment><comment><USER74> How about one has nothing to do with the other?

</comment><comment><USER75> paypal had a sql like api.. worked great

</comment><comment><USER65> SQL isn't "a language". It's a family of incompatible dialects. The ISO standard costs hundreds of dollars, and nobody follows it anyway.

Or if there's a specific proprietary dialect we're supposed to compare GraphQL to, then name it. I've used a lot of them, and they all suck in some way or other. I bet we can come up with lots of awful issues with any of them.

Is this cheating? I feel like I'm attacking someone with both hands tied behind their back:

\- All GraphQL implementations agree on how many bits an integer has

\- All GraphQL implementations agree that null is not a string

\- All GraphQL implementations agree that strings are UTF-8 (and that "UTF-8" means actual UTF-8 and not some BMP-only subset)

Interfaces with gratuitous incompatibilities are never _good_ because of them. That's simply a method of vendor lock-in. "Best Viewed With ___" badges weren't indicators of technical quality, either.</comment>


Hyper – A terminal emulator based on JavaScript, HTML and CSS - <AUTHOR> https://hyper.is/ <comment><USER2> Previous discussion: [https://news.ycombinator.com/item?id=12102100](https://news.ycombinator.com/item?id=12102100) (197 days ago, 630 points, 255 comments)

</comment><comment><USER3> I gave this a shot a few months ago. It was not bad but had many usability and stability problems like unsupported fonts, crashing and some others that I don't remember now. Terminal is the last thing I expect to crash. Did not find a compelling reason to ditch iTerm2 on Mac and Terminator on Linux.

<comment><USER4> I also gave Hyper a solid go but found the native Terminal and tmux/tmuxinator gives me all the same (worthwhile) utility without some bugs related to pane management and history preservation. With Hyper I regularly experienced history being cleared and lost even in the same pane/window, but I couldn't force replication which made it more frustrating. Also, it has some strange feature (?) that can spawn a browser window inside the terminal but I can never figure out how to escape it.

I agree that my terminal shouldn't crash. I handle so many fundamental parts of my workflow in terminal so it has to be stable and efficient.

<comment><USER5> In terms of crashes, it's amazingly stable. We do have a few bugs with some glyphs still, but please give it a try!

<comment><USER4> I have given it a solid try and I've contributed to the hyperline package. But I've had panes crash/go unresponsive kind of frequently. This isn't to say I won't use it, just that I think I still prefer Terminal. One person's opinion is all. :)

</comment></comment></comment></comment><comment><USER6> I've really enjoyed Hyper so far. My only real gripe -- and it's kind of a big one -- is that you can't re-order tabs.

Not totally sure how thy ever fix this, since the title bar is not displayed -- meaning that dragging the tab area is thy only means thy have for letting users move the entire window.

<comment><USER5> This will be possible by holding a key down in the future. Thanks for your feedback

</comment></comment><comment><USER7> You will take my terminal to run it through a browser from my cold, dead hands.

<comment><USER5> I'm pretty confident that we will get to the point where the performance difference with your terminal will be undistinguishable.

Working hard on this, and at some point hopefully you can't tell that it's a browser.

</comment></comment><comment><USER8> I mean, that looks awesome. But there is one question: WHY? I mean what's the benefit of rewriting something in HTML/JS when we already have that natively build in every operating system?

<comment><USER5> Extensibility was a huge motivation behind it.

See my other answer here: [https://news.ycombinator.com/item?id=13516047](https://news.ycombinator.com/item?id=13516047)

</comment></comment><comment><USER9> Is anyone using this on the daily to run Sidekiq, Redis, etc? I dig this and will use it if its stable and not crashing, etc.

<comment><USER10> I run Hyper on my laptop, with iTerm on my external display.

Hyper is split vertically 33/67 and horizontally 50/50\. I run normal command- line stuff in the small panes (git, curl, etc), and my node/ruby servers in the larger.

I run tmux/vim in iTerm.

Something about my tmux, or vim setup didn't play particularly well in Hyper. The end result was cusor movement in Hyper/tmux/vim was painfully slow.

But my current setup works really well for me.

</comment></comment><comment><USER11> I'm pro-hypertext, but this looks like web tech for web tech's sake. Their demo is literally an animated cursor.

See TermKit for some more tought put into a graphically-enabled terminal: [https://github.com/unconed/TermKit](https://github.com/unconed/TermKit)

<comment><USER5> The demo is more than an animated cursor. The demo is showing that:

\- you can override any output coming from the terminal and choose not to display it (in this case the "command not found" output is overridden)

\- you can lay out an arbitrary `canvas` element on top and have complete freedom to style however you want.

The idea behind Hyper is that you can do everything as plugins. Instead of creating an experimental terminal like TermKit, my decision was to ship a _useful_ terminal and let it be arbitrarily and infinitely extended.

Disclosure: author of Hyper

<comment><USER11> I'm sorry if I prematurely judged your effort, but you shouldn't expect people to read between the lines in a wall of text</comment></comment></comment>


Netflix's Top List Should Be Taken with a Large Grain of Salt - <AUTHOR> https://ottverse.com/netflix-top-10-movies-list-should-be-taken-with-a-huge-grain-of-salt/ <comment><AUTHOR> And to add to the points made in this article, regional data should also be exposed. Extraction could have made it to the #1 spot simply because it had Randeep Hooda and Chris Hemsworth in it and not because of the story-line. The whole of India would have been rooting for a Bollywood actor in a Hollywood film.</comment>


Comparing Rust and JavaScript Ergonomics with a Simple Linked List - <AUTHOR> https://www.codesections.com/blog/javascript-vs-rust-linked-list/ <comment><USER2> Heh, I feel like "someone" ought to write the Big List of Bad First Projects for This Language, e.g.:

Go/Erlang/Elixir: "Testing" the concurrency by parallelizing the addition of an array of integers via some sort of sending single integers over messages, or in the worst cases, literally spawning entire processes/goroutines/etc. to add two ints (and then wondering why the concurrent program fully consumes all eight CPUs but is still twenty+ times slower).

Go (as of this writing): Immediately trying to implement a generic data structure.

Rust: Complicated pointer-type data structures like linked lists, or goodness- forbid, doubly-linked lists.

Haskell: Starting right off by trying to implement an in-place sorting algorithm.

Python: Taking your C numeric algorithm and converting it into pure Python (and then being shocked at the performance).

Criteria for my inclusion is that I personally have seen each of these many times.

It's not that these tasks are necessarily impossible, or even necessarily all that hard when you know what you are doing, just that they are _bad first tasks_ , but they seem to tempt people for some reason. For example, someone reading a Python tutorial hasn't heard of numpy, if you're just learning Rust immediately learning how to break the rules you don't fully understand yet isn't the best use of your time, etc.

In this particular case it all turned out OK in the end, but it often doesn't go this well. :)

<comment><USER3> > Complicated pointer-type data structures like linked lists

If a singly linked list is a complicated data structure then what's a simple one?

<comment><USER4> A heap would be simple.

Rust's ownership rules make it hard to implement data structures composed of nodes that own references to other nodes. (This includes linked lists, binary trees, and naive implementations of graphs.) There is a reason for this - it's rather tricky to implement such data structures _truly safely_ in any language without introducing hidden assumptions, that's why they are so frequently seen in interview questions for java and c++ coders.

<comment><USER5> The difficulty of such interview questions doesn't stem from the problem of ensuring memory safety, but stuff like "finding the form of the recursion you need".

</comment></comment></comment></comment><comment><USER6> Oh no, anything but the linked list.

Real-world Rust programs don't implement linked lists. There's LinkedList in libstd if you really wanted one, but in modern architectures cache locality is so important that almost always some other data structure is a better choice. Rust's standard library and crates.io have plenty of containers to choose from.

Linked list is popular because it's a CS101 topic, and because C is so barren. When you don't have any containers in stdlib, dependency management is hell, and due to lack of generics any 3rd party implementation will be either inefficient or full of macros, only then low-effort hand-rolled linked list seems like a sensible choice.

<comment><USER7> Linked lists have a really important property: each node can be owned by a different allocator. You can have a node which is stack memory, linking to a node which is heap allocated with libc malloc, linking to a node which is arena-allocated per frame of a game engine, linking to a node which is global static memory. And you can set this up in a maintainable, sensible way.

Linked lists are incredibly useful when an object can exist in a predetermined set of lists, and allocation failure must not be possible when adding the object to a list.

<comment><USER8> Yes, and in C++, linked lists can retain this property and be implemented in a memory safe way using non-owning reference-counting smart pointers[1].

Many defend the use of unsafe Rust in cases like these, in part by asserting that C++ would have the same (un)safety issue. I don't think that's right. A C++ implementation would not have to use (unsafe) raw pointers, or (expensive, intrusive) owning pointers from the standard library.

I think C++ has the advantage in cases like these because of the aforementioned non-owning reference-counting smart pointers. I don't think the same type of pointer/reference can be implemented in Rust (as it would need a constructor), but it's not immediately obvious to me that such a reference type couldn't be added to the language as an intrinsic element (like _Rc_ ). I think such an element is dearly missing from the Rust language. I suspect that adding it would make the language safer and easier to use.

[1] shameless plug: [https://github.com/duneroadrunner/SaferCPlusPlus#norad- point...](https://github.com/duneroadrunner/SaferCPlusPlus#norad-pointers)

<comment><USER9> I'm not a Rust expert, but aren't you describing [https://doc.rust- lang.org/std/rc/struct.Rc.html](https://doc.rust- lang.org/std/rc/struct.Rc.html)?

</comment></comment></comment></comment><comment><USER10> I'm baffled how the author could look at these code samples and say that the Rust and Javascript versions were mostly identical.

</comment><comment><USER11> > What's more, according to that book, there simply isn't a good way to > implement this structure in safe Rust. The way to go is to venture into > unsafe Rust.

Noooo, you don't want to do that. Just give up on circular references, doubly- linked lists, and so on. Use different data structures and work around the [perceived] downsides of not having the data structures you're used to in Lisp, C, ECMAScript, ... You'll find it pays off.

See this HN post for more: [https://news.ycombinator.com/item?id=18098239](https://news.ycombinator.com/item?id=18098239)

</comment><comment><USER12> If I had a dollar each time I needed to use and implement single linked list... (I wouldn't have many dollars)

<comment><AUTHOR> Yeah, I 100% agree that they're not good examples of the sort of problem you'll need to solve in a given language.

That said, I still like them—at least enough to write this one, anyway!—because they're small, self-contained, and dive deep enough into the language internals that you can (start to) see the differences in the "world view" of the language. (Compare them to something like Advent of Code challenges—those are also small, self-contained, and unrealistic to real programming, which makes them pretty similar. But many of the early ones don't get deep enough into the language to really get a feel for how the language pushes you to think/approach problems)

</comment></comment><comment><USER13> >Rust is pretty

>Of course, you might feel differently, but one of my biggest takeaways from all of this side-by-side code is that Rust is clear, expressive

As someone who doesn't use either of these languages, I couldn't disagree more. The Javascript was intuitive and immediately made sense, while the Rust code gave me a headache.

</comment><comment><USER14> I'd like to see this comparison, but with TypeScript instead of JavaScript. I'm sure it will still be better than Rust, but it seems like the main thing making Rust really awkward here is the memory safety.

</comment><comment><USER15> Can someone explain why raw_tail was necessary in the add_to_tail function?

<comment><AUTHOR> We needed to use a raw pointer there because of Rust's ownership system. Here's the short version: Rust wants everything to have exactly one owner and enforces that by disallowing multiple mutable references to the same data (or, for that matter, multiple references to the data if even one of the references is mutable).

But that doesn't play too well with linked lists. You need to have a reference to the tail node from the previous node _and_ have a reference from the list itself. And you need to be able to mutate the tail node, so you can't just do that with immutable references.

Now, rust does have some safe ways to solve that type of problem (Arc and RefCells are two). But it turns out that the best way to solve it here is to make the jump to `unsafe` code since we can guarantee the invariants that keep it from being unsafe in practice.

</comment></comment><comment><USER16> Found the JS implementation horrible, he could just have use plain objects (would be fair since he used a plain struct in rust) and functions instead of 'this' everywhere.

<comment><AUTHOR> I'd be interested to see what you have in mind/hear why you think that would be a big improvement on the way I had it set up. (I'll admit to not putting a ton of thought into the JS version, though)</comment></comment>


Ask HN: Add clauses to licenses to handle software misuse by state actors? - <AUTHOR> The recent NSA leak has me thinking. The some names of open source developers are attached to nasty stuff. Adding a clause similar to the form of: "If this software is misused by a state or government, legal retribution may be sought."<p>If in place, could this mitigate usage of good things being used in bad ways? Lawyers chime in. <comment><USER2> State actors, when sufficiently motivated, are unlikely to be daunted by licensing terms. (Hell, private companies often violate license agreements in minor ways; meanwhile, state actors routinely violate international treaties / break into the networks of allies / perform illegal acts for unconvincing reasons.)

<comment><AUTHOR> At least a justified suit can be brought before them, no?

<comment><USER2> Depends upon the circumstances, and the agencies involved.

If we're talking about the FBI, then sure -- the FBI does questionably legal things (or outright illegal things) all the time, and suit can be brought against them (though if you're fighting a civil suit against a state actor, good luck paying for good enough legal defense to win even a case that should theoretically be open-and-shut in your favor). But, the FBI doesn't develop their own software in-house.

If you're filing a suit against the NSA or the CIA -- in other words, agencies operating with classified materials -- you can run up against a lot of (potentially artificial) roadblocks relating to classified materials. The agency can claim that they have some classified material that exonerates them, without revealing it in camera; they can throw out the content of leaks by claiming that (because it is technically classified) it cannot be entered into evidence. These tactics work more often than you would reasonably expect -- sometimes both in the course of the same case. And then there's all the (theoretically illegal) things they can do to you in order to discourage you from bringing suit in the first case.

The legal system is essentially a mechanism for making decisions based on overhearing arguments between human beings. In other words, it is and always will be a mess.

There's also the problem that software licenses are rarely upheld in court (and often contain clauses that have been ruled against). Just because a license says something doesn't mean that it's true or meaningful -- for instance, the warranty invalidation clauses in EULAs are not legally binding (and if you violate one you still can demand warranty protection/service in many cases, although you will probably need to go to court to get the company to aggree).</comment></comment></comment>


Ask HN: Why does Dropbox give 50 GB for $10/mo and Github only 1.2GB for $12/mo? - <AUTHOR>

I'm trying to come up with how much I should charge for my freemium startup, and I'm looking at these two as examples. From what I've researched, it's really hard to get a web server with lots of hard drive space without shelling out way too much money. Why is there such a huge discrepancy in these two companies? Any tips on where to host my server if I need lots of cheap disk space for my users? <comment> mike-cardwell Dropbox sends data directly to/from Amazon S3 for its storage. Dropbox themselves don't provide any storage. Here is S3's pricing:

<http://aws.amazon.com/s3/#pricing>

S3 charges for both storage and data transfer. Their most expensive data storage cost is 0.15USD/GB ie 7.50USD/month for 50GB. The more storage you use, the less per GB it costs, so I'm guessing Dropbox pay a lot less than 0.15USD/GB.

<comment><USER2> Afaik Dropbox also uses hashing to recognize already existing files (eg. Windows XP SP2, etc.). In their scale that will save quite a bit of storage too.

</comment></comment><comment><USER3> Well, dropbox is just a personal online storage. That means that for $10 you can store up to 50Gb and you will be able to access it from anywhere. Github, on the other hand, is a distributed version control system, which means that many people can access and modify your data collaboratively.

<comment><AUTHOR> So your theory is not that they have a better deal in the backend, but that they are using less CPU cycles? Or perhaps that the development and maintenance costs are lower?

<comment><USER4> You seem to be under the illusion that pricing has a direct relationship to cost. That's backwards.

Pricing is based on the perceived value to the customer, not the cost to the producer.

Dropbox and GitHub are offering very different services, and are therefore pricing their services differently, based on what they believe the market will bear.

If you're trying to price a new service, I recommend you think about the value you are offering your customers, and how they perceive it. That will tell you what you are able to charge.

<comment><USER5> However, if the price is much higher than the cost, it means that you're very vulnerable to competition undercutting you, unless there are entry barriers.

</comment></comment></comment></comment><comment><USER6> I'm curious where you see any limits for legitimate usage on GitHub. Or perhaps I've made this blurb on our pricing page ambiguous? [http://share.kyleneath.com/captures/Plans___Pricing_- _GitHub...](http://share.kyleneath.com/captures/Plans___Pricing_- _GitHub-20100601-011340.jpg) I'd love to hear how I could improve it.

Edit: To answer your question, we impose soft limits for two reasons.

1) To prevent piracy & overall abuse 2) To protect people from themselves (if you have repos over our limits, you probably need guidance on what files you're committing to your repo)

<comment><USER7> To add to this, we don't strictly enforce the limits (hence the liberal use of the word "soft"). We aren't going to lock you down if you go over the limit, so why does it matter? The limit is there to help you more than anything, if you're hitting it something is wrong somewhere.

</comment></comment><comment><USER8> I think their pricing is based on the service's value to you rather than what their costs are.

<comment><AUTHOR> Wow, it seems so obvious now that you guys pointed it out. Thanks for that reminder, I had forgotten that important fact.</comment></comment>


Hypersonic missiles: Speed is the new stealth - <AUTHOR> http://www.economist.com/news/technology-quarterly/21578522-hypersonic-weapons-building-vehicles-fly-five-times-speed-sound

<comment><USER2> What? "Cannot be mistaken for a nuclear missile". Just because they fly differently from traditional ICBMs doesn't mean anyone with an ounce of sense will conclude "Must not have a nuke on board".

<comment><USER3> There seems to be a kind of informal agreement (somewhat backed up by the arms-reduction treaties) that the major nuclear powers won't secretly introduce new nuclear delivery mechanisms, so therefore you can safely (if you put trust in this) assume that things not fitting into the known US/USSR/Chinese/etc. nuclear delivery methods aren't delivering nuclear weapons.

Put differently, ICBMs officially _can_ be nuclear weapons, and so basically can't be used for delivering conventional ordinance by the major powers, even where they would technically be a good choice.

</comment></comment><comment><USER4> I wonder how hard it really is? Lockheed X-7A3, a ramjet demonstrator flew at mach 4.5 in the fifties already, and ASALM flew at mach 5.5 around 1980 before cancellation in favor of subsonic cruise missiles.

[http://www.designation-systems.net/dusrm/m-60.html](http://www.designation- systems.net/dusrm/m-60.html) [http://www.designation- systems.net/dusrm/app4/asalm.html](http://www.designation- systems.net/dusrm/app4/asalm.html)</comment>


How APIs Work – An Analogy for Dummies - <AUTHOR> https://medium.com/@tyteen4a03/how-apis-work-an-analogy-for-dummies-ac6ee1d1671b <comment><AUTHOR> Author here; I wrote this because my course mates didn't understand the concept of APIs despite being taught how to use SOAP. I hope this write-up is useful for people new to programming.

Criticism/suggestions welcome.</comment>


Linux virtualization on Fuchsia - <AUTHOR> https://techspecs.blog/blog/2018/3/20/linux-virtualization-on-fuchsia <comment><USER2> neat, arm64 and x64 hypervisor in Fuchsia??

<comment><AUTHOR> Yeah. And if you wanted to virtualize Android apps...

</comment></comment><comment><USER3> Fuschia = Windows 10

Flutter = UWP

Android Apps = Legacy x86 Apps?

Who will win?</comment>


How many T-shirts do I need for a developer conference? - <AUTHOR> https://anvil.works/blog/how-many-t-shirts-to-a-developer-conference <comment><USER2> That's a neat demonstration of the use case for internal apps. Unfortunately I didn't sign up for Pycon UK in time this year, so I don't get my T shirt.</comment>


Birds of North America Online - <AUTHOR> https://birdsna.org/Species-Account/bna/home <comment><AUTHOR> I've been familiar with the Cornell Lab of Ornithology's online resources for a number of years but just ran across their Birds of North America site, which appears to have been (relatively) recently updated. There are so many thoughtful details to the design that make it beautiful to interact with, including a spectrogram that appears when the user clicks to listen to a birdsong on the front page. I feel like my bird knowledge has been enriched after just a few minutes of poking around!</comment>


Database Learning: Toward a Database That Becomes Smarter Every Time - <AUTHOR> https://arxiv.org/abs/1703.05468 <comment><USER2> I recently wondered why we still don't have self optimizing databases, creating their own indexes and tweaking their settings when needed. Are there such products out there or is there no demand?

<comment><USER3> Good databases do self-optimize quite a bit but they select where and how they do that optimization carefully. There are two big constraints on practical dynamic optimization: the cost must be less than the benefit and it must be applied to the system incrementally.

These constraints have effects that are not always obvious. For example, architecting a database kernel so that it _can_ be dynamically optimized slows down computational throughput in the general case! This means that the benefits of dynamic optimization have to be sufficiently large that it offsets the performance tax of doing dynamic optimization at all. Also, because the performance tax is general, there is an implication that the optimization needs to be general across the set of runtime operations as well i.e. you can't optimize one operation at the cost of slowing down others. Nonetheless, there many areas such as cache replacement algorithms where dynamic optimization has big performance payoffs for the added complexity.

Second, dynamic optimization needs to be applicable incrementally, otherwise it tends to have a "stop the world" effect on performance when it is being applied, which is bad. A good example of this is dynamically optimizing storage for the actual distribution of queries it sees. Dynamically adding a new secondary index imposes a huge background cost that is visible in performance, so databases don't do it. However, dynamically optimizing individual _pages_ for queries is common because it can be applied incrementally for pages you are already accessing anyway for a small one-time cost per page distributed over time as pages are accessed. And it only applies to pages you actually touch; building a new index pulls a lot of cold storage through the cache. Page level query optimization may be less effective overall than adding a second index but the fact that it can be applied incrementally makes it a preferred form of dynamic optimization.

In short, dynamic optimization has been studied and tried for many decades. Sophisticated databases do a lot of it but the mechanisms are chosen carefully to minimize adverse side effects in real world conditions.

<comment><USER2> Great Answer, thank you! Going further, would this mean one could build a DB where i could send off "optimize now" queries and telling thereby the DB that I accept stop-the-world type of effects for a limited time? Or even specifying the effects that i'm willing to accept?

My main issue is that most of the DB Stuff i do as a non-dba is so basic that it could be automated, but isn't for reasons that until your post came up weren't clear to me.

</comment></comment></comment><comment><USER4> How abou a database that uses behavioral pattern detection or behavior profiles to make data exfiltration through connected applications harder?</comment>


Airport seizes Virgin Australia planes to recoup debt - <AUTHOR> https://techxplore.com/news/2020-04-airport-seizes-virgin-australia-planes.html <comment><USER2> The shape of things to come.</comment>


Fairchild Symbol Computer (2008) [pdf] - <AUTHOR> https://gordonbell.azurewebsites.net/tcmwebpage/computertimeline/fairchildsymbol.pdf <comment><AUTHOR> "At the expense of extra hardware, the Symbol computer removed ''artifacts''—introduced to improve execution speed— of conventional computing languages. Undesirable programming artifacts, for example, included static data types and type declaration."

:-)

"Fairchild's Symbol computer had both the compiler and operating system built with hardware, not software."

8-)

"The translator hardware (compiler) operated at disk transfer speeds and was so fast there was no need to keep and store object code, since it could be quickly regenerated on-the-fly."</comment>


Why don't companies have "great programming skills" as part of their "culture"? - <AUTHOR>

I was told today by a company that very recently hired me that I am  "A very talented developer, but unfortunately not a cultural fit". This has been told to be literally by every single company that has ever hired me except for one. I have never been told by anyone that I can't get things done, or that my insights are wrong, or anything like that. The common thing amongst the companies I work with are the founders who are extroverted, love smalltalk, sales based, etc. While I'm the opposite: likes to work in quiet environment, introverted, strong engineering intuition, poor sales/people intuition.
I have this fantasy that I'll come across a company some day that will tell me "You're a great developer, who gets stuff done, and thats exactly why we want you on our team!", but I'm beginning give up on that dream because I've seen so many companies that are the opposite.
Should I give up hope, or maybe someone has a recommendation of a company in the NYC/Bay area that is looking for someone like me? <comment><USER2> Saying someone is "not a cultural fit" is a polite way to say that the person is difficult or impossible to work with.

In the workforce nowadays, you are almost never going to get direct feedback about what you might be doing wrong: you have to read between the lines.

If you have been told by several companies that you are not a cultural fit (and then let go), then you might consider what aspects of your behavior are causing friction.

Think of it this way: due to the high costs of acquiring and training employees, companies _hate_ to let people go, especially new hires. One has to be doing something very wrong to get let go, and employees are usually given many chances to get it together.

<comment><AUTHOR> """ Think of it this way: due to the high costs of acquiring and training employees, companies hate to let people go, especially new hires. One has to be doing something very wrong to get let go, and employees are usually given many chances to get it together. """

Yeah, thats not been the case with me. This last job I was let go this morning, and I started Monday. Most of my time there was spent making the case of simplifying the application. The code was horribly complex, and in my experience complexity is what does companies in. If I had been a good little employee and kept my mouth shut and did what I was told, I probably could have kept my job, but then I would have not contributed to the company in a way that would have made the company stronger. Sort of a catch 22. Put your foot down and be an asshole, or don;t put your foot down, let the company make crap software, but keep your job. There has to be a better way? Is there such a thing as a company that admires developers who have the guts to say no to a boss?

<comment><USER3> Keeping your mouth shut and doing what you are told sounds like a great idea for the _first week of employment_. You need to spend some time figuring out how things work, getting settled in, and even just learning your coworkers' names before you start asserting yourself.

If you came in to a brand new job and saying "no" to the boss was the first thing you did, it's not too surprising that they formed a negative impression.

</comment></comment></comment><comment><USER4> Note that social skills are needed in order to work within a team. I suggest you explore expanding your socials skills. If you want to, I can give you a few pointers over email.

</comment><comment><USER5> Not sure if you have, but try a company that allows remote workers

<comment><USER6> This sounds like 'hide in a closet'</comment></comment>


Shed Skin 0.3 - An Optimizing Python-to-C++ Compiler - <AUTHOR> http://shed-skin.blogspot.com/2010/01/shed-skin-03.html <comment><USER2> It is not a python compiler in almost any sense. It is more of a macro that allows you to write C++ in a python like syntax.</comment>