Ask HN: PostgreSQL or MySQL? - <AUTHOR> I have four questions related to SQL
1. PostgreSQL or MySQL? And why?
2. Is it possible to build a hybrid database schema? For example, SQLite+JSON?
3. Is it possible to convert XML(XMI) schema to SQLite schema automatically?
4. Is it possible to build a custom file format based on SQLite or hybrid one based on SQLite+JSON? <comment><USER4> The following isn't the top reason I recommend Postgres, but is the reason I think least likely to be echoed in a dozen other comments:

Postgres has some of the best documentation of any software product I've ever used. If someone wants to learn about SQL or databases, I always have to restrain myself from recommending that they just read the Postgres manual front to back. It's comprehensive, it's well written, it's easy to navigate, and almost every explanation comes with several clear examples. It's hard to overstate how valuable a property that is for a product as central to your architecture as your primary transactional database.

<comment><USER5> Too many Open Source developers discount the value of documentation. I all too often hear, "It's Open Source so the community should step up and write the documentation." To which I counter that the best person/people to at least start the documentation are the ones who build the product as they're the most knowledgeable about it. The community will gladly contribute. I personally believe that great documentation was a big reason for PHP's success in the early days.

<comment><USER6> Same for all of the GNU software distribution, Sun Microsystems' products, and SGI's. Even MSDN has had pretty good documentation.

I miss the days when I could understand how something worked just by reading the official documentation first.

To this day, in my open source projects, I do not accept contributions that lack either documentation updates or test cases.

<comment><USER7> > I miss the days when I could understand how something worked just by reading > the official documentation first.

This is probably why I don't read much documentation when I'm working with a library, et al. Never thought about it until I read the parent

</comment></comment></comment></comment><comment><USER8> On the first point, default to using Postgres unless you have a particular reason not to. In general, it offers more and better-executed features. There are some scenarios where MySQL offers a benefit and this advice doesn't apply, but it's unlikely you will need to worry about that if you're in a "what database do I use" position.

Your other questions are a bit too vague to be answerable. It's possible to build anything you want, including custom file formats or schema converters, if you want to invest the time. You might have more useful responses if you explain the use case instead of the solution you already have in mind.

<comment><USER9> As someone who has transitioned from MySQL->pg, this makes total sense to me. Partitioning is still a bit better in MySQL IMO (many types / options) - <USER2> pg is nearly there.

<comment><USER10> Did you mean replication rather than partitioning perhaps?

I've found MySQL replication quite easy to use, whereas PostgreSQL has been rather clunkier in my experience. However, I've had the exact opposite experience with respect to partitioning.

<comment><USER11> MySQL partitioning shines with one specific use-case: time-based data retention. e.g., by using RANGE partitioning over a temporal value, you can very efficiently drop large amounts of old data when no longer needed, and can automate this fairly easily.

For anything else, MySQL partitioning rule-of-thumb is you'll have a bad time, mainly for performance reasons.

<comment><USER12> Postgres's support for table partitioning got a massive boost in the last major release (11.x): [https://www.postgresql.org/docs/current/ddl- partitioning.htm...](https://www.postgresql.org/docs/current/ddl- partitioning.html)

</comment></comment></comment></comment></comment><comment><USER13> I will respond only to Q1, taking the perspective of analytics and data science.

Answer: PostgreSQL > MySQL

Postgres implementation of SQL includes a few useful clauses that are useful for analytics that MySQL does not support.

It used to be that MySQL had no window functions, and that made it wholly inferior to Postgres when it came to analytics. However, it seems MySQL began supporting window functions two years ago, so that is no longer a reason to choose one over the other.

There are at least two features supported in Postgres that are not available in MySQL that I use often enough to care:

• SELECT DISTINCT ON

• FULL OUTER JOIN

Having these saves dev time. It is possible to implement either using MySQL, but your code will be more verbose, creating more opportunities for error.

If you care about analyzing the data for data-scientific purposes, you would be better off using Postgres. It isn't just the couple of extra clauses. It's also having more (useful) indexing choices and little choices like being able to use two pipes (||) to concatenate strings instead of CONCAT().

<comment><USER14> Having consulted for hundreds of analysts writing SQL to explore their data and create reports, I would agree with this assessment. Additionally, I tend to avoid recommending MySQL because of a strange behavior when using Group By. In Postgres, every column must be either aggregated or grouped by when using a grouping expression. In MySQL, this requirement is not present, and the ambiguity of the query is resolved by returning basically random results which are quite often misunderstood to represent the desired report. In other words, you can screw up in MySQL easily and not notice, whereas in Postgres, you'll get an error message.

[https://stackoverflow.com/questions/1023347/mysql- selecting-...](https://stackoverflow.com/questions/1023347/mysql-selecting-a- column-not-in-group-by)

<comment><USER15> This was in older versions. From 5.7 ONLY_FULL_GROUP_BY is enabled by default.

</comment></comment></comment><comment><USER16> Personally,PostgreSQL. Both have their advantages and disadvantages, but my standard arguments in favor of Postgres are that it: 1\. Enforces data types natively. I've shot myself in the foot before with MySQL and 'polluting' data. 2\. JSONB support makes it easy to use a 'hybrid' schema if you want NoSQL- like behavior or a junk drawer to shove JSON into.

Regarding your conversion questions, it's common to have an app that communicates with several databases, but I don't know of any architecture that would allow ACID transactions in two engines simultaneously.

As far as converting data, it all depends on what your development preferences are. I work mostly in the Ruby ecosystem, and I can tell you that I've used ActiveRecord to a Postgres database, and exported the results into a SQLite file. I'm sure it would be possible to use an XML parser like Nokogiri to convert XML into a SQLite file as well. If you prefer another major programming language, the tools likely exist to accomplish the same thing.

<comment><USER17> > 1\. Enforces data types natively.

Quite a while ago MySQL changed the default to be strict on types and not doing truncations anymore. Recently it also (finally) got check constraints.

> 2\. JSONB support makes it easy to use a 'hybrid' schema

MySQL has a JSON data type with validation, binary storage, efficient updates and efficient replication.

<comment><USER18> My experiences are with a pretty old mysql version (5.5, iirc), but that version allowed inserts without supplying a value to a column that is NOT NULL and has no default value.

If that's the _strict_ mode, I shudder to think what the lax mode must be... :(

And upgrade to MariaDB 10.3 caught those cases, so at least there's some progress in the mysql/mariadb ecosystem.

> MySQL has a JSON data type with validation, binary storage, efficient > updates and efficient replication.

Postgres has, in addition, indexes over expression, which allows you to build indexes for specific queries into JSONB.

(I'd argue that if you need indexes, it's typically better to extract that data into the regular table and not keep it in the json, but it can still save your butt if you need to react quickly to a requirement change and no time for a data migration).

<comment><USER17> > My experiences are with a pretty old mysql version (5.5, iirc), but that > version allowed inserts without supplying a value to a column that is NOT > NULL and has no default value.

With recent MySQL:

create table ttt (col int not null); insert into ttt () values(); ERROR: 1364: Field 'col' doesn't have a default value

> Postgres has, in addition, indexes over expression, which allows you to > build indexes for specific queries into JSONB.

MySQL has that, too. I consider this quite useful, especially since MySQL also supports indexing an array of values from a JSON document. (Given `{"colors": ["red", "green", "blue" ]}` a search for `JSON_CONTAINS('$.colors', 'red')` benefits from the index.

</comment></comment></comment></comment><comment><USER19> Given the progress both Mariadb and postgresql has made I think you will not go wrong with either. In both you have support for json now which one you chose depends on your use case.

 _Mariadb_ \- If you want easy horizontal scalability with master-master replication go with mariadb.

 _Postgresql_ \- If you want reliable and powerful sql-92 compliance and willing to work with master-slave go for postgresql. You can do master to master with postgre-Xl [1]. But still given the ease of use with which you can work with galera cluster I would not recommend postgresql for master-master.

Anyways for your other questions:

2\. SQLite already supports json type [2] so its not SQLite+JSON its just SQLite.

3\. Yes you can write a simple python script to do it.

4\. Why to use a custom file format when SQLite has its own format with support for json.

[1] [https://www.postgres-xl.org/](https://www.postgres-xl.org/)

[2] [https://www.sqlite.org/json1.html](https://www.sqlite.org/json1.html)

<comment><USER20> In my experience, master-master replication with Galera is more troublesome than its worth. It's better to use the simple master-standby solution with automated fail-over.

</comment></comment><comment><USER21> 1\. Do you have experience running PostgreSQL or MySQL in production? Use what you know. Do you have friends or colleagues with that experience? Use what they know. Is the stack you end up using predominantly used with one or the other, so you have lots more people in the community who will have hit issues you might have? Use that one. Failing any of those? Default to PostgreSQL.

2\. I think you mean SQL+JSON, and the answer is PostgreSQL's JSONB columns, or dumping blobs into rows in MySQL (just keep your row sizes below 4kb).

3\. Possible? Yes. A good idea? Almost never. Trees and relations are quite mathematically different, and you will need to specify quite a bit of the mapping to make it come out in a way you would actually want to work with.

4\. Not really...generally you would have a SQLite database. If you need stuff that doesn't go in well as a blob, you would put it in a file and include a reference in the database.

<comment><USER22> Can you expand on the 4kb part of row sizes? I have been experimenting with storing json in MySQL tables and I'm curious about this limit

<comment><USER21> MySQL expects rows to be fairly small, which leads to decisions about how the system handles loading data. You can put larger blobs in, but it may have adverse performance implications.

</comment></comment></comment><comment><USER23> When considering between PostgreSQL and MySQL bear in mind that client-side drivers are licensed differently. I believe the MySQL connector is still GPL licensed; it used to be LGPL and then the license was changed with little fanfare. Commentators called this "weaponising the GPL". The goal is to get you to buy the enterprise version of MySQL.

I know MySQL way better but at a brief look I can see Postgres drivers are licensed more liberally.

<comment><USER24> Actually the goal is to get you to unwittingly use the GPL connectors, then backcharge you after an audit.

</comment></comment><comment><USER25> > 2\. Is it possible to build a hybrid database schema? For example, > SQLite+JSON?

Postgres and MySQL both have JSON support that can do fancy things which includes indexes, but it's extra work and I don't think it actually gains you anything. I'd recommend using regular old columns for anything you ever plan to want a query on. But if you have other stuff, it's perfectly reasonable to dump a bunch of random garbage into a JSON column. People use this for things like user settings, and are generally happy with it.

> 4\. Is it possible to build a custom file format based on SQLite or hybrid > one based on SQLite+JSON?

SQLite is already basically a file format. You can store JSON blobs in strings somewhere inside it if that's convenient.

<comment><USER26> Postgres jsonb column type has worked great without much effort. The old JSON type wasn't great and required special index, but that's decently in the past now. Not sure of MySQL in this aspect.

</comment></comment><comment><USER27> If you ask this at HN most likely it will be PostgreSQL.

If you ask scripting language it got to be python or javascript here.

I use HN daily and noticed as a group, HN readers do in general have their preference on things like this.

By all means, MySQL's market share is much larger than PostgreSQL, and MySQL 8+ is very impressive. I compared both and eventually picked MySQL for my projects, even though I really _hate_ Oracle.

<comment><USER28> You could use Percona MySQL, which isn't by Oracle and is much better anyway.

</comment></comment><comment><USER29> 1\. Depends. If you have a lot of update queries, MySQL wins [[https://eng.uber.com/mysql-migration/](https://eng.uber.com/mysql- migration/)]. MySQL also has better replication and sharding support at the moment.

2\. PostgreSQL and MYSQL also sport JSON... JSON functions can be a little tricky. Still, they work.

3\. Depends on the structure of your XML. Might even possible to use RegEx to achieve this objective. XML to JSON is easier though. That can be saved in JSON fields in a DB.

4\. You can build a custom file based on anything. The only catch, the SQLite client will only read a valid SQLite file. As such, you have to know what you're doing.

[Edit] Formating

<comment><USER30> That's incredibly well written, constructive feedback by the Uber team! I wonder if the Postgres team has taken this on board and attempted to resolve some of the design issues, e.g.: by moving the cache in-process.

<comment><USER31> The Uber team betrays their lack of knowledge about PostgreSQL in the article, so take it with a huge grain of salt. I recommend reading Markus Winand's article.

[https://use-the-index-luke.com/blog/2016-07-29/on-ubers-choi...](https://use- the-index-luke.com/blog/2016-07-29/on-ubers-choice-of-databases)

But that does not mean though that there are not also real issues with table bloat in PostgreSQL, and they are being worked on by creating a new storage engine with a different MVCC implementation.

</comment></comment></comment><comment><USER32> #1: depends on your scenario. Postgres is more like Oracle--there are types of query it can do natively that would have to be externally assisted with MySQL. MySQL is simpler, has all kinds of warts, but scales decently.

Here's why uber moved to MySQL: [https://eng.uber.com/mysql- migration/](https://eng.uber.com/mysql-migration/)

#2-4: My use of SQLite has always been very basic, so no idea. Per SQLite's author: " _SQLite does not compete with client /server databases. SQLite competes with fopen()._"

<comment><USER33> That's a very interesting article. Does anybody knows if Postgres 10 and 11 addressed any of those issues?

<comment><USER34> [https://news.ycombinator.com/item?id=14222721](https://news.ycombinator.com/item?id=14222721) is a continued discussion of the response.

</comment></comment></comment><comment><USER35> For me postgresql's schema, EXPLAIN, Column modification and UUID. But most of the time I stick to sqlite, it satisfies most of the project with low footprint and require for installation restricted servers.

<comment><USER36> Every time I've used sqlite for any kind of throwaway web app, I've regretted it due to the almost complete lack of support for concurrent operations. Sqlite is for file formats, not anything that might have concurrent writes.

<comment><USER37> TIL. I really thought SQLite had better support for concurrent behavior.

Do you know if the problem you cite is distinct separate processes of SQLite? Ie, can a single process (aka one web app) handle concurrent behavior properly with SQLite? Or will it fail even in that scenario?

<comment><USER38> Why write code to deal with that complexity when you can just use pg or MySQL?

<comment><USER37> Because depending on the use case you don't want to run a 2nd server. For example, I work on some FOSS that lets users self host. The intention is for minimally experienced users to self host, as well as host on small laptops/etc. For obvious reasons, spinning up a full database has issues - requiring them to install Docker or etc is not feasible.

With that said, knowing how DB concurrency behaves in inherently concurrent applications (anything HTTP facing, for example) seems vital. If it needs to be not concurrent, it's paramount to know that.

To be clear, knowing how and when the concurrency breaks down does not mean advocating for it. Merely that if concurrency wasn't supported, you may need to write your applications differently to properly lock the non-threadsafe behavior.

</comment></comment></comment></comment></comment><comment><USER11> To echo a few other commenters here, a major point of consideration is your team's experience and broader technical network. Generally speaking, either Postgres or MySQL is likely fine for most applications. But you'll have a better time with whichever one you can most smoothly build on, operate, and hire for.

Separate from that, and with the disclaimer that my career has been heavily MySQL-oriented, although I try to provide a balanced perspective:

MySQL can be a good choice for OLTP-heavy workloads, such as a high volume of relatively straightforward queries. It also supports a very wide variety of replication options. In some aspects, MySQL is simpler to operate than pg. (This can be a point of contention, as MySQL is more idiosyncratic and this leads to some land-mines that are well-known to the community, but unclear from the manual alone.)

At this point there are also many hundreds (thousands?) of engineers and DBAs worldwide who have extensive experience working on massive-scale MySQL deployments. This may or may not be relevant to your project, depending what you're building and its eventual best-case size.

MySQL is admittedly less "batteries included". Although MySQL/InnoDB does support things like windowing functions, fulltext, geospatial, etc they're not necessarily a best-in-class implementation yet. In a MySQL shop, if you need OLAP, you're better off doing ETL into a separate column store or Hadoop cluster; if you need fulltext search, you're better off using a separate search index system like Solr; etc. Depending on your point-of-view, this approach is either "more UNIXy" or a major operational headache, or both :)

Meanwhile Postgres may be a better choice if you want to handle a bunch of things using a single piece of software: mixed OLTP + OLAP workloads, fulltext search, geospatial. It also has some interesting indexing features MySQL lacks, such as partial indexes, and ability to index an entire nested JSON structure in a single index.

Postgres gets major points for having a true open source development model, cleaner core code, better adherence to SQL standards, etc. Conceptually, it is very well-aligned with the textbook description of a relational database. Its adherents love it, sometimes fanatically so, which can be a mixed bag to outsiders. I sometimes feel parallels to the Mac user community in the late 90s/early 00s: it's probably a better choice for many use-cases, but personally I raise an eyebrow to suggestions that it's the universal best choice for all possible situations.

</comment><comment><USER39> I'm not a very tech savvy and know very little about databases. But I have read countless posts/comments over the years and everywhere everyone recommends postgres.

Again, I have no clue about other database systems, but postgres documentation is just awesome. I believe you can practically everything from its documentation.

<comment><USER40> I find a comment like this to be underrated; it's like a meta-analysis.

</comment></comment><comment><USER41> 1\. Both are great, they're both absolutely fantastic products!

PostgreSQL does have slightly more popularity nowadays, especially since they came out with JSONB support first and have more advanced features built-in.

</comment><comment><USER42> I have used PostgreSQL+JSONB and it works great. Doing indexes on JSON fields adds a lot of value.

<comment><USER43> What do you use the JSONB column for?

We've loved the flexibility of JSONB while our schema wasn't nailed down, but eventually migrated many fields to be columns for ease of reading and writing nested objects.

</comment></comment><comment><USER44> There was an Uber engineering post years ago about why they switched to mysql. Not sure how relevant it is today

[https://eng.uber.com/mysql-migration/](https://eng.uber.com/mysql-migration/)

<comment><USER45> [https://use-the-index-luke.com/blog/2016-07-29/on-ubers-choi...](https://use- the-index-luke.com/blog/2016-07-29/on-ubers-choice-of-databases)

</comment></comment><comment><USER46> Use what you're most comfort with, not what others are.

Postgres and MySQL (MariaDB/Percona) are both big players with a huge community.

Maybe consider to use Mariadb or Percona instead of MySQL to avoid oracle vendor lock in.

</comment><comment><USER47> EXPLAIN is almost useless on mysql, but very helpful in postgres.

EXPLAIN is a very useful command once you have some degree of success (users).

<comment><USER17> EXPLAIN with the JSON or TREE formats in MySQL got a lot better. Also MySQL's optimzer traces give lots of information more [https://dev.mysql.com/doc/internals/en/optimizer- tracing.htm...](https://dev.mysql.com/doc/internals/en/optimizer-tracing.html)

</comment></comment><comment><USER48> 1\. Mostly, Postgres, as it's superior in most areas, but MySQL seems to still have advantages in replication (and, if you are hosting in AWS, there Aurora Serverless MySQL offering supports somethings, like their own Data API, that the equivalent Postgres offering doesn't yet which may tip the balance.

> Is it possible to build a hybrid database schema? For example, SQLite+JSON?

Yes, but many DBs will hold JSON natively and, conversely, lots of DB data fits nicely in JSON. What's the motivating use case?

> 4\. Is it possible to build a custom file format based on SQLite or hybrid > one based on SQLite+JSON?

Not sure what you are asking here; SQLite has its own file format.i suppose, as with many combined formats these days, you could stop that together with a folder of JSON documents and call it a custom format. What's the motivating use case?

<comment><USER49> Aurora PG Serverless is out as of July, but I've read some horror stories about it so far.

<comment><USER48> Don't know about horror stories, but Aurora PG Serverless doesn't (yet?) have the Data API support (which enables a couple of other Aurora serverless features) that Aurora MySQL Serverless has, and IIRC has some other feature limitations that aren't directly tied to Data API.

<comment><USER50> Data API now supports Aurora PG Serverless, [https://aws.amazon.com/about- aws/whats-new/2019/09/amazon-au...](https://aws.amazon.com/about-aws/whats- new/2019/09/amazon-aurora-serverless-postgresql-now-supports-data-api/)

</comment></comment></comment></comment><comment><AUTHOR> Thank you all, I've tried to read all the comments. I think I have to clarify the questions I asked:

1\. Is related to our upcoming startup, we chose MySQL + Redis, but some said use PostgreSQL

(However, my friend explained that MySQL before v8 was weak and after v8 has tried to come back to its salad days, this is why he chose MySQL)

2-3-4: In the industry, especially in the industry I'm active, Digital Built Environment Industry, there's a lot of invaluable schemas, especially XML(XMI) that I feel "these schemas convert structured data to semi-structured data" which is not good and efficient

For this reason, I'm thinking of SQL (Server) and SQLite (File) to preserve structured data

There's a vitally important schema in the built environment industry called Industry Foundation Classes (IFC) [1] which is based on EXPRESS/STEP and also has XML(XMI) schema too, but I want to convert the schema to SQL and SQLite

A friend did an invaluable job and after years introduced a correct method to convert it to SQL [2][3] but don't have any idea of developing the schema based on SQLite [4] that will accelerate the IFC development

[1] [https://www.buildingsmart.org/about/what-is-openbim/ifc- intr...](https://www.buildingsmart.org/about/what-is-openbim/ifc- introduction/)

[2] [https://forums.buildingsmart.org/t/ifc-for-relational- databa...](https://forums.buildingsmart.org/t/ifc-for-relational-databases- ifcsql/1524)

[3] [https://forums.buildingsmart.org/t/store-modify-and- retrieve...](https://forums.buildingsmart.org/t/store-modify-and-retrieve-ifc- data-with-ifcsql/1558)

[4] [https://forums.buildingsmart.org/t/ifc-sqlite- project/1359](https://forums.buildingsmart.org/t/ifc-sqlite-project/1359)

</comment><comment><USER51> For point 3 (Is it possible to convert XML(XMI) schema to SQLite schema automatically?), are you asking about a tool that takes an XML schema and gives you back an SQL schema?

I haven't heard of anything, but I would imagine it exists. I have, on the other hand, done a fair amount of XML to SQL conversion and have found that generating a good schema involves understanding the data so you know what to denormalize. You don't really want to have thousands of tables, each representing an XML tag. That would make your database very hard to use.

Trial and error, and a good knowledge of what kind of questions you are going to be asking your database to answer...

</comment><comment><USER52> Answering Q1: MySQL over Postgres. Postgres has 2 critical things going against it \- The query optimizer is a mess \- It's implementation of secondary indexes is not cache friendly, so it can be slow

<comment><USER53> I don't know what you're trying to say. PostgreSQL makes no distinction between primary and secondary indexes. MySQL, which only knows clustered indexes (see "Unreasonable Defaults" [1]) has to make such a distinction, so that when you use a secondary index it also has to go through the primary index in addition to the secondary index. Also

[1] [https://use-the-index-luke.com/blog/2014-01/unreasonable- def...](https://use-the-index-luke.com/blog/2014-01/unreasonable-defaults- primary-key-clustering-key)

</comment></comment><comment><USER54> I would also recommend Postgres for the features alone. Like the full text search.

One reason not mentioned often is the vast amount of plugins which can do lots of useful things.

Like PostGis or TimescaleDb which all work really well.

<comment><USER17> Instead of requiring a plugin, MySQL has good GIS support natively built-in.

</comment></comment><comment><USER55> As previously mentioned, I agree that there is much less difference between them than there used to be. Use what you know or have access to those who know.

Personally I prefer PostgreSQL and the main benefit of using MySQL is how easy it is to set up replication and operate in master-slave, master-master, or group replication.

For high write volumes, TokuDB and now RocksDB do work well, the former for sequential writes and the latter is LSM Tree based.

</comment><comment><USER56> Postgres is a long-term support stack at this point, like the JVM. Too many people rely on it. You need a very good reason to pick anything else.

</comment><comment><USER57> 1\. PostgreSQL. Fast, reliable, (relatively) small (for some reason mysql got a lot bigger). Had a few historic problems with replication but I think that's been resolved. I dealt a lot with large scale large data sets and I wouldn't recommend mysql for that at all, except if they were very simple and relations were either not used or used minimally. There are better database systems than either, but I can't think of a less reliable one than MySQL. If you use PHP though, mysql is easier.

(I'm going to assume the rest of the questions are SQL related and not SQLite, which is a particular binary format)

2\. why? SQL + json would make some sense, sometimes. I haven't needed it - <USER3> just do SQL. Others have better answers.

3\. There are tools to do some XML to and from SQL. YMMV. I always rolled my own.

4\. it's always possible. Is it worth it? I used CSV with embedded json for a long time with archival storage because it was reversably parseable and could scale time-based databases somewhat reasonably. There are smarter ways to do this too ...

<comment><USER11> > I dealt a lot with large scale large data sets and I wouldn't recommend > mysql for that at all

Was your use-case transaction processing (OLTP) or analytics (OLAP)? There are a ton of examples of massive-scale MySQL deployments for OLTP... Facebook, YouTube, Pinterest, Slack, Uber, Wikipedia, GitHub, Yelp, Etsy, Shopify, Booking.com, Wordpress.com, Tumblr, Box, Dropbox, Alibaba, Square, Venmo... a large chunk of the internet, in other words :)

<comment><USER58> It seems most people on HN (and to their credit, they also admit), they have limited recent experience with MySQL, so they don't know that they are 99% functionally equivalent (with edge cases being better for one or the other).

</comment></comment></comment><comment><USER59> You don't mention SQLite in question 1, but then include it in questions 2-4.

Did you maybe mean "SQL" and not "SQLite" here?

</comment><comment><USER60> For very simple usage, you won't see a lot of difference, but I would go with postgres. It has a lot of features that MySql doesn't, and isn't any harder to set up and use.

Point 2: as long as you also write software to deal with it, I guess so. Postgres has a JSON type. This question makes me want to ask what you are trying to accomplish.

Point 3: Yes, probably, and for all I know there's something out there to do it for you. I'm assuming basic usage; there is undoubtedly some weird corner of xmi that makes it a Turing-complete language or something equally ill- advised. So writing a schema converter for very basic stuff is probably quite easy, but really filling in all the junk is possibly less so.

4 appears to be a repeat of 2.

</comment><comment><USER61> I'm going to write a couple of things about the first point.

Several years ago, a knowledgable guy told me that the most compelling reason for choosing between PostgreSQL and MySQL was the expected I/O: "for read- intensive workloads (e.g. blogs), choose MySQL; for mixed workloads (e.g. forums), choose PostgreSQL".

But I honestly don't know if that may still be valid as of today.

Nowadays, I think that for basic things, it doesn't really matter; but for peculiar things, Postgres may have some advantages (both technically and not). Also keep in mind that, for some popular scopes, SQLite is likely everything you really need.

<comment><USER11> > Several years ago, a knowledgable guy told me that the most compelling > reason for choosing between PostgreSQL and MySQL was the expected I/O: "for > read-intensive workloads (e.g. blogs), choose MySQL; for mixed workloads > (e.g. forums), choose PostgreSQL".

This seems a bit backwards. MySQL's main strength is OLTP workloads, including mixed read-write workloads / high write volumes. A majority of the giant social networks are using MySQL (or previously used MySQL before moving to in- house custom databases) and have insane write volumes.

For OLAP workloads, or mixed OLTP/OLAP workloads, Postgres tends to be a better choice. OLAP is inherently read-heavy, which is why I would disagree with the advice above.

<comment><USER21> The biggest social network uses MySQL because they already dumped so much engineering into it that changing is basically impractical. But it's mostly used as a key-value store backing a custom graph database.

<comment><USER11> That is simply not correct. Please understand that I worked on MySQL at Facebook, so I know what I'm talking about here :)

Facebook developed an entirely new MySQL storage engine (MyRocks, which is a RocksDB-backed engine for MySQL) and then migrated their largest sharded tiers to it. This is basically just as much work as developing a new database from scratch, i.e. more work than something like migrating to Postgres. This completely debunks the "changing is basically impractical" claim.

And while Facebook's primary db tier (UDB) does have a restricted API / access pattern, calling it a "key-value store" is a gross oversimplification at best, or completely inaccurate at worst. Range scans are absolutely core to the UDB access pattern, for starters.

Many other social networks are also built on MySQL (linkedin, pinterest, tumblr; and several in China) or previously used MySQL before moving to a custom in-house db (twitter). I think reddit and instagram are the only two using pg? And I recall parts of instagram were being moved to mysql, although I'm way out-of-date on whatever happened there.

<comment><USER21> > This is basically just as much work as developing a new database from > scratch, i.e. more work than something like migrating to Postgres. This > completely debunks the "changing is basically impractical" claim.

I disagree for three reasons:

1\. The long tail of code using MySQL at the company, like at any large software company, is prohibitive. You would have to maintain MySQL and PostgreSQL in parallel for years. A new storage engine, on the other hand, is controlled by one team.

2\. Migrating from InnoDB to MyRocks consists of successively adding MyRocks replicas, letting them catch up, and removing InnoDB replicas. That is a dramatically easier proposition than migrating tiers to PostgreSQL.

The fact that RocksDB was a hard technical project is kind of irrelevant. The new storage engine provided major wins and could be done within a team, while migrating to PostgreSQL would provide at most small improvements and demand changes to huge amounts of code and massive data migration projects. That makes the former project deeply practical and the latter impractical. If the usual stack back in the day had been the LAPP stack instead of the LAMP stack, we would be having this discussion the other way.

> calling it a "key-value store" is a gross oversimplification at best

That's fair. The right thing to have said would be that the query patterns that are used are extremely simple selects over a single table, which is a place that MySQL has traditionally shone. MySQL's query planner still does strange things on complex queries from time to time. I had a case about six months ago where one shard decided it was going to reorder indexes in a query and load everything in the database's core tables before filtering it down instead of using the proper index order like the other nine hundred something shards. Easily fixed once we realized it (we forced the index order in the query), but the fact that we had to... I have heard that this has all gotten much better in MySQL 8.0.

<comment><USER11> You're _severely_ underestimating the amount of effort that went into MyRocks. The development and deployment was a 3+ year effort spanning quite a few different teams.

Automating the physical rollout (as you correctly described) is the easy part. That doesn't account for all the many difficult spots that occurred prior to it: the massive complexity of mysql storage engine development in general; huge numbers of various performance edge-cases; converting replication-related system tables to MyRocks in order to achieve crash-safe replication; developing online hot-copy for MyRocks from scratch; schema change support; adding tons of new status vars and system vars; fast bulk import to MyRocks which is necessary for the replica migration to even be possible; updating hundreds of thousands of lines of automation code written under the assumption of InnoDB being the only storage engine in use and using various InnoDBisms...

The MyRocks migration wasn't a project I personally worked on, but I'm very aware of what was involved. It appears you joined FB PE in 2017 and therefore missed much of this historical context? I'm not really sure why you would have such strong opinions about it.

You say that FB is using MySQL because "changing is basically impractical", but also say MyRocks "provided major wins", which seems to be a contradiction. In any case, I'm not aware of any pg feature that provides compression ratios _anywhere_ near that of MyRocks, and pg is only recently even adopting an arch that supports pluggable storage engines at all. In combination it's really hard to make a case that FB is using MySQL just due to historical investment and inability to change.

Honestly I would also not be surprised if FB moves some core tiers off of MySQL to a pure-RocksDB solution at some point in the future. The number of intermediate data services and proxies make this sort of thing absolutely possible. For the same reason, in theory a move to another db like pg would be completely possible without needing to run both in parallel for years (again, just talking in theory; moving to pg just doesn't make practical sense).

> The right thing to have said would be that the query patterns that are used > are extremely simple selects over a single table

For UDB, sure. What about all the other MySQL tiers? The non-UDB MySQL footprint at FB, despite being a minority of FB's MySQL fleet, is still larger than the vast majority of other companies' relational database fleets worldwide. The range of use-cases in the DBaaS (XDB) tier alone spans a massive combination of different database features and access patterns.

<comment><USER21> > You say that FB is using MySQL because "changing is basically impractical", > but also say MyRocks "provided major wins", which seems to be a > contradiction.

I think I must not be expressing myself clearly. 3+ year projects involving a large number of teams to get back to where you started are impractical. That's what migrating to PostgreSQL would be. Perhaps I should have written "switching from MySQL to PostgreSQL would be impractical"?

<comment><USER11> Apologies if I'm misunderstanding. To take a step back and paraphrase this subthread, as I understand it:

* `dezzeus said MySQL was better for read-intensive workloads, Postgres better for mixed read/write

* I replied saying there are a number of huge social networks with insane write rates, which is contrary proof against that claim. (Having personally spent most of the past decade working on massive-scale MySQL at several social networks / UGC sites, this topic is near and dear to my heart...)

* You replied saying, iiuc, that FB is only using MySQL for historical reasons and difficulty of switching. (IMO, your initial comment was tangential to the original topic of comparative read vs write perf anyway. Regardless of why FB is using MySQL, factually they are an example of extremely high write rate, previously via InnoDB for many years. That said, I wasn't the person who downvoted your comment.)

* I replied saying that's inaccurate, as FB demonstrably _does_ have the resources and talent to switch to another DB _if_ there was a compelling reason, and furthermore MySQL+MyRocks provides a combination of feature set + compression levels that other databases (including Postgres) simply cannot match at this time. At FB's scale, this translates to absolutely massive cost savings, meaning that MySQL+MyRocks is a better choice for FB for _technical and business reasons_ rather than just historical reasons or difficulty of switching.

I may have misunderstood, but it definitely felt like your original comments were throwing shade at MySQL, and/or publicly stating historically inaccurate reasons for why FB is currently using MySQL.

<comment><USER21> I think I have done a poor job of communicating, and I'm sorry for it.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER37> Question on this area. What have you found to be the best way to migrate an existing app to Postgres?

We plan to migrate to Postgres, but with every LOC written we're entrenching ourselves into Mysql. We all want Postgres, but at the same time I just can't see the transition happening with each passing day.

Thoughts?

 _edit_ : I should be clear too. I think our goal would be to not require a full rewrite spanning weeks. So an active translation layer, or replication.. or something tricky.. but I just don't foresee us being able to spend a month putting everything on hold so we can migrate.

</comment><comment><USER62> My company recently looked at MySQL vs PostgreSQL. While we chose Postgres there is one thing that stood out in my research, I haven't found any comments on this:

We were looking at scalable, HA, possibly multi-DC workloads and such, and yes, relational DBs are not necessarily the right choice, but MySQL looked a bit better in this regard, especially for master-master replication. There are several high-profile teams that have huge production deployments running on MySQL (IIRC Uber and Github) and there are a lot of things to be learned from them.

<comment><USER63> Did you evaluate Citus as well?

</comment></comment><comment><USER64> You can compare it from here [https://sql- workbench.eu/dbms_comparison.html](https://sql- workbench.eu/dbms_comparison.html)

</comment><comment><USER65> Is there a Galera-equivalent system available of Postgres? I.e., multi-master replication.

I know of BDR, but that doesn't seem to be publicly available for recent versions of Pg?

<comment><USER66> Not sure about master-master, but I've used repmgr[1] to manage a PostgreSQL master-slave setup and it works pretty well though it is by no-means plug and play. There are a variety of steps and manual processes to get it up and running.

[1] [https://repmgr.org/](https://repmgr.org/)

<comment><USER65> We find it handy to have multi-master, which leans towards being more active- active: we have keepalived doing health checks on two systems, and point our app server(s) at a vIP.

The app server runs Galera's _arbd_ so we don't need three DBs.

</comment></comment></comment><comment><USER67> My first goto database is always SQLite. Only when SQLite is unable to handle the requirements, for example, too many writes, do I look at PostgreSQL.

</comment><comment><USER68> Regarding 1): choosing a software stack (or any part thereof) without a clear use case is a recipe for failure. What's the use case?

<comment><USER69> To be fair, often you don't have your requirements completely nailed beforehand, or they change midway through the project.

That's why I choose Postgres: aside from extreme scale or latency, you can throw pretty much anything at it. Also, given expert advice, its performance degrades gracefully i.e. lots of easy ways to optimize and workaround issues.

Example: pg's type system is insanely flexible incl native support for high speed JSON, latlng, geometry, and more - and types are "batteries included" incl matching, inspection/extraction (e.g. JSON navigation), conversion, concurrency and concurrency control, ACID recovery, replication, etc. For example, it's one way line of code to create an index on the results of a JSON navigation expression whose result is then converted to another datatype e.g. timestamptz. Performance is then 100x+ on queries that use that expression.

Pg datetime handling is also world class incl timezones and conversion, query and manipulation functions. Want the records from last Tuesday across timezones, given data in seconds-since-the-epoch? The SQL is shorter than in English.

<comment><USER70> If the requirements are still up in the air, then I would aim to use the simplest, most evolvable solution. This to me means _no_ database.

<comment><USER71> If the requirements are still up in the air, but you know that you're going to be storing _some_ data, definitely go with a database, even if just a single SQLite file. Using plain files means figuring out the file format, which turns into a huge mess when requirements get tacked on later. The other option would be to store a JSON file or a bunch of JSON files in a directory, at which point you're building a shitty version of either SQL or MongoDB instead of focusing on your business logic.

<comment><USER70> The grandparent comment is right about one thing: you need to figure out what your use case is, in order to make a good judgment call. "Some data" can mean a lot of things. Transactional data? Sure, get a sql database in. Images, video, or plain json documents? SQL databases are a poor fit there.

One should also consider if they even need to manage data persistence. I've had success using Google Sheets as the backing data storage for a prototype. Below a certain scale, it is a lot easier to visualise and manipulate data in a spreadsheet than through a sql client.

<comment><USER68> This. Answer "pg!" without any clue of what the objective is, and you'll end up with surprises like "oh, you know, this is for a revolutionary concept where people can check in their source code at different stages of development, so that they have a history of their changes and can restore an older version if needed" or "hey, we've followed your expert recommendations but now we're facing some availability issue after a link to our shiny new search engine made the front page of HN and Slashdot". Migrating from pg to git or SOLR is not really enjoyable, and you can't reuse much of what you've done (unlike when migrating one SQL database to another).

</comment></comment></comment></comment></comment></comment><comment><USER72> Although there is already pretty unanimous consensus here, one side note is that administrating mysql is always a headache in my experience whereas postgres seems to just work.

Every time I start a new project and I see it's using mysql i sigh a little bit knowing that I'm going to end up spending hours fighting some dumb configuration issue.

</comment><comment><USER73> What does your app/framework support as a default store (what do the devs use and test against).

What features do you expect from the db? What versions of each database can you run?

Have you got an environment where one option is better supported than the other? (For example AWS aurora cluster and MySQL?)

Do you have an ops team with existing experience with either db?

</comment><comment><USER74> MySQL: Wait a sec, what do I use for this table, InnoDB or MyISAM? What are the tradeoffs again? Oh, right, do I want ACID transactions or fulltext search? Man, this tradeoff sucks.

PostgreSQL: Wait a sec, which schema (namespace) should I put this table in? Man, it's nice to have namespaces within a DB.

<comment><USER75> No one has been using MyISAM for the last 10 years.

<comment><USER74> Funnily enough, I have not been using MySQL for the last 10 years either, because I switched to Postgres and never went back.

<comment><USER11> Fair, but why comment on MySQL's supposed deficiencies if your knowledge of it is so out-of-date?

<comment><USER75> But vanilla PG didn't get good master / slave feature until "recently" so you were running without replicas in the last 10 years?

[https://www.postgresql.org/about/news/1235/](https://www.postgresql.org/about/news/1235/)

<comment><USER76> Your link is from 9 years ago.

<comment><USER75> The release from 9 years ago was not even close to what MySQL was offering at the time.

</comment></comment></comment></comment></comment></comment></comment><comment><USER77> MySQL or better yet AuroraDB if you want scalability. MySQL still has a sizable speed advantage, is easier to learn, maintain, and has a more vibrant world of tooling. MySQL is also easier to find experienced developers for. This has been my experience anyway.

</comment><comment><USER78> Are you confusing SQLite with SQL?

</comment><comment><USER79> Isn't MySQL owned by Oracle?

To each his own, but that would just about answer the question for me.

<comment><USER80> MariaDB isn't.

</comment></comment><comment><USER81> In case it's not been said already, Oracle own mysql, use mariadb instead

</comment><comment><USER82> I'd say Postgres because of tools like [https://hasura.io](https://hasura.io) that let you automatically layer graphql on top of it and jsonb :)

</comment><comment><USER83> I really like the direct DB to filesystem mappings of the MyISAM engine in MySql. A database is simply a directory. A table is simply 3 files in that directory:

        - One for the structure
        - One for the data
        - One for the indexes

You can simply copy a MyISAM database by copying the directory (while the MySql server is stopped).

MariaDB has forked the MyISAM engine and created the Aria engine. It is a crash safe version of MyISAM and is now their default engine for all system tables.

So in contrast to MySql, you can completely disable InnoDB in MariaDB. MySql does not support that anymore as their system tables are InnoDB.

Especially for non-concurrency workloads, MyISAM and Aria are super fast.

For these reasons, I really like MariaDB.

<comment><USER84> You also get "mariabackup" which is a fork of perconabackup for consistent, physical backups of InnoDB and Aria/MyISAM

</comment></comment><comment><USER85> 1\. Whatever you feel most comfortable using and administering because the alternative is dangerous for production.

2\. Probably.

3\. Probably, but you'll probably have to write the code to do it.

4\. Why bother?

</comment><comment><USER86> Postgres hands down.

</comment><comment><USER87> MySQL or MariaDB?

</comment><comment><USER88> Materialized views, common table expressions, outer joins, are all missing in mysql and essential. The Postgresql's stored procedure language blows mysql out of the water, and it's possible to use different language backends.

SQLite has json extensions. You need to select them at compile time though, so you can't just assume they'll be there by default.

<comment><USER11> MySQL has supported left/right outer joins for as long as I can remember, and I've been using it continuously since 2003.

MySQL doesn't support _full outer join_ , but it's rarely needed, and you can often get a similar result using a UNION. (Clunky, I'll admit... but I'd say the same thing about Postgres recommending against use of _NOT IN_ , which is a common construct and more readable than an anti-join.)

CTEs were added in MySQL 8.0 (~1.5 years ago).

Agreed on the other points.

<comment><USER89> > . but I'd say the same thing about Postgres recommending against use of NOT > IN, which is a common construct and more readable than an anti-join.

The recommendation against that is due to the semantics defined in the standard. If there is _any_ NULL in the list matched against, the entire a NOT IN (...) expression also has to evaluate to NULL. Which is extremely confusing to most people (and has some consequences in how efficient things can be executed).

</comment></comment></comment><comment><USER90> I have quite a bit of experience with #2.

We use a SQLite hybrid with JSON blobs (stored as regular TEXT columns) as the principal transactional store for our clients' business system state. The simplicity of this approach (which is definitely not for everyone) is what made it feasible for our small development team to manage the project as far as we have. If we were still using a big-boy SQL solution like PostgreSQL or SQL Server and managing all of the concerns specific to these dependencies (multiplied by the number of environments we are responsible for deploying these to), we probably would have been forced to close up shop by now.

Being able to deploy a single binary which brings every dependency (including the database) it needs along for the ride proved to be a lifesaver for us. The process before this involved spending an entire day getting SQL Server configured relative to the service accounts, and whatever ridiculous quirks may exist in each clients' specific AD. Today, it is literally a 30 second file extract, sc create, and it's running. We go from RDP into a blank box to a working production system within 5 minutes. This is also how we've been able to completely sidestep the docker question, since no one could justify the added complexity once we got the equation down to: a single binary folder + a bare-ass Windows Server 2016/19 box = good to go. This is also enabled by .NET Core's Self-Contained Deployments, so SQLite isn't a free lunch all on its own.

Again, the above is not for everyone. We are responsible for maintaining production services running across many client sites, and needed a solution that could scale not only in terms of our clients' requirements, but also in terms of our abilities to manage all of these environments all at once. Managing a hosted SQL service requires direct access to each of these environments which has tremendous overhead for us due to the email back-and- forth remote access game that has to be played each time.

If you are in an environment where its just a single instance of the production/test/qa/development stack, I would absolutely push for a hosted SQL option like PostgreSQL/MySQL/SQL Server. In these cases you have a single (or 4) points of touch, and it is very reasonable and consistent and all within your control. If you find yourself in a situation similar to mine, I would still advocate for the hosted SQL by default unless you are a very brave soul and understand the risks of going a bit off the beaten path. Disclaimer aside, there really are some interesting treasures out here. If you can afford to waste a month, try doing your entire persistence layer in SQLite with JSON serialized business models where appropriate. Assuming you budgeted for complete failure, worst case you walk away with a new perspective. Better case, you end up in the position I find myself in now.

Just don't forget to set PRAGMA journal_mode=WAL; if you decide to go on the adventure. It makes orders of magnitude difference in concurrent write throughput (SQL Server was never this fast for us).

</comment><comment><USER91> MySQL is simply garbage that shouldn't be used under any circumstance.

1\. No transactional DDL 2\. Implicit, confusing coercions 3\. Terrible Unicode handling 4\. the list goes on

<comment><USER17> 1\. Thanks to the new data dictionary in 8.0 there is light at the end of that tunnel ...

2\. Yeah, MySQL tried to be "simple" and it took a while to be confident for changing it's default to being strict (the option for opting exists for ages ...) But meanwhile defaults are proper.

3\. Especially the 3 byte Unicode type was a historic mistake. It reduced required storage space but didn't forsee the need for Emojis and other 4 byte sequences ... utf8mb4 charset fixes that though

4\. ...

</comment></comment><comment><USER92> MongoDB

</comment><comment><USER93> [https://m.youtube.com/watch?v=emgJtr9tIME](https://m.youtube.com/watch?v=emgJtr9tIME)

</comment><comment><USER94> 1\. For small databases (up to a few GB) my personal preference in the past 5 years was MS SQL (Express Edition, free), then PostgreSQL and last is MySQL. That means the answer to your specific question, PostgreSQL. Why? That is the order of features, ease of use and performance in my experience, other people may have seen different.

For larger databases I have no experience with either PostgreSQL or MySQL, I was spoiled to use MS SQL for up to Terabyte size and that's the largest I've ever worked with.

<comment><USER95> MS SQL Server definitely has some important features that are missing from PG. Top on that list IMO is real stored procedures that can return multiple heterogeneous result-sets with simple select statements. Another big one is client side tools. Query optimization and schema flexibility (being able to alter things instead of dropping in re-creating) might still be not as good in PG as well.

You won't get a ton of agreement from this crowd though, despite the majority of them probably never having used it.

<comment><USER96> Only this is not a way to select a DBMS. PostgreSQL adds important features much faster than competitors, it is free and much leaner and more flexible.

<comment><USER95> Free and leaner, yes. More flexible? That's debatable. As I pointed out, it's certainly not more flexible when it come to returning multiple result-sets from a single procedure.

I think that PG actually did just add a feature called stored procedures though. So maybe soon for returning multiple heterogeneous selects! It's one feature that I've been waiting a long time for…

I chose SQL Server in many cases for it's very high quality, support, superior ease of use and flexibility. Free and leaner weren't very high on my list comparatively. You think those are the best characteristics to go by?

</comment></comment></comment></comment><comment><USER97> You don't really benefit from MySQL speed if you are not Facebook, so I would always start with PostgreSQL.

<comment><USER98> We use PostgreSQL but also Firebird, which maybe is the great underdog in opensource databases.

<comment><USER99> I used InterBase (Firebird predecessor) 15 years ago, and recall it had some big limits around versioning/lots of updates. To reclaim disk space we'd have to periodically backup/restore the db. It got so bad/frequent that we moved to Postgres and haven't looked back since. I guess if one is considering SQLite that it's not too relevant, but is this still an issue with modern Firebird?

<comment><USER100> Many of IB's limitations and bugs have been removed over the years. I think it was around FB2.5 that it really became a better product than IB ever was. A cool thing, though, is that it's very backward compatible. If you have applications that expect IB6, they will happily connect to any version of FB, even using the old client lib. The pace of development has ramped up since FB3, which was focused on rearchitecting the core. That's mostly a good thing. Still, I fear a little that it will also affect reliability.

Not sure exactly what kind of situation got you in trouble, but I haven't had any issues (ab)using the database myself. Disk space is still not reclaimed. It does, however, get used when the amount of data grows again, of course. Effectively the database is always the largest size it ever needed to be, but no larger. Most of the time, that shouldn't be any issue. If you expect huge spikes, there are other ways around it.

</comment></comment></comment></comment><comment><USER101> This entirely depends on the domain and data model, and unless you need SQL - mongodb should be a consideration too.

<comment><USER21> No. MongoDB has a track record of losing data and not actually providing durability. It is considered a joke in the professional community.

<comment><USER102> interesting: [https://www.mongodb.com/jepsen](https://www.mongodb.com/jepsen)

A lot of MongoDB users who wouldn't be considered a "joke" [https://www.mongodb.com/who-uses-mongodb](https://www.mongodb.com/who-uses- mongodb)

Rather than listen to unqualified nonsense, they run tests. And gather facts to guide technology decisions. and yes, I work for MongoDB - just before anyone starts whining about disclosure You're welcome

<comment><USER21> I posted my problems with it as a response to a sibling comment. Since this was four or five years ago now, maybe all of this has been fixed, but that's not something I'm going to depend on in production any time soon.

Also, the MongoDB users you list must be taken with a grain of salt. For example, you list Facebook on that page. Nope. The description is "...adapted the storage engine API, extending MongoDB into new workloads and new capabilities." That is, Facebook acquired Parse, which used Mongo. Facebook ported MongoDB to run on RocksDB. Facebook shut down Parse. End of MongoDB at Facebook. You list Google, and the mouseover is basically that you can run MongoDB on GCE. I'm not saying that Mongo is unusual in this behavior. It's how most people listing "who uses X" build their lists.</comment></comment></comment></comment>


Don't Assume It's Difficult Until It Is - <AUTHOR> http://weblog.jamisbuck.org/2016/1/9/dont-assume-its-difficult.html <comment><USER2> The first part resonate a lot with me. I wanted to learn Russian (because why not ?), I intuitively thought the Cyrillic alphabet would be the hardest part of all, like it's another alphabet and you can't read anything right ? Sounds really difficult...

Actually it's the easiest part of the language because it's 99% phonetic with some few exceptions, the whole alphabet is in reality quite small and it was actually easier to learn how to read Russian than reading English. The hard part is actually more the grammar (with those cases) and the vocabulary more than the reading itself. I still have a very basic level but the alphabet part took like only 4 hours maybe to remember them all.

But it's also the same when coding, the hard parts are not always where I expect them to be.

<comment><USER3> > But it's also the same when coding, the hard parts are not always where I > expect them to be.

As you may know, this also applies to optimizing code. The 'hot' parts of the code are very often things that you don't expect.

<comment><USER4> The hard part of optimizing code is to find areas were your code is weak at and taking too long to do something.

In my best practices and I learned this from a video by Grace Hopper, is to use less code to do the same thing. Less code takes up less memory and usually runs faster.

Just writing code and then compiling it without any errors doesn't mean it is done. You still have to optimize it and do security checks as well as quality checks. You still have to check for bugs and side effects. You have to use it and think like a user would to see how the user would see it.

<comment><USER5> Less code is not really that important anymore (within sensible bounds) Clarity to whatever poor shmo has to maintain it is often much more important than saving 5 lines with some clever bit twiddling hack.

Unless of course you are having performance issues and you have identified this code block as the issue.

<comment><USER6> >> _Less code is not really that important anymore (within sensible bounds) Clarity to whatever poor shmo has to maintain it is often much more important than saving 5 lines with some clever bit twiddling hack._

I don't think many people will advocate reducing line count by stuffing the same processing into fewer lines using clever tricks. What should always be considered a good thing though, is cutting down on the line count by changing the algorithm/strategy, using better suited data structures, by trying to 'do less' and get the same result, not trying to support every thinkable corner case you will never run into, by preferring multiple simple implementations of the same thing for different corner cases, over one single complex kitchen- sink solution, by not re-inventing the wheel and using tried & tested libraries/frameworks, etc.

I know it sounds like a terrible cliche, but any line you don't write, is a line that can't contain bugs ;-). Over the years I've learned that almost always, simplicity is preferred over complexity, even if it implies a little bit of duplication, or an implementation that is 'less awesome' because it 'only' does what is needed for the application.

</comment></comment></comment></comment></comment><comment><USER7> This reminds me of babies. A lot of people assume that changing diapers or sleep deprivation is the hard part of having a baby. For me, though, the hard part is actually just that babies are kind of boring, and the boringness of having to watch them constantly grinds you down.

That definitely wasn't what I was expecting to be hard when considering whether to have a baby.

(Fortunately they get more interesting over time.)

<comment><USER8> I haven't heard yet that babies are boring, but I can imagine it's the grind that is the worst part.

Unrelated:

[https://www.youtube.com/watch?v=JzvwTyESiR0](https://www.youtube.com/watch?v=JzvwTyESiR0)

<comment><USER9> My mum has always said that babies are boring. Not uneventful, but boring.

For the first few months all they do is eat shit and sleep. Literally.

<comment><USER10> Shit has no nutritional value- it's just breastfeeding or formula for the first few months.

</comment></comment></comment></comment><comment><USER11> I feel like this is good advice but presents only one side of the coin. Underestimating how difficult something is, is exactly what leads to ''planning fallacy''. I would rather err on the side of caution and overestimate how difficult something is

<comment><USER12> The most important time to apply the advice from the article is after you have decided a project is worth your time and before you start planning in earnest.

The examples of difficult subject matter the author gives (particularly math) are so intimidating to some people that they miss out on a lot of great stuff because they conclude "I could never do that".

I don't think the author wants people to make poor time estimates and suffer budget overruns.

To me, the message of the article reads: "If you think something looks worthwhile but difficult, take a stab anyway. Often the hairiest, prickliest parts of the undertaking from an outsider's perspective aren't that tough at all once you're in the thick of it"

I 100% agree with you about how valuable conservatism is when you're planning, but I think the author's encouragement is intended for people who haven't even considered planning yet.

<comment><AUTHOR> Thanks for stating it more clearly than I did! You've nailed it. It's always bothered me when people dismiss their ability to learn a thing, because they think some part will be difficult. They really have no idea. Yes, the task as a whole will have difficult parts, but we as individual human beings are far more capable than we give ourselves credit for.

</comment></comment></comment><comment><USER13> Personally, when troubleshooting software, I've always found that stating assumptions, testing them both for false and true values, and writing the results down, makes for good (albeit sometimes slow) forward progress.

Of course, you get those bugs that occur with seemingly zero difference in your methods, and that's when you need to start thinking creatively...

</comment><comment><USER14> This is honestly how I felt about shaving with a straight razor. Everyone I spoke to about it was like "you're crazy! you'll cut yourself. At the very least use a safety razor to warm up to something like that." I had a gift card to Art of Shaving, and bought a shavette.

Two weeks in, I haven't cut myself once, and I shave faster and better than I ever have.

<comment><USER15> you never cut yourself? Parts of my neck were nicked repeatedly in the first month. Most interesting part was that not once did it actually hurt, it was just that sharp.

Still though, the blade price alone makes the effort worth it. Plus I had a few solid months of a feeling of superior manliness, and the glory of using a shaving brush has yet to wear off in the last year and a half. How did I go years without using a shaving brush?

<comment><USER16> For about the last year that I was in the military, I started using a shaving brush and safety razor. It was the best, though I usually had to shave twice (down then up) to get the closeness required of a military shave. Reading this and thinking back, perhaps I should have tried a straight razor. I never shave anymore so perhaps it could be something new to try.

</comment></comment></comment><comment><USER17> Good lesson to learn =/</comment>


Show HN: I converted my tech interview study guide into a newsletter - <AUTHOR> https://thedailycoding.com <comment><USER2> Man this looks good! Subscribed to your newsletter. Hope this helps me in understanding those tricky sorting algorithms. Looking forward to the emails!

<comment><AUTHOR> Thanks! I hope it helps you in someway.

</comment></comment><comment><USER3> Whats up with everyone starting a newsletter these days? Congrats anyways, looks good. Looking forwards to it.

<comment><USER4> Making use of time we're not wasting, I mean, spending commuting? I am all for it and would do one myself if I thought I knew something worth sharing. Would a "Bumbling into AWS Competency" be a worthwhile topic?

<comment><AUTHOR> Are you looking to share random aws content or follow a course walk through content?

</comment></comment></comment><comment><USER5> Congrats on the newsletter. Looks great!

</comment><comment><USER6> looks good, hope i follow it regularly ;)</comment>


AI aims to save kids from shooters - <AUTHOR> https://www.fastcompany.com/90214257/ai-gun-detection-school-shootings <comment><USER2> > While legislators and advocates wrestle over gun laws, a growing list of > companies are joining Shielded Students to fill school security gaps.

Gross. While we refuse to adequately address the problem of gun control, and while education continues to see its budgets defunded to the point where kids are media-illiterate and the Bible is considered a textbook, let's sell school districts million-dollar magnetrons that pretend to detect guns. "We need more training data" will be a common refrain muttered over the bodies of dead kids for a decade to come.

My kids' school had two lockdown events last year thanks to armed fugitives being on the loose in the area. One of the worse local campus shootings we saw recently involved a guy who literally shot his way into the building. "Firearm detected!" alerts add nothing but noise to these situations. We already fucking know he's got a gun. Money well spent.

And of the high-profile cases-- the Columbine kids supposedly had the foresight to call in fake bomb threats to lure police away from the target environment, extending their response time in the face of 911 alerts. The Virginia Tech shooter padlocked all the doors before pulling out any guns-- it was too late for "firearm detected!" alerts by the time he drew. And Sandy Hook wasn't perpetrated by a student, so heavier scrutiny of students would still have missed that.

And once the schools are locked into decade-long contracts with technology that has classified images of every amorphous shadow that could possibly be a gun, if chemistry class hasn't devolved into alchemical lessons on turning water and fish into wine and bread, some enterprising student will realize he can make peroxide-based explosives from common household materials and we start the cycle all over again, with another tech-vulture coming to the rescue to pick the carcass of our educational system clean.

Technology is not the answer. We built a much-too-efficient killing machine, and are trying to counter it with more machines. Trying to engineer a solution out of a social problem will just end up with our schools mirroring the structure of maximum-security prisons-- and contraband _still_ gets into those.</comment>


Splunk your Kafka Data with SQL and no licensing costs - <AUTHOR> https://lenses.io/blog/2020/05/visualise-Apache-Kafka-topic-data-in-Splunk/ <comment><USER2> nice, could be very handy</comment>


Jellies is a kid-friendly, parent-approved alternative to YouTube Kids - <AUTHOR> https://techcrunch.com/2017/11/27/jellies-is-a-kid-friendly-parent-approved-alternative-to-youtube-kids/ <comment><AUTHOR> Techmeme summary: _Sarah Perez / TechCrunch: Jellies launches on iOS as an ad- free, $5/month video service for kids, which handpicks all of its videos and aims to be a safe alternative to YouTube Kids_</comment>


Startup offers landlords continuous, deep surveillance of tenants' social media - <AUTHOR> http://boingboing.net/2016/06/09/uk-startup-offers-landlords-co.html <comment><USER2> Services like these are just the start. It's not going to be too long before background social media checks such as these get absorbed into your Experian credit report and become part of the general information a landlord or anyone else has on you. You won't even know about it or have any recourse if the algorithm gets it wrong.

China is already trialling a 'Social Credit' system that aims to achieve similar. The only difference between China and the West is that as an authoritarian state they can be more brazen about it.

Sadly, it being Britain, there will always be a fair share of people willing to go along with it because they 'probably know what they're doing' and the 'innocent have nothing to hide.'

<comment><USER3> I'm sure this is an incredibly unpopular opinion here, but I really believe that these things can be a net good. At least in some cases, like your example of credit checking.

I recall a startup nonprofit that gave loans to people in third world countries. The main problem was detecting fraud, and the only tool they had was checking social media profiles. Clearly this is a good thing if it maximizes the number of people who get loans. Even if a few good people get accidentally filtered, there is a net benefit. And yet this was really controversial in the HN comments.

There is also a net benefit to having more accurate predictions on who will make a good tenant. Rent is the largest cost of living for the poor in many places, and anything that helps reduce that cost is a net benefit.

Yet people react based on emotions. If a single false positive out of millions causes someone to be worse off, it's considered unfair and bad. Even if the net utility is positive and more people benefit than are hurt. It also rests on some underlying assumption that the existing system is perfectly fair, which is obviously false. Human's are way more biased at judging than algorithms. Bad loans and bad tenants have to be paid for by everyone else, which is hardly fair either.

Social Credit has nothing to do with it. The goal of social credit is to discourage dissent. Whereas credit rating agencies don't care what your political opinions are, just how likely you are to pay your bills.

<comment><USER4> You are suggesting that a background check system is doing a net good if more people benefit than are hurt.

The problem is that a system like this would strongly encourage _all_ people, good or bad, to avoid doing things that the system could object to. It's not enough to count the actual false positives, you have to take this loss of freedom into account as well. I don't think it's a good idea to have people self-censor because they worry how a surveillance algorithm will react to what they do. (The same way that you think twice, nowadays, about using problematic keywords or expressing a dangerously controversial opinion in a text message to a friend.)

<comment><USER5> Exactly. This is exactly why systems like this pose such a threat. And what about the few that decide to say "Fuck the system!", well they become a social underclass - they can't get loans, mortgages, jobs, etc. It's a very slippery slope. Everyone should be entitled to a private life. And not everybody's private life would chime well with 'the establishment'.

I could imagine myself falling foul of this because I produce music. It wouldn't take long on my facebook page to see that, so if a landlord could mine that information then I'd be unlikely to get a tenancy based on my potential to be a nuisance to my neighbours.

<comment><USER6> You gotta be a model FB citizen to rent or loan. FB's got it made, nobody can ignore them any more, even if they want to.

</comment></comment></comment></comment></comment><comment><USER7> That is really, really creepy. I will not pay money to or enter into any sort of business relationship with a real estate firm that wants to be on my Facebook 'friends' list.

<comment><USER8> So what can we collectively do to ensure that businesses like this fail, fail big, and fail publicly?

<comment><USER9> Note who works for/with them and remember that if you're ever in a hiring position (or able to influence hiring)?

Since the people working to put this together clearly don't have the ethical sense to not-make this, at least they might think twice if there's enough understanding that producing flat-out evil might hurt their career prospects.

<comment><USER10> If society collectively decided to refuse them service for making such an invasive service, it's not as if they can complain.

</comment></comment></comment></comment><comment><USER11> For my current job (banking software development) I had to undergo a background check. In addition to the usual work and address history stuff I was also asked for all my social media account details, to which I answered "N/A". I don't want to lie and said I had none, instead I just went with N/A as I felt my social media presence was not applicable to the background check. I have a lot of personal info and pictures of my children on my facebook profile and that is no ones business but for me and my family.

The background check company got back to me with a list of social media account handles they had found, some were mind, some weren't. They told me that they would be submitting this list to my employer stating that I lied about my social media accounts. I responded by telling them that if they did they would be submitting incorrect information about me and I could take action using the Data Protection Act.

I expected to fail the check, but somehow I passed and got the job.

The takeaway from this is that so long as you refuse to discuss your social media accounts and don't use your real picture as a profile picture, don't have a really unusual name and keep everything locked down and private then how can they prove otherwise?

<comment><USER12> They can't prove, but they Know. They also often Think they Know, and this will cause unexpected (from your perspective) repercussions.

<comment><USER11> But what this company is doing is inherently dangerous for itself. The UK Data Protection Act is pretty darn strict about personal data and the Information Commissioner can and frequently does levy hefty fines for mistakes and breaches.

So lets just say I tell them that I have no social media presence and they dig up some accounts and tell the landlord I lied. Well, they have to justify their claim, they have to be able to conclusively prove it is mine. If they get anything wrong then all it takes is a complaint to the Information Commissioner stating this company is holding, processing and passing on inaccurate data about me to give them a little shock. A few too many complains would then probably trigger a severe audit by the commissioner.

Looking at it from a wider perspective. How are they processing and protecting information in general. How do they protect all the data they harvest from your social accounts? Are staff able to share the info for fun? is there access logging? How secure are their PCs? What do they do with your data after your tenancy ends? This is the sort of stuff a business like this cant afford to make mistakes with. If they do and people complain in big enough numbers it could negatively impact their business.

It's also worth noting that where religion, sexuality or health data is involved things get a little bit more serious as regards the DPA.

If I were dealing this company I would be quite happy to act in manner which would let them dig their own grave even if it impacted me.

</comment></comment></comment><comment><USER13> Dear HN Monkeys,

You're cracking me up. You can not be pro-technology and pro-privacy at the same time - these things are mutually exclusive almost by definition. Pick a side already and stop whining. (Aaand here's a tip: the winning side in this will always be technology. Yes, that very same technology that you help create on a daily basis. )

<comment><USER12> I want to try raising a HN admin flag on you - but your voice needs to be heard, you are obviously writing what you believe is right. There are likely many others just like yourself.

First, the obvious; Public Key Cryptography, HTTPs, Digital Encryption Algorithms and their implementations are all TECHNOLOGY and there is a non- trivial number of us HN Monkeys involved in building them as well (I _wish_ I was one, not just yet).

Next. I get your concern. I really do, and I share it. The problem you're referring to is often incorrectly attributed to the proliferation of technology alone - but I think it's a problem of the free market. It's a problem of the commoditisation of Human Attention and Human Data in the quest for ever higher, ever better profits. This is not a new problem, rapacious and horrible entities have been born out of this. My blood boils when I see ads for fertiliser from DuPont on Indian TV. Yes, they're now advertising again as if Bhopal never happened. But hey, profits.

So your anger is misdirected. Please use it as fuel instead to try and build an alternate system. To try and think of ways in which the innate, inescapable human need for increasing individual rewards (today these rewards are expressed in the currencies of Money and Political Power), without compromising the needs of - well - _everyone else_. We need more thinkers and thinking. It's what humans do BEST.

</comment></comment><comment><USER14> If landlords and letting agents and real-estate agents do this, should not tenants do it too? Could we have a service where a tenant gets a "creepiness report" on a potential landlord/agent?

Actually, probably better not. Escalating mutual destruction scenarios are not a useful long-term strategy.

</comment><comment><USER8> This is exactly the dystopia we're willfully pushing ourselves towards. What a nightmare

<comment><USER6> This application is trivial compared to what NSA probably does. They surely have us all profiled, they know our interests and friends. It's very dangerous if they decide to impact on undesirables. These logs could even be used in the future based on the interests of those in power. Not even Eastern Europe in the height of Communism was able to have such detailed, deeply insightful information on every person.

</comment></comment><comment><USER15> > _making tenancy unbearably awful has the additional benefit of encouraging > people to do everything -- anything -- to become home-owners_

There is other option: leave. Prague and other cities have many foreigners from UK.

<comment><USER16> And it's quite unfortunate for them that the UK might vote for a Brexit, without them necessarily being able to chime in. We recently met UK retirees in Spain who were deeply worried. They had not lived in the UK for over 15 years so weren't eligible to vote for or against. And yet their life situation was directly affected by the outcome: If the Brexit occurred, they'd end up unable to afford local healthcare, and would end up needing to return back north.

<comment><USER17> "If the Brexit occurred, they'd end up unable to afford local healthcare, and would end up needing to return back north."

There is already a reciprocal agreement for healthcare and no reason to suppose it would end. Don't forget there are many Spanish people living and working in the UK right now.

<comment><USER18> Given the extent to which Brexit campaigners talk about ending immigration, I don't think it's safe to assume that. After all, deporting all the Spanish people living in the UK would be _great_ for the net migration statistics. And cause total chaos, but that doesn't seem to have registered with the Leave campaign.

<comment><USER17> Is anyone seriously considering deporting all the Spanish people?

<comment><USER18> I'm not sure if they've _considered_ it, but it's the logical outcome of the change. They're not UK nationals and they don't have any kind of visa or leave to remain - _because_ of the EU rules. So if the rules were changed to require visas for EU nationals, what then? Make everyone apply for leave to remain? Do nothing, and watch as people realise that although they can stay in the UK if they ever leave it to visit e.g. Spain they're not allowed back in at the border?

So given e.g. [https://news.ycombinator.com/item?id=11876453](https://news.ycombinator.com/item?id=11876453) , what happens?

<comment><USER17> It is going to be complicated but I don't think it's going to be such a big problem? There will be lots of time to find workable arrangements, it's not like everything changes overnight.

In the case of Spain and the UK there are lots of reasons to maintain a good working relationship.

Of course it may not work out that way but, going back to the original point, anyone who left the UK 15 years ago had to be aware there may be problems further down the line?

</comment></comment></comment></comment></comment></comment></comment><comment><USER12> A Decentralised Web is quickly becoming one of Humanity's most urgent needs.

</comment><comment><USER19> I wonder what approach Facebook will take to services such as this?

It could be a huge source of revenue for them, or they could decide this would damaged confidence in their service and discourage (my instinct says the latter).

</comment><comment><USER20> This is exactly why facebook will fail.

<comment><USER21> I doubt so, big ones such as facebook or google can dedicate themselves to be big umbrella after their main products lose direction/public/profit/all-of- the-above. Anyhow, as much as I'd love to see social media die out I am sure it will never happen

<comment><USER22> Yeah, maybe Facebook won't fail as a company, or at least not soon. But I can get OP's argument. I can see it stop being a cool thing people use to connect to their friends. Once a viable alternative is available.

</comment></comment></comment><comment><USER23> They can have that. I have thousands of friends, and essentially zero information on Facebook, good luck even filtering out my real friends from the Farmville and Mafia Wars (never seriously played, those games are in there exactly because they obfuscate real acquintances).

I've also got real social media accounts that I actually use and that are under a fake name and I regularly verify that obvious google searches don't find them.

You can use google alerts to automate this.

<comment><USER10> For all you know, they penalize people for not being oversharers.

<comment><USER6> > friends from the Farmville and Mafia Wars

Or they find some objectionable friend in his list, by random chance.

</comment></comment></comment><comment><USER24> It's a logical idea.... given today's world and the nature of society / capitalism at this point in time. I am writing an essay "On The Internet, You Should Pretend To Be A Dog". The same basic idea of strip mining a person's social media presence (along with whatever databases of information is handy) and machine learning the heck out of it to figure out sources of information or specific starting points for cracking your passwords (assuming you are one of the subset who base your password on personal information - how many non-IT people do that? - and don't bother paying attention to all the advice out there)

Really it's an essay about good password policies, e.g. for Grandma who thinks her cat's name spelled backwards is hard to guess... yet posts all over Facebook about Meowie.

Considering Mark Zuckerberg didn't bother to use LastPass or memorize a simple sort of random string of characters...</comment>


Extracting a Salt From an MD5 Hash - <AUTHOR> http://bigtrapeze.com/2012/04/05/extracting-a-salt-from-an-md5-hash/

<comment><USER2> "using MD5 encryption (which is what the majority of websites use to encrypt stored data)"

MD5 is not nor has it ever been advertised as being a form of encryption. No article that references it as such can be treated seriously even if you are recommending against using it.</comment>


Why is Twitter Destroying Itself? - <AUTHOR> http://www.theatlantic.com/technology/archive/2012/08/twitter-is-the-webs-best-public-identity-service-151-so-why-is-it-destroying-itself/261497/ <comment><USER2> This Atlantic blog piece engages so little with the question in its headline (it really doesn't engage at all, except by means of a flimsy metaphor in the last graf of the piece) that you have to assume someone else at the Atlantic wrote this headline. That's often the case at big publications; authors write reasonable pieces, and then editors slap crazy headlines on them to maximize views.

An Atlantic blog post that merely pondered the question of "Why is Twitter willing to alienate new Tumblr users" would obviously not drive as many views; people (rightly) don't expect random bloggers at The Atlantic to have a lot of insight into that question, and the only reason you'd want to read someone grappling with that question is if there was some promise that they'd have an answer.

The Atlantic's mostly poor coverage of technology issues is synecdochic of the decline of the whole publication in the wake of its pivot towards online journalism. They built a website with a pretty firm wall between The Atlantic Monthly and The Atlantic (Website), recruited some blog writers, and got some of their correspondants (Fallows, in particular) to write alongside them. Gradually, the blog writers seem to have swallowed the whole place, driving it towards Huffington-esque short-form provocations instead of essays and journalism; this mentality has infected the Monthly as well.

The same fate, of becoming a second-rate version of Slate with a reputation propped up by a second-rate version of The New Yorker, interestingly hasn't seemed to have befallen The New Yorker itself.

<comment><USER3> Agreed on the link bait title, pretty sure you misused "synecdochic". Synecdoche is like metaphor, but the former I've only seen used to explicitly refer to figures of speech, like a simile, while the latter is used interchangeably with "symbolic", so a thing could be metaphoric, but only written or oral communications could be synecdochic.

<comment><USER2> A small instance of a larger concept that does a good job of illustrating the contours of that larger concept; a part for the whole.

(Who the hell would downvote your comment? Weird! I fixed that.)

<comment><USER3> Right, I'm with you there, but I would say that referring to "the crown" in place of the King would be synecdochic (the expression is synecdochic), but the crown, in itself, isn't synecdochic of the king.

<comment><USER2> When people say "the crown" when referencing a monarch and their administration, that's a textbook synecdoche. Similarly, when people refer to the number of "seats" in an enterprise, as with a "10 seat license", that is also a synecdoche; obviously, the licensor could care less about seats per se; you pay even if you use a standing desk. :)

I am unapologetic about introducing this confusing word into the thread, as this discussion is more interesting than the blog post we're commenting on.

<comment><USER3> Similarly to rhizome, I appreciate your use of a word I hadn't seen in a long while (and that HN spellcheckers flag), and I agree this is much more interesting than the article, but I would have gone with something in {emblematic, symptomatic, representative}.

I'm holding firm in claiming that synecdoche should be reserved for referencing oral or written expressions, viz., intentional usages of a part to indicate the whole.

Thanks for a little diversion :)

<comment><USER2> I think the poor quality of their tech coverage, which quality is indeed a "symptom" of a problem, isn't the whole story. The total fact of their tech coverage --- that they feel the need to do it at all at any level of quality --- is more important to me, and so "symptomatic" didn't seem right. But I wrote unclearly enough that "symptomatic" is a better word in the context of my original comment.

I do own the fact that "synecdochic" is never a good word to use in clear writing. :)

</comment></comment></comment></comment></comment></comment></comment><comment><USER4> It seems dumb in the same way that vertical integration is often seen to be inferior to horizontal integration.

If for instance Twitter is doing this to support Posterous, a tumblr competitor, this will probably harm Twitter more than it will support Posterous.

When I heard they cut of Instagram, I figured it was to support TwitPic (or?) which again seems to be a losing move for them.

</comment><comment><USER5> Because they haven't properly understood how to monetize since day 1

<comment><USER2> They are by many accounts significantly (as in, not just pro-forma) profitable.

</comment></comment><comment><USER6> This guy just joined Tumblr on Monday? Doesn't this kind of discredit his opinion?

<comment><USER2> The article doesn't work if he's a long-time Tumblr user, because find-friends is something you typically do when you first join.

</comment></comment><comment><USER7> It isn't.</comment>