Was MongoDB Ever the Right Choice? - <AUTHOR> https://www.simplethread.com/was-mongodb-ever-the-right-choice/ <comment><USER3> I would argue that MongoDB is not—and has never been—the _best_ choice for solving any particular technical problem. But it had some other "advantages" over other, better solutions – in that it was easier to set up, didn't require schema definition, had a passable clustering story etc.

I have worked with at least one company that had been built using MongoDB as a primary data store from day one. This caused untold pain later on, but the trade-off is that it likely allowed the company to exist at all – the founder being more of a domain expert than a technical expert, but being able to use it to scale their idea pretty quickly without having to pay much attention to all that tedious "reliability" and "safety" nonsense :)

That said, it's not something that an experienced developer should be using for anything nowadays, and the solution might be to ensure that competing alternatives (like Postgres) can learn from why MongoDB became popular and seek to solve some of the pain points in their own implementations.

<comment><USER4> I once worked for a company where Lotus Notes fulfilled a similar role (this was back in the late 1990s). Eventually they ran an entire free to sign up web-based email system using 4 giant Notes instances. It was an absolute nightmare, but probably the company would never have happened otherwise because Notes was all that the founders knew. My job was managing the migration off to a normal SQL database, which took the best part of a year.

<comment><USER5> I worked in such a company too.

Every time we wanted to do something new a founder would say "Domino can do this!" and then they would spend the whole weekend setting it up for what we were trying to do.

All because we were some kind of IBM partner and they wanted to please the big wigs.

<comment><USER6> Once upon a time in a job far away, about 15% of my time was allocated to being the company sysadmin, and as all good sysadmins do, reduced that time to about 5%.....until....

One day, and despite my protests, I had to bin our perfectly fine and well maintained/loved Exchange server for fricken Notes/Domino just because one of the investors had some free licenses. After the migration, everyone hated it and I was persona non grata seeing as it was me who had to switch us over. It never worked properly. I left not long afterwards and the poor sod who was my replacement had the joy of looking after that dumpster fire.

edit: opening sentence

<comment><USER7> I wonder if that poor sod will be an HN reader too, and end up seeing your post

</comment></comment></comment></comment></comment><comment><USER8> I went to production with Mongo and it exceeded my expectations. It operated without a single issue for years under high load. It was a dream to administer and a vital piece of a multimillion dollar franchise.

Everybody else I know, however, has had nothing but headaches.

They're the same reasons I've seen people fail at using Cassandra, Redis, or Spanner. If you can't adjust to the limitations and paradigm shift, you get no benefits. And an ORM often makes everything worse.

The "no" in NoSQL doesn't seem to stop people from modeling join relationships in Redis, or chaining distributed queries with fully consistent writes on Cassandra.

I'm on a project right now, where a developer has selected an ORM for PostgreSQL that forgoes joins. They've managed to generate about 100 queries, in one case, where a single query is all that's needed. 2ms vs. 800+ms. That individual is incapable of using something more complicated. Substituting Mongo as-is will make everything worse, and they'll triumphantly proclaim how terrible NoSQL is and then write an article.

I feel confident using Mongo for any task. I don't feel confident letting most of my peers use it.

There are a LOT of legitimate gripes, but no article I've read mentions them. It's always the same superficial complaints from ten years ago. If you can't get past those, choose another tool. End. Of. Story.

<comment><USER9> I'd really like to read an article written by you talking about Mongo limitations.

</comment></comment><comment><USER10> Since everyone is sharing their opinion and experience with mongodb I think I'll share mine.

As an appeal to authority I would like to mention that I have relevant vocational qualifications on the subject (more geared towards scalability and operations). Although I don't believe it really matters - it will to those who assume I don't understand best practice.

MongoDB itself is not /really/ a valid choice in many scenarios that it was painted as solving. Their only fault is overzealous marketing, it has (in my opinion) very clear pain points that should be avoided, but those painpoints are antithical to why many people used it in the first place.

Most people pick up mongo because it's painted as being "beginner developer friendly", I don't mean new developers, I mean picking it up and running with it, without understanding it, was made to be incredibly easy. But MongoDB itself needs you to understand your data patterns before you start adding shards, so the technology itself depends on you actually sitting down and designing an architecture while understanding that. These goals are at odds with each other.

In MongoDB (as it was when I was using it in full prod 6+ years ago) you -needed- to understand how your data is going to grow and how it will be queried long before you ever created an index. You could not grow after creation. But using it as a plain document store with no searching and heavy sharding on the document ID is the best way to go. And in that scenario it is much better than most competitors.

In nearly every /other/ scenario its a less favourable choice than another technology of some variety.

I would argue the data loss point but I think if that's not a solved issue it will be, and I'm fairly certain you can configure it to be slower but correct (my memory is bad).

I am not a MongoDB advocate, nor do I hate the technology outright. I strongly dislike how it was marketed as being a panacea.

And for the same reason I avoid PHP, I will attempt to avoid MongoDB.

(As in; it can be done well but the majority of cases will be poorly implemented)

<comment><USER11> I'm a big fan of Mongo for the use case you described - searching by ID and all information in one document.

But people don't seem to understand that there are plenty of scenarios where you really either don't know the schemes in advance and/or the "schema" is defined by an external source.

I worked for a company that sold software that allowed users to create forms that could be filled out either on the web or via a mobile app.

The user created the form and the schema and the indexes were created on the fly - one collection per type of form. What would an RDMS have bought us?

<comment><USER12> PostgreSQL's JSON columns are pretty powerful

<comment><USER13> And if you create a table with a single ID column and a single JSON column you've essentially re-invented a NoSQL database. But I guess you can pretend it isn't.

<comment><USER11> And it's a lot worse and the tooling isn't as robust...

</comment></comment></comment></comment></comment><comment><USER14> As with anything, it depends on the project. I'm working on an internal service that uses Mongo as a single merged cache for a lot of mostly unchanging data from various data stores with different credentials for each, which are distributed around the world, that we otherwise have to fetch through multiple comparatively slow API calls. For this, Mongo is perfect: no messing with schemas as they change, unannounced, from upstream; I can index just the fields I want to search on; Mongo will expire things for me on a TTL; the query API is simpler than the API we're caching from; and we get results 20-50x faster. We looked into FoundationDB and Postgres, but they require a lot more initial setup. ElasticSearch is the closest solution, but it needs a lot more info about the schema up front and its query language is a nightmare compared to Mongo's, for no real gain in functionality that I can see.

Is Mongo the right tool to build your entire business on top of? Probably not, but it can be the best tool for the right job.

<comment> james-mcelwain Elasticsearch can index documents dynamically, and doesn't require a schema to create an index. Dynamic data types for fields may not always produce what you want, but it's possible to define a partial schema for the fields that are important and let Elasticsearch handle the rest.

The query language is verbose but I would hesitate to call it a nightmare. You can always search using the Lucene query language, and SQL support is landing sometime soon.

<comment><USER15> The query language can be a bit quirky if you're coming from a SQL background, but you're absolutely right. It is nowhere close to a nightmare.

</comment></comment></comment><comment><USER16> My anecdote: several years back I had a test lab for measuring the performance and scalability characteristics of various geospatial databases. We added MongoDB to the mix a couple years after they released geospatial support.

We always verified basic correctness with a new database by inserting several billion geometries as fast as the database would accept them, reading the entire data set back out, and comparing it to the original data set we inserted for any discrepancies. MongoDB never passed this test. It would apparently lose records semi-randomly every time, so we removed it from the test set. It was the only database we tested that had this issue.

<comment><USER17> I evaluated several distributed databases for a healthcare-related system. The ability to lose messages in sharding scenarios, and the specifics of how one would recover them, made me think I could never support MongoDB for anything more serious than Reddit.

<comment><USER18> The Jepsen tests [1] have been run against MongoDB - while older versions presented edge-case opportunities for data loss, that's no longer the case with recent versions. The Jepsen tests also specifically test sharded clusters. From Aphyr's report:

> MongoDB 3.6.4's sharded clusters offer comparable safety to non-sharded > deployments.

These tests are now integrated into MongoDB's regular test suite. Maybe MongoDB wasn't the right choice for you at the time you were evaluating it, but I just want to point out that MongoDB has matured and improved a great deal.

(Disclaimer: I work for MongoDB)

[1] [https://jepsen.io/analyses/mongodb-3-6-4](https://jepsen.io/analyses/mongodb-3-6-4)

<comment><USER16> FWIW, the data loss I was referring to happened on a single node. There may have been issues in a distributed environment as well but we never got there.

<comment><USER18> When did you perform the test? Can you remember what MongoDB version you were using?

</comment></comment></comment></comment></comment><comment><USER19> I'm still in charge of a production system serving around 2,000 small to medium websites from a 2-machine MongoDB cluster. It's been running on MongoDB since around 2010 and we have NEVER had any issues.

I accept that the unacknowledged writes was a bad decision, but IMHO if you deploy a new database without reading the documentation, you have bigger issues.

The reality is that there are some places where speed of movement is important and referential integrity is just not that big a deal. We're not all building banking systems.

<comment><USER20> 1\. Writing DDL is not hard. It's just not very hard.

2\. You can go from strict guarantees to looseness safely, when you demonstrably need to. The reverse isn't true -- it's easy to wind up realising, much too late, that you actually needed particular guarantees that you didn't even think of.

Relational databases didn't become incredibly popular by accident. It's because they were a drastic improvement -- theoretically and empirically -- on the generation of NoSQL databases that preceded them.

<comment><USER11> Writing DDLs are hard when you don't know in advanced what the schema is or the schema is changing frequently.

A lot of things "become popular" but that doesn't necessarily mean they are good.

<comment><USER21> All the SQL databases with which I'm familiar support large string columns. What's the downside to just stuffing your schema-less or volatile-schema data in one of those?

<comment><USER11> The downside is that you can't easily query against the individual fields and you can't index individual JSON fields.

I'm bringing up C# again...

Querying with Mongo using the Mongo driver in C#....

      var people = db.GetCollection<People>().AsQueryable();

      var seniorMales = from p in people where p.Age >= 65 && p.Sex == "M" select p;

Querying an RDMS with EF:

      var people = context.People;

      var seniorMales = from p in people where p.Age >= 65 && p.Sex == "M" select p;

Both queries get translated to their respective query languages and run on the server. C# enforces the types in either case and you get compile time type checking and IDE Intellisense. In either case you can index the Age and Sex fields.

NoSQL databases aren't "schemaless". Mongo understands the schema of JSON data and can query against it just like an RDMS understands rows and columns.

</comment></comment></comment></comment></comment><comment><USER22> It is still in production as the main database in a startup I joined in 2010 and kicked off their software's development. I've been through smaller ups and downs, but we never lost data. The reason why I loved it back then and still do today (I'm using mongodb in my own little webapp) is the speed of development. The few data migrations that I had to write in over eight years where nothing compared to what you'd have to do in a relational database. With new features we were always able to keep the db schema in some state of fluidity on our dev and staging machines until we we happy with the data's architecture. No back and forth that you'd do with a relational database. We have only a dozen collections in the db and only four where pulled during a normal user session but those translate not just into four models/classes but a few more embedded ones, which makes totally sense because except for reporting we didn't pull out the embedded data, even though it has become so much smoother with the aggregation framework.

<comment><USER23> It sounds like your product was in the sweet spot where there isn't much data complexity or evolution, and the schema is understandable and easy enough to modify over time without strictness.

The product I work on has ~400 tables active at the moment, and ~800 that have ever existed in it over the last 6 years. We depend on the database schema to reduce complexity at the application level.

I don't think the complexity would be manageable at the application level if we couldn't do this. At least not with the ~6 engineers working on it.

<comment><USER22> That's about six times the engineering power you have there.

Our data is indeed simple and beside data collection via a web app, the focus is on reports with that data; but always in small sensible junk and never across the whole available data for a model.

</comment></comment></comment><comment><USER24> I always supposed that document databases were based on object databases from the early 90s... And that the "NoSQL" craze was simply a continuation of that progression. I mean there is a difference between NoSQL and no schema. The early object databases had schemas. The reason they wanted to abandon SQL was because they believed that they never wanted relational data: they just wanted a persitance layer for their business models.

I might be wrong about that because I pretty much ignored DBs in the late 90's and 00s, so I never really followed what was happening. However, one thing I'm absolutely sure about: MongoDB had very few features that were really novel. I'm not really sure why people think everything started there (especially someone who knows about 4GL ;-) )

<comment><USER25> Mongo's debut was well timed to ride the JSON crazewave.

<comment><USER26> Sorry for my ignorance, but _what_ JSON craze?

<comment><USER27> As a serialization format, JSON is horrible. Way too loose regarding formatting and way too much noise. All that is typically needed is a standard text format for relational data. I.e. a fixed CSV.

The only "advantage" to JSON is that it maps directly to the object trees most developers use in their scripted programs (which is misguided IMHO).

<comment><USER28> Not sure I would call CSV a "standard text format" \- I've seen many many problems over the years with badly formed CSV files and bad Unicode handling. CSV appears to be an "almost standard" where 98% of the time it is fine and the remaining 2% are an utter nightmare.

<comment><USER5> Haha, yes I remember building my first HTTP-API in 2010.

It had to deliver three kinds of formats: CSV, JSON, and XML.

JSON was a one-liner, then came CSV and then XML.

But all the consultants working in customer projects said CSV was the most important one because it's industry standard.

When the consultants sent me example CSV files from like 5 customers I couldn't believe they all had a different format.

</comment></comment></comment></comment></comment></comment><comment><USER29> These "caveats" are mostly false, or outdated.

> Loss of transactions

MongoDB 4.0 supports ACID transactions: [https://www.mongodb.com/transactions](https://www.mongodb.com/transactions)

> Loss of relational integrity (foreign keys)

> Having a database enforce these relationships can offload a lot of work from > your application, and therefore from your engineers.

I've never seen anyone use MongoDB without also using something like Mongoose where you get all this for free. Zero work for your engineers. [https://mongoosejs.com/](https://mongoosejs.com/)

> Lack of ability to enforce data structure

Again, Mongoose. The work doesn't fall on your engineers, it falls on an awesome, heavily-used, heavily-tested library.

> Custom query language

Can you give an example of something that you realistically would want to do in SQL that you can't with a JSON query?

> Loss of tooling ecosystem

This might be the only valid caveat in the entire article.

<comment><USER30> The problem with thinking a library like Mongoose satisfies those constraints is that it only works with a single application. Every database I've worked with that made it beyond a prototype outlived the original application. So you either end up building an RDBMS app/api in front of mongo for every new application to use or you have to translate those constraints to every client.

I'm of the opinion to let the "awesome, heavily-used, heavily-tested" RDBMSs built to manage data and constraints do just that.

<comment><USER31> True, using a library to hide the limitations of a DB smells bad. It is fixing the problem in the wrong location.

I may have misread your comment but rarely do more than one client application access a database. Especially in today's often Microservices architecture. So using a library as mentioned is fine. And as the app evolves/gets rewritten you can keep the library or replace it with a similar one.

Over 10+ years ago I did join a few projects with existing architecture that had evolved to multiple applications accessing the same database. And they were a nightmare but that is rare to encounter these days as most people have learned that one app = one DB. (Data warehousing is a possible exception, but mostly those datasets are exported instead as well as streams for machine learning).

Probably preaching to the choir but multiple clients mean feature freeze/deadlock, no DB refactoring and spaghetti architecture.

<comment><USER21> > rarely do more than one client application access a database

That has not been my experience!

</comment></comment></comment></comment><comment><USER32> > Bandwagon effect – Everyone knows this, and yet it is still hard to fight > against. Just make sure that you're choosing a technology because it solves > real needs for you, not because the cool kids are doing it.

> Mere newness bias – Many software developers tend to undervalue technologies > they have worked with for a long time, and overvalue the benefits of a new > technology. This isn't specific to software engineers, everyone has the > tendency to do this.

> Feature-positive effect – We tend to see what is present, and overlook what > isn't there. This can wreak havoc when working in concert with the "Mere > newness bias", since not only are you inherently putting more value on the > new technology, but you're also overlooking the gaps of the new tech.

This is so much what I've always wanted to and tried to say on the topic. Adopting new technologies thoughtfully rather than reflexively is something I'd love to find out about a team before joining one again.

</comment><comment><USER18> Great article - it's definitely vital to fully understand the tradeoffs of any technology you choose, and not just going with the latest fad.

I work at MongoDB, so I do want to mention that MongoDB has changed a lot over the past few years. MongoDB now has multi-document ACID transactions. Also, schema validation means you can enforce a strict data structure if you want. On top of that there have been improvements in data durability and consistency (the Jepsen tests are now integrated into the MongoDB test suite).

As mentioned in the article MongoDB has matured a great deal. It's far more mature and fully-featured in 2019 than it was in 2012.

</comment><comment><USER33> I hate these kinds of articles, however, I do recognise that people are allowed their opinions, which is good. But these sorts of articles take a very polarised view of the world.

> Does this tool solve a real problem for us > Do we thoroughly understand the > tradeoffs

Well, no. But neither did running JS on the server (along with numerous other similar examples). I've used MongoDB plenty of times before (in production with 1 issue to date) and I absolutely will again.

<comment><USER21> > running JS on the server

Huh? Enough people apparently knew JavaScript well enough that running it on a server was really useful. Why's that not a real problem? (Not being able to do so.)

</comment></comment><comment><USER34> In my previous job, at a payment service provider, I wrote a "velocity" system. This was required to block transactions that met certain conditions within a certain time frame. A typical example was: block transactions that are from the same IP if that happens more than 5 times in a 10 minute period. Or: block transactions from the same card number if it is used more than 3 times in a minute.

The problem was that the "velocity" blocking requirements could be completely arbitrary, and could be anything a merchant required as long as they used any of the data that was available from the transaction/previous transactions - block a transaction if it's more than 500 GBP and we've had 3 other transactions greater than 500 GBP in the same country in the previous 10 minutes.

This essentially meant we would either have to do dev work whenever a merchant had a new "velocity" rule, write our own complex framework to handle the addition of arbitrary rules, or just stuff the data into a schemaless NoSQL store and then leverage the power of the engine's query syntax as part of the merchant's configuration.

We went for the schemaless NoSQL solution and we used mongodb v1.6 - <USER2> 2010 IIRC, before it reached peak hype, before it "fixed" a lot of the out of the box defaults, before it got a lot of hate. It worked perfectly and ran from the time we deployed it until i left the company a few years later. Maybe I left a mass of technical debt, but the solution ended up being so simple and so little code I doubt it.

The other nice thing was that the velocity system was not essential - if the time to do a velocity check took more than 0.05s we would ignore it. If the backend wasn't responding we ignored it. If the write to the storage failed it didn't matter. We didn't need to keep more than a couple of hours worth of data. If we lost all of that data it didn't matter.

I don't know if I'd take the same approach now, nine years later, but at the time the use of mongodb worked perfectly. It was only a couple of weeks work, and solved the problem elegantly. So in response to the original question: yes, but as usual RTFM and make sure the pros and the cons fit your use cases.

<comment><USER35> If you have almost no guarantees to provide, almost any solution would have worked.

In other words, you claim it worked for years, but anything would if you can lose all data, reboot it, have no real-time requirements, etc.

In summary, it is not really a useful data point on whether MongoDB is useful or not.

<comment><USER34> > If you have almost no guarantees to provide, almost any solution would have > worked.

Within reason. If the failure rate is 1 in 1000 then it's probably not as a big a concern as 1 in 10 _depending on your use case_

> In summary, it is not really a useful data point on whether MongoDB is > useful or not.

As I said in the last sentence - it worked perfectly. In other words the "no guarantees to provide" never actually came up and I don't recall us having any data loss, response time, or write issues.

</comment></comment></comment><comment><USER36> I used it before I understood anything about databases, and I got some jobs done with it. Now that I have some knowledge, I choose PostgreSQL for most cases. But mongodb got me up and rolling in a pretty DRY way when I first started programming for the web.

If you get to the point where your website is outgrowing it's initial mongodb implementation, you have a very good problem on your hands.

<comment><USER37> It doesn't take a lot of traffic to outgrow a poorly implemented mongodb structure with a lot of data. That's not a fault of mongodb as much as it is using the tool wrong. When I first started using mongodb I treated it like a relational database and hit performance issues very quickly.

</comment></comment><comment><USER38> Man, I remember the Mongo craze. You couldn't have a rational discussion about databases with some of the most rabid supporters. I made a comment on a post yesterday about tech having those few early and loud supporters who shut down any conversation that doesn't support their chosen technology. The Mongo craze was like that early on. It was really frustrating not being able to look at other databases when everyone was chanting "Mongo! Mongo!" Thankfully tech has moved on to Kubernetes and React and microservices and now we can finally have a rational discussion about databases.

<comment><USER39> If you haven't watched it recently, this was always my favorite look at "MongoDB is web scale": [https://www.youtube.com/watch?v=b2F-DItXtZs](https://www.youtube.com/watch?v=b2F-DItXtZs)

</comment></comment><comment><USER40> Off course it was, prototyping speed on mongodb was (and probably still is) always excellent.

Some features that don't scale are very nice to have when you don't have scaling issues. For example, if you add tags to your documents and you want to query on those tags (find all documents containing tag A and B), it's nice that's just a builtin.

I haven't found a single datastore that is as developer friendly that supports that use case, so for now, I'm sticking with mongodb for my pet project.

(if you know of a datastore that has support for this query out of the box, please let me know)

<comment><USER3> The other answers have confirmed that Postgres will do this with array fields, and it's good advice to follow. It's also in my view much easier to read than MongoDB's query language is!

      CREATE TABLE documents (name text, tags text[]);
      INSERT INTO documents VALUES ('Doc1', '{tag1, tag2}');
      INSERT INTO documents VALUES ('Doc2', '{tag2, tag3}');
      INSERT INTO documents VALUES ('Doc3', '{tag2, tag3, tag4}');

      SELECT * FROM documents WHERE tags @> '{tag1}';
       name |    tags
      </comment></comment><comment>+</comment><comment></comment><comment>-
       Doc1 | {tag1,tag2}

      SELECT * FROM documents WHERE tags @> '{tag2}';
       name |    tags
      </comment><comment>+</comment><comment></comment><comment>-
       Doc1 | {tag1,tag2}
       Doc2 | {tag2,tag3}
       Doc3 | {tag2,tag3,tag4}

      SELECT * FROM documents WHERE tags @> '{tag2, tag3}';
       name |       tags
      </comment><comment>+</comment><comment></comment><comment></comment><comment>
       Doc2 | {tag2,tag3}
       Doc3 | {tag2,tag3,tag4}

Postgres certainly isn't perfect, but it's usually a good answer to "how do I store and query data" where you don't have any particular specialist requirements.

<comment><USER41> Using this almost 1:1 in a production app where customers can filter by tags. Works great.

</comment></comment><comment><USER42> I've only ever heard of mongo successfully used for two use cases:

1\. As a cache, like redis 2\. As a log store, like Elasticsearch

In both cases, the data is somewhat ephemeral, and not the "source of truth" for the app. The minute it is used for holding real, customer supporting data, things start to get dire real fast.

<comment><USER18> Plenty of people are successfully using MongoDB for real, customer supporting data at a large scale. There's a selection of users on the website for a start: [https://www.mongodb.com/who-uses-mongodb](https://www.mongodb.com/who- uses-mongodb)

(Disclaimer: I work for MongoDB)

<comment><USER43> But we'll never know if those companies will be better off if they hadn't. You're just making a bandwagon argument

(Disclaimer: I hate MongoDb)

</comment></comment></comment><comment><USER44> I found developing with MongoDB a pleasure, until its lack of transactions became problematic. Fortunately, the project I was working on didn't go very far.

At the time, I concluded that MongoDB was the "Visual Basic of Databases." It was very easy get something simple running, much like Visual Basic classic was.

Quite honestly, something ACID-compliant with a MongoDB-like API is really needed for small-scale projects and prototyping.

<comment><USER45> It's called MongoDB. ACID multi-document transactions, here, now, introduced in version 4.0. [https://www.mongodb.com/transactions](https://www.mongodb.com/transactions)

<comment><USER46> Plus the feature, minus the reputation.

</comment></comment></comment><comment><USER47> I'm really glad this "Mongo all the things!" trend is reversing.

</comment><comment><USER48> After using mongodb for many years, I haven't run into a use case where I should have used it over SQL. The only benefit is rapid prototyping for me personally and it initially felt awesome to have mongoose in node be the schema as it was super easy to modify. But you pay for it big time as you go and find that anything difficult means you'll have to do backflips and more requests to get things done and get used to very strange and verbose nested json queries. It's the 10-15% use cases that are really hard. NoSQL is especially annoying when needing any type of joins because despite your best planning, it still happens. There were many cases I needed to fallback on postgres to get stuff done and have a second source of data to sync and then I was wondering why I didn't just do it all in postgres first.

</comment><comment><USER49> I feel like Bill Murray in Punxsutawney. Didn't we just see this article yesterday?

[https://news.ycombinator.com/item?id=19492562](https://news.ycombinator.com/item?id=19492562)

Here was my comment from yesterday morning:

 _> Question 1: What problems am I trying to solve?_

I wish I had really thought about this when I first wrote my largest web scraper. At the time, I was still relatively new to database design and programming in general. This web scraper, out of the thousands I've written in the interim, is--of course--the one that is still going strong many years later.

I eshewed MongoDB for all the reasons given to me on the internet and, because I was slowly gaining competence with SQL, ended up building a large and complex pipeline to send the data right into Postgres. In retrospect, this was a serious design mistake, and one that I regret the most.

Although I still contend that the data did eventually need to be normalized, I now believe that I was doing it far too early. By ingesting the JSON stream into a parser, splitting it up, generating foreign keys, and then forcing the whole works into a single Postgres database I severely limited the capacity of my web scraper (and also guaranteed the need for a very powerful server to run it).

Had I initially dumped all results into MongoDB (or some other efficient document store) and then, separately, parsed the output into normalized SQL, I would have dramatically simplified the operation, maintenance, and debugging of my web scraper. Plus it would have been much simpler to spawn work jobs on to different machines instead of trying to break up huge monolithic processes with poorly defined endpoints. There have been many lessons learned.

In short, Mongo likely serves a very good purpose for high-speed data storage and manipulation (although it's hardly alone in this space). However, it's still likely not a great all-around solution and works best when supported by an ACID-based normalized RDBMS. Unless, of course, things have dramatically changed in recent updates.

</comment><comment><USER50> It's been over 10 years and CouchDB has not let me down once. Still using it in production everyday.

<comment><USER51> I also really like CouchDB and can attest to it being a reliable part of the stack. I'm using it on a personal project (Node/Nano/Couch) that has the potential for needing to store a lot of data but hasn't gotten there yet, so I can't speak from experience yet on performance/scalability. It has been so far great in production and also great for the normal CRUD parts of this project.

The main reasons I chose Couch over something relational like MySQL was essentially 1. to have a a clear path for data to go as JSON from server/API/client without the need for mapping, 2. Schemaless to allow for quick iterating and development, 3. Easy to start with local first development and then rollout for deployment. Also, I hadn't worked on a stack with a persistence strategy that was solely document oriented so it was a fun and good learning experience.

A few things I learned along the way:

I still needed to create externalized "views" of the data that either combined data from multiple documents and the need to hide private data. I still needed to serve lists of data in pages. More importantly I needed to provide a lot of adhoc reporting over the various data I'm storing across multiple Couch dbs.

The views and paging are all easily solvable with Couch, but so much easier to implement using SQL and it feels like I'm just pushing that need or compensating somewhere else in the implementation. But the friction around quick reporting has made me second guess choosing Couch/document based vs. something like a MySQL/ORM.

The author's point of the custom query language (Mango for Couch) and loss of tooling ecosystem has been the biggest problem for me on this project and I'm considering migrating to a Node/Sequelize/MySQL stack just to avoid wasting future cycles trying to quickly report on the data I'm storing. When the project started the reporting aspects weren't as apparent as they are today since the project has evolved and other requirements became necessary.

If anyone has any experience or recommendations for tools that can easily do adhoc reporting against CouchDB or documents in general I'm interested in hearing about them.

<comment><USER50> I've had decent success so far with CouchDB 2.0 and Mango for performing ad hoc queries.

I believe Mango will automatically create the view index on the fly for any query you make and save the index for later. Before CouchDB 2.0 this would usually have to be done manually before the query was run.

</comment></comment></comment><comment><USER52> > Feature-positive effect – We tend to see what is present, and overlook what > isn't there.

This is such a problem. Just yesterday someone came to me and said, "We're now using x to do y. It makes z a lot easier. Just wanted to let you know so you can make the appropriate changes to your analysis & reporting applications."

Say what now? I'm embedded within the operational unit. I know _a lot_ about day to day operations in & out of the systems in use, and how they ultimately translate through in data. I asked half a dozen questions, and 4 of them were met with "oh, we hadn't thought about that." These are deal-break questions, things the current method handled without issue, so much so they became invisible to the users, until they decide to make a change and realize the new method doesn't make any provision for them.

</comment><comment><USER53> MongoDB with it's JSON model helped us develop features and iterate much faster than the previous MySQL/PG. So it was a huge win for use (but we had some pain with some issues).

Today we use PG everywhere with JSON, though still the library support for JSON in Mongo is often a little bit better than in PG drivers.

</comment><comment><USER54> The really interesting part of the article comes after the "What could have been done differently?" headline. I think the article would have been better if it wasn't focused as much on MongoDB because now everybody is discussing if the caveats still apply or ever applied.

</comment><comment><USER55> Could this article title be rephrased "Has Postgres essentially killed the MongoDB market?"? ;)

</comment><comment><USER56> If you need limited document storage capabilities I'd recommend just using S3 alongside a traditional RD like PostgreSQL. You can simply store object IDs in the primary database but the actual document/data in S3. I used this methodology for a cloud platform I built that required the ability to store large 3D models uploaded by users. Metadata was stored in PostgreSQL then the actual data in S3. This also facilitated generating a acquisition URL on the server which could be triggered client-side so that after initial creation, there was almost no primary server overhead (bandwidth or storage) for retrieval.

</comment><comment><USER57> MongoDB was great to me for prototyping, either manually or with tools like Meteor.

<comment><USER58> Yeah, Meteor was pretty amazing when it first came out. It felt like it was from several years in the future. As I recall, it relied on mongo on the server and a minimongo that ran in the client. It made it trivially easy to create universal real-time webapps, which was a game-changer for me at the time. Reactivity has since become mainstream, but at the time it represented a bright line / step function increase in both DX and UX.

</comment></comment><comment><USER59> The main use case for MongoDB I have seen are custom forms or data that can be nested a variable number of times.

The main problems I have had with MongoDB were that, as of several years ago, it did not integrate well with most (I would argue any in practice) data reporting / displaying third parties.

Also, some of the more advanced queries were not at all intuitive. In fact, I barely remember any of the syntax now. In Mongo's defense, that might have to do with the fact that we atempted some stuff in MongoDB that we would never attempt to do with SQL cursors.

<comment><USER60> > data that can be nested a variable number of times

I'm far from a SQL expert and have mostly done client-side work in my life, but SQL-based database does seem like a good match for things like these. You just need to write more foreign key relations, and constraints are a bit more complicated, but I see no reason to switch to NoSQL for things like these.

</comment></comment><comment><USER61> While I think MongoDB is deserving of a lot of the criticism it receives I think the bigger issue is with those who adopt it without a full understanding of their data and the use cases they might need to implement. There are countless stories now from people who assumed their data wasn't relational only to discover much later on that the opposite was true. There are cases where a document oriented datastore could be the way to go but everything I've read seems to suggest that they're in a minority.

<comment><USER29> There's no reason you can't use MongoDB with relational data.

Foreign references? They've got them. [https://docs.mongodb.com/manual/reference/database- reference...](https://docs.mongodb.com/manual/reference/database-references/)

ACID transactions? They've got them. [https://www.mongodb.com/transactions](https://www.mongodb.com/transactions)

<comment><USER61> But why wouldn't I just use Postgres instead?

<comment><USER29> But why wouldn't you just use MongoDB instead?

</comment></comment></comment></comment><comment><USER62> Great for prototyping, and cases where you can contain everything in one document, but as soon as you're referencing _model2_ from _model1_ , it's probably time to switch to a SQL based DB.

I know MongoDB supports references, but it's like using a flathead driver for a Phillips head screw.

</comment><comment><USER63> Very timely Show HN post: we just released Mongoku, a neat Web-based interface: [https://news.ycombinator.com/item?id=19500859](https://news.ycombinator.com/item?id=19500859)

</comment><comment><USER64> I've said it before, and I'll say it again: Mongo:2010s::MySQL:2000s.

</comment><comment><USER65> mongodb has its advantages and disadvantages. just like other databases.

</comment><comment><USER66> If one researches the company's history, MongoDB came from a specific need the founders had for another product: ShopWiki.

ShopWiki is a shopping price listing site, where it keeps tracks of prices for any item: computer, clothing, food, etc. ShopWiki needs to be able to store, access, AND search amongst all of these different items. So, MongoDB is the perfect solution.

If the application being built has similar requirements to ShopWiki or a retail site where it sells _everything_, then MongoDB IS the right choice, because the founders basically built MongoDB for ShopWiki.

To sum it up: Is MongoDB the right choice? If the product is similar to ShopWiki, then yes.

<comment><USER67> Untrue. MongoDB arose from the frustrations the founders had with scaling SQL databases while building DoubleClick.

</comment></comment><comment><USER68> There seems to be a lot of backlash over using mongodb recently. Which current hip tech will we see receiving the same treatment? graphql? golang?

</comment><comment><USER69> No

<comment><USER70> Betteridge strikes again!

<comment><USER71> Except this one is an exception!

</comment></comment></comment><comment><USER72> I've never used Mongo. How many of people's complaints revolve around NoSQL (Ex AWS DynamoDB) vs Mongo specifically?

</comment><comment><USER73> Highly nested data with a schema that might change from site to site = Electronic medical records.

</comment><comment><USER74> I actually think NoSQL might be better for MVP and prototyping. Why? Because they can handle churn more quickly, and you don't have to worry about scale.

Once the data layer starts to resonate around a solution ... then pick the right horse for it.

<comment><USER75> Why not just use a SQL db that handles json and stick everything in a couple columns? Changing databases is a lot of work, and as far as I can tell, mongo isn't really providing much value over SQL dbs that already support schemaless json columns.

<comment><USER76> > Why not just use a SQL db that handles json and stick everything in a couple > columns?

Honest question: what's wrong with just defining a domain model, adopting an ORM and a serialization framework, and simply go with a conventional RDBMS? Afaik all reference web application frameworks handle this right out of the box.

<comment><USER77> Nothing on paper, if you get the schema more or less correctly the first time. The problem is when you need to do a schema change that affects terabytes of data down the road.

<comment><USER78> That is going to be a problem regardless of technology.

With an enforced schema you will at least know that all of the existing data matches the schema. Without it you have to hope that you had zero bugs while collecting the terabytes of data.

<comment><USER77> Yeah, but it's a different type of problem.

If you've a schema, you sometimes need to rewrite an entire table as part of a migration, and that can mean heavy engineering if you want to avoid downtime or disabling writes during the migration. There are 1:1 relationships between tables in the wild that wouldn't have passed a sniff test as part of an early schema design review, but then got created regardless to avoid a lengthy table rewrite.

If you've no schema proper, by contrast, you can manage multiple variations of what the data might look like in code. Not that such a thing is simple; it's definitely not, for the reason you raised. But it's simpler to deploy and migrate, or certainly might appear to be so to someone who isn't comfortable with SQL.

Also, there's a class of apps where having a schema doesn't add much value and NoSQL actually makes sense. Think storing and mining logs, scraped data, ML training sets, etc. -- apps where it doesn't matter much however a big pile of data gets stored, so long as you can shovel through it in parallel or store it very fast.

</comment></comment></comment></comment></comment></comment><comment><USER79> For any team that ever avoided a six or seven figure SQL Server or Oracle license and managed to scale to more than 50 transactions per second with horizontal scaling, yes, MongoDB was absolutely worth it.

<comment><USER43> Uh... Postgres?

</comment></comment><comment><USER80> In my experience, MongoDB shouldn't be used in cases where you need relations, or the typical guarantees that ACID databases bring.

I think a lot of the "beef" people have towards MongoDB is from past experiences where it was used in this regard.

I usually use PostgreSQL at this point, but when I use MongoDB, it's more as a "document store" than a "database."

<comment><USER29> MongoDB is ACID compliant. [https://www.mongodb.com/transactions](https://www.mongodb.com/transactions)</comment></comment>


Using Git Wrong - <AUTHOR> https://dpc.pw/blog/2017/08/youre-using-git-wrong/ <comment><USER2> This guy is espousing cargo cult version control.

> This is how the Kernel team does it, so everyone should do it that way

99% of the time, your team has absolutely nothing in common with the Kernel team and it's foolish to adopt processes from other teams without knowing why.

If anything, I would say the right way to use git in a small team is to always merge and never rebase.

Whatever you decide to do, make sure to think about _your_ problem first, then use the tools to solve your problem.

<comment><USER3> He's also forgotten that git was invented before continuous delivery and the idea of releasing often from one stable master branch.

That said it can be useful _sometimes_ to push changes out of band straight to a colleague. I have used this a few times to great effect but don't often come across the need

<comment><USER4> Wait, what?

> git was invented before continuous delivery

git was invented 2005. So you mean when people used the term Continuous Delivery more frequently? Because using CI and automatically deploy is not "new" and was surely done before git - I remember doing that even with CVS and SVN.

<comment><USER3> Come off it. There's a lot more to continuous delivery than ci + deploy.

It's goes hand in hand with BDD and automated testing. I reckon there's a lot less people doing this stuff today in the wild than you'd think.

Sure there's the Netflix's n googles of the world. It was only a few years ago I would consult on projects that would have no CI. It happens.

The same thing is happening today, just because you read about kubernetes or swarm simply does not mean anything like the majority of the people out there are using it in production. Just go to a conference and and watch people stick their hands in the air.

Sure SOMEONE might have been doing continuous delivery back in 2005, it certainly wasn't me. I was too busy learning about data access objects vs ORMs and releases were 3 months away if you were lucky.

</comment></comment></comment></comment><comment><USER5> I think it's a fallacy to say that just because Git _allows_ a completely decentralized organizational structure, that not using one is somehow wrong. The beauty of git and any other DVCS is that it doesn't prescribe any specific organizational structure, so you're free to adopt any structure you like on it. And critically, you're free to change it when it makes sense to do so. I think the reality is that a centralized structure works best for most projects most of the time, but git has the flexibility to break out of that mold.

For example, maybe in one project most of the time everyone pushes and pulls from a centralized master and not to and from each other. But then two developers get together to try and solve a particularly troublesome bug. While they're doing this, they can push and pull directly from each other without worrying about master, and then once they're done, they can merge, clean up, etc., and then push the result to master. Git allows these kinds of pockets of decentralized development within an overall centralized workflow, and does so seamlessly.

Since Git allows any organizational structure, it's up to the people working on a project to choose what works best for them. That's a responsibility that they didn't have before DVCS, because only one structure was supported by the tools.

<comment><USER6> But why won't they just work on a separate branch still sitting against the main git repo? So long as the internet is up it offers the same thing basically.

<comment><USER7> If they work at the same company, yes. It's often (always) easier to use the central repo than to figure out connectivity or send patch files.

But if they're in different companies, and there is no central repo visible/accessible for both devs, then yes the D in DVCS is definitely appreciated.

</comment></comment></comment><comment><USER8> The author suggests one "symptom" of using git wrong is

>Have you ever did a code review, and gave feedback that was longer than doing the fix yourself, while you're at it?

But this seems to neglect the utility of teaching people to do things correctly. Sure, I could always just fix junior programmers' mistakes, but it seems better to educate them in the first place.

Sending the fixes back down doesn't seem like it would have the same impact.

<comment><USER9> Depending on the kind of mistakes they've made, better again would be to provide a style/implementation guide for the code etc. so that instead of having to review code or have arguments break out over how something should be written, you can just point to a definitive guide beforehand, and avoid the troubles.

I say this, unfortunately, from experience. It has become common in our team for a new developer's first commit to be met with a blasting review, telling them everything they've done wrong, and most of it is capitalisation/whitespace etc. stuff.

<comment><USER10> Better yet, replace the style guide with a script (à la go fmt). You'll save everyone time and make the new person's experience a million times more pleasant!

<comment><USER11> Not if the new person disagrees or prefers a slightly different style. Reviews at least provide a chance of discussion and a possibility of change. Not everyone wants to work in an authoritarian environment of "do it this way or leave."

<comment><USER12> Gofmt, or rustfmt or whatever language fmt are really one of the best things that happened to modern development. Even if you don't agree with the style, which nobody does fully, it's much better than having discussions about style all the time and having to format the code manually. Formatting just happens when you save the file, so you can forget spaces and everything when writing and your code will look fine when you save it.

<comment><USER13> > Gofmt, or rustfmt or whatever language fmt are really one of the best things > that happened to modern development.

Funny way to state that. There is this formatting tool `indent' for C and C++. It was already well known and rather rarely used back in 2003. Code formatters have not happened to "modern development" by any stretch.

</comment></comment></comment></comment></comment></comment><comment><USER14> > while it was created to be used like in Linux kernel development. [ ... ]

> * You have one online repository for every project (or even the entire code > in case of a monorepo).

> * Your repository/repositories have one branch that everyone branch from and > develop their changes against. (Release branches don't count.)

The kernel has one main repo that integrates things from the others, the "torvalds/linux.git":

[https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/)

This keeps Linux together as One Thing. This is Linux's CVSROOT, so to speek.

> _You have one process that everyone have to follow to review the code; one > review system and one place that runs all your tests._

No kidding; also: one payroll from which everyone gets paid and one product that customers understand: not seven versions of the product based on whose cubicle it came from. See the trend?

<comment><USER15> Developers however use Linus's repository only to find a convenient forking point. Most of the development happens either in subsystem trees, or in distribution forks (where "distribution" includes big companies like Google or Facebook); only bug fixes target Linus's tree directly.

And most users use stable trees or distribution forks. So Linus's tree has the distinctive feature that almost no one uses it, unlike a traditional CVSROOT.

<comment><USER16> That almost no-one uses _directly_ , but pretty much everyone you mentioned definitely pulls from it(or pull from things branched from it). It would be super interesting to see a path map of commits and what trees they came from for the big distributions of the kernel(i.e. android, debian, fedora, etc). But I'd bet that all of them pull from Linus's repo as a descendant.

My understanding of the general flow layout for the kernel is:

developer branch -> patch -> mailing list -> sub maintainers -> large maintainers -> Linus -> Stable branch X.X -> distribution (i.e. Debian/etc).

With some things cherry picked here and there that don't flow through this, but that most code does.

<comment><USER15> Sure, everyone's tree descends from it. However:

\- normal CVSROOT: developers develop on top of the head of the day.

\- Linus's tree: developers develop on top of someone else's head of the day, except for some bugfixes.

Linus's tree becomes "similar" to a CVSROOT only after he cuts a release. But at that point, version control is actually not part of the game anymore.

</comment></comment></comment></comment><comment><USER17> In our company, every developer must fork the main repo to their own account, and every change that they want merged into master needs to be done as a PR that is then reviewed by the right person. That's a completely distributed process, and works great for a small company. We obviously need a central repo because apps need to be deployed from a single history -- a central master branch is not an antipattern, as suggested by the author. Even Linux has a central master branch.

<comment><USER18> My perspective is that we might be looking at two different things: development and deployment. Of course, each deployment has a canon. I think the idea of the bazaar is that there does not need to be a single canon but maybe I'm misinterpreting it.

</comment></comment><comment><USER19> A linked list is a binary tree, and using git in a centralized fashion is a legitimate use of git.

Not everyone wants a distributed version control system, and not every team is ready for a more complicated process.

</comment><comment><USER20> > Have you ever did a code review, and gave feedback that was longer than > doing the fix yourself, while you're at it?

I always thought this process was sort-of intended. I feel it shows some respect to the author of the code under review to comment on it, and not to change it–especially not if that would result in changes that are then merged without first getting approval from the author. I'm having trouble finding the right words for this idea...

It's also a way to teach people about coding standards and expectations for little things, such as spelling in comments.

<comment><USER21> I'll provide contrast, then.

Code checked into version control is never changed. What happens is the reviewer creates a derivative work. If they use my code as a basis for their derivative work, that's a sign of respect just as much as anything else. It means that they think I actually got it mostly right.

The same argument applies if it's merged right away. Unless the reviewer(s) blatantly misunderstood something important, it's likely their fix is an improvement. And even if they did, the previous code is just a reset/rebase/oops-commit away.

That said, you should still discuss the code to iron out misunderstandings and learn things.

</comment></comment><comment><USER22> Author seems to assume that all software development is like Linux development and it isn't. I can't have 10 different versions of my order processing service floating around doing different things for different people.

<comment><USER23> Isn't his point that different people have different needs, and git was created with one typical use in mind? So maybe not everyone should use git exactly because their needs may be different? Disclaimer: not a serious programmer myself

<comment><USER22> But he starts by saying that people are using git "wrong" (or suboptimally). If git fits my needs why should I worry about how this dude or Linus or the prime emporer of gitland thinks it should be used?

I think the author has a great point about the differences between how git was "designed" to be used as compared to how it ends up being used in orgs most often. But I also balk at being told I'm using an open source tool incorrectly as a general rule.

disclaimer - I really want prime emporer of gitland to be a proper honorific.

</comment></comment></comment><comment><USER24> I see one major flaw in reasoning in this post: trust.

Lets pretend that architects are under team leads, seniors under architects, juniors under seniors, etc.. In this hierarchical model every layer forwards some responsibilities down the chain and _trusts_ the task to be broken down, distributed further down and completed. In this model code reviews function merely as tools to _roughly_ check for gaping mistakes and whether code fits "architecturally". People are trusted to do their job to the best of their ability and are allowed to have personal style.

In a model where seniors go over everything juniors made, fix according to their own view and then commit we have a problems that lower layers are not trusted to do their job correctly and in the end everything has to strictly conform to product owner's vision or will be "fixed" (rewritten). In this model there are no longer junior, mid-level, senior engineers, because the more minions one has under them, the less time they have to produce anything on their own. In this model there are junior, mid, senior code-monkeys and one mighty product owner who knows it all best. Not sure that environment where only career growth opportunity is ability to rewrite other's code to your own liking would be good for team morality.

<comment><USER25> You just described enterprise consulting with outsource/offshored projects.

</comment></comment><comment><USER26> What I really want is Subversion, but then without all the bugs and the weird file locking. And with easy release branches and a way to make commits locally and then share a bunch of them at once with my colleagues.

Of course no such thing exists, so I use Git which has these features but combines them with 10x worse UX and 10x more condescension from the community. But I put up with that, only because it's the lesser of two evils.

I can't imagine I'm alone in this.

<comment><USER27> What you (and I) want and need is Git with a better UI / UX. I want see a better visual representation of that's happened, and not just a list of commits.

For example (and simply thinking out loud so don't shoot me), why can't I modified a file. Then modify another file. Then create a commit object (so to speak) and drag the files on to that commit? And then, once I'm sorted, I drag the commit on to a push. Which perhaps goes into a queue for review. And then the tool somehow help enable that review knowing any other commits on the same files, etc.

Coding is building. Seeing it visually is going to help. It always just seems to me I'm forced to think too much about Git, when a UI would do. And the brain cycles I use on Git (however small) could be used to write code (not wrestle with managing it).

</comment></comment><comment><USER28> I find this post quite amusing.

Many of the suggested methods of code sharing do happen, especially on large projects with multiple teams.

But the "one master branch" _problem_ is not a result of missuing git... It's a result of there being one version used in production... No git process change is going to change that.

His real issue seems to be with code reviews... Which isn't really something unique to git...

</comment><comment><USER29> The longer you keep your branch around "to be battle tested and swapped across teams or modules", the more attention and time you have to pay keeping it updated. Cost increases as your branch "gets battle tested". If your company can afford that, it's fine.

This applies to pretty much any version control where you can create a remote branch.

</comment><comment><USER30> I have one comment for the article: LOL.

Seriously, the author probably never worked in teams with different sizes [1, 2], considered another workflows [3, 4] and skill levels of the developers (yes, that's a variable you have to consider also!)

Also if you use Git in the same manner as CVS, you should do it because it's much faster, more reliable and more flexible...

Edit: Ok, the author is saying things which I agree with, I was commenting on the click-bait title and the first paragraphs mostly, but still, there is more to the story.

[1]: [https://m.signalvnoise.com/the-majestic- monolith-29166d02222...](https://m.signalvnoise.com/the-majestic- monolith-29166d022228)

[2]: [https://blog.bradfieldcs.com/you-are-not- google-84912cf44afb](https://blog.bradfieldcs.com/you-are-not- google-84912cf44afb)

[3]: [https://trunkbaseddevelopment.com/](https://trunkbaseddevelopment.com/)

[4]: [http://nvie.com/posts/a-successful-git-branching- model/](http://nvie.com/posts/a-successful-git-branching-model/)

</comment><comment><USER31> These are nice ideas and I must confess that I never douted the way we use Git before reading the article. However, in the companies I worked for (and I think it applies to a lot of companies), the time constraints were too tight to allow more than form-based reviews. I do feel these reviews are sub-optimal and should be replaced, but checking out the repo and testing yourself is a very slow process that we mostly can't afford. Most of the time the "go fast, break things" motus is applied and it's fine if you are not building critical applications like a kernel...

</comment><comment><USER32> We need a git repo holding the production version.

This is also where the continuous integration tests usually take place.

What is a good point of the OP is that people may not be sharing and testing code enough before pushing to the reference repo.

<comment><USER33> Out of interest why do you "need" a repo/branch replicating production? I ask out of curiousity rather than criticism.

We also have the same at my place of work, but for convenience when requiring to patch production - ie just branch master rather than scramble to find the commit hash last released. It's not a requirement, but it definitely makes life easier.

<comment><USER32> Because it provides a unique reference point. If there wasn't such a a reference point, there is a risk that the production branch of programmers becomes desynchronized.

It is easier and safer to have n developpers synchronizing to one reference repository, than to keep n developpers and repositories synchronized with each other.

</comment></comment></comment><comment><USER34> One interesting thought this article provoked:

Code review often catches typos, or edge cases that should be tested. As the OP points out, you end up serializing the problem into a comment, which the other developer has to deserialize, implement, and send back. Worse, these sort of issues often come across as nags - I'd be happy to fix a typo myself, but feel like a jerk calling someone else's out. I rather like the idea of pulling down somebody's diff, amending it, and pushing it back up. I'll have to try that sometime.

<comment><USER7> You can't catch all the issues in code review every time, and it's not always you that does the reviews. If you fix everything yourself, the original committer won't learn anything and the next submissions will have the same issues. Or perhaps the edge case isn't really an edge case and there is nothing to fix after all, in which case you've wasted time for nothing.

Also, if you do this people will likely subconciously take advantage [1] of it in the long run, and if you ever leave the team quality will suffer.

[1] ever had flatmates? Do all the dishes and cleaning for a month and you'll see :-/

</comment></comment><comment><USER35> > _And you might just not be enthusiastic about that "distributed" idea at > all. Maybe more top-heavy, centralized system is actually what you need._

Talk about throwing out the baby with the bathwater!

Just because you have a blessed master branch, and a central repo, does _not_ mean imply you aren't making use of your DVCS. You can still do all development on topic branches, including CI, and you can do it all in an asynchronous manner.

This idea of pushing branches between individuals in an ad-hoc fashion with whatever tools everyone wants to use makes perfect sense for something like kernel development where A) there is no single product running on a single production environment, it's a freaking kernel deployed to billions of systems globally and B) you're dealing with a huge set of people who you could never hope to standardize anyway.

When you're shipping a complex product to a single production environment, it makes a great deal of sense to centralize certain things. Having everyone bike-shedding on a million different code review paths, and relying on each individual to pass their code around to other individuals with no single place where you can look to find out what someone was working on will lead a lot of chaotic inefficiency that will leave juniors confused, and all in the pursuit of some platonic ideal that is cargo-culted from a completely different project.

Don't get me wrong, I think most people have a lot to learn about how to use git better, and I am a huge proponent of the value of a well-curated VCS history, but this article is inappropriately prescriptive nonsense.

</comment><comment><USER36> >Have you ever did a code review, and gave feedback that was longer than doing the fix yourself, while you're at it?

Yes. Yes yes yes. I hate this so much. I want my code reviews to involve fixing little things. I don't even have suggested edits, I can only highlight comments and make comments, and even suggested edits wouldn't be enough. I want to just make the changes. It's all on a branch anyway.

<comment><USER37> Yeah, but you miss the training opportunity this way..

<comment><USER36> How so? It's on the code review. They see the change, and learn about it..?

99.9% of code review that I do, and that I get, is not training. So maybe don't make all the changes for new employees? I use vim. Most people at my work use vim. Our build process is complex. Everything we do is really complicated. It works for the experienced, yet it has a learning curve.

So, new people get a slightly different experience. That's fine.

</comment></comment></comment><comment><USER38> > The way code review is done: by looking on an output of diff command, > pretty-printed in a web-browser, and pointing out obvious mistakes is very > inefficient and fundamentally insufficient

> open in IDE/text editor of their choice with or along diff viewer, jump > around the new code, run tests and even write some new ones, toy with it

This doesn't scale. I know team leads in large projects where reviewing external contributions is their main responsibility; this style would be impossible.

Instead, I think the human, diff based approach with CI running against the reviewed code is enough. A reviewer going over the tests which are shown to be passing should be sufficient after the usual diff read-through.

</comment><comment><USER27> Ideally, shouldn't all "merges" come from PRs? That, there's a gatekeeper (or sorts) that helps to prevent sloppy, unnecessary and erroneous things from ever making it into the code base.

Just because (autonomous) you can merge and push, doesn't mean you should. Right?

</comment><comment><USER39> Under "Changing Code":

>It might seem like chaos, but it is not. It is just fluid synchronization, or to put differently: eventual consistency system.

Can anyone explain this? The example he used wasn't fleshed out enough for me to understand how this system wouldn't be chaos

<comment><USER40> The part about eventual consistency is just the author throwing around terms he does not understand in an effort to sound relevant.

</comment></comment><comment><USER41> Every once in a while there's an article like this. S __t, use it how ever you want as long as makes you productive.

</comment><comment><USER42> git as a tool not a workflow. If you want to use git like a central VCS that is fine. git allows it. If you want to go fully distributed like the kernel devs. Fine, git allows it. As always, pick the approach that works best for your environment and not tell people they are wrong because of X and Y.</comment>


Ask HN: What are some good certifications for a junior developer? - <AUTHOR>

Are there any skill-building programs or certifications out there that you could recommend to a developer at the very beginning of his (post-bootcamp) career? Something that could ideally be done in addition to a part-time or full-time job?Thank you <comment><USER2> There are a lot of skill-building programs out there but you gotta ask yourself first on which 'programming world' you would like to be part of:

Certifications are really good if you really want to build a career into middle-management, in-house software development departments a.k.a. mega-corp widget factories; these types highly appreciate certificates. A big pharmaceutical company would love to see a Microsoft Certificate over a Github profile since they don't care about the code, they just care about you passing a "industry standard test" so that you would fit X number of requirements and feel safe that they got the right person. Couple your certificate with a filled out LinkedIn profile; and they'll highly consider you.

Building a good Github profile/portfolio website is the better route if you want to build your career around working for companies that sell software as their core product, or well-known shops (Fog Creek, 37 signals, etc.). These type of companies highly value code and work output, and they'd understand the value in your profile. Don't get me wrong, a certificate would help in some way but if you can't show them you can code or be comfortable with the command line, they'll go for anyone else who has built something and deployed it.

If you're going for the certifications route, it depends on what stack you're going for:

For anything with Spring, go through springsource.com/training you'll find some links there for certification classes.

For anything .NET or Windows, go through microsoft.com/learning for more info.

Don't underestimate the amount of time and money spent on studying/getting a certification. If you already have a job, don't take a certification unless you really plan on moving up some sort of mega-corp corporate ladder or if you're targeting for another job that requires a certain certification. The better way to be better that has no strict commitments is to find yourself a user-group of your stack, work with a friend on a project you think is 'cool', or work on some open-source project.

</comment><comment><USER3> The programming world doesn't really work like that. There are a couple of places selling certifications, but they matter so little that it would be a waste of money to get one.

The best thing you can do is build stuff and show it off on a portfolio website or a github profile.

</comment><comment><USER4> [http://www.microsoft.com/learning/en-us/mcpd- certification.a...](http://www.microsoft.com/learning/en-us/mcpd- certification.aspx)

[http://education.oracle.com/pls/web_prod-plq- dad/db_pages.ge...](http://education.oracle.com/pls/web_prod-plq- dad/db_pages.getpage?page_id=504)

But if you already have a job, you don't need one. They might help you land a job if you don't have a CS degree. Also agree with the other comment: if you are doing JS, etc. then get active in GitHub.

</comment><comment><USER5> I'd have to agree with the original answer too, In terms of recruitment these days, it seems that agents frequently ask for either github urls or somewhere a client can see your work as in a portfolio. Best thing to do is find something that interests you and then find relevant tech to learn in order to implement it. You'll find that you pick a lot of things up from others you will work with and the way in which projects get set out etc. If you look to create a portfolio site you can also use that as a side project to pick up new tech. I recently started a personal project to create a personal portfolio site boilerplate that is a single page application that is easy for people to modify and make their own [0]. I made it because I needed to get underway with creating a portfolio site and also because it gave me another opportunity at using new things. Also maybe look at contributing to popular OSS because that will get credit too. That's my two cents anyway, hope it helps.

[0] [http://jh3y.github.io/pavo](http://jh3y.github.io/pavo)

</comment><comment><USER6> I went to the Epicodus programming bootcamp and got a junior dev level job after without much struggle, but most companies I talked to just wanted to see my github. Make a cool side project. I started writing two books and was contributing to an open source project as well as working freelance making simple static pages/CMS sites to get more stuff in my portfolio. If you want to know some people to talk to, email me. It's in my profile.</comment>


Review my site: FundStrap.com - Worldwide funding for your project - <AUTHOR> http://www.fundstrap.com First stage of my startup. Crowdfunding for your project, mass-production, book, documentary, service, etc. Anyone from around the world can signup. Please leave your thoughts. <comment><USER2> I'm still a little unclear (even with the FAQ) exactly what projects are acceptable and which are out of the boundaries. For example, you say international accessible. Does that mean projects on a national scope are not accepted? Will you curate projects like KickStarter?

<comment><AUTHOR> We website has a worldwide scope. Unlike kickstarter which has a lot of us- centric projects. That said: you can always try. A local project can be succesfull but has a smaller audience.

Project will be curated before publishing. Fundstrap is financialy responcible for the creditcard transactions. So submitters need to be verified.

Any project will be accepted by default when you can provide a product or service in exchange. So investments where interest or profit shares are returned are not accepted. If you have a startup which does not provide a product, you can give a (lifetime?) account as a service.

</comment></comment><comment><USER3> What is the difference with kickstarter.com? It seems like someone read about the 15% fee the get for every started project and though he could do better.

<comment><AUTHOR> Non USA citizens cannot start a project.

The 15% is for the beta-period. We hope we can lower the fee when we go live. Many examples on HN suggest you can better lower the fee in stead of raising. The latter causes a lot of angry customers. :-)

<comment><USER3> no, you are right. There is jile.eu. Launched some weeks ago.

</comment></comment></comment><comment><USER4> FAQ page has "Fundstap" as title, little typo.

<comment><AUTHOR> Fixed</comment></comment>


What programmers need to know about encodings and charsets (2011) - <AUTHOR> https://kunststube.net/encoding/ <comment><USER2> I was looking for the catch. Here it is: "It's really simple: _Know_ what encoding a certain piece of text, that is, a certain byte sequence, is in, then interpret it with that encoding."

That's like "knowing" the truth. How?

I have received some very interesting files that made Python yack unicode errors, again and again. Why? Not only did I not "know" what encoding it was in -- _the encodings changed_ at different points in the stream of bytes. I call this "slamming bytes together" because somewhere along the line, someone's program did exactly that.

Everything is simple -- until it isn't.

<comment><USER3> There is nothing you can do with text file with unknown encoding but treat it as an array of bytes.

If you start guessing the encoding, at best it won't work in some cases, at worst you are introducing security vulnerabilities. You can try, but there is just no way to do it right.

[http://michaelthelin.se/security/2014/06/08/web-security- cro...](http://michaelthelin.se/security/2014/06/08/web-security-cross-site- scripting-attacks-using-utf-7.html)

<comment><USER4> Generally you can safely treat text in an unknown encoding as UTF-8. Since you're expecting potential failures but want to press on anyway instead of causing an exception/ error you treat invalid sequences as U+FFFD the Replacement Character as you would in a language or API with no exception reporting mechanism.

There are lots of pleasing aspects to this choice. It's ASCII compatible of course, so anything that was actually ASCII is still ASCII, anything that was _almost_ ASCII is just ASCII with U+FFFD where it deviated.

The replacement character resolutely isn't any of the specific things, nor any of the generic classes of thing you might be expected to treat differently for security reasons. It isn't a number, or a letter (of either "case"), it isn't white space, and it certainly isn't any of the separators, escapes or quote markers like ? or \ or + or . or _ or...

... yet it is still inside the BMP so it won't trigger weird (perhaps less well tested) behaviour for other planes.

It's self-synchronising. If something goes wrong somehow, in a few bytes if there is UTF-8 or an ASCII-compatible encoding the decoder will synchronise properly, you never end up "out of phase" as can happen for some encodings.

Most usefully, whatever you're now butted up against works with UTF-8 now. Maybe some day that'll get formally documented, maybe it won't. As the years drag on the chance of specifying _anything else_ shrink more, and the de facto popularity of UTF-8 means even if it's never formalised anywhere everybody will just assume UTF-8 anyway and you haven't to lift a finger.

<comment><USER5> I probably don't need to say this, but it all depends. Many operating systems and GUI frameworks internally use UTF-16 because it was more common when they were built. Lots of old files use really obscure encodings. Sometimes you get a UTF BOM to identify UTF-16 and UTF-32, other times you don't. Then there are the pesky ways you can encode characters with HTML or XML entities, the occasional double-encoding of such, and so on.

When I worked with library records, I had to deal with text encodings that pre-dated SQL, though I suppose I should be thankful that ASCII existed by then so they were mostly ASCII compatible, but even today there are systems designed to output MARC-8 + UTF-8 as a fallback only when a MARC-8 character isn't available (MARC-21) instead of just using UTF-8.

I'll admit though, outside of MARC-8 and the various Unicode encodings, I'm having trouble thinking up systems that would still be incompatible today. Old documents, yes, absolutely would be encoded in different charsets, Windows still generally defaults to encoding in their Latin1 if I recall correctly, but most systems today do expect UTF-8 over the network at least, and UTF-16 for display perhaps...

Don't get me started on line endings though, and how many files use one, both, more than one ... and especially how much fun it can be with git repos cross- platform, or when automated tools use platform default line endings when they should be configurable, etc. CSV files that aren't properly escaped are also a special mini hell...

Data is never easy. :) And that's assuming it's written correctly - [https://rachelbythebay.com/w/2020/08/11/files/](https://rachelbythebay.com/w/2020/08/11/files/)

<comment><USER5> Wanted to amend this list of character encodings with another one I came across recently -- GSM-7, used for SMS. [https://en.wikipedia.org/wiki/GSM_03.38#GSM_7-bit_default_al...](https://en.wikipedia.org/wiki/GSM_03.38#GSM_7-bit_default_alphabet_and_extension_table_of_3GPP_TS_23.038_.2F_GSM_03.38) If a message you send includes other Unicode characters than that, including emoji, it will cost more to send and use UCS-2 encoding (which later became UTF-16).

</comment></comment></comment></comment></comment><comment><USER6> Note: This post is basically a TLDR of [https://www.theregister.com/2013/10/04/verity_stob_unicode/](https://www.theregister.com/2013/10/04/verity_stob_unicode/) by Verity Stob.

One of the reasons there is a lot of confusion about encodings vs Unicode is that Unicode was initially an encoding. It was thought that 65K characters was enough to represent all the characters in actual use across the languages and thus you just needed to change the from an 8 bit char to a 16 bit char and all would be well (apart from the issue of endianness). Thus Unicode initially specified what each symbol would look like encoded in 16bits. (see [http://unicode.org/history/unicode88.pdf](http://unicode.org/history/unicode88.pdf), particularly section 2). Windows NT, Java, ICU, all embraced this.

Then it turned out that you needed a lot more characters than 65K and instead of each character being 16 bits, you would need 32 bit characters (or else have weird 3 byte data types). Whereas people could justify going from 8 bits to 16 bits as a cost of not having to worry about charsets, most developers balked at 32 bits for every character. In addition, you now had a bunch of the early adopters (Java and Windows NT) that had already embraced 16 bit characters. So then encodings were hacked on such as UTF-16 (surrogate pairs of 16 bit characters for some unicode code points).

I think, if the problem had been understood better at the start that you have a lot more characters than will fit in 16 bits, then something UTF-8 would likely have been chosen as the canonical encoding and we could have avoided a lot of these issues. Alas, such is the benefit of 20/20 hindsight.

<comment><USER7> > see > [http://unicode.org/history/unicode88.pdf](http://unicode.org/history/unicode88.pdf), > particularly section 2

it's Fascinating to see how people can arrive at the answer No and conclude that the answer is Yes

</comment></comment><comment><USER8> Interesting article from Joel spoolsky on unicode and character sets. [https://www.joelonsoftware.com/2003/10/08/the-absolute- minim...](https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum- every-software-developer-absolutely-positively-must-know-about-unicode-and- character-sets-no-excuses/)

</comment><comment><USER9> If curious see also

2015 [https://news.ycombinator.com/item?id=9788253](https://news.ycombinator.com/item?id=9788253)

2012: [https://news.ycombinator.com/item?id=4771987](https://news.ycombinator.com/item?id=4771987)

</comment><comment><USER10> I personally had a good time re-reading this over and over again when I was migrating python 2 to python 3, it's a great resource: [http://farmdev.com/talks/unicode/](http://farmdev.com/talks/unicode/)

</comment><comment><AUTHOR> I think not explorer BOM UTF-8 [https://en.wikipedia.org/wiki/Byte_order_mark](https://en.wikipedia.org/wiki/Byte_order_mark)

</comment><comment><USER11> I love C++ so much, and it has brought me such joy as a hobbyist programmer, but good grief, this one aspect of it (dealing with encodings & charsets) is so depressing I just want to cry sometimes.

</comment><comment><USER12> F for respects for everyone who got wrecked by BOM (byte-order mark) and CRLF vs LF.</comment>


HN?: are you interested in seeing documentation for an ELK-like system? - <AUTHOR> http://imgur.com/a/5uu2H <comment><USER2> And how is it any different from what _everybody else_ uses?</comment>


How Amazon Rebuilt Itself Around Artificial Intelligence - <AUTHOR> https://www.wired.com/story/amazon-artificial-intelligence-flywheel/ <comment><USER2> One thing which is sorely missed in such discussions is the employee efforts. Out of all the companies, Amazon's hours seems to be the worse of the lot.

That said, I really like Bezos (or Amazon) that they took risks on thing like AI, AWS etc. I find that lot of companies lack that insight. Even when they are not doing great, I routinely find executives hem and haw about their "core business". Most of these companies end up scraping the bottom of the barrel for profitability and keeping the company afloat for couple of years more.

<comment><USER3> An insightful comment that had to be made - I wonder why people downvoted it.

Indeed, Amazon is probably the best and the worst about the modern tech: an ambitious leader, who manages to create paradigm shifts in areas where he is not a subject matter expert; and a sweatshop-like environment where the low ranking employees are treated like oompah loompahs.

<comment><USER4> As an Amazon employee, I tend to think Amazon's reputation is overstated. Some teams definitely have very high operational loads, but in general my hours have been comparable to friends in similar tech companies and certainly less than friends in industries like medicine, management consulting, etc.

It's very strange to describe Amazon (at least engineering roles) as a sweatshop considering how well compensated employees are.

<comment><USER5> Well compensated? Amazon pays significantly below other mega cap tech companies, and as a PhD student in machine learning with a lot of friends interning and applying for jobs I can confidently say Amazon is nobody's first choice due to low salaries and working conditions.

I'd rather say I am amazed they get anyone great working for them because it really is clearly and painfully obvious that almost everyone would rather work for Google, Facebook or Microsoft regarding machine learning.

Have first hand knowledge of salaries. Amazon offers 30-50% less than other top tech companies for machine learning. There is literally no silver lining or upside as far as I can tell.

In that context, it does not matter if Amazon pays more than <other job> in <other industry> or treats employees better than <other company> in <other industry>.

<comment><USER6> This is selection bias. You have first hand knowledge of what Amazon offers new graduates with little or no industry experience (and yes, PhD students fall into this category). The compensation for more experienced employees is substantially higher.

<comment><USER5> "The compensation for more experienced employees is substantially higher"

Citation needed. Higher than compensation for experienced employees at Google/Facebook? I am not even disputing Amazon pays more later, I am saying Amazon never pays top tier AND treats employees badly AND has no perks or particular benefits to make up for it, so nobody who has the option (that I know of) wants to work there.

<comment><USER6> Every company pays more later, I was trying to say that the pay gap between other companies disappears for mid+ level positions.

Also, they do not treat employees badly. Are there cases where certain managers treat their employees poorly? Of course, you can find the same anecdotal examples at company of sufficiently large size. Since the infamous 2015 NYT article, employees are treated very well on average. For perks, if you're referring to free catered meals then, yes, Amazon lacks there. But how about working in downtown Seattle instead of some random suburb outside of San Jose?

Finally, you ask for a citation about my claim while offering none for your own. My source: I work here so I actually have first hand experience.

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> I have to say I'm disappointed by the mild boosterism that runs through Steven Levy's coverage of large corporations. For example:

> Amazon's product recommendations had been infused with AI since the > company's very early days,

Really? Is it the kind of AI that impresses people today? Or was it just collaborative filtering. This narrative that "things were good and now they're getting better" is common in corporate communications, but shouldn't be part of neutral coverage.

Or this:

> "But they have really come on aggressively. Now they are becoming a force."

> Maybe the force.

That's absurd. When you look at where Google has taken TensorFlow, AlphaGo and Waymo, and compare it with really any other company in the world, it would be hard to anticipate that Amazon would become _the_ force.

Amazon was forced to adopt an outside machine-learning framework, MxNet, as its own when its in-house toolkit didn't get traction. That's in stark contrast to Google, again.

I'd say Amazon's main advantage is in its ability to sell infrastructure and platforms that enable developers to build apps -- which in this case will help productize ML in various ways. But I would not call it _the_ force in AI.

</comment><comment><USER8> Amazon's Alexa is a big win for the company. It will directly increase shopping on Amazon. Integration with music and Prime Video impact Apple

It might soon be the most popular, and perhaps best, consumer voice assistant:

[http://h4labs.org/amazons-alexa-blindsided-apple-and- google/](http://h4labs.org/amazons-alexa-blindsided-apple-and-google/)

<comment><USER9> > It will directly increase shopping on Amazon.

Is there any evidence to this speculation?

Linked article has a link that just points to the sales of the device itself as proof.

<comment><USER8> Sure:

[https://www.forbes.com/sites/johnkoetsier/2018/01/09/adobe-e...](https://www.forbes.com/sites/johnkoetsier/2018/01/09/adobe- echo-google-home-sales-doubled-in-2017-and-22-shop-by-voice/)

Amazon is definitely working on improving this.

"Alexa, what are my deals?"

[https://www.amazon.com/b?node=16924225011](https://www.amazon.com/b?node=16924225011)

Sounds like they will be investing significantly more into Alexa:

[http://www.businessinsider.com/amazon-ceo-jeff-bezos-says- al...](http://www.businessinsider.com/amazon-ceo-jeff-bezos-says-alexa-beat- expectations-2018-2)

<comment><USER10> That doesn't say there's an increase in sales, just that people buy over VUI. It's very likely that buying over VUI cannibalizes web and mobile sales.

<comment><USER11> yes there's also this surprise element attached to it. You won't know if you have ordered a right item in right qty until you receive it or you immediately open the orders tab in your phone to make sure you ordered the right item.

<comment><USER10> Or when you say, "Alexa, order me a Chromecast" and it ships you a firestick instead.

</comment></comment></comment></comment></comment></comment><comment><USER12> I've heard just the opposite from someone I know at Amazon. I've heard that they still gather a lot of their data by doing Sql queries.

<comment><USER13> SQL is highly underrated by engineers.

</comment></comment><comment><USER14> FYI, unless Ghostery enables Adobe Dynamic Tag Manager, this article will be blocked by Wired... made invisible.

<comment><USER15> [https://outline.com/ffNk2r](https://outline.com/ffNk2r)</comment></comment>


What are the most lucrative specialties for freelance software development? - <AUTHOR>

I'm finding that making $80-$100/hr is reasonably easy as a freelancer, doing standard things like client/server. This is quite fine but I'm wondering what are people's opinions on more specialized skills to acquire and/or niches to target. Personally I have a strong math background and was formerly doing bioinformatics, so I have a certain sort of "big data" experience, but I'm not sure how to parlay that into a more premium freelance niche. <comment><USER2> Money is not there with smallish clients. You should work for F500 companies as a C2C contractor. In the silicon valley, most jobs tend to be contract to hire. It is common to see 6 to 12 months contract gigs in the vallet. Wheras on the east coast, long term contracts (like 3 years) is very common.

If you are in the valley, GE San Ramon is hiring lots of big data guys. I know via an external recruiter that some big data architects are making $135 per hour C2C. Sure, they may ask you to become full time; however, if you talk to the hiring managers, they can keep extending the contract.

Here is how you shud do: 1\. Stop working on 1099/W2 contracting crap 2\. Do C2C with the primary vendors, which requires Incorporation docs, tax-id and $400 Insurance per year from that techinsurance 3\. Some companies dont like subcontracting. So, primary vendors want you on W2, and you simply say No to those primary vendors. 4\. Some vendors, even if the client insists on W2 subcontracting, are happy to give you C2C, by telling the client that you are on W2. 5\. Yes, feel the market for rates. You can easily charge $125 per her C2C in the valley with all big data drama: kafka, camus, java, etc.

<comment><AUTHOR> Interesting, well I already have an LLC that I intended for use for my business, but then it didn't seem like it would make any difference. But you're saying contract with larger clients basically as if I am a consulting firm even though the only employee is me.

<comment><USER2> 1\. Basically, larger companies don't like to have you as a direct consultant/contractor. In other words, these companies do business with a set of staffing companies (called primary vendors), or through a contingent workforce management solution (a super vendor, who sources contractors/consultants through other staffing companies).

2\. Yes, you are the only employee for your LLC. But you are contracting with a primary vendor, who will place you at the end client (a fortune 500 company, for instance).

3\. Contracting rates are not that great, esp if you are shooting for a longer contract. In other words, working for google/apple/fb as full time is better than a string of shorter contracts. There is an inverse relationship between the contract rate and the contract length. Even large companies have restrictions of having the same contractor for more than two years.</comment></comment></comment>


Socrates as a mentor? - <AUTHOR> http://ericbrooke.wordpress.com/2011/09/02/socrates-as-a-mentor/

<comment><USER2> Was the Socratic method ever a good idea?

Doesn't it only work in contrived, scripted debates against an interlocutor who seems to spend most of his time saying "Yes surely Socrates that must be the case"?

<comment><USER3> It is great for talking through deductive reasoning with children. My 7 year old niece responds great to it. She has the base knowledge to think through the questions, and provide a possible answers. And, it is a lot of fun because she is clever enough to catch me and make me think through things more carefully.

Now, her 5 year old sister still doesn't seem to have enough everyday background knowledge, and focus to respond as well to it yet.

</comment></comment><comment><USER4> People forget that Socrates was forced to drink poison ostensibly becuase he had not made many friends with his "method". So one might consider that. I think often times say "the Socratic Method" which is a method of aurgument when really, they mean to say "ask a lot of questions" but that does not sound as cool I guess.</comment>


Google Cloud Gets a Secret Manager - m-watson https://techcrunch.com/2020/01/22/google-cloud-gets-a-secret-manager/ <comment> m-watson Ah, should have just linked right to the announcement but it is posted now: [https://cloud.google.com/blog/products/identity- security/int...](https://cloud.google.com/blog/products/identity- security/introducing-google-clouds-secret-manager)</comment>


China's Next Big Innovation? A Used-Car Market - <AUTHOR> https://app.newsoveraudio.com/article/4341 <comment><USER2> Could some kind of audio tag be added to this?

<comment><AUTHOR> That's a good point. The domain is 'newsoveraudio' though.</comment></comment>