Facebook releases 300-dimensional pretrained Fasttext vectors for 90 languages - <AUTHOR> https://github.com/facebookresearch/fastText/blob/master/pretrained-vectors.md <comment><USER2> This has the potential to be very very useful and it is great that FB has released them. Some potential caveats. I don't know how well Fasttext vectors perform as features for downstream machine learning systems (if anyone know of work along these lines, I would be very happy to know about it), unlike word2vec [1] or GloVe [2] vectors that have been used for a few years at this point. Also, only having trained on Wikipedia gives the vectors less exposure to "real world" text, unlike say word2vec that was trained on the whole of Google News back in the day or GloVe that used Common Crawl. Still, if you need word vectors for a ton of languages this is looking like a great resource and will save you the pre-processing and computational troubles of having to produce them on your own.

[1]: [https://code.google.com/archive/p/word2vec/](https://code.google.com/archive/p/word2vec/)

[2]: [http://nlp.stanford.edu/projects/glove/](http://nlp.stanford.edu/projects/glove/)

<comment><USER3> This isn't a real downstream task, but one of the researchers at RaRe compared FastText to word2vec/gensim/skipgram word embeddings on the original testsets for the 'semantic' and 'syntactic' analogy tasks from the word2vec papers here:

[https://rare-technologies.com/fasttext-and-gensim-word- embed...](https://rare-technologies.com/fasttext-and-gensim-word-embeddings/)

The conclusion:

       These preliminary results seem to indicate fastText embeddings are
       significantly better than word2vec at encoding syntactic information. This is
       expected, since most syntactic analogies are morphology based, and the char
       n-gram approach of fastText takes such information into account. The original
       word2vec model seems to perform better on semantic tasks, since words in
       semantic analogies are unrelated to their char n-grams, and the added
       information from irrelevant char n-grams worsens the embeddings.

Personally I think those analogy testsets are not very good, because they just test all pairs of relations between a very small number of words from very limited domains (like capital and country names).

One advantage of FastText should be better learning on small amounts of data like Wikipedia.

<comment><USER4> It takes a certain kind of perspective for wikipedia to be called a "small amount of data." English wikipedia alone would run to about 2500 print volumes. Imagine telling an AI researcher from 1995 that that was "small".

<comment><USER3> I admit, I was being funny by being unclear. For word-embeddings English Wikipedia is a moderate-large dataset at 58GB uncompressed (13GB compressed). But most of those other language wikis really are tiny. Welsh is just 67MB compressed, and there are plenty of languages more obscure than that on the list. The point of word2vec was to make use of as much data as possible by being as fast as possible (processing billions of words an hour) rather than clever, so it would be impressive if fastText vectors for those wikis were at all useful.

</comment></comment></comment></comment><comment><USER5> Evaluation of [https://s3-us-west-1.amazonaws.com/fasttext- vectors/wiki.en....](https://s3-us-west-1.amazonaws.com/fasttext- vectors/wiki.en.zip) (unzips wiki.en.vec) on word similarity tasks (all numbers are Spearman rank correlation):

WS-353 Similarity: 0.781

WS-353 Relatedness: 0.682

MEN: 0.765

MTurk: 0.679

RW: 0.487

SimLex: 0.380

MC: 0.812

RG: 0.800

SCWS: 0.667

Impressive for a model trained on Wikipedia alone!

I will post analogy scores for this model as soon as they are done computing.

<comment><USER5> Google Analogy Task results ( detailed results at [http://pastebin.com/PF96nMfX](http://pastebin.com/PF96nMfX) ):

Semantic accuracy: 63.84 % Syntactic accuracy: 67.00 %

Here performance is not great (great would be >80% on semantic and >70% on syntactic).

As this task requires nearest neighbor lookups, performance is impacted by vocabulary size. Since models trained using Wikipedia alone usually limit vocabulary to something ~300k words, we can try that to get scores which are comparable to those posted by the GloVe [1] and LexVec [2] papers by only using the first 300k words in the pre-trained vectors, giving the following results:

Semantic accuracy: 77.75 % Syntactic accuracy: 72.55 %

Impressive stuff!

[1] [http://nlp.stanford.edu/pubs/glove.pdf](http://nlp.stanford.edu/pubs/glove.pdf) \- [https://github.com/stanfordnlp/GloVe](https://github.com/stanfordnlp/GloVe)

[2] [https://arxiv.org/pdf/1606.01283v1](https://arxiv.org/pdf/1606.01283v1) \- [https://github.com/alexandres/lexvec](https://github.com/alexandres/lexvec)

</comment></comment><comment><AUTHOR> One of the biggest things that I see with this release is trained vectors for Asian languages - Hindi, Kannada, Telugu, Urdu, etc.

This is huge - because most other releases have traditionally been in European languages. It is fairly rare to see Asian languages release.

One challenge is that typical linguistic use in Asia mixes the native language with English. For example, people in north india use "Hinglish". It is typically fairly hard to make sense out of this.

<comment><USER6> It's already a challenge to find stuff in German, it's annoying just how much research only focuses on English.

<comment><USER7> According to Ethnologue (2005), by number of speakers:

Hindi -- native: 370m | total: 490m

Bengali -- native: 196m | total: 215m

German -- native: 101m | total: 229m

<comment><USER6> I'm sure you know yourself that number of speakers is rarely the metric used for choosing a target market, and more commonly products are launched by the potential revenue to be made (which scales with GDP per capita).

Yet, most projects only target the Anglosphere, not even Europe is usually included.

<comment><USER8> Problem is that "Europe" means so many different languages... which is also our biggest remaining obstacle in trying to launch even just web products here (as compared to launching "in the Anglosphere").

</comment></comment></comment></comment></comment><comment><USER9> FYI: you can now use fastText directly from gensim (Python) [1]. This allows you to easily test / compare fastText to other popular embeddings, such as word2vec, doc2vec or gloVe.

[1] [https://github.com/RaRe- Technologies/gensim/blob/develop/gen...](https://github.com/RaRe- Technologies/gensim/blob/develop/gensim/models/wrappers/fasttext.py)

</comment><comment><USER10> Played a bit with fastText months ago. The issue I had with it is that unlike CNNs/RNNs, the _relative position_ of a word doesn't matter as much (only as a part of a context window during training the embeds), and so results can be worse depending on the case. However, for CPU-workloads, fastText is certainly faster, especially since subword information is also incorporated.

I have Python code to process text files into a fastText/friendly format so I may clean that up and see how these pre-trained embeds work.

(although, the English embeds are 10.36 GB; that might be a tough pill to swallow for training on machines with only 8 GB of RAM)

<comment><USER11> Regarding the size of the word vectors files: the text files are sorted by frequency, so it is possible to easily load the top k words only.

We might also release smaller models in the future, for training on machines without large memory.

<comment><USER12> fwiw I have 32gb on my workstation and my personal laptop is maxed out at 16gb. Keeping within these thresholds may be useful to others.

</comment></comment></comment><comment><USER13> Does anyone know if the languages all live in the _same_ 300-dimensional space, or are they each trained independently? (i.e. do words and their translations have similar vectors?)

<comment><USER11> Models are trained independently for each language. So unfortunately, you cannot directly compare words from different languages using these vectors.

If you have a bilingual dictionary, you might try to learn a linear mapping from one language to the other (e.g. see [https://arxiv.org/abs/1309.4168](https://arxiv.org/abs/1309.4168) for this approach).

</comment></comment><comment><USER14> Can anyone point me to any articles on what can be achieved with this and how?

[EDIT] I reply to myself here: [https://news.ycombinator.com/item?id=12226988](https://news.ycombinator.com/item?id=12226988)

<comment><USER15> Words with similar vectors have similar meanings. You use this in search, sentiment analysis, topic detection, finding similar text, and classification.

Of course there are tests for this "words with similar vectors have similar meanings" property, and I'm finding that the fastText vectors aren't doing that well on them, especially outside of English.

I'm glad they released them, particularly so anyone can run a fair comparison between different word vectors, but these things should come with evaluation results the same way that code should come with tests. These vectors are performing worse than ones that my company Luminoso released last year [1] (a better, later post is [2]), and if you don't believe me plugging my own vectors, I know that Sapienza University of Rome also has better vectors called NASARI [3].

fastText covers more languages, but most of these languages have no evaluations. How do you know the Basque vectors aren't just random numbers?

I think that performance hits a plateau when the vectors only come from text, with no relational knowledge, and especially when that text is only from Wikipedia. Text exists that isn't written like an encyclopedia. Meanings exist that aren't obvious from context. My research at Luminoso involves adding information from the ConceptNet knowledge graph, producing a word vector set called "ConceptNet Numberbatch" that just won against other systems in SemEval [4], a simultaneous, blind evaluation. The NASARI vectors are also based on a knowledge graph.

[1] [https://blog.conceptnet.io/2016/05/19/an-introduction-to- the...](https://blog.conceptnet.io/2016/05/19/an-introduction-to-the- conceptnet-vector-ensemble/) \-- linking this to establish the date

[2] [https://blog.conceptnet.io/2016/11/03/conceptnet-5-5-and- con...](https://blog.conceptnet.io/2016/11/03/conceptnet-5-5-and-conceptnet- io/)

[3] [http://lcl.uniroma1.it/nasari/](http://lcl.uniroma1.it/nasari/)

[4] [http://alt.qcri.org/semeval2017/task2/](http://alt.qcri.org/semeval2017/task2/)

<comment><USER16> We evaluated using ConceptNet Numberbatch but in the end went with fasttext because of the treatment of OOV words using sub word information. This is important for us because we work with Social Media where misspellings are very frequent and we have found this helps a lot. Are you also looking into these sort of enhancements? How do you usually deal with OOV words?

<comment><USER15> Very good question!

Our OOV strategy was pretty important in SemEval. The first line of defense -- so fundamental to Numberbatch that I don't even think of it as OOV -- is to see if the term exists in ConceptNet but with too low a degree to make it into the matrix. In that case, we average the vectors from its neighbors in the graph that are in the matrix.

For handling words that are truly OOV for ConceptNet, we ended up using a simple strategy of matching prefixes of the word against known words (and also checking whether a word that's supposed to be in a different language was known in English).

fastText's sub-word strategy, which is learned along with the vocabulary instead of after the fact, is indeed a benefit they have. But am I right that the sub-word information isn't present in these vectors they released?

There's a paper on the SemEval results that just needs to be reviewed by the other participants, and I'm also working on a blog update about it.

</comment></comment></comment></comment><comment><USER17> Would also be nice with a guide on how to use them

</comment><comment><USER18> Disappointing to see Facebook, a company with a huge Irish presence, neglect Irish in the list. There are plenty of minority languages in there, like Breton and Scots. And languages not spoken natively anywhere like Latin, Esperanto, Volapük.

Google Translate (with a similar number of languages)supports it fine.

<comment><USER11> Hi, because we trained these vectors on Wikipedia, we released models corresponding to the 90 largest Wikipedia first (in term of training data size). More models are on the way, including Irish.

<comment><USER18> I suspected it was something like this. Unfortunately the Vicipéid is not of very high quality. I just just hope Facebook doesn't forget which side its bread is buttered on.

</comment></comment></comment><comment><USER19> Can someone explain WHY do word vectors with similar contexts club together and are good? One of the papers suggest that "we don't really know" (Section 4)

[0] [https://arxiv.org/abs/1402.3722](https://arxiv.org/abs/1402.3722)

<comment><USER20> Consider the classic example of king and queen. "Queen" will tend to occur near "female" words, like "she", "her", "woman", etc. And vice versa for "king". But both words will tend to occur near words talking about royalty, e.g. "castle", "crown", "ruler", etc.

So you can learn a decent amount of information about a word, just by looking at the words around it. This is the same thing we teach kids learning to read with "context clues". If I talk about bolgorovs and how delicious they are, and how they are ripe and sweet, etc. You can probably guess "bolgorovs" are a fruit, just from the context.

<comment><USER21> Sometimes, context can be ripe for the picking while simultaneously leaving your subject rotting on the vine.

Regardless: I think you answered a question more about "what" than "why"; I know what these models are, and how they are used, but it is still somewhat surprising to me that you can get as good results as people claim without a nearly infinitely dimensional space (and honestly I kind of wonder if this might be more a model of "the kinds of questions humans most often like to ask about words when confronted with a word for the first time or wanting to test a data set" instead of "a true understanding of what is being said encoded as vectors", allowing things like "have opposite gender" to be extremely functional vectors but probably leaving _much more important_ concepts like "are classic opponent in war" on the floor, which is really important semantic information that isn't necessarily transtive and might not even be commutative, a thought process that seems to align with complaints about word2vec from ConceptNet.)

Put differently: I bet the set of 300 axes is actually a more useful result (though one that is more opaque and I don't hear much about attempts to analyze; but I am currently not in this field and haven't been paying attention to the literature) than the actual vector mapping (which is what people always seem excited about). I would love to see more talk of "what questions are these models weirdly good at answering versus questions where they seem so limited as to almost be useless".

<comment><USER22> _> it is still somewhat surprising to me that you can get as good results as people claim without a nearly infinitely dimensional space_

As I understand it, the _maximum_ number of dimensions required is equal to the number of words. That is, if you did no dimensional reduction, you have a vector that expresses exactly how close occurrences of the word in question are on average to occurrences of each and every other word.

That's a very large number of dimensions, but hardly infinite.

Reducing the number of dimensions turns "distance from every other word" into "distance from abstract concepts", except that "abstract concept" is overstating the case, as the "concepts" aren't features of human cognition per-se except to the extent that those features are reflected statistically in the corpus that was used. Besides, the choice of the _number_ of dimensions to reduce to is somewhat arbitrary, and no one knows right now what the "correct" number is, or even if there is a "correct" number. I'm not even sure whether much work has been done on the sensitivity of the models to the number of dimensions.

There is probably a _lot_ of productive work to be done on dimensionality reduction techniques that make the reduced dimensions map better to abstractions that a human would recognize, at least faintly, as well as work to create corpora that better sample the full range of human expression in as compact a size as possible.

</comment></comment></comment></comment><comment><USER23> I tried using the danish .bin with fastText predict and a single danish sentence but I keep on getting an assert error from vector.cpp around A._m not equal to _m. Am I doing something wrong?

./fastText predict wiki.da.bin fileWithASingleLine 1

<comment><USER11> These models were trained in an unsupervised way, and thus cannot be used with the "predict" mode of fastText.

The .bin models can be used to generate word vectors for out-of-vocabulary words:

      > echo 'list of words' | ./fasttext print-vectors model.bin

or

      > ./fasttext print-vectors model.bin < queries.txt

where queries.txt is a list of words you want a vector representation for.

</comment></comment><comment><USER24> If you want to try out fastText without having to do any local setup, see [https://github.com/floydhub/fastText](https://github.com/floydhub/fastText).

FloydHub[1] is a deep learning PaaS for training and deploying DL models in the cloud with zero setup.

[1][https://www.floydhub.com](https://www.floydhub.com) Disclaimer: I am one of Floyd's co-creators

</comment><comment><USER25> It doesn't say what data these were training on. That is kind of important information. I previously applied word vectors that were trained on news to social media posts and it didn't work well at all. Also I don't think there is a language called "Western".

Also interesting that this is hosted on S3.

<comment><USER11> These models were trained on Wikipedia.

It should be "Western Frisian" instead of "Western" ([https://en.wikipedia.org/wiki/West_Frisian_language](https://en.wikipedia.org/wiki/West_Frisian_language)). Thanks for the catch!

<comment><USER25> Thanks.

</comment></comment></comment><comment><USER26> Can someone please ELI5 why this is good, and what they can be used for? I'm assuming machine learning...

<comment><USER27> I haven't anything with fastText, but I have with word2vec. It embeds each word in a 300 dimensional vector, such that similar words have a large cosine similarity. (If you normalize each vector to have a unit norm, then cosine similarity is just a dot product.) So in short, it gives you a measure of how similar each word is to other words.

This has many uses in machine learning. You can extend it to documents and find similar documents, find misspellings, use them as features in a ML model, etc.

There haven't been good vectors in that many languages (that I know of), so that's a plus for these fastText vectors.

<comment><USER26> ah. Thanks!

</comment></comment></comment><comment><USER28> This any useful for sentiment analysis and plagiarism detection? I might give it a go after I'm done with my current projects

<comment><USER29> This will enable writing a plagiarism detector which will not be fooled by the simple strategy of replacing words with their synonyms. Given that synonyms have very similar embeddings, you can compute a distance between two phrases by computing the distance between their word embeddings. And that's just what comes to mind right now.

For sentiment detection, I could see a similar experiment to [1] working, but instead of discriminating between newsgroups, you classify sentiment.

[1] [https://blog.keras.io/using-pre-trained-word-embeddings- in-a...](https://blog.keras.io/using-pre-trained-word-embeddings-in-a-keras- model.html)

<comment><USER22> _> This will enable writing a plagiarism detector which will not be fooled by the simple strategy of replacing words with their synonyms._

I'm not sure how helpful that will be, as you may end up with a system that detects whenever a student expresses similar thoughts (and lets face it, the educational system is all about getting students to conform to conventional patterns of thinking) in their own words.

And if the system _doesn 't_ detect re-expression of the same ideas, then a system that automatically rewrites essays in a slightly different style (essentially, an English-to-English neural machine translation) will defeat it.

The endgame would be grading student essays on how well they express _an entirely original idea_ , which is an unreasonable standard.

</comment></comment></comment><comment><USER20> How does this compare to Conceptnet Numberbatch? [https://blog.conceptnet.io/2016/05/25/conceptnet- numberbatch...](https://blog.conceptnet.io/2016/05/25/conceptnet-numberbatch- a-new-name-for-the-best-word-embeddings-you-can-download/)

</comment><comment><USER30> On their Facebook page [1] they said they are planning to release models for 294 languages very soon.

[1] [https://www.facebook.com/groups/1174547215919768/](https://www.facebook.com/groups/1174547215919768/)

</comment><comment><USER31> What is this?

</comment><comment> torrent-of-ions What is the point of using Github for something like this?

<comment><USER32> It's part of fastText project, which is hosted on Github.

The files themselves appear to be hosted on s3.</comment></comment>


Finding Product Market Fit: Feedback to an Entrepreneur - <AUTHOR> https://medium.com/@nicolaerusan/finding-product-market-fit-feedback-to-an-entrepreneur-9ff0069362c5

<comment><USER2> Overall totally agree with the insights. The part where I wouldn't think about it differently per say but where I would describe it different is about the pricing issue. You have to figure out how much value you create and try to capture as much value as you can. I think you can approximate the value that is created in B2B situations albeit very roughly and not in an exact manner. Pricing wrong is always bad but I agree that how much it cost someone to make something is irrelevant to what price you should charge and maybe you will get people to realize that with your comment "charge a lot". If you increase price but quantity decreases such that profitability is reduced not ideal. Also it would also depend on if there were same or cross side network effects for your product (I.e more users equals more value for your two sides network)</comment>


Snowden Task Force Damage Report is Redacted - <AUTHOR> http://www.scribd.com/doc/256849173/Snowden-Task-Force-Damage-Reports-REDACTED

<comment><USER3> The idea that the government would release to a news agency (or anyone in the public) a document outlining how the "Snowden leaks" damaged national security is mind numbingly ridiculous. I agree with the comment that this at least acknowledges that these documents, and this analysis exist (for future requests). However, I wouldn't expect to see this information revealed until long after it's relevant to US security (think decades or more).

Unfortunately this means that most involved in this debate, will be without a complete picture of what benefits, or costs have incurred from either these "programs" or the leaks themselves.

We're (in the public) left to argue about whether or not we're "safer" in the most abstract terms. Both sides can point to terrorism, or the lack thereof without actually working with 100% of the data and therefore unable to make any conclusions.

<comment><USER4> I agree with you that it's unlikely we'll see this information any time soon, but I want to point out that I think the government did the right thing in their response: they admitted they had such a document while at the same time saying that they believed the content should be classified.

This allows the discussion to move forward in a meaningful way, as people can petition to have the classification reviewed, asked for it in other contexts, etc rather than having just been flatly rejected here without any recourse.

This is a discussion between the government and the public through the appropriate channels and following the appropriate means (at least, so far). I'm not sure what else people could want, in this context.

<comment><USER5> The problem is that the U.S. Government had adopted a policy of "classified by default". It's impossible for us to have a public debate on this issue when anyone who knows about it is legally not allowed to discuss it with anyone who does not have the proper clearance.

It's important to note that discussing the content of a classified document - even if that document has been leaked and is in the public sphere, and regardless of whether or not the information really should be classified - <USER2> punishable by serious jail time. This effectively prevents public debate about any program considered "classified", which means if you're doing something shady that would be embarrassing if it came out, you just slap a security clearance on it and claim its for "national security", thus precluding any government officials from acknowledging it.

As more and more government programs are considered classified, the people are given less of a voice in how our country is run. That is the most scary part for me.

<comment><USER6> And yet the same people complaining about classified by default as a de facto government policy are pushing for encryption by default everywhere.

<comment><USER5> This is actually kind of the opposite thing. Individuals should be able to have privacy from the government; but the government works for the people and should be more transparent.

There are definitely things that should be classified - but in classifying everything, you run the risk of the "classified" designation losing its meaning. In Citizenfour, this was something that Snowden explicitly stated: he was leaving it to the discretion of the reporters which information should be published and what should not, because his biases weren't necessarily correct. But this is because so much classified information shouldn't be classified that someone like Snowden can't really differentiate between what is important to keep secret and what isn't.

<comment><USER6> No more than encrypting information causes encryption to lose its meaning. Something that should be declassified still can be. It just means that the default state is different.

Would you say that if your innocuous slashdot browsing was encrypted even though it "shouldn't be", someone like you or me would become unable to differentiate between what is important to keep secret and what isn't?

</comment></comment></comment></comment></comment></comment><comment><USER7> While it's amusing, it's not nothing.

The release of the document, even redacted, means that it exists. Proof that it exists gives legal purchase to challenge the redactions moving forward or to seek alternative means of disclosure on other grounds.

</comment><comment><USER8> So much for "Freedom of Information". This looks like straight from onion news it's so ridiculous.

<comment><USER9> Well, I don't think anyone expects FOIA to apply to genuinely classified information. So I guess the issue is whether it actually needs to be classified.

<comment><USER10> Some of us, though, think that "classified information" held by governments is an anachronism on its face in the information age.

And so yes, I expect that FOIA will apply to every shred of knowledge held by government once this misconfiguration is corrected.

<comment><USER11> (Sigh). Some things absolutely need to be classified: nuclear launch codes, specific positions & plans of the military, all sorts of military secrets (spy information, knowledge that gives us an advantage against any enemies), login credentials for government officials, encryption keys for things, etc.

Do you really want everyone in the world to know the exact location of our nuclear weapons? Top speed of our submarines? Blind spots of our radar systems?

<comment><USER10> > the exact location of our nuclear weapons? Top speed of our submarines? > Blind spots of our radar systems?

I don't mean to sound brash or simplistic, but the simple answer is that I don't want my government to have any of those things. I'll be safer and more empowered when my government disarms, de-escalates, and makes its goings on completely transparent.

As a thought exercise: how much longer do you think the world will abide nuclear weapons? Are they just here forever?

<comment><USER11> I'm going to leave the whole first part alone, just so this doesn't devolve TOO far.

>> As a thought exercise: how much longer do you think the world will abide nuclear weapons? Are they just here forever? They're probably here as long as there's a desire to utterly wipe a large area off of the map. Maybe the USA doesn't want them anymore, but many, many other countries want them and will continue to work towards getting them. Having them gets you a seat at the power table, and people have to listen to you. Even if they're not used, they're an extraordinarily valuable tool.

<comment><USER12> >> Having them gets you a seat at the power table

What? A death sentence, you mean, only by trying to have them.

</comment></comment></comment></comment></comment></comment></comment><comment><USER13> I suppose releasing something that is completely redacted isn't the same as releasing nothing. That's the only scenario I can think of that doesn't make this look like self-sabotage. Makes you question the usefulness of FOIA requests.

</comment><comment><USER14> You are all idiots and don't understand, clearly, how FOIA works. B5, one of the exemptions the government uses here, is the delibertiave exemption that is one of the most abused and doesn't have to be used. It's done by choice. The fact that the government would classify EVERYTHING is ridiculous not that they wouldn't release a framework about the leaks. Try doing some research about the FOIA process.

</comment><comment><USER15> Link to the real article: [https://news.vice.com/article/official-reports-on- the-damage...](https://news.vice.com/article/official-reports-on-the-damage- caused-by-edward-snowdens-leaks-are-totally-redacted)

</comment><comment><USER16> Here is my CN:

G: You said something that hurt my feelings!

C: What did I say?

G: I'm not telling you but it hurt my feelings!

</comment><comment><USER17> amazing! more than 6 months of hard work to strip out 140 pages of info. My 5 years old son could do the job faster.

</comment><comment> Paul-ish There seem to be a an unredacted footnote, for whatever good that does.

</comment><comment><USER18> At least we know how many bullet points are in the documents

</comment><comment><USER19> this document is to be declassified in 2038, according to the poorly-blacked- out parts.

we won't know the truth for another 23 years.

</comment><comment><USER20> Welp, I'm convinced. Page 76 is key.

</comment><comment><USER21> Releasing evidence to support claims is a quaint relic of when America was a Republic beholden to its citizens.

<comment><USER22> This wasn't released to prove their point. This was released in response to a FOIA request.

<comment><USER21> " In fact, the existence of the DIA's report had been unknown until the White House secretly authorized the declassification of select portions of it so two Republican lawmakers could undercut the media narrative painting Snowden as a heroic whistleblower."

[https://news.vice.com/article/official-reports-on-the- damage...](https://news.vice.com/article/official-reports-on-the-damage- caused-by-edward-snowdens-leaks-are-totally-redacted)

<comment><USER22> And... ?

Also, no where does that article substantiate that claim, and calling a DIA report an internal administration report seems to indicate that they don't really know what they're talking about. The DIA has several congressional oversight committees, including the House Intelligence Committee, that would be seeking the same information as well and most likely would have requested and received the report independently of the administration.

Also, this is pretty breathless and baseless:

> "Rogers did not provide evidence for his claims. But the message was clear: > The Obama administration has authorized leaks of its own internal reports > about Snowden for political purposes, but any attempts by journalists to dig > deeper would constitute a national security threat."

No, that's not clear at all. But it sure sounds good.

<comment><USER21> The Obama administration did authorize leaks relating to Snowden for political purposes, are you claiming that did not happen? But please do go on defending the fantastic, transparent, freedom loving US government which strives to weed out corruption, protect whistle-blowers, and zealously defends constitutional rights.

<comment><USER22> What leaks, again? That the US gov thinks he caused national security damage? Well, duh, senior. That's not a leak, that's common sense that they would think that.

I just like to look at things objectively, but you clearly appeal mainly to emotion. My viewpoint on this has nothing to do with my viewpoint on the government.

<comment><USER21> "A Pentagon review has concluded that the disclosure of classified documents taken by former NSA contractor Edward Snowden could "gravely impact" America's national security and risk the lives of U.S. military personnel, and that leaks to journalists have already revealed sources and methods of intelligence operations to America's adversaries. At least, that's how two members of Congress who have read the classified report are characterizing its findings. But the lawmakers — who are working in coordination with the Obama administration and are trying to counter the narrative that Snowden is a heroic whistleblower — offered no specific examples to substantiate their claims."

[http://foreignpolicy.com/2014/01/09/congressmen-reveal- secre...](http://foreignpolicy.com/2014/01/09/congressmen-reveal-secret- reports-findings-to-discredit-snowden/)

try reading, vice links that article in the very quote I used. Well, duh, senior stooge.

<comment><USER22> So, to you, the leak is that the pentagon review thinks Snowden damaged the US's national security? That's not a leak.

The two members of congress had access to that report because of their position on the House Intelligence Committee. It was not leaked to them.

You've basically cited the same thing three times now, and it doesn't say what you think it apparently says. I understand you're deeply invested in this emotionally though. So I am done discussing this with you.

<comment><USER23> You are a fucking moron. The report was leaked to these two Republican lawmakers and these Republican lawmakers said the WH authorized the leak so they could "undercut" the Snowden narrative.

You need to do some serious reading comprehension. Seriously, you're a fucking idiot.</comment></comment></comment></comment></comment></comment></comment></comment></comment>


Google breaks their own rules and becomes evil - <AUTHOR> http://xebidy.com/google-breaks-their-own-rules-and-becomes-evil/

<comment><USER2> I miss hoodwink.d.

That said, I am unaware of any uproar over this. Am I living a sheltered life?

This sounds more like what the Web is _supposed_ to be like rather then anyone being evil.

xebidy, do you fear Greasemonkey as well?

I bet it takes less than 255 characters of JavaScript to thwart it if you don't want it on your site.

<comment><USER3> A recent Gillmor Gang got pretty hot over it.

</comment></comment><comment><USER4> What an _incredibly_ lazy, link-bait, uninformative, unoriginal headline.

It's about the Google Sidewiki - <http://www.google.com/sidewiki/> \- released just over a month ago.

</comment><comment><USER5> Imagine if Google actually did something evil? We'd have to invent a new term for it. This feature could be annoying, sure, but if you're the type of person who installs Google Toolbar that's probably not a big concern. There are many times I would like to be able to publicly flag a site for inaccuracies, outdated information, content theft, or simply say I think this guy is silly and you shouldn't listen to him. I understand why the site would want total control but it's the Internet -- it doesn't work that way. The end user, more or less, has total control over how they want to get information.

</comment><comment><USER6> Anything that requires a toolbar installed first does not qualify as 'evil', at worst it qualifies as 'stupid' by the user that knows how to download a toolbar but then doesn't know how to disable it.

linkbait + blogspam = flagged.

</comment><comment><USER3> Question: Does the sidewiki area indicate in any way that it is not part of the website? Does the url bar only show the sites url, or does it also show some indicator of the side wiki?

<comment><USER7> It comes with a border, it's blue, it has a title "Google Sidewiki" and it displays your name. I would say it can't be confused with the components of the website itself, although with Web 2.0 sometimes sites get built in a way that components that are parts of the site look like they're not.

</comment></comment><comment><USER8> You mean, in the same way that Hacker News, Slashdot, Reddit, Digg, etc., is evil for allowing side-channel discussions of other people's websites? Give me a break.

</comment><comment><USER9> Sidebars = Evil?

We've come a long way 'Evil' from Nazis to sidebars....</comment>


An Inside Look at CloudKitchens - <AUTHOR> https://superfood.substack.com/p/exclusive-an-inside-look-at-cloudkitchens <comment><USER2> I'm pretty bullish on this concept finding market fit, probably less confident about the business model being great. The super interesting feature IMO is how much this lowers the cost to enter the market. I imagine this are super short term leases (maybe...per shift?), no need for POS, simple bulk ordering etc. shifting capex to opex for small businesses will open up the market to a lot of new entrants. I think a comparison to AWS is fair here.

On the financial side, it sort of seems like a race to the bottom for the landlords. I don't think they'll have anything like the lock-in AWS has and given how significant the expense will be I would imagine teams being pretty willing to jump for a small savings (which goes to straight to the bottom line). IMO this would be a great move by one of the apps to own, as controlling distribution would lock tenants in.

<comment><USER3> I'm bullish on the concept and business model from an investment standpoint, I think they are likely going to become very successful, but as a consumer I can't say I'm looking forward to delivery platforms being flooded with these "cloud kitchens".

It seems to be designed to enable mediocre restaurants to "pivot" at a moment's notice and rebrand themselves after building up a well-deserved crappy reputation, ready to start anew and fool more unsuspecting consumers. On the other side of the coin, well-loved restaurants could disappear at a moments notice because the owner/chef got tired of making the same type of cuisine and want to experiment with something new or more lucrative/trendy.

I think the agility this model offers restaurants is both a blessing and a curse in this sense, as with traditional restaurants the physical presence involved makes the brand more enduring, for better or worse.

<comment><USER4> >> well-loved restaurants could disappear at a moments notice because the owner/chef got tired of making the same type of cuisine and want to experiment with something new

So let's say you're such an owner. Just sell your business, and start something new.

But this could make things really interesting: What if you could order any meal you read about on the web, from anywhere ?

It's not possible for every dish, for the lack of local ingredients. But maybe it's possible for many dishes ?

<comment><USER5> A robotic ingredient picker could be really useful in that context. Make a large kitchen with a ton of ingredients. When an order comes up, a robot picker automatically brings the sauce ingredients to the saucier, the sautee ingredients to the sautee station, etc...

</comment></comment></comment></comment><comment><USER6> For those that haven't been following this, CloudKitchens[0] is a still somewhat stealth "startup". It was started by Travis Kalanick (of Uber fame) and he has been doing a fairly good job keeping it hidden[1].

[0] [https://www.cloudkitchens.com/](https://www.cloudkitchens.com/)

[1] [https://www.ft.com/content/769409f1-fb0b-4761-b2aa-e75b9792c...](https://www.ft.com/content/769409f1-fb0b-4761-b2aa-e75b9792c514)

<comment><USER7> Wonder if one part of their secret sauce will be skirting health code.

<comment><USER8> Do you mean the kitchen skirting codes or the tenants?

I don't know that much about health codes, but my guess is that they are not expensive to follow beyond the cost of having suitable equipment, which is something they probably won't be able to avoid.

<comment><USER9> Health codes in saner municipalities which are the municipalities occupied with people that have the most disposable income are extremely expensive to follow as they dictate enormous number of variables, ranging from what can be stored and how to what kind of cooking process one must follow.

For example, AC banned partially pre-cooking of poultry around 2012(?) at which point Revel's restaurant had to can its most ordered and one of the most profitable dishes as its guests were not interested in waiting for nearly an hour for a chicken to be cooked from scratch. They used to partially cook it, stick it into the fridge, wait for the order and finish cooking it in ~15-20 minutes. It was a total disaster for the restaurant.

</comment></comment></comment></comment><comment><USER10> Rent is typically only 7% of total restaurant operations costs. See Chipotle's quarterly earnings filings for example, and any restaurant industry analyses.

Labor and food costs make up 60% of costs, with them being roughly equally divided. Restaurant margins are around 5-7%. I don't see how CloudKitchens is supposed to fundamentally alter the restaurant industry, making it cheaper. Maybe I'm missing something.

<comment><USER11> One of the biggest problems of a kitchen is having food go bad. Nightly we have to throw away food because we don't have another purpose for it. When serving a singular cuisine it is tough to find ways to create new dishes that fit into the menu. Ghost kitchens become interesting because assuming you're running 3-10 "restaurants" out of the space, the overlap in ingredients becomes large which allows us to reuse and repurpose ingredients easily. This plus the volume of orders allows us again to reduce overhead costs of purchasing. Often times, it takes the same amount of effort to cook for 50 covers vs 500 but clearly the latter yields more profit.

<comment><USER12> I'd like to see a willingness for restaurants to say "I'm sorry, but we are out of that this evening." This would help eliminate food waste by not having to make a batch of something for only 1-2 customers.

<comment><USER13> Some restaurants do. I've certainly run into exactly that response on a number of occasions.

</comment></comment></comment></comment><comment><USER14> Many of the restaurants that can go delivery only already did, like pizza places and some Chinese restaurants. But the problem is two fold, many food items don't lend themselves to delivery, and then delivery itself is expensive and doesn't scale either.

A large part of the restaurant experience is dining in, even if that doesn't include the hospitality, the freshly prepared food will be irreplaceable. The moment it's packed in boxes a lot of the food loses its edge and items prepared by many chefs will all seem similar. If this concept takes it will take the quality of food down globally.

However I am highly skeptical about it and the main reason is delivery, which I still think is a fundamentally unviable business.

<comment><USER15> The problem I have with arguments that are bearish on food delivery is that they are all focused on supply-chain challenges and ignore the obvious massive demand for the product. Of all the problems you can have in business, the only one that cannot be overcome is a lack of demand.

Widespread 1-day delivery of retail goods would have seemed totally absurd and unscalable just 20 years ago ("What, are we gonna FedEx overnight everything? Ridiculous!") and yet here we are. When there is demand, supply will rise to meet it almost every time.

Someone will figure out the logistics to make this work and become the Amazon of food (perhaps Amazon?).

<comment><USER14> Well theoretically you are right but practically speaking even amazon couldn't really change distance. The whole allure of shipping things from a warehouse and not paying rent is true but at the end of the day amazon ended up setting warehouses all over the country to meet the demand for same day delivery. So it did reduce the real estate costs but it's still has to stock all items close to the customer. So the efficiency is in cutting display and showroom costs. Food doesn't fall under that category as it's very different even from perishable grocery items. That is the source of skepticism. I agree there is demand but it only means that the business will never die, doesn't mean that it's going to overhaul the industry. Let's see though. I do hope some innovative idea comes along.

</comment></comment></comment><comment><USER16> At some point in my life I became extremely fussy about wanting to know where my food came from — particularly the meat — and the conditions under which all of it was produced. Vegetables, cereals, meat, dairy, the lot.

No more spring onions from farms employing exploited labour. Certainly nothing imported from countries without labour regulations or farming standards. No more indoor only "free range" chicken etc.

I've met the cows I eat and the hipsters who grow my cabbage. Both are very pleasant and show no signs of duress. All my suppliers have first names and Instagram accounts. There's lots of evidence, alongside the produce itself, to cross reference their credentials. The market exists for more expensive, more carefully reared and more kindly cultivated produce of all kinds.

I trust only a handful of restaurants to get these details right and have had to accept that eating out is even more of a luxury than it used to be. None of the trusted restaurants are the local ghost kitchens, nor are they likely to be. Competing on price and efficient goes completely against the, ahem, grain.

I don't want food to be cheap, fast, or efficient, and rarely — if ever — _to be prepared by strangers I will never meet._

<comment><USER17> This is an amazing level of privilege to the point that it almost seems like Silicon Valley parody. I completely agree with the aim (animal welfare, fair wages), but just want to note that 99.9999% of the population don't have the means (money, time, local availability, etc.) to live this way.

<comment><USER16> I disagree that I am privileged to have access to the fat of the land, but if you want to talk privilege then remember this in the context of:

1/ paying someone else to cook for you

2/ paying another person to deliver this food to you

3/ funding a fleet of SWEs and marketers to run the whole thing for you, by paying a double digit surcharge over what you might have paid if you'd gone to the restaurant in person.

I would say that the privileged are those funding the Seamless / Deliveroo serfs. I can't afford to.

<comment><USER17> Maybe things are different in the UK, but in the US the only people with the time and money to be as choosy as you seem to be are the very privileged. Millions of people barely have their heads above water and they'll take the most efficient (in terms of time and money) calories they can get. Having the time to do the kinds of vetting you seem to do is a privilege itself. I hope the day comes where everyone has the means to be this conscious of their food, however, as I agree with your motivations (at least insofar as I understand them in an online post :P).

<comment><USER16> Ironically, aren't jobs like _ghost kitchen line cook_ and _gig economy delivery rider_ behind the lowering of the quality of life for the "have- nots"?

All this while, at the same time, the "haves" continue to order their brunch using Colins-Organic-Chicken.app.

</comment></comment></comment></comment></comment><comment><USER18> "It will soon be just as cheap to have your food prepared and delivered as going to the grocery store and making it yourself."

I doubt this very much.

<comment><USER19> Exactly.

Until preparation and delivery become roboticized, it is _absolutely not true_.

I live in New York City, eat meat, and it's easy to spend only ~$300/mo. on groceries for the month if you're paying attention to prices and cooking yourself.

That's $3.25 per meal. Please show me how exactly I can get nutritious full prepared meals at this price, _let alone_ delivered...

<comment><USER20> Thought exercise: I've often thought that some sort of "CDN for food" would be necessary to make food delivery competitive with groceries and home cooked meals.

How it would work is:

1\. You tell the service that within a certain time slot, you'd like your order of [insert order here] to be delivered.

2\. All your neighbours do the same

3\. At the time slot, a vehicle (or robot) deposits the food in some kind of storage locker for you and several of your neighbours to pick up.

This would substantially drive down the per-order delivery cost.

A dense city where everyone lives in apartments, such as New York, would be an ideal place for this idea, since orders for everyone in your building could be dropped off at one point

<comment><USER19> It sounds great in theory.

The problem is that people want 30 different types of food (Chinese, Italian, Ethiopian), they don't want it cold, and while NYC is dense it's not dense _enough_.

Even in a building with 100 apartments (which is much larger than the norm), at _most_ maybe 2 apartments will want the same type of food in the same half- hour slot. It's not going to be much different from what a delivery driver already does, which is take out 3 orders from the same restaurant to 3 different buildings at the same time.

And remember that's just the delivery aspect. The costs of food prep are massive too.

<comment><USER4> What if you had 30 different restaurants at the same place ?

<comment><USER20> Exactly what the OP article is about, but yes, that would be perfect for this type of application.

Cloud kitchens, with dozens of restaurants, delivering food to a single drop point, serving dozens of residents at once, would enormously drive down delivery costs. Likely to well under a dollar per order.

I think the biggest stumbling block with this idea is that enormous amount of customers that would be needed to make this operation efficient. You'd need to convince hundreds of people within a neighbourhood or apartment block to give up cooking, and rely on the delivery service

</comment></comment></comment></comment></comment></comment><comment><USER21> I hope the copy writer got paid up front. With almost half a billion in VC debt, GhostKitchens can afford it.

<comment><USER22> The writer is ex + early Uber, I'm sure he's doing fine.

<comment><USER21> Well, they do say organic is best if you can get it.

</comment></comment></comment><comment><USER23> Cloud kitchens also consolidate geographically associated, demographically rich, heavily identifiable consumer data in ways we haven't seen outside of retail, and with more richness and more intimacy than retail is capable of.

Amazon might sell you everything else ever made, but they aren't feeding you a hot meal, not learning what you're in the mood for, what you like to share with your friends and family, what experiences you prefer and reject down to individual condiments on an individual item, not yet, and then find you and physically go to you where you live, and where you work, and where you hang out — these vertical cloud kitchens will.

This is a goldmine for a very narrow slice of the tech economy, and like many tech disruptors the efficiency gains wrung out of the model won't make your life better as much as it will hurt the places you already use, all to benefit someone else the most.

The price or quality won't (and in places where these are already common, doesn't) matter as much as the information these extremely vertical operations can aggregate.

</comment><comment><USER24> I was introduced to the idea of ghost kitchens a couple years ago. Initially, I was pretty keen on the idea. But at the time, we already had ghost kitchens operating in LA, and since then we've seen more open up. What have I not seen? Cheaper costs as a result. Now, delivery apps are flooded with a bunch of companies I've never heard of, I don't know if they're good, I don't know if I can trust them, and at the end of the day they're simply not cheaper for the consumer (one of the touted advantages).

I still think the idea has potential, and it sounded like Cloud Kitchens was genuinely doing some innovative stuff (I interviewed with them﹡), but I'm not convinced that potential will actually be realized. If the market shows it is willing to bear $X for some Nashville hot chicken, what motivation does an operator in a ghost kitchen have to undercut that rather than try to increase their razor thin margins?

At the end of the day, my excitement around the idea is around enabling entrepreneurial chefs, but it's sort of like a talent show - for every 1 great talent, there will be hundreds of less stellar entries, plus the inevitable business folks. It's an interesting idea, and it'll be interesting to see how it plays out over the coming years. That said, I'm more likely to support good brick and mortar companies that help build up the neighborhood community (especially in a city like LA that generally lacks a sense of community).

﹡They spent more time touting their FAANGU(ber) employees than selling the company and ultimately the in-person experience was one of my most loathed interview experiences. This was awhile ago, things could have improved since then.

<comment><USER25> > They spent more time touting their FAANGU(ber) employees

To me this seems like a major red flag and one more sign of them planning to build an engineering playground as opposed to actually solving the problem their business claims to want to solve.

</comment></comment><comment><USER26> How often do other HN users get takeout and delivery in an average week?

For myself I usually grab one takeout meal a week. I haven't had food delivered in years.

The proliferation and mainstreaming of food delivery with all the food apps is one of those big trends that totally passed me by. I'm not really happy about the whole thing since it seems to only be working because of squeezed independent restaurants, low paid gig-workers, and lots of VC cash injections. In it's current incarnation, it's not really sustainable but with Covid killing off a huge swath of restaurants that calculus has totally changed.

I expect that to only last for a while though. Restaurant people are an independent and cantankerous group. Ghost kitchens might be a good place to start - like food trucks - but most will want to move on to their own place eventually.

<comment><USER27> Even before COVID there were a lot of trends supporting food ordering among the working class: depression, anxiety, car-lessness, weed legalization, etc. I'm not happy about the volume I've been hitting ranging from maybe 2-5 times a week.

<comment><USER26> Yeah, if you don't have the space to cook in, or are shuffling between multiple jobs and don't have time to shop/cook then takeout is the only real option.

I'm not knocking takeout and delivery at all. The huge proliferation of it is just alien to how I operate so it's interesting to hear what other people are up to.

<comment><USER27> I knock the monopolistic-capture by organizations like Ubereats who have to take a huge chunk to justify their VC investment. Restaurants need every ounce of margin they can get if they're ever going to have a hope of adequately compensating waiters, linecooks, and other staff, who are extremely exposed to covid and poorly treated in general - not to mention drivers.

</comment></comment></comment></comment><comment><USER28> > As with any new technology,

Not a technology.

> the naysayers have already published conclusive takes on why this is an > episode from Black Mirror

This is a fairly dismissive treatment; it's quite reasonable to raise concerns about this sort of new business practice.

All in all, a strange, low-content article.

<comment><USER29> I think they mean "technology" in the modern Silicon Valley sense of financial tech, commodification of services, and eventual monopoly rent seeking.

</comment></comment><comment><USER30> This seems very ambitious:

"It will soon be just as cheap to have your food prepared and delivered as going to the grocery store and making it yourself."

It depends on how many variables you bake into the equation. Such as the total time cost spent shopping/storing/cooking/cleaning vs ordering. The amount of food waste in a home kitchen vs commercial. Economies of scale implied in commercial food preparation vs home etc...

It's certainly conceivable to have the overall costs be similar. Or at least close enough that it becomes an irrelevant change to improve convenience. In the same way that we could all grow our own potatoes for less than the store cost, but we don't.

</comment><comment><USER31> > better for restaurants and chefs by lowering the entry barrier to start a > business

Ultimately this winds up better for the owner of the CloudKitchens and worse for restaurants and most likely consumers. Already these services create their own restaurants to compete with the businesses paying them for the space.

How long before they are using the data of a successful place to create a knockoff that ranks higher? It's probably already happened.

</comment><comment><USER32> Is anyone interested in working there? I know they are looking for strong SWEs. dm me (y<username> at gmail.com) if interested.

<comment><USER29> It's crazy how easy it is to rehabilitate your image in the valley. There's literally a book written about early Uber's toxic work culture yet I'm sure you'll get tons of message from those willing to work for Kalanick.

<comment><USER32> I did read the book and i didn't find anything too toxic.

Especially is you compare it to other industry giants (hello stack ranking and sports team mentality.)

</comment></comment></comment><comment><USER33> Not sure if this is mentioned but it sounds similar to another restaurant disruption that gained ground recently, I'm talking about redesigned food courts. The idea was an artisanal food courts that chefs could rent stalls in. I am sure they are still around but I know there was a lot of fuss over their exploitative practices.

</comment><comment><USER34> ghost kitchens serve companies, but who has the ingredients (pun intended) to serve consumers with fresh, personalized meals?

* fresh ingredients? * locations near consumers? * food and cooking licenses? * space for cooking equipment?

amazon, by way of its whole foods acquisition. whole foods already uses humans to create personalized coffee, juices, sandwiches, and burritos on demand.

it would be unsurprising if they are exploring robots to automate simple meal preparation and expand into other areas.

</comment><comment><USER35> "It will soon be just as cheap to have your food prepared and delivered as going to the grocery store and making it yourself."

That's a strong claim.

</comment><comment><USER36> In 18 months, if there is a vaccine, won't the demand for delivery decrease? I order food now for delivery I have no interest in delivering once it is safe to go out.

</comment><comment><USER37> WeWork for kitchens is an interesting mixed bag.

> It will soon be just as cheap to have your food prepared and delivered as > going to the grocery store and making it yoursel

The techno-utopianist I cannot beat out of me is excited. It is rediculous that we avoid economies of scale to save money.

> Drivers

Damn, to think the rest of the US is doing car delivery is ridiculous. NYC's ubiquitous bike delivery is a great example of why density is just more efficient.

> > The "ghost kitchen" would change all of that. In place of small businesses > there would only be a series of tractor trailers staffed with > interchangeable "freelance contractors" cooking, presumably with no benefits > or fixed hours.

Lol I don't think much people get benefits at restaurants already. Still, the quote is absolutely correct, they won't start getting it either. IMO every "gig work" company should be pushing hard as hell for universal healthcare etc., as removes a huge competitive advantage of their competition. Do it for the $$ not the altruism, if you must.

\-----

Here's what I'm wondering about. The food seen for rich people has dramatically improved in the last 30 years. One way to think of this is rich Californians setting an example of "slow-food" / embracing unproductivity at every link of the supply chain to side-step all the grain subsidies and other perverse incentives.

What we're seeing now is the beginning of others cooking for you as a non- luxury item. This is good. I think this means a lot of that bad stuff will be harder to at the same price-point. This is bad for me but perhaps good for society. Anecdotally, I feel like after getting better for a long time, delivery food has been getting worse in NYC as the margins get tighter and competition fiercer.

There's two outcomes;

\- we all eat like shit, as increasingly not only is ordering cheaper than cooking, but the expectation of ordering means people will not be able to afford to take time off to cook[1]

\- Or, since nothing motivates political action like the loss of privilege within leaving memory, this will put new wind in the sails of the fight against the grain subsidies.

[1]: Employment always takes a greater share of our time despite productivity improvements, probably due to [https://en.wikipedia.org/wiki/Jevons_paradox](https://en.wikipedia.org/wiki/Jevons_paradox). this is Capitalism's #1 bug.

<comment><USER38> > _others cooking for you as a non-luxury item_

Interestingly, this sounds more like a regression, since in most urban but poor countries it's already cheaper to eat out than prepare food yourself. This comes down to the cost of labor: a restaurant can buy in bulk cheaper than you can and prepare in batches larger than you can, so they'll be cheaper than you as long as their labor cost doesn't obliterate the advantage. In Europe or Australia it does, so eating out is a luxury; in India or China it does not.

Obviously this is a generalization, there are countries that buck the trend (eg Japan) and some meals that are still cheaper to DIY. But I don't think we'll get to techno-utopia here until robotic food preparation goes from gimmick to mass market, and we're still a long way off.

<comment><USER37> Yes I absolutely agree. While I think it's generally good when luxuries become widespread, this isn't so good if just means some sort of mass middle class is exploiting some underclass. And we must always be vigilant for low labor costs being laundered as some sort of efficient tech (e.g. Uber in suburbs/rural is just inherently inefficient). It's a cruel irony as-under priced labor is _the_ bellwether for _in_ efficiency.

That said, I hope there's something between now and full italicization that can be economically while paying the few remaining chefs well. Cities should absolutely price-control the shit out of these cloud kitchens since they will be so dependent on the public good that is efficient land use.

</comment></comment></comment><comment><USER39> >It will soon be just as cheap to have your food prepared and delivered as going to the grocery store and making it yourself.

No, no it won't. Why would a person claim something so ridiculous? I can understand if you never leave your little Soho / Bay bubble, but do real actual people honestly think this?

<comment><USER40> This seems to be already the case in India with the Dabbawalla system [1], so it's not an outrageous idea - it's simply replacing home cooks with tiny commercial kitchens. They can buy produce in bulk at _much_ cheaper rates than you'll find at your grocery store.

[1] [https://en.wikipedia.org/wiki/Dabbawala](https://en.wikipedia.org/wiki/Dabbawala)

<comment><USER39> It's not the case, because it's not possible for food to be cheaper after prep and delivery. It's cheap, but not cheaper than making your own food from scratch. And there's no oversight or health inspections for this mass prepared food either.

But you know this, it's why you said "seems to be" instead of "is," very nicely done.

<comment><USER40> You're completely ignoring the food cost I just mentioned. For example, the cost of a Big Mac is around 1 dollar. Try making a burger with ingredients from the supermarket for $1, it is impossible. Things easily cost 5-10x as much in retail vs their bulk prices, it is _much_ cheaper to cook for 50 customers at once than for a single meal.

This system is removing all of the other costs of operating a restaurant, so unless you also buy ingredients in 50kg bags for your home, they can definitely make it cheaper.</comment></comment></comment></comment>


People say the music business is suffering, but it's not - <AUTHOR> http://stevemartin.com/stevemartin/2010/12/people-say-the-music-business-is-suffering-but-its-not.html

<comment><USER2> Maybe I'm ignorant, but what do these three checks mean? LA Films Music got paid $0.13 by HFA in October?

<comment><AUTHOR> They are royalty checks, sent to Steve Martin by his recording company.

</comment></comment><comment><USER3> Please please please stop calling it the music business. Music/Movies/Games are not all seperate businesses. They all fall into the category of entertainment. They are in the entertainment business and their competitors are more then just pirates, its everything else out there trying to get your time.

<comment><USER4> There are lots of artists and companies that deal just in music. Sure they have to compete with other forms of entertainment, but "music business" seems like a useful subcategory for discussion. Do you have other reason for disliking the distinction?

<comment><USER3> Mostly because it allows those who use the term to say they are suffering as a direct result of piracy and not increased competition. The more people who are in this mindset then the greater chance there is for people to question the figures that the record lables spit out.

</comment></comment></comment><comment><USER5> It'd be useful to know what those are for. I'd be impressed if he's still making that much from "King Tut" (1978).

<comment><USER6> Probably his banjo album, The Crow.

<http://www.discogs.com/Steve-Martin-The-Crow/release/1979914>

<comment><USER7> ...which won a Grammy last year for "Best Bluegrass Album", and for which he is actively touring right now.

So, yeah, his point is "here are the royalty riches you can expect when you make an award winning album and drag yourself all over the country promoting it."

Kind of explains the whole "Pink Panther 2" and "Cheaper by the Dozen 2" things.</comment></comment></comment>


Introducing Lektor – A Static File Content Management System For Python - <AUTHOR> http://lucumr.pocoo.org/2015/12/21/introducing-lektor/ <comment><USER2> Direct links to project:

* Website: [https://www.getlektor.com/](https://www.getlektor.com/)

* Github: [https://github.com/lektor/lektor](https://github.com/lektor/lektor)

Would be really curious to hear how people like it :)

<comment><USER3> Very nice. I had been building the same thing after wanting something more than pelican and less than wordpress. This seems to fit perfectly.

Is there a roadmap for contributing to the project? I failed to see one on github.

<comment><USER2> > Is there a roadmap for contributing to the project? I failed to see one on > github.

Nope. Hoping to get some feedback to see what people need. It does what I need pretty much at the moment. Just figured i add some docs and share it as some sort of Christmas or whatever else you celebrate present.

</comment></comment></comment><comment><USER4> My feeling is that every serious programmer will end up inventing tools like this simply because the overhead of incorporating other people's tools eventually starts to overtake the overhead of maintaining one's own system.

<comment><USER5> Let's not forget that Lektor solves actual problems. Show me another CMS that offers:

\- Static HTML export with elegant dependency tracking by design (reducing attack vectors and maintenance cost on the frontend server)

\- Admin interface (I can't tell a non-techie to connect via SSH, fire up vi and run Jekyll)

\- Flexible content structure (Title/Author/Content doesn't really scale beyond a simple blog)

\- Elegant design and implementation (I could do anything with WordPress and half a dozen plugins, but I fail to even find a bare-bones theme without kilobytes of code that I will never need; the whole ecosystem revolves around the novice user who understandably prefers "quick and dirty" over touching code)

I have been looking for a CMS that fulfills these exact criteria for quite some time. Lektor looks ideal for projects where the developer prepares a non- trivial page structure (think book listings, music albums, anything that goes beyond _bold_ and _italic_ ), but the client should be able to edit content. Thank you, Armin!

<comment><USER6> A few alternatives....

[http://getgrav.org/](http://getgrav.org/)

[http://getcockpit.com/](http://getcockpit.com/)

[http://harpjs.com/](http://harpjs.com/)

<comment><USER7> Checkout SpudPress - Static WordPress hosting. All the same except you can use WordPress theming and Admin. If you know how to distinguish between static and dynamic. It's insanely useful

[http://spudpress.com](http://spudpress.com)

<comment><USER8> I think a lot of people who would love to use a self-hosted version of this. However, paying for $20/m is usually not worth for a hobby/blog site.

</comment></comment></comment></comment></comment><comment><USER9> I always love Armin's stuff. In quality, it's often near that oh-so-very- special spot that is reserved for Steve Yegge's best posts. Now I will simultaneously be lynched by both the people who think that Steve sucks, and the people who think that Steve's stuff is 1 million times better than anything Armin could ever post. Not to mention 500 other groups who I don't even know.

<comment><USER10> nope, you're upvoted b/c i also appreciate source-level userfriendliness of pocoo projects

</comment></comment><comment><USER11> From the original post:

> There is very little innovation in that space and that's a bit unfortunate

I'll have to respectfully disagree with Armin here, but perhaps I map the terrain a little differently than he does.

In addition to the half-dozen tools he names, which are more narrowly targeted at blog-like static sites, the broader space of asset pipelines and static generation is seeing a lot of innovation. I'd point to Gulp and Rails' latest as a sign that there's more work to do here. Even strictly in the Django world, there are a zillion asset-pipeline-like-apps; django-compressor and django-pipeline, the two most used, are quite different in their approach and philosophy.

I'm excited by this proliferation of tools for generating static web content "offline" and expect to see far more in the future. Lektor looks like another interesting piece to the puzzle, particularly in its focus on end-users rather than programmers. (Anyone remember Movable Type? That's probably the first user-centric static site generator to gain widespread use?)

<comment><USER2> > I'd point to Gulp and Rails' latest as a sign that there's more work to do > here.

If you want to throw them into the same category, then sure, there is different stuff happening. But Gulp is even further away from something non programmers can use.

<comment><USER11> I definitely do see gulp et. al. as belonging to the same category of tools, although they're much more general-purpose. I think that's where I map the terrain differently than you. I do so because I'm hopeful that future innovations _there_ will find their way back to tools like Lektor, etc.

(And completely agree on the programmer-centric nature of these tools.)

</comment></comment></comment><comment><USER12> Armin Ronacher just makes more and more awesome stuff. I think I'm going to ditch my pelican site, and go for Lektor.

</comment><comment><USER13> I'm really suffering from an acute case of Analysis Paralysis when it comes to Static Site Generators. The offerings on the market are overwhelming and you don't know which one to pick, do you go with an incumbent or mainstream like Jekyll with a huge selection of functionality at your disposal or go for the underdog and minimal one offering just the basics or just build your own framework and adapt to the evolving needs of your projects over its lifetime.

If you're thinking that the scene of MVC JavaScript frameworks resembles more of the good ol' days of the Wild Wild West, wait to see how things are currently like on the SSG front.

<comment><USER14> i had this problem a week ago- ended up building an ultra-low-frills low- dependency one in an afternoon just a few weeks ago. Seems silly - i am pretty sure with sufficient time investment i am better off learning how to use popular frameworks (because learning how to use frameworks will make me better at learning how to use frameworks).

</comment></comment><comment><USER15> Considering that one of the project's main goals is to address the needs of non-programmers, do you plan to add an improved Markdown editor (with some syntax highlighting and maybe buttons for things like titles, bold, etc.)?

<comment><USER2> Absolutely. See also the open ticket here: [https://github.com/lektor/lektor/issues/7](https://github.com/lektor/lektor/issues/7)

</comment></comment><comment><USER16> This looks great! The native desktop app is a powerful way to drive adoption with non-technical people.

Cactus was on the right track with that although I'm fairly sure the author, Koen Bok is now focussed on his company, Framer. Are there plans to build clients for other platforms? I guess the community can jump in at any time.

Sidenote: do people know of native clients for the other major static site generators like Jekyll, Hugo, Middleman?

Shameless plug: If you're looking for somewhere to host your static site, please check out my side project, [http://www.knownly.net](http://www.knownly.net)

<comment><USER17> I want to also vote up Cactus despite the lack of attention paid to it lately. IMO it's very well designed and deserves to be worked on by a wider community.

It's open sourced via BSD 3 clause: [https://github.com/koenbok/cactus](https://github.com/koenbok/cactus)

I especially liked the "deploy to s3" feature, IMO the cheapest way to run a highly scalable static site today.

Alternatively, a "deploy to GitHub page" would be neat as well.

I would work on extending Cactus if it was a bigger problem personally, but so far what's released fits my needs already.

</comment></comment><comment><USER18> Very neat. Static site generators are hard to get right (I've been hacking away at a hybrid solution at [http://sushy.no-bolso.com](http://sushy.no- bolso.com) / [http://blog.carmo.io](http://blog.carmo.io) \- same engine, different themes).

I do have two questions, though. Why the new file extension (.lr) and the field separator?

Jekyll's front matter format is easier for most people - and my engine uses a plain newline to separate front matter from text, and supports multiple file extensions to make it easier for desktop editors.

<comment><USER2> The .lr format is easy to explain: it conforms to data models and is easy to parse and generate. Frontmatter does not fit into these requiements. I tried it quite a bit but it makes everything more complex. Not worth it.

<comment><USER18> Well (and this is only an opinion) having to separate each field makes it harder to edit manually, and a new file extension will be a hassle for non- technical users who want to use their favorite editor.

(I went with .txt/.md/.textile to support multiple markup engines and make it easier for people to edit with literally anything)

<comment><USER2> > Well (and this is only an opinion) having to separate each field makes it > harder to edit manually, and a new file extension will be a hassle for non- > technical users who want to use their favorite editor.

A non technical user would use the admin interface.

Obviously a custom file extension is not particularly nice but there is really not that much one can do around that. Renaming the file to `.md` will not magically make things better because the contents are just wrong and the Markdown format just does not do what Lektor needs.

</comment></comment></comment></comment><comment><USER19> > Lektor belongs to a separate org and the project does not use any resources > only I have access to (other than the domain name and the server travis-CI > deploys to).

Really appreciate you starting it out this way from the getgo.

> Because I run so many Open Source projects and maintenance of all of them > turns out to be tricky I figured I do this better this time around.

Looking forward to seeing Flask & Flask-SQLAlchemy moved to a dedicated Flask org on github. Ideally with another person or two having Pypi creds for pushing releases.

</comment><comment><USER15> Armin, are you not afraid that having to install Lektor locally could be a serious obstacle for "people that have no understanding of programming"?

<comment><USER20> I wonder how hard it'd be to bundle up into a single executable that the user could run. I know I've been battling with Python -> exe tools ever since I started using Python, though.

<comment><USER2> There is a bundled cersion for osx with a ui. Just didn't have the time to do it for windows and linux.

</comment></comment></comment><comment><USER21> Respectfully, I don't think this puts this project outside the scope of any other static site generators as listed.

In my view a content only API would be more disruptive. A smart content API as a service could be used for any project. Give it a beautiful interface for authors and focus on the writing experience.

Maybe this is Mediums potential new market/ platform. What if you could grab and query your content with a smart api with graphql? No updates, servers, etc.

Any new project I used, I grab my content from this saas and use whatever tech/framework I want. Why aren't more devs looking at this as an option going forward?

<comment><USER22> Services such as the one you describe do exist, e.g. [Contentful]([https://www.contentful.com/](https://www.contentful.com/)). I've used it on a few projects and it's pretty awesome.

The only thing it's missing from your description is graphql. :)

</comment></comment><comment><USER23> This looks really interesting. I always love seeing new static site builders. Each one is a bit different, which is cool.

I'm working on making hosting for static sites easier. My static hosting service is still in beta, but we have a sign up form if people are interested. [http://www.statichosting.co/](http://www.statichosting.co/)

<comment><USER24> Thanks for the link Brian. The "Get In Line For The Beta" button does not work for me (it does not do anything).

</comment></comment><comment><USER25> Out of topic: When the next version of Flask will be released? Does Flask has a roadmap or it is "abandonware"?

<comment><USER26> [https://www.reddit.com/r/Python/comments/3xi6o9/whats_the_fu...](https://www.reddit.com/r/Python/comments/3xi6o9/whats_the_future_of_flask/)

</comment></comment><comment><USER27> This is great! I'm using pelican and while a very good tool, I was considering building out a desktop/local-front-end for it...But this is _MUCH_ better than what I could build (with my current level of expertise)...i think i'll be using this, and seriously consider contributing! Kudos @mitsuhiko!

</comment><comment><USER28> I love Armin's work, especially Flask, but this seems like lot of other products out there. I have no incentive to switch to this from the generator I'm already using, and for non-techy people I think it's still a big jump in terms of just opening up a tumblr blog.

<comment><USER29> What other static file CMS are you using?

<comment><USER28> I don't use a static file "CMS", but I use my own generator[1] which I made because I wanted something really fast. If you want a flat file cms, then a Google search will throw a lot of them. Kirby is the most popular, IMO. But it's not FLOSS. PicoCMS ticks all those boxes though.

[1] [https://github.com/pawandubey/griffin](https://github.com/pawandubey/griffin)

<comment><USER29> I wanted a "full" CMS and a GUI, and it looks like finally I got one now. I hadn't seen another static generator that had a GUI of any kind (that seemed to be supported.)

</comment></comment></comment></comment><comment><USER30> Nice stuff! (Again and again :)) What's about multi-language website? Tips? Advice?

<comment><USER31> Have a look at [https://www.getlektor.com/docs/content/alts/](https://www.getlektor.com/docs/content/alts/)

<comment><USER30> Thank you ! :-)

</comment></comment></comment><comment><USER32> Urubu (A micro CMS for static websites, with a focus on good navigation practices.) might be an alternative: [http://urubu.jandecaluwe.com/](http://urubu.jandecaluwe.com/)

</comment><comment><USER33> Does it support some kind of live preview mode (like with HarpJS) ?

<comment><USER2> Depends on what you mean by live. If you hit the save button the page renders immediately and you can see it. It rebuilds as files change.

</comment></comment><comment><USER34> Hmm - gonna have to steal this for a project in new year. This comment is more a bookmark than any contribution to society

</comment><comment><USER35> I found sculpin pretty much perfect, but having a built-in admin panel is a nice touch.

</comment><comment><USER36> Why did you call it Lektor?

<comment><USER37> It's German for editor. I'm gonna guess for the same reason that his WSGI library is called Werkzeug (tool).

</comment></comment><comment><USER38> i like [http://www.mkdocs.org](http://www.mkdocs.org), it's also python based.</comment>