A list of command line tools for manipulating structured text data - <AUTHOR> https://github.com/dbohdan/structured-text-tools <comment><USER2> Also csvkit.

      in2csv data.xls > data.csv
      in2csv data.json > data.csv
      csvcut -c column_a,column_c data.csv > new.csv
      csvcut -c column_c,column_a data.csv > new.csv
      csvgrep -c phone_number -r 555-555-\d{4}" data.csv > matching.csv
      csvjson data.csv > data.json
      csvsql --query "select name from data where age > 30" data.csv > old_folks.csv
      csvsql --db postgresql:///database --insert data.csv
      sql2csv --db postgresql:///database --query "select * from data" > extract.csv

[https://github.com/wireservice/csvkit](https://github.com/wireservice/csvkit)

(Submitted a pull request.)

<comment><USER3> Oh man where were you a few months ago where I had to write simple apples just to do some of this stuff... also this will be so damn handy!:

        csvsql --query "select name from data where age > 30" data.csv > old_folks.csv

<comment><USER4> Also see DBD::CSV which been around since 1998! - [https://metacpan.org/pod/DBD::CSV](https://metacpan.org/pod/DBD::CSV)

</comment></comment></comment><comment><USER5> awk is really, really powerful. It is fast, and you can do a lot very efficiently just playing with the 'FS' variable. And you can find it on all *nix boxes. And it works nicely with cli other tools such as cut, paste, or datamash ([http://www.gnu.org/software/datamash/](http://www.gnu.org/software/datamash/)). As soon as it becomes too complex though it is better to resort to a real language (be it python, perl or whatever - my favourite is python + scipy).

<comment><USER6> Awk is great at what it does, but I find myself unable to keep it cached in my brain long enough to reuse it. Using awk usually means a google search of how to use it, which defeats quickly working at a term.

<comment><USER7> awk is a really productive constraint one can oblige to. If a problem looks like a text processing problem, use awk, and let the idioms set in. It is an awesome tool to have on one's belt.

It is, in a sense, liberating to not have any library support for common problems: No need to learn a library (hej!) and, by the way, what you need in any given problem is an easy subset of what that library would do anyway.

Awk has made me focus more on the data there is to analyze, rather than the framework to analyze it with.

As the idiomatic use of awk is also very succinct, I hardly can imagine working efficiently on the command line without it.

<comment><USER8> I take a different stance. I never use awk, because I can use Perl. In using Perl, I also have access to the many modules that can extend it's usefulness. One of the core reasons Perl was created was to fill in where awk wasn't as useful as it could be.

<comment><USER9> I use awk only if it's a trivial one-liner. The only awk I can remember is selecting specific columns in whitespace-separated text. If it's just a single column, I'll try to use cut(1).

For anything more complicated, I'll use Python. Because if it's something that is going to have a somewhat long life, and that I would need to feed into plotting, I'd rather use Python. Because I remember the bad old days of plot(1).

</comment></comment></comment></comment></comment><comment><USER10> HIGHLY Recommended Book:

Data Science on the Command Line

[http://datascienceatthecommandline.com/](http://datascienceatthecommandline.com/)

</comment><comment><USER11> [http://www.nongnu.org/txr](http://www.nongnu.org/txr)

<comment><USER12> I've used some very basic TXR for refactorings that were a bit beyond my IDE's capabilities, which gave me a taste of how powerful it could be. One thing that's slowed me down in experimenting with it is having to save the script, rerun TXR and refresh the output file each time I make a change. Do you have any tips for quickly and interactively building complex scripts?

<comment><USER11> Not in the TXR pattern language, I'm afraid! TXR Lisp has those interactive properties of just re-loading a module on-the-fly to redefine some functions without redefining the entire program, but the pattern language isn't interactive in that way.

This is from the ground up, pretty much, emanating from the way it is parsed in its entirety before being executed.

You can develop the logic in small pieces and test them in isolation on some sub-segment of the data, then integrate the pieces into a larger script with @(load ...).

Speaking of code refactorings, I also use it for that myself. When using it to interactively rewrite code, I invoke it in a pipeline out of vim. E.g. select some range of code visually with V, then "!txr script.txr -[Enter]" to filter it through. Vim deletes the original text and replaces it with the output. If the output is wrong, I hit u for undo to restore the original text. Then of course I have to fix script.txr and save it, and recall and repeat that filtering step just with ":[up arrow][Enter]".

To avoid refreshing the output file during development, don't have one; let it dump to standard output.

Something useful is that we can also monitor the first N lines of an output file that fit into the screen using the watch utility that is found in many GNU/Linux distros. Watch repeatedly executes some command that is given (once every 2.0 seconds by default) and splashes its output on a clear screen. If we "watch cat file", we can monitor the changing contents of file.

<comment><USER12> I didn't think of calling TXR from within vim, thanks I'll try that.

Watch could get me closer to the workflow I'm used to from web development where the browser auto-reloads whenever a change is made. Your mentioning vim (of which I'm not exactly a power user) actually prompted me to check if it had the ability to run a command on writing a buffer or to automatically reload a file when it changes. Turns out it does indeed have ways to do both these things, so that might be another way to achieve what I'm after.

</comment></comment></comment></comment><comment><USER13> Check out pup for parsing HTML. [https://github.com/ericchiang/pup](https://github.com/ericchiang/pup)

pup uses CSS selectors to select elements from HTML documents. Used in conjunction with curl, it gives you a very simple and low friction way to scrape data in scripts.

</comment><comment><USER14> I would add to that list Nokogiri, "The Chainsaw". xsltproc is ubiquitous, but writing xslt is akin to having a pack of wild monkeys compose a mural with their excrement.

</comment><comment><USER15> It isn't easy (perhaps not even possible) to get the name of the fruit Bob grows in his farm using any of these tools and the following data:

      {
        "models": [{
          "title": "fruits",
          "fields": [
            {"name": "Name", "key": "3746"},
            {"name": "Colour", "key": "4867"}
          ],
          "entities": [{
            "_id": "372612",
            "3746": "Orange",
            "4867": "orange"
          }]
        }, {
          "title": "farmers",
          "fields": [
            {"name": "Full name", "key": "8367"},
            {"name": "Address", "key": "3947"},
            {"name": "Fruits", "key": "5243"}
          ],
          "entities": [{
            "_id": "747463",
            "8367": "Bob, the farmer",
            "3947": "Farmland",
            "5243": ["372612"]
          }]
        }]
      }

<comment><USER16> That's true even in Clojure, arguably the simplest and cleanest language ever invented for complex data transformation and extraction.

The Clojure solution to this still ends up requiring temporary variables and some sort of model transformation functionality. (Will try to post my Clojure solution in 5 hours after my next noprocrast timer is up.)

If the data could first be transformed so that it doesn't require temporary variables or ad-hoc transformation function definitions, instead making use of "paths", then it would be easier with command line tools. Such a transformation could be possible as its own command line interface.

<comment><USER17> Yup, lots of temporaries (I haven't written much Clojure recently, so I'm sure I'm missing lots of simplifying core fns).

      (let [; input
            js (cheshire.core/parse-string (slurp (clojure.java.io/resource "awful.json")))
            ; utils
            get-match (fn [k v coll] (first (filter #(= (get % k) v) coll)))
            model-for (fn [title toplevel] (get-match "title" title (toplevel "models")))
            key-for (fn [name* model] ((get-match "name" name* (model "fields")) "key"))
            ; models
            fruit-model (model-for "fruits" js)
            farmer-model (model-for "farmers" js)
            ; keys
            fruit-name-key (key-for "Name" fruit-model)
            farmer-name-key (key-for "Full name" farmer-model)
            farmer-fruits-key (key-for "Fruits" farmer-model)
            ; values
            bob-entity (get-match farmer-name-key "Bob, the farmer" (farmer-model "entities"))
            bob-fruit-keys (bob-entity farmer-fruits-key)
            bob-fruit-entities (map #(get-match "_id" % (fruit-model "entities")) bob-fruit-keys)
            bob-fruit-names (map #(% fruit-name-key) bob-fruit-entities)]
        bob-fruit-names)

      => ("Orange")

</comment></comment></comment><comment><USER8> I've found fsql[1] to be extremely useful in the past.

1: [https://metacpan.org/pod/distribution/App- fsql/bin/fsql](https://metacpan.org/pod/distribution/App-fsql/bin/fsql)

</comment><comment><USER18> Don't forget cut and sed.

</comment><comment><USER19> pgloader: load from CSV (and others) to postgreSQL.

See [http://pgloader.io/howto/quickstart.html](http://pgloader.io/howto/quickstart.html)

</comment><comment><USER15> Does anyone know of a tool like ranger[1] for visualizing JSON on the terminal? There is a Chrome Extension[2], but nothing useful to browse JSON on the terminal (it doesn't have to be like ranger, I'm looking for any tool that makes it easier to take a look at a JSON file).

      [1]: https://github.com/hut/ranger
      [2]: https://chrome.google.com/webstore/detail/json-finder/flhdcaebggmmpnnaljiajhihdfconkbj

<comment><USER20> If you use vim there's [https://github.com/elzr/vim- json](https://github.com/elzr/vim-json) which gives you folding, highlighting, etc.

</comment></comment><comment><USER21> For converting arrays of objects between formats like CSV, JSON, YAML, XML (WIP), etc... I built aoot[1] which stands for "Array of objects to". It's written in Node.js and uses upstream packages whenever possible.

1\. [https://github.com/montanaflynn/aoot](https://github.com/montanaflynn/aoot)

</comment><comment><USER22> [http://www.commandlinefu.com/commands/browse/sort-by- votes](http://www.commandlinefu.com/commands/browse/sort-by-votes) has some cool tips

</comment><comment><USER23> [http://neilb.bitbucket.org/csvfix/](http://neilb.bitbucket.org/csvfix/)

</comment><comment><USER24> And not a single tool for s-expressions?

<comment><USER25> I was after something like jq for s-expressions a while ago, but didn't find anything other than full-blown Lisps, Schemes, or libraries written in other languages: [http://stackoverflow.com/questions/31232843/jq-or- xsltproc-a...](http://stackoverflow.com/questions/31232843/jq-or-xsltproc- alternative-for-s-expressions)

</comment></comment><comment><USER26> xsv for doing queries against CSV files probably belongs to the list too: [https://github.com/BurntSushi/xsv](https://github.com/BurntSushi/xsv)

</comment><comment><USER27> yacc, antlr, lemon, bison.....

<comment><USER14> Definitely. Most software vulnerabilities are from failure to write formal parsers on on all inputs. Is there a command line YACC for compiling simple stuff?

<comment><USER27> "Most software vulnerabilities are from failure to write formal parsers on on all inputs."

That's a good quote

<comment><USER28> This might also interest you? [http://langsec.org/](http://langsec.org/)

<comment><USER27> thanks. Have you attended the conference?</comment></comment></comment></comment></comment>


"Abrupt thaw" of Arctic permafrost releasing more CO2 than predicted - <AUTHOR> https://www.nationalgeographic.com/science/2020/02/arctic-thawing-ground-releasing-shocking-amount-dangerous-gases/ <comment><USER2> I agree with Neil Gaiman and the many, many climate scientists watching this unfold that the Australian, Canadian and Californian fires are only the tip of a very large iceberg. Policy wise I think politics needs to plan to cope with living in a 4-5C world and action wise we need to do everything we can to try and plateau in a 2-3C world.

<comment><USER3> I think 4-5C is a pretty reasonable assumption, over the next century.

The thing is that most of our current action plan for climate change consists of reducing CO2 emissions, and this is woefully inadequate for a world that the most recent science indicates has already passed the tipping point. Among the second-order changes that we should be preparing for:

1\. Migration as the new norm. A 4-5C temperature rise will shift equivalent biomes over 1000 miles northward/southward, and inundate large coastal areas where major populations are currently located. Major densely-populated areas will become uninhabitable, and currently uninhabitable areas will become fertile.

2\. The collapse of governments, and of the nation-state system. When migration becomes the new norm, borders become irrelevant, as do settled homogenous populations. What's the new political organization look like?

3\. Large _temporary_ food shortages. Higher temperatures, higher rainfall, and higher CO2 levels actually mean that in the long-term, plant growth and agricultural productivity will increase, and the earth's carrying capacity should go up. But in the short term - a lot of our current crops are hyper- specialized to their existing climate, and land rights, water rights, & physical crop location become problematic. Food _distribution_ becomes critical - we'll likely make enough food to feed everyone on the planet, but the people who grow it may not want to share (or may have the inability to trade) with the people who need it.

4\. Expect to see more diseases emerge. Large migrations, changes in biomes, and increased human habitation of formerly virgin ecosystems all spell chances for pathogens to jump from animal hosts to people and spread widely.

Overall, I'm a lot more scared of the likelihood that humans will kill each other than that the environment will kill us. We have a unique tendency to blame each other for unwelcome changes rather than to adapt to them.

<comment><USER2> We should also consider that as the ambient CO2 density rises we're also going to lose a lot of our cognitive function. If we're going to rely on nuclear energy to see us through we might need to rely on AI to carry on operations for us.

<comment><USER4> I'm skeptical of that claim without some good supporting evidence. Submarines tend to maintain CO2 concentrations 10x higher than atmospheric, and they have a better reactor safety record than civilian power plants.

</comment></comment></comment></comment><comment><USER5> I am the most worried I've ever been, there's not a lot I can do personally, since it's such a big collective action problem and it's not a good feeling.

<comment><USER6> I am also concerned so I reoriented my career to support environmental advocacy. Forget sorting your recyclables. Go make a career in alternative energy, alternative transportation, geoengineering or environmental advocacy. I guarantee you will be able to do more for this issue than tweeting or donating online. I have saved my organization more in infrastructure costs than I will ever earn in my lifetime.

The paperclip maximizers are here and they will turn the entire surface of the earth into plastic wrap and data centers. Choose to dedicate the productive capacity of your life to something else.

<comment><USER7> Can you tell us more? I'm really curious about this.

<comment><USER6> Sure! There's many jobs out there that will help solve climate and energy. You don't have to found the company yourself or personally invent the new battery technology. Many companies exist already and need all kinds of professionals: software dev, finance, marketing, data analysis and more. Many companies work on products that will help us adapt and mitigate climate change. Tesla is the famous one, but there are entire industries. Then there's the nonprofit space. There are dozens of green groups working in the USA alone, ones you've definitely heard of and ones you haven't. Universities are conducting important research and educating on these issues. And there's government as well, scientists, regulators and policymakers.

I've been working in the nonprofit space for 10 years and trust me when I say that we need highly capable and motivated tech professionals. Go here and start looking at job postings in your country: [https://en.wikipedia.org/wiki/List_of_environmental_organiza...](https://en.wikipedia.org/wiki/List_of_environmental_organizations)

No one person will solve a planet-scale issue like climate change, but each person who decides to pick up a bucket and start bailing water will mitigate the impact. Be the change.

<comment><USER8> More undergraduates emerge each year, with eco-oriented training, than there are places to fill, in the USA. Good "jobs" in this space are often filled with people with advanced training and some level of being independently wealthy, working for below-cost-of-living, no children, etc.. It is ordinary for big dot-orgs to ask for free work, even when there is a budget, here in the US. Behind the scenes, certain groups have pooled money and invested it, while fundraising and asking for volunteers. The truth is, this is partly necessary, because "money" is not being created, money is being spent. There is a long-term fundraising need for this work, and it shows in unexpected ways, sometimes tough to see in action.

</comment></comment></comment></comment></comment><comment><USER9> I wonder why geoengineering proposals, such as seeding the oceans with iron[1], aren't more often discussed as possible mitigations to climate change.

[1] - [https://en.wikipedia.org/wiki/Iron_fertilization](https://en.wikipedia.org/wiki/Iron_fertilization)

<comment><USER10> I wonder why we're still in emitting large amounts of greenhouse gases now instead of trying to fix the problem later.

40% of the worlds electricity is generated using coal.

If we changed this 10 years ago, we'd have less problems today.

If we change this today we'll have less of a problem in 10 years.

We're coming up with all kinds of clever ideas to deal with the problem in 2050. This is a slow train wreck happening over decades.

<comment><USER11> > I wonder why we're still in emitting large amounts of greenhouse gases now

Because enough people make a LOT of money from the status quo

</comment></comment></comment><comment><USER12> The "clathrate gun" is terrifying. If it's happening twice as fast as we expected, that sounds bad.

<comment><USER13> We'll all die. It's not so terrifying as depressing. Clathrate gun means death for essentially everyone. Maybe Bezos and Musk are right to make off planet life temporarily viable. Modern society is based on relatively predictable large scale agriculture being able to redistribute goods (transportation infra, tech to support distribution). We simply, as a society, are not capable of adapting to those circumstances. Most humans would die and the survivors would not enjoy a modern lifestyle.

<comment><USER14> The nice thing is I'm not saving anything for retirement since the world is ending soon. I'm laughing at all my coworkers worrying about their 401K matches.

<comment><USER13> That's not smart. Just look at the simplified game tree. What's your utility of what you're putting in a 401k? If we save the world you won't be destitute in old age.

<comment><USER15> Not op but I'd be very happy to be destitute in old age just by knowing humanity saved the world.

</comment></comment></comment></comment></comment><comment><USER16> I feel like I keep seeing articles covering climate-related events/disasters happening Sooner Than Expected™. I'm already doing my best to reduce my personal impact and raise awareness in my social circle, but it's worrying that the biggest doomsday predictions might actually come true.

<comment><USER17> The change needs to be systemic, which is the problem. Logistics, industry and agriculture make up the vast majority of emissions. But for these things to change prices will have to rise probably quite dramatically.

We've gotten so used to not paying for the externalities associated with emissions it's damn near impossible to shift gears. Even at the individual level people are often only willing to do it when the cost increase is marginal.

If we taxed emissions at every level things would get so expensive people would riot.

I think we just have to accept that our society is not designed in a way that we can deal with this. We will have to let the catastrophe happen before there will be any real change. Even then their may not be change. We just have to wait for enough people to die for the system to fall back into balance.

<comment><USER18> Upvoted.

Your observations, calculus, and conclusions are correct.

I'd only suggest that we're already paying for those externalities. It's just that those costs are off book. So the feedback loops are broken. Making improvements all but impossible.

A carbon tax would update our accounting rules to capture those costs in a direct and efficient manner.

<comment><USER19> The fundamental problem with the Carbon Market is that we are missing the entire second half of the equation. There is no viable large scale carbon removal technologies that would allow us to set a price per ton of carbon released. The market doesn't work because it is all supply and no demand.

If we were serious about this there would be hundreds or thousands of companies competing on the most cost effective way to remove carbon from the atmosphere. All companies would be required to buy enough carbon removal to make them zero net emissions. When you buy a gallon of gas it would have whatever the current market value of 5.5lbs of carbon removal factored in right at the pump. Or that would happen at the refinery and be reflected in the final price of the gas.

Unfortunately if you wanted to jump start this market it would be very disruptive to the global economy. Governments worldwide (it's too easy to export pollution, this needs to be a worldwide solution) would have to agree on a very high price to get the market started to avoid undercutting companies wanting to get into it and let the money start forming a pile so terrifically large that people would be scrambling to develop carbon capture technologies faster and cheaper than the competition. There would be a lot of cheating and lying that would need to be policed worldwide. Our current system of governments is not capable of pulling this off.

Carbon markets are something people try to entice capitalists with, but only because they don't think people will look at the details. To actually implement them would send those same capitalists howling and screaming to their nearest governmental body with exceptions and rollbacks in hand.

<comment><USER20> Most serious carbon market proposals are not primarily related to removal/sequestration. They are related to reducing emissions. You get credits for emitting less and/or producing low emission technologies (solar panels, EVs, etc.), and can sell them or use them to reduce your tax burden.

This type of thing works, when done right.

<comment><USER21> Its not that they aren't primarily related to it, most proposals are neutral as to whether a ton of CO2 is avoided or sequestered. It just so happens to be the case that its relatively expensive to capture and sequester CO2 from the atmosphere compared to emissions avoidance. The lowest hanging fruit is efficiency (better insulation, more efficient appliances, cars, etc), followed by lower carbon forms of electricity, etc.

We'll likely need to do sequestration but, assuming a gradually rising carbon price, it won't make economic sense to do it at scale until we've done the more cost effective things first.

<comment><USER19> The nice thing about working markets is that it would naturally pressure people to do the cost effective stuff first. If buying carbon offsets is appropriately expensive you have a major incentive to avoid buying them.

</comment></comment></comment></comment></comment></comment></comment><comment><USER22> I've been reading about and following climate change daily since 2012 and every damn year it's the same thing:

 _Worse Than Expected™_

<comment><USER23> It is because the flow of money that powers climate research, green industry, and climate activism is largely dependent on stirring up fear in the public to garner acceptance and support. Climate change is a real problem but so are the perverse incentives in our research institutions.

</comment></comment><comment><USER24> With the rapidly declining costs of getting payloads to outerspace, I think we should discuss blocking radiation from the sun in some way. We clearly are not going to transition our entire energy system fast enough, but using mirrors/reflectors is a reversible and reasonable way to slow the warming of the planet.

Let's say we take rockets from SpaceX out to 2050: wouldn't that be ample time for cost declines to make trillions of small mirrors possible?

<comment><USER25> This might help contain temperature changes, but it won't prevent ocean acidification due to rising CO2 levels. I'm curious what percent of the damage is done by CO2 levels vs directly by temperature.

<comment><USER15> But which metric do you use for "percent of damage" ?

</comment></comment></comment><comment><USER26> ESAS/methane gun hypothesis: hold my clathrates.

If you want more, current scientific analyses and surveys of CC literature/papers/talks/conferences, I would look at Dr. Paul Beckwith's channel on YT.</comment>


The Little MongoDB Book available on GitHub - <AUTHOR> http://openmymind.net/2011/3/28/The-Little-MongoDB-Book

<comment><USER2> Great little book to get you up to speed quickly on MongoDB. No religion about it, just here it is, make your own mind up. Plus it has unicorns fighting vampires !

</comment><comment><USER3> Nice web book! I thought the coverage of map reduce was especially good. I also like the more complete book "MongoDB, The Definitve Guide" but this free book should probably serve the needs of most people.

<comment><USER4> fat fingers downvoted, luckily inspection and edit html let me correct the error. odd that I had to do so.

cleaned out the left and right braces

a href="vote?for=2378453&dir=up&by=messel&auth=MY_AUTH&whence=%69%74%65%6d%3f%69%64%3d%32%33%37%38%32%34%30" onclick="return vote(this)" id="up_2378453"

img vspace="3" hspace="2" border="0" src="<http://ycombinator.com/images/grayarrow.gif>

/a

</comment></comment><comment><USER5> Does anybody have a good way of converting this to .mobi for Kindle?

<comment><USER6> Do you mind posting the .mobi if you get it converted?

<comment><USER5> very basic generation script at <https://github.com/robduncan/the-little- mongodb-book>

</comment></comment></comment><comment><USER7> Congratulations for the book, also for using simple and effective publishing tools like markdown and Pandoc.

</comment><comment><USER8> Great stuff! Perfect timing for me too, as I had decided to spend this week brushing up on Mongo. Thanks!

</comment><comment><USER9> Perfect, just what I needed. Thanks!

</comment><comment><USER10> great book! thank you for sharing</comment>


Times Pulls Article Blaming Encryption in Paris Terror Attack - <AUTHOR> http://www.insidesources.com/new-york-times-article-blaming-encryption-paris-attacks/ <comment><USER2> "One key premise here seems to be that prior to the Snowden reporting, The Terrorists helpfully and stupidly used telephones and unencrypted emails to plot, so Western governments were able to track their plotting and disrupt at least large-scale attacks. That would come as a massive surprise to the victims of the attacks of 2002 in Bali, 2004 in Madrid, 2005 in London, 2008 in Mumbai, and April 2013 at the Boston Marathon."

[https://theintercept.com/2015/11/15/exploiting-emotions- abou...](https://theintercept.com/2015/11/15/exploiting-emotions-about-paris- to-blame-snowden-distract-from-actual-culprits-who-empowered-isis/)

<comment><USER3> It's even more absurd than that. The premise is that we can have a public worldwide debate that emphasizes how important encryption is to successfully carrying out terrorist attacks, convince the world to give up their privacy for the sake of safety, and after the majority of the protestors have been defeated by public awareness that encryption and terrorism go hand-in-hand, the terrorists will go back to using phone calls and unencrypted email.

<comment><USER4> If (almost) everyone who is not a terrorist were to give up encryption, then it would be much easier to track down/narrow down the terrorists if they keep using it, no?

<comment><USER5> I don't think so, as far as I know encrypted data doesn't really have a signature that can be easily spotted

<comment><USER6> It does, it looks random (hight entropy).

Yes, measured data and unmarked compressed data have this same property, as do actual random data. But is does not look like 9 nines of false positive rate are a concern to those people.

<comment><USER7> You can set the entropy to any amount you want. You need to consider encryption methods that put in at least a bit of effort to hide themselves. It could select random phrases and pretend to be a spambot.

<comment><USER6> Well, where's the boundary between cryptography that hides itself and stenography? Is there one?

If you include stenography, yes, it's certainly not easily recognizable. I don't think good stenography can be recognized at all, but that's not my area and I've got people contradict me at this (without further info), thus I'm not sure.

<comment><USER7> There's not much of a boundary, but that wasn't exactly the point I meant to make. You can do something like encode as ASCII 0s and 1s and have low entropy without that hiding anything.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> Increasingly we're seeing 'encryption' discussed in mainstream media as a tool that can only be used for nefarious ends. Kind of like how they talked about "cookies" in the late 90s and early 00s.

It's unfortunate that there's no way [without a lot of money] to launch a campaign that highlights the fact that most sensitive online transactions rely on encryption - banking, purchasing, etc - to protect you from The Bad People who want to steal your critical financial information.

That would also highlight that it's not just about keeping secrets from the government - in fact, that's the smallest fraction of it.

<comment><USER9> We have to get rid of this narrative of bad versus good people while we're at it. It removes every bit of nuance from the discussion.

<comment><USER10> That's the point. What the Paris attackers did makes them bad people. It's not up for debate.

<comment><USER11> Surely you can see that they think the exact opposite. In order to do something like that, they have to convince themselves that they the good guys, and it has to be done to "purify the evil in the world" or something. The truth is that people are not inherently bad. Evil feeds on itself; if you insist on calling them bad, then you are falling into the same trap that they did. Hopefully you wouldn't act in such a heinous manner, but Western societies kill thousands of innocents all the time, and it is masked with the same rationalizations; if a drone strike kills someone, we define them as a bad guy.

<comment><USER12> > _The truth is that people are not inherently bad._

I don't know about that. I'd argue that most people don't think of themselves as bad, but it doesn't make them not-bad. And humanity definitely has base instincts - if we didn't, we wouldn't need governments and police.

<comment><USER13> > _but it doesn 't make them not-bad._

I disagree. I've never found a coherent absolute ethical framework. Not even "Don't kill people."

When anyone is arguing that something is bad, that person has to appeal an authority or belief. Often the reasoning leads to utilitarianism: "If we want society to continue, we should ban murder." But that's still an if/then statement. Beneath the if/then is an appeal that society is good or desired.

> _And humanity definitely has base instincts - if we didn 't, we wouldn't > need governments and police._

This viewpoint becomes popular with Hobbes in the 1600s. I disagree with the term "base instincts," which negatively connotes those things. I'll change it to "randomness"; i.e. in a society of nondeterministic people, some will try to kill the others. Government and police try to reduce that kind of randomness. But from that same randomness we get music, science, justice. I'm speaking loosely of course.

I think of my American government as social contract, not as protective parent.

<comment><USER14> > I'll change it to "randomness"; i.e. in a society of nondeterministic > people, some will try to kill the others.

I think "randomness" is even worse, because it makes it sound like these actions could be expected from anyone at anytime. Statistically, a small portion of the population is responsible for most of the violence, through repeat offenses. Also, in many situations there are clear warning signs (e.g. mentally ill with clear homicidal ideation, member of a gang). Are you really arguing that a dice roll is a good fundamental model for human behavior?

Nondeterminism (assuming humans are truly nondeterministic) doesn't really matter here, except for the fact that we don't have a way of precisely predicting people's behavior (and may never have one). If it turns out that humans are just complicated, deterministic machines that we can not feasibly predict, the reasons for which we have developed societal structures do not disappear.

> But from that same randomness we get music, science, justice.

I think it's pretty easy to distinguish between the first two and violence. I agree that the third is a bit trickier. If what you mean is that the exercise of both these and the dispositions-formerly-known-as-base-instincts are a result of allowing for a significant measure of personal freedom, I'll agree with you. But writing human behavior off as having no more structure than a random number generator ignores a lot of predictive and explanative power that we do actually have.

<comment><USER15> > it makes it sound like these actions could be expected from anyone at > anytime

They can. A neurotypical person's brain could spontaneously generate a violent psychotic episode due to a stroke or adrenergic tumor or somesuch. Just like any area of the sky could spontaneously throw a lightning strike at you at any time. It's _low probability_ , certainly, but murder is a low-probability event to begin with.

The important point is that a model with "non-deterministic" people in it has more predictive power, _epidemiologically_ , than a model where it's impossible to become a murderer without "warning signs." It's not at all "writing human behavior off"; the fact that the model includes randomness can actually help you prevent murders more effectively, by leading you toward strategies to cope with unpredictable murders—e.g. building education toward methods of "de-escalation" for psychotic episodes, crimes of passion, etc. into your society—rather than simply trying to reinforce policing and social work.

<comment><USER14> > They can. A neurotypical person's brain could spontaneously generate a > violent psychotic episode due to a stroke or adrenergic tumor or somesuch. > Just like any area of the sky could spontaneously throw a lightning strike > at you at any time. It's low probability, certainly, but murder is a low- > probability event to begin with.

I think if we eliminated all murders except these, we would be in excellent shape. My point is these are not the ones worth focusing on, because we don't have good tools to deal with "random, history free, psychotic break."

> The important point is that a model with "non-deterministic" people in it > has more predictive power, epidemiologically, than a model where it's > impossible to become a murderer without "warning signs."

Well, if your probabilistic model has no "warning signs", then how does it provide any information at all? If you don't have a method of using information to differentiate the probabilities when given a person/group of people/location etc. then you have no predictive power at all, except for the average murder per capita.

> building education toward methods of "de-escalation" for psychotic episodes, > crimes of passion, etc.

De-escalation of psychotic episodes is an impossibly hard thing to teach without protracted work with a mental health professional. In addition, I seriously doubt that there would be any effectiveness when taught to people who have not experienced psychosis. Teaching this to everybody would be inhibited not only by cost, but by the fact that there is not likely enough people in a society that would be good enough therapists to do this on a large scale.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER16> This link implies that the NYT is backtracking about a piece apparently "blaming encryption." The NYT article referenced did not at all focus on encryption, and it didn't go so far as to definitively blame encryption.

The only paragraph discussing encryption is buried in the middle of the article. """ The attackers are believed to have communicated using encryption technology, according to European officials """

The lede did not mention encryption at all: "The attackers in Friday's terrorist assault in Paris communicated at some point beforehand with known members of the Islamic State in Syria, officials on both sides of the Atlantic say."

[https://web.archive.org/web/20151115191248/http:/www.nytimes...](https://web.archive.org/web/20151115191248/http:/www.nytimes.com/2015/11/16/world/europe/paris- attackers-communicated-with-isis-officials-say.html)

<comment><USER17> you're saying its only possible the NYT "blamed encryption" if encryption was the article's main topic or the article's lede?

<comment><USER18> No, actually they're not saying that at all. They're saying that the article only barely mentions encryption and doesn't try to push an opinion on the matter.

<comment><USER17> I'm confused. Can you say more?

<comment><USER19> What are you confused about?

</comment></comment></comment></comment></comment><comment><USER20> I blame prime numbers, even co-primes should be banned. Please contact your representative and help us free the world from evil math.

<comment><USER21> I blame it on unicode. Without unicode these terrorists would not be able to communicate with impunity.

<comment><USER22> In fairness, character encoding is clearly the work of terrorists.

<comment><USER23> If you come to America, you should have to learn the _AMERICAN_ Standard Code for Information Interchange. Speak the language. None of this Unicode i18n nonsense!

<comment><USER22> "Press  for billing, or please hold for an attendant."

</comment></comment></comment></comment></comment><comment><USER24> Even without having encryption in main stream applications, like WhatsApp, it's child's play to communicate secretly. There are so many open source applications that allow you to do this, it wouldn't even be a speed bump for criminals.

Targeting main stream applications only hurts main stream users. NYT should write an article about that.

<comment><USER25> You seem to be overlooking the obvious argument that if a small group of people use encryption, then you greatly reduce the number of messages that you need to flag. Furthermore, if someone on a Watch List starts using encryption then perhaps you have an imminent problem.

I'm not on the side of reading messages but you missed the real argument being made.

Typical HN. How about someone answering the argument that will really be made by governments instead of the pointless one the NSA, etc have already answered. You can downvote me but you don't get a downvote in governments around the world when they outlaw real encryption.

<comment><USER26> > You can downvote me but you don't get a downvote in governments around the > world when they outlaw real encryption.

I'm pretty sure you can downvote them by voting for representatives that do not support such laws. At least if you're in some kind of democracy.

<comment><USER27> There should be a name for this fallacy. This is easy to say for any issue in isolation, but what is a voter to do when they only have two or three choices? When there are dozens of issues at any given time the chances of having a candidate that aligns with you on all of them is nil.

</comment></comment></comment></comment><comment><USER28> You could plan terrorist activity using postcards. Should we ban the Post Office? You could use phone calls from pay phones (where they still exist) and people used to do that all the time to plan crimes. You could go on vacation together and plan something. You could use voice or text chat in an online game to plan stuff. The people in charge of these organizations are not the suicide bombers; the head folks know full well what to use to avoid getting caught.

<comment><USER29> I have a friend who used to get erotic messages from her mathematician boyfriend steganographically encrypted on postcards. I had to show her how to decrypt them. It was pretty standard cryptanalysis once I guessed that the messages existed. Some of them were filthy.

<comment><USER30> Stenographically encrypted in the printed image? How would that work?

<comment><USER31> Probably on the side you write on.

<comment><USER30> Yep, that would make far more sense... For some reason I assumed it was an image encoded on the postcard. Rereading the original post, that's not what it says, is it...

</comment></comment></comment></comment></comment><comment><USER32> It's true that the NYT pulled (or dramatically rewrote, close enough) an article claiming: "The attackers are believed to have communicated using encryption technology..." You can see the original version here thanks to Archive.org: [https://web.archive.org/web/20151115191248/http:/www.nytimes...](https://web.archive.org/web/20151115191248/http:/www.nytimes.com/2015/11/16/world/europe/paris- attackers-communicated-with-isis-officials-say.html)

But then the NYT doubled down on the terrorists-and-encryption angle in a separate story here: [http://www.nytimes.com/2015/11/17/world/europe/encrypted- mes...](http://www.nytimes.com/2015/11/17/world/europe/encrypted-messaging- apps-face-new-scrutiny-over-possible-role-in-paris-attacks.html)

Which says in the second sentence: "Obama administration officials say the Islamic State has used a range of encryption technologies over the past year and a half, many of which defy cracking by the National Security Agency..."

HN discussion of that second story is here: [https://news.ycombinator.com/item?id=10579201](https://news.ycombinator.com/item?id=10579201)

</comment><comment><USER33> _A link to the NYT article now redirects readers to a separate, general article on the attacks, which does not contain the word "encrypt." The original piece can be found on the Internet Archive._

[https://web.archive.org/web/20151115191248/http://www.nytime...](https://web.archive.org/web/20151115191248/http://www.nytimes.com/2015/11/16/world/europe/paris- attackers-communicated-with-isis-officials-say.html)

<comment><USER34> The language seems pretty reasonable...

"The attackers are believed to have communicated using encryption technology, according to European officials who had been briefed on the investigation but were not authorized to speak publicly. It was not clear whether the encryption was part of widely used communications tools, like WhatsApp, which the authorities have a hard time monitoring, or something more elaborate. Intelligence officials have been pressing for more leeway to counter the growing use of encryption."

<comment><USER35> Sure, if you think unproven hearsay in NYT that we all ought to take for granted is "reasonable". If the authorities believe it was the encryption to blame, then they should come out and say it outright - with details, of course. They shouldn't hide under the "anonymous officials" tag which "believe" it was encryption the culprit that stopped them from learning about the attacks, and nothing else. It could've been just some random cop's opinion, who heard it from someone else, who was also wrong about it.

<comment><USER36> What's wrong with newspaper reporting hearsay, while describing it as hearsay?

NYT didn't seem to pass it as anything else than it is. What's your logic here? Should all anonymous sources be banned? Or only the sources with opinions that you don't like?

<comment><USER37> Actually, most reputable newspapers have very strict rules about anonymous sources, including some who ban them altogether.

Here is the NY Times Standard and Ethics Statement, which describes the newspaper's "distaste" for anonymous sources:

[http://www.nytco.com/who-we-are/culture/standards-and- ethics...](http://www.nytco.com/who-we-are/culture/standards-and-ethics/)

In fact, that's probably the reason it was pulled: a complaint was made to the NYT Public Editor about the piece's sources.

<comment><USER38> If so, they should say so. Silently pulling the article and redirecting the URL is a little misleading.

</comment></comment></comment></comment></comment></comment><comment><USER39> What is the deal with insidesources.com? Something feels a "little off" after browsing the rest of the site. It feels like a undercover PR machine for someone or some group. It also seems like they hate copy editors.

<comment><USER40> editor of insidesources' twitter handle is @warroomalerts

</comment></comment><comment><USER41> > writing messages via in-game functions, like spelling words with dropped > items or shooting walls

I'm kinda ashamed I haven't thought of that.

<comment><USER42> I think this may be the most important thing in the article. If the conspirators really were communicating via in-game ephemera, the communication probably wasn't encrypted at all. If so, the hurdle that the security forces faced is not that they couldn't access the communication, but that they couldn't interpret it.

And that just shows how little a mandatory backdoor policy would help: being able to read something doesn't mean you can understand it. It's always possible for someone to come up with a communication scheme that you didn't anticipate or can't interpret, and you can't legislate away that capability.

So a backdoor policy doesn't trade privacy for safety -- instead it trades privacy for a chance at safety, and pretty much just safety from careless, unprepared attackers.

<comment><USER43> It would not be the first time that obscurity is confused with security.

</comment></comment></comment><comment><USER44> Shame..that they wrote it in the first place.

<comment><USER45> Care to elaborate?

<comment><USER44> Encryption is a basic human right. The freedom to speak without being judged.

<comment><USER46> So is life, what do you do when 2 basic rights clash? It's definitely a tricky question, and I certainly don't have the answer.

<comment><USER47> Encryption or the use thereof doesn't threaten your life. Screwdrivers or the use thereof don't either, however if someone misuses the screwdriver and stabs you in the temple you will die.

Does this mean we should outlaw the use of screwdrivers ?

</comment></comment></comment></comment></comment><comment><USER48> The best encryption is simply to communicate the old fashioned way. Just because they had some playstations and access to "whatsapp" does not mean the terrorist trusted electronic devices.

<comment><USER49> If your favorite tool is the pervasive surveillance hammer, you try to make everything look like a nail.

</comment></comment><comment><USER50> I believe the conclusion of the article is the most important takeaway: (so, points for good writing, I guess :) )

> Former CIA Deputy Director Michael Morell said he suspects the Paris attacks > will weigh heavily on the encryption fight ongoing.

> "I think what we're going to learn is that these guys are communicating via > these encrypted apps, the commercial encryption, which is very difficult, if > not impossible, for governments to break, and the producers of which don't > produce the keys necessary for law enforcement to read the encrypted > messages," Morell said on CBS' "Face the Nation" Sunday.

> "We need to have a public debate about this," he continued. "We have in a > sense had a public debate — that debate was defined by Edward Snowden, and > the concern about privacy. I think we're now going to have another debate > about that — it's going to be defined by what happened in Paris."

So, "They" want to misuse the attacks on Paris as an excuse to attract mind and screen-time for an alternative counter-debate about privacy and encryption. But Snowden got to "define the debate" because of the new information (as well as evidence for old information) he brought to the public, it's an entirely different thing if you're a politician grabbing screen-time and "define the debate" to a shocked audience over a terror-attack --which is exactly what the terrorists' want. Any freedoms or security taken away from us as a response to a terror-attack, is a win for the terrorists.

</comment><comment><USER51> > "'The most difficult communication between these terrorists is via > PlayStation 4,' the minister said, three days before the terrorist attacks > in Paris. 'It's very, very difficult for our services — not only Belgian > services but international services — to decrypt the communication that is > done via PlayStation 4.'"

The Onion is going to have a really hard time topping this.

</comment><comment><USER18> Honestly, I don't doubt at all that intelligence officials would love to exploit FUD about this situation to further their agendas, but this article's claim that the NYT article was "blaming encryption" is kind of a stretch. Read the article linked on archive.org and decide for yourself:

[https://web.archive.org/web/20151115191248/http:/www.nytimes...](https://web.archive.org/web/20151115191248/http:/www.nytimes.com/2015/11/16/world/europe/paris- attackers-communicated-with-isis-officials-say.html)

</comment><comment><USER52> The government and business have built a state of surveillance far beyond anything dreamt of a couple of decades ago, with massive continuing investment.

Yet the tool of surveillance was unable to stop this attack. Now they want more surveillance, but maybe it's the wrong tool for this job.

Surveillance seems to have reached that place on the technology adoption curve where people get a little over-enthusiastic and blindly think it solves every problem, and they haven't yet realized it's a tool which, like all tools, is good for for things but not for others.

(That ignores the massive cost to individuals and to our societies of surveillance.)

</comment><comment><USER53> In my mind, it comes down to what a society's priorities are.

If the priorities are Freedom > all, then encryption is a necessary tool.

If the priorities are Prosperity > Safety > Freedom > all, then society will run roughshod over freedom.

<comment><USER54> >A politic minister will study to lull us into security, by granting us the full extent of our petitions. The warm sunshine of influence would melt down the virtue, which the violence of the storm rendered more firm and unyielding. In a state of tranquillity, wealth and luxury, our descendants would forget the arts of war, and the noble activity and zeal which made their ancestors invincible. Every art of corruption would be employed to loosen the bond of union which renders our assistance formidable. When the spirit of liberty which now animates our hearts and gives success to our arms is extinct, our numbers will accelerate our ruin, and render us easier victims to tyranny. Ye abandoned minions of an infatuated ministry, if peradventure any should yet remain among us!—remember that a Warren and Montgomery are numbered among the dead. Contemplate the mangled bodies of our countrymen, and then say, What should be the reward of such sacrifices ? Bid us and our posterity bow the knee, supplicate the friendship, and plough, and sow, and reap, to glut the avarice of the men who have let loose on us the dogs of war to riot in our blood, and hunt us from the face of the earth? _If ye love wealth better than liberty, the tranquillity of servitude, than the animating contest of freedom—go from us in peace. We ask not your counsels or arms. Crouch down and lick the hands which feed you. May your chains set lightly upon you, and may posterity forget that ye were our countrymen._

-Sam Adams, American Independence Speech, August 1, 1776.

[https://en.wikisource.org/wiki/American_Independence](https://en.wikisource.org/wiki/American_Independence)

</comment></comment><comment><USER55> This same narrative has been massaged into hundreds of stories on the attacks. These news outlets are doing surveillance-crazed politicians' bidding for them by creating the public perception that these people would still be alive were it not for encryption. Perhaps the Times realized how dangerously stupid this narrative is and is working to do something about it.

We are told that if we stop going to the mall or sporting events out of fear, then the terrorists win. They also win if we lose our privacy rights in response to this or any other attack.

</comment><comment><USER56> This issue brings up a larger point that many in the tech world have ignored for awhile. Now that end to end encryption is easily accessible by everyone, how do the authorities figure out beforehand when a terrorist attack will take place? The answer is not banning encryption technologies or giving authorities a 'key' as the FBI director has proposed. But now authorities have a very difficult problem on their hands. Encryption can't be contained, and the authorities need to figure out how to still catch the bad guys.

<comment><USER57> It takes what it has always taken: good police work.

Encryption is like envelopes for the mail. They can be broken into with enough force, but their purpose it to shield the contents of the message from those handling it in transit. A post card offers no such protection.

So suppose the banned encryption (envelopes). Im sure the government would still reauire envelopes. As would anybody dealing with health records. Or banking, financial stuff has a legitimate need for secrecy. So they ned envelopes.

Pretty soon what you have is a class of people allowed to USE envelopes and a class of people not allowed it use them.

Like envelopes, encryption is an idea, not an inplementation. They may block you from using this tool or that tool, just like a government could make selling pre-made envelopes illegal - but the IDEA of encryption is simple just like the idea of an envelope. Anybody with paper, scissors, and glue can fashion their own envelope. Encyption is the same, one person can dream up an build their own encoder and you can never ban or prevent that idea from being used.

<comment><USER58> [https://www.philzimmermann.com/EN/essays/WhyIWrotePGP.html](https://www.philzimmermann.com/EN/essays/WhyIWrotePGP.html)

"Why I Wrote PGP", by Philip Zimmermann

We really need to emphasize the envelope metaphor a lot more. It's much easier for non-technical propel to understand than any discussion of key/signing/etc.

</comment></comment></comment><comment><USER59> I've started seeing this like locking down all coffee houses just because it is a convenient place to meet and discuss things and some people happen to plan terrible acts in these places.

</comment><comment><USER60> As someone said[1] the damage is already done.

Now everyone thinks that the terror attacks were perpetrated by Syrian refugees that were communicating through the PS4 network using encryption, and that nothing of this would have happened if videogames didn't exist, encryption was forbidden, and they hadn't sheltered refugees.

[1] [https://openparachute.files.wordpress.com/2015/01/amount.jpg](https://openparachute.files.wordpress.com/2015/01/amount.jpg)

</comment><comment><USER61> If the secret service is relying on snooped messages as the main affective method to catch criminals then it's doing it wrong. They're stuck in the WW2 mentality where they could listen on radio and decrypt codes. Back then it was very effective but that was 60 years ago.

Secret services should consider decrypted communication as a nice to have, but they should be completely capable of being effective without decrypting communication channels and listening on private citizens.

</comment><comment><USER62> > Dang it, guys. Encryption is now illegal for us to use, everyone uninstall > PGP.

Said no terrorist or criminal. Once the cat has been let out of the bag there is no getting it in again.

<comment><USER63> or ask Google, Apple, and Microsoft to automatically un-install the nefarious applications from all computers and phones... not sure what they would do about Linux ...

</comment></comment><comment><USER64> Typical New Pravda SOP. Drop an unsubstantiated rumor blaming something the government doesn't like then retract it once e damage has been done.

</comment><comment><USER65> Waiting for the day when it is insisted that only criminals/terrorists use end-to-end encryption.

I'm not surprised by liberals who want this and think government is mother/father (to be clear I am progressive) but I am REALLY surprised by conservatives that now want government monitoring every part of their citizen's lives.

</comment><comment><USER66> One of the main problems with the NSA programs is that it targeted _everyone_. That just makes no sense. If you have a lead or probable cause on someone, great, track them and hack their messages. Collecting everyone's information to catch a terrorist only dilutes your search.

</comment><comment><USER67> Kind of like how they used guns!

</comment><comment><USER68> I read that unsubstantiated assertion in the print edition (I know, dinosaur. I like a morning paper) and called it out to my family. I'm saddened and amused at how the press is led around by the nose.

</comment><comment><USER69> I don't know why people are considering these terrorists to be so smart. They arrived at the Stade de France and were naively thinking they'd be able to come inside with explosives tied to them. In January, they didn't even know where the Charlie Hebdo meeting room was, they had to ask for directions and were even sent the wrong way (at first). Really, I'm far from worrying about their communication's encryption as much as I worry about the lax government.

<comment><USER70> > as much as I worry about the lax government

You mean the same governments that are already collecting everyone's private communications and swabbing babies for bomb residue at airports under the guise of stopping these types of things? It's not a lack of effort or resources that should be concerning, it's the sheer incompetence.

<comment><USER69> No, the point I was trying to make has nothing to do with communication. Molenbeek has been known to host radicalized Islamists for twenty years, I've heard on French TV this weekend. Encrypted communication or not, as long as governments sleep on such facts, we will see further attacks.

</comment></comment></comment><comment><USER71> Wow, I read this article on Sunday, and had I not seen this post I never would have realized the Times no longer stood by what they said.

</comment><comment><USER29> Countries fail to police immigration, promising us that it will go well and we'll all benefit. Then they demand our civil liberties so they can police the immigrants they invited. And our taxes have to rise to pay welfare to them. But don't worry; a few million more and we'll all be better off.

<comment><USER54> Howgh, my fellow native american or indigenous friend. I too grow weary of these pale face immigrants taking our land, genociding us, and telling us we'll all be better off.

</comment></comment><comment><USER72> I encourage open thought rather than mob attacks on unpopular opinions. Anyone have a link to the original article?

<comment><AUTHOR> A quick search of this thread reveals:

[https://news.ycombinator.com/item?id=10580586](https://news.ycombinator.com/item?id=10580586)

[https://web.archive.org/web/20151115191248/http://www.nytime...](https://web.archive.org/web/20151115191248/http://www.nytimes.com/2015/11/16/world/europe/paris- attackers-communicated-with-isis-officials-say.html)

<comment><USER72> I don't really see anything too terrible about the article.

</comment></comment></comment><comment><USER73> [http://www.newsdiffs.org/article- history/www.nytimes.com/201...](http://www.newsdiffs.org/article- history/www.nytimes.com/2015/11/16/world/europe/paris-attackers-communicated- with-isis-officials-say.html)

</comment><comment><USER74> The biggest problem with the "encryption is too good" crowd is that they simply refuse to recognize the simple fact that if there is a way in for them, there is a way in for others. If there is a back door, it's simply no longer effective encryption.

</comment><comment><USER50> > The Times later posted a second article citing an anonymous "European > counterterrorism official" who was quoted saying authorities' "working > assumption is that these guys were very security aware,"

We should make laws against security awareness if it enables terrorists!

</comment><comment><USER75> What I found interesting is the method of communication was outlined in "Homeland" by Cory Doctorow (Or perhaps it was in "Little Brother"). I suppose we need to outlaw books now also.

</comment><comment><USER76> Could it not be that what the article said was wrong, but that someone twisted their arm for correctly exposing how they were communicating?

</comment><comment><USER77> Many are proposing a false dichotomy. Fact is, we can have privacy AND mitigate terrorist activity.

</comment><comment><USER78> Jim Bamford of Foreign Policy said encryption was part of the way this was done.

On the Diane Rehm Show, when asked what we know, he said "not much, but that encryption or anonymizers, like Tor, might have been used to hide their communications".

So they're already working to plant that narrative in listeners' heads.

<comment><USER79> "was done" vs "might have been" \- you're making a big jump.

If you weren't aware, Bamford is actually one of the good guys in this debate. Don't make him say things he didn't say.

</comment></comment><comment><USER80> This is all covered pretty well in the latest Bond movie.

</comment><comment><USER81> > citing unidentified "European officials"

That was the Belgium minister.

> It was not clear whether the encryption was part of widely used > communications tools, like WhatsApp, which the authorities have a hard time > monitoring, or something more elaborate.

"PlayStation 4 is even more difficult to keep track of than WhatsApp," Jambon [Belgium minister] said at a debate in Brussels. "The most difficult communication between these terrorists is via PlayStation 4," he said. "It's very, very difficult for our services — not only Belgian services but international services — to decrypt the communication that is done via PlayStation 4." [http://www.telegraph.co.uk/technology/video- games/playstatio...](http://www.telegraph.co.uk/technology/video- games/playstation/11997952/paris-attacks-playstation-4.html)

So: It is less about encryption (PGP, Tor) than it is about companies running their own communication networks with encryption, and the intelligence agencies have increasingly more trouble tracking extremists using games or phone apps to plan their attacks.

Is that Snowden's fault? No. Did Snowden's leaks contribute to companies making their networks harder to tap? Definitely.

Another tidbit that is coming out is that nearly all terrorists were already on the radar of the intelligence services and had documents tracking their radicalization. [http://www.theguardian.com/world/2015/nov/16/french-and- belg...](http://www.theguardian.com/world/2015/nov/16/french-and-belgian- intelligence-knew-paris-attackers-had-jihadi-backgrounds)

Apparently there was a failure to share and to make actionable sense of this information.

Is that Snowden's fault? No. Did Snowden's broad indiscriminate leaks cause less willingness to share information between intelligence agencies? Definitely.

This is, in my view, a side-effect of these revelations, but a real effect too. I don't want to weigh up these two on a scale, as that will be difficult and everyone will have different priorities anyway.

BTW: This gaffe/Chinese whispers started with a Forbes article and New York Times fell for it: [http://www.forbes.com/sites/insertcoin/2015/11/14/why-the- pa...](http://www.forbes.com/sites/insertcoin/2015/11/14/why-the-paris-isis- terrorists-used-ps4-to-plan-attacks/)

"Correction: It has not been confirmed, as originally written, that a console was found as a result of specific Belgian terror raids. Minister Jambon was speaking about tactics he knows ISIS to be using generally. "</comment>


Build a Static Website Using React JS: Part #1 Project Setup and Website UI - <AUTHOR> https://www.dhruvkumarjha.com/articles/building-a-static-website-using-react-js-part-1-project-setup-and-website-ui <comment><USER2> I love WordPress for its cool feaures and ease of use. I've just finished creating my news portal website using a premium news wordpress theme ( [https://www.templatemonster.com/category/news-portal- wordpre...](https://www.templatemonster.com/category/news-portal-wordpress- themes/) ). It looks quite pretty.</comment>


Singularity and problem with humanity - <AUTHOR> http://alexsblog.org/2014/04/25/singularity-and-problem-with-humanity/

<comment><USER2> Please make one or more editing passes on your blog entries before posting them. Your writing is filled with grammatical errors and missing words.

<comment><AUTHOR> Thank you, I will take a look at it. Also, this will not happen again.</comment></comment>


Ask HN: Is there a url for newest asks? - <AUTHOR> I know there is news.ycombinator.com/newest and a newest for 'show': https://news.ycombinator.com/shownew but is there a newest for ask? <comment><USER2> I don't think so.

There should be.

Right now I am looking at 100% of the Asks that go by on HN with a script that scrapes the web site, classifies posts, and shows me ones I want to read. It is amazing how many questions go by and never get an answer. Maybe that's just because I want to be a fermion in a world of bosons.

<comment><AUTHOR> I hear you. That's exactly how I feel about it.

When I am in an answering mood I'd like to be able to see the newest asks without the distraction of the other posts.

</comment></comment><comment><USER3> [https://hn.algolia.com/?query=ask%20hn&sort=byDate&prefix&pa...](https://hn.algolia.com/?query=ask%20hn&sort=byDate&prefix&page=0&dateRange=last24h&type=story)</comment>


4chan's hackers come to the defense of WikiLeaks by attacking its enemies - <AUTHOR> http://venturebeat.com/2010/12/07/4chans-hackers-come-to-the-defense-of-wikileaks-by-attacking-its-enemies/

<comment><USER2> If anything, 4chan is giving defenders of wikileaks a bad name. It's like when peaceful protests are invaded by hooligans intent only on starting fights and setting fire to cars.

<comment><USER3> The problem with any debate is that idiots exist, and they also choose sides.

</comment></comment><comment><USER3> First sentence begins "4chan, a hacking web site..."

I stopped reading.</comment>


Andrew Yang Probably Isn't Who You Think He Is - <AUTHOR> https://www.lamag.com/citythinkblog/andrew-yang-los-angeles/ <comment><USER2> I like Andrew Yang, but is UBI a proven model yet?

Progressive Finland ran an experiment.

YC itself ran(is still running?) a UBI experiment.

Is UBI a proven model or does it yet provide compelling evidence to further experiment or roll out?

Seems very vapourware-ish or too soon to tell.</comment>


Ask HN: What is the best DB for versioned time series? - <AUTHOR> Dear all,
time series DBs are a hot topic these days but given the focus on IoT -- or more generally measurement data -- the underlying data model typically assumes that for one time series there is only one data point per period of time, so it's really just one dimensional.
However, if you work for example with forecast data (say for a stock price) you might wanna store every version of a forecast and not overwrite the previous forecast.
What are in your experience the best time series databases that (natively) support two or more dimensions and also allowing queries on these other dimensions like "get forecast for delivery-time from x to y where forecast_time = z"?
Thanks!<AUTHOR> <comment><USER2> [https://spotify.github.io/heroic/](https://spotify.github.io/heroic/) is a multi dimensional timeseries database with a rich query language.

</comment><comment><USER3> I have been using the ELK stack as a time series database for a long time, I know it isn't built for it but there's so much documentation and resources online that makes it a very "easy" choice, it's very easy to scale any part of it, use the elastic as a nosql database, have kibana to visualize and easily query. Curator to move old data to cold storage / shrink. It's been a no brainer.

</comment><comment><USER4> Have a look a TimescaleDB.

<comment><USER5> I've used this with decent success. Their support slack channel is great as well, and there is a lot of active development on the project.

Postgres's built in partitioning is also being actively developed. If it gets good enough, I wonder what will happen to timescaleDB as a company...

</comment></comment><comment><USER6> Wouldn't that be solved with tags? Most time series databases support tagging values.

<comment><USER7> Yeah. You can treat the version as just another attribute to identify the series.

</comment></comment><comment><USER8> Amazon just launched their Time Series DB. It is probably worth a look.

</comment><comment><USER9> Clickhouse - [https://clickhouse.yandex](https://clickhouse.yandex)

</comment><comment><AUTHOR> Great, thanks all!</comment>


Hands on with Apple's new OS X: Mountain Lion - <AUTHOR> http://www.macworld.com/article/165407/2012/02/hands_on_with_apples_new_os_x_mountain_lion.html <comment><USER2> I had a couple comments on Gruber's posting, but one of the things that really jumped out at me about this hands-on is the Notification Center trigger taking Spotlight's coveted top-right Fitt's Law trigger. Also, while I say "ew" to more skeuomorphism, I could see how things like are enjoyed by some users; not that it makes this type of UI "correct", but, whatever.</comment>


The Chicken Is Local, but Was It Happy? GPS Tells the Life Story of Your Poultry - <AUTHOR> https://www.npr.org/2019/02/24/697185878/the-chicken-is-local-but-was-it-happy-gps-now-tells-the-life-story-of-your-poult <comment><USER2> that still technically doesnt tell me if it was "happy" since we have no actual way of measuring a chickens level of happiness, this disappoints me greatly as im sure if i ever do eat chicken ill want to eat one that was killed in the middle of enjoying its life than one that was ready to die</comment>


Nokia mocks iPhone 4 - <AUTHOR> http://www.iclarified.com/entry/index.php?enid=10329

<comment><USER2> Glass houses, stones, etc.

Heres someone demonstrating the same problem on a Nokia E71 phone.

[http://www.youtube.com/watch?v=amPG52DVQuk&feature=playe...](http://www.youtube.com/watch?v=amPG52DVQuk&feature=player_embedded)

</comment><comment><USER3> Poor Nokia took the bait. Now every story covering this will almost inevitably include links to Nokia's own phones exhibiting this issue.

</comment><comment><USER4> Usually when companies mock their competitors it shows how desperate they really are. But then again, Apple also does this (They did mock Windows in one of their ads).

<comment><USER3> In more than one of them, that's for dang sure.

</comment></comment><comment><USER5> The actual blog post: [http://conversations.nokia.com/2010/06/28/how-do-you- hold-yo...](http://conversations.nokia.com/2010/06/28/how-do-you-hold-your- nokia)

</comment><comment><USER6> [http://techcrunch.com/2010/06/28/apple-weve-sold- over-1-7-mi...](http://techcrunch.com/2010/06/28/apple-weve-sold- over-1-7-million-iphone-4-devices-in-3-days/)

"most successful launch in Apple's history" ~ Steve Jobs

When was the last time anyone stood in line for 3 days for a Nokia product launch?

<comment><USER7> The iPhone 4 launch was popular. I agree.

But how is this related to reception problems and Nokia's tongue-in-cheek post?

<comment><USER8> Because given that context, it's hard to resist seeing this sort of thing as sour grapes.

I'm not claiming it is, but that impression is hard to avoid even if they are very gentle about poking fun indirectly as the case is here.</comment></comment></comment>


Ask HN: Best way to make a website go viral - <AUTHOR>

Hey HN -- the title says it all. What is the best way to market and make your website go viral? <comment><USER2> To make it go viral: Make it interesting, original (if possible), authentic (people can smell "fake") and the rest will follow on their own.

The truth is that you have very little control over what truly goes viral and the last thing you need is for your site to go viral for all the wrong reasons. There is such a thing as bad publicity.

And just remember that Marketing != Going viral, which in some cases may be anti-marketing.

<comment><AUTHOR> Thanks! I'm trying to get my website ([http://mappedinsiliconvalley.com/](http://mappedinsiliconvalley.com/)) out there into the Silicon Valley community. What do you think?</comment></comment>


Please Buy Less - <AUTHOR> http://pleasebuyless.com/ <comment><USER3> Wow, getting some negative responses here. Personally, I'm rapt to see a message like this (especially on front page of HN)

</comment><comment><USER4> I like the message, but it seems to complain more than it actually provides solutions.

The term 'retail therapy' is a very real thing.

I think a lot of our consumerism is connected to our disconnection and lifestyle, and this is related to how we set up our cities and communities. From population density, to walk-ability of cities, etc.

When I lived in the third world, in an area with pretty high population density, mixed use zoning, and high walk-ability I would consume less 'stuff' and more experiences. For instance, instead of driving to TJ max to buy some new clothes as I would here, I might take a walk and buy some pastry... which I'd be hungry for from walking. It was from a small producer, who used less materials for packaging than a large one would. I'd also get to know people who I'd regularly see, and that also reduced consumption of good I think as I would stop and connect with other people. We are pleasure seekers, and if consumerism is the easiest way to satiate that desire, it will be done in a consumerist way.

I'm not saying our consumerism is ONLY about lifestyle, but I think going into what causes consumption is more effective at reducing it than simply saying 'hey you, don't do that'. I think it's why in some parts of Europe people seem to buy less junk and more quality, artisanal made items.

</comment><comment><USER5> FYI: [https://www.history.com/news/whats-the-real-history-of- black...](https://www.history.com/news/whats-the-real-history-of-black-friday)

</comment><comment><USER6> I would, I use it all the time, but it's open source[1].

:-)

[1] [http://www.greenwoodsoftware.com/less/](http://www.greenwoodsoftware.com/less/)

</comment><comment><USER7> We must enable and provide technology to countries that are on there way to western worlds living standard to get there with low co2 emission. Because they want it with or without co2 emission.

</comment><comment><USER8> Political bullshit that has no place on Hacker News.

</comment><comment><AUTHOR> I was not sure about posting this. The world is weird in that you are a single person in a sea of 7+ billion people, it is very hard to get your message across and even harder to make a change that will actually make a dent. Hope you think twice before buying and save some money too - have a nice extended weekend :)

<comment><USER9> I'm glad you did. It's an important message and Americans -- at least the ones on the internet -- seem now to be more receptive to critiques of capitalism than they have been at any time in the last 50 years.

</comment></comment><comment><USER10> Demanding politely to a command I'm going to obviously say no to is rude and obnoxious.

<comment><USER11> That's possibly the least "rude and obnoxious" message about rampant consumerism I've ever come across. Your response speaks volumes about yourself as opposed to the message on the site.

<comment><USER10> After deciding to read the message at all after being put upon by an author who doesn't know me, I found the message to be a bit of a cliche nowadays. Who's his audience? Hungry consumers aren't busy reading and following commandments handed down by domain names.

</comment></comment></comment><comment><USER12> Please do not travel, either. Togetherness and family are idols at least as wasteful and environmentally damaging as consumerism.

<comment><USER13> Certainly electric night at light are a waste, right?

Running lightbulbs in an average home for a year uses more resources than travel.

Lighting is 17% of global carbon emissions, I think [0]. And air travel is 2.5% or so.

Many people don't even have the option not to travel, but everyone can choose not to use electric lights.

[0] [https://www.ledonecorp.com/using-led-lighting-to-reduce- your...](https://www.ledonecorp.com/using-led-lighting-to-reduce-your-carbon- footprint/)

[1] [https://www.npr.org/2019/11/27/778692814/choosing-not-to- fly...](https://www.npr.org/2019/11/27/778692814/choosing-not-to-fly-home-for- the-holidays-for-the-climates-sake)

<comment><USER14> I simply do not believe LED One's claim lighting is 17%, apparently based on an all tungsten world, nor that electricity is 50% of emissions.

Electricity comes out at 50% only when the category is "Electricity and heating" [1], usually it comes in at 20-30% depending on country. A lobby group promoting adoption of LED lighting in 2006 while all domestic use was still tungsten claims 6% is lighting[2]. Post LED adoption, perhaps 0.5-1%. Does anyone still use tungsten aside from some specialised use given LED bulbs are down to tungsten prices?

It appears to be a small enough percentage I can't find anywhere picking out lighting individually.

[1] [https://ourworldindata.org/grapher/carbon-dioxide- co2-emissi...](https://ourworldindata.org/grapher/carbon-dioxide- co2-emissions-by-sector-or-source)

[2] [https://www.theclimategroup.org/project/led- scale](https://www.theclimategroup.org/project/led-scale)

</comment></comment></comment><comment><USER15> "have one fewer child" is the most efficient way to decrease your carbon footprint (see [https://www.sciencemag.org/news/2017/07/best-way-reduce- your...](https://www.sciencemag.org/news/2017/07/best-way-reduce-your-carbon- footprint-one-government-isn-t-telling-you-about) )

Easy, right? However, I am not sure it is correct. For instance, if you produce less than 2 kids (e.g., 0) AND it was your decision, would not it mean that all the carbon footprint went to adapt/improve your genetic material (including your grand-grand-parents) is a waste in some sense? You are the last one of your branch. Wasn't you fooled to believe you should have no kids? Makes a lot of sense for other branches to fool you and gain a greater ratio of the entire genetic material (this is basically the definition of fitness function in evolutionary biology - to have more offspring / a greater ratio of genetic material). Evolutionary processes at play.

<comment><USER16> 1) This is a sunk cost fallacy. Think of all the fossil fuel infrastructure, wouldn't it be a waste to stop using it? 2) Your goals and goals of your genes are not the same. You should care about as much about your genes as your genes care for you (not at all)

<comment><USER15> To truly answer such questions you should have a (set of) metrics to measure what is better and run parallel universes for a significant amount of time to check which version is better according to your metrics.

>> Think of all the fossil fuel infrastructure, wouldn't it be a waste to stop using it?

Most likely, it would be a waste to stop using it right now.

>> Your goals and goals of your genes are not the same. You should care about as much about your genes as your genes care for you (not at all)

Living organisms are genetically wired to care about their genes, e.g., to express them (e.g., to write posts on HN and publish scientific papers) and spread them (to have kids).

<comment><USER16> Wired, but not directly. We don't have an explicit urge to have as many kids as possible, we just like having sex. It kinda leads to the same outcome. Or it did, before we invented contraceptives.

Writing posts on HN is not expressing your genes, it is an entirely accidental effect of intelligence, which is another mechanism we have solely to better propagate our genes, but it kinda runaway.

<comment><USER15> >> Writing posts on HN is not expressing your genes, it is an entirely accidental effect of intelligence

Your opinion is a function of your genes (sex, race, physical conditions, etc). Your HN post does not have to be imprinted in your genes but our genes affect our opinion whether we want it or not. It is rarely about which sorting procedure you will peak but when discussing things relevant to genes (again, racial and sexual stuff) they will inject their biases.

</comment></comment></comment></comment></comment><comment><USER17> We've been advocating "Reduce, Reuse, Recycle" for over 40 years, and yet emissions and consumption have gone up every year. You will never be able to convince enough people to voluntarily reduce their lifestyle. To cut waste & emissions we need legislation, not shaming of consumers.

<comment><USER18> Yes, proper incentives need to be in place. Taxes & regulation (and in some cases simply staffing up to enforce existing laws) are what are needed.

That said, I have personally stopped profligate purchases on Amazon/Express - promised the wife we'd clean out the garage before buying new non-essentials and have reduced online spending to 1/5th of last year (YMMV our garage was chock full of stuff).

<comment><USER19> Most of the times when people realize what you realized and go for a change of habits - less buying and doing the same with less/minimal lifestyle - <USER2> first step they take is to throw stuff away. I've done it myself. Now I really stress the reuse and the repurpose before throwing anything away. Otherwise sell. At the end of the day is one crap less that ends up in some landfill.

</comment></comment></comment><comment><USER20> The easiest way to get people to buy less is to raise prices.

If we had a carbon tax that correctly priced the environmental impact of goods, it would decrease consumption. Without having to shame people into removing themselves from the economy.

<comment><USER21> Yes. I've become convinced that Pigovian taxes [0] (connected to a basic dividend) are the answer to climate change, and to ecological externalities in general. (In addition to greasing the wheels of political viability, a dividend ensures that paying the true cost of carbon is not a de-facto regressive tax, as that cost hits the working class the hardest.)

[0] [https://en.wikipedia.org/wiki/Pigovian_tax](https://en.wikipedia.org/wiki/Pigovian_tax)

<comment><USER20> I agree that we will definitely want to do it in a way that minimizes how regressive the outcome is.

</comment></comment></comment><comment><USER22> I think this is some serious Marxist agitprop. It is absolutely appalling that people think we _ought to buy less_ as if that's what the problem is. The problem is not unethical consumption! The difficulty is not in alienating labor from capital! These posadist fools think we will be saved by aliens.

Look fellas, buy your girl that Gucci bag if she wants it. The gal may as well looks fashionable while we descent into collapse.

<comment><USER23> > your girl

This is so wrong on so many levels. I pity your female work colleagues. Unless you were talking about a dad purchasing some luxury stuff for his daughter, but I guess you didn't have that in mind.

<comment><USER24> Would you please not take HN threads further into ideological flamewar? It leads to nasty and stupid discussions that we neither need nor want.

[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

<comment><USER23> That's nothing ideological about telling someone that their way of seeing other human beings as inferior is wrong, as talking about a different person as belonging to someone else (the "your" part) means seeing the other person as being inferior. Yeah, as members of this community we could maybe ignore when other users talk about such views out loud, but that won't really solve anything, would it?, it would just mean hiding everything under the rug, even empowering said user in his/her views.

<comment><USER24> That's vastly overinterpreting what the commenter wrote. Please follow the site guidelines when commenting here. They include:

" _Please respond to the strongest plausible interpretation of what someone says, not a weaker one that 's easier to criticize. Assume good faith._"</comment></comment></comment></comment></comment>


Understanding the magnitude of Tesla's $11b+ triumph - <AUTHOR> http://calacanis.com/2016/04/04/understanding-the-magnitude-of-teslas-11b-triumph/ <comment><USER2> Jason Calcanis has always been a Tesla fan. I know this because when he used to run Mahalo (in Santa Monica), he managed to open the door of his Roadster and scratch up the paint on my not near as cool Scion tC parked right beside him. In typical form, he didn't care at all when I mentioned it to him at the Django meetup, in the Mahalo office. Small world :)

This was maybe 6 or 7 years ago.</comment>


Show HN: The Queue - Find Content, Read It, Save it , Print It - <AUTHOR> http://thequeue.org

<comment><AUTHOR> Including a Bookmarklet, Save to Instapaper button and a print feature.</comment>


Redis: Zero to Master in 30 minutes - <AUTHOR> http://openmymind.net/2011/11/8/Redis-Zero-To-Master-In-30-Minutes-Part-1/

<comment><USER2> If you don't already know how to use Redis, it's hard to think of a technology that offers as good a return on time invested - you really can pick up everything you need to know about it in a few hours, and once you add it to your tool kit you'll find no shortage of problems that it can help solve.

<comment><USER3> Your post got me thinking: what other things can one learn that give ridiculously good return on time invested? Not just technologies, but anything. Here's an incomplete list:

* Graph flow: a lot of interesting problems can be solved efficiently by converting them to graph flow problems.

* Basic microeconomics: the world looks different once you understand it, and it's not that hard to learn. A cheap enlightenment boost!

* Monoid-cached trees. Guy Steele explains better than I could. Video and slides:

<http://vimeo.com/6624203>

[http://labs.oracle.com/projects/plrg/Publications/ICFPAugust...](http://labs.oracle.com/projects/plrg/Publications/ICFPAugust2009Steele.pdf)

Any others? I'm sure I haven't done more than scratch the surface.

</comment></comment><comment><USER4> One complaint: Don't actually use the KEYS command. It's not very efficient. Instead, keep a set of all the keys you would want to delete. When you call KEYS, it has to do a linear scan of every key in the database, and since it is stored as a hash table there really isn't a way to optimize it. It's useful for inspecting things in redis-cli, but not much else.

<comment><USER5> KEYS is indeed only a debugging command, as clearly stated in the documentation.

<comment><USER6> Probably it should be named DBG_KEYS or D_KEYS - some evil hard to type name, that says - I'm to be used very rarely.

But what do I know, just giving friendly advice...

</comment></comment></comment><comment><USER5> Since this was not posted on HN maybe it is a shameless plug that's worth it:

<http://antirez.com/post/short-term-redis-plans.html>

<comment><USER7> Actually it was: <http://news.ycombinator.com/item?id=3208387>

</comment></comment><comment><AUTHOR> Direct link to part 2: [http://openmymind.net/2011/11/8/Redis-Zero-To-Master- In-30-M...](http://openmymind.net/2011/11/8/Redis-Zero-To-Master- In-30-Minutes-Part-2/)

</comment><comment><USER5> What's wrong with the name 'strings' for Redis strings?

<http://en.wikipedia.org/wiki/String_(computer_science)>

If you change the title of the wikipedia article you can read it as a description of Redis strings almost verbatim ;)

<comment><AUTHOR> edit: I don't really want to get into a CS/data structures argument with you, 'cuz I like having a chance at winning when I argue stuff.

It's not wrong. I think when you say String people think of:

       String name = "Leto Atreides";

In Redis (and I guess as a Computer Science term) it means:

      byte[] anything = ....

Although, the Wiki page does say:

      generally understood as a data type...using some character encoding

With the redis implementation not having any character encoding.

It isn't a big deal, but I think you'd find that it can be a little confusing to us some.

<comment><USER5> but strings in most languages such as C, Python or Ruby are exactly random access sequences of bytes. And btw 'binary' is a form of encoding in a string. So Redis strings are strings ;) in a full sense.

I think that if calling what are strings, "strings", can generate confusion, go figure how much confusion can be created by calling them with another term...

<comment><USER8> A byte array is not necessarily a string.

Is an image a string?

Is a video on YouTube a giant string?

They happen to be equivalent at a data storage level, but they're not semantically equivalent. I think the article explains the difference quite well to be honest.

</comment></comment></comment></comment><comment><USER9> Read the source code of <http://lamernews.com> on github <http://github.com/antirez/lamernews> .It's written by the author of Redis.

</comment><comment><USER10> I love that this is so direct, and it's helped me learn more about redis.

Some of the explanations are a bit confusing though. I'm not sure I know what a List is - though I suspect, knowing JavaScript, Python, and PHP pretty well, I know what a List is. Descriptions like "If a hash is like a string with an extra field layer" are just plain confusing. Why not put it in terms of commonly understood programming languages? The examples are in Ruby, why not let us know which Ruby structures these most closely correspond to, for instance?

I do appreciate the particular names in the example text, however so I'm willing to overlook this.

<comment><AUTHOR> You are right. I cleaned up a bit of the garbage and just used a more straightforward explanation. Thanks.

<comment><USER10> Thanks for listening to my suggestions! Again, I appreciate the article.

</comment></comment></comment><comment><USER11> At that rate of learning, you'll surpass antirez in a few days.

<comment><USER12> I started last week and compiled a stable Redis 2.8 this morning.

</comment></comment><comment><USER13> What is the recommended configuration for in-memory stores like Redis? On a separate machine than its client application?

If it's all going in memory, why not run the client on that machine and use native data structures?

<comment><USER5> There are obvious reasons for needing networking access, but even when you don't need it:

1) Redis is much more memory efficient than your average programming language. 2) Redis handles persistence and replication for you. Not a joke. 3) Accessing data with atomic operations on complex data types from your application using multiple threads can create more problems. 4) Your application is free of doing other stuff if you use a non blocking client, if you are asking Redis to do computationally intensive tasks. 5) There is no easy way to model Redis sorted sets, in many very high level languages, at the same speed, without writing a C extension.

<comment><USER14> Another benefit I run into is sidestepping the GIL in Python or Ruby. Instead of using multiple threads to manipulate data in-process, use multiple processes to manipulate the data in Redis.

</comment></comment></comment><comment><USER15> My first questions about any new dev tool/tech are "Where does it run?" and "What is the primary interface for working with it?"

So I'd love to see the paragraph that begins "There are various ways to install Redis" expanded. In particular, what's the workflow between a) I just downloaded something called Redis, and b) I am issuing commands to learn along with these examples.

Otherwise very clear and concise. Well done.

<comment><USER16> Only used Redis on *nix, but there it's very easy: Just download, unpack and compile.

No need to install it just to play with it..

Compiles in no time at all =)

</comment></comment><comment><USER6> I really want to try redis, but Windows is big part of our development. Is there any port of redis to Windows?

<comment><USER17> <https://github.com/dmajkic/redis/downloads>

<comment><USER6> Thanks!

</comment></comment></comment><comment><USER18> In part 2, correct me if I'm wrong, but isn't the member you're adding for jobs a key? Isn't the code to see if it's already a member unnecessary? It shouldn't add duplicates since the member's value doesn't change (it's a key). Granted, it's unnecessary to re-add the key to the jobs sorted set as the code does, but the explanation doesn't seem right. On the other hand, you could call zadd again to change the score.

<comment><AUTHOR> uh..ya, ur right. I'll fix it

<comment><USER18> I see you fixed it. Cool. By the way, I've been using Redis extensively for a while and your tutorial is quite good. Congrats!

</comment></comment></comment><comment><USER19> Redis is just a good piece of software. I don't see any reason why a web programmer couldn't benefit from spending time to learn it.

I simply don't think a remotely complex web application can do without it these days.</comment>


Asking for volunteers to test a videochat product (for free) useful for elderly - <AUTHOR> https://www.kinokochat.com/ <comment><AUTHOR> Hey HN, we're a group of former IoT colleagues who have built a way to video chat with your family on the TV. Yes, we know, another video chat product. But this one is deliberately dead simple. No accounts, no passwords, no room links, no collection of personal data. There's not even a mute button. It's just a direct connection between two units that sets up a video call on your TV. All you need is a Wi-Fi connection and a TV with an HDMI port. It takes one click of the included bluetooth button to enter the chat, and you can keep it on for as long as you'd like.

Why did we build it? Because we were tired of crowding around a laptop with our entire family, or telling our parents to hold their iPad a little higher and make sure it's charged. Because we inevitably shared the wrong zoom link, which was awkward. Because we're all a bit wary of putting a Facebook device in our living rooms.

And because for an older demographic, the experience of sitting on the couch with a big screen and sound is just a lot more pleasant. We think this product could be especially beneficial for caregivers and folks who, especially given the current circumstances, can't visit their older loved ones.

We have a limited number of units available for public testing, and we'd like to give them away for free in exchange for some real feedback. If you're interested, please reach out to us at firstname.lastname@example.com and tell us how you'd use the product. Thanks for your support.</comment>


Trump's Policies Are Sending Precious Startup Jobs to Canada - <AUTHOR> https://www.wired.com/story/pausing-international-entrepreneur-rule-sends-jobs-to-canada <comment><USER2> There's more to startups than just hiring immigrants, sheesh.

Our VCs are hilariously risk-averse for the pennies they give out.

Our digital infrastructure sucks, you need to be in the Toronto core if you need a fat internet connection.

Your labour pool is spread out over all of southern ontario and the transit system is brutal.

At my last job, 2 developers left for the states because they "couldn't deal with winter anymore".

The market salaries are 50% less and the canadian dollar is worth 20% less than the USD, so you'll take a ~70% paycut if you move from the US to Canada.

On top of all that? If you're running a business here, most of your clients will be from the states anyways. Our population is roughly the size of California. So you'll be travelling there anyways for conferences and big sales.</comment>