How the Dat Protocol Works - <AUTHOR> https://datprotocol.github.io/how-dat-works/ <comment><USER2> The diagrams on this page are FANTASTIC.

The dat homepage[1] is also nicely done. It would be interesting to see how much and how it's being used. Since there is a central tracker does anyone know if there are stats available?

[1]: [https://dat.foundation/](https://dat.foundation/)

<comment><USER3> Any specific tooling generating these or just a good designer?

<comment><AUTHOR> Duncan Keall is just talented. He also did the Scuttlebutt Protocol guide ([https://ssbc.github.io/scuttlebutt-protocol- guide/](https://ssbc.github.io/scuttlebutt-protocol-guide/)).

Here is a blog post describing how it was made: [https://blog.datproject.org/2019/01/21/how-dat- works/](https://blog.datproject.org/2019/01/21/how-dat-works/)

</comment></comment></comment><comment><USER4> Dat (and Beaker Browser) is one of those things that I think about playing with every time I see it pop up somewhere. The technology truly feels like something _special_.

Then I sit down to actually start playing with it I struggle to think of anything to build that would actually take advantage of the tech aside from purely static sites. I would really love to see more complex examples folks have seen in the wild.

<comment><USER5> A while ago I made a demo of a P2P chat application that runs on the beaker browser and uses the experimental P2P APIs. It's running at dat://feathers- chat.hashbase.io (slides from the talk at [https://vanjs- decentralized.hashbase.io](https://vanjs-decentralized.hashbase.io)). If the peers can see each other you should be able to register with any email address and send chat messages between browser instances.

What I realized is that this technology ticks a lot of the boxes of what we right now think only the big cloud providers can do. By using a more decentralized protocol it is by design

\- Actually Serverless

\- Offline-first

\- Real-time

\- Auto-deploying

\- Live-updating

\- 100% uptime

I really think there is something there from both, a developer and user experience perspective. The problem is that a lot of it is still very experimental and far from the usability and maturity of e.g. a Firebase or Heroku.

<comment><USER4> This is a really cool idea, but how should I be using it? Trying to test it out just now I opened up two tabs in Beaker but both rooms just have the one user in them.

Love the design, by the way.

<comment><USER5> I believe this is happening because it is using the same peer connection. If you start to separate instances of the browser (e.g. in a VM or another machine) you should be able to see both users.

The design is taken from the Chat guide for [https://feathersjs.com](https://feathersjs.com). Feathers is a JS library that allows to architect APIs in a way that they are protocol independent. Which worked great in this case because I just had to swap out the existing REST/websocket Feathers adapter for a DAT/Beaker API Feathers adapter.

</comment></comment></comment></comment><comment><AUTHOR> Also see the Scuttlebutt Protocol guide: [https://ssbc.github.io/scuttlebutt- protocol-guide/](https://ssbc.github.io/scuttlebutt-protocol-guide/)

Discuss: [https://news.ycombinator.com/item?id=20364154](https://news.ycombinator.com/item?id=20364154)

</comment><comment><USER6> As a tech illiterate, what makes DAT superior to the standard p2p and blockchain technology? If my understanding is wrong, can someone explain in laymen term why DAT matters for the future?

<comment><USER7> Say you have a 1 GB file that you want to share with everybody in this thread. How do you do this? What if you have thousands of such files? In theory, DAT and similar DHT-based file sharing protocols, can enable us to implement something like YouTube, but without the enormous costs for running the datacenters. In practice, ISPs won't let this happen and DAT-like protocols will be limited to the world of VPSes that have static IP addresses.

<comment><USER8> My hope is that asynchronous swarm based applications ("dweb") will work so well without ISPs (e.g. on dynamic local networks and mesh networks) that they will pose an existential threat to ISPs.

"Think WAN, act LAN"

or is it

"Think Wan, net LAN"?

I don't know.

</comment></comment></comment><comment><USER9> Genuine question to hopefully help others as well: What are the major differences with dat and IPFS?

<comment><USER10> > _What are the major differences with dat and IPFS?_

• [https://docs.datproject.org/docs/faq#how-is-dat-different- th...](https://docs.datproject.org/docs/faq#how-is-dat-different-than-ipfs)

• [https://github.com/ipfs/faq/issues/119](https://github.com/ipfs/faq/issues/119)

</comment></comment><comment><USER11> DAT seems very well thought through, and having never seen beaker before that looks amazing! However, how private is a P2P service like this? One major problem for BitTorrent is how trivial it is to snoop other peoples history (esp if you're the RIAA or MPAA, or even iknowwhatyoudownload). Is overcoming this a design goal for dat? Is it even possible to overcome it?

<comment><USER12> There's some information about this in the protocol description:

"Discovery keys are used for finding other peers who are interested in the same Dat as you.

If you know a Dat's public key then you can calculate the discovery key easily, however if you only know a discovery key you cannot work backwards to find the corresponding public key. This prevents eavesdroppers learning of Dat URLs (and therefore being able to read their contents) by observing network traffic.

However eavesdroppers can confirm that peers are talking about a specific Dat and read all communications between those peers if they know its public key already. Eavesdroppers who do not know the public key can still get an idea of how many Dats are popular on the network, their approximate sizes, which IP addresses are interested in them and potentially the IP address of the creator by observing handshakes, traffic timing and volumes. Dat makes no attempt to hide IP addresses."

<comment><USER13> So as I understand it, if there was a popular movie shared via dat and people were sharing it (the key would have to be publicly known for people to discover it at all), it would be trivial for the MPAA to snoop and see what IPs were downloading it, just like torrents now.

Is that correct? Would those snoopers be able to prove who is sharing and who is downloading?

</comment></comment></comment><comment><USER14> The first thing that popped to mind for me was that this seemed like a good protocol to build some sort of decentralized/private social network, then in their docs I managed to end up looking at this: [https://github.com/Rotonde/rotonde- client](https://github.com/Rotonde/rotonde-client)

</comment><comment><USER15> "Hypercore assumes a linear history. It can not support branches in the history (multiple entries with the same sequence number) and will fail to replicate when a branch occurs. This means that applications must be extremely careful about ensuring correctness. Users can not copy private keys between devices or processes without strict coordination between them, otherwise they will generate branches and 'corrupt' the hypercore."

[https://www.datprotocol.com/deps/0002-hypercore/#the- linear-...](https://www.datprotocol.com/deps/0002-hypercore/#the-linear- history-requirement)

</comment><comment><USER16> That's a beautiful guide. Anyone know how the diagrams were created?

</comment><comment><USER17> The accessibility of that guide with a screen reader is horrible. The text is readable, but that's about it. Why, oh why couldn't people just write good, textual articles like they used to, instead of all this picture based crap? Instagram, Tinder and snapchat are bad anough as they are, but tech was mostly free of this, but then we see stuff like this appearing. I won't really be surprised if I will not be able to read 50, 60% of the content in 10 years if current trends continue.

<comment><USER8> What would help you screen read a visually oriented guide? Would detailed textual descriptions of the figures help?

Personally I think that the way they're using diagrams to illustrate some of these technical concepts is really helpful. I wonder if the format would translate to spoken text? I.e. in parallel with the figures, having little descriptive "imagination breaks" in the text which would provide opportunity for focusing on some key concept or relationship that might otherwise be illustrated using a visual figure. Maybe even using spatial language?

IMO there should be way more aurally oriented technical materials.

Recently I was listening to math lectures on YouTube while driving and realized that they were much easier to follow than I expected. Still, I'd love it if there was an aurally focused higher math practice. Sometimes I find it easier to focus when I'm listening rather than using my eyes.

<comment><USER17> In this case, not really descriptions, but equivalents. I don't care that a figure shows two red boxes connected in a particular way, I care that packets flow from node x to y, but not in the other direction (just an example).

</comment></comment></comment><comment><USER18> I noticed the other day _GNUnet_ [0] here on HN, and it seems to me that there could be a bit of overlap here, especially for the filesharing system... Is there any collaboration happening between these two systems? From my couch perspective, they both look great and seem as though they could compliment each other really well.

[0] [https://gnunet.org/en/#gnunet-0.11.5-release](https://gnunet.org/en/#gnunet-0.11.5-release)

</comment><comment><USER19> They really need to work on a better explanation of what exactly this is! Something like "It's similar to BitTorrent except..."

<comment><USER13> [https://docs.datproject.org/docs/faq#how-is-dat-different- th...](https://docs.datproject.org/docs/faq#how-is-dat-different-than- academic-torrents-or-bittorrent)

>Academic Torrents [13] uses BitTorrent to share scientific datasets, and BitTorrent has many drawbacks that hinder direct use by scientists. BitTorrent is for sharing static files, that is, files that do not change over time. Dat, on the other hand, has the ability to update and sync files over the peer-to- peer network. BitTorrent is also inefficient at providing random access to data in larger datasets, which is crucial for those who want to get only a piece of a large dataset. BitTorrent comes close to the solution, but we have been able to build something that is more efficient and better designed for the data sharing use case.

<comment><USER19> Much better. They should stick that on the home page, not in the FAQ.

</comment></comment></comment><comment><USER20> Is this similar to Syncthing?

<comment><USER8> It's kinda like syncthing crossed with git. Lower level than syncthing, focused on stuff like p2p shareable, versioned data structures and cryptographic verification.

Although you can build applications on top of syncthing's API/protocol, it's primarily a high level application, whereas Dat is a protocol meant for building applications.

It would make sense to build something like Syncthing on top of Dat.

</comment></comment><comment><USER21> I didn't see IPFS mentioned on the page so I was wondering if someone here could help me understand the difference between this and IPFS which seems to share at least some of the same goals.

<comment><USER22> IPFS as I understand it doesn't have a built-in ability to alter files once uploaded and has to use IPNS to handle pointing to newer versions over time. Dat Protocol does this for you as I understand it haven't dug in too much.

</comment></comment><comment><USER23> Dat seems like it could be a good way for those infosec people and others that don't like youtube to share their videos.

Is there any way to stream video using OBS via dat?

</comment><comment><USER24> I'm sorry, but accessibility and UX is a primary focus, and then you explain that urls consist of a gigantic hex string?

I can't share a file without having to copy paste the url, and I can't look at two similar urls and immediately determine which is which. No thanks.

This seems like yet another product that seems good on paper, and seems great if you're a nerd, but this will never catch on with the average consumer.

<comment><USER8> The keys are akin to IP addresses, which are also not so human friendly.

People are still trying to wrap their heads around decentralized naming systems. There are lots of ideas, and nothing has stuck yet in a groundbreaking way.

But yeah, the naming problem is independent of what this protocol gives us.

<comment><USER22> I think the best idea for naming is probably going to be something like giphy or what3words (though their implementation could use work since they're not directly translatable between multiple languages) where a hex address gets encoded in N words which are generally easier for people to grok than N*m (m being 2/4/8 depending on how many hex characters are encoded in which words). Choosing the words is hard though especially as you add more languages.

</comment></comment></comment><comment><USER23> Can you share on dat via tor?

</comment><comment><USER25> is this different from ipfs?

<comment><USER13> Yes.</comment></comment>


Ask HN: How would your life change if Facebook/Twitter disappeared? - <AUTHOR> Thinking about the big tech giants, it's quite obvious the world would pretty much fall apart if Google disappeared overnight. On the other hand, I wouldn't lose anything if Facebook/Twitter were to meet the same fate. We then have the likes of Netflix/Amazon/Uber who provide useful "convenience luxuries" but nothing life changing in my opinion. Curious to get the HN perspective on this, specifically, a view on the company valuation in the context of obsolescence. <comment><USER2> I wouldn't lose anything if Facebook disappeared, since I don't use the site all that much.

As for Twitter? Well that'd be a bit more of an issue, though I'd probably move to a competing service like Mastodon/GNU Social or Gab.

Still, I'm not exactly a social media addict, so I guess it'd be more interesting to hear what'd change for those.

</comment><comment><USER3> If FB suddenly disappeared, I'd lose almost all my good friends and acquaintances. And it would be hard or impossible to find them all again; they're scattered around the globe, mostly FB is the only contact point I've needed/had, dont have their phone, email or address. I should get that info, huh. On FB I almost exclusively just chat with friends.

Never used Twitter or wanted to.

ps Why would the world fall apart if Google disappeared? It's not obvious to me.

<comment><USER4> Most people don't know how to use the internet without Google.

I'm sure many industries are supported with the likes of "Just Google it"

<comment><USER5> actually, while you could replace google search with bing or duck duck go, what would really destroy the world is the sudden absence of gmail/google docs/android updates/youtube, all of those things could be replaced, but it would take more time...

</comment></comment></comment><comment><USER6> They're a great source of news. I just activated my Twitter after a decade or so of avoiding it.

Malaysia in particular is vulnerable to propaganda, because political parties own all the major free channels and newspapers. Social media has helped balance it. While the government has blocked Medium, places like Facebook and Twitter bring world news to the masses, including any stories like local corruption that a news site like WSJ would investigate but none of the local ones could.

Also about 80% of my jobs come from Facebook connections, so it is very valuable to me.

If both of those went down, we'd probably move to Reddit, Instagram, LinkedIn, Pinterest. I think those sites are even worse for society.

</comment><comment><USER7> I have a lot of Facebook friends that I would completely lose touch with, because I have no backup contact info. If I knew ahead of time that Facebook was going away, I could procure a backup method but if Facebook just disappeared one day, I would have little chance of contacting a number of acquaintances and former colleagues. In the past, I probably would have had a Rolodex, a collection of business cards or even a local phone book to fall back on, but Facebook has pretty much replaced all of that. Even though I have mainly stopped browsing timelines, this is the main value of Fb now for me.

<comment><AUTHOR> Really? I have almost everyone on LinkedIn which is to be expected once you've graduated.

<comment><USER8> Seems like only my lawyer or software friends are on LinkedIn.

</comment></comment></comment><comment><USER9> FB: my friends would probably return to using email instead of FB (which I avoid for a number of reasons) to organize events and announce life changes like births and marriages. I'd be back in the loop. This would be a good thing.

Twitter: no effect whatsoever.

</comment><comment><USER10> I would lose contact with some people on FB. I haven't posted in about 5 years so I don't think I would really miss it. I'd like some warning so I could get some contacts out of it first before it closes

</comment><comment><USER11> I'm Facebook and Twitter free. Ironically HN is the most social media-y you get.

</comment><comment><USER12> Facebook: no change.

Twitter: I think I'd lose access to a lot of real-name graphics/game programmers who I like casually interacting and sharing with. On the other hand, I'd be spared their frequent overbearing political opinions.

</comment><comment><USER13> It would make it basically impossible for me to know what my overseas and East Coast friends have been up to in their lives.

I don't get my news or amateur opinions from social networks so it doesn't affect me there.

</comment><comment><USER14> I don't use FB, Twitter or any other social media so nothing would change there.

I do use Netflix though - ad-free entertainment for less than £8 a month - bargain!

</comment><comment><USER15> Not using them or any similar services, so I wouldn't care if they disappeared overnight.

</comment><comment><USER16> It would not change at all, I don't use them...

</comment><comment><USER17> I don't use those.</comment>


Naps more effective than either caffeine or getting more sleep at night - <AUTHOR> http://www.psychologytoday.com/blog/sleep-newzzz/200907/nap-vs-caffeine-vs-more-nighttime-sleep

<comment><USER2> Anecdote: For most of his adult life, my grandfather, a now-retired apple farmer, slept 5-6 hours a night, but supplemented this with 20 minute naps after lunch (which was a huge hot meal) and dinner (which was more like a lunch). Never have I seen him as exhausted as I am after my 3rd cup of coffee and 8 hours of sleep a night. And I have a desk job!

</comment><comment><USER3> Who the heck can sleep for just 20 minutes? Do certain people have a little "sleep now" button they can press?</comment>


High-Voltage DC Breakthrough Could Boost Renewable Energy - <AUTHOR> http://news.nationalgeographic.com/news/energy/2012/12/121206-high-voltage-dc-breakthrough/ <comment><USER2> Ohms law: V = IR

Power: P = VI

Power loss due to resistive heating in the wires: P = I^2 * R

Note that current (I) quickly dominates the loss calculation because it is squared. So for transmission lines, efficiency is obtained by keeping the current as low as possible. Therefore, to transfer the same power you need to make the voltage as high as possible.

The I^2*R heating loss applies equally to DC and AC transmission lines. So the DC lines still need to be high voltage for efficiency as well.

By using AC we can 'transform' high-voltage-low-current to low-voltage-high- current power very easily. It's literally just two coils of wire around a chunk of iron. Step-down AC transformers are used both at your neighborhood substation and on the pole outside your house.

But the article didn't say how they planned to transform the high-voltage-low- current DC into anything usable.

<comment><USER3> Historically the limit on voltage has been the dielectric constant for air. Once you exceed it, the air ionizes, and you get a short. Visible as 'lightning' going through the air from source to its ground counterpart. Transmission line effects limit the ability to send AC power long distances as the every time you switch the potential you generate magnetic fields and some of the energy is radiated off into the space around the wire.

AC has been more desirable to date because of a relatively easy infrastructure for converting it, and for sharing it on the same line. Transforming DC into AC is easily done by driving a generator set but for most things you'd want the genset to convert to 'standard' three phase AC and then re-use existing city wide infrastructure.

What bothered me about the article was that nowhere does it explain how you can make a DC breaker that can actually disconnect at those voltages. Since the dielectric constant is a function of distance, when the breaker first opens there is an arc because, the current really really wants to keep flowing. And the dielectric constant of ionized air is quite small. "Regular" breakers have a fan that kicks in once they get to a minimum distance apart to "blow out" the arc (it pushes it out to extend it and thus 'break' it, see the video here <https://www.youtube.com/watch?v=hIkNY5xjy5k> for how this works)

Anyway, would love to hear actually what it was they invented.

<comment><USER2> As ususal, looks like the best description made it into Wikipedia <http://en.wikipedia.org/wiki/High-voltage_direct_current>

_The ABB breaker contains four switching elements, two mechanical (one high- speed and one low-speed) and two semiconductor (one high-voltage and one low- voltage). Normally, power flows through the low-speed mechanical switch, the high-speed mechanical switch, and the low-voltage semiconductor switch. The last two switches are paralleled by the high-voltage semiconductor switch.

Initially, all switches are closed (on). Because the high-voltage semiconductor switch has much greater resistance than the mechanical switch plus the low-voltage semiconductor switch, current flow through it is low. To disconnect, first the low-voltage semiconductor switch opens. This diverts the current through the high-voltage semiconductor switch. Because of its relatively high resistance, it begins heating very rapidly. Then the high- speed mechanical switch is opened. Unlike the low-voltage semiconductor switch, which is only capable of standing off the voltage drop of the closed high-voltage semiconductor switch, this is capable of standing off the full voltage. Because no current is flowing through this switch when it opens, it is not damaged by arcing. Then, the high-voltage semiconductor switch is opened. This actually cuts the power. However, it only cuts power to a very low level; it is not quite 100% off. A final low-speed mechanical switch disconnects the residual current._

<comment><USER3> Oh, I like that. A pretty clever design.

</comment></comment></comment></comment><comment><USER4> If I am not mistaken we already use HVDC on high power runs due to its much lower loss ? How does this beaker change that ?

<comment><USER5> According to the article HVDC is only used for point to point electrical runs, because managing grids of it is very hard. What the breaker does is allow for easier managing of the grids by allowing parts under going surge, like from a lighting strike, to be segmented off.

<comment><USER6> Exactly. The modern AC power grid is very resilient due in part to the multi- interconnect nature which allows for parts of the grid to go down without taking down the whole network.

On an AC grid, there are breakers that automatically disconnect and often reconnect.

<http://en.wikipedia.org/wiki/Recloser>

I guess that this HDVC hybrid breaker accomplishes more or less the same thing.

</comment></comment></comment><comment><USER7> DC doesn't have phase and frequency issues AC does. Presumably a shared DC grid would be constant-voltage. What surprises naive me is that gigawatt inverters seem to be a done deal. Once we have gigawatt-hour ("utility-") scale _storage_ , then we can have a fully renewables-based future (again, naive me thinks).

<comment><USER8> The government should be building out infrastructure like this.

<comment><USER9> Or maybe someone who could profit from it.

<comment><USER10> Rent seeking doesn't lead to innovation.

<comment><USER9> Hence why the government shouldn't be involved. I live in a state (Pennsylvania) with energy choice. The one with the most "efficient" system gets the lowest rates, which allows me to preserve my own little slice of the "social wealth". If I so choose, I can go for an eco-friendly solution as well, if that sort of thing made me feel good.

<comment><USER8> By this logic, the interstates, the railroads, and the internet should not exist.

<comment><USER11> That makes no sense. First, he's advocating private ownership of electrical infrastructure, not the elimination of it. Built by private interests doesn't equate to non-existence.

Second, advocating private ownership of electrical infrastructure doesn't equate advocating private creation and ownership of all infrastructure.

I rather like a good straw man; this one isn't.

<comment><USER8> _> Built by private interests doesn't equate to non-existence._

Sometimes it does.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> I would love to have my house run on DC. Almost everything I own either switches to DC internally, or doesn't care. If it were practical to run the grid on DC, or we all go to solar for residential and light commercial, maybe I'll get my wish.

<comment><USER12> The stuff in your house that "switches to DC internally" isn't switching to 100,000 to 800,000 volts[1], which is what the HVDC is that the article is talking about. Running DC in your house isn't going to eliminate any voltage conversions that are already going on, they will just be DC-DC converters with 48vDC (or whatever) input rather that 120vAC input.

Most of the stuff in your house isn't even running on the same voltages internally... most of the electronics nowadays use many voltage "rails":

* 17-19v laptop chargers (the voltage has to be higher than the battery's voltage in order to charge the battery)

* 12v for older hard drives

* 5v for not quite so older hard drives and older logic

* 3.3v for newer logic and newer hard drives

* A veritable plethora of sub-3.3v rails for CPU chips and other high tech chips.

* 120vDC for "universal" motors unless you replace all your appliance motors with 48vDC (or whatever) motors

In addition, the current requirements of running at a non-lethal DC voltage inside your house would require much larger wire gauges than your current wiring in order to keep the resistive losses reasonable[2]. The rewiring costs would be staggering.

[1] <http://en.wikipedia.org/wiki/High-voltage_direct_current>

[2] Charts for 12v wiring, 48vDC would be 4x better, but still not good - [http://www.westmarine.com/webapp/wcs/stores/servlet/WestAdvi...](http://www.westmarine.com/webapp/wcs/stores/servlet/WestAdvisorView?langId=-1&storeId=11151&catalogId=10001&page=Marine- Wire)

<comment><USER11> "The stuff in your house that "switches to DC internally" isn't switching to 100,000 to 800,000 volts[1], which is what the HVDC is that the article is talking about. Running DC in your house isn't going to eliminate any voltage conversions that are already going on, they will just be DC-DC converters with 48vDC (or whatever) input rather that 120vAC input."

Being an electrician, I'm aware of that. The reason I don't have a DC house is because the grid couldn't transmit DC long distance, back when the War of the Currents was fought. Now, maybe I'll get my D/C house soon.

</comment></comment></comment><comment><USER13> When I was a kid, Scientific American had a cover featuring a high voltage transistor, which was supposed to revolutionize power transmission.

My very weak understanding is that energy transmission is directed by a delicate balancing of loads. Which is inefficient. Where using a solid state switching solution would be more efficient.

Does this sound familiar to anyone?

</comment><comment><AUTHOR> I didn't follow why the article stated that AC is not ready for the grid of the future. This DC grid seems to be an incremental improvement, 10%? But IMO not a huge breakthrough.</comment>


Ask HN: Front end framework with focus on productivity - <AUTHOR>

Let me clarify by what I mean with productivity: I'm developing a couple personal apps (ie. one user and one developer), and because of that, the most important "feature" for me is my ability to crank out code. Compatibility, maintainability (or to be specific, the usual concern with multiple developers and such), speed, scaling etc. be damned.<p>In a sense, I'm looking for the front end equivalent of Arc/ Lisp for UI developing. What should I be looking at? <comment><USER2> If you're thinking small, something like [http://vuejs.org/](http://vuejs.org/) or [http://knockoutjs.com/](http://knockoutjs.com/) for views / data binding, and whatever else needed to structure the rest of your logic.

I'm not mentioning the big popular frameworks because more than enough has already been said about those elsewhere.

</comment><comment><USER3> AngularJs all the way.</comment>


Ask HN: I love astronomy and I would like to do something about it. - <AUTHOR>

I love astronomy, and I wanted to become an astronomer since I was kid. I build web sites now.
I would love to do something about it. I would buy a telescope, but I rent an apartment on the 9th floor and I have windows only on one side of the room. I don't even know where to start. After work hours, I am working on my pet projects, and I would love to do something with astronomy.
Any suggestions? <comment><USER2> Have you tried going camping? There's too much light pollution in cities anyway.

I come from Pune (India) and we were lucky to have a world class astronomy research institute a stone's throw away from my house. They often organised outdoor events for young people. Check it out - <http://www.iucaa.ernet.in>. They have some community projects like the Virtual Observatory which you might be able to do something with remotely (I think you can get loads of data). I know the director of the institute personally, so if you do find it useful and want to explore further, drop me an email (in profile).

<comment><AUTHOR> I am no where close to pune, but I will check it out when I come to visit my family.

</comment></comment><comment><USER3> Is there an open-to-the-public observatory anywhere near you? About a half- hour from where I live there is a small observatory with some reasonably heavy-duty telescopes, and areas to set up your own telescope, with monthly "guided tours" of what can be seen at each respective time of the year.

<comment><AUTHOR> Yes, I found one. It is open to public only on tuesday's, so I will have to check it out. Thanks for the tip.

</comment></comment><comment><USER4> Read 'The Age of Wonder' By Richard Holmes. It will get you outside ;)

<comment><AUTHOR> Sounds like a good book

</comment></comment><comment><USER5> Are there any astronomy groups that meet where you live?

<comment><USER5> Also, as for a first telescope you might consider the galileoscope:

<https://www.galileoscope.org/gs/></comment></comment>


White House seeks to end subsidies for electric cars, renewables - <AUTHOR> https://www.reuters.com/article/us-usa-trump-autos-idUSKBN1O22D4 <comment><USER2> Good but unilateral.

If subsidies for electric cars are removed, then subsidies for ICE cars should be removed.

If subsidies for solar electric generation are removed, then subsidies to coal electric generation should be removed.

<comment><USER3> What subsidies do ICE cars have that electrics don't?

<comment><USER4> Cheap extraction leases on public land.

Huge military investment in oil-producing countries and regions.

Building and transport codes built around the needs and technologies of dinosaur burners.

Lack of significant restrictions on emissions.

I don't think you're gonna get back the sum of the decades of subsidies of various sorts that cut the cost of technology developments.

(yes, I know they aren't really "dinosaur" burners but alge burners)

<comment><USER5> Tarrifs are also a tax.

<comment><USER4> Indeed but this was responding to a request for a list of _subsidies_ for internal combustion.

</comment></comment></comment></comment></comment><comment><USER6> Not against this. Ethanol subsidy has caused a lot of environmental damage. Wind subsidy has created a crisis as there is no money to make repairs. Solar can stand on it's own.

<comment><USER7> Just as soon as oil and gas stands on its own (it doesn't) and correctly prices in the negative externalities (it doesn't). Then the free market can knock itself out.

</comment></comment><comment><USER8> Clickbait. FTFA

 _This story corrects from "will end" throughout, to make clear the White House cannot end congressionally mandated subsidies on its own, and adds quote_

</comment><comment><USER9> So, so, so dumb. Dumb decision. You should subsidise renewables and tax all others for the externalities.

<comment><USER3> What are the externalities of the renewables?

<comment><USER10> Space usage and maintenance. Modifications of electric grid. Rare metal use.

Compared to health and environmental damage done by mining, gas and oil prospecting, it's cheap, but not as cheap as nuclear.

<comment><USER9> What makes you think nuclear has less severe externalities?

And I am not even including in my considerations that countries have totally mismanaged the storage in the past (e.g. UK dumping it into the ocean, Germany dumping it into an old mine etc.)

[https://en.wikipedia.org/wiki/High- level_radioactive_waste_m...](https://en.wikipedia.org/wiki/High- level_radioactive_waste_management)

</comment></comment></comment></comment><comment><USER11> What about petroleum and natural gas?

</comment><comment><USER12> This is only one half of the story, there is also the export drive going on to force LNG and other hydrocarbon products that 'fracking' is delivering on the rest of the world. Lame NATO friends of America such as Poland have to buy expensive LNG from Uncle Sam instead of their Russian neighbours.

The same thing happens with that current hotspot of Ukraine where these policies have led to a very cold winter for the people that live there. They are being 'forced' to by that coal that Trump has insisted on getting dug up from the ground, a fuel nobody wants any more (even though coal fires are so nice to have).

Renewables are cheaper than fossil fuels, the move to them is unstoppable due to this price advantage. Much progress is unseen, in a world where people are so fixated on being negative and nit-picking those building out the carbon neutral future are quietly getting on with it. They don't need the State Department and the Pentagon to bully others into buying windfarms. They also don't need naysayers that claim windfarms are too difficult to maintain or other hysterical nonsense about 'China and rare earth metals', 'lithium and polluting wars', 'the power is never there when it is needed' and all of that tripe. Funny how some people can't collaborate and have to have hissy fits just so they can stay stuck in the dirty fuel past. Trump is not the problem here, pettiness from weak minded people who believe what they read in newspapers is the bigger downer.

<comment><USER13> The 1% don't care how money is made (who it kills or destroys the planet) so long that it flows. An existential threat from externalities is just another new inelastic market to exploit.</comment></comment>


Where can I find great/new programming tech talks? - <AUTHOR>

I'm a 4th year CS major about to graduate in May, but not starting work till July. I'm really interested in soaking up as much CS knowledge/experience during these few months so I can try to bring myself up to speed with the 'latest and greatest' in industry. Here are some examples of talks/videos I've watched recently that I found interesting and would love to find videos similar to:<p>* Arista's EOS: http://www.aristanetworks.com/en/products/eos<p>* Palantir White Videos: http://www.palantirtech.com/government/videos/whitevideos<p>* Scala at Twitter: http://ontwik.com/scala/scaling-scala-at-twitter-by-marius-eriksen/<p>Thanks <comment><USER2> Talks at conferences tend to be as much about entertainment as education. Let's be honest, there's only so far you can go with a subject within the confines of a one hour speech. So if you're looking to "soak up as much CS knowledge/experience" as you can, try reading some of the classics of CS literature.

Jeff Atwood's list of recommended reading is a good place to start: [http://www.codinghorror.com/blog/2004/02/recommended- reading...](http://www.codinghorror.com/blog/2004/02/recommended-reading-for- developers.html)

This discussion on Stack Overflow also has some good entries in it: [http://stackoverflow.com/questions/1711/what-is-the- single-m...](http://stackoverflow.com/questions/1711/what-is-the-single-most- influential-book-every-programmer-should-read)

<comment><AUTHOR> That's a good point, but as I was looking over some of the chapters of the highest-rated books, they seemed to be teaching mostly high-level, abstract concepts and ideas without discussing specific technologies or tools. I don't want to come off as naive for dismissing books like these, but a significant part of my CS education in college has used some of these textbooks and I wanted to learn more about specific technologies and services.

I feel most inadequate reading about how start-ups are using actual new technologies like AWS or Couchbase. It seems as if tech talks often provide a good overview of the applications and constraints of new technologies. So given the fact that I do not really have any idea what specific projects I will be working on in the future, I feel that having at least a general overview of what tools are available will prove to be useful.

<comment><USER3> This is one of the biggest challenges with teaching programming, everything changes incredibly quickly. If you want to write a book that people will recommend for decades, or even for years, you need to gloss over any specific technology and only talk about the concepts that will not change for quite a while.

With that said, InfoQ.com has a lot of videos that sound like what you're looking for.

</comment></comment></comment><comment><USER4> The talk of Rich Hickey (Clojure) on Persistent Data Structures and Managed References gives a very interesting view on programming in general.

[http://www.infoq.com/presentations/Value-Identity-State- Rich...](http://www.infoq.com/presentations/Value-Identity-State-Rich-Hickey)

</comment><comment><USER5> PyCon 2011 talks: [http://pycon.blip.tv/posts?view=archive&nsfw=dc](http://pycon.blip.tv/posts?view=archive&nsfw=dc)

</comment><comment><USER6> You should check out <http://ontwik.com>

</comment><comment><USER7> <http://infoq.com>

<http://www.google.com/events/io/2010/sessions.html>

<comment><AUTHOR> Thanks, infoq looks really helpful

</comment></comment><comment><USER8> Webstock has all of its conference talks online. 2011 (<http://www.webstock.org.nz/talks/events/webstock-11/>) has some awesome talks (I went, I should know :P ), and I suggest watching them all.

Speakers include Marco Arment, Doug Bowman, Josh Clark, Merlin Mann, John Gruber, Jason Cohen... so many amazing speakers. (Skip Mark Pilgrim though.)

</comment><comment><USER9> If you're interested in iOS development you should definitely check out the "Developing Apps for iOS" Stanford classes. They're available for free.

It's really high quality learning material, I highly recommend it.

[http://itunes.apple.com/us/podcast/cs193p-student-final- proj...](http://itunes.apple.com/us/podcast/cs193p-student-final- projects/id395605774?i=90218598)

</comment><comment><USER10> Java specialist newsletter. Discusses design patterns and advanced topics. <http://www.javaspecialists.eu/>

IBM's developer works. Focused on libraries and frameworks used in the enterprise and new technologies. <http://www.ibm.com/developerworks/>

</comment><comment><USER11> I liked channel 9 lectures on functional programming: [http://channel9.msdn.com/Shows/Going+Deep/Lecture-Series- Eri...](http://channel9.msdn.com/Shows/Going+Deep/Lecture-Series-Erik-Meijer- Functional-Programming-Fundamentals-Chapter-1) It helped me to start programming in Haskell after I got stuck with a book.

</comment><comment><USER12> Probably not what you're after but the computer history museum videos are great: <http://www.youtube.com/user/ComputerHistory>

</comment><comment><USER13> googletechtalks on youtube.

<comment><USER14> This and infoq seem to have a lot of the most useful videos that are not just randomly out there (people posting a few videos they made themselves or the like).

</comment></comment><comment><USER15> I can always find something good to watch at yui theatre <http://developer.yahoo.com/yui/theater/>

</comment><comment><USER16> I love these high-quality videos on different modern databases.

<http://nosqltapes.com/>

</comment><comment><USER17> Chaos Communication Congress videos: <http://media.ccc.de/>

</comment><comment><USER18> here's another good place, altho more of culture rather than the technical stuff.

[http://nushackers.org/2011/03/movies-documentaries-and- video...](http://nushackers.org/2011/03/movies-documentaries-and-videos/)

</comment><comment><USER19> parleys.com has a decent amount of videos, focusing mainly on java.</comment>


The Increasing Trend of Online Extortion - <AUTHOR> http://www.troyhunt.com/2014/11/ransom-is-new-black-increasing-trend-of.html <comment><USER2> I saw Cryptolocker in action, and what surprised me about it was how professional it was. For those not familiar, it would come as an attachment in an email, and once run, would encrypt files locally and on any shared drives. Particularly useful against companies, with many unsuspecting users and lots of sensitive stuff to encrypt.

It left behind lots of text files giving you instructions and an address you can access via Tor. When you went to that address, there was a web app to allow you to upload an encrypted file to confirm if it was Cryptolocker. If you pay the ransom, they would send their "decrypter tool" that had the encryption key embedded in it.

The real criminal breakthrough, in my opinion, is that all of these utilities worked. When people are able to do some research and find out that if they pay up, they really will get their stuff back, often times they will pay up. It's certainly very disturbing.

<comment><USER3> I suspect it's because criminals don't do A/B testing on whether following through when their demands are met gets them better results.

<comment><USER4> Why would you suspect that? Just because criminals more often commit crimes that do not allow for easy A/B testing (or dont offer a worthwhile return on it) does not mean smart criminals wouldn't do such testing. If you want to get technical about it american banks have been doing A/B testing for a long time, and often are just as big of crooks as extortion virus creators.

</comment></comment></comment><comment><USER5> Due to the pseudo anonymitiy of cryptocurrencies, a whole range of perfect crimes is possible. Say you've got a some bitcoins (as little as 3 BTC makes you attractive, but let's say you've got 100BTC ) and you order a pizza with bitcoins from that wallet. The pizza guy fires up blockchain.info and notices that your address has a hefty balance, then picks up the phone and tips Oleg and Boris, giving them your home address.

Oleg and Boris mean business: they have a hammer, clippers, a soldering iron and an AK-47 in the trunk. Their target: your Bitcoins.

They wait patiently for your to exit your apartment or they follow you around or they just come knocking at your door. They are ready to patiently torture you until you transfer all your bitcoins to their address. They will enjoy the process, you .. not so much.

After you finally give in and transfer them your bitcoins, they leave and disappear. Now you've got absolutely nothing to show to the police: You cannot prove that those were your bitcoins and you cannot prove that the address you've transferred your balance to isn't yours. And you have no idea that it was the pizza place that tipped the bad guys.

For Oleg and Boris, it's the perfect crime: They just made $30k in 20 minutes and they didn't even have to kill anyone, a finger here, a finger there and they are rich!

This, in my opinion, is a great risk for cryptocurrency owners, because it offers potentially great returns for the bad guys and the risk is pretty small, plus it's very hard to prove that the theft/extorsion did happen.

<comment><USER6> Simple: keep multiple addresses (all clients are built for this) and diversify your balance among them. Keeping them all on a single address makes even less sense than keeping large savings in a checking account.

It shouldn't be very hard to design the UI to make it hard to fall into this trap, either. Easy way: when doing a small payment from an address with a large balance, the client can automatically add intermediate transfers to avoid the direct link.

<comment><USER5> Of course, everybody should do that. But since we live in an imperfect world, not everybody will do it. As you're saying, this should be implemented in the BTC clients themselves and they should not let you make large transfers without first warning of the dangers. But what if you're buying a boat or a car or a house ?

I've lived in the ex Soviet Union in the 90's and extorsion through torture was a common thing back then. But now we might see the 21st century version of those guys, probably a bit more gentle - no need to shoot or kill people, just force them to press Enter and you're done.

<comment><USER6> If you're buying a boat or a car, you'll have to do a big transfer. But how is that any different than buying something expensive using a bank account? It's not a new problem with cryptocurrencies.

<comment><USER7> Your bank wire doesn't leak the balance of your account to the recipient. It is a new and unique problem with _some_ cryptocurrencies. Of course, we have already proposed solutions to problem this with ring signatures, stealth addresses, 'coinjoin', 'zerocash', 'TITAN', etc.

<comment><USER6> As I wrote, you can simply move money around so that the final transfer doesn't reveal your original balance, e.g. if I have a balance on address A of 30 and I want to pay something that costs 4, I can just create multiple addresses where I move and subdivide the 30 until the last address only has a little over 4.

The recipient has no sure way of knowing that the original address was mine - I might just have withdrawn from a service like Coinbase.

</comment></comment></comment></comment></comment></comment><comment><USER8> I keep thinking these two things are related: 1) reliable identity 2) good security.

If you had a reliable identity - well implemented private key crypto or signatures, or perhaps just a fixed IP address - Communications protocols could be created that don't allow anonymous communication. You don't provide identity and they won't accept messages. The thing is, this would also allow private communications which neither corporations or governments want to happen.

So the internet will remain insecure so long as companies want to read your stuff for "ad targeting" and governments want to read your stuff to "stop crime". Got that last part?

<comment><USER9> There is no connection between CryptoLocker and ad targeting. What you're proposing would do very little to thwart tracking (imho strong identities would make it easier) and completely kill the open Internet where I can send and receive information from people I don't know, maybe even people I wouldn't want to know otherwise.

<comment><USER8> Just using public key crypto allows anyone to communicate securely with anyone else - including people you don't know. If you have end-to-end security, your communication can't be read and you can verify who a message or threat came from. It's all good with the exception of 3rd party listening. It's probably easier to see that data went from one person to another (which is no change) but that doesn't tell you what the data is (which is a big change).

I don't know why people don't seem to understand this. Or perhaps the lack of understanding is why it's not here.

</comment></comment></comment><comment><USER10> Intense, if not shocking.

It's the "Nigerian Prince wants to give you $100M" email scam scaled up another notch.

Where that was a pure numbers game, the criminal knows if they send out 1M emails, 1% will engage, and of that 1% they will get 1% to send money, that "scam" could easily be thwarted with a simple click of the delete button in you inbox or the increase of spam filtering to help the unsuspecting or unknowing not start the process at all.

This online extortion is certainly more aggressive and has an immediate effect on your life. Criminals are always looking for the next way to get ahead, I wonder where the "spam filter" to thwart this effort will come from and what they will move on to next.

</comment><comment><USER11> And this is where the thieves start getting creative. Backups, setting a PIN, using decent passwords--all of these steps prevent the low-hanging fruit of locking up someone's data (and can be installed by technically-savvy friends or family). The bigger question is how to protect your reputation, especially when directory entries can easily be modified on Google, Bing, and Yelp, complaints can be submitted to the health or licensing agencies online, and even cranking out some SEO can get "bad press" filtering near the top of search results.

<comment><USER12> These viruses can encrypt connected storage too, so your backups better not be all on a network drive or external hard drive. I believe they can encrypt Dropbox folders, not sure about Crashplan or similar services.

<comment><USER13> Crashplan retains old versions of files (it's a backup service) and you can choose to restore everything before a certain date, so you should be able to get everything back.

I just searched and they actually have a page showing how to recover from such an attack: [http://support.code42.com/CrashPlan/Latest/Troubleshooting/R...](http://support.code42.com/CrashPlan/Latest/Troubleshooting/Recovering_Files_Infected_By_CryptoLocker_Or_CryptoWall)

<comment><USER14> Doesn't dropbox also keep old versions?

<comment><USER13> Yes, but only for 30 days unless you pay more. If you don't realise there's been a problem quickly enough, you could be in trouble.

I'm not sure there's an easy way to restore everything from before certain date, either, though I'm sure it'd be possible to knock something up to do it.

</comment></comment></comment></comment></comment><comment><USER15> Crypto ransom is the premise for the novel Reamde by Neal Stephenson

</comment><comment><USER16> cryptocurrency encourages this type of behavior and makes online extortion possible.

</comment><comment><USER17> Extortion is a weird, upsetting crime. It seems to be a waste product or high- entropy end state in a stagnant economic system.

First, the powerful extort constantly. When VCs use their social connections to other VCs (the culture of co-funding) to pull the "we'll turn off the whole Valley" card, also known as "the reputation threat", and get people to sign bad term sheets, that's extortion. Most bad reference and negative reputation issues that exist in the Valley come from people who refused to be extorted. But when people in power do it, they don't call it "extortion". They call it "power".

Likewise, most people who acquire power did so by extortion. Not in the hold- up sense, but by happening into important information on powerful people and being able to leverage it into the investment of said powerful people in their careers. All that said, it takes a certain social skill to pull off. You can't just send an email saying, "I know <X> and will release it unless you provide <Y>, <Z>, and <W>." You'll piss that person off, and it's a felony, and even though you'll probably never do jail time (because the extortion target still doesn't want "X" to see daylight) that person now holds the cards. You have to be really subtle and it's best if the extortion threat is unsaid. One of the reasons why fraternity affiliations are so powerful is the implied mutual extortion that comes from living together for 3-4 years at a time of life in which people tend to be impulsive and do incredibly stupid things that their adult selves will regret.

If you're in the same frat, you're bound to have dirt on that person, and if that person becomes powerful, you'll cash in. If you say, "I'll reveal <X> unless you <Y>" you won't get anything, and you could end up in jail... but if it's a tacit agreement, that person will support your career in perpetuity. Perversely, the network effect of this tacit, mutual extortion is _positive_ for the group of people it covers because it spreads good fortune around.

Extortion seems to be a by-product of stagnation, because it's the ultimate zero-sum activity. It's what people start doing to each other when they give up on new contribution. Oddly, the frat culture (for all its ugliness) seems to be an adaptation to this because, while it creates a low-level tacit extortion field, it also prepares people to handle external extortions (from "the proles" who "have no right") and to exercise power ( _cough_ extort others).

What's strange about this rash of online extortions is that it seems to be coming at a time when, objectively, there _shouldn 't_ be a sense of economic stagnation. There are, arguably, more opportunities for positive-sum contribution than there ever were. But the social distance between capital and effort/talent has never been greater, especially on a global scale, so that might explain the problem.

<comment><USER18> I was in a fraternity. Did some people do some stupid shit, sure, it was college after all. But I have a hard time seeing how any crazy extortion schemes would come into play as you described. I don't know how I could extort a brother because I once saw him puke in the street or streak thru campus. For someone as prolific on HN as you, I'm surprised to see you come off as a conspiracy nut. Or, maybe me living in the midwest where everyone is nice and cooperative in general has given me a warped view of the world (albeit a positive view).

<comment><USER17> So, there's a wide spectrum of organization given the name "fraternity" and not all of them are bad. I'm talking about the abusive frats that (a) cover up disgusting and almost invariably illegal behavior by well-connected, privileged men, (b) disburse connections from the upper class into its next generation, and (c) are heavily responsible for the injection of bro types into the VC-funded founder ranks.

There's a wide spectrum of "fraternity". Just look at initiations. There are some where initiation is being driven 5 miles off-campus and having to find your way back, and others where it involves bodily fluids, physical abuse, and dangerous levels of alcohol.

When people complain about "frat boys" or "bros" infesting the Valley, we're not talking about guys who like to drink occasionally or streaked a football game, because none of that's a big deal. No one has a problem with an occasional game of beer pong. We're talking about entitled, well-connected people (e.g. the Spiegels and Duplans of the world) who've lived for decades in a world where there are absolutely no consequences for their actions.</comment></comment></comment>


Benchmarking C++ Sets - <AUTHOR> https://constantsmatter.com/posts/cpp-benchmark-container-lookup/ <comment><USER2> Always measure!

In a realistic application (with cache pressure) you may find an array performs even better because the CPU can trivially predict and pipeline the entire operation vs jumping around to random locations in memory.

Another approach is a B-tree with lots of nodes stored at the leaves. This can get you the array's linear performance for small datasets, while preserving other big-O benefits of sets/trees (eg insertion time).

As the side of your dataset grows, there is a performance cliff at L1, L2, and L3 cache size. For very large datasets, creating data structures with predictable access patterns often trumps Big-O entirely. If following a linked list thrashes the cache it turns out the constant factor can and does dominate.

So again, always measure with your application on your intended hardware. Be sure you measure with realistic loads and access patterns.

<comment><USER3> Vector is supposed to essentially optimize to direct array, except on insert or append when it may resize.

Std::Array is expected to have almost identical performance characteristics.

</comment></comment><comment><USER4> These results don't make any sense and I don't believe them.

The author is generating random strings of length 64. They then generate N random string and search for them. The probability of finding a match is effectively zero.

> Interestingly, up to about 75 elements, the vector actually outperforms both > sets.

No way. The unordered_set will perform one hash, one lookup, and return false. That is not going to be slower than SEVENTY FIVE string comparisons.

Looking at the code I _think_ that randString() for each of the N search strings is included in the timing code. That, uh, is not good? Line 29: [https://github.com/akshaynanavati/benchmarks/blob/master/ben...](https://github.com/akshaynanavati/benchmarks/blob/master/benchmarks/lookup_string.cpp)

I don't know how Google Benchmark works. So I wrote my own simple test. I could not find any combination of string length and num searches where a vector was faster than an unordered_set.

[https://pastebin.com/raw/mfBmGa5B](https://pastebin.com/raw/mfBmGa5B)

Example results:

String length: 8 Num Elements: 75 Num Searches: 10000 Gen Time: 1.94252 Vec Time: 4.12721 Set Time: 0.144987 Num Found: 0

<comment><USER3> Not it won't for this small strings. A 64 byte string likely fits in a cache line so you're doing a cache line comparison which may even have a microop in the CPU. Alternatively using SSE or AVX to vector compare the strings which is still really fast.

I suspect the author made critical mistakes too.

If you are actually hitting a match or do many operations with warmed cache, vector may actually win big time even compared to unordered_set as there may be no pointer chasing involved. (since both all keys and all values are in cache) The vector scan essentially boils down to a memcmp for a sufficiently smart compiler. Computing the hash may or may not be faster than this.

<comment><USER4> std::vector<std::string> is a pointer-chase cache-miss every compare.

In my testing unordered_map is faster so long as numberOfStrings > stringLength. This true for any size string greater than roughly 4. As stringLength grows the unordered_set advantage grows.

Original post says vector is faster until 75 elements. By my test the unordered_set is 4x faster with 64 char strings/75 elements. At 250 numberOfStrings its ~10x faster.

Sorted vectors + binary search are very very good though. Author said vector faster until 20,000 elements. My testing finds the numberOfStrings > stringLength limit is still basically true. But the unordered_set's advantage grows only to 1.9x by 20,000 numberOfStrings.

This advantage will grow as number of matches / near-matches grows. Random strings are not a great test case. Data sets matter!

Author is fixing a couple of bugs and will be updating their post. Curious to see if his updated results match mine. :)

<comment><USER5> _"std::vector <std::string> is a pointer-chase cache-miss every compare"_

Nitpick: that is (almost certainly) true for strings of length 64 on any system, but not necessarily so.

On existing systems, it isn't true for short strings (with 'short' being implementation dependent), due to "short string optimization" (See [https://stackoverflow.com/a/10319672](https://stackoverflow.com/a/10319672), [https://stackoverflow.com/a/28003328](https://stackoverflow.com/a/28003328)).

There's nothing in the standard that forbids putting extra room in the base object to accommodate larger strings (but of course, it would surprise many users, possibly lead to stack overflows in lots of code, etc)

</comment></comment></comment></comment><comment><USER6> Somewhat interestingly, C++ STL unordered_map is so much slower than it needs to be: [https://www.youtube.com/watch?v=fHNmRkzxHWs&t=47m13s](https://www.youtube.com/watch?v=fHNmRkzxHWs&t=47m13s)

Using one of the policy-based data structures instantly gets you about an order magnitude of speedup (for accesses).

<comment><USER7> Kinda makes you wonder why those decisions (to provide bucket iteration, bucket iterators) were made. It's hard to picture a use for such a feature that makes it worthy of adding constraints to a standard container. I would love it if someone could provide some insight into the decision process that led to this state of affairs.

<comment><USER8> IIRC boost::unordered_map/set offered those, and I'm pretty sure the API was taken more or less verbatim from that. I suspect some desire of compatibility was to blame, but don't know.

Hopefully in the future mistakes of these sort will be avoided, but that might be asking for too much (certainly std::optional opted to break compat with boost's (`std::optional<T&>` is disallowed) so there's some hope).

</comment></comment></comment><comment><USER9> I have a lot of issues with this blog post:

\- Compiler, c++ library, compilation options, and platform are missing

\- The author doesn't benchmark vector and unordered_map insertions with reserve()

\- If I read correctly, the author benchmarks up to 32,768 items. This is way too low. There should be tests with millions of entries.

\- Random string/numbers isn't a realistic scenario and very favourable to unordered_map. I would use data from real life examples, e.g. list of cities, list of people, etc.

\- Ordered arrays are missing from the benchmark (e.g. boost::container::flat_set), it's a very powerful data structure. Why just do a sorted vector at the end?

\- Vector outperforms unordered_set probably because of the hashing function used.

<comment><USER10> Even if they had did all that, some person on the internet would have had "a lot of issues with the post" in order to show off how smart they are.

You have to draw the line somewhere, and this post is very interesting as it is.

<comment><USER11> The original article was updated as the results were completely incorrect. The only time a linear search is faster is for a very tiny in-place integer search.

</comment></comment></comment><comment><USER12> It is very rare that I have seen a use of std::set that seemed justified, although it does happen. People just use it because it's easier to type than unordered_set!

One way to speed up set's usually appalling performance is to give it a custom allocator so that instead of N calls to allocate tiny tree nodes all over the heap, you get all your tree nodes on an arena. This can dramatically speed up short-lived sets (for example you are trying to build the unique set of some non-unique input).

</comment><comment><USER13> > In one setup, I used std::strings of length 64 and in the other I used 64 > bit numbers. The main motivation behind measuring both numbers and strings > was that comparing 64 bit numbers takes a constant number of clock cycles

Pedantic: comparing 64 character strings is O(1).

<comment><USER14> But not a constant number of clock cycles. A comparison that fails on the first bytes can bail out early.

<comment><USER15> To be fair, Intel's recent offerings include 512-bit registers [1].

I highly doubt std::string takes advantage of them.

[1] [https://en.wikipedia.org/wiki/AVX-512](https://en.wikipedia.org/wiki/AVX-512)

<comment><USER16> It shouldn't because it slows down the whole core not to get too hot. 256 is the best size to use.

</comment></comment></comment></comment><comment><USER17> For longish strings (length 64 qualifies), binary_search is missing an important optimization: it ends up walking the same prefix over and over.

A better algorithm is called "radix quicksort with a three-way partition." Rather than the classical qsort < and >= partition, you maintain three regions of <, =, and >. Crucially, at each step, you sort based only on a single character: the whole range via the first char, then the = region via the second char, then that = region via the third char, etc. This allows for super-fast comparisons without re-walking prefixes.

All of these data structures (std::binary_search, std::set, and and set::unordered_set) are quite sub-optimal. The design of unordered_set with closed-hashing separately-allocated buckets is especially distressing.

For a much more efficient take, see LLVM's DenseSet which does NOT separately allocate buckets. This burdens the client with finding two values to represent empty and tombstone entries, but if perf is a concern that's much better than additional mallocs.

</comment><comment><USER18> It seems like many commenters here (the ones who aren't questioning the methodology) are drawing the lesson that you should use vector rather than set.

From these charts, isn't unordered_set clearly the best all-rounder? Which is exactly as I'd expect. Am I missing something?

Sure, a simple vector might be a bit faster for 10 or 100 or 1000 elements, but after a certain point performance is awful.

</comment><comment><USER19> definitely check out flat_set too! most of the time using a sorted vector as the backing store for these kinds of things ends up being a net win, especially for smaller sets, but the size it starts to become a hindrance may surprise you.

<comment><USER20> Flat sets are great.

By writing my own implementation (to avoid boost dependency) I kinda understood why they aren't standard: iterators have different invalidation rules and some special sorted-vector-derived useful properties, the map/multimap keys can't be const as in other standard associative containers, etc.

</comment></comment><comment><USER21> Very interesting. This makes it sound like there should be a way to control `std::lower_bound` to switch to linear probing once it's narrowed the range down enough. (In the specific example, "enough" is about 125 elements.)

Likewise for a hash table with separate chaining, wouldn't one better-than `std::unordered_set` way to do it be to keep the separate chains as a `std::vector` that is kept sorted, and then use `std::lower_bound` (or the above better version of it) to probe the table? Those separate tables are O(1) in size, so lookup and insertion are still O(1) on average. Worst case goes to O(n log n), but that doesn't seem too bad.

Related: Is there a widely-used "frozen set" C++ data structure? I'm picturing something that is constructed from a sequence but that internally is a sorted `std::vector<T>` that provides a set-like API but that uses `std::lower_bound` to find data. It's easy enough to write, but it seems like it should be in the standard or at least Boost.

<comment><USER22> There is one in boost: [https://www.boost.org/doc/libs/1_68_0/doc/html/container/non...](https://www.boost.org/doc/libs/1_68_0/doc/html/container/non_standard_containers.html#container.non_standard_containers.flat_xxx)

<comment><USER21> Nice! [https://www.boost.org/doc/libs/1_50_0/doc/html/boost/contain...](https://www.boost.org/doc/libs/1_50_0/doc/html/boost/container/flat_set.html) and [https://www.boost.org/doc/libs/1_65_1/doc/html/boost/contain...](https://www.boost.org/doc/libs/1_65_1/doc/html/boost/container/flat_map.html) for reference.

<comment><USER21> Interestingly, when I dig deep into `find`, I wind up in `priv_lower_bound` which bisects to the bitter end rather than doing a linear find at the end. [https://www.boost.org/doc/libs/1_50_0/boost/container/detail...](https://www.boost.org/doc/libs/1_50_0/boost/container/detail/flat_tree.hpp)

</comment></comment></comment></comment><comment><USER23> The use of unsorted vectors for sets was covered and benchmarked in a Scott Meyers [0] post from 2015. Facebook's folly library supports linear containers and maps for these small sets (~100 elements, as a rule of thumb, but measure), and I've rolled my own for specific applications.

[0] [https://scottmeyers.blogspot.com/2015/09/should-you-be- using...](https://scottmeyers.blogspot.com/2015/09/should-you-be-using- something-instead.html)

</comment><comment><USER24> The article should say which C++ compiler and standard library they're using. Each implementation is different, and these results may not be true for all of them.

</comment><comment><USER10> Great! I'd love to know what percentage of the world's std::set could be replaced by vectors, and how many megawatts would be saved.

</comment><comment><USER25> This is completely useless if you don't mention what STL library version you're using.

C++ sets define the interfaces and not the implementation.

</comment><comment><USER26> It's actually not hard to beat the performance of standard set and map containers in specific scenarios. For example in one application I have a treap implementation that's much faster than the standard ordered map, but it is much less featureful (doesn't support deletion for example).

</comment><comment><USER27> Anyone compared this to sparsehash recently?

[https://github.com/sparsehash/sparsehash](https://github.com/sparsehash/sparsehash)

I read a while ago that it beat all the competitors at almost every size.

</comment><comment><USER11> Are these benchmarks done with cold caches and multithreading? If not then we are not seeing the whole picture here.

<comment><USER28> They use the Google Benchmark library. (I would read into it to answer your actual questions)

<comment><USER11> Also looks like randString generation will dominate for small arrays for (size_t i = 0; i < kLookups; ++i) { benchmark::DoNotOptimize(std::find(vec.begin(), vec.end(), randString())); }

Also does that allocate a new string every call?

<comment><USER3> Yes, this benchmark is relatively flawed.

Allocating a new string simulates heap fragmentation.

<comment><USER11> The post was updated as the results were incorrect.

</comment></comment></comment></comment></comment><comment><USER29> In the first table:

        set @ 4096: 10.80
        set @ 32768: 1.31

uh?

</comment><comment><USER30> Comparing lookup time of unsorted vector and sorted set is unfair. The vector should be sorted and then we might compare plain binary search against BST performance.

<comment><USER31> It is not unfair, and there is a comparison with A sorted vector as well. And the author correctly indicated the reason as of why the sorted vector is performing better than the unsorted vector. The cache friendliness is key. And it shows that for small sizes of elements an array or vector would perform better. It simply depends on your needs.

I would be interested in comparing an stack based array with integers to the sets as well.

<comment><USER30> My fault, I just skimmed through the article and missed that comparison.</comment></comment></comment>


OpenShot 2.5.0: Video Editing and Hardware Acceleration - <AUTHOR> https://www.openshot.org/blog/2020/02/08/openshot-250-released-video-editing-hardware-acceleration/ <comment><USER2> I used open shot while on vacation recently and it was a pleasant experience. At the very least it was among the better open source productivity apps that I tried recently.

By this I mean it has a complete installer for my os, it installed without a hitch, the UI is "normal" and intuitive enough, it handled my dpi just fine, and the functionality is solid.

The only issue is that it was a bit sluggish at times.

Got my video done and will surely drop a donation if I adopt it as my go to video editor.

</comment><comment><USER3> I used open shot quite a bit years ago for YouTube content creation. It is approximately a million times easier to use than pretty much any other video editor but back then it was slow and crashy to the point of frustration. I just gave 2.5.0 a try and sometimes it was a little slow and stuttery at times but I managed to edit a fairly simple video with it.

The GPU acceleration feature is pretty nice. On my dated machine, encoding a 1080p MP4 on an integrated Intel video chip is a little more than twice as fast as hammering all 4 of my CPU cores. (Effectively rendering in real time.)

</comment><comment><USER4> Related from 2019: [https://news.ycombinator.com/item?id=19725823](https://news.ycombinator.com/item?id=19725823)

</comment><comment><USER5> How well does it work on Windows? Every time I've tried it there, it was crashy and I wound up going to kdenlive.</comment>


How The Apple Tablet Could Ruin Computing (2009) - <AUTHOR> http://www.popsci.com/gear-amp-gadgets/article/2009-08/how-apple-tablet-could-ruin-computing

<comment><USER2> Link is broken for me, but here it is from Google's cache:

<http://goo.gl/OVPCV></comment>


Populism (03/2017) – by Ray Dalio - <AUTHOR> https://www.scribd.com/document/342835546/Populism <comment><AUTHOR> First 5 pages are gold. This part trips me out on how similar the Trump's administration and Trump himself are doing. It's like their playbook one by one.

"And while their political ideologies vary, the 1930s populists shared most of these core beliefs and policy goals:

o They aligned themselves with "the people" or "the common man." o They were anti-establishment and attacked the current ruling interests (government, corporations, wealthy individuals, etc.), calling them elites who were out of touch and had failed the people. o They sought to undermine those elites in favor of others by, for example, advocating wealth redistribution or the nationalization of industry. o They were strongly nationalist and held national unity as a key aim. o They detested the debate and disagreement inherent in democracy, and sought to empower the executive branch, using strong-arm tactics to prevent others from getting in their way and, in more extreme cases, undermining democracy. o They tended to be anti-international, anti-global trade, and anti-immigrant. They often railed against foreign influence in their countries. This often translated to hostility toward other countries, which pushed those countries to embrace political extremes as well.

• Conflicts between factions became increasingly intense, leading to great obstructionism, crackdowns on opposition and free media, etc. This led to more autocratic leadership. Those that had the weakest norms/shortest history of democratic institutions were quickest to move away from democracy to dictatorship"</comment>


Dropping out: Those who found success without a college degree - <AUTHOR> http://www.businessinsider.com/john-carney-ten-people-who-make-dropping-out-of-college-look-weirdly-smart-2009-9

<comment><USER2> I completely agree that having or nor having college degree does not necessarily indicate chances for success. As article indicated we have many successful drop outs but how many drop outs are actually become successful, probably less than 0.0001%. I believe in power of dedication and desire. If you are motivated and know what you want you will learn and pick up knowledge much faster and in more efficient way then any college can teach you and will succeed. As the meter of the fact, most successful drop outs quit school because it was not build for them and held them from growing quick. They are fast paced and proved it. However, for average person who does not have extraordinary ambitions, college might be a good place to learn how to learn, to learn how to organize thought, to figure out what they need and meet good people. Nevertheless, completely support college degree but it should not be an indicator of success when preceded by public, by employer by business, however you have to have some way to measure you potential success if all you have is resume consisted of one page and few facts.</comment>


History of networks – Niall Ferguson - <AUTHOR> https://www.theguardian.com/books/2017/oct/12/the-square-and-the-tower-by-niall-ferguson-review <comment><AUTHOR> Here is also a link to a great talk on the thesis that networks are the important thing to study in history.

[https://www.youtube.com/watch?v=4cADSlk5CHU](https://www.youtube.com/watch?v=4cADSlk5CHU)

<comment><USER2> Could you recommend something perhaps an hour or so briefer?

<comment><AUTHOR> Conference talk:

"Networks vs. Hierarchies: A Delicate Relationship"

[https://www.youtube.com/watch?v=Mcy_E-231LE](https://www.youtube.com/watch?v=Mcy_E-231LE)</comment></comment></comment>


Killer Reasons to Join A Startup (And Debunking Myths on Why You Shouldn't) - <AUTHOR> http://bostinnovation.com/2011/04/11/10-killer-reasons-to-join-a-startup-and-debunking-3-myths-on-why-you-shouldn%E2%80%99t/

<comment><USER2> The article misses another startup benefit: flexibility. As a single mom raising a special needs kid, I'm often looked at as _crazy_ for getting involved with start-ups. People assume I'm really rolling the dice with regard to my child's care.

The reality is just the opposite: I like working with start-ups because BigCorp can't really handle anything different than the norm. StartupInc on the other hand, in order to recruit me (granted, for a lower base salary than BigCorp), will happily agree to my needing to leave early on Thursdays (time to be made up remotely on Thursday night) for supplemental speech therapy, or work remotely on days when my son is home sick, and so on.

<comment><USER3> You might find things have changed at a lot of big companies since you last looked. Many of my friends who are parents get a lot of flexibility with their jobs due to the workplace diversity programmes that urge managers to cater for people in different situations.

<comment><USER4> I had to turn down a job in the last 6 months because BigCorp couldn't allow me to take unpaid leave for the summer to take care of my son while he was out of school. So maybe not as much as you think.

</comment></comment></comment><comment><USER5> I'm not sure I'd laud startups for 'career security'. Outside the valley echo- chamber, few companies have any respect for startup experience. ('startup' sounds a lot like 'small company' if you aren't a hacker...)

Also, for every startup where young employees get a chance to punch above their weight, there's another where the devs grind all day long on customer support issues and maintenance.

On the whole I'm personally more inclined toward startups than large companies, but I've been around long enough to be wary of mythologizing them. They have their own set of dysfunctions that can be as stressful as anything you'll find in a bigcorp.

<comment><USER6> Alternatively, working at a startup may not have the same cache when not in the Valley, but you might get a chance to work with some awesome tools that are ahead of the curve that would get you experience for a future job.

<comment><USER3> I think it totally depends on what you do there. I'd probably hire someone who has the full gamut of dev, sysadmin and customer interaction experience from a startup. Less so if they couldn't effectively describe how they'd helped get the business running.

> working at a startup might not have the same cache ...

By the way, I think you mean 'cachet': <http://en.wiktionary.org/wiki/cachet>

</comment></comment></comment><comment><USER7> I really wish we'd reframe this perennial question as "how do I detect crap startups?" That seems the biggest question for my friends in Bigco: they're afraid of finding out in a year that an enterprise was never going anywhere.

<comment><USER8> This one is somewhat straightforward, though:

Can I describe to my mom, in one sentence, what it is they do?

What proportion of the company is people who do real, useful shit as opposed to "leadership" or other management? It's tiny, right?

Are they making something that can be genuinely useful or enjoyable for people?

Are they funded by a secretive billionaire or have you heard of their investors?

There are others, but this seems a good start. One or two no answers might not disqualify a company but no to all four would leave me deeply suspicious.

<comment><USER7> Yes those questions are a great start. But it takes some expertise to interpret the answers. If you're new to the space all investors are new to you. You get random hangers-on claiming to have invested in skype. 'useful for people' is also hard to check. A company's product may be useful to a niche you have no experience with. 'Describe to my mom' seems to bias people towards consumer startups - it's hard to explain companies that occupy a niche in a specialized eco-system. But many of those are very valuable.

In the end it's an arms race. Crap companies are out there improving their posturing everyday. Bullshit coevolves with bullshit detectors. Which is why this should be _the_ perennial question to discuss every few months. "Ask HN: any crap companies out there?" "Ask HN: anybody fed you a new line of bullshit?" "Ask HN: new questions to ask startups?"

(I don't attribute malice to 'crap companies'. A company is crap if it mistakes the map for the territory, if it focuses on its 'positioning' without changing its product. Other such levels of incompetence.)

</comment></comment></comment><comment><USER9> Joining a startup is tricky business. The nature of a startup is so dynamic that what you joined a startup for may not be what you end up doing at a startup. So if you joined a startup for something you felt passion of, particular tech or idea. Then be wary as this may change and you might find your self working on something totally different, especially if the control is not in your hands. More importantly make sure you have worked with people or a person you are about to start a startup with because if trust/chemistry is not there then nothing of much worth can be achieved.

<comment><USER10> Very true. Tons of change in a startup always and as a startup grows, it often requires different approaches to working as well as different types of work and, therefore, different people.

</comment></comment><comment><USER11> So very first sentence of Myth #1:

'Employees in a startup have a chance to make money when the company exits, i.e., gets acquired or goes public'

This attitude that a purchase or IPO is the only possible way for a startup to earn money for its employees would be laughable anywhere outside computerland.

<comment><USER10> Good point. Cash flow is king. I was trying to keep it simple. The fact that nowadays it's easier than ever to build a profitable small company in high- tech is another reason to go to startups.

</comment></comment><comment><USER12> Spot on points here by Sim regarding the fact there is never a bad time to join a good startup and never a good time to join a bad one.</comment>


Ask HN: Are we using too much internet? (Bell & Rogers seem to think so) - <AUTHOR>

After reading this short article, the last paragraph really stood out to me:
"The CRTC has agreed with arguments from large ISPs such as Bell and Rogers Communications that usage-based billing is a way to encourage heavy users to reduce their usage. The major internet providers, including cable giant Rogers, had argued that it is needed to deal with booming online traffic and increased network congestion as people do more online — including downloading music and watching movies and television programs."
http://bit.ly/i9R1NP
So are we (Canadians) using too much internet? Or have Bell and Rogers just found another way to make us pay even more for for an essential service? (well at least I'd consider internet "essential") <comment><USER2> I'm from Canada too and I simply think they are trying to take another buck from people's pocket.

Unfortunately for some of us with or without the regulation the situation is already "regulated", personally I pay about 65$ a month for 100GB of combined upload/download.

Bell, Videotron and Rogers already have limits for the Internet usage, what they are trying to do now is to force the small providers to impose limitations, basically they will kill the concurrence.

You will see a similar situation in the mobile phone offer ... huge prices versus Europe for e.g.</comment>


Microsoft: The Open Source Company - <AUTHOR> https://arstechnica.com/gadgets/2019/05/microsoft-the-open-source-company/ <comment><USER2> Things I'd like to see from Microsoft:

\- Adopt OpenDocumentFormat whole-heartedly

\- Stop suing Android OEMs for software patents

\- Support Mozilla and/or Canonical in a serious way</comment>


Decrypting the Encryption Debate: A Framework for Decision Makers - <AUTHOR> https://www.nap.edu/catalog/25010/decrypting-the-encryption-debate-a-framework-for-decision-makers <comment><USER2> We at EFF didn't appreciate this report that much when it came out last month: [https://www.eff.org/deeplinks/2018/02/new-national- academy-s...](https://www.eff.org/deeplinks/2018/02/new-national-academy- sciences-report-encryption-asks-wrong-questions)

<comment><USER3> That sounds like a good endorsement. Those of use who are competent _and_ honest have been pretty tired of the FUD and downright lies around the benefits of single-user encryption.

Specifically, if admin master keys are so dangerous, why does Google, Microsoft, and every other tech company have one into the employee's hard drives?

<comment><USER4> Because the problem corporations face is pretty much the reverse of the problem at issue here. In a corporation, the only entity being protected is the corporation itself. Master keys exist to provide access to all of a corp's devices, even if employees forget their passcodes etc. The entity controlling the master keys is the entity being protected, and while having those keys compromised would not be great, the loss of data from not having master keys is a much bigger threat.

OTOH, for citizens, a master key with access to their private data is a problem. The data owner is not the owner of the master key in this context and therefore the very existence of such a key is much more problematic even if the key is never leaked to unauthorized parties.

</comment></comment></comment><comment><AUTHOR> OP here,

I ran across this from the Cyberlaw Podcast notes[0]. Their brief description of it is why I thought it'd be interesting for the HN crowd:

>The National Academy report on encryption access has demonstrated that access is well within the zone of plausible technology policy, with support from a group of prominent tech experts, such as Ray Ozzie, all of whom know math.

I certainly don't agree with that statement, but I'm a long way from a cryptographer or security expert. I'm mostly looking forward to the EFF refutation, though I'm a little surprised it isn't already up...

[0] [http://reason.com/volokh/2018/04/02/keeper-loser- weeper](http://reason.com/volokh/2018/04/02/keeper-loser-weeper)

<comment><USER5> It's possible to design a system that lets the government decrypt (essentially) everything. The same technology that lets you send a PGP e-mail with two recipients allows you to send a message that can be decrypted by the NSA as well as your recipient.

What's not possible is protecting against bad actors within government; preventing the oppressive governments from insisting on the same powers, allowing them to further oppress their citizens; and eliminating the backdoor- free encryption that evildoers will instantly move over to, which has to be eliminated if trained terrorists are the true targets of the back doors.

But it's possible legislators don't care about these issues as much as freedom advocates do.

<comment><USER6> > It's possible to design a system that lets the government decrypt > (essentially) everything. The same technology that lets you send a PGP > e-mail with two recipients allows you to send a message that can be > decrypted by the NSA as well as your recipient.

It's not that simple. That "solution" has the government key as a single point of failure: when it leaks, everything is compromised.

<comment><USER5> So put it in a Hardware Security Module [1] and store it in Fort Knox.

Competent certificate authorities manage to store their private keys and not leak them. And the security community loves SSL, even though it depends on that.

[1] [https://en.wikipedia.org/wiki/Hardware_security_module](https://en.wikipedia.org/wiki/Hardware_security_module)

</comment></comment></comment></comment><comment><USER7> Almost 30 years later, it sounds like they're still arguing for the Clipper Chip in some manner:

[https://en.m.wikipedia.org/wiki/Clipper_chip](https://en.m.wikipedia.org/wiki/Clipper_chip)

</comment><comment><USER8> Even when the whole San Bernardino thing happened, the FBI could already unlock the device. But they kept quiet so they can get their legal precedent.

[https://assets.documentcloud.org/documents/4424780/Fbi- Iphon...](https://assets.documentcloud.org/documents/4424780/Fbi-Iphone- IG.pdf)

Also, let's assume this "solution" gets implemented into every smartphone out there. And then we discover that it actually doesn't work as well as theorized, and it can now be abused by hackers.

What then? Will the FBI agree to put that genie back in the bottle and have all OEMs remove support for this solution? Or will they go whine to the media and Congress that this solution can't be removed because "it's been so helpful in catching criminals and that odd kidnapper out of 1,000 cases", as they usually do when defending surveillance powers they've already had for a decade, whether illegally or legally, and then someone tries to rein in on those powers?

Criminals are not "going dark". If anything, there's been an Enlightenment Age of Surveillance over the past two decades. They have more data and more tools than ever (including the recent FISA law that allows the FBI to see all raw data going through ISP cables). And perhaps that's the problem, and why they keep missing so many cases. Too much noise that's captured.

It doesn't look like the FBI and the US government will ever be satisfied with anything other than complete surveillance of everything US citizens are doing - which is exactly what China is implementing right now.

[https://www.rfa.org/english/news/china/surveillance-03302018...](https://www.rfa.org/english/news/china/surveillance-03302018111415.html)

(By the way, in relation to that article, the FBI is already seeking access to whatever Alexa and other such devices are recording, which could be increasingly more data as the tech improves).

Does anyone here actually believe that this is _not_ actually FBI's goal? Or do they think that if this solution will be implemented, they won't find some other thing to complain about? Maybe open source encryption?

<comment><USER9> > _If anything, there 's been an Enlightenment Age of Surveillance over the > past two decades._

For me, a watershed moment was the suppression of the Green Revolution in Iran. Organizing on social media got the world to notice, but it also provided the Iranian government with an indelible audit trail leading back to the dissidents.

</comment></comment><comment><USER10> If "decision makers" don't already respect the framework of personal liberty and individual rights, then they have no business being "decision makers" in the first place.

</comment><comment><USER11> Edit: nevermind, thought it was paywalled as the CTA is to pay, but there's a second link on the side to download the PDF.

<comment><USER12> I could download it just by providing an email address, no payment was required.</comment></comment>