Commento: Open-Source Disqus Alternative - August-Garcia https://www.256kilobytes.com/content/show/4957/what-is-commento-the-open-source-disqus-alternative <comment><AUTHOR> [https://news.ycombinator.com/item?id=19210697](https://news.ycombinator.com/item?id=19210697)

This is the second time this account has submitted an article about a project that had major attention on HN a short time earlier. If this is with the idea of promoting your blog here, that is not the way. What we want is interesting content that _hasn 't_ been seen recently or (better still) ever.

<comment><USER2> To be fair, the person who posted this instance is not the author of the library/old article you linked. I was wondering why this suddenly disappeared from the front page. I guess it's possible they are doubling dipping with accounts? If not, they can't really help if someone else liked it and shared again.

Anywho... looks fantastic, I'm certainly interested in trying this out! Speaks to the interest this project is gaining that it rose to the top twice! As someone who missed the post a few weeks ago, I am very happy to have caught this one.

<comment><AUTHOR> Submissions can count as dupes when they cover the same story. The story here is substantially the same as 19210697, and the pattern I was trying to pointing out is visible at [https://news.ycombinator.com/submitted?id=August- Garcia](https://news.ycombinator.com/submitted?id=August-Garcia). (It has nothing to do with the creators of this project or the other one, but yes there has been more than one form of double dipping.)

I agree that it is a cool project! Marking this post a dupe isn't to deny that. It's just that front page real estate is the scarcest resource on HN. More explanation here: [https://news.ycombinator.com/item?id=19103247](https://news.ycombinator.com/item?id=19103247).

Not seeing all the cool things that show up on HN is a problem all of us have. One thing we made to try to mitigate this is [https://news.ycombinator.com/front](https://news.ycombinator.com/front), which you can get to by clicking 'past' in the top bar.

<comment><USER2> Okay, I see what you're saying. Yeah, your updated response is more clear. I was confused before, but you are actually addressing August-Garcia. For sure seems like they are trying to capitalize on popular posts, and driving traffic to 256k website is clearly the goal. I'm definitely not a fan of this behavior & have ranted about similar activities in the past... lol

Also, props to the 'past' feature. Great idea! I'm a big fan. Thanks for all the hard work!

<comment><AUTHOR> You're welcome, and thanks for the kind words.

</comment></comment></comment></comment></comment><comment><USER3> Lol it's so surreal seeing your project when you're randomly browsing the frontpage. Anyway, creator here, AMA!

>adtac has succeeded in reducing download to just 11 kB and isn't done yet

Someone raised an issue to support Brotli in Commento (which I wasn't aware of) [1] and I did a quick test; it'd decrease the JS+CSS payload even further to under 8-9 KB. Nearly all modern browsers support Brotli [2], so I'm really looking forward to this.

I now realise this could be a modern take on the famous C10K problem, just with a different meaning: developing web services with under 10 kilobytes of payload :)

[1] [https://gitlab.com/commento/commento/issues/125](https://gitlab.com/commento/commento/issues/125)

[2] [https://en.wikipedia.org/wiki/Brotli#Browser_support](https://en.wikipedia.org/wiki/Brotli#Browser_support)

<comment><USER4> The brotli experience has been frustrating.

All modern browsers support it, so you'd think it would be an easy win. The top nginx search is an official looking Google project which is long abandoned and doesn't support current versions of the library. There's an official nginx module which is commercial, and then finally there's a supported fork that that sits on about page three of Google.

Shared hosts don't generally support it. AWS Cloudfront doesn't support it, along with several other CDNs. Big + to Cloudflare for proper support.

It reduced my bundle sizes measurably. We all hate bloat and want to reduce bundles, but there seems to be a community inertia around mass deployment.

<comment><USER5> Anyone know why brotli needs to be generated at the web server level? Just curious why/if it can't be done at the web app level.

<comment><USER6> Its generally used as a data transfer compression system. So the web server needs to compress the data before its sent to the browser. If you did it at the web app level (client/browser side), the data would already be transferred so there are no size savings.

<comment><USER7> Many web servers will serve file.ext.(gz|br) as a pre-compressed option. In order for brotli to beat gzip on size, you need more CPU time, so it's best if precompressed. This also means it's not really an option for dynamic content.

</comment></comment></comment></comment></comment><comment><USER8> My experience with comments on my site has not been great. My general sense is that, beyond the occasional "thank you", which I appreciate but which is not very actionable, the best discourse happened by email anyway, and comments were mostly by people who wanted me to solve their problems for them.

I made my email address even more prominent and disabled comments on my site after realizing it would be a net improvement.

<comment><USER9> On this note, what are some examples of comments providing considerable value?

I can think of: comments in torrent trackers, where comments sometimes provide useful information; and, if this even counts, forum sites, like Reddit or HN, where comments host discussions.

<comment><USER10> For blog-posts which contains technical how-to information, the comments often contain corrections or other additional information which may be relevant for later releases or other distros.

I've certainly found that useful more than once.

<comment><USER8> Yes, this is one example of useful comments. It's just that, for my site, those tended to be over email, which I generally respond to immediately, so it's equivalently useful.

Comment sections are especially good for sites which the author doesn't update, so you can read the correction in the comment.

</comment></comment></comment></comment><comment><USER11> Kudos! And nice to see you're using GitLab as well.

Not to get off topic but my theory is, something like this is one of the reason why Automattic's / WordPress' recent Gutenberg efforts were misguided.

That is, it's not only about the content (of the individual node sites / blogs) but (in a networked / graph world) also the importance of the connections between the nodes, as well as the participation of third-parties (i.e., those leaving comments).

Long to short, commenting and connecting could result in an (informal social) network (of sorts). With WP powering ~30% of the web, laying the ground work for such a graph could have been revolutionary. Content __and__ connections.

Instead, they went with Gutenberg (and imho missed a massive opportunity to move away from the dated silo-based content model).

p.s. If Commento can (easily) work with the current copy of static site generators, what would be worth mentioning somewhere in your marketing / feature / benefits copy.

Also, since Disqus is a pseudo social network, how would Commento work if the same instance were used across multiple sites / blogs? In such a situation, could Commento be used as a proxy-network among the sites that share a given instance?

<comment><USER8> Actually, it would be great if comments had ActivityPub integration. Then you could comment on articles from your Mastodon account.

</comment></comment><comment><USER12> I've summarized many ways to include comments on a static site in my blog post [1]. It's great to see how many alternatives are around.

[1] [https://darekkay.com/blog/static-site- comments/](https://darekkay.com/blog/static-site-comments/)

</comment><comment><USER13> I see on Commento.io it explicitly mentions being able to import from Disqus comments, but the "Import from a different service" is behind a signup wall.

Rather than signing up for a service that may or may not work for my site, I'm curious, does it allow importing from existing native Drupal and/or WordPress commenting systems?

That's my biggest pain point right now. Exposing public login to my CMS is a security risk (see: plugins/modules with permission escalation bugs) and frankly a pain I don't want to manage anymore. Migrating old comments to a new system is preferable to keeping the legacy comments system (and user accounts) in place while maintaining a second system on top of it for new comments.

<comment><USER13> Related (if the creator is still around): Is there a roadmap/timeline for SSO?

<comment><USER3> I plan on implementing SSO for the next release (v1.7.0), but I've been a bit busy with school work for the past week or so. Still about ~2 weeks away, by my estimate.

<comment><USER13> Oh wow, great.

I'm used to roadmaps and software planning cycles in months or years so "weeks" is probably before I could even get somebody to open an email to start thinking about this. :)

</comment></comment></comment></comment><comment><USER14> [https://gitlab.com/commento/commento](https://gitlab.com/commento/commento)

<comment><USER15> Commento is hosted on Gitlab now: [https://gitlab.com/commento/commento](https://gitlab.com/commento/commento) The Github page is only a mirror.

<comment><USER14> Yes sorry about that, it took me few seconds to notice it and updat the link in my message

</comment></comment></comment><comment><USER16> I'm actively bootstrapping an alternative to Disqus called Remarkbox ([https://www.remarkbox.com](https://www.remarkbox.com)).

It's a real struggle to get people to actually make the switch.

</comment><comment><USER17> How this compares to Isso [1]?

[1]([https://posativ.org/isso/](https://posativ.org/isso/))

<comment><USER18> > Even compared to other open source Disqus alternatives, such as Isso and > Schnack, Commento is much faster and requires less download data and RAM.

If you click on the link, there is also a table comparing Commento, Isso and Schnack

</comment></comment><comment><USER19> I note that putting lines of text between ```...``` can be used to put them in a monospaced font.

This is useful when discussing programming, as one can include excerpts of code. To do this in Reddit, you have to prefix each line by 4 spaces, which is hard to do because the comment box doesn't use a monospaced font.

Maybe Commento could have an option of a monospaced font for the comment box?

<comment><USER20> Reddit supports ``` BTW. You are thinking of StackOverflow probably (which is very annoying). Extra bonus points to sites like Github for supporting ```language-name, e.g. ```js / ```ruby / etc.

<comment><USER21> StackOverflow finally added ```language fenced code blocks a few months ago.

[https://meta.stackexchange.com/a/322000](https://meta.stackexchange.com/a/322000)

<comment><USER20> You are a life saver! Now they just need to fix the reputation for new users but that's a lot more complex.

</comment></comment></comment></comment><comment><USER22> There was some discussion about adding comments to beta.observable.hq a while back[0][1]. One of the offered solutions was to embed disqus into a notebook manually[2]. Could you do that with Commento as well?

[0] [https://talk.observablehq.com/t/comments-on- notebooks/1662](https://talk.observablehq.com/t/comments-on-notebooks/1662)

[1] [https://talk.observablehq.com/t/let-s-talk-about- observable-...](https://talk.observablehq.com/t/let-s-talk-about-observable- suggestions/1675)

[2] [https://observablehq.com/@bumbeishvili/comments-on- notebooks](https://observablehq.com/@bumbeishvili/comments-on-notebooks)

<comment><USER3> Yep, looks doable. All you need to do is insert two tags: a <script> and a <div>. You can also disable auto loading and manually trigger loading comments (see `data-auto-init` in the docs [1]).

[1] [https://docs.commento.io/configuration/frontend/](https://docs.commento.io/configuration/frontend/)

</comment></comment><comment><USER23> I've used a similar solution called TalkYard [1] which provides a interesting combination of commenting and forums, and has super-helpful support.

[1] [https://www.talkyard.io/](https://www.talkyard.io/)

</comment><comment><USER24> Another kudos to the project, adtac!

A few days ago, I self-hosted Commento to slowly replace Disqus on my sites. It was pretty painless to get the release binaries working on a little $5 Digital Ocean droplet, and get SMTP + Google OAuth configured.

To keep my page loads sane when I used Disqus, I had to delay the Disqus scripts from loading until the user hit a "Load Comments" button. It's nice that Commento comes with that sort of functionality right out of the gate, along with the ability to pass in a CSS file to override the styling.

</comment><comment><USER7> Very cool as an alternative... the link at the top though, I thought it was like 1997 all over again.

I'm glad to see alternatives... Though, I kind of miss when newsgroups and BBSes were more popular.

</comment><comment><USER25> You have a "Your Price" option? Love it! [https://commento.io/pricing](https://commento.io/pricing)

</comment><comment><USER26> Honestly, Disqus has been so unhelpful with fixing issues (such as users not being able to post comments from iPhones) that this might be worth a look.

</comment><comment><USER27> [https://github.com/posativ/isso](https://github.com/posativ/isso)

<comment><USER28> [https://github.com/umputun/remark](https://github.com/umputun/remark)

<comment><USER29> This should be upvoted more.

</comment></comment></comment><comment><USER30> From:

[https://gitlab.com/commento/commento](https://gitlab.com/commento/commento)

> ... half a second increase in page load time results in a 20% decrease in > engagement and site traffic.

An intriguing stat. Wonder if the 20% figure is specific to new visitors, who have never seen the site before?

</comment><comment><USER31> Instead of using another data silo, consider webmentions. Post a reply on your own blog with rel="reply-to" and send a webmention notifying the author, who uses a plugin or webmention service to display replies.

It's decentralized web comments. See indieweb.org for more information.

</comment><comment><USER19> How does the spam filtering work? Is it just Akismet, or is there something else you do locally?

</comment><comment><USER32> Cool project! I am also creating a similar product [https://just- comments.com/](https://just-comments.com/) It's great to see so many alternatives to be created recently.

</comment><comment><USER7> Suggested a feature on GH project. Would be cool to see an extension for this, similar to dissenter, that could inject related comments, particularly for websites that don't even have comments.

</comment><comment><USER33> What are some competitors to Disqus re: hosted comments?

What are the challenges with implementing one?

Does Disqus store the comments against a url? What happens if you add some nonsensical GET parameters? Do the comments disappear?

</comment><comment><USER34> Coral Talk commenting system is a good alternative: [https://coralproject.net/talk/](https://coralproject.net/talk/)

</comment><comment><USER8> I made a comment below about my negative experience with comments but forgot to mention an important detail: I took down my comments section because of spam.

I had found your solution back when I was researching comment systems, and it looked (and still does look) great! I especially like the clean UI and tree- style comments. However, the big issue for me is how it handles spam. What sort of tools are there to save me from spending time moderating comments?

Even something as simple as "initially hide comments that contain URLs" would go a long way towards removing lots of spam.

</comment><comment><USER35> The demo page locks up for ~5 seconds before the comments appear. I wouldn't call this "lightweight".

<comment><USER3> 500+ comments and 5k+ page loads in the last few hours, but it turns out the backend was still pretty fast. It was the JavaScript that dynamically generated the comments web page that added a couple of seconds of delay. Looks like I know where to look for performance improvements next :)

</comment></comment><comment><USER36> This has reminded me to enable it on my blog (temporary testing). Thanks @adtac for creating it!

</comment><comment><USER37> Cool name!

<comment><USER3> Funny story behind that. I originally thought the name was Latin for "comments". Naming things is hard, so I figured I'd just take the easy way out and translate from a dead language. I realised it was Italian only months later lol. Google Translate failed me.

<comment><USER37> And Spanish & Portuguese more or less

</comment></comment></comment><comment><USER38> This is great. Is disqus still in business?

</comment><comment><USER39> dang. requires javascript.

</comment><comment><AUTHOR> Url changed from [https://www.256kilobytes.com/content/show/4957/what-is- comme...](https://www.256kilobytes.com/content/show/4957/what-is-commento-the- open-source-disqus-alternative), which points to this.

</comment><comment><USER40> Only I can't open this page?

<comment><USER41> Count yourself lucky. Hoorah! for any alternative to Disqus, of course, but OMG the layout of their WWW page is heinously awful example of what can happen when children are permitted to do CSS.

<comment><USER42> The actual site for the project is [https://commento.io/](https://commento.io/).

But you're right, the site in the op is super ugly.</comment></comment></comment>


Papers I like (part 5) - <AUTHOR> https://fgiesen.wordpress.com/2017/09/02/papers-i-like-part-5/ <comment><USER2> I found the commentary on Tomasulo's paper particularly fascinating. I had no idea those concepts were so old! Thanks for the history lesson!</comment>


China Bans Bitcoin Executives from Leaving Country, Miners "Preparing for Worst" - <AUTHOR> http://www.trustnodes.com/2017/09/19/china-bans-bitcoin-executives-leaving-country-miners-preparing-worst? <comment><USER3> So the PBOC is taking further steps to plug the leak. Not unexpected.

China has exchange controls. Internal yuan cannot be easily converted to euros or dollars. There are limits on capital outflows. The controls are leaky; the classic leak is to send all your relatives to Hong Kong loaded up with the maximum allowed amount of yuan in cash. The current limit is about $50K/year/person. Small leaks are thus tolerated. Big leaks get plugged, eventually. Bitcoin has apparently been moved to the "big leak" category.

(The US does not have exchange controls. If you want to wire transfer a million dollars from the US to Switzerland and buy a house in Geneva, the US has no objection, provided that you report it for tax purposes. Some Bitcoin enthusiasts are unclear on this. Why China has tight exchange controls is beyond the scope of this discussion.)

<comment><USER4> > _China has exchange controls_

Bitcoin is irrelevant, compared to still-tolerated and massively-more voluminous alternatives, for capital controls. This is a red herring.

Bitcoin was fine until ICO promoters started pilfering from the masses. In any case, China is currently loosening its capital controls [1].

This isn't a problem that can be constrained to China. China is just reacting first. The rebranding of this action away from fraud and towards capital controls is dishonest.

[1] [http://www.reuters.com/article/us-china-economy-capital- cont...](http://www.reuters.com/article/us-china-economy-capital- controls/china-relaxes-some-cross-border-capital-curbs-as-yuan-steadies- sources-idUSKBN17L0M2)

 _Disclaimer: I am not a lawyer. This is not legal nor financial advice. Do not launder money._

<comment><USER5> How is it irrelevant? You can buy Bitcoin with yuan, transfer the bitcoin to a foreign exchange, and then buy another currency like the US dollar with the bitcoin, thereby circumventing capital controls.

<comment><USER4> > _thereby circumventing capital controls_

You can also invoice hundreds of millions in fake orders. This is how the bulk of exfiltration occurs.

Addressing the rounding error before the bulk means other concerns are at work. I am ignoring, in this argument, that recent policy—preceding these orders—calls for more capital export, to keep down the yuan.

<comment><USER6> The difference I think is that bitcoin allows someone with no connections to bypass capital controls. I'm not saying bitcoin isn't a rounding error, I'm sure other priorities (like ICO scams) are at work, but that might be another point.

<comment><USER7> But someone with no connections would presumably not have that much capital to move either, right? They'd not move much volume even if they bypassed capital controls.

<comment><USER8> It can be dangerous though. If suddenly several million citizens start transferring small amounts you can easily get outflows of several billion USD a month. Done by a few that would be an amount easily noticed.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> What's interesting to me is that no amount of bad news appears to affect the price of BTC. I'm not a professional or amateur trader (I've got no position on BTC at all) but the price fluctuations really seem dominated by sentiment over any sort of fundamentals.

<comment><USER10> It's a 24/7 market; the price responds a lot faster than other markets. So looking at day-over-day price isn't a good gauge. Additionally, by the time a story is known at large, it's usually been known for a bit among those who follow Bitcoin, so you'll see it already priced in.

<comment><USER11> Are you sure? Doesn't the need to mine block slow down transaction to a bare maximum per block?

And if miners in china close operations won't time between block increase?

Thus maybe BTC is actually responding slower than other market... what do you think?

I'm by far not a keychain expert so I will gladly be corrected if I'm wrong.

<comment><USER12> Bitcoin adjusts block generation to 10 minutes irrespective of the miners. What does change is difficulty, hashrate and transaction fees.

The bitcoin hashrate has remained more or less the same in last 2 months: [https://blockchain.info/charts/hash- rate?timespan=60days](https://blockchain.info/charts/hash- rate?timespan=60days)

This is in contrast with last 3 months where the hashrate nearly doubled: [https://blockchain.info/charts/hash- rate?timespan=180days](https://blockchain.info/charts/hash- rate?timespan=180days)

Fees peaked around 25th August but still pretty high: [https://bitinfocharts.com/comparison/bitcoin- transactionfees...](https://bitinfocharts.com/comparison/bitcoin- transactionfees.html)

<comment><USER13> Hashrate has not remained constant:

\- 7 day avg of last 3 mo: [https://blockchain.info/charts/hash- rate?timespan=180days&da...](https://blockchain.info/charts/hash- rate?timespan=180days&daysAverageString=7)

\- 7 day avg of last 2 mo: [https://blockchain.info/charts/hash- rate?timespan=30days&day...](https://blockchain.info/charts/hash- rate?timespan=30days&daysAverageString=7)

Going from 6 to 7.5 EH/s it's not more or less the same (it's a 25% increase)

</comment></comment></comment></comment></comment><comment><USER14> BTC noob here, please forgive me. My understanding is that you can send from any wallet to any other wallet, right? So what's to stop me from having wallets on both Huobi and LakeBTC, buying a BTC on Huobi for $3524 then transferring it to my LakeBTC wallet and selling it for $4282?

I understand arbitrage so I'm sure there's a step I'm missing, and I know there's often a delay between the order and fulfillment, fees etc, but a 21.5% spread between Huobi and LakeBTC seems like a _lot_.

<comment><USER15> It's still an option now, however it looks like they too will lose banking access in China soon.

>The impact of the Chinese ruling on LakeBTC is limited. The only major change is that our services to all Chinese citizens will be suspended. LakeBTC is a legitimate business which follows applicable laws and regulations.

Source: [https://www.lakebtc.com/p/9352](https://www.lakebtc.com/p/9352)

The type of arbitrage you described has been going on for years, especially for ETH where 20% price differential isn't uncommon at all. The difficult part, however, is getting money in and out of China without raising suspicion. By the time you have paid off all middleman the margin isn't that great anymore.

</comment></comment><comment><AUTHOR> Is it just me or has China been doing some crazy regulatory stuff recently. Idk if there's an overall mosaic I'm missing or if these are unrelated.

<comment> EZ-E Yes there has recently been :

\- promise to block all VPN services from accessing blocked website

\- law to hold chat group owners responsible to what the members say (controverse political talk or gambling for example can lead to arrests)

\- most of casual barbecue that were held in the streets last summer have been forced to close this year in my area

\- tightening and more scrutinity on visa applications for foreigners (there has been some abuse so this is legitimate)

\- more random drug tests and work visa checks on foreigners (again there are many illegal teachers so can't blame them)

\- ban of many cheeses (like blue cheese)

\- ban of Winnie the pooh from social media following a picture mocking Xi as looking like Winnie

China is getting more and more authoritarian and strict. Some of these moves are because of the big party congress in one month where Xi is excepted to renew his term

<comment><USER16> Why would they ban blue cheese?

Is drug use high among foreigners or is that just a harrassement control to keep them in mental check, so they fear the unpredictability of the government and the power they hold over individual fate?

<comment><USER17> My guess is food safety. Not quite real food safety, but also not made up. Some European cheeses are not available in here Australia because of regulations about how dairy products have to be processed (Pasteurisation I guess).

Not only might that logic work in China too, but the culture is less tolerant of a food-stuff what is after all curdled milk with a fungal infection. (And we in the west get grossed out by chicken feet!).

<comment><USER18> Food safety is a great way to backdoor agricultural import restrictions. Europe has been doing it for _decades_ on GMO's.

<comment><USER19> Wasn't the American GMO's studies bullshit for decades as well? If anything, Europe has led the way in strict and unbiased testing.

</comment></comment></comment></comment></comment></comment><comment><USER20> China's fast descent into totalitarianism is fascinating and scary. One can easily replace 'bitcoin executives' with 'foreign executives' or 'foreign assets' and see where this is going. I wonder if those companies that choose to outsource all the jobs in their country to China realized what they've done.

<comment><USER21> I think people tend to gloss over the fact that China has a population of more than 1.4 billion people in a land area that's slightly smaller than the US, multiple territories in outright revolt, ongoing territorial conflict, some cities that are approaching inhospitability due to environmental damage, and our increasingly aggressive nation is something less than friendly with them.

I think one thing causing the apparent increase in political discontent in the US is the increasing metaphorical distance between congress and the people. Now imagine if you increased our population by more than 400%, had Hawaii declaring independence, and various states trying to separate either to go completely independent or join Mexico or Canada. And now of imagine that a coalition of China, Russia, Iran and other unfriendly nations were constantly posturing towards an invasion of Mexico which would put them right at our doorstep while their motherland remains far out of reach.

I'm in no way defending China, but I think it's safe to say that the level of authoritarianism we would respond with if we hit even a fraction of the issues they're facing would be unlike anything the world has ever seen. Our current political systems and technologies, regardless of specific political ideology, do not scale well.

That said there is one macabre issue that benefits China. About 50 years ago Chinese were starving to death by the tens of millions. While perhaps controversial, I think contentedness paradoxically breeds discontent. Consider in the US that much of the political discontent is heavily centered at some of the most privileged areas in the nation. We can wax poetic about justice and the nuance of social interaction. People struggling to just get by have more immediate issues to deal with. China's successful industrialization of their nation is already producing an increasingly large contented class to whom the struggles China has had are as personally familiar to them as the lynchings or Cold War of US history are to us. That newfounded contented class is likely to spark discontent towards the very actions that provided their luxury - <USER2> China will have to manage that somehow.

<comment><USER22> >had Hawaii declaring independence, and various states trying to separate either to go completely independent or join Mexico or Canada. And now of imagine that a coalition of China, Russia, Iran and other unfriendly nations were constantly posturing towards an invasion of Mexico

Can you map these metaphors to their counterparts in China?

<comment><USER23> I believe it's the western states which are in rebellion, which have significant Muslim populations. North Korea is being threatened with invasion and China is encircled by US allies and military bases.

<comment><USER24> I wouldn't call the situation in Xinjiang or Tibet a rebellion. There are independence movements of some importance, but AFAIK the majority population in both regions are Han Chinese by now, who have much less reason for separatism.

A comparable situation for the US might be native Hawaiians campaigning for independence, while the majority population just doesn't care. There are probably such groups already, they just haven't resorted to violence enough/encountered a strong enough reaction by the state to become notorious.

<comment><USER25> You are really overestimating the substance of the native Hawaiian independence movement. The vast vast majority of people living in Hawaii are not even native Hawaiian to begin with. And even most local Hawaiians don't want independence from the United States. It is really a tiny minority offshoot with some specific political grievances. Independence isn't even seriously floated by the majority of these independence folks. The whole idea is just a political and marketing tool except to a very small group of people.

</comment></comment></comment></comment></comment></comment><comment><USER26> All comments so far have been pro-bitcoin... does nobody think this could actually be a sensible decision?

<comment><USER27> For a totalitarian country, yes.

<comment><USER28> In almost any category conceivable (police presence on the streets, incarceration weights, bureaucratic oversight), the PRC is significantly more libertarian than the US. It is true that the CIA has invested a tremendous amount of energy and money into political issues designed to delegitimize the CCP -- Tibet, Falun Gong, and Taiwan. Liberalizing speech around these topics would likely be more efficient counterstrategy than the current approach, which is a remnant of an older model of socialist statecraft.

<comment><USER29> "Any category conceivable" here means "a bunch of categories I cherry-picked."

I can "conceive" of many categories in which the US is less totalitarian: freedom of speech and of the press, judicial independence, not having to show ID to buy train tickets or prepaid SIM cards, freedom of assembly, severity of punishment for minor drug crimes, ...

The US isn't perfect, but let's not exaggerate.

</comment></comment></comment></comment><comment><USER30> Curious if this will have an impact on "liquid' domain names (3 letter .com domains). They spiked a few years back as a way to get money out of China, but slowed recently due to bitcoin being a better option.

Would be interesting to see if that activity picks back up.

<comment><USER31> What's the going prize on 3L.com's right now? Say, a bad combination (Q, X, Y, V, etc.)?

Last time I exited this market (2008), lowest tier 3L.coms were selling for $3k

<comment><USER32> I have a 4 letter domain that sounds like a common word. What should I sell for?

<comment><USER30> You're gonna have to share way more info than that...

</comment></comment></comment></comment><comment><USER33> I'm most interesting in how the unfolding events affect the miners. Over 50% of mining compute is happening in China, would the state step in and try to control this aspect as well? My understanding is this would give them to power to make arbitrary changes to the blockchain.

As the article mentions, miners have expenses to pay which are likely funded by bitcoin conversion to local currency, if they can't operate in China (especially favorable because of inexpensive power) the controlling nodes will be moving or shifting to other mining firms.

</comment><comment><USER34> One would be naive to think that a government like PBOC would just let it be. But this is long-term positive news for Bitcoin.

It is good that China is taking steps to regulate the cryptocurrency world (starting with an outright ban, I guess) Especially the ICO's, which are mostly scams, and even those that aren't should probably be treated as securities.

Bitcoin (and other crypto-currency) exchanges also could use some regulatory oversight - if you watch the order books even on US-based ones like GDAX there's non-stop spoofing and other trading illegalities that would normally get you banned from any exchange and perhaps land you in jail in a jiffy.

It will be interesting to see what happens if we lose the China hashing capacity. If that happens (and it looks like it already is), we will have slow blocks until the next difficulty adjustment.

In general the more of this sort of thing happens, the better it validates the Bitcoin model. What doesn't kill it makes it stronger.

<comment><USER8> To be fair, in the past exchanges weren't the one advancing rules against spoofing. Only when regulators forced them to act they actually started stepping in. Exchanges have an incentive to maximise trading volume, high volatility is good for them.

</comment></comment><comment><USER35> "China's decision to shut down exchanges took many by surprise"

Really? It's a "communist" country. If anything, what's surprising is that it took China this long to start cracking down on Bitcoin.

</comment><comment><USER36> If miners shut down, mining rate slows, supply goes down, so price should go up. Right?

<comment><USER37> No.

Bitcoin algorithm automatically adjusts every 2016 blocks, to make the average block time 10 minutes. If the number of miners drops, the algorithm will adjust the difficulty of finding a 'good' hash. Meaning as an individual miner you have a better chance of finding the correct hash, but the overall network has the same rate.

<comment><USER38> If all the chinese miners drop out suddenly, how long does it take to mine those 2016 blocks? More precisely, how much longer than 20160 minutes (2 weeks)?

If Bitcoin transaction rate would be significantly reduced for weeks or months, would that kill it in the sense that Litecoin or Ethereum would take over?

<comment><USER37> It should happen right away I believe. As soon as the next block after the 2016th is accepted, the difficulty updates.

The transaction rate would only be affected for a max of two weeks as I understand it. As for how the 'market' would react if all Chinese miners dropped out, I have no idea :)

</comment></comment></comment></comment><comment><USER39> So... Are we going to see a surge of cheap second hand GPUs?

<comment><USER32> They mine with ASIC chips they themselves manufacture in China. If anything this hurts our ability to buy mining hardware from china, the main supplier.

</comment></comment><comment><USER40> This is good news for Bitcoin. They are holding the executives so they can apologize and issue them licenses.

/s

</comment><comment><USER41> First they ignore you, then they laugh at you, then they fight you, then you win.

-Mahatma Gandhi

<comment><USER42> People always quote this for all sorts of causes, but the fact that you're at one stage doesn't mean you're going to move to the next:

\- First they ignore you. Nobody ever notices you.

\- First they ignore you, then they laugh at you. Nobody takes you seriously.

\- First they ignore you, then they laugh at you, then they fight you. You lose.

\- First they ignore you, then they laugh at you, then they fight you, then you win. Congratulations! Now who are you ignoring?

ISTM you need to have a plan for crossing each of the three filters, rather than saying "they're fighting us, therefore we'll win".

EDIT: It's even funnier if you realise that the quote is symmetric and using it as an argument also proves its opposite. Activists often sub in government/"the system" as the "they", but activists could be the "they" from the system's perspective. They start by ignoring the system, then mocking it, then fighting it, then the system wins?

<comment><USER43> You have a very valid point here. But to me this quote is more of a "Hey, it's normal that you have setbacks, but keep going, it's part of winning".

So to prevent: "Damn, they are ignoring me, I might as well give up".

</comment></comment></comment><comment><USER44> China is a closed economy, it makes sense that they would put a ban on a technology which outside of speculative investment is largely used to facilitate money laundering.

<comment><USER45> Money laundering is currently not an issue, it has more to do with capital controls.

<comment><USER16> Serious question: how are capital controls vs money laundering different from the perspective of the one with the money?

<comment><USER46> Depends if you obtained your funds legally or illegally? With former you are betting against Chinese economy with latter you are looking to leave less traces of activities that got you those funds.

<comment><USER47> That's not such a strong distinction, I think. I mean, "legally" and "illegally" are defined by the State. And there's plenty opportunity for selective/surprise enforcement.

<comment><USER48> Yeah but that's what laundering is. Making money that would be designated dirty by the state look clean.

This is more of just hiding it outright, I think?

<comment><USER47> Well, domestic capital leaving China is largely illegal. So circumventing laws through Bitcoin is arguably money laundering. And indeed, US laws against money laundering apply regardless of the source.

<comment><USER49> > Well, domestic capital leaving China is largely illegal

Says whom? I imagine anyone who got rich legally in China would also try to get the capital out and diversify in more developed countries.

<comment><USER47> I meant that there aren't many legal ways to get it out.</comment></comment></comment></comment></comment></comment></comment></comment></comment>


Facebook Reveals New Security Settings Amid Privacy Concerns - <AUTHOR> https://www.bloomberg.com/news/articles/2018-03-28/facebook-announces-new-security-settings-amid-privacy-concerns <comment><USER2> They absolutely did not "reveal new security settings" – they revamped the security settings menu & page. From the article:

> Facebook announced on Wednesday that it's redesigning the settings menu on > mobile devices, consolidating privacy options in one place, rather than > sending users to some 20 different screens

There is no indication in the article that they are allowing users more control over their data, and fb remain free to do what they will with your data once you sign up for their service.

This does nothing to resolve the problem that people are up in arms about:

> Under the revamp, users still won't be able to delete data that they had > given third-party apps on the platform previously, even if it was used for > reasons other than what was agreed to. That data, generated over years of > games and personality quizzes that had access to private information, is > largely still stored outside of Facebook's grasp by the private individuals > and companies that built those applications.

<comment><USER3> > users still won't be able to delete data that they had given third-party > apps ... is largely still stored outside of Facebook's grasp by the private > individuals and companies that built those applications.

I don't believe it's possible to resolve _that_ problem.

</comment></comment><comment><USER4> What's really needed is a way to delete old data in batches, like "delete everything I've posted over 1 year ago".

Currently you have to delete thousands of individual items (like played Spotify songs) one-by-one, using a tediously slow and inconvenient UI that individually confirms each delete.

<comment><USER5> The "tedious to delete, by design" approach crosses all companies (create account = 1 click, delete account = call this number between the hours of 1:15 and 1:16 on Tuesdays and wait half an hour to speak to a representative, enduring their sales pitches first).

There should be a simple law: the mechanism for deletion of any user data must be no more complex than its creation.

It would also be nice to have deletion terms always specified at point of creation, e.g. "bookmark this link to remove what you're about to create".

<comment><USER6> >the mechanism for deletion of any user data must be no more complex than its creation

I don't think that'd work too well... indeed, it's already in place. You can create a single post with one click, you can delete a single post with one click. I guess we're looking for "batch deletion".

<comment><USER7> > ...you can delete a single post with one click.

Can you? Or, like the parent states, is there a confirmation for every deletion, thus two clicks?

<comment><USER6> Well multiple actions in both cases. If you're making a text post, you need to 1) write the text, 2) click submit. Likewise with an image, 1) upload image, 2) click submit.

I suppose the confirmation isn't strictly necessary but for a normal user's use-case, but having a confirmation is a good thing (accidental irrecoverable deletions are bad). But they could definitely have a one-click delete with an Undo button.

</comment></comment></comment></comment></comment><comment><USER8> The real question is: will they let me delete all the call logs, address books, and SMS data they've scraped off my android phones over the years? If they do, will it actually be deleted? How long will it take to be removed from all backups?

<comment><USER9> GDPR comes in in May. That's the interesting development

<comment><USER10> What's gdpr

<comment><USER11> General Data Protection Regulation. I can't wait for it ... and to see how facebook handles all delete requests.

</comment></comment></comment></comment><comment><USER12> What's amazing to me is how the media can stir the public into a fever pitch with such ease. I've told people about FB and Google data collection for years and nobody cared, at least not until the talking heads told them to care.

What FB does is innocuous compared to the CIA vault7 leaks, and yet the media shoved that down the memory hole. The real question is why is this being pushed now when literally of this has been public info for years?

<comment><USER6> It's just part of the media machine. FB is currently the "bad guy" so any media agency who can push the narrative is rewarded with more user attention. This has a huge snowball effect and will result in FB getting hammered into the ground until a more interesting antagonist comes around.

Another interesting "bad guy" right now is Russia -- justified or not, notice how media outlets are trying their hardest to find _any Russian links whatsoever_ to every bit of political news, even things completely unrelated. For instance, try searching for "cambridge analytica russia" and look at the number of results.

</comment></comment><comment><USER11> Even if they offered a delete button any time before GDPR comes into effect I would not trust and use it. I also don't like the idea of using one of those chrome extensions mentioned here. To me, currently this is synonymous to hiding everything they have on me from myself.

But doing this after May 25 would mean I definitely want it gone for good and if they don't fully delete what I manually remove, I expect them to get themselves into trouble.

<comment><USER13> > But doing this after May 25 would mean I definitely want it gone for good > and if they don't fully delete what I manually remove, I expect them to get > themselves into trouble.

You're technically right here, but it seems at present that most companies won't be complying with this for quite some time (perhaps an opportunity for some litigations?)

Backups appear to be the "black hole" in GDPR, as nobody spent the 2 years actually planning and preparing. Therefore, most people I've spoken to are planning to simply add a "gdpr_deleted" flag to each row in the database, and set it to true when it happens. They don't return those rows in queries. The inter-dependent nature of many databases (you shared a post with 10 users, you delete your account now, FB wants to retain the view records), coupled with nobody taking GDPR seriously until the last minute, means we'll likely see "soft deletes" for quite some time.

It would be interesting though to make a GDPR request to companies for data held specifically on backups, and held in databases records that would normally not be shown (i.e. these hidden records), as that could get them in trouble if you could prove they lied!

<comment><USER14> > Backups appear to be the "black hole" in GDPR

To delete backed up data, first encrypt the data with a key. To delete the data, delete the encryption key, leaving your "backed up data" meaningless bits.

</comment></comment></comment><comment><USER15> Here's a good security/privacy option: Provide a fully featured HTML5 app (specifically one that allows you to access the inbox). Then there is no reason to install the native app or messenger.

<comment><USER16> I only use my browser to use FB. But I don't message with Messenger - I use phone's native text message feature.

<comment><USER15> I'd still like to access the Facebook Inbox when I'm on my phone and using the Facebook web page. Right now it forces you to the messenger native app.

</comment></comment></comment><comment><USER17> >""Last week showed how much more work we need to do to enforce our policies and help people understand how Facebook works and the choices they have over their data," Facebook Chief Privacy Officer Erin Egan and Deputy General Counsel Ashlie Beringer wrote.

Facebook Chief Privacy Officer. Now there's a job title.

Seriously what does a Chief Privacy Officer at FB do? What are the responsibilities of a Privacy Officer at a company whose entire existence is predicated on their users having no privacy? Here's 15 years worth of their boss's views on privacy:

[https://www.cnbc.com/2018/03/21/facebook-ceo-mark- zuckerberg...](https://www.cnbc.com/2018/03/21/facebook-ceo-mark-zuckerbergs- statements-on-privacy-2003-2018.html)

</comment><comment><USER18> Who is ever going to trust that these settings actually do anything except for the button animation after the previous two weeks of privacy disaster for Facebook?

Probably most of the country, actually. =\

Mark my words, their stock price goes right back to $180 over the next two weeks.

<comment><USER19> people who don't wear tinfoil hats

<comment><USER20> That's kind of a silly comment when we have proof that Facebook's poor design lead to abuse.

<comment><USER19> there's a distinction between abuse from a third party and outright lying on the part of fb. suggesting that deleting data leads to nothing more than a CSS animation is asserting the latter.

<comment><USER21> I think the reality is that data will no longer "look" like it's in the data download, I'm sure FB will keep it to continue performing ML algorithms to pursue more and more ad revenue. Hopefully they will at least remove the code that is collecting it so that it stops in the future. No tin foil hat needed to believe this route.

</comment></comment></comment></comment></comment><comment><USER22> Something curious about facebook is, they let anyone access user profile/pages (or whatever it is called), so visitor can see the user profile picture and might be able to see other stuff. Facebook "protects" the privacy of the "visitor" because the user cannot see what visitors have visited their profiles.

So, users have no privacy, but visitors have their privacy protected.

Facebook should list "who" visited user profiles. I dont know why they dont do this already. And even more, they could somehow list what photos the visitor chose to see, how long visitor stayed on user profile, which 'full resolution pictures' visitor had downloaded. If they only saw thumbnails then no need to list, but if visitor made the effort to download _all_ full resolution user pictures then the owner of those pictures should be able to see who has downloaded them.

So this way, the user would be able to see how frequently visitor X, Y, Z has stayed on their profiles, and how many/which full res pictures they have downloaded/seen.

If the user profile is not a person, but a store, then the store would be able to benefit from this data and send some message saying "We noticed you have been visiting us lately, and you seem to have a lot of intrest in product X. Do you want to know the price or make an offer?"

Or, if the user is a person and has seen weird behaviour from weird users, then user would be able to take precautions.

So in the end, the user profiles are public and have no privacy. But the visitors/consumer of other people profiles can snoop on other lifes and have their privacy protected because the page they visited is never able to know they have been visited by visitor X Ntimes.

<comment><USER23> This is a pretty interesting proposition. I'm personally not a fan of the store idea --I'd rather be able to visit business pages anonymously and not get messages from them-- but I actually think seeing who has seen your profile could be a good idea, especially in the realm of the picture download/viewing.

</comment></comment><comment><USER24> Again, it wasn't "amid privacy concerns". It was because they were trying to be compliant with EU's upcoming GDPR regulation:

[https://www.reuters.com/article/us-facebook-sandberg- privacy...](https://www.reuters.com/article/us-facebook-sandberg- privacy/facebook-to-hand-privacy-controls-to-users-ahead-of-eu-law- idUSKBN1FC1Q6)

</comment><comment><USER25> What I want is to download my history _including_ the external links I posted. But they seem to not include that stuff. I'm sure they tracked which links I posted, but they don't consider that part of the private data I'd own in any way, so it's not there in the download. Which I means I can't find that article I stupidly posted to FB and forgot to bookmark otherwise…

</comment><comment><USER26> The real problem is data collected about you from other people. For example all of my lesser friends install the android facebook app which gives them tons of information about me from my interactions with them. I need a way to tell facebook to delete and not collect data about me from other people's devices.

</comment><comment><USER27> F#ck the hypocrites at Facebook. That's all.

</comment><comment><USER28> "The Facebook Privacy Setting That Doesn't Do Anything at All

'I really can't make sense of it.'

Gergely Biczok, CrySys Lab"

[https://www.wired.com/story/facebook-privacy-setting- doesnt-...](https://www.wired.com/story/facebook-privacy-setting-doesnt-do- anything/)</comment>


An Ancient Kernel Hole Is Not Closed - <AUTHOR> https://grsecurity.net/an_ancient_kernel_hole_is_not_closed.php <comment><USER2> After reading through the original Qualys discussion[1] of this new vuln I was definitely left with a feeling that this class of bugs has not yet exhausted its treasure trove of vulns. This article only confirms it.

Very interesting discussion that will likely continue as more people wrap their heads around this tough problem.

[1] [https://blog.qualys.com/securitylabs/2017/06/19/the-stack- cl...](https://blog.qualys.com/securitylabs/2017/06/19/the-stack-clash)

<comment><USER3> So the title should read "is not closed"? The parenthesis are just annoying. It's one or the other, just state it!

Readers shouldn't have to solve a puzzle to read a title.

<comment><AUTHOR> I'm sorry, I'm really confused now.

Here's the title of the article:

An Ancient Kernel Hole is (Not) Closed

I thought that I copy-pasted the article title to be the title of this submission.

Did I omit the parenthesis? If so, it was unintentional, and I apologize.

<comment><USER4> HN automatically removes certain punctuation from post titles. In a lot of cases though, it creates more confusion. Definitely not your fault.

<comment><AUTHOR> I did not know that! I'm tempted to experiment now 8-)

Thanks for the heads-up!

</comment></comment></comment></comment></comment><comment><USER5> Too bad this comment never got a reply: [https://lwn.net/Articles/401227/](https://lwn.net/Articles/401227/)

<comment><AUTHOR> It seems to me that this is a clash of personalities. Linus has been as acerbic before. See [https://lkml.org/lkml/2012/12/23/75](https://lkml.org/lkml/2012/12/23/75), for example.

Open Source Security (the company behind grsecurity) is still existing, so it's got enough customers to support the grsecurity developers (both the ones we know, as well as (presumably) PaX Team).

It seems to me like grsecurity aren't interested in upstreaming their patches. That's one reason (among others) why their patches are distributed as one big file, instead of (for example) a patch set or a Git branch.

Why are they not interested? Well, it seems to me like Linux devs may not have placed as high a value on security, when compared to other metrics (such as performance). I understand what it's like to be advocating for a position, not finding acceptance after a long try; finally saying "screw it" and going their own (commercial) way.

So any 'debate' related to grsecurity is going to include people (including LWN commenters) who feel very strongly against grsecurity's position, and how they should be more open, and free, and willing to compromise (if that's the right word) to get the grsecurity/PaX Team patches upstreamed.

And then you have very strong personalities in the kernel dev community. That makes compromise very difficult.

It my opinion, it's the same reason why having a conversation related to politics is nigh impossible: Your strong views affect what you say, how you say it, what you hear and how your perceive it.

<comment><USER6> Honestly, grsec has rightly earned a reputation for being hostile and even irresponsible with their conduct, regardless of how good their patches are (of course, the quality is known to be high- but there have been some blunders. Don't run `script /dev/null </dev/zero` on an older grsec kernel ;)

</comment></comment></comment><comment><USER7> Can someone give a short explanation of this exploit please? Something a bit deeper than "an attacker can exploit this confusion to overwrite the stack" from the OP, but not the super detailed PDF he references. I'm hoping to get a feel for the root cause of this exploit.

<comment><USER8> The classic vulnerability is that you manage to grow the heap or the stack so that they collide (they start at opposite ends and grow towards the middle) then you get all kinds of exploits possible by stack overwriting the heap or heap overwriting the stack.

While you can prevent the heap from being allocated over the stack, the stack may grow "autonomously" during normal processor instructions where the OS can't intervene.

The classic fix was to put a "guard page", i.e. a segment of memory marked as forbidden, so if stack would grow into there, it'd trigger a page fault and then the process can be stopped.

However, while stack _usually_ grows in small increments, you may allocate some very large data on the stack _and not write or read_ anything in that buffer - so if you do it correctly, the end of the stack will "jump over" that guard page, leaving it in the middle of the stack but not touching it in any way; and the end of the stack will overlap with the heap, enabling all the fun exploits again.

<comment><USER7> OK, if I understand correctly, there is no privilege escalation. Eg., running this exploit in user code doesn't get you root. When they described it as a "kernel hole" I assumed it was much worse. So the word "kernel" is there because the design flaw is in the kernel, not because it allows a complete system takeover (unless it's against a privileged program).

<comment><USER9> > So the word "kernel" is there because the design flaw is in the kernel

The vulnerability is in the userland process. However, the kernel can utilize a stop-gap measure that makes exploitation difficult in most scenarios (but this is arbitrary).

<comment><USER10> Then why is nobody trying to fix the vulnerability at the source ie. userland process?

<comment><USER8> There exists a fix for the vulnerability at the source, namely, a compiler feature that will add extra code that will "touch" every 4k page of the stack if large amount is allocated.

However, for _that_ fix to be useful, every single distribution needs to recompile every single userland package and distribute it to every single user - i.e., instead of downloading a security patch, you need to download a new version of every executable on your hdd.

<comment><USER11> It could be limited to only programs that are setuid.

<comment><USER12> This applies generally to any program running as root (or a user with other privileges than yourself) if you can find a way to allocate a large enough segment of memory.

<comment><USER11> So setuid programs and network servers, for example.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER13> Interesting. I independently dreamt up this class of vulnerability in 2007 or so. I didn't know there were any instances of it in the wild. When I realized it ought to be possible, I just wrote a PoC vulnerable app and an exploit for it, and then sort of forgot about it. Cool to see it's a real thing now.

<comment> yjftsjthsd-h Link? It might be useful to see a simple example

<comment><USER13> Unfortunately I don't think I have the code anymore. But the gist of it is that say you have two threads in a process. One of the threads contains a function like this:

      void recurse (int m, int n) {
        char arr[n]
        recurse(m--, n)
      }

If 'm' and 'n' are user-controlled, 'recurse' is executing in thread A, and thread A's stack is above thread B's stack, then you can cause 'recurse' to recurse down to the point that it makes its way into thread B's stack region. Normally, this is prevented by 'guard pages' in between the two thread's stacks. However, if you have a variable length array, as in the example, it's possible to allocate enough memory to just skip right over the guard page. Once you've done this, you're now writing directly into the other thread's stack, with all the fun stuff that entails.

</comment></comment></comment><comment> rob-olmos Red Hat's vulnerability page: [https://access.redhat.com/security/vulnerabilities/stackguar...](https://access.redhat.com/security/vulnerabilities/stackguard)

Implemented the 1MiB guard page and glibc hardening.

</comment><comment><USER14> That's a bit of a mouthful for a ever more commonplace occurence. I propose we refer to this class of vulnerability as "k-holes".</comment>


word2vec – Tool for computing continuous distributed representations of words - <AUTHOR> https://code.google.com/p/word2vec/ <comment><USER3> I just gave an invited talk at KDD about deep learning in which I covered this algorithm, so it's great to see this code appear now.

For anyone interested in text analysis: PLEASE study and use this code and the referenced papers. It's importance is hard to overstate. It is far, far better than all previous approaches to word analysis. These representations are the dimensional compression that occurs in the middle of a deep neural net. The resulting vectors encode rich information about the semantics and usage pattern of each word in a very concise way.

We have barely scratched the surface of the applications of these distributed representations. This is a great time to get started in this field - <USER2> techniques are almost totally obsoleted by this so everyone is starting from the same point.

<comment><USER4> I have previously used Explicit Semantic Analysis (ESA) algorithm for individual word similarity calculations. ESA uses as a basis the text of Wikipedia entries and its ontology as a source and worked quite OK.

Do you / does anyone know if there is an easy way to use word2vec to compare similarities of two different documents (think of TF-IDF & cosine similarity)? It is stated on the page that "The linearity of the vector operations seems to weakly hold also for the addition of several vectors, so it is possible to add several word or phrase vectors to form representation of short sentences [2]", but the referenced paper has not yet been published.

It would be super interesting if there was a simple way to compare the similarities of two documents using something like this.

</comment></comment><comment><USER5> Could anybody explain (or provide a pointer to an explanation of) the details of how the individual words are mapped to vectors? The source is available, but optimized such that the underlying how's are a bit opaque, and the underlying whys even more so.

<comment><USER6> You can think of this as a square matrix W. The size of the matrix is the size of the vocabulary. If we look at the 100k most frequent words in our corpus, W will be a 100k x 100k matrix.

The value of W(i,j) is the distance between words i and j, and a row of the matrix is the vector representation of that word. Research around word vectors is all about computing W(i,j) in an efficient way that is also useful in natural language processing applications.

Word vectors are often used to compute similarity between words: since words are represented as vectors, we can compute the cosine angle between a given pair of words to find out how similar the two words are.

<comment><USER7> Does that mean there actually is an answer for "What do you get when you cross a mosquito with a mountaineer?"

<comment><USER8> TL;DR: The answer to your query is a person named Chaudhry Sitwell Borisovich who is definitely an entomologist-hymnist and probably is also a mineralogist- ornithologist.

A google search suggests that he was born in 1961.

I ran a few queries using the code and its default dataset, trying to use neutral words for substraction: "mosquito -small +mountaineer", "mosquito -big +mountaineer", "mosquito -loud +mountaineer", "mosquito -normal +mountaineer", "mosquito -usual +mountaineer", "mosquito -air +mountaineer", "mosquito -nothing +mountaineer".

The most frequent words for these queries are:

6 times: "borisovich" "chaudhry" "entomologist" "hymnist" "sitwell"

5 times: "mineralogist" "ornithologist"

<comment><USER7> Well done, sir.

</comment></comment></comment></comment></comment><comment><USER9> Just as a hint, you'll get better results if you apply some dimensional reduction to this. LDA is an old standby, but I like what these people are doing...

[http://www.cept.at/](http://www.cept.at/)

<comment><USER10> This already does dimensional reduction using neural networks (for details see papers cited at the bottom of the page).

</comment></comment><comment><USER11> If you've ever complained on HN about NSA technology ruining our society, then time to take this tool and save society. This is powerful and cool technology. It could expose corruption. It could be a bullshit detector. The same types of tools that have stripped our liberties can be the same tools to re-balance democracy.

But please write free and open source software. Otherwise no one can ever trust the software or consider it secure and safe.

</comment><comment><USER12> This is great! It would be amazing to have a list of other tools/libraries like this.

One project I'm working on I was looking to compare the word/topic frequencies of a document/text against common usage and I'm sure there is something out there like this that does this already as opposed to me doing it (poorly) from scratch.

</comment><comment><USER13> "... and vector('king') - vector('man') + vector('woman') is close to vector('queen') [3, 1]. You can try out a simple demo by running demo- analogy.sh..."

I downloaded the code but I dont understand how to get queen as a result. I tried submitting king - man + woman, but the bin dont understand it.

<comment><USER14> Use just "man king woman"

</comment></comment><comment><USER15> I'm working in a experimental library to do a similar thing, but using echo state networks ([https://github.com/neuromancer/libmind](https://github.com/neuromancer/libmind)). It will be nice to compare both approaches with a SVM to classify words.

</comment><comment><USER16> Isn't it similar to what LSA([http://en.wikipedia.org/wiki/Latent_semantic_analysis](http://en.wikipedia.org/wiki/Latent_semantic_analysis)) does?

<comment><USER17> From the linked paper:

 _Many different types of models were proposed for estimating continuous representations of words, including the well-known Latent Semantic Analysis (LSA) and Latent Dirichlet Allocation (LDA). In this paper, we focus on distributed representations of words learned by neural networks, as it was previously shown that they perform significantly better than LSA for preserving linear regularities among words_

</comment></comment><comment><USER18> Our technology gives a much better identification of noun senses ([http://springsense.com/api/](http://springsense.com/api/))</comment>


Ask HN: Is a centennial computer feasible? - <AUTHOR>

As far as I know (very little), we can't make a computer that lasts for more than a few decades.<p>Fictional challenge: if software were not an issue, how would you design a computer whose primary goal is to last for 100+ years with little or no maintenance? I realize such feature would be barely useful, I'm just interested in delayed obsolescence. <comment><USER2> See [http://longnow.org/](http://longnow.org/)

<comment><AUTHOR> Thank you for the insightful link. I wasn't aware of such project!</comment></comment>


Names Hurt: How the Media Manipulates Our View of a Programmer - <AUTHOR> https://shecancode.io/blog/names-hurt-how-the-media-manipulates-our-view-of-a-programmer <comment><USER2> The Wired article was probably bad, but I don't think blaming "the media" is a good idea, since "the media" is often understood as "any media". It creates confusion.

The author has written an article. Somehow the author is also a media.

The media must not manipulate our view of the media.</comment>


Show HN: GPG on macOS and YubiKey - <AUTHOR> https://github.com/DataDog/yubikey <comment><USER2> Nice repo.

For those starting from scratch with a YubiKey I always recommend this guide:

[https://github.com/drduh/YubiKey-Guide](https://github.com/drduh/YubiKey- Guide)

Then they know how this stuff works and how to fix it when it breaks.

<comment><USER3> I just got a yubikey and found this guide today. It's quite good.

One thing that I still haven't found a good answer for that's not mentioned in the guide: what's KDF for?

The new yubikey firmware has release notes here: [https://support.yubico.com/support/solutions/articles/150000...](https://support.yubico.com/support/solutions/articles/15000027139-yubikey-5-2-3-enhancements- to-openpgp-3-4-support)

This is the bit that has me lost:

> To remove the transmission and on-card storage of OpenPGP PINs in plain > text, the YubiKey supports the Key Derived Function (KDF) functionality. > With the KDF function enabled, the PIN is stored as a hash on the YubiKey. > When entering the PIN to the OpenPGP Smart Card, the OpenPGP client will > only pass the hashed value, never passing the PIN directly. KDF > functionality is set on the card itself, and communicated to the client; it > is transparent to the user. Should the KDF functionality not be enabled, the > PIN function will work as previously. The KDF function is listed in section > 4.3.2 of the OpenPGP Smart Card 3.4 spec.

Can someone explain to me how KDF matters at all here?

It seems like the keys are encrypted on the yubikey via pin, or at least protected in hardware via pin, and that the pin is stored on the device. KDF seems to take that plain text pin and replace it with a hashed pin. If you steal my yubikey, it looks like KDF would prevent you from... dumping the PIN? But if you could dump the pin, wouldn't you just dump the key instead? I can't seem to figure out the threat model for this feature.

<comment><USER4> I'm guessing it's to protect against MITM of the USB interface

<comment><USER3> How would that help though? If you have a compromised USB interface, and you're entering your pin on that machine, you could just capture the keyboard input anyway.

<comment><USER4> That's a good point!, it has to be for another reason.

Interested to hear what it's for

<comment><USER5> Nope, you were right. It's for USB MITM. I guess the assumption is that the keyboard is wired in a different way (a laptop?) or in wireless scenarios (NFC Yubikey).

<comment><AUTHOR> Quality discussion, both of you, thanks!

</comment></comment></comment></comment></comment></comment></comment><comment><USER6> I've been very pleasantly surprised by how easy it was to get everything set up on NixOS:

``` programs.gnupg.agent = { enable = true; enableSSHSupport = true; };

services.udev.packages = [ pkgs.yubikey-personalization pkgs.libu2f-host ]; services.pcscd.enable = true;

environment.shellInit = '' export GPG_TTY="$(tty)" gpg-connect-agent /bye export SSH_AUTH_SOCK="/run/user/$UID/gnupg/S.gpg-agent.ssh" ''; ```

The only surprise I had was that I forgot to tell gpg to trust the imported key after I imported it.

Combine this with GoPass... its the start of something good :)

<comment><USER7> Are you using Gnome? I remember when this was impossible to get working on Gnome for a couple of releases. Hoping that's fixed now.

<comment><USER6> I'm using awesome wm.

What was the problem with gnome? I vaguely recall a problem with polkit or something.

</comment></comment></comment><comment><USER5> > Optional: verify public key on Keybase.

For organizations publishing employee keys via Web Key Directory can also be an additional signal that the key is trustworthy.

It's also quite simple: for example exporting the key 5C090ED7318B6C1E (binary, _not_ armored) and putting it on this exact URL: [https://datadoghq.com/.well- known/openpgpkey/hu/964aj6q73iat...](https://datadoghq.com/.well- known/openpgpkey/hu/964aj6q73iatngoya1q7qs4r6utpmb4g) is enough to discover the key using e-mail address.

This post goes into more detail: [https://spacekookie.de/blog/usable-gpg-with- wkd/](https://spacekookie.de/blog/usable-gpg-with-wkd/)

WKD is used by Linux distros (ArchLinux, Gentoo, Debian...) and kernel.org itself: [https://www.kernel.org/category/signatures.html#using-the- we...](https://www.kernel.org/category/signatures.html#using-the-web-key- directory) as well as some OpenPGP sites (e.g. ProtonMail).

<comment><AUTHOR> Good idea, will make a note, thanks!

</comment></comment><comment><USER8> > For usability while balancing security, cache PIN for at most a day.

[https://github.com/DataDog/yubikey/blob/master/gpg.sh#147](https://github.com/DataDog/yubikey/blob/master/gpg.sh#147)

This statement has no effect when using Yubikey - the PIN is cached by the key itself and it will remain unlocked indefinitely until it's physically unplugged. See [https://dev.gnupg.org/T3362](https://dev.gnupg.org/T3362)

<comment><AUTHOR> Interesting. This hasn't been my experience, so not sure what's going on yet...

</comment></comment><comment><USER9> It's not polished, but here's a vagrant box which can provision Yubikey's PGP and PIV applications. Additional hardening can be performed.

[https://github.com/justintaft/yubikey-gpg-piv- provision](https://github.com/justintaft/yubikey-gpg-piv-provision)

</comment><comment><AUTHOR> Main contributor here! Let me know if you have questions.

<comment><USER10> Any issues using Catalina? mine shows Error: No YubiKey detected!

<comment><AUTHOR> No, but try unplugging the YubiKey and trying again. Also, the script officially supports only YubiKey 5, but I've heard that 4 works if you remove the code that checks the version.

</comment></comment></comment><comment><USER11> Using a hardware token to protect the subkeys is important but it is also necessary to protect the primary key by keeping it offline.

The best method I know is to store it on paper with the help of paperkey:

[http://www.jabberwocky.com/software/paperkey/](http://www.jabberwocky.com/software/paperkey/)

The tool also supports a raw output mode which can be piped to a QR encoder. 4096 bit RSA secret keys fit in binary QR codes and they are much easier to use compared to manual data entry.

Current versions of zbar can't decode binary data in QR codes properly. I've sent some patches that fix the problem but they haven't been reviewed yet. Hopefully it will be possible to automate this process with zbarcam soon.

</comment><comment><USER12> Personally I mostly use it for SSH via PKCS11. I found that simpler than GPG because it's already integrated with SSH.

Still looking forward to SSH supporting U2F (some day).

<comment><USER13> It's here: [https://marc.info/?l=openssh-unix- dev&m=157259802529972&w=2](https://marc.info/?l=openssh-unix- dev&m=157259802529972&w=2)

</comment></comment><comment><USER14> Yubikey as a second factor for macOS login is still not possible, right?

<comment><AUTHOR> I don't think so. That's what TouchID is for, and we are thinking of storing signing keys in that secure enclave in the future.

</comment></comment><comment><USER15> Is signing every commit really that useful?

<comment><AUTHOR> If you care about who produced your source code, yes.

<comment><USER16> I think "sign every commit useful?" is less about whether signing is useful, and more about the trade-off of signing every commit vs just signing a tag on a commit.

Signing every commit is going to make it a mindless task. It's easier to be vigilant when signing if you sign less frequently.

What trade-offs should be considered?

<comment><USER17>
        git config --global commit.gpgsign true

this turns commit signing on for every commit, you don't have to explicitly sign commits as a separate step.

the criticism I've always heard of this practice is, what do you do with those signatures? github displays a little widget showing that the commits are signed, but beyond that I don't think it cares which public key they were signed with, so it's not really helping anything.

<comment><USER18> It is simply an audit trail to trace back to the origin, since anyone can set any email address as the author of a commit. GitHub will show the signature fingerprint I think when you hover over that widget, so you can at least distinguish between your key and an impostor's.

I use this but it causes weird usability issues for me when using multiple iTerm tabs or git GUIs when the GPG daemon's unlock timeout expires and I need to input my passphrase in again. I wish it could just be tied to the mac's keychain, so when I unlock my computer it is ready to go. Or at least tie it into TouchID or Watch like unlocking the mac does.

<comment><AUTHOR> Yes, it's so easy with this YK setup that there's no reason not to do it. The only exception is during rebase, but there's an option in GPG to disable signing then.

I sign all my commits so that everyone knows it was most likely me. You can even turn on branch protection in GitHub these days that rejects unsigned commits.

With all due respect, Mr. Torvalds isn't exactly famous for having designed the most secure kernel.

</comment></comment></comment></comment></comment></comment><comment><USER19> Is there a similar guide for TouchID?

</comment><comment><USER20> Slightly related: Anyone know why Google Chrome (not upstream Chromium, nor any derivative) is the only browser on Android that implements WebAuthn as intended? Do they implement it as a proprietary component?

<comment><AUTHOR> No, sorry.</comment></comment>