New phenomenon breaks inbound TCP policing - <AUTHOR> https://forums.whirlpool.net.au/forum-replies.cfm?t=2530363 <comment><USER2> Title is wrong; updates come over TCP that has been modified not to perform link sharing.

I have seen this with my wife's computer. Though I haven't seen the large- window phenomenon (haven't looked), what I do see is dozens of active connections opened to the same destination, which of course defeats both TCP's link sharing and standard QoS algorithms.

I work around the issue by bucketing any Akamai IP ranges I find into a very- low-priority queue, and let those TCP connections fight it out. Seems to have worked well.

For those interested, here are the Akamai IP ranges I use:

        188.8.131.52/12
        18.104.22.168/11
        126.96.36.199/14
        126.96.36.199/13
        126.96.36.199/10
        2001:428:4403::/48
        2001:428:4404::/48
        2001:428:4405::/48
        2001:428:4406::/48
        2600:1400::/24

<comment><USER3> It's also not Windows 10 specific, the post also mentions Office updates

<comment><USER4> There's a mention somewhere in the middle of the thread that someone had seen something similar with Apple's software updates, too: maybe it's a more general Akamai issue?

Even if it is isolated to the servers distributing MS's updates, I'm having trouble seeing how this could even be MS's fault-- it's the server side (owned and operated by Akamai) that's misbehaving here.

<comment><USER2> I'm not convinced either way. Windows Update is the one opening the dozens of connections (it has to if it's behind a firewall), not Akamai; Akamai's just serving the data. But Windows Update is possibly just calling out to an Akamai library that opens the dozens of connections on its behalf.

If there are also congestion-algorithm shenanigans like the forum post suggests, then Akamai is definitely at fault. But the issue I see is that of connection hogging.

<comment><USER5> * [https://developer.akamai.com/stuff/Optimization/TCP_Optimiza...](https://developer.akamai.com/stuff/Optimization/TCP_Optimizations.html)

* [https://www.akamai.com/us/en/about/news/press/2013-press/aka...](https://www.akamai.com/us/en/about/news/press/2013-press/akamai-speeds-downloads-and-online-video-quality.jsp)

</comment></comment></comment></comment></comment><comment><USER6> Hey cool, someone forgetting _yet again_ why we use TCP.

We don't use TCP because its fast. We don't use it because its reliable (although that's really useful). We use it because _we kept breaking the internet_. Once you get above a certain threshold, the network can't keep up with you and packets start getting dropped. The problem is that backing off just a little doesn't allow the network to recover.

Instead, we need to use exponential backoff in the face of packet loss to ensure that the network as a whole can recover.

But if you're pretty much the only connection misbehaving, and everything else backs off, then you can kinda get away with not using exponential backoff. The problem is that the applications that is was "kinda okay" to do this for was VOIP and friends, where realtime delivery is really important and exponential backoff causes noticeable drops in quality.

For a great read about these kinds of issues, check out the TCP-Friendly rate control RFC: [https://tools.ietf.org/html/rfc5348](https://tools.ietf.org/html/rfc5348)

<comment><USER7> > Once you get above a certain threshold, the network can't keep up with you > and packets start getting dropped. The problem is that backing off just a > little doesn't allow the network to recover.

Another aspect of this problem is that the network is too hesitant to drop packets [1], so by the time you've noticed packet loss things have gotten bad enough that the drastic backoff is needed. Widespread deployment of ECN and AQM would allow for more rapid feedback before any huge backlog develops, and consequently a less extreme response to congestion signals could be used.

[1] Arista would rather their 10GbE switches add up to 100ms of queuing delay per port than drop a packet: [https://lists.bufferbloat.net/pipermail/cerowrt- devel/2016-J...](https://lists.bufferbloat.net/pipermail/cerowrt- devel/2016-June/010701.html)

<comment><USER8> Slightly OT, but that link is _astonishing_.

That anyone can think adding 100ms of latency to a 10Gbe switch, even under heavy contention is a good feature is absolutely staggering.

<comment><USER9> It's not quite 100ms. A bit less. The explanation is simple : if tcp exponential backoff fires, you will have a very bad time on any tcp connection. Site owners, obviously, don't want that.

Try this : iptables -A INPUT -m statistic --mode random --probability 0.001 -j DROP

And see how your internet works. TLDR: sometimes loading times go through the roof, some instant messages go through in <0.1s, and on occasion it takes 30+ seconds, on occasion it's a DNS query that gets dropped and a page load suddenly takes 1 minute for no identifiable reason, large downloads always "get fucked" (suddenly lose 90% of their bandwidth and take several minutes to recover). Burstly traffic doesn't work. If you start your firefox with 20+ tabs open 80% of them will never load.

You will not enjoy the experience.

So yes, people think that adding 100ms of latency is better than dropping a packet under contention.

<comment><USER7> Your numbers are ridiculous. There's a huge gulf between buffering _millions_ of packets per switch port before a single drop, and a 1 in 1000 drop probability. You're also assuming that the drops are indiscriminate when a refusal to consider AQM and fair queuing is what led Arista to this absurdity in the first place, and you're presuming that latencies would still be astronomical in a world without massive queues.

A 10GbE network in a datacenter without bufferbloat would have RTTs orders of magnitude smaller than the 100ms queuing delay Arista considers acceptable; the effects of a congestion event would be ancient history by the time Arista's queues could drain. Even outside the datacenter, 100ms is a pretty long time for most connections in a managed-queue world. A congestion event on a device using fq_codel won't kill your DNS request or TCP handshake; it'll slow down an established flow and if you're using ECN you won't even lose a packet. It's only in a DDoS-like scenario of thousands of unresponsive connections (such as TCPs with a large initial window) beginning to transmit simultaneously that you'd see some flows getting unfairly penalized, but things would equalize within a few RTTs if the traffic was real TCP and not a true DDoS. You only see it take _minutes_ for a download's throughput to recover if you're going over multiple satellite links or through a severely bloated queue.

<comment><USER9> Okay make it one in a million. You will still be able to tell, and still see the phenomena I'm talking about.

</comment></comment></comment></comment></comment></comment><comment><USER10> I was the OP in the Whirlpool post. I use Mikrotik extensively to manage connections and have done for many years. This has only become an issue recently and it has happened enough times now for me to ascertain that it is when there is a Windows 10, or Office 2016 using the new Windows update doing its thing. I have tried to limit the issue by creating limited new tcp connections per second to any given IP address and even limit maximum concurrent connections.I have seen on occasion during this issue occuring a sudden huge burst of new outbound connections. I was thinking this would cause a type of DOS attack with thousands of SYN ACKS coming back.

The real kicker is that the connections are all to servers (Akamai) on port 80, so any serious blocking breaks all web browsing. The cynic in me says the whole Windows 10 update thing has been made to operate in lockdown environments when non-well known ports are blocked. Intentional, or not, the Internet is basically broken while this happens as Windows is ubiquitous and people all over the world who have successfully used inbound rate limiting to create successful shared Internet connections are going to be getting angry support calls. I hope my post goes viral so it starts to get seen by the likes of Microsoft and Akamai engineers. The local ISP I spoke to where I initially noticed this problem pretty much fobbed me off with the old "nobody else has reported the issue."

<comment><USER11> Well, the good news is that this is a temporary problem. There's only so many computers out there that will get the Windows 10 upgrade and presumably that number will drop like a rock as PCs either get upgraded or the users switch to Linux ;)

<comment><USER12> Windows 10 updates going forward will use the same method.

</comment></comment></comment><comment><USER13> Where did the HN submission title get UDP from? I don't see anyone in that thread suggesting that the updates were done in UDP, and all the traffic in the trace file is all TCP.

The trace is indeed a total mess, but I'm not convinced it's anything to do with TCP acceleration. There's absolutely massive levels of reordering and packet duplication happening in ways which are not consistent with TCP acceleration at all. It's much more likely that it's some kind of configuration problem elsewhere in the network.

From eyeballing the trace, almost half the payload segments there are duplicates, while a much smaller proportion are retransmits. (You can tell the difference e.g. using IP ids or by TCP timestamp TSvals / TSecrs).

<comment><USER14> We changed the title back to what the article says. The submitted title was "Windows 10 updates via UDP bypassing QoS restrictions".

Submitters: the HN guidelines ask you please not to rewrite titles except when they are misleading or linkbait. It seems like in this case the rewrite made it more misleading.

If anyone suggests a better title, we can change it again.

<comment><AUTHOR> Apologies for that, I was trying to summarize the linked article in the title, not to mislead.

</comment></comment></comment><comment><USER15> PCAP is at [https://forums.whirlpool.net.au/forum- replies.cfm?t=2530363&...](https://forums.whirlpool.net.au/forum- replies.cfm?t=2530363&p=3#r52). It's not UDP.

I'll have to see whether this is what causes my 100 megabit downlink to behave as if it's capped at 30 megabits sometimes. The router can barely keep up as it is.

<comment><USER11> I've had a problem similar to this with my 150 megabit downlink (sigh, wish I had that much upload). The thing to remember when troubleshooting these sorts of problems is this:

        Not all iptables rules are created equal.

State tracking has _significantly_ more overhead than other types of rules/filtering/shaping (even though state tracking is required for certain types of shaping). You may or may not have already done this but if not try this:

        iptables -t raw -A PREROUTING -i eth0 -p tcp -m tcp --dport 443 -j NOTRACK
        iptables -t raw -A PREROUTING -i eth0 -p tcp -m tcp --dport 80 -j NOTRACK

(replace eth0 with your Internet interface)

State tracking on port 443 and 80 is mostly useless and it's where you're likely to see the majority of your (high bandwidth) traffic. Setting NOTRACK on those ports can make a HUGE difference while still enabling your squirrel- powered router (let me guess: It requires no active cooling? haha) to shape traffic like "teh big boys."

<comment><USER15> For years and years I was on ~8 megabit ADSL. I thought 100 megabit fibre would solve all my problems but it just moves my problems into a different class.

Thanks for that info. As soon as I can work out why the default congestion control / single connection speed on FreeBSD 10 is so bad, I'll be running that as a router.

</comment></comment></comment><comment><USER16> I've actually seem the same thing recently - updates will soak up all available bandwidth, to the point where web browsing is basically impossible.

<comment><USER17> For clarification, Is it soaking up bandwith because its doing the (I forget the term) where it uses your connecion for other Windows 10 users while they update?

<comment><USER2> "Peer-to-peer transfer" is the term. You mean this hellspawn? [https://www.akamai.com/us/en/solutions/products/media- delive...](https://www.akamai.com/us/en/solutions/products/media- delivery/netsession-interface-faq.jsp)

I have seen my wife's Windows 10 computer upload crap from time to time and I suspected but could not confirm that it is this. How is this even possible, given that we're behind a NAT? TCP simultaneous open, I suppose?

I've not yet been able to figure out how to block these shenanigans. I _really_ don't appreciate MS/Akamai profiting off of my (rather limited) upload bandwidth.

<comment><USER17> Ah yes thats the term, I believe (not for downloading) but another MS app Skype uses the same thing when youre talking to someone on a lower bandwith/ADSL connection. I believe it uses the same TCP mechanism of some sort which I havent looked in to.

<comment><USER18> SuperNodes, which Skype used up until Microsoft bought them out (in May 2011) and stopped doing that (in April 2012)?

[http://arstechnica.com/business/2012/05/skype- replaces-p2p-s...](http://arstechnica.com/business/2012/05/skype- replaces-p2p-supernodes-with-linux-boxes-hosted-by-microsoft/)

<comment><USER17> Oh I see. I figured they still used them. Skypes Ui sure has suffered sadly.

<comment><USER11> It's not just the UI that suffered. The latency and performance of Skype calls dropped significantly after they switched from P2P to Microsoft Notification Protocol 24 (aka MSN Messenger Protocol)...

[https://en.wikipedia.org/wiki/Microsoft_Notification_Protoco...](https://en.wikipedia.org/wiki/Microsoft_Notification_Protocol)

They also introduced serious security problems when they made that change... So instead of Skype messages going from one client directly to another they go through Microsoft's servers (where they are stored and intercepted by TLAs) _unencrypted_.

They also introduced a new "feature" whereby their systems read everything you write:

[http://www.h-online.com/security/news/item/Skype-with- care-M...](http://www.h-online.com/security/news/item/Skype-with-care- Microsoft-is-reading-everything-you-write-1862870.html)

</comment></comment></comment></comment></comment></comment></comment><comment><USER19> It's interesting how on the forum this is totaly not about Windows 10, but more about Akamai and here all people are bashing on Windows.

<comment><USER10> Sadly my findings are that it is the new Microsoft update protocol and Akamai in combination.

</comment></comment><comment> thomas-b I've seen on multiple win10 laptops where it just get all download bandwidth (no noticeable upload) to a point where websites won't open and skype looses connection. That's on a 5Mb line. I do see many connections opened by a single process on Microsoft IPs mentioned.

I believe it was really only update related but I saw it happen when actually no updates were available. I just ended up limiting the corresponding process bandwidth whenever it gets annoying.

One into the other, I'm mainly just very surprised this kind of thing can happen. Except that I'm fairly happy with win10 though as opposed to the usual MS bashing we can hear. One thing into

</comment><comment><USER20> A key part:

" _It was the same range of source addresses and this was with Windows server and then Office updates. What seems to be happening is that instead of the sending server reducing its window size when packets are dropped, it just keeps re-sending large windows, which are obviously being dropped at my end. The queue algorithm has no idea of this and it will be letting packets through at a rate it thinks is correct, so the flow continues even though much of the traffic is dropped. However as the traffic keeps coming, the link is totally saturated._ "

Translation: someone broke TCP flow control.

</comment><comment><USER21> Windows has a feature to perform _low_ priority downloads of updates called Background Intelligent Transfer Service: [https://technet.microsoft.com/en- us/library/cc776905(v=ws.10...](https://technet.microsoft.com/en- us/library/cc776905\(v=ws.10\).aspx)

There's also the Windows Update Delivery Optimization P2P feature: [http://windows.microsoft.com/en-us/windows-10/windows- update...](http://windows.microsoft.com/en-us/windows-10/windows-update- delivery-optimization-faq)

<comment><USER2> The problem with BITS is that it assumes that the computer it's on is the only computer on the network. (In a switched network, how could it know otherwise?) So it will start bulk downloads while other users are trying to use the network interactively, with predictable consequences.

I basically just configure my network as if everything connected to it – both clients and the ISP – are bad actors trying to DoS me (albeit, a DoS regulated by TCP Vegas). Between my ISP's bufferbloat and these shenanigans from MS/Akamai, it's not a bad approximation.

<comment><USER21> IIRC BITS used Vegas-like (or these days we'd say uTP-like) heuristics to detect cross traffic and back off, but who knows what the latest version is doing.

</comment></comment></comment><comment><USER22> This isn't a new problem for us VOIP'ers. Bad actors have been breaking TCP for a while. We solved by putting a great big server/router up at a datacenter with a fat link. Any connection that misbehaves by ignoring TCP drops or flooding with UDP for more than a few seconds gets rerouted through that server. (New connections are made through this route.) It has _outbound_ rate limiting down the pipe to the local networks.

Keeps the local networks happy and fast. Isn't as expensive as routing everything through a datacenter because only misbehaving IP's get rerouted. Had the added side benefit that I could "protect" offices from the "involuntary" win10 upgrade.

</comment><comment><USER23> New phenomenon? I think it was first VOIP apps I ever used with Windows 3.11 which already had some protocol level tricks to work around TCP limitations. Like using ICMP and or UDP traffic. Hardly news. I've been thinking this since reading about TCP for the very first time. I've even published a concept of "Bandwidth Hog", which is transfer protocol designed to optimize your transport. By stealing others bandwidth, aka not sharing it fairly. [http://www.sami-lehtinen.net/blog/bandwidth-hog-quick- concep...](http://www.sami-lehtinen.net/blog/bandwidth-hog-quick-concept- draft)

<comment><USER2> It's "new" in that it's been deployed by the most widespread desktop OS, so networks _without_ compromised machines or bad actors have to deal with it now.

</comment></comment><comment><USER24> Ah, this looks like using multiple TCP streams to counter latency based throughput. Its not really new, its used in VFX to shuttle large amounts of data about.

</comment><comment><USER25> Is it not OK to criticize Microsoft on HN? This title was changed to remove any mention of Windows. Just curious.

<comment><USER26> I think it's that they (HN mods) like to use the original title.

<comment><USER12> Especially when the submitted title contains relevant context for the article. Can't be having that.

<comment><USER27> The submitted title was just plain wrong.

<comment><USER12> After looking at what the original title was, I agree. In this case it needed to be changed but just dropping the part about UDP would have been fine. In most cases though it honestly seems like when HN titles are changed it's changed for the worst. It looks like I'm not the only one who feels that way either.

[https://news.ycombinator.com/item?id=4102013](https://news.ycombinator.com/item?id=4102013)

<comment><USER27> Go ahead and fight it in those cases, but this is not at all good example.

</comment></comment></comment></comment></comment></comment><comment><USER28> Really, pushing Windows 10 has now become so urgent we can't let TCP slow us down?!</comment>


Apache Kafka Applications Can Work Without Apache Kafka Cluster? - <AUTHOR> https://dev.to/azure/apache-kafka-applications-can-work-without-apache-kafka-cluster-3en5 <comment> st-isidore I found it easy to use Event Hubs for Kafka for simple producers and consumers, but quickly realized that EH for Kafka doesn't support the Kafka Streams API after trying to do something more complex (yet still simple enough not to want the added cost of an entire Spark or Databricks cluster).

<comment><AUTHOR> many significant updates are coming for EH, definitely a thing to watch</comment></comment>


What Will California Do with Too Much Solar Power? - <AUTHOR> http://ww2.kqed.org/science/2016/04/04/what-will-california-do-with-too-much-solar/ <comment><USER2> So here's the question. How much time would it take the Gigafactory to produce enough storage capacity to store all that excess sunshine so we can start turning off the natural gas plants?

</comment><comment><USER3> towards the end of the article you get an interesting question. If you have the State of California mandating that the energy is 50% renewable but share energy with other states how do you account for the grey area especially if the other states are burning a huge amount of coal.

Secondly slipped in there is that we're rejecting free renewable energy because of the way the natural gas contracts are setup. That's a travesty right there, shouldn't let business in the way of free energy imo. Of course most likely when those contracts were signed there was no huge solar buildouts imagined.</comment>


OTPW – A one-time password login package - <AUTHOR> http://www.cl.cam.ac.uk/~mgk25/otpw.html

<comment><USER2> Very similar to Steve Gibson's Perfect Paper Passwords scheme: <https://www.grc.com/ppp.htm> I wrote a Java implementation for phones ([http://www.jgc.org/blog/2007/10/java-client- implementation-o...](http://www.jgc.org/blog/2007/10/java-client- implementation-of-steve.html)). This was my one and only foray into program for J2ME. I suspect that's something you only do once unless someone pays you a lot of money.

</comment><comment><USER3> This would be a great feature in a lot of web-based apps. Generate a few hundred passwords when you are on a secure connection, use them when needed.

Edit: I found a few implementations.

<http://wordpress.org/extend/plugins/one-time-password/>

<http://alexking.org/blog/2008/06/27/phonefactor-10>

<http://henrik.schack.dk/yubikey-plugin/>

[http://blog.fastmail.fm/2008/07/21/one-time-and-sms- password...](http://blog.fastmail.fm/2008/07/21/one-time-and-sms-passwords/)

<http://squirrelmail.org/plugin_view.php?id=276>

Yubikey looks interesting and they also have a web service API.

<http://www.yubico.com/products/yubikey/> <http://www.yubico.com/developers/api/>

MyPW is another web service: <https://www.mypw.com/>

OpenOTP supports both hardware tokens (also supports Yubikeys) and all kinds of phones.

<http://www.rcdevs.com/products/openotp/>

<comment><AUTHOR> Before you trust Yubikey, you might want to read about its weaknesses:

<http://security.dj/?p=4>

</comment></comment><comment><USER4> So this is basically solve the same problems as <http://en.wikipedia.org/wiki/SecurID>, but using a paper sheet instead of a hardware/software generated key?

<comment><AUTHOR> They are similar, but each has its advantages and disadvantages. According to the OTPW site:

 _"Admittedly, the security obtained by OTPW is not comparable with that of a challenge-response system in which the user has a PIN protected special calculator that generates the response. On the other hand, a piece of paper is much more portable, much more robust, and much cheaper than a special calculator. OTPW was designed for the large user base, for which an extra battery-powered device is inconvenient or not cost effective and who therefore still use normal Unix passwords everywhere."_</comment></comment>


The Dutch revolution in journalism: all newspapers behind one paydike - <AUTHOR> https://medium.com/p/a2031594e430

<comment><USER2> I'm one of the few lucky ones with a beta account. I love being able to read articles from different news papers without having to buy them all. Articles usually cost about 15 to 25 cents and you can ask your money back when the article is not what you expect it to be. This is pretty useful because the moment you click on an article you pay for it. I haven't found this to be a problem for now but I guess people will sometimes click without noticing. Money is being refunded without questions asked.

The service works great on an iPad and the crew behind Blendle is really responsive to questions. Great work.

</comment><comment><USER3> This overcomes one of the big issues today - how can you charge for what others give away for free? Amazing that they got everyone on board, and that there are no regulatory hurdles. It takes real Chutzpah by the founders to think they could off something so audacious. Bravo!</comment>


Ask HN: Freelancers, how do you track your income? - <AUTHOR>

I've been freelancing for a few years now, and I never found any tool (or Excel magic) that could solve my problem.
I'm working 40+ hours and when I look into the numbers, I should be doing really good. But at the end of the month, I'm not even close to that number. Sure, I'll eventually get there, but I'd like to know how others are coping with this.
I'm doing fine for the daily expenses. But before I make big purchases, I have to spend hours to project the state of my finances in the coming months.
Anyone in the same situation? How are you coping with this? Is there any tool that can help out?
An example:
Right now I have a project that spans over 3 months. I'm 2 months in. So now I'm wondering: - How much per month am I making with this project, considering 30% was paid upfront? - The initial payment was cleared last week (delayed cheque). I should be noting that somewhere right? - Am I hitting my monthly target at all? Projects spanning on several months are very common. <comment><USER2> I highly recommend using an accounting package, even if it's just quickbooks online. it helps so much when it comes to tax time.

For my business we use Xero so we can track expenses against the income for tax purposes.

Are you tracking hours so you can confirm your hourly rate? I use a tool called Toggl for that although there are others.

you can look at your bankstatements and focus on your cash flows in, then project those out.

Also, try to get 50% upfront wherever possible, especially if you have to pay others. Because that last payment seems to drag out a while....

</comment><comment><USER3> Let me give you an example.

Assume, your monthly target is $1,000. Now, you are taking 30% upfront money. So, if you able to get $3,500 projects a month then you can able to get 30% upfront money that equals $1,050. So, you can fulfill your target.

Here the key is buffer projects and you need to set milestones with buffer timeline. So, once you get the first 30% then target the next milestone or 30% to get within 20 days time. Then, even if client delays you still have buffer time.

Hope this will help you.

</comment><comment><USER4> I can't speak to whether or not it's good, but I think [http://cushionapp.com/](http://cushionapp.com/) is a really nice idea in terms of helping freelancers track and manage cash flow. Wish I'd thought of it :o)

</comment><comment><USER5> Quickbooks online self employment edition. It tracks income, expenses, and does quarterly taxes for you.</comment>


Ask HN: Could you build an entire web app in just one language? - <AUTHOR>

I was tutoring someone in web app development recently, and the monumental task in front of him really hit me.  He was trying to learn and use at least 9 "languages" at the same time.
This was just one possible stack, but in our case it was.
Ruby - Rails - MySQL - Bash (command line usage) - CSS - HTML - Javascript - JQuery - Git - (Capistrano, Yaml, nginx, ??)
Even though these aren't all true languages, they appear this way to newcomers since they are each a new syntax to learn.
If you slowly built up these skills over 15 years, they are clearly separate concepts in your mind.  But for a newcomer trying to use them, it's not even clear which are which.
Is that a Ruby method or a Rails method?
Is "script/server" a shell command or is "ls" part of rails?
Is <a href="..." onclick="..." style="..."></a> html, js, or css?
He made a comment along the lines of "wouldn't it be great if you could build all the pieces of a web app in one language", and I started thinking about it.
GWT (Google Website Toolkit), ActiveRecord, CoffeeScript, and Heroku are all steps in this direction.  You could classify them generally as trying to "eliminate a language in the stack" or allowing you to do a piece of the stack in a language you already know.
Obviously there is a trade off here between power and simplicity, but I'm wondering - would it be possible or desirable to get an entire web app down to just one language?  If not that how few could you use?
Btw, I think there are benefits to seasoned developers here as well.  I remember Lars Rasmussen (creator of Google Maps and Wave) mentioned something to this effect at Google IO in 2009, that GWT allowed him to spend his mental CPU cycles at a higher level and be more creative (not having to worry about cross browser css or js).  So the benefits may not only be for newcomers. <comment><USER2> This is exactly why JavaScript is going to win. I'm using JavaScript back to front using CouchDB -> NodeJS -> client-side JavaScript. The cognitive overhead of switching from language to language is pretty much gone. Being able to serialize and deserialize objects in the same language is pretty cool. Doing REST calls to Couch spits out JSON stringified data, then a JSON.parse()->JSON.stringify in the middle, and then another JSON.parse() on the client.

<comment><AUTHOR> This is pretty awesome and I think you're right. If we had to pick one I really wish it wasn't javascript :) But having just one would be very powerful.

</comment></comment><comment><USER3> There's also a language/tool called Monkey Coder which attempts something along these lines; though it's more aimed at creating apps for the different kind of platforms (iphone, android, html5, mac/pc/linux app, etc). I haven't played with it yet, so can't comment, but if you're interested, check it out here: <http://www.monkeycoder.co.nz/>

</comment><comment><USER4> Some of the lisp style stacks get very close as do some smalltalk environments (squeak). Of course this means generating the markup from code but seems to work well enough.

HN is an example (see hn challenge) as is teepeedee2 (<http://john.freml.in>). Maby clojure is worth a look.

</comment><comment><USER5> There's XAML, which brings it down to just three languages (Batch, XAML, and C#), but I would not recommend Silverlight for web applications at the moment.

</comment><comment><USER6> Pretty much anything significant will need SQL, even if Node.JS includes databases, so you write it in JS I'd rather do the sql myself.

</comment><comment><USER7> Well nodeJS is doing a decent job by having the same language on the server side as on the clientside.

css and html are both markup-lanugages and I find it very strange to add turing-completeness to a markup language.

well there are examples where this IS the case, look at LaTeX for example.

But having html as the frontend, and both clientside and serverside javascript as the backend is the closest that we can get at this moment.

<comment><USER8> There are JavaScript UI libs that handle the HTML/CSS for you like ExtJS. You'd still need to at least supply a single HTML file that just loads the js files though, but that hardly requires knowing HTML/CSS. After that the whole presentation layer is pretty much defined with JS and JSON.

That's in theory though -- in practice I wouldn't expect to get very far before not knowing html and css became a significant hurdle while developing a web app.

Anyway, these days I wouldn't even suggest building web apps as a starting point for someone's programming related career; it's just too big of a mess of languages, frameworks, and libraries. I think just about any academic curriculum aimed at application development starts off with basic console applications, works up to GUI apps, and then goes to the web. By then hopefully you see some patterns emerge that make learning new tech easier on you.

Then again, ironically, I did start to learn programming via the web through PHP, so I guess it can be done. The thing that helped me was that I already had a few years of experience working with static HTML, and I understood the problems PHP was trying to solve and felt the need for a solution myself. This was in the late 90s or early 2000s though, things were still a lot simpler back then than they are now. Websites didn't absolutely depend on JavaScript, you didn't have to know anything about AJAX, seeing CSS was fancy since having style information in your html was still common.

I don't know, if I think about learning all of it at once from scratch these days I want to vomit and have a seizure. So yeah, to answer the OPs question, I guess I do see how the benefits of unifying and abstracting a lot of the pieces out might be useful.

</comment></comment><comment><USER9> This whole site is written in arc. period.

<comment><AUTHOR> ...with html, css, sql, a deployment solution, and some handfuls of js.</comment></comment>


Ex-Ars Technica tech reporter Peter Bright convicted of soliciting minors - <AUTHOR> https://nypost.com/2020/03/19/ex-conde-nast-tech-reporter-peter-bright-convicted-of-soliciting-minors-for-sex/ <comment><USER2> What's the point of pulling Ars Technica's name through the mud for something one of its reporters did outside of working for Ars? Seems unnecessary unless Ars made a stink about vetting its people and claiming the most perfect reporters... this guy is gross and did a sick thing and is being punished let's not take down a good publisher with it</comment>


NJ Inventor Builds World's 'Worst' Alarm Clock - <AUTHOR> http://www.youtube.com/watch?v=KX7ZenkgxiE&feature=player_embedded

<comment><USER2> I built a similar tool. I wrote a GTK app that runs on my desktop machine and puts an application indicator into the panel, from which I can set alarms and turn them off. It connects to a netbook in my bedroom which, via cron, plays a sound repeatedly until I get up, go to my main PC in the living room and turn off the alarm. However, I've found that a lot of the time I'll just go straight back to bed...</comment>


Moving Camera, Moving People: A Deep Learning Approach to Depth Prediction - <AUTHOR> https://ai.googleblog.com/2019/05/moving-camera-moving-people-deep.html <comment><USER2> The best part of this paper is that they used mannequin challenge videos as their training dataset. That's super clever.

<comment><USER3> I have seen people suggest that the "10 year challenge" was created to build an age-related training dataset. While the mannequin challenge was probably just spontaneous, I wonder if we will see an increasing number of viral challenges in the future that center around the creation of structured information.

</comment></comment><comment><USER4> You can tell that the authors have a very fast internet connection by the fact that this website weights in at 91.6 Mbyte and takes over a minute to fully load on a 25 Mbit connection.

<comment><USER5> Jesus. Why couldn't they use embedded video files instead of 30 megabyte gifs?

<comment><USER6> A state-of-the-art deep learning neural net designed by digital video experts within one of the most technology savvy companies in the world...

What do they use to reveal it to the world? GIFs!

</comment></comment></comment><comment><USER7> Always worth looking at a point cloud versus a disparity map.

Grayscale disparity/depth maps are somewhat misleading - the large regions of constant intensity suggest that the algorithm is good at segmenting areas of constant depth. However, the flickering in the map suggests that if you actually tried to plot this in 3D, it'd be pretty noisy. Not to disparage the result, but 2D depth/disparity maps tend to look better than what they represent.

You can see this in the synthetic camera wiggle video, focus on the actor's hands, for example.

You can also see this effect in the Stereolabs Zed promo video.

<comment><USER8> to visualize depth maps it is best to look at derivatives of them (e.g., a directional derivative or the laplacian). Mapping the depths to indensities directly loses a lot of information.

</comment></comment><comment><USER9> I can't wait until techniques like this find their way in to open source photogrammetry pipelines. I came up with a way of training neural nets for robotics using a monocular camera, photogrammetry, and a simulation environment with the captured 3D scene, but the photogrammetry was error prone and computationally intensive even on a beefy cloud server.

I'd love for OpenSFM or OpenMVS (check github) to get this kind of software.

Also would love to see an implementation of this on github, but hopefully that will follow in time.

<comment><USER10> I personally do not believe that depth generated purely from deep learning can be used as input to photogrammetry anytime soon.

Photogrammetry works exceedingly well because the depth maps that they generate are quite precise and accurate, and mesh reconstruction usually assumes that these points are quite close to ground truth.

Deep learning approaches usually have medium accuracy but low precision, which causes the flickering and smooth surfaces that you see on the person. Even the background has flickering despite being computed through stereo, likely because the camera motion is primarily forward-backward (vs. more accurate side-to-side motion), the baseline is likely small, and the depth isn't globally optimized.

This type of research is super great for applications requiring lower accuracy, typically visual-only applications (e.g. selective blurring, faking stereo on a frame, etc.). But as an input to photogrammetry — probably not anytime soon, until the problems above get resolved.

<comment><USER9> Interesting. Perhaps my idea of this being inserted in to existing algorithms would not work.

However I do ultimately seek a low accuracy "visually approximate" 3D scene that I could use for simulation purposes. I guess I could rephrase my desire as: I'd love to see this kind of approach used to train an end to end deep learning photogrammetry system. I feel like the parallel nature of neural nets as well as their ability to approximate results could result in a much less computationally intensive solution to my photogrammetry desires.

(I want to train my four wheel drive robot to follow forest trails using the training method described in the "world models" research paper, which requires a simulation to work.)

<comment><USER10> Some of my friends recently put out [http://gibsonenv.stanford.edu/](http://gibsonenv.stanford.edu/)

Full simulation with realistic 3D spaces, enables embodied agents to interact and learn from real-world spaces. Not forest trails, but a real world environment.

If you really want to create a 3D model of forest trails, photogrammetry should be sufficient, because forest scenes are richly-textured.

<comment><USER9> Yes I did come across Gibsonenv and it looks great for indoor scenes.

As far as photogrammetry of forest trails, I found it to be very computationally intensive (taking a GCE 32 core instance 30+ hours using 90+GB of ram to compute a scene, only with errors that made it unusable). It felt very heavy handed and given all the great work I've seen in scene understanding using neural nets, it seems like deep learning would be a promising approach here. Maybe there is commercial photogrammetry software that has better pipelines, but I want to be able to compute my scenes on linux and use hundreds of images.

I did my computation with OpenSFM and OpenMVS. Both wonderful projects for being free and open source. I did get a lot of great results. But I am convinced a simpler way is possible with deep learning.

<comment><USER10> OpenSFM is quite out of date, so it's quite inefficient and rather inaccurate (e.g. exhaustive matching is O(n^2), and there are a lot of smarter ways that are closer to O(n))

Also, one of the main steps of mesh reconstruction is depth map generation. It typically takes anywhere from 30-75% of compute time for dense reconstruction, _IF_ it's parallelized thru GPU. If you're using the CPU only to calculate depth maps, you're probably slowing yourself down by an order of magnitude.

If you have a GPU, and use a better SFM-MVS solution, then you can quite easily reconstruct datasets of 1k-10k images within 24 hours.

<comment><USER11> What would you recommend as a better SFM-MVS solution?

</comment></comment></comment></comment></comment></comment></comment><comment><USER12> Why wouldn't they use 3d renderings as a large part of their training data set? You could have perfectly generated depth outputs generated alongside the image input, and you could adjust things like focal length to all kinds of values that would make this able to understand how shifting items correlate to depth across a variety of focal lengths. To be honest I'm not even sure how they're training them with live footage, how are they even getting the depth maps from the training footage to begin with?

<comment><USER13> > we make use of an existing source of data for supervision: YouTube videos in > which people imitate mannequins by freezing in a wide variety of natural > poses, while a hand-held camera tours the scene. Because the entire scene is > stationary (only the camera is moving), triangulation-based methods--like > multi-view-stereo (MVS)--work, and we can get accurate depth maps for the > entire scene including the people in it

I suspect the reason for not using 3D rendering is the desire to cope with the noise and variability of real video.

</comment></comment><comment><USER14> Here is the actual paper [https://arxiv.org/pdf/1904.11111.pdf](https://arxiv.org/pdf/1904.11111.pdf)

</comment><comment><USER15> Could this technology one day become so good as to eliminate the need for lidar for self-driving cars? Or will lidar be so inexpensive by that point that there will be no need to eliminate it?

</comment><comment><USER16> This is a great hack, but I'd love to see more detail on how they did pose initialization to approximate ground truth on depth/pose from the Mannequin set. The paper says they are using ORB-SLAM2, but AFAIK ORB still needs a height label.

Maybe it's the case that this system doesn't actually return an X,Y,Z camera pose, but rather just a pixel specific depth, and not a recovered pose for new inputs.

</comment><comment><USER17> The predictions seem to have rapid flickering, which means the model is saying lots of items are moving back and forth extremely quickly. Since this seems common in video analysis (rapid changes per frame) is it that smoothing or taking into consideration multiple frames is slow? Or does it cause more issues than it solves?

</comment><comment><USER18> DeepMind seems to be conducting research in a similar direction. Any insight on how are the two projects related or different?

[https://deepmind.com/blog/neural-scene-representation-and- re...](https://deepmind.com/blog/neural-scene-representation-and-rendering/)

</comment><comment><USER19> At this point I feel like I'm psychic. Every single time I see an image processing project posted on here I think to myself "I bet the only examples are tiny low resolution thumbnails" and _every_. _single_. _time_. I'm proven right. Whyyyyyyy?

To be fair, this particular application doesn't _really_ need more to show it's improvement over other approaches, but still.

<comment><USER20> Because it's a lot less data to crunch for the network.

</comment></comment><comment><USER21> I'm not sure if this is due to different mapping into greyscale or is their method completely killing far distance details?

Compared to "Chen et al" which is a bit flickery in the foreground, but full of stable background details, their result is almost completely black 3m in.

</comment><comment><USER22> Can other companies use YouTube database for free say for research in Computer vision?

<comment><USER23> I think it's a gray area, but researchers often just do it. Better to ask for forgiveness than permission I guess. You could never collect datasets like ImageNet if you had to obtain individual permissions.

<comment><USER24> At least some jurisdictions have research exemptions in their copyright laws, so at least I don't need the copyright owner's permission to use any data for research purposes.

I'd still prefer to use explicitly open datasets because it allows for simpler data sharing and easier reproducibility, however in cases where that's not possible whatever is available will do even if I'm restricted in how I can redistribute that data.

</comment></comment></comment><comment><USER25> Seems related to the Tesla video-based depth perception work?

<comment><USER26> Tesla approach works chiefly on video scenes with static objects like parked cars.

They train a DepthCNN to infer depth from monocular images (lidar or stereo for supervision) and make sure it's temporally consistent by adjusting with pixel transformations from the previous and next frame using a PoseCNN. [https://arxiv.org/abs/1704.07813](https://arxiv.org/abs/1704.07813)

The guys at Google use Optical flow (only previous frame) to make sure their model trained on static object video sequences works when the scene is dynamic using a mask for a specific class an object (humans here). They do have to make sure nothing but humans are dynamic in the scene.

</comment></comment><comment><USER27> This can improve fake bokeh on smartphones to a pro-camera level quality

</comment><comment><USER28> Is it as good as LIDAR

<comment><USER29> I do not think so</comment></comment>


Show HN: A database of everything (over 55M keys) - <AUTHOR> https://www.outpan.com/ <comment><USER2> There's some more complex and curated projects out there, among them:

[https://www.wikidata.org/wiki/Wikidata:Main_Page](https://www.wikidata.org/wiki/Wikidata:Main_Page)

<comment><USER3> +1001 for Wikidata

compare [https://www.outpan.com/data?k=germany](https://www.outpan.com/data?k=germany)

to [http://netbase.pannous.com/html/verbose/germany](http://netbase.pannous.com/html/verbose/germany) (wikidata mirror)

or [https://www.wikidata.org/w/index.php?search=Germany](https://www.wikidata.org/w/index.php?search=Germany)

<comment><USER4> Netbase/pannous seems to be cool but I guess it still got some work to do as it's suggesting Bonn as capital of Germany, unless I'm misinterpreting the statements. Kind of weird result from Google when you search for: "bonn capital germany". For me, it displays the wiki summary of Berlin but the link goes to Bonn. Does it usually work like that?

Edit: It was the Bonn Summary of wikipedia

<comment><USER3> Bonn was the capital of Germany for many years.

</comment></comment></comment></comment><comment><USER5> Signup page is blanking out for me.

Really curious to try it. It's a really neat idea. Has this been done before? I've never seen anything like it.

The full value of this would likely come from interesting, productive, insightful visualizations of the underlying graph that is being built.

Questions that come to my mind:

\- What if you write bots that scrape Wikipedia, Twitter, etc. and output entries from semantic analysis performed on these sources?

\- If many people write such bots, how similar would the graphs be? What are the parameters that determine graph overlap?

\- Can you use this to tie in to the real world?

 _" A key can be any unique string such as product barcodes, book ISBNs, email addresses, URLs, domain names, names, phone numbers ..."_

Interesting stuff there... A way to make this into an interesting social network is if people curated their own graphs, e.g. of books and webpages and favorite restaurants, and other people could browse these graphs in a read only mode. (perhaps they can clone it to their graphs or so and then start to edit them)

The temporal aspect might be interesting too. Would there be value in seeing how my graph has changed over time? When I was in academia, none of the tools to keep track of the papers I read suited me. I could see this working well for this use case scenario.

(if you could write a Prolog against this, would it have interesting properties?)

I see that a submitter just posted 7630028603780 -> volume -> 75ml (the first number being probably the bar code of some beverage), that's neat.

Someone else just posted the following entry:

 _5010438013621 - > ingredients -> Water, sugar, mixed fruit juices from concentrate 10% (Grape, blackcurrant, raspberry), acid (Citric acid), Vimto flavouring (Includes natural extracts of fruits, herbs, barley malt and spices), colouring food (Concentrate of carrot, hibiscus), acidity regulator (Sodium citrate), preservatives (Sodium benzoate, Potassium sorbate), Vitamin C, Sweeteners (Sucralose, Acesulfame K)._

Now what the site should be doing is converting this to something like:

 _5010438013621 - > ingredients -> water

5010438013621 -> ingredients -> sugar

5010438013621 -> ingredients -> vitamin C

... _

(Outpan person/people, I'm in SF, if you want to chat more around coffee, there's an email is in my profile)

<comment><USER6> > Has this been done before?

Depends what you mean by "this"! RDF [1] and most of the technology surrounding it and the "Semantic Web" are based on (subject, predicate, object) triples almost exactly like this, where each element is often a URI, and objects are often strings just like they are here.

It even has taken this idea to the next level where the statements expressed by such a triple can themselves be given an "anonymous" ID, which can then be used as a subject or object – meaning you can make meta statements about the statement itself, all while still using this simple system of triples.

There are even entire languages built around querying graphs of such triples: [https://www.w3.org/TR/sparql11-query/](https://www.w3.org/TR/sparql11-query/)

DBpedia [1] is one such project that attempts to encode data from Wikipedia in triples like this; their About page says that the 2014 version of the database had 3 billion triples, so that number is probably much higher now. Here's a preview if you want to see what these triples look like:

• Homepages of things: [http://downloads.dbpedia.org/preview.php?file=2015-10_sl_cor...](http://downloads.dbpedia.org/preview.php?file=2015-10_sl_core-i18n_sl_en_sl_homepages_en.ttl.bz2)

• Genders of things: [http://downloads.dbpedia.org/preview.php?file=2015-10_sl_cor...](http://downloads.dbpedia.org/preview.php?file=2015-10_sl_core-i18n_sl_en_sl_genders_en.ttl.bz2)

etc. You'll notice that RDF predicates are all namespaced by URIs; that way you can unambiguously know in what sense "homepage" and "gender" are used (consider more ambiguous properties like "length"). That means there can be other uses of "homepage", "gender", "length" etc. that mean different things, and those will be namespaced by a different URI.

Anyway, this Outpan project is obviously a more loose and freeform version of that – but only slightly; RDF is not very strict at all, it's just that people have thought a lot about how to successfully model the entire world's information, and so real-world RDF ontologies end up looking somewhat complicated. I'm not sure if a freeform version like this has been widely attempted before.

[1] [https://en.wikipedia.org/wiki/Resource_Description_Framework](https://en.wikipedia.org/wiki/Resource_Description_Framework) [2] [http://wiki.dbpedia.org/](http://wiki.dbpedia.org/)

<comment><USER6> To borrow a subject matter that's currently popular on the Outpan homepage, here are the first 500 facts DBpedia knows about Donald Trump:

        SELECT DISTINCT ?property ?value WHERE {
            <http://dbpedia.org/resource/Donald_Trump> ?property ?value
        } LIMIT 500

Results: [http://dbpedia.org/sparql?default-graph- uri=http%3A%2F%2Fdbp...](http://dbpedia.org/sparql?default-graph- uri=http%3A%2F%2Fdbpedia.org&query=SELECT+DISTINCT+%3Fproperty+%3Fvalue+WHERE+%7B%0D%0A++%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FDonald_Trump%3E+%3Fproperty+%3Fvalue%0D%0A%7D+LIMIT+500&format=text%2Fhtml&CXML_redir_for_subjs=121&CXML_redir_for_hrefs=&timeout=30000&debug=on) (although, note, not every dataset they have is loaded into their SPARQL endpoint)

As you can see there's a lot of metadata type properties, but scroll down and you can see his birthdate, children, alma mater, etc.

This page is just a prettified version of that data: [http://dbpedia.org/page/Donald_Trump](http://dbpedia.org/page/Donald_Trump)

<comment><AUTHOR> This is great! thanks. I will look into adding the dbpedia.org data.

<comment><USER6> Check out [https://www.wikidata.org/](https://www.wikidata.org/) for another similar project with additional data! Their keys tend to be more opaque [1], but otherwise it's a very similar approach.

[1] e.g. the key for "Earth" is Q2: [https://www.wikidata.org/wiki/Q2](https://www.wikidata.org/wiki/Q2)

</comment></comment></comment></comment></comment><comment><USER7> does silicon valley now suddenly discover the semantic web and triplestores :O

if you want really all the data: [http://lod-cloud.net/](http://lod- cloud.net/)

but still, neat project.

<comment><AUTHOR> Semantic web is too good of an idea not too iterate on constantly :)

</comment></comment><comment><USER8> As much as I like the idea, I can already see someone called "Drumpf" on the front page linking attributes about Hillary Clinton and Donald Trump to their Twitter accounts and not their names. I guess if you're after "everything" then curation makes no sense but you'll end up with a bunch of nonsense.

<comment><AUTHOR> Twitter url seems like a strong key since it is referenced in many other contexts.

As for curation, it is intended to provide examples of what key, attr and values are regardless of their content. This is an experimental feature and might be removed/tweaked...

</comment></comment><comment><USER9> Great concept but I can see it falling out of line very quickly, on the homepage I spotted "berlin -> country -> germany" Followed by "england -> capital -> London"

If you search for the key "germany" it has no results, if you search "london" it finds no results.

The fluidity of the data is definitely a hindrance, if you wanted to use the dataset you'd have to already know what you are looking for to find the value.

</comment><comment><USER10> Is 55 million keys a lot for tracking "everything"? My expectation was that "everything" would need more than 55 trillion keys.

<comment><AUTHOR> "a journey of a thousand miles begins with a single step"

<comment><USER11> "a journey of a everything begins with a 55M keys"

<comment><AUTHOR> haha. We should use this as the project mantra.

<comment><USER11> I agree :-) you are welcome

</comment></comment></comment></comment></comment><comment><USER12> So it's RDF without the globally unique identifiers?

</comment><comment><USER13> So searches like "Linux" "California" "lincoln" "Hitler" "Disney" and "red" return 0 results

<comment><AUTHOR> I was driving so couldn't stay on the phone. The number of keys for different concepts is just very large (if not infinite for the sake of avoiding philosophical debates). It will take a some time to have enough data to cover even popular keys. Fortunately I have a lot of time :D

</comment></comment><comment><USER14> What sort of backend storage does this use?

<comment><USER15> Given that it's effectively key->key->value, I'm guessing Cassandra for the main backend. The data model fits very well, and it would give you the kind of scalability you would need for this sort of thing.

<comment><USER16> Given how ridiculously basic the website is, I think you're right. It should have been a real triplestore with a SPARQL endpoint though.

<comment><USER17> There are whole databases built around the concept: [http://www.datomic.com/](http://www.datomic.com/)

<comment><USER16> I browsed the website, but couldn't grasp what is the main value of that product or how could it be useful for the showcased website. Could you please educate me a bit?

</comment></comment></comment></comment></comment><comment><USER18> Is it possible to lookup all the attributes for a given value, say Trump?

</comment><comment><USER19> Shame it fails for the trap of natural language ambiguity. So on the front page I see "England -> capital -> London" and at a glance I thought the capital (as in money) flows from England and is accumulated in London.

<comment><USER20> I agree in general (and it's the fault of freely user-definable keys/attribute) - but that's not a great example, since the arrow aren't read as a direction of flow.

Perhaps that reveals that a forward slash might be a better separator though - like a URI.

</comment></comment><comment><USER21> The header looks like fastmail :)</comment>


Ask HN: Must a company be a Corporation to award equity? - <AUTHOR> Are the only types of corporate structures that allow equity allocation Corporations? Or would this type of offer work with LLCs? <comment><USER2> There are ways of awarding "equity" in LLC's as well, but it works differently than a traditional S or C corp. Even in a S corp you are limited to how many people can have shares before you are forced to convert to a C corp but you are still at least dealing with shares.

In an LLC you do not have shares, you generally have "units" which equate similarly but are geared towards "profit interest in the business", which usually ties back to a capital contribution but doesn't necessarily tie to cash.

IANAL so you should seek one out if you are trying to do anything along these lines. Anytime you set up a business pay the money for an attorney to go over it so you have the correct information and structure, it is so worth it and doesn't cost that much in the long run. And fixing things later can cost way more.

Personally, I always prefer LLC's as I don't seek venture funding for my own companies, and even if you do raise some money from friends/family or high net worth people an LLC is generally fine in these cases. If you plan to raise funds from large Angels or VC's always setup the proper C corp from the beginning and use an attorney that has done it before. If you aren't sure, you can always convert an LLC to a C corp, just tell your attorney that it is possible you will do this. That way they can include the proper language in the operating agreement to make that conversion as tax neutral as possible.

<comment><AUTHOR> Thanks for the detailed answer!</comment></comment>


Getting back to the core - <AUTHOR>

"Kozmo started out as a business dedicated to getting consumers what they needed every day," he said. "It morphed form that into being superstore for everything you might possibly want, from books to CDs, and having deliveries as a tack-on. We're going back to the core." ~ http://money.cnn.com/2005/09/14/technology/kozmo_redux/index...
Some of the failed dot-com crash-outs from the late 90's had some interesting ideas that might have worked had they stuck with the core of the original idea? Can you think of any others? My case in point is a re-born kozmo as maxdelivery. The difference? (aside from taking $USD230M from VC's) ...

    * charges for delivery reflect reality ($10 minimum order, and there's a $4.95 delivery charge for orders under $50)
    * simplifying inventory (focus stays on DVDs, groceries and other daily needs)
    * no VC's (company isn't beholden to investors who are more focused on growth than profitability)
    * sticking to favourable geography with high living density of
 Manhattan NYC (instead of having physical presence in 15 states)
Chris Siragusa has obviously learned his lessons the hard way. Do you know of other failed sites that could have been successful getting back to the core? <comment><USER2> Kozmo was great while it lasted in Boston. Movie & candy delivery right to Bexley on the cheap! The demand was there at their original low prices--not sure what it would be farther up the demand curve, i.e. if the price was significantly higher. I don't think the $10 minimum would hurt much (students can pool), but the $5 delivery charge might.</comment>


Tor Open to Attack! (pdf) - <AUTHOR> http://www.cs.colorado.edu/department/publications/reports/docs/CU-CS-1025-07.pdf <comment><AUTHOR> If any of you guys are familiar with anonymous Internet systems, this is a really cool read. It basically outlines a possible attack against something like Tor by linking together paths of entry/exit to trace (identify) a user.

Tor was one of the items covered in a presentation on online identities that I did last week here on my campus.

<comment><USER2> How many nodes would have to be compromised to make this a viable attack vecotr? (yeah, yeah, I know, I should RTFM)

<comment><AUTHOR> Without going into the detail of the article (I wouldn't want to spoil it for you!): a lot.

But you know how it is. A single proof of insecurity is all you really need.</comment></comment></comment>


Squarespace Developer Platform Beta - <AUTHOR> http://developers.squarespace.com/ Squarespace's developer platform is now in beta, featuring Git and SFTP access, total template code control, less.css on the server, JSON behind every URL, and more.
http://blog.squarespace.com/developer-platform-beta
Developers also get an infinite trial (free until you publish). <comment><USER2> Very cool! Can't wait to play with it. Been a fan of Squarespace for years.</comment>


Learn with Me: Containers from Scratch, Part 1 - <AUTHOR> https://psbarrett.com/blog/learn-with-me-containers-from-scratch-part-1.html <comment><AUTHOR> I fully expect that no one is really going to find this post interesting, but I've both been meaning to blog and meaning to learn how to deploy stuff with containers, so I figured I'd kill two birds with one stone.

This post is extra not interesting since it's entirely introduction with next to no technical information, but I thought it was an important place to start. I'm posting it here in the hopes that someone who knows more than me will correct me on anything that I got shockingly wrong before I get too far going down the wrong path.</comment>


Google command line (2010) - <AUTHOR> http://google-opensource.blogspot.com/2010/06/introducing-google-command-line-tool.html

<comment><USER2> What's new about this Friday, June 18, 2010 | 8:55 AM ??</comment>


Ubisoft CCO Serge Hascoet resigns in wake of sweeping allegations - <AUTHOR> https://arstechnica.com/gaming/2020/07/ubisofts-most-powerful-creative-force-resigns-in-wake-of-sweeping-allegations/ <comment><USER2> Weird to see this here.

I have limited knowledge on these things; FD: I work for Ubisoft in their Massive Studio.

Ubisoft itself is incredibly silo'd; each studio and each function operates like a loosely coupled bundling of companies.

You can think of it like:

* Marketing has offices (or colo's in some offices) but has a completely seperate reporting structure all the way to Yves.

* (same for other business functions such as server hosting, exchange administration etc;)

* Studios operate with an entirely different culture from HQ.

* Studios develop games, those games are submitted for "stage gates" which is when Editoriale can say if something is fun, or if something needs to be cut or added.. Editoriale in HQ can only speak during those times (big milestones). Otherwise the Studio doesn't really talk to HQ. (save: marketing, I guess)

I can say that Massive is incredibly progressive, this isn't just a result of being Sweden, even some (not all) of the Swedes who work here think we're overcorrecting for progressiveness. We had Sarkeesian on site a few times to consult on games (and paid her handsomely for the privilege)

However.. and this is a big however:

There is also a culture of partying in the upper echelons, self-promotion, congratulatory/promotional incest and an "untouchable ego" that a lot of the upper management seem to have, this means that people at the top stay there, enrich themselves and keep others out. This doesn't apply to Yves himself, strangely, I've met him many times and is a fantastically humble, down to earth and generous guy. In fact he's made many moves that are directly against enriching himself in order to ensure that nobody can get the idea of shuttering a studio and putting people out of a job- and has actively worked (personally) against crunch culture.

\--

What I'm trying to say is that: each "head" is responsible for the culture they create; and some people are given the freedom to abuse that culture.

Yves see's that ability to abuse as a failure of HR and that's why (I speculate) that Cecile has been let go.

As far as I understand Serge has had no allegations levied against him; however it's unconscionable how much power he had- and it would have been easy to abuse accidentally (an off-hand comment/joke being taken as an instruction). Not saying he's not guilty of anything but that kind of power should not have been levied.

Notably the Editoriale team (gatekeepers for games) was recently expanded to be more diverse.. that diversity was all middle-aged white french guys who came from the same school and had worked in that department before.. so.. not very diverse.

<comment><USER3> I was just thinking "Huh, I used to work with someone who went to work for Massive" until I realized you were the same someone :) Interesting to hear how the Massive/Ubi editorial relationship works from the inside

<comment><USER2> Oh interesting! Hello friend! it's a small internet!

Now I will wonder who you are for weeks! xD

</comment></comment></comment><comment><USER4> I seem to recall Ubisoft working with a number Sarkeesian-type consultants in the years following gamergate. Seems like a case of the guiltiest being the first to accuse if I remembered that right, but I don't recall where I got the idea. Does anyone here know where I might have got that impression?

<comment><USER5> I don't think Sarkeesian could even possibly be the problem if Ubisoft HR changed their definition of harassment to allow sexual harassment and non- gamer journalists are reporting on it.

Companies have "woke" consultants in to make speeches all the time but it's performative and it never changes anything.

<comment><USER4> I'm more wondering about the value of woke consultants as an indicator - if a company is very concerned about making such performances, could it be an indicator of a guilty conscience?

<comment><USER2> I can only speak for my admittedly hyper-progressive studio: We brought Sarkeesian in to find our blind spots. (there was also the fact that the studio wanted to fund her work on a more personal level).

But if you consider that roughly 86% of the studio at that time was male, it would have been easy for something to slip through that could have been perceived as anti-women.

A great example is in the game Hitman; there is a moment where you're infiltrating a strip club and an option exists (although, is penalised) for murdering a scantily clad stripper. This was caught by Sarkeesian and used as an example of poor game development that allows for the objectification of women. (or, women as background objects)[0]

We didn't want that.

[0]: [https://youtu.be/4ZPSrwedvsg?t=603](https://youtu.be/4ZPSrwedvsg?t=603)

</comment></comment></comment></comment><comment><USER6> M

</comment><comment> drevil-v2 I am deeply troubled that we have normalised public executions on the basis of allegations. No person or group should have that power to effectively end someone's career on accusations alone.

<comment><USER7> I'm more worried about the 95%+ of sexual assaults that perpetrators get away with.

<comment> drevil-v2 Citation needed for that "95%+" claim. And While you are at it, can you spell out what exactly you mean by "sexual assault"? It seems to me that the meaning has been corrupted in order to further political and ideological aims of the accusers and executioners.

<comment><USER7> Yes no problems it's all out there.

19% of women report sexual violence to police. (1)

10% of those reports result in a conviction. (2)

So actually I was being conservative, it's actually

98%+ get away with it based on these two sources.

And we haven't even got to child sexual assault conviction rates.

(1) [http://www.ausstats.abs.gov.au/ausstats/subscriber.nsf/0/056...](http://www.ausstats.abs.gov.au/ausstats/subscriber.nsf/0/056A404DAA576AE6CA2571D00080E985/$File/49060_2005%20\(reissue\).pdf)

(2) [https://www.abc.net.au/news/2018-11-14/why-do-so-few- sexual-...](https://www.abc.net.au/news/2018-11-14/why-do-so-few-sexual- assault-result-in-convictions/10492256)

<comment><USER8> > 98%+ get away with it based on these two sources.

Your first stats are 15 year old and are only applicable to Canada. You can't make broad generalizations like that. Different countries have different culture and very different legal systems.

<comment><USER7> Thanks for your reply.

I'm not sure who you were replying with as the cited data was not from Canada and was not 15 years old.

If you have a more nuanced reply with better data sources please do so.

</comment></comment></comment></comment></comment></comment><comment><USER9> Can people not just work somewhere without sexually abusing staff please?

<comment><USER9> Apparently not, according to the downvotes. I've noticed the quality of commenters and submission decline on this site of late. Keep it coming /s

<comment><USER10> The downvotes I don't think indicate others believe that it's okay to just sexually harass people. In fact, I'd venture to say that people here would be in nearly unanimous agreement that it's not okay. With that in mind, your comment to me appears to either be seeking in-group approval ("does anybody else?" When yes, almost everyone else) or is an attempt to be funny. All this to say, neither of those things are really valued here. Hence, downvotes.</comment></comment></comment>


India: WhatsApp 'child kidnap' rumours lead to lynchings - <AUTHOR> https://www.bbc.com/news/world-asia-india-44435127 <comment><AUTHOR> Sometimes the cost of misinformation is tangible, this is sadly one of those cases. I launched [https://www.quod.us](https://www.quod.us) as a way for people to track and catalog misinformation, and I've been following this story for the last week.</comment>


Bread & Circus: Wikileaks makes the TSA debacle go away - <AUTHOR> http://www.google.com/trends?q=tsa,wikileaks&ctab=0&geo=us&date=mtd&sort=0

<comment><AUTHOR> I know, this has nothing to do with hacking.

Still, given the amount of threads dedicated to TSA and Wikileaks on HN, I thought it'd be amusing to point out this little graph on Google Trends.

It fits extremely well with what we can observe on HN: hardly anyone is discussing the TSA issues anymore. Which begs the question: how aggravated are we with the TSA, really, if the scandal-du-jour can so easily makes us forget what we were passionately discussing just weeks ago?

<comment><USER2> Thanks for posting this. It demonstrates nicely what is a common phenomenon. I'm also sure it's frequently used in politics and elsewhere to to hide a scandal behind a seemingly more important but in fact less important (or untrue) event.

Nevertheless, in this particular case I think WikiLeaks indeed is the bigger story.</comment></comment>


Update on GoDaddy Transfer Issues - <AUTHOR> http://community.namecheap.com/blog/2011/12/26/godaddy-transfer-update/

<comment><USER2> I also ran into a problem with GoDaddy when trying to transfer a domain name that was protected by Domains by Proxy. I first got a notification that I needed to enter into a new agreement, so I did. Then I got a notification that I needed to cancel my private registration, so I did. Then I got this:

        "The transfer of MYDOMAIN.ME from Go Daddy to another
        registrar could not be completed for the following
        reason(s):

        Express written objection to the transfer from the
        Transfer Contact. (e.g. - email, fax, paper document or
        other processes by which the Transfer Contact has
        expressly and voluntarily objected through opt-in
        means)."

So it looks like they're auto-rejecting domain transfers if you're using Domains by Proxy?

<comment><USER3> I transferred two domains away from GoDaddy that were protected by Domains by Proxy. They did require that the WHOIS guard be dropped, even though both registrars offered a WHOIS-protecting service.

It did feel like GoDaddy was moving the goalposts.

Since I had no deadline and no feeling of urgency, I just allowed the entire process to play out naturally. There were periods when I was busier elsewhere and didn't press the issue for a month or two at a time, but I spent the better part of sixteen months getting both transfers completed.

Update: I bought a standard membership to DomainTools to see if the info ever leaked. It looks like the patient approach paid off, and that the exposed WHOIS info was not crawled during the transfers. Taking the time to make sure both ends of the transfer were prepared was worth the effort, but I think it's probably luck that kept the info from being picked up.

<comment><USER4> Domaintools may be the best-known whois database, but it isn't the only one and not all of them are as easy to check against.

</comment></comment></comment><comment><USER5> Devil's advocate here: The idea that godaddy appears to be intentionally stalling transfers is pure speculation. Not saying that they're incapable of doing it, but that whole "don't attribute to malice" thing.

Let's not turn this into another ugly internet lynch-mob. Just move your domains, and be done with it. Namecheap (and others) look like they're more than happy to help out all their new customers.

<comment><USER6> What's the motive to make this process painful? Would people really say Oh screw it, I'll just stay with GoDAddy? Ive never transferred a domain before so I don't know what the process or delay is normally like

<comment><USER7> This is standard practice to increase the friction of unsubscribing. How many clicks does it take you to logout of Facebook or Google...and see if your grandmother could even figure out how.

<comment><USER8> Two clicks. Google is easy: click your name, click sign out. Facebook is slightly harder: click the menu triangle in the upper right hand corner, click sign out.

<comment><USER9> I think he meant to say "close your account" rather than "log out".

<comment><USER8> Oh. That makes sense. My bullshit filter kicked in before I finished interpreting the post as a whole.

</comment></comment></comment></comment></comment></comment><comment><USER10> Yup, same issue here. Transfer has been hanging over 24 hours now. Edit: Transfer in Process - Acquiring Current Whois for Transfer Verification

<comment><USER11> I wonder if GoDaddy's WHOIS server is just applying rate limiting. I suspect their transfers are spiking after the SOPA aftermath. Namecheap's servers might have been sending a fairly consistent level of WHOIS queries that would not be anomalous but after the coupon and lots of press about Namecheap that threshold might have been exceeded several times over.

Now GoDaddy should have cleared up the WHOIS throttles by now. But I can understand there being a temporary issue with mass transfers like this.

</comment></comment><comment><USER12> Some definitive, unbiased proof of this needs to be seen before I can join the lynch mob. I have no doubt that GoDaddy is desperate and/or inept enough to do something like this but I also can't take their competitor's word for it.

<comment><USER13> Same. I hate hate hate GoDaddy, but Namecheap assuming malice in this case might blow up in their faces, and honesty, just for speculating (with a pretty sure-sounding tone) that GoDaddy is doing this on purpose, I hope it DOES blow up in their faces. This is really just dirty business.

Move to name.com instead because they don't practice this type of distasteful PR.

<comment><USER14> What if they are telling the truth?

<comment><USER13> Then that sucks for GoDaddy, but what I'm getting at is that they should make a call to GoDaddy engineering and say "Hey, what's up with this, are we being throttled or something?". The risk/benefit for this type of thing just doesn't work in favor of Namecheap. Posting what they did, if GoDaddy is indeed dragging their feet on purpose, they'll be able to say "we told you so!" which isn't worth much. If GoDaddy is NOT intentionally delaying this and GoDaddy comes out against this, Namecheap gets a lot of bad PR, and rightly so.

If they just said "we're having issues with transfers in from GoDaddy right now, we've contacted them about the issue and are waiting to hear back, and we'll update you with any of their responses and/or progress on the matter", I'd be much more impressed. It seems that they're trying to kick GoDaddy while they're down instead of working to actually resolve the issue for their clients.

<comment><USER15> Conversely, the mobs are already upset about bad behavior on GoDaddy's part. If Namecheap is right, this story gets even bigger and makes things even worse for GoDaddy.

If they're wrong, well, everyone is pissed at GoDaddy anyway, so this angle fades quietly from view.

I don't see the PR downside for Namecheap here.

</comment></comment></comment></comment></comment><comment><USER16> Just to add data to the discussion;

I kicked off migration of my only 3 GoDaddy domains to namecheap last night. One .net, one .com and one .cc domain. None had any kind of whois protection or anonymisation.

The .net and .com migrations went very quickly and smoothly. Within two hours I had the confirmation e-mails from both namecheap and GoDaddy, and within 4 hours the migration was complete.

The .cc domain took a little longer, as when I started the move with namecheap it didn't seem to want an EPP code for a .cc domain, but then later changed its mind and asked me to enter one. I entered the code within 2 hours, and 8 hours later the migration was complete.

GoDaddy's goodbye was actually pretty professional;

      </comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment>=
       SORRY TO SEE YOU GO. WE'LL ALWAYS WELCOME YOU BACK.
      </comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment>=

      Dear Colm MacCarthaigh,

      We're sorry you transferred your domain name(s) away from GoDaddy.com.
      We are committed to providing quality services and products and hope
      that we met your needs.

      If you feel your transfer was in error, or if you have changed your
      mind, please contact our 24/7 sales department at (480) 505-8877.
      They'll assist you in transferring your domain name(s) back to us.*
      Keep in mind that we continue to offer low prices and $7.49 transfer
      rates on some domains.

      Sincerely,
      Go Daddy

      P.S. Visit GoDaddy.com (http://www.godaddy.com/default.aspx?prog_id=GoDaddy&isc=gdbba1365)
      and SAVE 15%* off your order of $50 or more. Just use source code
      gdbba1365 when you check out to get your special savings. Start
      shopping now at GoDaddy.com or order by phone at (480) 505-8821.

      *Please note that ICANN's Transfer Policy may prevent you from
      transferring your domain name within 60-days of a transfer.

      - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
      *Not applicable to ICANN fees, taxes, shipping and   handling, sale
      priced domains and transfers, bulk domains   and transfers, premium
      domains, Sunrise/Landrush domain   registrations and pre-registrations,
      memberships or   maintenance plans, additional disk space and bandwidth
      renewals, additional email addresses, Search Engine Visibility
      advertising budget, Managed Hosting, custom page layouts, brand identity
      services, Go Daddy branded   merchandise or gift cards. Discount
      reflected in your   shopping cart - cannot be used in conjunction
      with any other offer, discount or promotion, or in connection with
      special partnership discount programs. After the initial purchase term,
      discounted products purchased with special offer discounts will renew
      at the then-current renewal list price.

      Copyright (C) 2011 Go Daddy All rights reserved.

</comment><comment><USER17> GoDaddy has responded to the allegations from namecheap at TechChrunch, saying that the blocks were part of standard practice to limit the volume of Whois queries from a single IP, which is apparently common practice, unless the registrar is notified that there may be a large number of queries. [1]

If this is indeed true, then it seems namecheap are trying to score cheap PR points, although they have responded saying they attempted to reach out to GoDaddy.[2]

I'm inclined to believe that namecheap did try to reach them, although I'm not sure that they are above trying to slam more bad press onto GoDaddy....

[1] [http://techcrunch.com/2011/12/26/godaddy-responds-to- nameche...](http://techcrunch.com/2011/12/26/godaddy-responds-to-namecheap- accusations-removes-normal-rate-limiting-block/)

[2] [http://community.namecheap.com/blog/2011/12/26/godaddy- trans...](http://community.namecheap.com/blog/2011/12/26/godaddy-transfer- update/#comment-1709)

</comment><comment><USER18> Well that explains why my transfer is taking so long...

</comment><comment><USER19> NC is an enom reseller, so I'd guess that all of enom would be affected as well. Nothing on the eNom status page, so not sure... <http://www.enom.com/registrynews.asp>

<comment><USER3> Does eNom own NameCheap?

      http://davezan.com/does-enom-own-namecheap.html

No, it seems they don't, but the article provides details on a longstanding partnership.

</comment></comment><comment><USER20> I use godaddy to buy .com.mx and .mx domains .. PLEASE, start selling this domains.

I really hate to give money to godaddy .. i feel like a dollar for them is a dollar for killing-elephants-for-joy safari.

</comment><comment><USER21> I finally was able to get my domains transferred to Namecheap today. It's one of those "I never liked GoDaddy anyways" kind of things and the entire SOPA ordeal finally pushed me to it.

</comment><comment><USER22> If the (alleged) issue is that GoDaddy is throttling WHOIS lookups, does using [<http://help.godaddy.com/article/3681](GoDaddys> Export Lists tool) and including WHOIS info help? Or are the "real-time" lookups required by a new registrar?

</comment><comment><USER23> I moved all my domains from GoDaddy and one of them had trouble doing the whois information. I did it myself and it in fact did look different from my others. It could have just been the difference between a .org and a .com, but I entered in the epp auth code with the transfers and it helped it along.

</comment><comment><USER24> I just got the e-mail from GoDaddy to ask if I wanted to allow or decline the transfer. I'm glad that got resolved. That was my one to test that I was doing things correctly before I moved about 20 other domains off GoDaddy.

</comment><comment><USER25> Thanks. I noticed this too.

</comment><comment><USER26> This doesn't strike me as one of the classier communications. Does it resonate well with other people?

</comment><comment><USER27> I bet namecheap's investors are smiling right about now.

</comment><comment><USER28> I wouldn't exactly be calling my employees in over Christmas to make it easier for customers to leave.

</comment><comment><USER29> did you know that EA games and Sony also support SOPA?

<comment><USER4> Are you really surprised? Both EA and Sony are known for installing rootkits on their customer's computers.

</comment></comment><comment><USER30> A bunch of my transfers appear to have been delayed because Namecheap themselves have not generated the initial authorization email to begin the tranfer process. The ones for which they did generate the email (about 1/4 of my transfers) went through fine with no delays from Godaddy. Not sure what's going on (I sent an email to support about 4 hours ago but have no response yet), but so far I'm a little underwhelmed by their service.

<comment><AUTHOR> That authorization email is affected by what's discussed in this blog post -- they can't send it until they get all the domain info.

<comment><USER31> If I were a registrar, I would have rate-limiters on domain info downloads to prevent harvesting by spammers. And those rate-limiters would be set so as not to interfere with normal demand, but might be tripped if, say, everyone tried to transfer out their domains all at once.

Just sayin.' Never ascribe to malice what can adequately be explained by reasonable engineering heuristics...

<comment><USER13> Hey Namecheap, since I know you're on here (Anthony?): did you try contacting some engineering folk at GoDaddy to try to resolve this delay for your clients? I hope you already have, and that they told you to sit on it, given the tone of your last blog post.

If you're just slinging non-verified mud at a competitor, kicking them when they're down instead of focusing all your effort on actually fixing this, how are you better than GoDaddy and why would a potential new client choose you over them if you're both shady?

<comment><USER32> Amusing that you'd hope that GoDaddy would continue screwing their (soon to be former) customers because they don't like that a competitor thinks they're playing dirty pool.

Unsurprising, but amusing nevertheless.

<comment><USER13> ....what? My point was that it seems like Namecheap is just making efforts to blame GoDaddy instead of efforts to actually fix the problem. I hope GoDaddy dies in a fire. I hope everyone gets away from them ASAP and I hope they get out of the domain business after that (it'd be great to at least see ICANN call them out on it and fine them or something.)

As I see it, the more GoDaddy fights and does stupid things, the more it increases the Streisand effect and the more people will become aware of how awful they are.

<comment><USER32> Publicly lighting a fire under GoDaddy's rear is probably the most effective way to make them act right.</comment></comment></comment></comment></comment></comment></comment>


Bitcoin value is over $200 - <AUTHOR> http://markets.blockchain.info/

<comment><USER2> It's fascinating to watch this, largely because Bitcoin has no "natural" value. The only utility it has is as a transfer of value from one place to another, and it can do that equally well with a bitcoin being worth $1 or $200.

But it's really interesting to watch the value ratchet up, and easy to see how you can do this - if people will only ever sell a bitcoin for (say) 1% more than they bought it for, and people buying things with bitcoins don't care if the things they're buying with it are priced 1% higher than they might, theoretically have been, then the price of bitcoins can keep increasing indefinitely.

Until, that is, some people decide that they want to get rid of a lot of bitcoins, and thus are willing to accept less than that, to shift them quickly. Then the price drops. And then more people might also decide to sell, so as to convert the bitcoins they have at the value they're now worth.

It's classic sentiment pricing - but without any of the value "stickiness" that you get when there's some utility to be gained from the objects you're trading.

(I'm not trying to talk Bitcoin down here, I find the whole thing fascinating. Just thinking out loud.)</comment>


GnuTLS: TLS 1.3 session resumption works without master key, allowing MITM - <AUTHOR> https://gitlab.com/gnutls/gnutls/-/issues/1011 <comment><USER2> The HN title is currently "TLS 1.3 session resumption works without master key, allowing MITM" which only mentions TLS 1.3, while the CVE description mentions TLS 1.2 as well:

> GnuTLS 3.6.x before 3.6.14 uses incorrect cryptography for encrypting a > session ticket (a loss of confidentiality in TLS 1.2, and an authentication > bypass in TLS 1.3).

[https://nvd.nist.gov/vuln/detail/CVE-2020-13777](https://nvd.nist.gov/vuln/detail/CVE-2020-13777)

<comment><USER3> According to this tweet, "For TLS 1.2, this enables passive decryption of traffic to/from GnuTLS servers when the client supports session tickets (which is common)".

[https://twitter.com/__agwa/status/1270054738344792070](https://twitter.com/__agwa/status/1270054738344792070)

<comment><USER4> Right. Specifically what's going on here is that for TLS 1.2 resumption your _client_ remembers the secret keys it was using last time, but the _server_ doesn't want to keep secret keys for every single client that might re-connect so it has its own secret Session Ticket Encryption Key (STEK) and it sends any client a "ticket" with the server's state for that client's connection in it, encrypted using the STEK.

The not at all obvious problem in TLS 1.2 is that the state encapsulated in the ticket will work not only in some hypothetical future TLS resumption (which you might never do) but also the session you are in already now. And that ticket is sent over the wire if resumption is permitted, during session setup, before the symmetric encryption for the TLS channel has switched on.

So what that means is that if you know the STEK (which now for affected GnuTLS servers you do) you can read any session you captured in which resumption was allowed, _including_ fresh sessions which were never resumed, and passively decrypt the entire thing. Nothing anybody can do about it.

</comment></comment></comment><comment><USER5> > [T]he [GnuTLS] code is fundamentally broken; most of its external and > internal APIs are incapable of passing binary data without mangling it. The > code is completely unsafe for handling binary data, and yet the nature of > TLS processing is almost entirely dependent on secure handling of binary > data.

-Howard Chu (2007)

[https://www.openldap.org/lists/openldap- devel/200802/msg0007...](https://www.openldap.org/lists/openldap- devel/200802/msg00072.html)

</comment><comment><USER6> Different versions of TLS should not interact with each other. There are too many behavioral permutations to audit.

<comment><USER7> They don't. It's always the same version server and client negotiate.

The vulnerability is in the way session tickets are encrypted (or rather the lack of it), which is present in both versions, but in different formats; for example in TLS 1.3, the session ticket is sent _after_ change cipher command.

It's vulnerable in 1.2 and 1.3 because the way session ticket is encrypted. GnuTLS used effectively an all-zero key until it's rotated in the next tick.

<comment><USER6> Thanks, I must have misread the post:

> "The issue applies to TLS 1.3, when using TLS 1.2 resumption fails as > expected. Because the ticket can be used for resumption without knowledge of > the master key I assume (but haven't tested yet) that it can also be used > for passive decryption of early data."

I assumed that the server was using a ticket generated using TLS 1.2 to somehow resume a TLS 1.3 connection.

<comment><USER8> The bug report downplays the severity of the problem. This comment comes the closest to explaining the severity:

> ticket encryption key and decryption key are all-zero [...] in TLS 1.2, it > may allow attackers to recover the previous conversations

[https://gitlab.com/gnutls/gnutls/-/issues/1011#note_35383063...](https://gitlab.com/gnutls/gnutls/-/issues/1011#note_353830633)

</comment></comment></comment></comment><comment><USER9> This is your weekly reminder that There Is No Such Thing As Computer Security. Treat your computing devices appropriately.

</comment><comment><USER10> This needs a "GnuTLS:" or CVE reference in the title.

<comment><USER11> Indeed otherwise this might look like the SSLv3 Poodle issue to some.

</comment></comment><comment><USER12> I was wondering how commonly used gnutls was.

Gnutls is required by (among others) ffmpeg, emacs, vlc and wget.

<comment><USER13> This bug is on the TLS server code, so the relevant question would be how commonly used gnutls is _as a TLS server_. Luckily, gnutls seems to be more popular as a client library than as a server library.

<comment><USER14> I use it on a server. Way easier to use than opennsl, to my taste.

</comment></comment></comment><comment><USER15> So this isn't a vulnerability inherent to TLS 1.3 that we need to start designing TLS 1.4?

Edit: The title has since been fixed.

<comment><USER16> No, just a GnuTLS bug that was fixed in its recent release

</comment></comment><comment><USER17> The words "GnuTLS" or "mbedTLS/PolarSSL" strike fear into my heart. When picking a TLS library written by amateur cryptographers, make sure it is the one used by everyone else (OpenSSL).

<comment><USER18> GnuTLS has a non-crazy well documented API with loads of good quality examples, which is why we chose it for libvirt. (It was impossible for us to understand how to use OpenSSL for basic stuff like opening a connection at the time we made that choice - I sure hope it's improved now.) The lesson is if you want adoption you need to have a clear API with no foot-guns, lots of examples and documentation.

Also I would dispute that GnuTLS is somehow written by "amateurs". I know people who contribute at Red Hat and they are very much professionals, and experts in math and crypto engineering.

<comment><USER8> Yes, a good API with good documentation is important. Fortunately, OpenSSL has improved significantly in the last few years.

Unfortunately, I've talked with crypto experts about GnuTLS's bizarre session ticket key rotation system and nobody can figure out what the heck GnuTLS was trying to accomplish with it. It doesn't rotate anything important, and by implementing it they accidentally introduced this critical all-zero key bug. This is the hallmark of amateurs.

<comment><USER19> >Fortunately, OpenSSL has improved significantly in the last few years.

After Heartbleed was discovered. I think it's the TLS specification itself that's the problem. It's ridiculously complex and difficult to implement correctly to the point even the experts struggle with it.

</comment></comment></comment></comment><comment><USER20> In a way, maybe this is an (inadvertent) throwback to early GNU ideals: > "[When] passwords first appeared at the MIT AI Lab I [decided] to follow my belief that there should be no passwords," Stallman would later say. "Because I don't believe that it's really desirable to have security on a computer, I shouldn't be willing to help uphold the security regime."

[https://www.oreilly.com/openbook/freedom/ch07.html#86161](https://www.oreilly.com/openbook/freedom/ch07.html#86161)</comment>