How Google Meet's noise cancellation works - <AUTHOR> https://venturebeat.com/2020/06/08/google-meet-noise-cancellation-ai-cloud-denoiser-g-suite/ <comment><USER2> Most people have no idea of the amount of incredibly advanced signal processing that goes into echo cancellation and noise cancellation in videoconferencing.

This post is on noise cancellation specifically, and it actually has the potential to be a _huge_ step forward.

One of the big audio problems with group meetings is that the background noise from each participant adds up, to a point where it quickly becomes unbearable. For that reason, videoconferencing generally only plays audio from one or two participants at most, using a fairly simple estimation of whichever audio signal is currently loudest. The problem is that this can make it really hard to interrupt (people will literally not hear you), or tell the difference between two people going "mm-hmm" versus the whole group. If you've ever been in a group meeting where everybody applauds something, this is why you _see_ everyone applauding but only hear a smattering.

But if this noise cancellation really succeeds, it could be a huge leap forward because audio cues and overlap will actually work for the first time -- hearing the "mm-hmms", hearing everyone pipe up, and so on. Videoconferencing will feel more like an actual single shared audio environment, rather than the kind of "walkie-talkie" effect it so often feels like now.

I'm really looking forward to this.

<comment><USER3> > _The problem is that this can make it really hard to interrupt (people will > literally not hear you)_

This is driving me crazy with Google Meet in these COVID19 times. Even in a relatively small conference, I have a really hard time interrupting someone to ask a quick question, even when the speaker is expecting interruptions. It's always "excuse me!"; delay as person continues speaking; I stop; the other person says "yes, please ask away"; when I restart my question the other person already assumed I've changed my mind and continues speaking; repeat ad infinitum. And this is _if_ they even hear me over the audio breaking up.

It's very, very frustrating. If they solve this it would hugely improve quality of life in remote conferencing for me.

<comment><USER4> Yeah, I wish there was a simple non-verbal option to signal intent-to-talk.

I want to just be able to hit my self-view and have it have a big icon on it or something so the person currently speaking (and everyone else) can see that I want to say something. Maybe sort these in chronological order so the speaker can see who wanted to talk first?).

In theory you could do this with a good chat, but for some reason the chat in Zoom and the others is kind of an afterthought and nobody uses it.

One of the reasons I prefer text based chat is multiple people can talk at the same time without needing to deal with interrupting audio. If you can type well, the bandwidth is higher for group communication (and you get a log).

At least with video you can kind of tell when someone is waiting to speak by seeing their expression. Audio only is worse (but maybe wouldn't be, if you had good intent-to-speak tools built into the app?)

<comment><USER5> I really like how Jitsi Meet puts the hand raising/lowering button right on the bottom bar, where there's just empty black/white space in Zoom/Google Meet, and not buried inside a menu labeled "Participants" (???), where it's a hassle to access (Zoom).

</comment></comment></comment></comment><comment><USER6> > A musical instrument will probably also get filtered out. "To a pretty large > degree, it does," Lachapelle said. "Especially percussion instruments. > Sometimes a guitar can sound very much like a voice — you're starting to > touch the limits there. But if you have music playing in the background, > usually it'll cut it all out."

This is a big issue with hearing aids. The whole industry is focused on optimizing for voice intelligibility and as a musician you end up doing trial- and-error with the audiologist to turn all that stuff off.

We need more open source hearing aids - I've read of a few but they're not mainstream.

<comment><USER7> First, I'll say this to everyone: Get hearing aids if you need them. They can change your world.

About music, this is getting much better in hearing aids. I've been from analog thru digital over 15+ years of hearing aids, and my latest (3 months ago) pair from Phonak (no affiliation) is an honest leap forward. It has a built in Music profile that disables all sound optimizations in general, while still attempting to correct the hearing ranges that you have a deficit in. I was on the verge of no longer being able to hear with hearing aids, that has probably been extended by 3-5 years with these new models. At that point I will be approaching cochlear implant level hearing loss. I happily embrace my cyborg future!

On top of Music, I have a Walking profile that attempts to focus on the person that is walking to the left or right of my and can pick with side on the fly. And they make great ear plugs when things are loud.

The Normal program, auto-magically selects between 8'ish profiles to pick the best one for the environment. And it has finally got it right. Older models I would daily need to force it into the best mode because it guessed wrong. The latest model I only have to tell it what to do once every few weeks.

And to the original topic, noise cancellation, hearing aids bluetooth'ed to the phone/PC for conference calls is hands down the best possible audio experience. Built in noise cancellation, amazing microphones that can be used for your voice portion of the call, tuned to your hearing, with some of the finest sound output possible. Just amazing. These things are so good these days that they are finally being labeled as assistive devices for people without hearing loss. They can give someone with normal range hearing essentially bionic hearing. Tinnitus? They play customized white noise to make the ringing less noticeable. Doesn't help everyone, but it's really nice for me. I hear more ringing when I take my aids out.

Oh, and it does all of this on a device that fits in your ear with a battery the size of a few grains of rice and all in a few milliseconds so your brain sees the mouth move at the same time it actually hears the audio.

Again, get them if you need them.

<comment><USER6> They have some great features and I agree that people should get them (or upgrade), but they are not optimized for musicians.

I just got a similar model I assume (M90-R) and it's definitely not switching to music mode automatically when I play music. (Maybe it's different for listening.) I just had the audiologist add a music mode that I can switch to manually, but getting acceptable timbre for the instruments I play (accordion, melodica, and piano) is work in progress. Making an expensive instrument sound like cheap trash is disappointing, though of course I can take them out.

Having Bluetooth is nice, particularly for phone calls, but I find the sound quality is unsatisfying for listening to music, so it won't be replacing speakers for me.

</comment></comment></comment><comment><USER8> I've been using [https://krisp.ai/](https://krisp.ai/) to great effect with Zoom while sitting outside on the laptop with road traffic, birds, etc nearby - My team really had a "wow" moment when i turned it on the first time

<comment><USER9> Krisp is embedded into Discord (enable beta settings) and the voice chat quality far exceeds any of the "business" focused software I've ever used.

Not to mention the screensharing is infinitely better as well. It's pretty pathetic of the busines sapps, we went through a day where I was trying to screenshare something and my remote coworkers kept complaining of lag, blurriness, or the app would just crash (slack). We went through ms teams, zoom, slack, and google meet. All had issues. Convinced everyone to install Discord and suddenly I was able to shared my desktop perfectly at 1080p without noticeable lag and crystal clear audio.

<comment><USER10> +1

Discord's lack of lag in audio makes a huge difference for voice comms. I've only used it for gaming, but you can really tell the difference when you switch to the game's voice chat feature which has probably a third of a second of latency. And of course Zoom et. al. have a lot more lag and it really hurts the experience. In addition to low latency, the sound is also very good quality.

<comment><USER11> Note also that no messages in Discord, including individual messages/DMs, are end-to-end encrypted. This is precisely the same security issue that Zoom has (and Slack, and IRC without OTR).

Discord can and does log all messages through the system, and has many internal tools that operate on the plaintext. Anything you communicate through Discord you should assume any/all Discord staff may read.

They claim that the voice comms are e2e but there are no further details available (like where the keys are generated).

</comment></comment></comment></comment><comment><USER12> As far as my experience goes, the single best way to deal with background noise is... the mute button.

In every video conf I've been, you can instantly tell when "one of them" who can't be bothered to mute themselves joins. The audio quality immediately goes down the drain. It's always the same subset of people who do it, too. As soon as they're enjoined to please mute, the audio quality is restored.

No amount of magic signal processing will ever match it.

While perhaps misguided to use it that way, the mute button thus act as a social-clueness meter.

<comment><USER13> "Hold space to talk" is not a bad solve for this, also makes folks ramble less.

<comment><USER14> Would probably have some major accessibility issues.

Plus I'd hate to be the intern that has to sit and hold the space bar while the boss delivers a presentation

<comment><USER15> It could be built so that the meeting organizer can exempt certain people and so the system can exempt certain accounts.

</comment></comment></comment></comment><comment><USER16> I might be unusual, but my experience with videoconferencing has been that ambient noise is rarely a major problem. The big issue is audio cutting out due to a shaky network. When ambient noise is a problem, it's not so much someone typing as their spouse talking in the background or a fire engine going by -- and at that point the solution is for them to hit mute.

<comment><USER15> > _at that point the solution is for them to hit mute_

From a technical point of view, that is really the best thing. It works, and sometimes it's the only thing that works.

But if you try to get people actually do it, you run into problems:

(1) They don't realize it's them. AFAIK the system doesn't play their audio back to them, so while everyone else hears the noise, they don't. The one person who needs to take action is the one person who doesn't know action is necessary.

(2) They are distracted. When their spouse is talking, they are focused on whatever their spouse is saying, not on how it affects the meeting audio. Or the meeting is boring and they're not paying attention.

(3) They just don't care enough. They are there to attend a meeting, not fiddle with computer stuff. Some people will never take the time to learn where the mute button is in the software.

Perhaps #1 could be improved, though, with some kind of blindingly obvious indicator in the UI. If "YOUR MIC IS WHAT EVERYONE IS HEARING RIGHT NOW" flashes when your mic takes the floor, maybe you'd notice it lighting up when you didn't intend for it to.

<comment><USER17> AWS Chime has significant drawbacks, but one of the things I most liked about it was that anybody could mute anybody else. The number of calls where that significantly cut down on audio discomfort was surprising.

For those wondering, unmuting is a privileged operation that only the user could do themselves.

<comment><USER18> Same with Meet, at least if you're using the regular enterprise/GSuite version (probably different for schools and/or the nerfed version you get with a personal account, since kids muting the teacher gets old really quick).

</comment></comment></comment></comment><comment><USER19> I was not so impressed with this demo - especially when he was scrunching his potato chip packet, the degradation in his voice quality made it almost impossible to understand what he was saying and his voice sounded very synthesized and processed, and that's through a $200 Yeti professional microphone. Seems like some of the other noise cancellation technology options from Nvidia RTX and others are more effective.

<comment><USER2> Microphone quality has very little to do with it. Noise separation is just an incredibly hard problem, particularly when a noise is loud. Scrunching potato chips, there's no scenario where his voice won't become degraded unless you can isolate the scrunching sound separately (microphone beamforming can help here, but is still never perfect).

Running this economically on servers at scale in realtime, I consider this very impressive. I can't say how it compares with RTX, but I wonder if it has anything to do with the amount of computing resources that can be dedicated to it. A single expensive card dedicated to one audio stream, versus a single Google server than needs to process hundreds (thousands?) of audio streams.

<comment><USER20> RTX Voice uses less than 10% of a 2060 (non-super)

Given how common voice communication is in our world, I am sure Google can build ASICs for this (if not just run it on TPUs), and get the marginal cost of vocal processing to be negligible.

Heck, they probably would just need to divert <5% the resources of Fuschia or any other of their "senior engineer retention" projects.

</comment></comment></comment><comment><USER21> I've been doing online guitar lessons since Covid-19 started and all these algorithms just suck hard for that. Even in a 1:1 call.

Two repeated notes and the noise cancellation just immediately shuts you down... we've been using Zoom and luckily you can turn all the audio processing off if you go in "Advanced" and enable "Turn on original audio".

<comment><USER22> I'm surprised there's not an option to disable it?

<comment><USER23> What do you mean? ben7799 said there is an option to disable it.

</comment></comment></comment><comment><USER24> This would have been useful for my co-worker. They went on a trip to Europe years ago and had conference call scheduled when they got there.

Unfortunately for them they decided to lie on the bed in the hotel and the jet lag hit them pretty hard. Next thing you know they are asleep and they started snoring and I guess fairly loudly and everyone on the call could hear. So then the people on the call spend some time trying to figure out who is the person snoring, going through all the attendees. Eventually they figure out who it was and they started yelling trying to wake them up, which they did after awhile. Needless to say my coworker was very embarrassed about the incident at the time, but it did make a good story to tell people :)

</comment><comment><USER25> Simple non-AI solution: Require all meeting participants to use push-to-talk. Support foot-pedals, mouse buttons, phone volume up button, and bluetooth play button.

For large meetings, organizers can enable a single-talker mode. Holding the talk button puts you in a queue. Your screen indicates when it's your turn to talk. This prevents folks from talking over each other. This eliminates echo by muting the talker's speakers while recording their voice. Also, attendees see the current talker, not the person whose dog just barked.

<comment><USER26> Unfortunately push-to-talk is not standard on anything. I've hacked together a hotkey to do it, but there's always too much lag. Somebody should make a wired headset with a button and two modes, one is push and hold to talk and the other is push and hold to mute.

</comment></comment><comment><USER27> To all those here who complain about algorithms messing with their audio when they don't want them to. Use an app called TeamTalk. It lets you disable all that processing, so it works great for high-quality music transmission etc. I have no affiliation with them, I have been using it for a few years and I'm very happy.

</comment><comment><USER28> All this work to elim background is great. But we also will need business oriented group meets which emulate real life: Allow breakouts, 2-4 people in group "sidebar" to chat, the burble of other convos drift in, providing a gentle low background hum. The sidebar, unless marked private, would also contribute to hum of other users in a main group or in sidebars of their own. Acoustic effects can even allow directional it of the sounds.

Yes, we can do VR worlds that still look like Second Life, but while we are working on fixing that, we might solve for near term things to improve interactivity.

Well, and solve the "one person speaks, all must listen, lag for response, loop" which I find is similar to how morse code discussions work.

</comment><comment><USER29> Everybody makes a stab at this, and very little of it works consistently. I applaud Google for attacking this head-on! It is a big issue and deserves attention.

My biggest issue (when I worked in videoconferencing) was echoing, and locking onto the delay window where echoes could occur. Depending on the distance from a conference room speaker to all the walls, echoes could occur at one or more offsets (appear at microphone input with some delay after presenting at the speaker). And ambient noises could masquerade as echoes. The filters tend to be IIR filters, and get wound up easily. It was awful.

</comment><comment><USER30> Microsoft Teams' noise cancellation has been driving me absolutely crazy the past few weeks. I live on a busy road, and whenever a car drives past Teams will reduce the volume of anyone else talking in a meeting. Even if I'm muted - Teams gives me the "your microphone is muted" notification. And I use closed-back headphones for meetings so I don't even hear the outside noise. So this results in my having to constantly have the Windows volume slider open, one earpiece off, and listen for a car coming so I can raise the volume in advance. Is anyone else dealing with this?

<comment><USER31> Try Control Panel > Sound > Communications > Do Nothing

</comment></comment><comment><USER32> I'd say this is more like a demo. From the "how it works" in the title I was expecting to see some implementation details.

Edit: I had only watched the video. The article does indeed contain a lot more detail.

<comment><USER33> from the "venturebeat.com" domain though, this is about what i would have expected.

<comment><USER34> This is not the time to be snarky, I think we should congratulate the Google Meet team on placing this great article, not to mention the obnoxious integrations into other beloved Google products.

Can't wait to see what the Google _Duo_ team will come up with in response. I mean, we saw the blog post on their great new video codec (AOMedia Video 1 was it?) but I personally felt it left much to be desired.

What happened to the Hangout guys? Are they still in this one? Product middle management wants to be wooed.

<comment><USER35> Duo is targeted to the general public and has E2E encryption though, so cloud denoising is not a possibility.

</comment></comment></comment></comment><comment><USER36> Anyone know what kind of system they're using to do this? Any papers?

I messed around for a few months with speech enhancement last year and didn't really get anywhere beyond sort-of-reproducing a few existing models: [https://github.com/MattSegal/speech- enhancement](https://github.com/MattSegal/speech-enhancement)

All the published "state of the art" examples I could find were pretty crap, whereas Krisp AI were doing much better than what I've seen released publicly.

<comment><USER37> I've worked on my last gig for a startup that did speech-in-noise recognition and there we've used a recurrent neural net approach to separate background noise from speech. This model was trained on many many hours of audio data as well.

</comment></comment><comment><USER38> Serious question - what's the risk that someone with a high pitched, outside- the-norm voice will get denoised? If it filters out kids in the background, will kids no longer be able to use google meet?

<comment><USER39> I haven't been able to confirm this, but I swear it happens to my mom on Zoom. When I video chat my family on Zoom and she isn't sitting directly in front of the laptop, her words rarely come through. I can see her lips moving, I can hear my dad grunting in agreement next to her--but she's silent. If they switch places, I can hear my dad without issue.

I don't know if it's a combination of her cheap hardware or what, but it's... odd.

EDIT: grammar

</comment></comment><comment><USER40> I would really like to use a Google Noise for meeting cancellation. Excessive use of meetings now "because we can" instead of thinking longer and harder and asking a well structured question.

Meetings should be for review/discussion and decision making not vocal exercise and grandstanding.

Would also be great if meeting providers would have a dial to show current latency for all participants to make easier to interject.

Lastly I do recommend using meeting tools that have features like letting you vote, raise a hand, chat all in a sync to main voice. Will make life easier for meeting moderators..... And if you don't use moderators then start the practice of doing it - quality of meetings will improve hugely.

</comment><comment><USER41> What I'd really like to see is effective source seperation and nulling. For example, if you could mute the screaming baby in the background of a VC speaker (this has been fairly common occurrence now that we are WFH and it's hard to get day care).

<comment><USER42> I was really looking for the baby noise test in the demo, but I guess for now its human vs non-human cancellation ?

Edit: apparently can also remove kids crying; just not included in demo

</comment></comment><comment><USER43> As someone with 2 dogs this is going to be a good reason to switch to Meet whenever possible.

</comment><comment><USER44> None of this fancy technology is necessary IMO.

Just implement push to talk with mute-by-default. 90% of the audio issues would be resolved. Another 5% could be solved by buying everyone a decent headset which hopefully has a push-to-talk button on it as well.

<comment><USER45> In theory yes. But what about users that are mobile on a call and even when they "push" to talk, ambient noise from the metro, crowd, or traffic is present enough to be troubling?

You don't always get to choose if background noise is present.

Also, you just asked that people push a button, and wear a headset. That is a lot, and this is about lowering the bar needed to get a good experience.

</comment></comment><comment><USER46> I wonder how much benefit you would get from targeting specific microphone/speaker setups for noise cancelling rather than treating everything the same. I would imagine that the noise cancellation requirements are far different for someone video conferencing over a laptop mic and speaker versus a good pair of Bose headphones. If you could specify what type of device you are using it could tune the noise cancellation accordingly - if I am using a good pair of headphones, I don't need echo cancellation, but I still need to filter out some amount of background noise.

</comment><comment><USER47> Funny timing. Just got off my first Google Meet call an hour ago and was thinking they need to add noise cancellation. It was awful.

<comment><USER3> I feel you! Did you also have your kid shouting in the background? If they found a way to specifically mute kids crying or shouting it would be a huge deal :P

</comment></comment><comment><USER48> Might be interesting to train the model using user's own voice. Maybe this would help filtering out co-workers in open office or family members.

Maybe you could also use this personal model to hide very short network interruptions. Other party could use this model to constantly predict my next piece of audio and switch to prediction in case packet is lost.

</comment><comment><USER49> One of the things I picked up from HN a few months ago was bettering my remote setup -- I picked up a HDMI capture card for my mirrorless camera, and bought a few lights to brighten up my office, and then I purchased a cardioid microphone and a pop filter.

The difference is night and day based on some of the recordings I've heard.

</comment><comment><USER36> Anyone know what kind of system they're using to do this? Any papers?

I messed around a little with speech enhancement last year and didn't really get anywhere: [https://github.com/MattSegal/speech- enhancement](https://github.com/MattSegal/speech-enhancement)

</comment><comment><USER50> > Google also made a conscious decision to put the machine learning model in > the cloud, which wasn't the immediately obvious choice.

Oh good. Meet is already a _huge_ battery-hog on my laptop, so adding fancy signal processing client-side was worrying me.

</comment><comment><USER51> I need to get hearing aids soon and heard about all the advances and limitations. Particularly the problems with noise cancellation. I hope this kind of technology trickles into hearing aids also.

</comment><comment><USER52> I am still waiting until AI reaches the ultimate in noise cancelling- that meeting could have been an email. AI will automatically send meeting cancellation and most likely meeting notes.

</comment><comment><USER53> Wouldn't this be a bit more trivial with multiple microphones?

</comment><comment><USER54> Journey of a datascience feature - Start to End (still Work in Progress)

Key Takeaway - Its fine to be not 100% accurate, roll it out and learn.

tl;dr

\- Approval from Execs

\- Data -> Learning -> Training -> Variability -> Training -> Tuning

\- Privacymatters (for all digital educated , uneducated)

\- What & Whys of UX -- ultimately what user says

\- Definitely Cloud -- Its 21st Century

\- Optimised for Speed , Cost (a bit irrelevant if I am Google ;) )

\- Release (with presentation) -- Timing matters

\- Feedbacks (On permission)

A summary on the "Denoiser" and not "Noise cancellation" [Don't want to get ranted out by Data Science folks] feature of googlemeet by PM. Applies to any such feature.

</comment><comment><USER55> Any battery life tests of this tech on phones?

<comment><USER56> None. Because the processing doesn't happen on a phone.

> When you're on a Google Meet call, your voice is sent from your device to a > Google datacenter, where it goes through the machine learning model on the > TPU, gets reencrypted, and is then sent back to the meeting.

<comment><USER55> Something's obviously going wrong. Google meets runs the iPhone vs a standard call or Zoom.

Obviously theoretically there is basis to what you're saying but, end-to-end somewhere its problematic.

<comment><USER23> >Google meets runs the iPhone vs a standard call or Zoom.

What does this mean?

</comment></comment></comment></comment><comment><USER57> Or you could just mute your damn phones

</comment><comment><USER58> Weirdly, it seems the simplest phones already solved those problem a long time ago.

Seems like over-engineering. The issue is either with the microphone, with the hi-def stuff or something else.

Every normal phone never had an inch of a problem, so I'm really confused why computers have this issue.

<comment><USER59> It's because people find wearing a headset or holding a microphone too onerous.

Better for them to just point their laptop microphone at the whole room and let the poor saps at the other end suffer.

<comment><USER60> Ridiculously, I learned the other day through trial-and-error that the mic on my headset isn't actually getting used; rather, the one on the laptop is. The device is Bluetooth, and Bluetooth has two modes of operation: A2DP, which is output-only (no mic), and the default. But the laptop mic gets used. The other mode, HSP/HFP, can use the mic, but the audio quality drops _tremendously_. Like, to garbage levels. My best research said this was because BT lacks the bandwidth to carry both mic input & sound output simultaneously.

The headset at least still has the benefit of isolating the output from the conference to my ears, s.t. the laptop mic picks it up, but it would be nice to not be tethered to the laptop and be able to pace the room.

<comment><USER59> Bluetooth handsfree is not great quality but seems to work semi acceptably for phone calls in my car. I guess with multiple parties on a video call it might be worse?

Especially if the other end was not on a headset.

</comment></comment></comment></comment><comment><USER61> > How Google Meet's noise cancellation works

Very poorly. Of all the available alternatives (Zoom, Skype, FaceTime), Google Meet seems to have the worst audio _and_ video quality. This is inexplicable for a company very easily capable of technological and product leadership in both of those things.

<comment><USER62> Did you even read the article? This is talking about a new noise cancelation feature being introduced today.

<comment><USER61> Quite obviously not, just like almost everybody else in the comments.

Shouldn't the title be "How the _new_ Google Meet noise cancellation works" then?

</comment></comment></comment><comment><USER63> Not very well. Watched a Google Meet meeting for the neighbor's Honor Society induction and the quality was horrible. Video kept freezing and audio cut in and out. Was probably only about a dozen attendees in the meeting room. Wasn't the neighbor's connection either, they have a solid Fios 200/200 service.

<comment><USER62> Did you even read the article? This is talking about a new noise cancelation feature being introduced today.

</comment></comment><comment><USER64> It would be fun if it canceled out screaming.

We somehow have this sexist social expectation that women who show their feelings (crying, screaming) are "hysterical" (really a nasty word) and not taken seriously. If so, men screaming should be equally considered a sign of immaturity and lack of self-control.

Also could help with customers ("Sorry, I can't hear you!").</comment>


Were Intelligence Agencies Using Heartbleed in November 2013? - <AUTHOR> https://www.eff.org/deeplinks/2014/04/wild-heart-were-intelligence-agencies-using-heartbleed-november-2013 <comment><USER2> I helped write this post. Note that we're very interested in anyone who has been keeping raw packet logs from before the Heartbleed vuln. was public. If you find 18 03 (01 | 02 | 03) 00 03 01 in them, please let me know or post pcap files. Contact info: [https://www.eff.org/about/staff/yan- zhu](https://www.eff.org/about/staff/yan-zhu)

<comment><USER3> Are heartbeats typically visible in the raw traffic, or (after some point) do they wind up inside the secured stream?

(If the latter, this could be an unfortunate case where Perfect Forward Security, when enabled, also helps obscure exploits from later forensic discovery...)

<comment><USER4> It appears that you might be right, from the RFC:

"However, a HeartbeatRequest message SHOULD NOT be sent during handshakes. If a handshake is initiated while a HeartbeatRequest is still in flight, the sending peer MUST stop the DTLS retransmission timer for it. The receiving peer SHOULD discard the message silently, if it arrives during the handshake. In case of DTLS, HeartbeatRequest messages from older epochs SHOULD be discarded."

But that doesn't make sense to me because the PoC code didn't complete the handshake did it?

Edit: according to Google the reason is that OpenSSL does not honour the "SHOULD" part of the spec :/

</comment></comment></comment><comment><USER5> I must admit to being suspicious about this. I would consider myself very very careful about password and other security issues because of various human rights projects I work on, yet on 16th March at very unusual but clever time for attempting such a thing against me (at the time I would have tried this, if I was targeting me and collected relevant pre-attack information) someone from the UK used my exact and recently changed password to login to my email service - traced back to a very unusual location for attempting such a thing. Luckily the service I use for low-level mail security noticed this strange login and blocked it.

It has puzzled me quite a bit as nothing like this has (knowingly occurred to me before) and I take a lot of precautions (which for obviously reasons I'm not going to go into) against keyloggers, malware, MITM, etc etc. With such target hardening I was very suspicious of how it occurred.

Ofcourse maybe I was sleep talking my passwords again :)

<comment><USER6> A similar thing happened to me. Someone was repeatedly trying to access a gmail account of mine, which is strange because that account had not been active for over 5 years. They supplied the correct credentials every time, and the IP originated from some small village in China. I had also recently changed my password, so I don't think it was merely a coincidence. It is possible that I have been keylogged for 10 years without knowing it, but the timing is uncanny.

Edit: Keylogged for the past 10 years without knowing it, across 5 different machines, with different architectures and operating systems. :-)

<comment><USER5> Interesting...

If you don't mind me asking the question I always ask people when helping with their security (both cyber and physical) and eliminating an element of potential paranoia:

Would your work/life make you a worthwhile legitimate target? (don't mean to sound rude but I guess it differentiates between random attacks and targeted ones)

<comment><USER7> How do you even know what a legit target is anymore after Snowden dropping docs they spied on charities and Jr sys admins.

<comment><USER5> Valid point. I guess what I meant by legitimate target was "do something they want to specifically know about enough to relatively targeted attack" (ie: analyst wants to know something) as opposed to everyone else who they just want to scoop us as much data about but is currently of less interest.

<comment><USER7> I would say after watching this almost everybody could be a target of state interest from VCs to a janitor with a cellphone who works at a network they want into [http://youtu.be/3jQoAYRKqhg](http://youtu.be/3jQoAYRKqhg) (FosDem2014 presention) and especially if you have any kind of trust in an open source community and your patches are accepted blindly.

Personally if I were an evil intel agency I'd be going after GPU developers and manufacturers at all costs to get at their firmware sources or even possibly find ways to sabotage it at the source. It's the final frontier of awesome evilware potential.

\- the execution of GPU code, and transfer of data between device and host do not require admin privs so it will always run regardless of what the host system privilege settings are.

\- Malware w/Nvidia GPUs can be statically linked with the CUDA library in a standalone hidden file that never touches the operating system.

\- GPU memory is not shared with the CPU so encrypted malware can reside there undetected.

\- Run-time polymorphism: malware GPU code can be re-encrypted with a new random key thus mutate in completely random ways that would be difficult to detect even if you dumped the GPU memory on a regular basis.

\- GPU NSA code can easily access the screen framebuffer, and broadcast a live link of whatever somebody is doing.

\- GPU NSA code present the user with a nothing is wrong desktop pretending the virus scanner is still running, hiding daemons, presenting false browser screens hiding the fact SSL certs have been rejected, all sorts of evil.

</comment></comment></comment></comment></comment></comment><comment><USER8> This would be so easy for the NSA etc. to do that I think we have to consider it as inevitably having occurred.

All they would have had to do is take a close look at any new changes committed to OpenSSL and other critical infrastructure software. Surely they have people doing that -- they would be remiss not to.

<comment><USER9> Even easier, I would bet a lot of money that they have at least some rudimentary static analysis tools to detect potential targets, and this sort of memory error is pretty low hanging fruit for such a tool. To me it seems almost certain that they knew about it and they certainly exploited it if they knew.

The bigger question to me is how many of these bugs have they rooted out that have not been made public yet?

<comment><USER10> Why don't we have groups doing that sort of analysis on our behalf? Programmers are at a fundamental disadvantage when it comes to testing and verifying their own code. You can't trust a shop to verify itself when it comes to infrastructure this critical.

<comment><USER9> Yeah--it should be a no-brainer. Running such critical code through as many static analysis tools you can get your hands on should be standard practice. I wonder why Coverity and the rest havent taken it upon themselves. I remember a story about Coverity running their tool on random open source projects and emailing them about issues they found. Maybe OpenSSL is too far in the hole to start that now.

</comment></comment></comment></comment><comment><USER11> What worries me, is that the Snowden leaks didn't seem to have a strong emphasis on SSL encryption suggesting to me that they could circumvent it.

For reference take a look at this article from September. [http://www.reuters.com/article/2013/09/05/net-us-usa- securit...](http://www.reuters.com/article/2013/09/05/net-us-usa-security- snowden-encryption-idUSBRE98413720130905)

<comment><USER12> Snowden's files predate the existence of this vulnerability.

<comment><USER13> No, Snowden's files predate the _public knowledge_ of the vulnerability.

As far as I know, we presently have no way of determining whether or not the NSA had knowledge of the bug.

From the CVE[1], we see that OpenSSL versions from the very start in 2012[2] were vulnerable.

1 [https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-01...](https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-0160)

2 [http://www.openssl.org/source/](http://www.openssl.org/source/) (Jan 3 14:41:35 2012 openssl-1.0.1-beta1.tar.gz)

<comment><USER12> Snowden's files predate the _existence_ of the vulnerability. Many of his files were years old when he exfiltrated them. This vulnerability was created by a specific check-in that has been identified. That does not, of course, mean the NSA didn't use it, or even create it. Both are possible.

<comment><USER13> Oh, I see what you mean. Fair point.

(It's a wide time range of files he's released so far though, right?)

<comment><USER14> You'd think that, if any of his files actually covered such a possibility, he would have released that file by now.

</comment></comment></comment></comment></comment></comment><comment><USER15> Pardon me for being cynical about this, but from what we've heard about NSA hacking and industry collaboration I would say it's highly likely that a large number of the Certificate Authorities themselves are compromised by the NSA or GCHQ and so it renders the question moot.4 Certificate Authorities control > 90% of the market 3 of them based in the US and 1 in the UK. With access to the CA's keys they can sign any number of certificates they want.

<comment><USER16> Pardon me for being realistic, but I would say exactly the opposite. If the CAs were compromised, that would be the biggest story by far in Snowden's documents, and it would have appeared in the newspapers by now.

<comment><USER7> I would say they are compromised just by watching Moxie Marlinspike's presentation about the shitty state of CAs and how he was able to find signing certs just laying around in unprotected directories [https://www.youtube.com/watch?v=Z7Wl2FW2TcA](https://www.youtube.com/watch?v=Z7Wl2FW2TcA)

</comment></comment></comment><comment><USER17> As I've mentioned elsewhere, heartbleed combined with bulk data collection means all your historic communications can be read unless your provider was using Perfect Forward Secrecy.

I don't think this aspect is getting as much publicity as it should.

<comment><USER18> > _bulk data collection_

Including whatever the McDonald's free wi-fi might store? I'm not insinuating they were an actor, but is that how simple it could've been? Anything communicated over unsecure/not-secure-enough wi-fi could've been captured & apparently now decrypted using newly-acquired information?

I'm halfway sure that's what it means. But that would just be _crazy_ , right?

<comment><USER17> _Anything communicated over unsecure /not-secure-enough wi-fi could've been captured & apparently now decrypted using newly-acquired information?_

Yes. An attacker would have to collect that information, AND have grabbed the private keys from a vulnerable site. But there is nothing technically stopping that from happening. (And of course I expect there may be a market for those keys now)

 _But that would just be crazy, right?_

Yes. Crazy but possible.

</comment></comment></comment><comment><USER19> GCHQ have been known to attack IRC networks: [https://www.networkworld.com/community/blog/eff-cyber- attack...](https://www.networkworld.com/community/blog/eff-cyber-attack- against-hacktivists-cfaa-you-impunity-nsa-and-gchq)

</comment><comment><USER20> It should be illegal for a government to make use of botnets this way.

<comment><USER21> It is. They don't care.

<comment><USER22> All is legal for the sovereign. After all, the Law is _his_ tool: why would he consent to its use _against_ him?

<comment><USER23> I don't know if you're trolling or genuinely don't know how this works. If you don't, please read up on the constitution of whatever country you live in.

The respect for those constitutions has eroded significantly since the beginning of the century, but they still exist and we must still insist on them. Don't give up the achievements of the past that easily.

<comment><USER22> While we're handing out reading assignments, I'd encourage you to read something that _wasn 't_ assigned in junior-high civics class; perhaps Machiavelli? Political power has been exercised for millennia, and its nature is far closer to the caricature I offered than anything written in any newfangled constitution.

The point is that a constitution is not an _achievement_ , the "unlocking" of which would transform a society in any lasting way. It _might_ be more accurate to say that a constitution or similar document is an aspiration, but since few such have been fulfilled it's foolish to be surprised when we fall short. The fault is not in our constitutions, but in ourselves, that we are underlings. We knew when we built this monstrous war and imprisonment machine that it would be turned against us, yet we built it anyway.

<comment><USER23> The problem is that your statement "all is legal for the sovereign" is just plain false according to the most widely accepted definition of legality.

By propagating it as if it were true, you are in fact playing into the hands of those who want society to bow to a different notion of legality - one where the rule of law has been eliminated. That is, by writing what you write, you are needlessly conceding ground to the bad guys.

I know it's tempting to play (or be?) the jaded cynic. But it seems to me that if we want to keep politicians and government officials accountable to the laws, then a necessary (though of course not sufficient) condition for that is that we insist on calling their actions illegal when they are illegal.

</comment></comment></comment></comment></comment></comment><comment><USER24> As I cant access this page from Chrome (doesn't let me because "it's not secure") here is the archive.org link

[https://web.archive.org/web/20140410171401/https://www.eff.o...](https://web.archive.org/web/20140410171401/https://www.eff.org/deeplinks/2014/04/wild- heart-were-intelligence-agencies-using-heartbleed-november-2013)

Could it be that because of heartbleed now i can't access eff.org?

<comment><USER2> Are you joking? If not please report what error you're getting in Chrome.

<comment><USER24> No joking, where can I report that?

<comment><USER2> For anyone following along at home, we looked into this and it seems to be caused by the fact that you're using an older operating system that doesn't ship with the StartCom CA cert that eff.org uses. So probably not an attack. :)

<comment><USER25> EFF uses StartCom?!

</comment></comment></comment></comment></comment><comment><USER26> Probably not, If they did they would have raised these security flaws to the general public in the interest of security.

</comment><comment><USER27> This wouldn't surprise me one bit. Governments employing hackers to exploit whatever they can get their hands on is not something new.

Also, makes one think what other exploits are out that are being used, yet, we're not aware of it?

<comment><USER28> My theory is that basically all of our traffic is compromised, we just don't know it yet. It seems clear that the NSA has been actively working to find and exploit every vulnerability they can, and they have the power of a well-funded concerted effort, secret physical access, and gag orders all on their side. I bet they can do a whole lot more than what we know.

</comment></comment><comment><USER29> Very shameless plug: we just launched a t-shirt campaign with teespring.com. All proceeds will be donated to the OpenSSL Software Foundation:

* Campaign: [http://teespring.com/hbts](http://teespring.com/hbts)

* HN thread: [https://news.ycombinator.com/item?id=7567461](https://news.ycombinator.com/item?id=7567461)

<comment><USER30> I don't get the number 1396891800 - what does it mean?

<comment><USER31> Number of seconds between Jan 1 1970 and the discovery of Heartbleed, I suppose. The time_t time stamp.

<comment><USER30> That makes me feel less dumb - I thought I was missing something obvious. Thx!

</comment></comment></comment></comment><comment><USER32> [http://en.wikipedia.org/wiki/Betteridge's_law_of_headlines](http://en.wikipedia.org/wiki/Betteridge's_law_of_headlines)

I don't think so, mostly because to get useful information out of memory after only _one_ heartbeat would be quite lucky.

If this were an actual attack, I think we'd see many more heartbeats in Koeman's logs.</comment>


Why American Airlines Chose Samsung Over Apple When It Bought 17,000 Tablets - <AUTHOR> http://www.businessinsider.com/american-airlines-picks-samsung-not-apple-2013-3 <comment><USER2> The article probably should have stated the obvious - it is cheaper! When it comes to buying a thin client for enterprise applications, there must be sufficient feature parity between an iPad and Galaxy Tab.

As the article mentions, they did this only for volume i.e. flight attendants and technicians. The pilots still get an iPad.

<comment><USER3> The Galaxy Notes are more expensive than iPads...

<comment><USER4> Since the price was not mentioned, who knows what they paid for the tablets. I doubt that it was a usual retail price for 17000 tablets.

</comment></comment></comment><comment><USER5> This reads as basic marketing fluff for Samsung. Why is the Note preferable to the Galaxy Tab, let alone the iPad? The article certainly doesn't give a reason.

I'm actually interested in knowing the advantages of the Note in that use case. To me the Tab 7.7 I gave a friend for Christmas seemed like the best possible form factor for a mid sized Android device. 'Phablets' strike me as a wacky compromise.

<comment><USER6> The Note fits in a pocket. For the Tab you have to have a special pocket or a purse to carry it everywhere. Phablets are a compromise, but after you use the Note for one day as a phone, you will say "Wow, what a small screen my old phone has! How did I manage to use it?!"

<comment><USER7> > The Note fits in a pocket

Maybe in a large man's jeans or trousers but not women's jeans. Even the iPhone 5 doesn't properly fit.

</comment></comment></comment><comment><USER8> I feel Apple will largely win the mobile Enterprise market (now fairly open with exit of RIM) while Android will win over the Education market (newer growing market).

To oversimplify, IMO businesses are a little less cost sensitive than consumers and very concerned about security. I think Apple will push the security message against 'open source' to get their product across the line. Plus they do work a little more simply, so less training required.

Androids 'cheap' will let them win Education. The simple fact that if a school uses Apple products there will always be a few parents battering the principles door saying why do I have to buy this _insert apple product_ when I can but an _insert android product_ at 30% less.

And somewhere in there is Windows Mobile for enterprise if they get it right and a million other factors I don't know, don't exist yet or just make up the grey shades in my point I didn't bother to include...

Tl:dr Article is not worth reading but the tittle does initiate an interesting conversation as to what are the levers that drive businesses to choose their tech.

<comment><USER9> There is another reason why Android might win the education market: it is not as closed as the iOS universe. You can build your own OS (AOSP) and you can easily deploy custom applications on it. No need to get a certificate from Apple first.

</comment></comment><comment><USER10> The battle of which mobile device will over enterprise deployments is still on going. I think Android will win though Microsoft has a strong position. I don't see a winner yet because of one question:

Which vendor has the best support?

By support I mean someone to call when something breaks. Or when you need to code out custom widgets. Or when a device is buggy and needs to be replaced. Microsoft has been traditionally good in this area. Apple kind of sucks at it. Android is starting to build out an ecosystem but the customizability is all there.</comment>


One Biohacker's Improbable Bid to Make a DIY Covid-19 Vaccine - <AUTHOR> https://www.bloomberg.com/news/articles/2020-06-25/one-biohacker-s-improbable-bid-to-make-a-diy-covid-19-vaccine <comment><USER2> The first livestream has been shifted from Sunday to Saturday June 27th at 11AM PT.

<comment><AUTHOR> Thanks for the update!

</comment></comment><comment><AUTHOR> Here is a link to his course for making the vaccine: [https://docs.google.com/document/d/1l8_5vY83bh- fG1G6S6lO7F4N...](https://docs.google.com/document/d/1l8_5vY83bh- fG1G6S6lO7F4NLIW- _FgZsnLZq0mL_Tw/edit?fbclid=IwAR3p4W1SGAQD3G5L5D_0Lg54AvuDkp6bJVHrv9CXgoMASk3He7qwHGUEGZc)

And a link to the paper they are following: [https://science.sciencemag.org/content/early/2020/05/19/scie...](https://science.sciencemag.org/content/early/2020/05/19/science.abc6284)</comment>


Wind River VxWorks Platforms 6.9 RTOS [pdf] - <AUTHOR> https://www.windriver.com/products/product-notes/PN_VE_6_9_Platform_0311.pdf <comment><USER2> Last time I checked, ~US$30k per development license...just in case anyone was wondering.

<comment><USER3> I suppose the idea of selling only to those who can pay is in Wind River DNA.

I worked for a company that WR bought in 2000. We had a product where it was possible for a small shop to get a license for 1000 USD or so. Wind River canceled that and the cheapest license became at least 15K. The claim was that the money came from big customers and it was "wasteful" to deal with smaller guys. Yet it was the small guys that provided effective advertisement for the product to bring attention of bigger customers.

<comment><USER4> I worked for a company that WR bought around that time. WR management seemed to be into "retail therapy" \-- buying stuff for the sake of the purchase with no concern or intent on how to integrate it in.

</comment></comment></comment><comment><USER5> Any comments on this submission? 6.9 is somewhat old by now,and the dev tooling around it "works" but is quite buggy sometimes...

<comment><USER6> My "favorite" bug on 6.9 is that if number of symbols in your binaries exceeds some magic limit, workbench will fail 100% of the time to connect to the remote debug server... We exclusively use logs to debug on VxWorks.

<comment><USER7> It's been a while since I used VxWorks, but I hated Workbench as well (prefer using a plain old makefile). One of my favorite features of VxWorks was the shell you could get on a serial port. It was quite powerful, even letting you do REPL with C functions which was pretty awesome for debugging.

Another "favorite" feature of VxWorks was how they disabled floating point by default in tasks (at least on the LEON and PPC). Always led to confusion and frustration by someone new to the platform.

</comment></comment></comment><comment><USER8> Can someone explain to me why anyone needs this? I once had to try to use it with some weird Intel SOC but I gave up. So...Real question.

<comment><USER9> Are you asking why anyone needs an RTOS? It provides a very robust solution that sits somewhere between running a plain old OS like Linux and running baremetal C on a microprocesser. You get the speed benefit of baremetal but also get a scheduler and a whole bunch of multiprocessing structure to run lots of applications at once without worrying about manually switching or trying to create a massive main.c to do everything. VxWorks specifically runs on lots of industrial computers such that tasks can get done in a timely manner with little overhead.

VxWorks powers the Mars Rovers Sojourner, Spirit, Opportunity, Curiosity, Mars Reconnaissance Orbiter, and SpaceX Dragon. VxWorks also powers BMW iDrive, the Apache Longbow helicopter, and the Apple Airport Extreme and Linksys WRT54G routers. It's in a lot of things and what it's not in, FreeRTOS is probably the RTOS of choice.

<comment><USER10> According to the SpaceX flight software team neither Falcon or Dragon run VxWorks, they run a custom Linux (see [0]).

[0] [https://space.stackexchange.com/questions/9243/what- computer...](https://space.stackexchange.com/questions/9243/what-computer-and- software-is-used-by-the-falcon-9)

<comment><AUTHOR> They won't be using "off the shelf" Ubuntu or Fedora... this'll be the linux kernel with RTOS patches (either done on their own or using something like "RTLinux/Wind River Linux").

<comment><USER10> I never said they would be running a vanilla distro though. I said "custom" for that reason.

</comment></comment></comment></comment></comment><comment><USER11> ok, VxWorks exists. Is there something new and interesting?</comment>


Oh, hey, DHH loooovvvess the MacBook Air - <AUTHOR> http://www.37signals.com/svn/posts/858-the-macbook-air-could-easily-be-the-only-machine <comment><USER2> I'm a little puzzled as to why this got up voted. :( Would it still be significant news if it was Everyday Joe's love letter to the MacBook Air? And if it's because it's DHH, does his love of MacBook Air significantly change a market or industry? And if all else fails, is it interesting, and did I learn anything? In my mind, it's a resounding no.

<comment><USER3> Obviously, you have no idea who DHH is.

Obviously, DHH is a god. He's the man. He's awesome.

Obviously, you have no significance in your life. Live through DHH my friend.

<comment><USER4> Actually, before I read this article I had no idea who DHH was. Oh, he wrote Rails? I guess I assumed somebody must have, but I never especially cared what the guy's name was. Is this guy really famous enough to be referred to by his initials, RMS-style?

And I still don't care whether he likes the Macbook Air.

<comment><AUTHOR> fame isn't the sysadmin that allocates accounts. if i want to call him dhh or rms or pg, i can.

</comment></comment></comment></comment><comment><USER5> Note that DHH is using the solid-state drive version of the Air, so consider that when reading, "The machine is plenty fast for everything I do with a computer."

</comment><comment><USER6> Seriously, you can program on a 13" screen? My brain gets angry that I can't see enough code even on my non-portable 17" laptop at 1920. Or is that only when attached to the external 24"? (kinda cheating...).

</comment><comment><USER7> 3 years ago when people said they didn't need to pay a premium for speed and would take a cheap Dell that did everything they need, Mac fans said "But I do it faster!"

Now speed doesn't matter.

Everyone's insane.

F __* marketing.

<comment><USER7> Note to self: don't point out the silliness of Mac addicts.

<comment><USER8> No, more like:

1\. Don't treat "Mac fans" as a homogeneous group, and don't attribute some random opinion to them 3 years ago, observe that one particular Mac fan has a different opinion now, and accuse "Mac fans" in general of hypocrisy. That makes no sense.

2\. Don't equate "a cheap Dell laptop that does everything you need" with a MacBook Air. That is like comparing apples and oranges.

<comment><USER9> Even if you consider this mac fanboyism, I must say the air is simply a feat of design and engineering. Its hard to deny that. And the price, while being expensive because it is Apple hardware, is sill fairly reasonable. Its not mac fanboyism to praise something that is genuinely aesthetically pleasing.

I do however, wish that I could in fact afford one. Which, I can't.

<comment><USER10> I was a pretty skeptical regarding the Air. But, just today I went down to the CambridgeSide Apple Store and played with one of them. Its really quite nifty. The weight and thickness are really astounding. The numbers don't really mean much to me... having felt it now though, I can understand why some people love it.</comment></comment></comment></comment></comment>


Ask HN: How to get started writing as a programmer? - <AUTHOR>

Hi, I'm a programmer. Like most of us are.
There are a number of people, programmers, if not very good programmers, but yeah, programmers who are great writers, like for example Pual Graham, Aaron Swartz, Quinn Norton and others in the line.
But my simple question is, how and where do I get started writing ? I mean, how do I "get" to my audience ? And how do I tell people I write ? ... <comment><USER2> The best way to get started is to do it. It is just like moving up the developer ladder from journeyman (not knowing anything about it and working with junior or above developers to learn the ropes) to becoming a senior or master developer. Starting small is the best way to get started, just as with everything in life so you gain experience with each project without being overwhelmed and have time to do things the write way. You can start by writing a nice 150 page book on going over your workflow on creating an app, from no code to fully functional code, encourage no copy and pasting and include pictures of the finished product at the end of each code section. Spend some time working on a theme, front and back cover and insure you have the copy proofread by professionals.

</comment><comment><USER3> Writers write because that's what they do. Paul Graham has been at for more than 20 years...there was a PhD dissertation and then two books on Lisp before all the essays. Joel Spolsky and Dave Weiner developed a following because they wrote something interesting...regularly without worrying about audience. What they focused on was writing.

There's no such thing as writing in your head. If it's not written its not writing. So start a journal for yourself. Start a blog. Write. Everyday. If it's reasonably good, a few people may care.

Good luck.</comment>


Sony Ericsson's new Virtual Lab enables testing of phone apps through your browser - <AUTHOR> http://www.fiercewireless.com/press-releases/sony-ericsson-and-mobile-complete-launch-virtual-lab-service-remote-testing-java-me-a

<comment><USER2> I've been offering this service for Hecl for something like a year:

<http://builder.hecl.org/>

<comment><AUTHOR> This is cool. So you turn your simple Hecl scripts into Java ME code? Do you use Sun's reference WTK for the emulation?

Supposedly the deal with Sony's new service is it uses the ACTUAL phone hardware connected to actual networks to test on. You must reserve timeslots or something like that, but then when you are testing (via the browser), you are seeing what is running on the actual hardware on a real wireless network. And you will soon be able to test on phones not yet released too.

But the deal is it costs money.. And I'm not sure how much. Might be too expensive for a small startup.

<comment><USER2> The emulator I use is an open source one (linked to from the site).

Obviously, they're providing something that is more specific to their hardware, but the general concept is certainly not that difficult.

Hecl is an interpreter, so I don't really need to turn it into anything, just wedge it into a .jar file that the emulator can digest.</comment></comment></comment>


Overlay.NET: An easy-to-use overlay library written in C# - <AUTHOR> https://github.com/lolp1/Overlay.NET <comment><USER2> I hoped to see a magnificent hack implementing [https://en.wikipedia.org/wiki/Overlay_(programming)](https://en.wikipedia.org/wiki/Overlay_\(programming\)) (beter description at [http://www.i-programmer.info/babbages-bag/205-virtual- memory...](http://www.i-programmer.info/babbages-bag/205-virtual-memory.html)) for no reason other than _because we can_ , but alas.

</comment><comment><USER3> Hello everyone,

I am the author of this library. I'd like to say before anything else, thanks for all the attention the community here gave the project.

I went from 60 stars to 200+ overnight. I only found out it was from here when I looked at the traffic data for the project. That is motivating and rewarding as a newer developer who started during C# 6.0.

I intend to review the rules and general rules of thumb over here and contribute where possible.

Second, yes it will be under MIT in its current form. You have my word you are safe to use it under MIT conditions. If you would like to use it before I add the license.I can not guarantee all versions in the future will, but any release out and for the next foreseeable release MIT it shall be.

I intend to fix the NuGet and bugs that exist this week and get a stable version, a NuGet, and license up. From there I will improve what I can and add in stable d3d support with an easy way to load your dll and the clr into another process (using this [https://github.com/lolp1/DomainWrapper](https://github.com/lolp1/DomainWrapper)).

From there you will be able to detour Present or EndScene in both x64/32 d3d11/9.

Thanks again!

</comment><comment><USER4> In windows 8/10 can you still use the 'glass' blur in overlays? I wrote a similar thing on top of XNA a few years ago and remember there existing a flags/clear color combination which made the background of the frameless window look like the blurry aero glass.

The start menu in W10 has that effect, so there must be still a way to do it?

<comment><USER5> Indeed, you can use the same effect as the start menu via the undocumented API SetWindowCompositionAttribute [1]. Note it only works properly with frameless windows, if you have a titlebar there are slight artifacts around the window.

[1] [http://withinrafael.com/adding-the-aero-glass-blur-to- your-w...](http://withinrafael.com/adding-the-aero-glass-blur-to-your- windows-10-apps/)

</comment></comment><comment><USER6> What would you use it for?

<comment><USER7> I write tools to interact with games - ctrl+p to do a price lookup for item under mouse, track which cards have been played/are in your deck, that sort of thing - and it'd be really useful there. DLL injection is kind of a pain (and it can be detected as cheating, which this little framework probably doesn't solve), so the fastest route for prototyping ideas tends to be AutoHotKey or something else that can spawn tooltips easily on top of D3D windows. Obviously that has tons of drawbacks, so if this is quick & easy and runs on a 'real' platform like .NET, it'd be a lot more powerful.

</comment></comment><comment><USER8> Can I use this to overlay everything on the screen? Similar to what Humanized Enso does: [http://zapp3.staticworld.net/downloads/graphics/screenshots/...](http://zapp3.staticworld.net/downloads/graphics/screenshots/70391f.jpg)

<comment><USER9> You could probably modify it to do this, yes. It's creating a transparent window that overlays the host window, so you could use the same technique to overlay the desktop with a transparent topmost window.

I've done this before for similar input purposes and the only caveat is that some applications are able to 'push' topmost windows to the bottom - there are some workarounds, though.

You can also do this with raw GDI (if you do the pixel pushing yourself) or with GDI+, since all you need at the base level is an ARGB buffer to feed to the windows compositor.

For example, I wrote an overlay for my dualshock4 driver that would visualize touchpanel inputs at the bottom right corner of the screen and describe the inputs it had synthesized from them. [https://github.com/kg/DS4_PSO2/blob/master/DS4_PSO2/GestureO...](https://github.com/kg/DS4_PSO2/blob/master/DS4_PSO2/GestureOverlay.cs#L366)

<comment><USER2> _" I've done this before for similar input purposes and the only caveat is that some applications are able to 'push' topmost windows to the bottom - there are some workarounds, though"_

Are there? I would think that is a weapons race you cannot win. If you can force your window to the top of all other windows, including those of other applications, so can others.

The only way out of that is if the OS has a notion of "this is the one and only topmost full screen window".

<comment><USER10> > Are there? I would think that is a weapons race you cannot win.

You can win. One implementation is to have a separate thread whose sole purpose is to send set topmost messages in a tight loop (I've done this).

You win, or hang the UI up in a topmost live-lock.

<comment><USER2> At best, I would call that "You win or you lose". Yes, there are all kinds of hacks that might give you a better chance, and you may be able to make it very likely you will get what you want, but to do so you have to be very naughty.

For example, another hack involves removing the real screen from the OS and creating a virtual screen of the same dimensions and bit depth, intercepting calls to the virtual screen, and essentially writing one's own compositor in the video memory of the real screen. If my program does that, your messages to move your window to the top will never work. Your window may end up on top on the virtual screen, but that's not what the user looks at. If you counteract by using the same trick, how do you know that want you think is the real screen isn't already a virtual one created by a program that ran earlier, of that a program running later doesn't enumerate the hardware and figures out where the real memory lies?

I also could try and patch your program when I detect it, patch it in memory to send fewer or no such messages, intercept its messages, or even crash it when I see it, etc.

It remains competition for a scarce resource; because of that, I don't see how two identical copies of such a program could both win, let alone if I run ten of them.

</comment></comment></comment></comment></comment><comment><USER11> It doesn't seem to support old school WinForms applications, or does it?

<comment><USER3> Next version has a form based overlay (not WPF/Directx) .. does that count? :)

</comment></comment><comment><USER12> License?

<comment><USER13> On a reddit thread, author had stated preference for MIT but I don't think he has added the license file yet which is unfortunate.

<comment><USER14> This looks like the thread that you're referring to. [https://www.reddit.com/r/csharp/comments/5b6qe2/overlaynet_n...](https://www.reddit.com/r/csharp/comments/5b6qe2/overlaynet_now_with_basic_directx_support/d9nyl6s/)

In response to a question about the license, 7 days ago the author stated "Forgot to add it. It will be under MIT."

<comment><USER3> I am the author and I will be releasing it under MIT - you can take my word for it you can use it freely or wait a week or two for another commit and license add. Sorry for inconvenience.

</comment></comment></comment></comment><comment><USER15> This is great, thanks!</comment>


The 10,000 Year Clock - <AUTHOR> http://longnow.org/clock/? <comment><USER2> I haven't heard anything about this in 3+ years; don't assume that the page is super-current. But that's kind of the point of the whole project: to make you think of yourself on a longer timescale. The engineers don't want it to be ready in a couple years; the process mirrors the art that it is producing with lots of planning, careful thought about how to get it all to last, a gradual drilling process for the cavern, etc.

I do still wish that I knew how long it'll be before I can visit it, but I take some comfort in the idea that I'll need to look at the bigger picture and wait patiently.

<comment><USER3> Due to its fascinating subject matter, it ends up being one of the most- submitted stories on HN. There are various others articles and background stories around its engineering and funding as well: [https://hn.algolia.com/?q=10%2C000+year+clock#!/story/foreve...](https://hn.algolia.com/?q=10%2C000+year+clock#!/story/forever/0/10%2C000%20year%20clock)

</comment></comment><comment><USER4> I see the future. There will be a bunch of dummies that think the word will end when the clocks time is up.

<comment><USER5> Some Mayan probably said exactly the same thing a few thousand years ago.

<comment><USER6> That's the joke.

</comment></comment></comment><comment><USER7> I just finished reading Anathem by Neal Stephenson. Highly recommended for anyone who's interested in the concept.

<comment><USER8> It's a great book, but it takes a LONG time to read. Not just because of the length, but the concepts make it slower to read than a simple fiction book.

<comment><USER9> Also, the first 200 pages seem to take forever. I almost stopped reading. Then next 200 pages go by much faster, then the next 200 even faster, and so on...

<comment><USER10> I like most of Stephenson's writing, but the first fifty to a hundred pages made me want to throw the book against a wall. But I persevered, and I think it's one of his best books, and well worth the initial frustration.

<comment><USER7> I think the initial difficulty expertly captures for the reader just a small taste of the experience of the culture shock the avout have when they come into contact with the outside world after being closed off from society for x years.

</comment></comment></comment></comment></comment><comment><USER11> The Interval ([http://theinterval.org/](http://theinterval.org/)) is a bar/cafe/museum in SF by Long Now that has details on this project as well as a pretty cool cocktail bar. Worth a visit.

<comment><USER12> Indeed! They also recently had installed a cool chalkboard drawing robot:

[http://biorequiem.com/photoflurry/chalk-drawing-robot- unveil...](http://biorequiem.com/photoflurry/chalk-drawing-robot-unveiled-at- the-interval)

</comment></comment><comment><USER13> Looks like I'm not the only one who followed this up after reading about Danny Hillis and Applied Invention today.

</comment><comment><USER14> Never could figure out if this group is some sort of a cult or just a bunch of yahoos but it's damn cool either way. Reminds me kinda of the Georgia Guidestones en.wikipedia.org/wiki/Georgia_Guidestones</comment>


Service Changes for U.S. Customers - <AUTHOR> https://www.bitfinex.com/posts/216%C2%A0 <comment><USER2> This doesn't surprise me, and I expect other large exchanges to follow suite.

Although the intentions of many US cryptocurrency regulations are to protect US residents/investors, they are instead leading to the exclusion of US residents from everything cryptocurrency-related.

No one wants to touch US residents when cryptocurrencies are involved. Exchanges turn you down. ICOs tell you that you are not allowed to participate or outright block you. Gambling websites don't want to touch you either, of course. Neither do derivatives platforms, some sites with margin trading, and many sites with other types of speculation, such as sports betting, prediction markets, etc.

It's easier to just exclude US residents instead of dedicating significant time and resources in trying to comply with new regulations that don't fit new technology as well as they should.

<comment><USER3> Additionally, most international banks refuse US citizen unless you bring a lot of money, because the US taxes all citizen (not just the money made in the US, or money made by people while they were in the US).

And this goes even further, for example due to US patent laws, and export regulations (encryption are weapons of war, etc) even lots of open source software is not directly legal in the US (and many projects outright ban US citizen from working on these, while living in the US - this is also why the OpenBSD hackatons originally were a thing[1]).

        ________________________

[1] [http://www.openbsd.org/hackathons.html](http://www.openbsd.org/hackathons.html)

<comment><USER4> _Additionally, most international banks refuse US citizen unless you bring a lot of money, because the US taxes all citizen (not just the money made in the US, or money made by people while they were in the US)._

It's less about the taxation per se, which has been true for decades but not generally dissuaded foreign banks. The big thing that makes banking Americans annoying is FATCA compliance: [https://en.wikipedia.org/wiki/Foreign_Account_Tax_Compliance...](https://en.wikipedia.org/wiki/Foreign_Account_Tax_Compliance_Act)

FATCA levels a highly, highly asymmetric weapon against Americans who desire to bank abroad: if the bank is unwilling to pay a $X ~ $Y0 million dollars in compliance costs and technology changes (amortized over all of its customers), they will lose access to USD clearing. That's an unacceptable outcome for most banks, even banks that one does not generally think of as being heavily exposed to America.

Thus my recommendation for Americans living abroad: if you want to get an account with the least hassle, you either want the most gigantic bank you can think of in your country (which can absorb the compliance costs over their thousands of US customers because, hey, your expat population does have to bank somewhere).

<comment><USER5> It seems like there's a missing `or` clause, if I didn't go with the most gigantic bank.

<comment><USER6> There isn't always an "or" when it comes to feasible, tolerable solutions - especially when working with something designed in part for anonymity and institutions, both financial and government, that will not tolerate anonymity.

<comment><USER7> But there is usually an "or" following the word "either"

</comment></comment></comment></comment></comment></comment><comment><USER8> According to this video (possibly fake, as the source has been removed), Bitfinex often played cat-and-mouse games with regards to banking partnerships in the US:

[https://youtu.be/62cvxPIDBGY](https://youtu.be/62cvxPIDBGY)

The spoofy and Tether thing does lead to some suspicious stuff as well

</comment><comment><USER9> Between this and the Bitcoin split thing[1], I wonder if there's something going on behind the scenes at Bitfinex? I read through an article about alleged "spoofing" at Bitfinex[2] but not sure if I 100% believe it. Does anyone have any ideas on what's going on the general shadiness of Bitfinex?

[1]: [https://www.bloomberg.com/view/articles/2017-08-02/bitcoin-e...](https://www.bloomberg.com/view/articles/2017-08-02/bitcoin- exchange-had-too-many-bitcoins)

[2]: [https://hackernoon.com/meet-spoofy-how-a-single-entity- domin...](https://hackernoon.com/meet-spoofy-how-a-single-entity-dominates- the-price-of-bitcoin-39c711d28eb4)

<comment><USER10> After Bitfinex was hacked in 2016[1] to the tune of 120,000+ bitcoins, I wouldn't trust them with a penny of my money. There's no way they've secured their infrastructure and process to be anywhere near even basic PCI-DSS, let alone institutional banking standards. Big surprise they couldn't comply with USA KYC/AML laws.

[1] [https://en.wikipedia.org/wiki/Bitfinex_hack](https://en.wikipedia.org/wiki/Bitfinex_hack)

<comment><USER11> >There's no way they've secured their infrastructure and process to be anywhere near even basic PCI-DSS

This simply couldn't be further from the truth.

<comment><USER10> They're in Canada, right? I would like to see what kind of financial data security standards they adhere to, and what kind of physical/process security they've implemented since "the hack".

For all we know Bitfinex is just a few guys in an open layout office in Toronto running their Node.js "exchange" on Linode.</comment></comment></comment></comment>


Amazing OpenGL demo: Never Seen the Sky - <AUTHOR> https://github.com/unconed/NeverSeenTheSky

<comment><USER2> Note:

> Sorry! This demo requires WebGL and Web Audio support. Please use Google > Chrome to view it.</comment>


Why Americans Just Can't Quit Their Microwaves - <AUTHOR> https://www.theatlantic.com/health/archive/2019/07/microwaves-eternal-promise/593767/ <comment><USER2> The premise of the article is a little weird. Like, nobody's suggesting you should abandon your blender or toaster or dishwasher. Why this?

A microwave is exceptionally safe, and highly efficient at heat transfer. There really is no device that can compare for its specific set of use cases. Sure, it's not so great for making toast or grilling a steak or whatever, but it's a device that serves a purpose and does it well.

<comment><USER3> Because microwaves are mostly used to reheat ready meals, not to cook from scratch. That's potentially a public[1] health problem if there's little regulation of the sugar / fat[1] / salt[1] content of those meals.

[1] Public health talks about the whole population, not just the individual. This means you look at risks across a large group of people. Some people eat fat and are fine, some people eat salt and are fine, but across the population these things are eaten too much and they cause harm.

<comment><USER4> How about reheating leftovers of the previous nights home cooked meals? I mostly use mine for that.

</comment></comment></comment><comment><USER5> I make soups or curries in large quantities and freeze them in single servings. When I get home from work I can serve a healthy meal I cooked in six minutes. Why would I get rid of the microwave? What's wrong with it?

<comment><USER6> > What's wrong with it?

Nothing, the article mostly talks about frozen / pre made low quality meals.

</comment></comment><comment><USER7> I do use the microwave for actual cooking. In particular, if you follow the keto diet, the microwave can be your best friend.

Some examples:

1\. bacon: either straight on a plate, or on some paper towels to absorb the excess fat

2\. eggs: I content that there is no faster way to cook eggs than with a silicone egg poacher [1] in a microwave

3\. seafood stew: just put a pound of frozen seafood in a microwaveable steam bag [2], some cream and salt, and in 5 minutes you have a delicious stew

4\. roasted vegetables: you do this in the oven, but before you put the vegetables in the baking tray, you steam them in a microwave using the steam bag [2] for 2-3 minutes. This reduces the cooking time from one hour minutes to 20 minutes.

[1] [https://www.amazon.com/gp/product/B007PC4FAM/ref=ppx_yo_dt_b...](https://www.amazon.com/gp/product/B007PC4FAM/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1)

[2] [https://www.amazon.com/Glad-SimplyCooking-Microwave- Steaming...](https://www.amazon.com/Glad-SimplyCooking-Microwave-Steaming- Bags/dp/B001KYS5LG/ref=sr_1_6?keywords=glad+steam+bags&qid=1563444730&s=gateway&sr=8-6)

</comment><comment><USER8> It never even occurred to me to give up my microwave until a coworker recommended cooking popcorn in a pan. We have a popcorn maker which is already better than a microwave at making healthy unflavored popcorn, but the idea stuck. What else was there that I needed a microwave for? Everything else can be cooked in a pan or oven.

<comment><USER9> uh I think the main use for microwaves is reheating food. I've never used it cook anything.

<comment><USER10> Heating water-rich food uniformly and gently from the inside rather than by propagating heat through the food surface, which is a rather unique property that allows specific techniques and results.</comment></comment></comment>


UBiome Offices Searched by FBI - <AUTHOR> https://www.wsj.com/articles/ubiome-offices-searched-by-fbi-11556301287 <comment><USER2> I was stupid enough to sign up for their SmartGut program. They never clearly disclose how much the test costs if you opt to use your insurance. They're $2700. Not only that, they make it seem like you purchased 6 tests, but the $2700 charge is per test. Their billing practices are questionable, at least from the perspective of the consumer. On the BBB page for uBiome tons of people are complaining. While I can afford the surprise charge, I feel bad for the many that surely cannot.

<comment><USER3> They promised to only take what your insurance paid and not bill you any remainder. My insurance refused one and paid another. They ate the first one. But it's a ridiculous amount to bill my insurance so i didn't do anymore. They got shady when they started sending emails where just by clicking they would "resequence" your sample.

<comment><USER4> >They promised to only take what your insurance paid and not bill you any remainder.

This sounds like insurance fraud. [0]

[0] [https://www.ajmc.com/contributor/andria-jacobs-rn-ms-cen- cph...](https://www.ajmc.com/contributor/andria-jacobs-rn-ms-cen- cphq/2015/07/waiving-copays-and-deductibles-waves-a-red-flag)

<comment><USER3> What about all the $5 drug copay promotions? A drug company just refunded my copay.

<comment><USER5> It's ok with private insurance, but not public insurance. The federal govt considers it an inducement.

<comment><USER6> It's not Ok with commercial insurance either.

<comment><USER5> If by "ok" you mean legal, yes, it is legal for drug companies to offer co-pay assistance.

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> I did uBiome a couple of years ago, paying out of pocket ~$100-$200 or so for a kit. It was ahead of its time and I think it could have a very bright future.

At the time the it was overhyping the present-day usefulness of the data but wasn't lying about it, similar to 23andMe. I'm hoping this is a 23andMe-like incident and not a Theranos, where the latter I guess blatantly lied about the accuracy of its tests. IMO (I occasionally work with microbiome data in our bioinformatics lab), your microbiome data today is nearly useless but will be indispensable in the future as the technology improves, and we need early adopters to use services like uBiome to get there.

I know nothing about uBiome's newer more expensive products and/or how they bill insurance companies. I hope they're not doing anything illegal there, or if there is a simple settlement that can be reached a la 23andMe's FDA case. I can't imagine they are doing anything more unethical than anyone else in the health insurance industry, which IMO is rotten to the core (I just saw my friend's insurance bill for a normal healthy birth + 2day stay at a hospital in SF for over $60,000 before insurance).

<comment><USER5> Reimbursement is everything in the healthcare industry and the gov't loves to lay the smack down on companies who play fast and loose with the billing rules, particularly if Medicare or Medicaid is involved.

Yes, your friend's hospital bill is ridiculous, but that's doesn't mean it was fraudulent. If uBiome is breaking the rules, they are going to be severely punished.

</comment></comment><comment><USER8> Video of employees walking out of the building: [https://twitter.com/sallyshin/status/1121854424727425024](https://twitter.com/sallyshin/status/1121854424727425024)

<comment><USER5> Anyone else notice the "DCIS Police" on the cops shirt?

 _DCIS protects military personnel by investigating cases of fraud, bribery, and corruption; preventing the illegal transfer of sensitive defense technologies to proscribed nations and criminal elements; investigating companies that use defective, substandard, or counterfeit parts in weapons systems and equipment utilized by the military; and stopping cyber crimes and computer intrusions.

Priorities: Health care fraud committed by providers that involves (a) quality of care, unnecessary care, or failure to provide care to Tricare‐eligible service members, retirees, dependents, or survivors; or (b) significant direct loss to DoD's Tricare Management Activity._[1]

Maybe they just needed help with the raid and DCIS was available? Or maybe uBiome was ripping of gov't insurers?

[1][https://en.wikipedia.org/wiki/Defense_Criminal_Investigative...](https://en.wikipedia.org/wiki/Defense_Criminal_Investigative_Service)

<comment><USER9> DCIS explicitly, from the same source, is responsible for investigating, among other things:

"Health care fraud committed by providers that involves (a) quality of care, unnecessary care, or failure to provide care to Tricare‐eligible service members, retirees, dependents, or survivors; or (b) significant direct loss to DoD's Tricare Management Activity."

Given the general concerns about UBiome, that responsibility has to be why DCIS is involved.

</comment></comment></comment><comment><USER10> There's not much in the way of details in this article. Some other news sources are suggesting the company is being investigated for how they're billing insurance companies specifically: [https://www.cnbc.com/2019/04/26/the-fbi-just-raided- ubiomes-...](https://www.cnbc.com/2019/04/26/the-fbi-just-raided-ubiomes- office-for-billing-practices.html)

</comment><comment><USER11> So who are the board members who signed off on its current business plan? Don't see anything on the site about the corporate board, just the SAB. The Crunchbase list ([https://www.crunchbase.com/organization/ubiome/advisors/curr...](https://www.crunchbase.com/organization/ubiome/advisors/current_advisors_image_list#section- board-members-and-advisors)) seems outdated in light of the funding it has received from Andreessen, OS Fund, and 8 VC.

</comment><comment><USER12> My gut bacteria was destroyed by antibiotics plus after my latest round I ended up with the c-dif toxin that can be fatal. My microbiome results were so important in getting my gut bacteria recovered and taking the right probiotics to do it. So, anyone saying the results are useless obviously never saw a set of them. I'm disappointed that this has happened because I was just starting to get my gut bacteria in line through good diet and probiotics. No snake or essential oils necessary! (Puts crystals away and looks at that poster with sarcasm!).

</comment><comment><USER13> I wonder if all the microbiome sequence data that UBiome collected will wind up in some FBI database now.

<comment><USER14> uBiome was mostly selling pseudoscience to engineers (who should know better) and the essential oils crowd. Once you have the data you have to do something meaningful with it. For most uBiome users that means heading to YouTube to figure out which guru they are going to follow.

</comment></comment><comment><USER15> Are they seriously carrying shields and body armor?

<comment><USER16> The FBI and most US law enforcement is notorious for ridiculous overkill and roughing up subjects of search warrants.

<comment><USER17> The FBI engaged in similar militarized theater for the recent arrest of Roger Stone. Dozens of armored SWAT officers, automatic rifles, and a CNN war correspondent van parked outside - all to arrest a solitary man sleeping his pajamas.

For historical context, this trend is due to the pendulum swinging back too far in reaction to the Miami FBI shootout [1], where officers were woefully outgunned.

[1] [https://en.wikipedia.org/wiki/1986_FBI_Miami_shootout](https://en.wikipedia.org/wiki/1986_FBI_Miami_shootout)

<comment><USER18> Roger Stone had posted multiple videos of him at a shooting range so its reasonable for them to assume he was in the possession of firearms.

<comment><USER19> so anyone who has been at a shooting range needs to be raided by a SWAT team when arrested for white-collar crimes?

<comment><USER9> The thing about a heavily armed society is that it necessitates law enforcement preparing for armed resistance in routine tasks as a default rather than exceptional case, if nothing else to reduce the probability of such resistance by reducing the expectation of it being successful.

You can't reasonably both have a pervasively armed populace and law enforcement unprepared to deal with armed resistance from suspects even when the crime of which they are suspected is not itself violent.

<comment><USER16> SWAT raids are a terrible way to get the drop on anyone who is actually intending on armed resistance. They are on the other hand an excellent way to make sure that "accidents" happen, as they routinely do. They are an obvious terror tactic.</comment></comment></comment></comment></comment></comment></comment>


Giphy Celeb Detector - <AUTHOR> https://engineering.giphy.com/giphys-ai-can-identify-lil-yachty-can-yours/ <comment><AUTHOR> Didn't work that well for me. Tried on 2 GIFs:

1\. [https://media.giphy.com/media/I533LqvPn1R9S/giphy.gif](https://media.giphy.com/media/I533LqvPn1R9S/giphy.gif) Results: [https://imgur.com/TkRkeRx](https://imgur.com/TkRkeRx) (100% incorrect results)

2\. [https://media.giphy.com/media/10mysBwISOvX6U/giphy.gif](https://media.giphy.com/media/10mysBwISOvX6U/giphy.gif) Results: [https://imgur.com/HspROf6](https://imgur.com/HspROf6) (100% incorrect results)

A naive implementation of reverse-looking up GIPHY database for GIFs would have worked better (in this case, not their usecase).</comment>


Show HN: 5+ Billion Passwords in Order of Most Popular - <AUTHOR> https://github.com/berzerk0/Probable-Wordlists <comment><USER2> This list is immediately useful for validating user-created new passwords. Just stop with the bizarre rules about having uppercase, lowercase, symbols, numbers, length, etc. Instead require a string not in the top 10K (or 100K, or 1M) most popular.

<comment><USER3> "Your password must be between 12 to 46 characters long and must include at least one number, upper case character, special character, kanji character, rune and quadratic equation"

<comment><USER4> I used to just tack on Aa$0 to the end of all my (already good) passwords to satisfy these idiotic rules. Until one website decided that '0' was not a number, and then I had to change everything to Aa$1.

<comment><USER5> I tend to revolt against this kind of bullshit by creating terrifyingly primitive, pathetic passwords, and flagrantly sharing what my password is, in front of relevant admins, while complaining loudly about the concept of " _character diversity,_ " as a way of fomenting popular rebellion.

When I'm met with periodic expiration, I employ constant direct password assistance as a means of unpopular two-factor authentication, to piss people off, and stir the pot.

I then fail my password reset as many times as necessary to recover my preferred password for that system, to enforce to freedom of password selection.

I control my password. Me.

Not you. Me.

<comment><USER6> Im glad Im not the only one that does this.

</comment></comment></comment></comment></comment><comment><USER7> you should package this up as a bloom filter and simple js routine web developers could use to do client-side checks to validate passwords.

edit: on 2nd thought looks like a bloom filter for 5B entries at p=0.01 would be ~5GB, so not exactly convenient

<comment><AUTHOR> The largest list is 20GB, but it's not the only one.

Popularity was based on how many they appeared in files that had all duplicates removed (in reference to themselves)

The smallest file had passwords that appeared 75+ times, and the largest file had passwords that appeared 2+ times.

The top 195 Thousand (which appeared 25+ times in analysis) clocks in at 803kb as a text file with nothing but the passwords themselves

</comment></comment><comment><USER8> Somehow I wonder if life is significantly different for people named Daniel.

<comment><USER9> Or Alexander or Victoria. It's weird that these first names appear in the Top196-probable.txt file.

</comment></comment><comment><AUTHOR> Having issues with the Seedbox, torrents are down temporarily.

The main page contains links to Mega.NZ alternative downloads. Will be fixed shortly, apologies for the inconvenience.

<comment><USER10> Is there really any reason to multiply the downloads by 4 just to have different compression methods? It seems to me like that just needlessly dilutes the seed swarm and wastes space, since pretty much any modern archive reader can unpack all the provided formats. Sure, specialized command-line utils can't, but if you're using one of those then you probably know which one to use for a given format.

<comment><AUTHOR> You know, you're not the first person to bring that up - and I am having seed problems.

Perhaps next rev I'll only include the .7z (smallest) and greppable formats.

</comment></comment></comment><comment><USER11> I don't understand: Why do you assume that password checkers keep their lists in alphabetically sorted form rather than just to load the whole thing into a db table with an index on it?

<comment><AUTHOR> As I was looking around for the files to make this project, on SecLists, Weakpass, and Hashes.org, most of the files were in alphabetical order. This was especially true for the larger files.

<comment><USER11> Yes, but it doesn't matter what order they are in if you put them in a DB. I don't know anybody that would do a sequential scan over a file in production.

All you've done is shuffled around the most frequent cases to the beginning, which is great if that's what you are looking for, but now something else occupies the last slot and that case is just as bad as before.

The real solution is to see these files as input to an indexed table so that you can get to the entry you want in log(n) time.

<comment><USER12> _I don 't know anybody that would do a sequential scan over a file in production_

Well, that's pretty much how one would try to crack a password using a wordlist.

EDIT: If the goal is to crack a bunch of properly hashed (PBKDF2, scrypt, etc.) and salted passwords then a lookup table is not very practical.

<comment><USER11> If you're trying to crack 'a password' you're doing it wrong. Alternatively, you're doing something illegal.

That's not why these lists are made public (though I can appreciate the fact that they are 'dual use').

</comment></comment></comment></comment></comment><comment><USER13> Not to sound negative, but is this something we should _really_ be exposing? It feels like the only ones who gain from this are hackers and password crackers, no?

<comment><USER14> Audits. Education. Change.

It's not like bad guys don't already have wordlists, or that they are new in any way. Having good ones available, in the open, for everyone to use, provides a net benefit in the long run imo.

<comment><AUTHOR> This is similar to my train of thought.

Year after year people still use the same passwords that were the most popular for the year before. I think we should make it as obvious as possible that it is worth your time to make secure passwords.

I think people know that they shouldn't use "password" as a password, but it just needs to be made very clear through safe means (such as publishing wordlists that don't have any associated user data) that it leaves you very vulnerable. Many people simply haven't stopped to wonder how secure their password is.

My goal with this project is to flip the status of the passwords on the list - and make them the LEAST likely to be used.

</comment></comment></comment><comment><USER15> Wow this is so useful for um security

</comment><comment><USER16> If you pick the 5 billionth one you've probably picked a gooden.

<comment><USER17> We've banned this account for abusing the site, including posting many unsubstantive comments after we asked you to stop. Please don't create accounts to break the site rules with.

</comment></comment><comment><USER18> _•These lists are for LAWFUL, ETHICAL AND EDUCATIONAL PURPOSES ONLY._

Yeah, like that is going to stop people from doing nefarious things with this info. If you feel the need to post this screechy, all caps disclaimer, maybe rethink your project entirely?

Geez.</comment>


Full Hacker News - <AUTHOR> http://www.fullhn.com/

<comment><USER2> Really awesome, fix the broken images for articles, I am in. Thanks for creating this.</comment>


Github down - <AUTHOR>

It is not just down. It is completely down. There does not seem to be a http server running even. I just get a connection refused.<p>Anyone know what happened? Migration issues? <comment><USER2> In the event of complete down-itude, they have a page with advice (which I was able to reach while they were down):

<http://github.s3.amazonaws.com/misc/down.html>

I can't say how current the advice is or how well it works, since shortly after I found it, they came back up.

<comment><USER3> Hey that's kindof a cool failsafe. Maybe a good idea to have some redundancy in our web services. As in some functionality, vs. no functionality.

</comment></comment><comment><USER4> Connectivity problems between their front- and backend servers.

<http://twitter.com/github/status/4835070431>

<http://twitter.com/github/status/4835640473></comment>