Ad Blocking Extensions Tested for Performance - <AUTHOR> https://www.raymond.cc/blog/10-ad-blocking-extensions-tested-for-best-performance/view-all/ <comment><USER3> The methodology is questionable on several fronts:

\- you can't use the load event to compare advert-heavy vs. advert-free load times. The load event does _not_ fire when the page is usable by the viewer, it fires when every last request is done, in particular even requests fired from various advert iframes. However, those requests may not impact page usability. The load event may even never fire - any offscreen 1px spacer gif that times out will cause that! I don't think there is a built-in event for "done-enough", but the load event is certainly misleading.

\- opening the developer tools causes various side-effects; those can cause the page to load more slowly than usual. You shouldn't benchmark with developer tools open (unless you're explicitly targeting usage with devtools open...)

\- comparing load times across browsers as reported by the browsers themselves _may_ be valid, but it's not obvious. You definitely want to check that carefully.

\- Measuring "peak" CPU usage is almost meaningless without considering how long the CPU is used.

\- Measuring the chrome extension process CPU usage and memory usage isn't very helpful, because running this kind of extension causes CPU usage and memory usage in every content tab. Both of these statistics for chrome in this use-case are meaningless. You'd need to measure the memory usage and CPU-time of the entire chrome process tree to get meaningful results. Even in Firefox without e10s it's not valid to measure just the main process CPU and memory usage because plugins are in separate processes (and things like flash or h264 decoding can definitely use CPU and memory).

The only thing this page really makes a decent case for is that Chrome loads pages faster than Firefox - but even there, it's not clear we're dealing with an apples-to-apples comparison.

<comment><USER4> > you can't use the load event to compare advert-heavy vs. advert-free load > times

I don't see why not. It's exactly what they want to show, how much quicker pages can load if you don't have to load the ads. And I personally am not satisfied with a page that has loaded to the extend of usability. Either a site loads fast or it doesn't.

> opening the developer tools causes various side-effects

As long as it's done for all the tests it should not matter. Whatever "side- effects" are introduced will be applied for all tests. Of course this means that the values themselves can't be used to compare the result against other tests, but the goal here was anyways to compare between the different ad blocks and not to generate values for general statistics.

<comment><USER5> Some pages will be becoming usable before the DOMContentLoaded event is fired. Some will become usable shortly after DOMContentLoaded is fired. Some will become usable when load is fired. Some will become usable several seconds after that. Some will never become usable. (Cynical aside: the ones with excessive ads might fit in that classification.) There is no event that measures _readiness_ of a page, nor can there be; there can only be a collection of reasonable heuristics.

<comment><USER6> This is where a human with a stopwatch may come in handy. The clock will start when the page request is sent. Stop the clock when, in your opinion, you think the page is ready to read. Alternately, if you feel the page is loading too slowly, and have reached the limit of your patience, press this button instead.

The event does not have to come from within the vanilla browser, without human intervention. It could be from a plugin you create to collect human feedback on load times.

Of course, the human element will significantly decrease the number of trials you can do within a fixed time period.

<comment><USER7> > Stop the clock when, in your opinion, you think the page is ready to read.

Even that is not enough. A number of sites ( _especially_ ad-heavy sites on mobile devices) look ready to read, but as soon as you try to scroll, are jerky and semi-unresponsive. It's particularly a problem on ad-laden sites of small news publications. Blocking ads sometimes mitigates this, though not always, as some sites insist on using Javascript to implement "fancy" scrolling which inevitably breaks.

(Semi-related sidenote: please, for the love of God, _don 't_ use Javascript to hijack browser scrolling. It's almost certain you will not get it perfectly right and some users will suffer. It's almost never worth it. This is the #1 reason I end up having to switch to Reader Mode in Firefox).

<comment><USER6> The maybe you stop the clock when the user navigates away from the page. If it takes 30s to read the content without ad-loading and 40s to read it with ad- loading, then by subjective user time, the page is 10s slower. That may be because the ads take 10s to load, or because the distraction causes the user to read at 75% normal speed, but it's still a measurable result from a single variable change.

<comment><USER8> You're going to get a whole lot of noise in your signal from readers who get sick of waiting for/looking at the ads and leave early.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> They didn't seem to know about e10s in Firefox Nightly which would have solved their "no separate process" thing.

Measuring peak CPU is very, very stupid. The best Ad Blocker should have near 100% CPU but for an extremely short amount of time. Using 100% CPU is making the most efficient use of the hardware, not being inefficient. Now, I think due to the way they're measuring, it's actually closer to being average CPU because task manager samples over a period.

Also: "Firefox didn't like this page at all and we couldn't get consistent readings to run tests, on every refresh the browser would simply not respond or crash."

Huh? I think somethings wrong there. Firefox obviously renders tmz.com. A bug in the thing they used to monitor page load times? Did they check the performance impact of it?

<comment><USER10> > Using 100% CPU is making the most efficient use of the hardware, not being > inefficient

Not necessarily with modern processors. A power-managedCPU at 50% clock-speed actually uses less than half of the power as at 100%.

The relationship of speed to power is not linear, so for a laptop user the most efficient use of hardware might actually be running one job for twice as long at half-speed.

<comment><USER4> Also if it's really using 100% of the CPU (not just one core), then it will affect the overall PC experience since it uses up all available resources and doesn't leave anything for other processes.

I personally dislike it when an application makes my "PC" lag every now and then. CPU time and RAM are shared resources, thus applications should make sure to not act selfish unless asked for (e.g. encoding a video, etc).

<comment><USER11> It's not up to a random application to make sure you have other resources available, that's your OS's job.

</comment></comment></comment></comment><comment><USER12> So it looks like about 1/4 of the time spent loading a page these days is spent on the page and the remaining 3/4 are waiting for ad networks. There's got to be an opportunity there.

I previously worked at a digital agency and one of my final projects was building the website for a large TV channel. At one point I had to have a meeting with the ad network to go through the integration.

The technical people involved didn't know anything about how their products worked or the implications of integrating them. No idea if they were blocking or async.

At one point I was trying to understand what, if any, changes I might need to make to handle their ads that did full background take overs. "No no no, it's an _expanding mpu_ " \- sure, fine, what does that even mean!?

(Don't get me started on having to swap using js for switching images in the gallery for using individual pages because the onmiture page tracking numbers were the metric that everything was measured by for selling ad space)

/rant

You'd think with all the smart people that have put their minds to ads on the internet, there would be lightning fast, targeted ads that actually worked by now.

<comment><USER13> I've worked on lots of ad networks and am now building my own. The sad fact is that most ad companies are not built by top engineers. It's an industry that has lots of shady international players with lots of fraud and politics. Even when there is good code, it's all usually focused on the backend with no thought about the end user experience. Add to that both buyers and salespeople who are clueless about technical details and this is what happens.

<comment><USER14> > The sad fact is that most ad companies are not built by top engineers.

This is an interesting thought. Working in a environment close to advertisement (travel recommendation engine), I have a few possible explanations on that.

\- First, working in with a product that have no (or close to) added value for end users can be very bothering, and top engineers care about doing useful things.

\- Second, to design a fast, simple and honest ad network you must fight your own business/marketing team (because they always want more data, higher _click through rate_ , etc). Since they have most of the power here, they end up hiring more diligent engineers who won't argue about privacy, curated content and other stuff.

\- Third, top engineers are working in ad companies, and secretly conspire to make the whole system collapse from the inside.

<comment><USER13> Ad companies are not typically considered "sexy". This is just an image problem because few industries drop you into such massive scale and engineering challenges from day 1. There is a lot of cool stuff to build.

It's not necessary to get computer science geniuses for this, and frankly I need people who get stuff done rather than trying to save the world through code. But the talent has been subpar because most of the companies outside of the top few (google, facebook, twitter) are smaller and not tech driven. They just pick up whoever can code up some simple server and get an ad on the page. Add in all the shady companies and it gets worse. Optimizing for the user just hasn't been a requirement.

We have a honest and fast ad network, probably the best you'll find. Quick, simple, static ads that are non-intrusive. And we actually perform quite well. It's not somehow at odds with the model, it's just harder... the question is, who's willing to work harder at this?

</comment></comment></comment></comment><comment><USER14> Sure it's a nice test, and gives a keen idea of how much performance you can grab by using uBlock Origin.

However, what about pages _without_ ads? Am I going to experience a significant performance hit when running an adblocker on genuine websites? This is important. Because, assuming the advertising industry is going to self-destruct tomorrow morning, am I going to experience a slower web if I let my add-on activated, and thus letting an opportunity for the ads to reappear later?

<comment><USER15> uBlock Origin seems pretty comparable to most of the other adblockers with the exception of the pretty horrible AdBlock Plus.

</comment></comment><comment><USER16> Would be interesting to have tested using a hosts file (like from [http://someonewhocares.org/hosts/](http://someonewhocares.org/hosts/) ) instead of an ad blocker. Seems to work very well for me, and it doesn't cause the browser to use loads of RAM.

<comment><USER17> I'm not sure: isn't it faster to just refuse the request than directing it to localhost and waiting for a time out?

<comment><USER18> As long as you don't have a firewall blocking traffic to/from localhost, the TCP connection will be reset almost immediately (assuming you don't have an httpd running on port 80) so you won't have to wait for a timeout.

</comment></comment></comment><comment><USER19> In reality, people want a page that becomes _usable_ as quickly as possible. Specifically, people use hovers and scrolling to test that a given page is 'ready.'

That is very different from a page that fires DOMContentLoaded as quickly as possible. That metric is pretty much irrelevant to real people.

Which is part of why ads aren't quite as nasty as a lot of tech industry people make out.

<comment><USER20> When I start scrolling and the page seems ready to use, and then ads start popping in late and breaking the document flow in the first few seconds - <USER2> better believe I 'm blocking them on your site_.

<comment><USER19> I quite agree. There's no excuse for that. Although most of the time it isn't the fault of the website or publisher, but rather of an adtech partner.

<comment><USER13> It's both. It's a shared execution and the publishers haven't really put the work in either.

Check how well [http://www.theguardian.com/](http://www.theguardian.com/) loads and you can see that it's perfect possibly to have plenty of ads while still having a great reading experience.

<comment><USER19> Do you know what their architecture is? It is fast.

<comment><USER13> Some details here: [http://www.theguardian.com/technology/developer- blog/2014/de...](http://www.theguardian.com/technology/developer- blog/2014/dec/29/what-is-the-guardians-tech-stack)

It's not really the backend that matters, these days its the front-end setup that makes the biggest difference. Less javascript, less huge unnecessary images, less "interactive" crap and streamlining the loading of content first then ads async can make a site load pretty quick. They also style the ad slot dimensions (since all banners are standard sizes) which keeps the content from moving around when an ad does load.

Its just simple effective optimizations, something that should be done but for some combination of bad talent, bad management and poor priorities, just isnt.

</comment></comment></comment></comment></comment></comment><comment><USER21> I heard about µblock origin on HN and blindly trusted the positive comments. Glad to see my trust was well deserved, thank you all.

</comment><comment><USER22> I've found uBlock Origin increases memory usage heavily on [https://vimcolorschemetest.googlecode.com/svn/html/index-c.h...](https://vimcolorschemetest.googlecode.com/svn/html/index-c.html)

<comment><USER23> To be fair that page has 430 iframes. I'd say that's fairly atypical.

<comment><USER24> Atypical? Amazon does (did?) heavily use some <iframe>/<object> trickery to preload resources.

</comment></comment></comment><comment><USER25> I would have really liked to have seen an ad-reduced hosts file compared to these in-browser blockers.

[http://winhelp2002.mvps.org/hosts.txt](http://winhelp2002.mvps.org/hosts.txt)

</comment><comment><USER26> At the end of last year, I spent a while working with an adtech firm, helping them figure out their product strategy. They all use Ghostery and I started using it too. Subjectively, my web-browsing experience improved massively, particularly page load times. It's nice to see that experience borne out by objective testing.

I often think that companies like Ghostery are missing a trick by not advertising themselves as a method of speeding up your internet connection because that's really what it does.

<comment><USER27> "Ghostery, NoScript.. add-ons frequently phone home"

[https://mailman.stanford.edu/pipermail/liberationtech/2015-A...](https://mailman.stanford.edu/pipermail/liberationtech/2015-April/015236.html)

<comment><USER28> Well yeah, because Ghostery is a product of Evidon, which helps adtech businesses to get visitors to opt-in to tracking. It's a clever strategy, because they give valuable feedback about which ads and trackers are blocked for which reasons. If you can't beat them, outcompete them!

(I worked as CTO of an adserving platform and had to reject sales calls from Evidon on an almost monthly basis)

</comment></comment></comment><comment><USER29> Lowest overhead: Disable JS (add it to a hotkey) and use DNS level filtering: [https://github.com/jakeogh/dnsmasq- blacklist](https://github.com/jakeogh/dnsmasq-blacklist)

With surf+tabbed, each tab is it's own process, so you can enable and disable JS and or plugins per page.

</comment><comment><USER30> Next we need to find the best blocker cocktail. I use uBlock Origin, Ghostery and Flashcontrol on Chrome. Bit over the top possibly - can take a bit of trial an error to see what's blocking the thing if something you want doesn't load.

<comment><USER31> Have you tried uMatrix? If you have, I'm curious how you think it compares to your current mix.

<comment><USER24> seconded, for people who want to sacrifice some convenience for control µmatrix may be a better choice.

</comment></comment></comment><comment><USER32> Why is uBlock somewhat frowned upon here on HN? From what I've seen published it's still a good solution - works well for me. I thought the original developer gave up the project and was being rather difficult about the whole thing.

<comment><USER9> He came back and has been improving it actively, and wasn't too happy about technical decisions in the fork.

</comment></comment><comment><USER33> Forgetting the accuracy of the OP, I cannot be the only one who doesn't care in the least that adblockers slow load times. Between my ISP's shoddy service, the popups and distracting animations adblockers stop, the net effect is a faster overall experience. A few seconds on page load is a fair price to pay for the increased privacy and security.

</comment><comment><USER34> Are there any ad-blockers out there that also block Google ads (inline in websites, and also on top of search results)?

<comment><USER23> uBlock Origin does, though I'm pretty sure all do.

</comment></comment><comment><USER35> A much overlooked alternative to installing sketchy addons is to simply disable Javascript by default.

<comment><USER36> This also breaks 95% of the web

<comment><USER35> You're spreading FUD. Disabling Javascript _by default_ is one of the easiest ways to reduce browser attack surface, improve page load times and to remove the most annoying ads. The vast majority of pages work (to some degree) without scripts.

If you really need scripting on a page, _it 's very uncomplicated to add an exception in Chrome_: just click the "Javascript blocked" icon in the address bar, and choose "Always allow Javascript on example.com".

<comment><USER7> > improve page load times and to remove the most annoying ads. The vast > majority of pages work (to some degree) without scripts.

In my experience using NoScript for the past year on all my desktops and laptops, this is at best only partly true.

Load times do definitely decrease, but this is dwarfed by the effect of having to reload the page for most new sites after re-enabling Javascript (since so many sites are unusable without Javascript[0]). In the end, it's a much slower experience browsing the web, not faster, and even with aggressive whitelisting, it's still rather frustrating.

I actually have separate Firefox profiles that I use for various websites that tend to be particularly problematic with extensions (like online banking), so I'm not even including those in this summary.

I wish using NoScript provided a faster browsing experience overall while providing better security, but the simple fact remains that right now, it does not. It provides better security at the expense of slowing down your browsing experience.

[0] So many static blogs now literally fail to display static text without Javascript!

<comment><USER37> Sadly, that's true. I'm wondering how many feel that this is not a good direction where things are going -or if it's just me being "old".

Why would someone want to use javascript on the client-side for things where it's not necessary?

Year after year, the browser and the web are turning into "something else", and while the possibilities are exciting, it seems the use cases where javascript is used for things that actually make a positive difference, are vastly outnumbered by the cases where it is used to re-implement in a more flashy, complex and resource-intensive way things that have been usable, performant and functional, 10 years ago.

IMO displaying content should be done just by HTML, using javascript if it's not to provide _useful_ functionality, is wrong.

Whether it is browser HTML/CSS incompatibilities, flash atrocities or complex script bloat, it seems the popular web always manages to fix one problem and create another.

</comment></comment></comment></comment></comment><comment><USER38> blogspam, stop upvoting this please.

<comment><USER39> Is that fair? The article looks a bit spammy, but seems to be an actual test not copied from elsewhere.

<comment><USER40> The article's solid and substantial, it's the first I've seen to quantify the effect of ABP's acceptable ads policy. Also they have a view all button that works.

</comment></comment></comment><comment> the-dude One of my pet projects is 'wijVrij', a cheap TP-Link router with OpenWRT and a bootup-script which installs the winhelp2002 hosts file.

Works wonders on all your devices. [https://wijvrij.nl](https://wijvrij.nl) ( dutch )</comment>


Show HN: Use CouchDB to auth users for your Clojure/Script app - <AUTHOR> https://github.com/deckeraa/couchdb-auth-for-ring/ <comment><AUTHOR> CouchDB is a convenient place to store Clojure/Script objects across sessions. To make this even more convenient, couchdb-auth-for-ring lets you use CouchDB's _users database to auth users for your Clojure/Script web app. It's middleware for Ring.</comment>


Monzo: How our security team handle secrets - <AUTHOR> https://monzo.com/blog/2019/10/11/how-our-security-team-handle-secrets <comment><USER3> (Disclosure: I work on Kubernetes/EKS Security at AWS)

I'm curious why they didn't look into using Kubernetes ProjectedVolumeTokens for authenticating to Vault? The tokens Kubernetes issues are not stored in etcd, and they contain pod-specific metadata so they are invalidated as soon as the pod dies (when using TokenReview). Alternatively, they can be used to directly authenticate with Vault since they're OIDC-valid tokens [1].

The semantics around secrets in Kubernetes aren't nearly as robust as Vault, so I was surprised to not see this more clearly called out (ex: list secrets == get all keys and values). Even if you use KMS/AES encryption (which they reference) that doesn't help with access control.

[1] [https://www.vaultproject.io/docs/auth/jwt.html](https://www.vaultproject.io/docs/auth/jwt.html)

<comment><USER4> Interestingly, Googling `ProjectedVolumeTokens` yields this very post. I think that says something about its maturity.

<comment><USER3> That would by my error, its actually `BoundServiceAccountTokenVolume` and `TokenRequestProjection`

</comment></comment></comment><comment><USER5> The intersection of microservices + Vault is something I've long had to deal with - as far back as Vault 0.5.0, so I'm a bit "surprised" that isn't something that is turn-key with Vault+k8s today.

I was working with Mesos (before k8s had taken the world by storm), and had a similar issue - how do services get vault tokens without having a workflow that may include storing "secrets" in configuration. What I ended up writing was a tool[1] that a service could query, with it's Mesos Task ID, to get a token. The tool would then read it's own configuration, as well as the current mesos state to determine if the request was valid or not.

Unlike the k8s solution, as I understand it, you don't need to treat the 'service account token' as a secret (reducing the attack surface of when someone steals that token _and_ also has access to Vault). This is accomplished in two ways:

1\. You can determine if a request is valid by looking at how long the service is running. If someone steals a Task ID, but the service has already been running for 2 minutes, then the Task ID is useless.

2\. The Vault token is only issued once per service launch. This means if an attack steals the Task ID, but the token has already been given out, the Task ID is useless. If the attacker beats the the service in asking for that key, then the service should raise the alarm bells about it's key potentially being stolen. If you are even more paranoid, you could even decide to invalidate all active keys and reduce the amount of time the attacker has a valid Vault Token.

This was largely designed years ago with some cues I took from one of Vault's lead engineers so I expected that the k8s integration would work similarly. I'm not too familiar with k8s however so there might be other constraints I'm overlooking in why Vault's k8s is integrated the way it is.

[1] [https://github.com/nemosupremo/vault- gatekeeper](https://github.com/nemosupremo/vault-gatekeeper)

<comment><AUTHOR> This sounds interesting. I think with bound service account tokens ([https://github.com/kubernetes/community/pull/1460](https://github.com/kubernetes/community/pull/1460)) things may improve; pods will be able to prove their identity without it being equivalent to giving away your k8s api access.

I don't think there is anything with the same guarantees as the Task ID, sadly.

</comment></comment><comment><USER6> > We run about 1,100 microservices written in Go

Asking as a university student: is this a common number of microservices to have running in production? It looks like monzo has about 1,351 total employees [0]. If all of them were software engineers, this would be a little less than one microservice per engineer. How do you handle code reuse and reliability among thousands of microservices? It seems like the number of possible failure states would be unthinkable.

[0] [https://en.wikipedia.org/wiki/Monzo_(bank)](https://en.wikipedia.org/wiki/Monzo_\(bank\))

<comment><AUTHOR> I don't think its particularly common for a company of our size. They are mostly very small and handle very specific things. It's one approach, with some benefits, although it makes projects that affect all services quite tricky!

<comment><USER6> Does 1,100 microservices mean 1,100 distinct programs or 1,100 running instances of dozens or hundreds of distinct programs?

<comment><USER7> I assume they're mostly RPC, and one procedure is one 'microservice'.

There seems to be a bit of difference in what people mean by 'microservices' \- some orgs will have a few REST collections per 'service' so you might end up with 'users', 'products', 'transactions' as your three services, and it being totally unimaginable that you'd ever break 1000.

I'd argue that's still a Service-Oriented Architecture (SOA), but I'm sure it's not anywhere near as 'micro' as what Monzo counts over a thousand of.

<comment><AUTHOR> Microservices generally have 0-10 RPC handlers (we use [https://github.com/monzo/typhon](https://github.com/monzo/typhon)). We have a lot of different object types at Monzo, it goes way beyond 'transaction' or 'user' and each object type is going to be at least one service, but can be several. Services that mostly just consume off queues are often separated from services that handle requests so they can scale separately.

<comment><USER7> Oh I wasn't suggesting user/product/transaction were what Monzo would use! (Should have made that clearer since 'transaction' is right domain...) Just the first things that came to my mind for very basic sort of service splitting that I'm familiar with.

Thanks for replying, that does clear things up. My experience didn't really embrace RPC, so used gRPC for one small bit (< all of the service that contained it as I recall) but most was JSON HTTP APIs - the service boundaries pretty much just being team boundaries, though some (incl. mine) had a few services to the team.

</comment></comment></comment></comment></comment></comment><comment><USER8> The bit where a secret gets pasted into the Very Secure system is a clear problem. Because before it was pasted, and while it was being pasted, it's not in that secure system it's on some dev's laptop.

Most of your secrets will be/ should be just random bits maybe in some particular format that was convenient for a particular application e.g. a 4-digit PIN or a sixteen character Hexadecimal string, or 10 characters of A-Za-z0-9

So for these cases there's no reason that secret is ever on a developer's laptop. Best case the developer made a truly random secret, maybe they (like me) keep a set of hexadecimal dice on their desk for random choices. Just as likely it's tainted, the developer ran a randomizer until out popped a string they liked - or even they found one on a web site, or used the same one as in their test environment.

Either way, since what you wanted was random bits it makes sense in most cases (not all cases, obviously a secret key you were sent by somebody else, for example an API key for their system will have to be copied somehow) to have a feature that just spits the right format of random bits into the secure system without any human seeing them at all.

Even better, in cases where it's an option, is not to rely on stored secrets at all. I think Monzo's post is not worrying about this difference, but it can be critical in terms of decisions about debugging to prefer to have entirely ephemeral secrets. When a pod goes away, the ephemeral secrets that pod had vanish with it, and so you aren't storing them anywhere anyway. If they aren't stored, they can't get stolen by anybody and you've got one less thing to go wrong.

<comment><USER9> (Disclaimer: founder of HashiCorp, creator of Vault)

You hit on a good point. Vault has features to eliminate this security risk, if I'm understanding correctly.

The first feature is dynamic secrets: this generates an ephemeral, leased set of credentials that are unique per client. For a Kubernetes pod, it would get a unique set of DB credentials, for example. These are tied to the service account (used for auth). When the auth expires, so do the credentials (they're dropped from the DB, and if the DB supports it we also drop connections).

The second feature is root credential rotation. To use the above feature, a user had to at some point "paste" the superuser credentials into Vault. As you pointed out, there's a risk here. So what Vault can do is _immediately_ rotate that credential so after configuring Vault, it is no longer valid and only Vault knows the real credential. We support this for most database backend, for example.

If you combine these two elements, you get fully ephemeral secrets that are unknown by anyone except the necessary user. There's a lot more we can talk about, there's a lot more features we have around this, but this is just the high level point!

<comment><USER10> As one of your commercial customers, I would give you the feedback that there are a lot of features that get closed (on the github vault issue tracker) with terse and not very user friendly reasons. This is off putting to a lot of people.

We've recently been escalating via sales for github issues we see use for and hopefully that gets back to product management. Not everyone is able to do that however.

<comment><USER9> Thank you, I heard this feedback this week as well. We're hiring for a couple roles right now in the Vault team that will be dedicated to community management and process (that will expand more broadly). We hope this helps with this.

The feedback from customers definitely gets back to product management, so for your case, that works.

</comment></comment></comment></comment><comment><USER11> I so, so, so, badly want to use Vault everywhere.

The one thing keeping me from adopting isn't that it won't be secure enough. I worry _constantly_ that I'll lock myself out of my data, my infrastructure, etc.

Have others had that worry? How'd you get over it and just start using Vault? (probably through incremental, low-risk adoption first?)

<comment><USER12> Some quick points as a very early adopter of Vault:

\- Absolutely take an incremental approach, there is a learning curve

\- Use a high availability backend. I started out with just consul, but have since went with consul for HA, and cockroachdb as the data store.

\- Practice your disaster recovery a many times before you go all in

\- use a combo of paper plus offline digital storage for the unseal keys and root tokens

\- be diligent about roles/policies

\- rely on tokens more than any other auth method(username/pass, certs)

\- don't be afraid to use many vault instances that can talk to a core instance for things like transit auto unseal

All that said, I absolutely think it's worth the investment if you have the infrastructure to back it(aka a cluster without a SPOF)

The PKI backend is phenomenal, and it makes a great sidecar for any app that needs auth, secrets, or general crypto stuff using the transit backend.

I really want to create a user friendly Password Manager/TOTP front end backed by vault. Someday…

<comment><USER13> Something like this? Chrome plugin: [https://chrome.google.com/webstore/detail/kvasir/kabfjaeebjd...](https://chrome.google.com/webstore/detail/kvasir/kabfjaeebjdpgbifhipejjdebeodlbip?hl=en)

Src: [https://gitlab.com/Dreae/kvasir](https://gitlab.com/Dreae/kvasir)

</comment></comment></comment><comment><USER14> Vault interests me but seems to come with a lot of complexity/requires an army of devops. There's a YC backed company called Envkey[0] that looks interesting + simple, but there is no option to self host yet

[0] [https://www.envkey.com/](https://www.envkey.com/)

<comment><USER15> I'll plug envwarden[0] which is a simple wrapper to manage your keys as environment variables in Bitwarden (which can be self-hosted I believe).

[0] [https://github.com/envwarden/envwarden](https://github.com/envwarden/envwarden)

<comment><USER16> Yes, Bitwarden can be self-hosted. It is also a really nice secret management tool for humans, although it does have a CLI and API which can be used for machines.

</comment></comment></comment><comment><USER17> One thing bugs me about Vault and I rarely see it beeing discussed: how come a design where Vault simultaniously stores secret AND is able to access other systems these secrets are for is deemed as good and secure?

Vault is a company-wide "root account", reachable from every part of internal network, storing all the company secret data and have reach to many internal and external systems like databases, where it have full rights to dynamically configure short lived credentials. Doesn't it put too much trust into a single system?

<comment><USER18> The real question might be if having it in one place makes monitoring and revocation easier than distributed trusted systems? Also, traditional passwords/secrets don't expire, and if they do expire, how would you maintain trust in a distributed fashion outside of... complicated multi-part keys, something less secure like DNS, and/or more permanent tokens like the private keys used by a CA system? Somebody somewhere has to maintain a private or secret key, or you need a human to intervene. And even if you store the key on hardware, any users of the key could be compromised.

I guess what I'm getting at is—there probably isn't a perfect answer, just tradeoffs. And if history has taught us anything, it's a case of "when" not "if" something is attacked/broken. If so, perhaps you should partition your data, including infrastructure, to not rely on just one Vault server for everything? Outside of that, or monitoring, the only other clear answer everyone leans on is "Store it in the cloud," under the assumption that the hardware and people processes at cloud companies will be more secure overall than anything you'd develop. Which is then the attack vector vault prevents: saving permanent access tokens. Off the cuff that's how I see it. I'm not actually in SRE or security so I'd welcome other opinions.

</comment></comment><comment><USER19> Ha I love how they explain what "cryptography" means, as if anyone reading this wouldn't know.

Also I love the level of openness. No other bank would do this. (If you're in the UK, Monzo is honestly amazing. I've cancelled all my other bank accounts.)

<comment><USER20> Slight tangent on that off comment at the end - why did you cancel your other accounts? If Monzo services fail or briefly go down (which is a somewhat medium risk since they've had a couple this year which lasted days) you are screwed.

<comment><USER21> Their outages have not affected everyone and often affect a small slice of the service.

Also, other banks have the same kinds of outages, often, but are not as transparent or communicative as Monzo.

I've been full Monzo for the last 18 months and have never had an issue.

<comment><USER22> > _Also, other banks have the same kinds of outages_

One of the unexpected advantages (if you can call it that) of OpenBanking integrations is that you get to see, in practice, just how often the high- street banks have problems. Or how long it takes for them to recover. Whatever has been said about politics and sausages sure applies...

I mean, good grief. Payment handling for a given bank may be down for a week and that's apparently not a cause for concern. Authorisation messages may be missing for days. Incoming queue can be offline for two days and customers just have to deal with it.

From the stats I've seen, Monzo is actually among the best performers with their reliability and recovery. (A recurring complaint on the receiving end is that their app makes it _really_ easy to generate payments outside of the OpenBanking flow. That has generated extra work for our payment and customer service teams.)

<comment><USER23> Oh I'd love to see these stats, do you have a link?

<comment><USER22> The team who handle the payments should have a post coming up soon.

</comment></comment></comment></comment></comment></comment><comment><USER24> >> We check that a secret exists in our staging environment (s101) when writing to prod, and warn if it doesn't.

My first thought ... so you have a script that can see prod and non prod at the same time?

I think I may be developing paranoia.

<comment><USER18> Theoretically, you could have a dump of staging data that your prod script could read and use. What's key here is recognizing that staging and prod don't have to talk to each other or be trusted, as such. Instead, what's trusted is the deploy script, and your system is only as weak or as strong as the code used to deploy the deploy jobs. There aren't many good answers here because you can fat-finger a security bug or data wipe, or have a bug that only presents itself in certain states. Or is created by human input. This quickly devolves into the "trusting trust" problem. Is my paranoia worse? ;-) I'd suggest monitoring all the things, and doing as much code review and fuzzing as you can afford... what normally improves your code and processes will probably help prevent security defects too.

<comment><USER24> I don't think dumping production _secrets_ to anywhere qualifies as a good idea

I am going to guess this is comparing the key part of key value stores - <USER2> regularly add a new configuration value to dev and wonder for hours why pre- prod is failing

somehow i am missing the details and in this kind of case details is all that matters

<comment><USER18> Comparing keys is interesting, it's a way of versioning the set of credentials you're using.

Here I meant dumping staging values and then comparing prod values as retrieved to staging ones. As in, only compare values in prod as they are used by a deploy script or system configuration tool. That said, if you have one server, one place where secrets are kept, then it's probably safe enough to send non-prod secrets to servers as a way of ensuring the secrets are invalid. Meaning, you don't actually need to know the prod secrets to test staging or known weak secrets against prod.

That said, if in practice your secrets are randomly generated as services deploy, you'll likely need to validate by observed behaviour rather than using hard-coded credentials. And if you're practicing blue/green deploys, then staging might be just as production as prod...

</comment></comment></comment></comment><comment><USER25> Slightly related, but what's a good practice for storing secrets that need to be recovered as plain text?

I'm thinking of a system where the user can register her/his API keys to other third-party systems.

<comment><USER5> If you have the operational manpower, Vault is a good solution as well. With Vault you can create an endpoint that will encrypt/decrypt data with a key that only Vault knows. You can have a unique endpoint for each user.

Then in your application when the user submits some data, you ask Vault to encrypt it before you store it in your main data store. When you need to read it, you get that value from your store and ask Vault to decrypt it.

If you have different services doing the reading/writing you can even setup your permissions so that one service can only decrypt and the other can only encrypt.

</comment></comment><comment><USER26> One exposure/exercise rotating secrets and it is curtains for you.

It gets better...stop building rube goldberg secret contraptions.

</comment><comment><USER27> Any opinions about Vault vs. AWS secret management solutions?

<comment><AUTHOR> Vault is a lot more fully featured and generally has more granular access control than equivalent AWS products, but is tricky to run in a way that is available and secure. Obviously outsourcing that has huge value

</comment></comment><comment><USER28> Nitpick: k8s has supported encryption at rest for secrets for a while.

[https://kubernetes.io/docs/tasks/administer- cluster/encrypt-...](https://kubernetes.io/docs/tasks/administer- cluster/encrypt-data/)

<comment><AUTHOR> This exact link is mentioned in the article! We use it!

<comment><USER28> Nice, posted that when I came across:

> Kubernetes stores the data in plaintext in etcd, a database where it stores > all configuration data

Just finished reading the article.

</comment></comment></comment><comment><USER29> Related: I have seen lot of people make mistake of hardcoding the secrets in the android app, please make sure you do not do that. I have a tool to check these embedded secrets: [https://android.fallible.co/](https://android.fallible.co/)</comment>


Windows projected file system - <AUTHOR> https://docs.microsoft.com/en-us/windows/desktop/projfs/projected-file-system <comment><USER2> How old is this?

Because I know that Window's "Zip" utility seemed to "project" into Windows Explorer. I don't know if that's a Windows Explorer plugin, or if its this file system.

<comment><USER3> Windows Explorer since very early (95-era) has supported complex "Shell Objects" and a COM-based Provider model for showing and interacting with "folder-like things" in Windows Explorer (and some other file-system APIs and tools that support it). In addition to "Zip" folders, it's what the old school Control Panel used for a long time, and Windows WebDAV support, and a lot of other things over the decades.

This new API seems a very different approach than the classic Shell Object provider model. It is directly rooted to the filesystem as opposed to the wider "object space" that Windows tracks, and it includes more things like concerns for converting virtual objects into physical NTFS files (and back).

</comment></comment><comment><USER4> It's very cute to see Microsoft's programmers discover 25 year old ideas from Plan 9 :)))

</comment><comment><USER5> Does this mean we can more easily create a custom filesystem in Windows, thanks previously possible?

</comment><comment><USER6> Seems like a Windows version FUSE ?</comment>


European court rules companies must tell employees of email checks - <AUTHOR> http://www.reuters.com/article/us-privacy-emails-echr/european-court-rules-companies-must-tell-employees-of-email-checks-idUSKCN1BG0YA <comment><USER2> It will as useful as EU's cookie law. Every employee on joining any company will be notified that corporate emails are monitored, similar to 'CCTV in operation' signs. It will be another checkbox on the contract.

<comment><USER3> The cookie law didn't kill cookies but it did inform people about them. It's responsibility for bring the discussion about tracking and cookies into popular discourse was greater than zero. This law and the new checkbox may have a similar affect.

<comment><USER2> Completely agree, despite the implementation of it being somewhat of a joke I believe it brought some needed awareness about the spread of user tracking or at least it existence to non-tech people.

<comment><USER4> But at a huge cost

</comment></comment></comment></comment><comment><USER5> The judgement highlights the balancing of "respect for his private life, on the one hand, and his employer's right to take measures in order to ensure the smooth running of the company, on the other".

While it is fair to expect that one "follows the rules", breach should not imply that your private life is something that the employer can do with what they wish, Especially, as the court decision notes, when the risks (e.g. damage and liability) are theoretical.

Occasionally my wife will send me an email to my work address, because she thinks I may see it quicker. Does this use of company resources mean that the contents of the email can be used by the company, or is there an expectation that it is still private?

It seems that the judgement is less about the monitoring and notification thereof, but whether or not you have an expectation for private things to remain private at work.

<comment><USER6> > Occasionally my wife will send me an email to my work address, because she > thinks I may see it quicker. Does this use of company resources mean that > the contents of the email can be used by the company, or is there an > expectation that it is still private?

I think that any expectation of privacy when using work resources to communicate is unreasonable — IOW, I believe it's illogical to expect that the company will not be aware of the fact that one has communicated.

The question of whether the company may use the contents of such messages for its own purposes is a bit trickier, and I can see arguments in either direction. The best course of action is _don 't use an employer's resources for private purposes_; then one cannot go astray.

<comment><USER7> > I think that any expectation of privacy when using work resources to > communicate is unreasonable

I assume this must be depend on the society, as I have a very strong expectation in the contrary. I already mentioned this in another thread; here the employer may never read e-mail contents, and reading metadata requires very specific legal steps to be taken.

<comment><USER8> So your work email doesn't have a spam filter?

<comment><USER9> It is like saying because Google mail goes through the text of the mail to guess whether a message is spam, it no longer matters that they don't use the contents of the message for targeted ads. This is clearly not true assuming Google keeps the spam training data and the ad serving days separate. Am I missing something?

<comment><USER8> They aren't equivalent. But they are both examples of processing data about a person. The GDPR doesn't say anything about advertising vs. security and network management purposes.

</comment></comment></comment></comment></comment></comment><comment><USER10> Everything about this is fine and dandy, but I don't understand why anyone would use their work email for personal use. An email address isn't a difficult to obtain resource. Anyone can make a hundred email addresses and use all of them for different things.

I guess the guy just never thought of it as an issue even if he was being monitored until it became an issue. I don't know.

<comment><USER11> I'm finding them harder and harder to obtain privately. I wanted a free throw away email address recently, so I tried a few of the big names - yahoo, hotmail, outlook, gmail. I couldn't see a way to sign up for any one of these without giving out an existing email address to them. If I wanted something unconnected to anything else, I had no way to do that. I'm not sure how someone getting on the internet for the first time would do it with any free service. I'm not opposed to paying for an email account, but it does create a barrier to entry.

<comment><USER12> Gmail asks you for your current email address, but it isn't actually required. I just tried it and made a new account in two clicks.

I assume the same applies to at least some of the others.

</comment></comment></comment><comment><USER13> Hmmm... I am pretty pro-privacy, but I am unsure that an employee's corporate email account should qualify, particularly given that anything said by it could be construed as a statement by the company itself. If the employee was not notified email monitoring could happen when they were hired, I suppose that's a problem, but I almost feel like that should even be common sense.

The article suggests more employees are using their work email for personal purposes, and that is really the problem, I think. Particularly because it places your personal data inside your employer's control, and your employer can terminate that account at any time. It would be much more preferred if you connect your personal account to your work devices as well, so that you still have a clearly denoted personal/corporate firewall in your communications, but can still access both from work.

I don't think employers which heavily monitor their employee's corporate email are... good employers, persay, but I'd question any suggestion they shouldn't have the right to.

<comment><USER14> In one of my previous jobs I was specificly encouraged (during introductory employee training) to use my work email for personal stuff, as all access to private email boxes was cut off by proxy servers. I wonder how many people fell for that.

<comment><USER15> If they are blocking access to private email, it probably because of a regulatory requirement (common in finance) that they absolute must monitor and record everything going in and out. If that the case, and they didn't make that clear, then that's of course bad.

</comment></comment></comment><comment><USER16> The ruling itself is at [1]. It's a long slog at approx 200 paragraphs of legalese. The conclusions were that Article 8 [2] had been violated, and that as a result some of the applicant's legal costs should be paid (EUR 1,365) but dismissed the applicant's claim for substantial damages (loss of earnings and social standing, fiancée terminating their relationship).

[1] [https://hudoc.echr.coe.int/eng#{%22languageisocode%22:[%22EN...](https://hudoc.echr.coe.int/eng#{%22languageisocode%22:\[%22ENG%22\],%22documentcollectionid2%22:\[%22JUDGMENTS%22\],%22itemid%22:\[%22001-177082%22\])}

[2] Any person shall be enabled: (a) to establish the existence of an automated personal data file, its main purposes, as well as the identity and habitual residence or principal place of business of the controller of the file; (b) to obtain at reasonable intervals and without excessive delay or expense confirmation of whether personal data relating to him are stored in the automated data file as well as communication to him of such data in an intelligible form; ... (d) to have a remedy if a request for confirmation or, as the case may be, communication, rectification or erasure as referred to in paragraphs b and c of this article is not complied with."

</comment><comment><USER17> I don't understand why this is even remotely controversial. I am as jealous a guard of my personal privacy as anyone, but I would expect my employer to be able to monitor my work-related email. The resolution is simple: conduct your personal correspondence through a separate email account that is not owned by your employer.

In fact, I keep strictly segregated work and personal email, and I run my own company!

<comment><USER7> And I expect my employer to leave all my communication unmonitored, be it e-mail, snail mail or private face-to-face discussions. Here, controversy created.

<comment><USER17> OK, please explain to me why you think that's reasonable?

<comment><USER7> That's what the law says in Finland. Also, I personally believe the confidentiality of communication should never depend on the used medium or the ownership of the equipment used for the communication.

<comment><USER17> It's not about the ownership of the equipment, it's about whether the communications are official actions performed by an employee on behalf of the company. If they are, then the employer obviously (IMHO) has a right to know about them. And the best way to keep things from getting complicated (again IMHO) is to segregate work email (which the employer can read) from personal email (which no one but the user can read) in separate accounts.

I really don't see how any reasonable person could disagree with that.

</comment></comment></comment></comment></comment><comment><USER18> This case is confused. The title and some of the editorial say "email" but the messages were sent via Yahoo Messenger, which isn't email.

> The company had presented him with printouts of his private messages to his > brother and fiancée on Yahoo Messenger as evidence of his breach of a > company ban on such personal use.

I know the distinction is a fine one, but it is still a difference.

They didn't need to produce a message, just having a non-business contact would have sufficed.

<comment><USER19> I don't think the court really cares if the messages were actual emails or if they were IMs. To them, the difference is immaterial.

</comment></comment><comment><USER20> You can grab a PDF with more details from the European Court of Human Rights here:

[https://hudoc.echr.coe.int/eng- press?i=003-5825428-7419362](https://hudoc.echr.coe.int/eng- press?i=003-5825428-7419362)

And judgement: [https://hudoc.echr.coe.int/eng#{"itemid":["001-177082"]](https://hudoc.echr.coe.int/eng#{"itemid":\["001-177082"\])}

</comment><comment><USER21> I get there is a different tradition in Europe than compared to what I'm used to and that's cool.

For me my company email is a company provided resource that they own and it belongs to them entirely. That includes their ability to monitor, read, or do most anything.

No matter what the legal protections it seems like mixing personal life with work resources / tools carries an infinite number of possible issues and complications.

</comment><comment><USER22> Will this apply to automated email screening, the kind the big companies very likely do to get alerted to IP theft and the like?

I would have to believe the likes of FB, Aapl, GOOG, MSFT, etc. likely have some process in place to detect exfiltration.

</comment><comment><USER18> While I have no desire to read an employees private correspondance, am even less desirous of paying money to store data I will get prosecuted for looking at!

<comment><USER23> Then don't.

<comment><USER18> People leave / die / are sick / go on holiday

"I can't look in that Inbox, there might be an email from his brother"

Puhleeze

<comment><USER19> I've never needed to look into someone else's email because they weren't there.

<comment><USER24> It happens all the time. Say that Adam handles technical support for some of your best customers, but he's out with pneumonia, or some other serious illness. He's not going to be responding to that urgent request for help with a new API feature, so you need to assign his inbox to Brian.

Yes, if Adam had notice he was going to be out there'd be a controlled handoff, but that didn't happen, so you have to just open up his email and move onwards.

<comment><USER25> I have never in my life seen this happen. Typically we use mailing distributions where I work to avoid this issue. We also can only respond to clients using specific email addresses, like the one tied to the mailbox that multiple people have access to.

</comment></comment></comment></comment></comment></comment><comment><USER26> Yay, another useless piece of information, like the "we use cookies" and "this entire building contains chemicals known to the state of California to cause cancer".

</comment><comment><USER27> Yeah.. I can't really find anything wrong or onerous with requiring a disclosure. Everyone _should_ know, but just in case they don't, a piece of paper explaining this during onboarding isn't a huge deal.

<comment><USER28> Where doesn't it end?

You yourself just said that everyone _should_ know. Rather than creating needless paperwork for everybody they'd be better served with a public relations campaign by the government to teach people that your work email is owned by your company which means they can read and monitor it.

<comment><USER27> Somewhere around where the slippery slope fallacy starts.

Snark aside, there's no real downside to mandating something in an employment contract that is probably already in most employment contracts anyways. This catches the stragglers.

<comment><USER29> Why don't governments produce standard draft employment contacts? They already do so for many things (e.g. buying/renting property).

Employer could ignore the standard or modify it and the employee could still amend it but it would help small companies get this sort of thing right and help with protecting both parties (which is the point of an employment contract).

<comment> vertex-four In practice, lawyers will produce/modify standard contracts for extremely cheap, and as a bonus, if something goes wrong, you now have a relationship with a lawyer. If you're _really_ cheap, in my country, you can literally go to the local book store and pick a model contract off the shelf.

I don't think this is a problem that needs solving by the Government, since these contracts are essentially a commodity and have wound up being priced as such.</comment></comment></comment></comment></comment>


Q2VKPT: Fully path traced Quake 2 - <AUTHOR> http://brechpunkt.de/q2vkpt <comment><USER2> That's super interesting, like the detail is low, obviously but the realness of the lighting almost feels better than a lot of triple A games I've played lately in a way that makes _this_ almost feel _more_ real.

<comment><USER3> recently saw someone used deep learning to create higher resolution textures for old games. might be something to add on top of this to get more details:

[https://www.overclock3d.net/news/software/classic_doom_has_r...](https://www.overclock3d.net/news/software/classic_doom_has_received_an_amazing_graphical_overhaul_through_the_power_of_ai/1)

[https://www.resetera.com/threads/ai-neural-networks-being- us...](https://www.resetera.com/threads/ai-neural-networks-being-used-to- generate-hq-textures-for-older-games-you-can-do-it-yourself.88272/)

<comment><USER4> What about.. AI-powered texture compression.

Train the game upscaling AI with high resolution textures, then ship the game with lower res textures. High quality rendering with smaller file sizes, though won't be as good as the original pictures.

Not very useful in reality, but would be a pretty neat proof of concept.

The question is: would the AI data be smaller than shipping the high quality textures in the first place?

<comment><USER5> Shipping size is usually not the limiting factor for texture size, GPU memory is. Texture compression must work on small blocks for random pixel addressing.

<comment> 0-_-0 Which doesn't entirely rule out a deep learning based compressor, just makes it much harder :)

</comment></comment></comment></comment></comment><comment><USER6> A couple of things that would be interesting to try with this. Artificially increasing the geometry with simple subdivision to give a measure of the impact of geometry on the scene.

Do a bloom post-processing filter to give the impression of brighter than maximum. I can see how this might not be considered the point because it's not a ray-tracing solution, but it is a simple screen-space operation that's independent of the technique used to generate the base image. Including such a filter can add a lot of visual appeal, particularly enhancing the lighting benefits of path tracing.

</comment><comment><USER7> Huh, I wonder if applying that denoising filter to the OpenGL compute version [https://raytracey.blogspot.com/2016/06/real-time-path- traced...](https://raytracey.blogspot.com/2016/06/real-time-path-traced- quake-2.html) is going to be as good

</comment><comment><USER8> A very cool project, kudos to the developers. I don't want to take anything away from it, but I did stop noticing the graphics pretty quickly and just started watching the gameplay. Maybe it's the nostalgia of remembering the original overriding the cool-factor of ray tracing.

<comment><USER9> Same here. Maybe it's partially because I remember the game, and partially because graphics here are good enough to not to distract. In a similar vein, I recently played the remastered version of Modern Warfare 1, and it felt just like I remember playing the original - i.e. a good contemporary game. Were I to play original now, it would feel _dated_.

Except full scene water reflection. This is pure awesomeness, and completely distracted me from watching gameplay.

</comment></comment><comment><USER10> That reminds me of the [http://www.q3rt.de/](http://www.q3rt.de/) project. I wish someone tried to port that one to rtx.

</comment><comment><USER11> Seeing Q2 bring back all sorts of memories. That was right about the time for me when I started seeing the benefits of dedicated graphics cards, and Q2 felt magical with one.

<comment><USER12> TRESPASSAH!!!

<comment><USER12> Here's the reference for those unfamiliar with quake 2. [https://youtu.be/-rWifmDuJEA](https://youtu.be/-rWifmDuJEA)

</comment></comment></comment><comment><USER13> I guess if there's ever a reason to replay Q2 single-player, that would be it.

<comment><USER14> That and the soundtrack ;)

<comment><USER13> You need a disk for that as in CD-DA, unless that was patched to use files.

</comment></comment></comment><comment><USER15> I'd love a version of Half-Life (using the original engine) with this, that's for sure.

<comment><USER16> You're in luck: [https://www.dsogaming.com/news/classic-half-life-gets- esrgan...](https://www.dsogaming.com/news/classic-half-life-gets-esrgan-ai- enhanced-hd-textures-available-for-download/)

<comment><USER17> Raytracing, not textures.

</comment></comment></comment><comment><USER18> It should probably be noted also that this doesn't even seem to be using textures updated with normal & specular maps. This is probably why all the surfaces appear a bit plasticky.

</comment><comment><USER19> Those water reflections were oddly satisfying to see.

</comment><comment><USER20> They might have wanted to build the demo on some game that has more realistic geometry and textures to showcase the benefits of raytracing. Quake 2 still looks 90's mostly. I can spot the "hard" features in the demo myself to some extent but given well-known shading techniques you could get a close enough similar emulation of the effects that would fool most people.

</comment><comment><USER21> > Fully dynamic global illumination using path tracing, with raytraced > shadows, glossy reflections and one bounce of indirect lighting.

I wonder how different two, three and four bounces of indirect lighting would make it look.

Would be super interesting to see a side-by-side comparison.

</comment><comment><USER22> How hard would it be to implement varying resolution across the monitor with this approach, e.g. to combine with an eye-tracker to ensure that only the parts that the user looks at, are rendered in full resolution?

<comment><USER23> Varying resolution is trivial with raytracing, however a big part of the work here is not tied to screen resolution. Lighting is sampled far below screen resolution and needs to be reconstructed over multiple frames. Eye position adds no useful information here. Doing full-resolution "cinematic quality" path-tracing would require hardware that is magnitudes faster.

Having said that, variable shading rate is a feature of the newest NVIDIA GPUs (and supported by Wolfenstein: New Order) and it could be used with a sufficiently low-latency eye tracker rather trivially as well.

</comment></comment><comment><USER24> It feels like the effect could be improved by disabling most of the texture maps. The low resolution and baked-in lighting distracts from the physically correct surface shading.

</comment><comment><USER25> To be honest, I'd really rather see this for Quake 1, which imho had better overall gameplay.

<comment><USER26> Quake 2 introduced (and overused) coloured lights, which are great showcase of path-tracing's capabilities.

</comment></comment><comment><USER27> woa, the water reflects

</comment><comment><USER28> Why, oh why not the first Quake?

</comment><comment><USER29> core i9 x are coming awfully close to larrabee performance as demonstrated in 2008, we should be close to be doing this on the cpu unassisted

</comment><comment><USER30> Very strange, I'm on a JetBlue flight and the website appears to be blacklisted on their in-flight WiFi ("Fly-Fi")?</comment>


Deep Learning for Programmers Book (GPU and CPU) Release 0.10.0 - <AUTHOR> https://aiprobook.com/deep-learning-for-programmers?release=0.10.0&src=hn <comment><AUTHOR> \+ chapter on Momentum and Nesterov Momentum

Uses fast open source libraries to build everything from scratch on CPU & GPU. [https://github.com/uncomplicate](https://github.com/uncomplicate)</comment>


Ask HN: How do you prefer to spend your weekends? - <AUTHOR> Actively swtiched off? Time with family? Working on side projects? Errands? A little bit of each? <comment><USER2> Saturday - Chess club in the morning followed by a leisurely lunch/reading in a comfortable coffee shop.

Sunday I spend with the family.

It wasn't intentional but having Saturday to myself til the evening meal is awesome for unwinding from a busy work week, everyone needs (or at least should try) genuine unplugged alone time (particularly so for natural introverts like me).

</comment><comment><USER3> I just like working, so on weekends, I either work on a side projet or I work for a startup.

Working let me build things, and be creative, I can't manage to stop working. If I don't work I feel like my time is lost

</comment><comment><USER4> Every other Saturday I help out at a cat shelter, then have lunch at my "Saturday" cafe and work on my personal project while there. That evening I usually make cat food, watch dumb YouTube videos, and work on my project some more. When I'm not going to the shelter I sleep in a little and then go to the gym instead.

Sundays are usually chore days focused on animals and cleaning (well, we're trying to get better with the cleaning). This is when I do a water change for the fish, go shopping for meat for the cat food (and then prep the food), get any other cat supplies I might need, take the cat/s for a walk or supervise them on the porch, and tidy the house. Once all the pressing chores are done I work on my personal project and watch dumb YouTube videos.

</comment><comment><USER5> Spending as much time not under a roof as possible. This typically means whitewater kayaking or climbing. This is the only way I can stay sane while having a desk job.

</comment><comment><USER6> Ever actively switched off: videogames, gym ( if I didn't go enough in the other days), cleaning the house, going out, oversleeping and, in general, relaxing

</comment><comment><USER7> I prefer not to distinguish weekends from weekdays. So every day is a mix of work, hobbies, and play.

</comment><comment><USER8> With 3 & 5 yr old there is a lot to do and not much choice to do a tonne of leisure activities. If I get some spare minutes I'll do some hacking on some side projects, go for a walk, grab a coffee or something.

</comment><comment><USER9> Playing games and reading HN/the internet. Going out for drinks. There should definitely be more gym in there in place of all of those things.

</comment><comment><USER10> I think I get more work done on the weekends then I do the rest of the normal 5 day work week.

</comment><comment><USER11> Do as little as possible... e.g. sleep, eat, drink with friends, read books and watch tv...

</comment><comment><USER12> how would i _prefer_ ? Couch surfing... how do i _usually_ spend ? Working by morning and running errands with mrs.

</comment><comment><USER13> Playing golf.

</comment><comment><USER14> Reading HN.

<comment><USER15> Working :/</comment></comment>


30 Days Of Sexism - <AUTHOR> http://www.kotaku.com.au/2013/04/30-days-of-sexism/

<comment><USER2> I don't see how changing into a proper shirt is such a big deal – imagine anyone making a film review only wearing boxers and no jeans whatsoever – that's roughly how much you reveal if you wear a 'low cut' shirt.

Nobody would expect to get away with making a film review in their underwear without comments about putting on clothes. And if everybody did that, those who actually put on 'uncomfortable' jeans would obviously be thanked.

And regarding these 'marry me' comments: Go to imgur and you will find plenty of people identifying as female making exactly these comments any time an interesting male picture comes up. Hardly sexist.</comment>


Show HN: MediSim – A 3D simulation of emergent Behavior in little civilizations - <AUTHOR> http://stewart-taylor.github.io/MediSim/ <comment><AUTHOR> Hey Creator here. I actually wanted to post this link instead with the ReadMe as it tells you what is going on but I messed up the submission and it would not let me post again. [https://github.com/Stewart- Taylor/MediSim](https://github.com/Stewart-Taylor/MediSim) This link will give you more details about what is actually going on if you are confused.

I was inspired by this talk [https://www.youtube.com/watch?v=0HJPilemNns](https://www.youtube.com/watch?v=0HJPilemNns) by Simon Swain and wanted to make something similar.

If you have any questions about it feel free to ask me. I would also love to see if anyone has made anything similar.</comment>


Ask HN: Sent out emails about my product, now what? - <AUTHOR>

Update: I'm bootstrapping my project https://scrape.it
I sent about a hundred emails about my product to my mailing list. I want to know what I should do next. Should be split testing the landing pages or should I be waiting for an early adopter to give me feedback?
I still have another 900 or so emails to blast to. I'm just not sure what I should be optimizing here or what I should be aiming towards.
Like the questions I want to know about the user, did they or did they not like what was on the landing page? Did the price not make sense? Why did they stop engaging? etc. <comment><USER2> Give us a bit more context? Is it a consumer or business focused product? Consumer based products require a lot of attention to marketing strategies, value propositions stated clearly, landing page design, etc. Focus on ways to continue to get visitors or references to your site. Only at sustained volume do items like A/B tests matter.

<comment><AUTHOR> It's a business focused product. I updated the question to include my project.

</comment></comment><comment><USER3> The data should tell you. How many went to the landing page? did anyone ask questions? did you have good answers? I personally wouldn't be worried about price until I had up to 10 alpha users. that are willing to get at a free trial if they're willing to let you ask them questions.</comment>


Ask HN: What is your favorite way to read online content? - <AUTHOR> I've been spending a bunch of time recently browsing online content now that I'm at home a lot more.
I find that the reading UX between newsletters, news sites, Medium, Reddit etc is super varied.
Is there anyone site that makes reading content particularly enjoyable for you? <comment><USER2> I use pocket. It is installed by default in Firefox (my default browser). From there I save the articles I want to read. And then i read them on my Kobo eReader (which syncs pocket saved articles automatically). The perfect combo to save my eyes a few "back-lit screen hours".

<comment><USER3> Pocket and Kobo as well (with Newsblur for RSS feed aggregation).

That said, if there was an ereader that worked with Instapaper instead I'd probably jump across. Frequently enough Pocket screws up an article which Instapaper handles with aplomb.

<comment><USER4> Interesting.. I have just the opposite experience. Instapaper sends a weekly epub with my articles to my Kindle. More ofter than not, technical articles (i.e. math, code, many images) do not get converted well, to the point of making some of them completely unreadable.

</comment></comment></comment><comment><USER5> I am happily addicted to NetNewsWire 5.0 + Feedbin

I get extremely high quality content and no advertisements, tracking or pop- ups. Also, NetNewsWire is blazing fast.

If you're looking for some RSS recommendations:

[1] An extremely good blog for the latest in Covid related pharmaceutical research: [https://blogs.sciencemag.org/pipeline/](https://blogs.sciencemag.org/pipeline/)

[2] Math with bad drawings: [https://mathwithbaddrawings.com/](https://mathwithbaddrawings.com/)

[3] A blog that analyzes military details in fictional battles: [https://acoup.blog/](https://acoup.blog/)

</comment><comment><USER6> I don't read much of anything online these days. Most "reading" environments treat you as a user, not a _reader_. Poor typography, endless scrolling, and an endless supply of content that isn't worth reading in the first place. So I mostly consume text on screen only so that I can accomplish tasks. In other words, "reading" online is a means to an end.

All that said, if I stumble on a long form piece of writing that looks interesting, I'll print it!

<comment><USER7> I also tend to print anything substantial that I plan on reading. It's just easier on the eyes (though Firefox's reader mode and usejournal.com also help) and I can re-use the blank side of the paper for my notes. After that, it makes an okay fire starter.

I've also started subscribing to magazines and newspapers again. Most of the 'online' content I read is available in print form.

<comment><USER6> I am very much a fan of the printed word. I've been thinking about it like this: Online, text _delivers itself_ to you, almost always in short-form streams, everything is ephemeral. With a physical book, you have to _bring yourself_ to the text. Language, dialog, form, typography all come together on the page in a way that is highly satisfying. The act of reading printed pages is an "event" of sorts.

Any magazine recommendations?

<comment><USER7> I really enjoy both the Economist and the New Yorker. They tend to take very different stances on the same issues, but both have really excellent long-form articles that I enjoy. I also recommend hobby magazines (like Shutterbug or Adventure Kayak) that fit with your interests.

</comment></comment></comment></comment><comment><USER8> Firefox's Reader View, with custom userContent.css to fix the unnecessarily low contrast:

      @-moz-document url-prefix("about:reader") {
        body {
          background-color: #FFFFFF !important;
          color: #000000 !important;
        }
      }

In recent versions of Firefox this requires toolkit.legacyUserProfileCustomizations.stylesheets to be set to true in about:config

The most legible style is the style you're most familiar with, so "design" is almost always a negative. I don't care about your unique branding.

</comment><comment><USER9> For any text with more than a couple paragraphs, I always hit the "reader mode" button. If a webpage doesn't work with that, I'm not interested.

</comment><comment><USER10> Like many others I use RSS pretty much exclusively (using native clients on MacOS and iOS). I use reader mode by default in those apps and for my web browser too. HN is the only non-RSS sites read regularly.

The one site I'd _like_ to read but don't is Scott Galloway's -- he doesn't have an RSS feed and it's just not worth visiting the site to see if there's anything new.

<comment><USER11> I assume you're aware HN has an RSS feed? That's actually how I get to HN. I never actually read the front page directly...

<comment><USER10> How does that work? I'm interested in the number of votes but I figure once I've cached the feed file that won't change.

Are you using hnrss?

<comment><USER11> To be honest I don't know! I consume HN's feed at:

[https://news.ycombinator.com/rss](https://news.ycombinator.com/rss)

But that might just be a feed of things in order of submission...

</comment></comment></comment></comment><comment><USER12> RSS. Feedly to manage sources. Newsify on iOS and ReadKit on macOS to consume content. Love Newsify's extensions to share and publish

</comment><comment><USER13> Fraidycat: [https://fraidyc.at](https://fraidyc.at)

helps a lot to read LESS and focus more

<comment><USER14> Looks interesting, thanks !

</comment></comment><comment><USER15> My system primarily revolves around Instapaper[0].

If I run into anything that looks like it might be intereseting I save it to Instapaper using the browser extension on my workstation, or the app on my phone / tablet.

I also have a handful of email lists that automatically forward into instapaper, and a couple rss feeds that auto send to instapaper (Via ifttt[1]).

When I want to actually consume content I use the instapaper iPadOS app, which has a simple and clean UI. (This can also be done in your browser if you prefer)

[0]: [https://www.instapaper.com/](https://www.instapaper.com/) [1]: [https://ifttt.com/](https://ifttt.com/)

</comment><comment><USER16> Earlier last year, I wrote a tool which scrapes a ton of web feeds (industry news, news about investments, cooking blogs, etc..) to my email. (shameless plug[0]). Everything gets organised with labels that fit my morning reading workflow well.

Something else I've also been practicing is not letting online reading disrupt me at work. If I ever find my concentration drifting, I use an extension which bookmarks it for later (either Google Keep or Pocket). Usually I have a nice backlog for when I'm in the reading mood.

[0][http://feedsub.com](http://feedsub.com)

</comment><comment><USER17> Push to Kindle [1]. There are extensions for both Chrome and Firefox.

I wrote myself a simple python script so I can send articles from terminal [2].

[1] [https://www.fivefilters.org/push-to- kindle/](https://www.fivefilters.org/push-to-kindle/)

[2] [https://pypi.org/project/url2kindle/](https://pypi.org/project/url2kindle/)

</comment><comment><USER18> From principles: you want to minimize the left right action of your eyes/neck, to reduce fatigue, and not blast your retinas with a wall of white pixels.

One column of centered text with a non-busy black and white background looks great. For a while I read Amazon books like that because I put the online reader in an iframe.

Now I use a HiSense A5 e-Ink screen with a remote control for page turning and I couldn't ask for more.

</comment><comment><USER19> I like to read my online content as offline content. Paper prints FTW.

</comment><comment><USER20> This is mostly aimed at night-time reading and is an extension of f.lux but on linux you can apply custom glsl scripts to the screen with the compton compositor. I invert the screen and then highly red-tint it. This way I get 'native' darkmode on everything black-on-white, without having to depend on developers making a dark mode, buggy userscripts, and it works on all applications, even videos. ([https://github.com/JZL/linux-color- inversion](https://github.com/JZL/linux-color-inversion))

I have a script which changes my terminal and texteditor colorscheme then calls this script

</comment><comment><USER21> I like to bound the amount I read (fewer endless scrolling sites) by subscribing to a limited number of email lists.

It prevents fomo, idly browsing sites with content I've already read, etc

And if the content stops being high quality (or starts bring diluted with low quality content), unsubscribing is cheap

Ideally, I would pay ~$1000 per year and have high quality content (no more than 1-2 hours of reading per day) ranging my interests be delivered to my inbox

If something is overly topical, it'll quickly get archived if I didn't end up reading it the day it arrived

</comment><comment><USER22> Pocket + Kindle. Kindle doesn't do it automatically, but (e.g.) P2K handles the automation side.

[https://p2k.co/](https://p2k.co/)

<comment><USER23> This seems like a better free service. [http://www.klip.me/sendtokindle/](http://www.klip.me/sendtokindle/)

<comment><USER22> Will try it out, thanks for the heads up.

</comment></comment></comment><comment><USER24> Been reading a lot of longer articles and papers lately. If the author doesn't provide a PDF, I roll my own -- using Reader Mode and Print To File. I store a copy in Zotero so I can keep track of things and so I can add notes.

This enables distraction free reading and lets me archive the stuff I'm reading.

Used to have a similar workflow with Polar instead of Zotero, but switched to Zotero.

<comment><USER25> I'm using Polar for this and am pretty happy with it, especially with the page marks for reading progress and with the highlighting/annotations.

I'm not too familiar with Zotero. Does it have similar features, or do you just use it for storage and read/annotate in your PDF viewer?

</comment></comment><comment> Avi-D-coder Pocket and @Voice apps tts. @Voice allows you to jump through text by double tapping and supports more sites and formats (PDF, etc.), but pocket manages your article list so much better so I use pocky 90% of the time.

Pockets uses a readermode view, so it's non tts UI is nicer, so if you don't care about tts just use pocket.

Firefox reader mode is also nice, Firefox preview is great.

<comment><USER26> I think Pocket also supports tts.

[https://help.getpocket.com/article/1081-listening-to- article...](https://help.getpocket.com/article/1081-listening-to-articles-in- pocket-with-text-to-speech)

<comment> Avi-D-coder Yes it does, I mostly use pocket for tts, but the ui is worse. If you use android's local you can only skip forward and backwards 15 seconds, and it frequently loses your place if you pause it. @Voice allows you to tap on the text to seek.

</comment></comment></comment><comment><USER27> I use Innoreader for feeds and a lot of email newsletters. If you don't like things in multiple places, Innoreader can also generate feeds from websites (with mixed results depending on the structure of the website) and can give you temporary emails that you can subscribe to newsletters in so that they'll show up in the same interface.

</comment><comment><USER28> You can give [https://intrgr.com](https://intrgr.com) a try. It standardizes the Article format (removing clutter), gives you related articles from around the web, and recommendations if you make an account.

I made it because the web has become unreadable for me.

Disclaimer: Mobile view comes out next week

</comment><comment><USER29> Pocket + Kobo (my eReader)

Kobo has no RSS client, though, so for catching up on news feeds, Feedly on my Android phone or tablet. But I've largely stopped using Feedly at all, as my backlog would simply get so far that it started feeling like a chore to get through them.

I recommend longform.org for finding great longform article content.

</comment><comment><USER30> I started aggregating a few finance sites via RSS, and then added twitch trading streams, and some discord servers.

It was my own personal feed for a while, but I've now put it on the homepage of a side project - [https://topstonks.com](https://topstonks.com)

</comment><comment><USER31> The author's website. I'm done with big aggregated content platforms, I started using RSS again recently and trend to my own digital garden of blogs and newsletters.

I use Slack's RSS integration to get new articles notifications, then read them as they were intended to be read by the author.

</comment><comment><USER32> I use a RSS feed aggregator, which I curate with blogs and websites I discover and want to follow.

</comment><comment><USER33> I find Reeder for iOS to be exemplary in terms of UX. I use inoreader to sync the state and find it to work well as a tool for that.

I get everything I can in there. Email newsletters, hacker news etc are all in here to save my inbox.

</comment><comment><USER34> For longer articles I save them to pocket and read them on my kobo e-reader.

</comment><comment><USER35> the nytimes website (with an adblocker) is about as good as it gets for me. my wants are pretty simple:

\- nothing jumping around as i try to read, whole article on one page

\- decent contrast (accessibility guidelines says >= 4.5:1)

\- ~150% line height

\- ~80char line width

\- ~16pt font size

</comment><comment><USER36> If anyone has a Kindle and is willing to help us beta test a new RSS/newsletter reading service for the Kindle please email me at the email address in my bio. I would love to get your feedback.

</comment><comment><USER37> Miniflux + Pocket.

RSS / Atom / Json feeds are the way I subscribe to content and then read it either in Miniflux or Pocket. I rarely visit sites directly.

</comment><comment><USER38> All my feeds (rss) in one daily email - [https://rssmailer.app](https://rssmailer.app)

</comment><comment><USER39> I save everything to pocket that I want to read later while browsing the web.

I use feedly for RSS feeds that go to pocket.

I have a script that tags pocket articles.

<comment><USER40> Care to share the script and how it tags the articles? Sounds interesting.

</comment></comment><comment><USER11> tt-rss (with plugins to inline full content into the feed) + wallabag + calibre RSS push to Kindle (for offline reading of long-form content).

</comment><comment><USER41> Android:

Fwiw, pocket is great. ReadEra is perfect for PDF reading.

</comment><comment><USER42> RSS in InnoReader (web and Android)

</comment><comment><USER43> Hacker News, Twitter, and QuiteRSS

</comment><comment><USER44> RSS using Inoreader

</comment><comment><USER45> Instapaper!

</comment><comment><USER46> Feedly.

</comment><comment><USER47> On latest news: I'm hooked on TLDR Newsletter.</comment>


Pi digits in music - <AUTHOR> http://3.14-pi.net/

<comment><USER2> Too bad this has been buried. This is the sort of thing that HN needs more of.</comment>


How does dynamic dispatch work in WebAssembly? - <AUTHOR> http://fitzgeraldnick.com/2018/04/26/how-does-dynamic-dispatch-work-in-wasm.html <comment><USER2> Kinda annoying, but not surprising. VMs and bytecode formats make simplifying assumptions based on the languages they expect to be hosted on top of them, and that usually means "Something with control flow like C."

Creative uses of computed jumps, messing with the stack, dynamic codegen, all sorts of weird things your new language might do to efficiently implement some new control or data structure aren't likely to be possible.

At least in the short term nobody is going to be too upset. Today if something needs to wring all the power available from your CPU it isn't reasonable to put it on the web. That will continue to be true. WASM is the wise 80% solution, not a toy for ASM hackers and people messing around with weird prototype programming languages.

<comment><USER3> The reason behind the limited branching instructions in wasm is validation, not compatibility with C. C compilers on real architectures emit all sorts of crazy indirection mechanisms.

<comment><USER2> Right -- the control flow allowed is basically the simplest thing that will allow you to make a C-like language in a relatively efficient way. It isn't aimed at any particular compiler, it's aimed at supporting some semantics.

Providing additional flexibility on top of those semantics is "expensive" in terms of implementor-time and effort to get safety, portability and future- proofing. There's no promise that the fun extensions they might want today go "in the right direction", and the cost of going down a bad path is high for standards.

</comment></comment></comment><comment><USER4> Someone should make a CPU that does this. Intel CET is a bit of a start, but a CALL instruction that takes a "type" operand and only calls functions of that type would be a big improvement. So would a totally separate return address stack.

<comment><USER5> There is also pointer authentication in ARM: [https://lwn.net/Articles/718888/](https://lwn.net/Articles/718888/)

It seems pretty difficult to define function types at the ISA level, since anything to do with typing is language/VM-specific. What types can arguments have? Are varargs supported? Multiple parameter returns?

Maybe the type would just be an integer that the ABI would assign meaning to. But if you did that, how would the function's type be determined? Some pseudo- instruction at the CALL target? I guess looking at CET it does have some things like this: the ENDBRANCH instructions notate valid indirect branch targets.

<comment><USER4> > Maybe the type would just be an integer that the ABI would assign meaning > to. But if you did that, how would the function's type be determined? Some > pseudo-instruction at the CALL target?

Exactly.

</comment></comment></comment><comment><USER6> How do you implement co-operative multitasking in WebAssembly?

Do you have to analyze the source program & know where every possible call to a yield is, and store all resulting suffixes of a function as new functions?

</comment><comment><USER7> I'll be interested to read about the first WASM vulnerabilities that are revealed.

</comment><comment><USER8> So how are function pointers in (say) C implemented?

<comment><USER9> From playing around with [https://mbebenita.github.io/WasmExplorer/](https://mbebenita.github.io/WasmExplorer/) : it seems like any function you ever take the address of gets stored in the table, and wasm just passes the index in that table around. And if you do things like cast an integer to a function pointer, you actually just cast it to an index in the table.

C is high-level enough that IIRC it doesn't guarantee that arbitrary addresses can successfully be used as function pointers; the only valid function pointers are results of addresses of functions, or (possibly) values that have been cast from and then to them. Which is enough for wasm.

Try compiling this (to wat, which is an S-expression format that otherwise resembles the assembly in this article): [https://wasdk.github.io/WasmFiddle/?5zbjb](https://wasdk.github.io/WasmFiddle/?5zbjb)</comment></comment>