Myths Programmers Believe about CPU Caches (2018) - <AUTHOR> https://software.rajivprab.com/2018/04/29/myths-programmers-believe-about-cpu-caches/ <comment><USER2> If cache coherence is relevant to you, I strongly recommend the book "A Primer on Memory Consistency and Cache Coherence". It's much easier to understand the details of coherency from a broader perspective, than an incremental read-a- bunch-of-blogs perspective.

I found that book very readable, and it cleared up most misconceptions I had. It also teaches a universal vocabulary for discussing coherency/consistency, which is useful for conveying the nuances of the topic.

Cache coherence is not super relevant to most programmers though. Every language provides an abstraction on top of caches, and nearly every language uses the "data race free -> sequentially consistent". Having an understanding of data races and sequential consistency is much more important than understanding caching: the compiler/runtime has more freedom to mess with your code than your CPU (unless you are on something like the DEC Alpha, which you probably aren't).

If you are writing an OS/Hypervisor/Compiler (or any other situation where you touch asm), cache coherence is a subject you probably need a solid grasp of.

<comment><USER3> Disagree on that last part. If more programmers understood cache coherency, maybe their programs would not run like a giant turd.

<comment><USER2> Most engineers don't write code with hard performance constraints. Game devs probably need to be fighting to get every frame.

For the bulk of the eng I work with the concept of StoreLoad reordering on x86 would be an academic distraction.

<comment><USER4> > Most engineers don't write code with hard performance constraints.

Only because most software "engineers" don't give two shits about the actual user experience of their glacially slow over-engineered garbage.

<comment><USER5> 99.999% of performance issues in development are related to the abstract model, and not the underlying implementation details. Things such as searching in a big unordered list, repeating the same work, stupid SQL queries ect.

<comment><USER6> The way I usually follow it is 1) Is this OK? For example It is only called once in the code in an error path? 2) Did I do something silly? For example, I left in some extra debug code. 3) is the code doing something silly? For example extra work, dragging in extra data that is not needed? 4) is the code written in a way that causes extra work, re-init on inner loop, loop hoisting, etc 5) Is there a better algorithm for this? Binary search vs linear? 6) is the code doing too many things at once. De-serialize it re-serialize it 7) Is there a better abstraction for this? Monolith code vs microservice? 8) Is there any compiler switches that are 'easy' wins? Packing, -O2, etc? Usually not. 9) What sort of memory arch does this machine have? It varies between machines even for the x86 world. For example if I rearrange my memory structures do I use less cache and emit less code. The cache line on this box is x bytes and on that box is y bytes. Some languages do not lend themselves well to this one due to the way their VM/ISA is written. 10) is there some asm that would help? usually not.

Usually 1-7 are all you need. If you get all the way to 10 you are in the deep end for most things.

Big O is good for many things. But in reality big O is O(N)+C where the C can get you. That is where the later steps help. But usually you can totally ignore it. Most of the big wins I get are just from flipping out a bad search for a O(log(n)) search, or removing 'extra' code.

<comment><USER5> On the BigO notation, you have to remember is that it's an upper bound on the runtime. There is no guarantee other than the growth of the function.

<comment><USER6> Oh I agree. It is just that +C bit. What happens when your O(nLog(n)) basically just trashes the cache because of your data structures? Yet maybe something 'worse' would run better because of the way cache works? That +c bit can sometimes turn into its own big O. It is a nasty little gotcha when it does happen.

It is a good framework to get you in the ballpark of the correct thing. Even usually 99% of the time it is right. But sometimes the arch bites back due to your data.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER7> A good, introductory, high-level overview of what is going on with cache coherence... albeit specific to x86.

ARM systems are more relaxed, and therefore need more barriers than on x86. Memory barriers (which also function as "compiler barriers" for the memory / register thing discussed in the article) are handled as long as you properly use locks (or other synchronization primitives like semaphores or mutexes).

Its good to know how things work "under the covers" for performance reasons at least. Especially if you ever write a lock-free data-structure (not allowed to use... well... mutexes or locks), so you need to place the barriers in the appropriate spot.

\</comment><comment>

I think the acquire/release model of consistency will become more important in the coming years. PCIe 4.0 is showing signs of supporting acquire/release... ARM and POWER have added acquire/release model instructions, and even CUDA has acquire/release semantics being built.

As high-performance code demands faster-and-faster systems, the more-and-more relaxed our systems will become. Acquire/release is quickly becoming the standard model.

<comment><USER8> > ARM and POWER have added acquire/release model instructions

They have implemented the acquire-release consistency model since day one (or, the day they started supporting multi-processors). Yes, there are some subtleties there that have in some cases been tightened later on, e.g. multi- copy atomicity.

<comment><USER7> IIRC, there was a big stink because ARM and POWER historically implemented consume/release semantics, which is very slightly more relaxed than the now de-facto standard acquire/release semantics.

ARM and POWERPC CPU devs worked very hard to get consume/release into C++11, but no compiler writer actually implemented that part of the standard. As such, consume/release can be safely forgotten into the annals of computer history (much like DEC Alpha's fully relaxed semantics)

Then in ARM8, ARM simply added LDAR (Load-acquire) and STLR (Store-release) instructions. [https://developer.arm.com/docs/100941/0100/barriers](https://developer.arm.com/docs/100941/0100/barriers) . So the ARM CPU how fully supports the acquire/release model. Apparently IBM's POWER instruction set was similarly strengthened to acquire/release (either POWER8 or POWER9).

ARM / POWER "normal" loads and stores are still consume/release semantics. But compilers can simply emit LDAR (load-acquire) for the stronger guarantee.

\</comment></comment></comment><comment>----

I remember at least one talk that showed that consume/release is ideal for things like Linux's RCU or something like that (that acquire/release is actually "too strong" for RCU, and therefore suboptimal). But because compiler-writers found consume/release too hard to reason about in practice, we're stuck with acquire/release.

It seems like the C++ standard continues to evolve to push for memory_order_consume ([http://www.open- std.org/jtc1/sc22/wg21/docs/papers/2018/p075...](http://www.open- std.org/jtc1/sc22/wg21/docs/papers/2018/p0750r1.html)), but all the details are still up for discussion.

<comment><USER8> > ARM and POWERPC CPU devs worked very hard to get consume/release into C++11

AFAIR Paul McKenney was the primus motor, and the motivation was largely RCU. Then again, McKenney also worked for IBM at the time and certainly had an interest in pushing a model that mapped well to POWER.

But it turned out to be both somewhat mis-specified and hard to implement cleanly, so most compilers just implemented it as an acquire.

As you mention, there is ongoing work to fix it.

As for ARM, it seems the big thing they've done since the initial release of ARMv8 is to banish non-multicopy atomicity. See [https://www.cl.cam.ac.uk/~pes20/armv8-mca/armv8-mca- draft.pd...](https://www.cl.cam.ac.uk/~pes20/armv8-mca/armv8-mca-draft.pdf)

</comment></comment><comment><USER9> One thing not mentioned here (nor in previous discussions of the article, it seems) is that DMA is typically not coherent with the CPU caches. This is kinda visible from the little diagram at the top, with the disk sitting on the other side of the memory, but it should be explicitly spelled out. If you're using a DMA device (memory<->device or memory<->memory copies), you might end up in a state where the DMA and the CPU see different values. This usually means data transfer to/from a Disk or GPU, though other peripherals might use it too.

Your options here are either to manually invalidate your caches and synchronize with the DMA (e.g. via interrupts), or to request from the OS that the given memory section be entirely uncached; or in some cases, you can get away with a write-through cache policy, if the DMA is only ever reading the memory.

<comment><USER10> I think DPDK does some user-level trickery to achieve per-core caching through DMA, do you happen to know how they go about it?

</comment></comment><comment><USER11> For those interested (in 2014!) I did a rather simple analysis of CPU caches and for loops to point out some pitfalls:

[https://austingwalters.com/the-cache-and- multithreading/](https://austingwalters.com/the-cache-and-multithreading/)

Hope it helps someone, I tend to link it to my co-workers when they ask me why I PR'd re-ordering of loops & functions OR when they ask how I get speedups without changing functionality.

</comment><comment><USER12> I've never heard of the MESI protocol before so that was really interesting to read, and I liked the comparison to distributed systems.

I'm wondering if the MESI protocol could be used in a networked database manner? I feel like you need master node though to coordinate everything though (like the L2 does in the example).

<comment><USER13> The essential point of MESI is that it _doesn 't_ need a single master. In a sense, anyone who has exclusive ownership of a cache line is the "master" for that one cache line.

The downsides of MESI are that (a) it requires broadcasts, which don't scale very well; and (b) it doesn't tolerate partitions -- which also imposes an effective scaling limit, since large systems are always partitioned (usually with a partition of N-k and k partitions of 1, due to k nodes having failed).

<comment><USER8> > The downsides of MESI are that (a) it requires broadcasts

No, it can be implemented with directory instead, e.g.

[https://en.wikipedia.org/wiki/Directory- based_cache_coherenc...](https://en.wikipedia.org/wiki/Directory- based_cache_coherence)

Or various combinations of snooping and directories ("snoop filters", or directories that act as "bridges" between broadcast domains, etc.).

In current Xeon processors (and presumably AMD EPYC as well, thought I don't yet have first-hand experience with those), you have a couple of directories per CPU with snoop filtering, as with tens of cores broadcasting becomes a scalability bottleneck. In the BIOS you can change the mode how it operates, with slightly different names and semantics depending on the CPU generation.

</comment></comment></comment><comment><USER14> Something I don't understand is how to deal with cache coherency when you need the same data in a bunch of different configurations.

Take a typical game loop and assume we have a list of Transforms (e.g. world matrix, translation/rotation/scale, whatever - each Transform is a collection of floats in contiguous memory)

Different systems that run in that loop need those transforms in different orders. Rendering may want to organize it by material (to avoid shader switching), AI may want to organize it by type of state machine, Collision by geometric proximity, Culling for physics and lighting might be different, and the list goes on.

Naive answer is "just duplicate the transforms when they are updated" but that introduces its own complexity and comes at its own cost of fetching data from the cache.

I guess what I'm getting at is:

1) I would love to learn more about how this problem is tackled by robust game engines (I guess games too - but games have more specific knowledge than engines and can have unique code to handle it)

2) Does it all just come out in the wash at the end of the day? Not saying just throw it all on the heap and don't care... but, maybe say optimizing for one path, like "updating world transforms for rendering", is worth it and then whatever the cost is to jump around elsewhere doesn't really matter?

Sorry if my question is a bit vague... any insight is appreciated

<comment><USER15> Assuming that once determined at the start of the frame (e.g. camera position changes after user input handling), the transform matrices are not written to. They can then be freely shared across multiple cores without causing problems with coherency. The cache lines associated with the transform will be set to 'Shared' across all cores. Cache coherency will start to bite you in the ass in this situation if you start mutating the transforms while other threads are reading it, causing cache invalidations and pipeline flushes across all caches owning those lines.

In short, write a transform once and treat it as immutable. Do not reuse the Transform allocation for a good while for subsequent frames to ensure that its cache lines are no longer in cache. If you do need to reuse right away, you can force invalidate cache lines by addresses, so that the single-writer in the next step is the single (O)wner and no other caches need to invalidate anything.

<comment><USER14> Thanks - I'll have to do a bit more learning to really understand this, e.g. how mutability relates to cache lines and what "Shared" means in that context, but this gives me some good practical direction and insight to take it further :)

</comment></comment></comment><comment><USER16> Discussed last year: [https://news.ycombinator.com/item?id=17670095](https://news.ycombinator.com/item?id=17670095)

</comment><comment><USER17> This might be a naive question: how did we decide as an industry that cache should be controlled by the hardware, but registers and main memory by the compiler?

<comment><USER15> What do you mean by 'controlled by hardware'? The registers the compiler chooses for you are an abstraction themselves, one of the first things a CPU does is register renaming. Same for main memory, you are presented mostly an abstraction of main memory, with a ton of layers in-between (load/store buffers, write-combine buffers, coherent caches, etc)

Turning the argument the other way, you as a programmer can control a lot about caches: you can prefetch cache lines, invalidate them and use streaming instructions to bypass them.

<comment><USER17> What you said makes sense. The question is more from the perspective of the compiler/assembler. At these layers you explicitly say, move this number to this register, or read a number from this address. But you very rarely are able to say, copy this chunk of data into this cache line. Sure there are exceptions (like the GPU), and there are cases where you can hack it to do what you want. But in general you don't get to control the cache in a very specific way.

</comment></comment></comment><comment><USER18> Anyone else start thinking of Rust's mutable/immutable borrow system when reading the MESI algorithm? It's not quite the same - with Rust, mutable borrows are never shared, and you can never mutate a shared read-only borrow - but the principle seems like a simplification of the full MESI protocol.

It seems like this would be generally applicable for a wide variety of distributed & concurrent applications.

<comment><USER19> Keeping the directory coherent is the difficult part when translating directory-based cache coherence protocols to other distributed systems problems. The directory is like an oracle that sees every transaction in order. This is hard in most network distributed systems problems where you have to worry about availability, network partitioning, or durability of this node.

<comment><USER15> Indeed, the evolution will probably in the other direction, with the on chip network adopting algorithms from wider networks to deal with scaling problems. DRAM interfaces use to be pretty simple, now they are being trained almost like a DSL line.

</comment></comment></comment><comment><USER20> AMD's Rome processors are an interesting case, with 8MB of L3 cache per core. So the 64 core processor has 512MB of L3 cache. It wasn't that long ago that 512MB was a respectable amount of DRAM in a big server. An early 90's fridge sized Sun 690MP maxed out at 1GB of DRAM and had 1MB of L2 cache, no L3.

<comment><USER21> Half that - 4 MB per core so the 64 core CPU has 256 MB (dual socket is where the 512 number comes from but that's 128 cores and NUMA).

It's also not fully accessible, each core can only directly access the 16 MB in its group of 4. Everything else is the same as a cross cache read.

<comment><USER20> Ah, yeah. Mixed up their CCD and CCX terms. The 690MP was dual socket though, so still a somewhat valid comparison.

</comment></comment></comment><comment><USER22> > "different cores can have different/stale values in their individual > caches".

Different processes can certainly have different versions of the same state, different values for the same variable, and different values at the same virtual address.

And what about virtual caches? Non-coherent cache lines?

Moreover, even in the face of cache coherency you can still have race conditions.

<comment><USER23> > Different processes can certainly have different versions of the same state, > different values for the same variable, and different values at the same > virtual address.

what do you mean? Either two caches agree on the content of a cacheline or one of the cacheline is marked invalid (and the stale content is irrelevant). There are components of a core that might not respect coherency, like load and store buffers and arguably registers, but not caches (on cache-coherent systems of course).

Virtually addressed caches are an issue and that's why they have fallen out of favor.

</comment></comment><comment><USER24> The one-line summary seems to be that one should never worry about caches themselves introducing concurrency bugs.

I mean after we account for memory operations reordering on each core, the memory address storing a single value that is visible to all cores is a correct model from the concurrency-correctness point of view, right?

<comment> Tuna-Fish On x86, the correct model is that on any core, all reads are in order, all writes are in order, and no write will ever be moved earlier than a read on that core.

Or in other words, the only kind of visible reordering that is allowed to occur is that writes can be delayed past reads.

An example of a situation where this is significant:

        thread 1       thread 2
        mov [X], 0     mov [Y], 0
        mov [X], 1     mov [Y], 1
        mov r1, [Y]    mov r2, [X]

After this sequence of code, r1 == r2 == 0 is legal. (As is any other combination of 1 and 0.)

(edit:) And just to add, all this reordering is of course impossible to detect on just one core, as when a read request hits a recent write on the same core, it reads it out of the store queue. This can sometimes be really bad for performance, though, as if you read a value that is partially in the store queue (such as, write 16-bit value to x, the immediately read 32 bits from x), some cpus will stall that read, and all that follow it, until the entire store queue is flushed. Since the store queue can easily take tens if not hundreds of cycles to clear, this can be very expensive.

</comment></comment><comment><USER25> It's worth pointing out that the L1 cache and its associated logic is the only* way a core talks to the outside world, including all I/O ever. With that in mind it is easy to understand why it is so crucial to performance.

* there might be some minor exceptions

</comment><comment><USER26> The biggest myth is that any of this matters to anybody but a tiny fraction of niche programmers.

The reason some "app" is slow is not because of cache coherence traffic. It's because somebody chose the wrong data structure, created some stupid architecture, wrote incomprehensible code that the next guy extended in a horribly inefficient way cause they didn't understand the original. My web browsing experience is slow because people include heaps of bloated JS crap and trackers and ad networks so I have to load 15 megabytes of nonsense and wait for JS to render stuff I don't want to see. None of this was any better if anybody involved understood CPU caches better.

Even in the kernel or HPC applications, most code is not in the hot path. Programmers should rather focus on clean architectures and understandable code. How does it help if you hand-optimize some stuff that nobody understands, nobody can fix a bug, nobody can extend it. That's horrible code, even if it's 5% faster than what somebody else wrote.

TL;DR: This is interesting, but likely totally irrelevant to your day job. In the list of things to do to improve your code, it comes so far down that you'll never get there.

</comment><comment><USER27> Half a decade - woow! Sounds like the author has spent half a century there..

</comment><comment><USER28> the entire design of unix is realized in a moment when a motherboard is seen as a network.

</comment><comment><USER29> Should include tl;dr your concurrency fears are real, but for registers and not caches.

<comment><USER30> If you have two threads reading then writing values in memory, you still need synchronization/atomic changes at the software level.

</comment></comment><comment><USER31> Note that this is quite specific to x86, on other architectures like Power there are much weaker guarantees that will lead to problems when assuming the same model.

<comment><USER23> which part is x86 specific?

<comment><USER31> To quote from [https://fgiesen.wordpress.com/2014/07/07/cache- coherency/](https://fgiesen.wordpress.com/2014/07/07/cache-coherency/)

"Memory models

Different architectures provide different memory models. As of this writing, ARM and POWER architecture machines have comparatively "weak" memory models: the CPU core has considerable leeway in reordering load and store operations in ways that might change the semantics of programs in a multi-core context, along with "memory barrier" instructions that can be used by the program to specify constraints: "do not reorder memory operations across this line". By contrast, x86 comes with a quite strong memory model."

<comment><USER23> Yes, I know the difference between the memory model of x86 and, say, ARM. I'm asking what's x86 specific on this article about cache coherency.

<comment><USER31> The article explicitly mentions two times things that are only true for x86 (grep for it). In addition, the statement at the end is definitely not true for POWER: "As soon as the data is read/written to the L1 cache, the hardware- coherency protocol takes over and provides guaranteed coherency across all global threads. Thus ensuring that if multiple threads are reading/writing to the same variable, they are all kept in sync with one another."

<comment><USER23> Those two things are not x86 specific (the author only gives x86 an example). And the statement you quote is certainly true for POWER or any other cache coherent architecture.

<comment><USER31> Sorry you are right, I confused this with reordering.</comment></comment></comment></comment></comment></comment></comment>


How to Go to College for Less Than $200 - <AUTHOR> https://medium.com/@mattr555/how-to-go-to-college-for-less-than-200-47a3a0c91b73#.eqfh6ebca <comment><AUTHOR> Yes, that really is how Notre Dame advertises dorms on their website: [http://housing.nd.edu/undergraduate/hall-galleries/alumni- ha...](http://housing.nd.edu/undergraduate/hall-galleries/alumni-hall/)</comment>


OpenGrid – Chicago Releases User-Friendly Open Data Tool - <AUTHOR> http://opengrid.io/ <comment><USER2> The readme is a bit short. In the docs are the actual requirements for the "OpenGrid Service":

      A web server for hosting User Interface application
      An Java EE application server for deploying service layer (the service template implementation has been tested with JBoss EAP 6.2)
      MongoDB version 3.0.6 for storing data and template service layer configuration

Man, I know mongo has some geolocation tools but I would really like to see this with a better database. Good on Chicago for doing a professional job releasing this software though!

</comment><comment><USER3> I'd love to try this out, but Chrome has disabled geolocation for any non- HTTPS sites: [https://sites.google.com/a/chromium.org/dev/Home/chromium- se...](https://sites.google.com/a/chromium.org/dev/Home/chromium- security/deprecating-powerful-features-on-insecure-origins)

</comment><comment><AUTHOR> The code is located here: [https://github.com/Chicago/opengrid](https://github.com/Chicago/opengrid)

</comment><comment><USER4> That picture of Chicago is pretty cool, but it's really annoying to have to load a 5.3 MB picture when opening a site.</comment>


Can't upgrade my Mac - <AUTHOR>

	I bought a new Mac with retina display (4gb/cheapest retina version). I used it for a month without any problems, but after that it started freezing/crashing. I tried contacting apple saying this is not what I expected (with 4gb of ram) with all the crashing's and they said they couldn't do anything about it since it's past the return date. I can't even upgrade the ram (which is the reason for the freezes). Now, they come out with 8gb ram and faster ones for the same price. I can't even sell this for as much as I would have liked either.<p>Shouldn't they be able to help me get 8gb of ram, since my Mac doesn't even function properly? <comment><USER2> I keep encountering gripes that people can't "buy" upgrades to new Apple products. No... but you can totally add more RAM if you up your surface mount soldering skill game. ;)

Semi seriously though... you can totally keep old Mac Pros up to date if you machine the heat sinks, patch & flash custom firmware and solder some things to your main board. Laptops & iMacs too.

It's not impossible, just harder and seriously out of warranty.

We can't achieve miniaturization and faster clock speeds if we keep bulky connectors and forced interconnect design. It takes A LOT to make a CPU upgradable. Memory too.

Most automobiles have gone through this. It used to be that an owner of a vehicle could take apart and rebuild their own engine block. You could adjust your timing, fuel pump output, etc. Not anymore. Just as with cars, the average joe cannot work on their own computer anymore.

Your crashing and freezing is likely to due with shit software. I'd suggest a full backup, a reimage / full reinstall, and a gradual reintroduction of the crap that's breaking it.

</comment><comment><USER3> Hi Stasy.

Hacker News is not a venue for handling customer support complaints.

I know you are frustrated. But you will get better help elsewhere.</comment>


Samsung Galaxy Tab or iPad? - <AUTHOR>

I am planning to buy a tablet. Should I opt for Samsung Galaxy Tab or iPad? <comment><USER2> Me, I'm a Google fan so the Galaxy Tab running Froyo (Android 2.2) is always going to be a winner unless you want to get seriously into iPad synthesizer aps (where it has a huge advantage currently). Ultimately, though, it comes down to deciding what you want it FOR. Games and sound aps? You'll need an iPad. eReading, techy geek stuff, reading news / Google integration? Gotta go with the Android.

<comment><AUTHOR> If I go for iPad, Should I wait for iPad2. I heard its going to be lot better. BTW, I ll be using tablet for techy geek stuff, reading news and ocassional gaming.

<comment><USER3> rofl THEN you will have to wait for ipad 3 because will have a better camera and it will be slimmer XDDDD

dont be a fool dude

<comment><AUTHOR> Hm,,I heard iPad2 will be launched early 2011. I can wait for couple of months

</comment></comment></comment></comment><comment><USER4> The iPad is essentially twice as big. So screen size and portability preferences should steer you well for a start.

<comment><USER5> I'm a fresh baked Galaxy Tab user. So far I've used it for browsing the intertubes and reading ebooks, and I must say it does both pretty well. However, I'd agree that some more screen estate wouldn't hurt, especially for PDF files that don't play well with text reflow (pretty much every book with formatted source code samples).</comment></comment>


Ask HN: Where can I, a non-American, legally bet on the U.S. election? - <AUTHOR> Every site I've found at the top of Google search requires you to be a U.S. citizen. I am a Canadian citizen if that matters. <comment><USER2> Don't know if Interwetten is available in North America; it would be my choice as a European. [1] Also, William Hill has a politics market.

PredictIt is a US-based prediction market, which should work in Noth America, but is not available for many jurisdictions in Europe. [2]

[1] [https://www.interwetten.com/](https://www.interwetten.com/) [2] [https://en.wikipedia.org/wiki/PredictIt](https://en.wikipedia.org/wiki/PredictIt)

<comment><AUTHOR> I think PredictIt is only available in the U.S., their identity verification requires a U.S. address.

</comment></comment><comment><USER3> Find a local bookmaker. One of those will very likely give you odds. But it won't be particularly good odds, seeing that it's a 'pick one of two' kind of bet.

</comment><comment><USER4> The UK? Brits bet on anything.

</comment><comment><USER5> [https://ftx.com/en/trade/TRUMP](https://ftx.com/en/trade/TRUMP)

[https://ftx.com/en/trade/BIDEN](https://ftx.com/en/trade/BIDEN)

[https://www.augur.net/](https://www.augur.net/) will also probably have a prediction market for it on Tuesday when they launch v2

<comment><AUTHOR> > "Trading Trump 2020 and other President 2020 contracts is not permitted by > residents of the United States, Canada, the European Union, the United > Kingdom, Singapore, the UAE, Cambodia, Turkey, mainland China and Hong Kong > SAR, and other FTX prohibited jurisdictions."

[https://help.ftx.com/hc/en- us/articles/360039118112](https://help.ftx.com/hc/en-us/articles/360039118112)

</comment></comment><comment><USER6> [https://www.pinnacle.com/en/politics/2020-presidential- elect...](https://www.pinnacle.com/en/politics/2020-presidential-election- usa/matchups)</comment>


Seeing a Dip in Google Search Traffic Probably Because of the June Update - <AUTHOR> I run a small resource site (https://laravelcollections.com)
I am already seeing pretty deep dip in my Google Search Traffic. https://i.imgur.com/Ij55P0U.png
I think this is because of the June Update ( https://www.searchenginejournal.com/google-june-broad-core-update/310659/ ), what do you think? <comment><USER2> The article you linked to says it's rolling out today, June 3rd, but your screenshot ends on Saturday June 1st, doesn't it? So, probably not.</comment>


HTML5 massive-multiplayer using websocket - <AUTHOR> http://rumpetroll.com

<comment><USER2> This is really cool. It's also quite similar to a game I have in the works. It feels very polished, and there's very little latency.</comment>


Dying in Solitude: First-Hand Accounts of the Coronavirus Horrors in Italy - <AUTHOR> https://www.spiegel.de/international/europe/dying-in-solitude-first-hand-accounts-of-the-coronavirus-horrors-in-italy-a-371012bf-67eb-4bf8-a5dc-828e12c2be06 <comment><USER2> Italy is known for having particularly resistant bacteria compared to the norm in its hospitals ([https://www.ncbi.nlm.nih.gov/pubmed/30124425](https://www.ncbi.nlm.nih.gov/pubmed/30124425))

Can anyone shed light as to whether this is a contributor to the heightened level of deaths compared to other similar European countries?

</comment><comment><USER3> [https://archive.md/1HsQ2](https://archive.md/1HsQ2)

</comment><comment><USER4> Paywalled article. Any chances for a summary/key notes?

<comment><USER5> Disable javascript and it loads fine.</comment></comment>


A first grader could dream up a better way to do this - <AUTHOR>

While most thing are getting better with Android in general and the Play store specifically. The latest change to the Play store app not only sucks but it is beyond apprehension they ever released it like this.
The first page you used to see was a nice spotlight page. While more often than not it didn't displayed anything of interest to me I did buy some apps advertised on this page (It is still there but not as the landing page).
Enter the new recommended system. I think the only thing they do is to look at what you installed at one point in time on your device and randomly show apps that other people installed. This is completely useless and utterly lazy implementation of a recommendation system.
as an example: Out of curiosity I installed a launcher called Dodol. I played with it for 5 minutes, removed it from my device, removed it from the "My apps" section. I did this weeks ago and my recommendation page is still littered with Dolo themes, even after explicitly telling Google Play that I am not interested I still get the same recommendations.
I get recommendations for apps actually installed on my device because it is popular with people that are using application X, and application X is something I removed from my device and from the "My app" list!
A first grader could dream up a better way to do this. I'm really amazed that Google is willing to release something so unbelievable sup par.
In the end it is bad for the developers and Google, because I'm not discovering any new apps.
Just a small frustration :) <comment><USER2> A similar situation happened with the Apple App Store. Many developers were(/are?) unhappy. Many users were unhappy. Within a few months, no one really cared anymore and they adapted. I can't even remember what the old App Store used to look like.

<comment><AUTHOR> I like the idea of recommendations but the way they did it is useless. If they make it better I would be happy.</comment></comment>


Algorithm predicts sexual orientation of men with up to 70% accuracy - <AUTHOR> http://www.theguardian.com/society/2015/oct/08/algorithm-predicts-sexual-orientation-men-70-accuracy <comment><USER2> Given that studies show that over 90% of men are heterosexual, algorithm always returning "heterosexual" result is strictly better.</comment>


Why Facebook Still Seems to Spy on You - <AUTHOR> https://www.wsj.com/articles/facebook-ads-will-follow-you-even-when-your-privacy-settings-are-dialed-up-11551362400 <comment><AUTHOR> [https://outline.com/22aYFw](https://outline.com/22aYFw)</comment>


Verizon Names Data-Draining and Battery-Killing Apps - <AUTHOR> http://www.businessweek.com/articles/2013-02-11/verizon-names-data-plan-and-battery-killing-apps#r=rss

<comment><USER2> I'm going to take this opportunity to complain about Google Navigation app, on every device I've owned.

Their directions really should say "ETA: 3 Hours from now. ETToBatteryDrained: 1.5 hours from now. Shoulda remembered your car charger, dumbass". 'Cause I do forget to pack it every goddamned time.

I don't know if this is really possible, but they really should have a "low- accuracy" mode, where if you're on a freeway with no turns for an hour, they should turn off the GPS, and just use inertial or cell-phone navigation mostly. Then only periodically check the GPS. If you are going to turn soon, or appear to have deviated, they can go back into active GPS mode, but the goal is to avoid wasting battery when you're probably just following the road.

<comment><USER3> It is the nav app keeping the display lit up like a Times Square billboard that is sucking the majority of your battery life down, by the way. Your suggestion stands, however.

<comment><USER4> You can lock the display and the voice directions still subsist. The battery drain is from the constant polling of GPS information in my experience though.

<comment><USER5> > The battery drain is from the constant polling of GPS information in my > experience though.

Why would that use a lot of power though? It's just receiving a signal. I have the impression that it's when sending out a signal that a lot of power is used. And in that sense electromagnetic radiation at GSM, Wifi or visible spectrum-frequencies are all the same.

I'd say it's both keeping the screen on and constantly downloading new map information from the cellular network.

</comment></comment></comment></comment><comment><USER6> One of my friends does research on this for another of the big carriers, and has told me some war stories.

* No one caches, or they cache really badly. They've seen 20-30% reductions in battery usage from implementing a caching scheme from one version to the next. His theory is that they're coming from the web world, where caching is baked in.

* One app killed the battery by waking up the radios every 2 minutes to check for updated scores, sending a few bytes each time. Network traffic, in aggregate, was very small, but it just killed the batteries.

<comment><USER7> That second one seems like an easy mistake to make. If you're thinking of ethernet or WiFi, then a few bytes every couple of minutes is no big deal. But on 3G, every time you activate the radio when it's not being used, it's a few seconds of fairly intensive setup, then your communication, then a few seconds before it times out to a lower-power state, then a few more seconds before it goes fully back to sleep. It'll be something like 10-20 seconds total, so your quick blip every 2 minutes ends up keeping the radio on a 10-15% duty cycle!

<comment><USER8> I've never done mobile dev. I get the problem, but what's the preferred alternative? For, say, email, sports scores, VOIP messages, stats dashboards, etc?

<comment><USER6> For truly time critical stuff, push notifications, done however the platform vendor does it for optimum battery life.

If not, then some sort of exponential backoff helps so that you're not updating scores every 2 minutes all night.

</comment></comment></comment></comment><comment><AUTHOR> I think they made at least one error: Need for Speed: Most Wanted was not developed by Rovio.

Also of interest is that none of the streaming radio services suffered in their rather proprietary ranking: [http://support.verizonwireless.com/information/app_ranking/i...](http://support.verizonwireless.com/information/app_ranking/index.html)

<comment><USER9> It makes sense for a streaming radio app to acquire a wakelock, unless you want it to stop playing when the screen shuts off in 2 minutes. It does not make sense for any game to acquire a wakelock, and acquiring one is a sign that your app does not respect the Activity lifecycle and probably doesn't handle notifications or task switching well anyway. I know GTA3 absolutely sucks at this.

</comment></comment><comment><USER9> The actual list: [http://support.verizonwireless.com/information/app_ranking/h...](http://support.verizonwireless.com/information/app_ranking/high_risk.html)

It is interesting to note that 100% of the apps listed are games with wakelocks. While some may argue that this is necessary for games, I counter that your game should pause after a period of inactivity and allow the device to sleep. Just because your app is a console port does not mean it is running on a console.

Kudos to Verizon for naming and shaming.

<comment><USER10> > It is interesting to note that 100% of the apps listed are games with > wakelocks. While some may argue that this is necessary for games, I counter > that your game should pause after a period of inactivity and allow the > device to sleep.

Fucking meh. Seriously. They list a bunch of games which keep all the most power-consuming subsystems running full tilt (including radio for some), here's why they drain the battery: they're being used and doing their job.

Seriously, this list is idiotic, you've got complex games, you're playing them, it's going to drain the battery when you use them. Period. That's been pretty much a constant of my 6 or 7 years of using smartphones. Users using their phone will play, and when they don't play they'll quit the game or manually lock the phone.

Some of them _do_ have issues ("Hill Climb Racing" seems to be a good example there, keeping radios running when in the background) but aside from this hint you have to hunt for the list is utterly worthless.

<comment><USER9> This isn't about the complexity of the game or how much battery it drains while playing. This is about you setting your tablet down for half an hour with the game paused, then coming back to find it has drained half your battery unnecessarily. Or you switch out to check your email, get distracted, and find that the running game has drained half your battery unnecessarily.

Android has an Activity lifecycle for a reason, and these games do not respect that. Users should not be worrying themselves with managing resources. If your game gets shoved out of the foreground, it should have the good graces to save its state and allow the device to sleep, like _every other app on the system_.

<comment><USER10> > This isn't about the complexity of the game or how much battery it drains > while playing. This is about you setting your tablet down for half an hour > with the game paused, then coming back to find it has drained half your > battery unnecessarily. Or you switch out to check your email, get > distracted, and find that the running game has drained half your battery > unnecessarily.

If _that_ is the issue, there is _no indication whatsoever_ of it in the list. What it says is "When running, this app keeps the device from going into sleep mode." Not "this application keeps running in the background". If these applications keep running in the background and you have evidence of that I'd be happy to note it and their misbehavior, but the page _you_ linked is — as I noted — completely worthless. Because that's not what it says, whether it's due to terrible copywriting or because that's not the issue.

> If your game gets shoved out of the foreground, it should have the good > graces to save its state and allow the device to sleep, like every other app > on the system.

Sure, if that's what happens I'd be happy to concede the point, but that is not the way I read the page, considering the way these blurbs are written I interpret them as "we opened the application in the foreground and put down the device". Especially given they _specifically note_ when applications misbehave in the background, in both cases of network misbehaviors.

<comment><USER9> Play GTA3 on a Nexus 7 sometime. Switch out to another task (or hit the Home button). Let it sit for half an hour, then check Settings -> Battery. You'll find that GTA3 is at the top of the list and is pegging the CPU.

Their workaround for not obeying the Activity lifecycle is to display a persistent notification that the game is running. This is is not a solution for their actual problem, which is that they have no way to serialize the game's running state, and thus will lose any unsaved game data if they do obey the lifecycle and let their app get culled. This is evidenced by the need to go to the ingame save points in order to save your game, like on a console.

Even if the game is open in the foreground, there is no reason why it can't give up its wakelock after a period of inactivity. Obviously this is a lesser issue as you have described, but it is still contrary to the UX of most Android apps.

</comment></comment></comment></comment></comment><comment><USER11> Impose Draconian data caps and try to con your users into leaving their unlimited data plans. Then, call apps out for using too much data.

Thanks, Verizon.

<comment><USER9> Is it not their prerogative to ensure a good quality of network service and user experience? These apps harm everyone who uses the service _unnecessarily_.

Is Google also being hypocritical by encouraging users to use networks for Search et al, at the same time encouraging developers to optimize for network usage and battery life? [1]

[1] [http://developer.android.com/training/efficient- downloads/in...](http://developer.android.com/training/efficient- downloads/index.html)</comment></comment>


PfSense 2.3-Release Now Available - <AUTHOR> https://blog.pfsense.org/?p=2008 <comment><USER2> With the crappy, underpowered consumer routers with gaping security holes pfSense is a god send. Last few years I've been running pfSense and the experience has been fairly great.

I had a backup server running Ubuntu with ZFS which was always on anyways so I just had to install pfSense on OS disk and it imported the ZFS disks right away. So for a long time it was both my backup server and the router/firewall.

But lately with all the upgrades (including 2.3 betas and RCs I ran) SAMBA throughput was reduced significantly - not entirely sure if it were the drivers, ZFS or something else. And I realized maybe it is not such a great practice to make your router/firewall double up as a backup server. So I virtualized the box using ESXi and ran pfSense and Debian in two VMs on it.

The box being weak and running virtualized along side another backup server VM exposed the performance issues with pfSense - I had given a single CPU and 512Mb to it and with the GUI and other process overheads - DNS cache, pingers etc. meant that I had noticeable slowdowns when both VMs were active.

I am giving VyOS a try - it's a debian based router distro and purely command line based but resource utilization wise it seems to be doing fine with 512Mb and single CPU.

Anyone having a low power dedicated box for pfSense should just use pfSense though - the UI is way too good in 2.3 release and stuff just works - IPV6, Tunneling, VPN are very easy to configure along with a bunch of other things limited only by the box you're running it on.

<comment><USER3> I recognized your reply from /r/pfsense. Dude, you are running unsupported scenario and using your pfSense as NAS. That is a big no no

<comment><USER4> If you're a BSD fan, why not use FreeNAS + pfSense on separate VMs? Let them both do what they're good at :)

<comment><USER5> I put Bhyve support in pfSense so people could do this.

</comment></comment></comment></comment><comment><USER6> Not that a firewall needs to look pretty, and yes, I know it's Bootstrap, but the new UI is so much more pleasant to look at! Worth it for this alone.

<comment><USER5> And even then, the largest benefit of all that new GUI code is not the look, but rather the regularity it brought to the source code for same, after over 12 years of organic growth of same.

</comment></comment><comment><USER7> I'd really love to see an API or a unified configuration system. At $WORK we have a need for many small firewall appliances, but I refuse to use anything we can't deploy zero-touch, automate, monitor and back up/restore in a semi- sane way.

Either it needs a configuration language like Cisco/Juniper than can be dumped/imported/configured over SSH, or a web API (supposedly on the roadmap for pfsense 3.0 ... ?). I can't say I'm too familiar with the project, but it feels like it's just not a priority and that it's is going to remain very web UI focussed.

<comment><USER8> pfsense was forked a year or two ago, their 2.3 is just to keep up with the new kid on the block opnsense. And guess what, they have an API ;) [https://opnsense.org/](https://opnsense.org/)

<comment><USER6> I don't really see what OPNsense has done too much differently, other than commence re-writing the UI in a different PHP framework and Bootstrap it.

To look at it, you're largely looking at pfSense.

I couldn't find a clear map of what has been changed or added to.

<comment><USER5> One of the team members at OPNsense "liberated" the source to his former employer's fork of pfSense (they were using it for their product.)

That's how they got to where they are. The pace of innovation in that project has slowed since its launch.

</comment></comment></comment></comment><comment><USER9> Such a wonderful product! Thank you everyone that works on the code!!! I'm finally able to buy them from the store, feels good to give back after all these years.

</comment><comment><USER10> Hmm, it looks like pfSense doesn't support the one requirement I have of any router: per-IP (not _interface_) statistics. Is this really such an unusual thing to want? I want to know how much traffic each device on my network is producing, in each direction. Ideally also: for suspect devices, I'd like to know _who_ they are talking to.

<comment><USER11> Sounds like you're looking for the darkstat package, which is available in pfSense.

<comment><USER10> Awesome. I got a Qotom Q190PG4 and am up an running.

</comment></comment></comment><comment><USER12> I just want to say I absolutely love pfSense! After going through so many routers I decided to just build my own... For the same cost of a high end consumer router I built my own 802.11AC Gigabit router that can handle anything without crashing every day!

</comment><comment><USER13> Yeah, old design was feeling dated when I had to make couple changes from my phone.

Only wish list I have -- allow ZFS on root with zraid. Hope, it's possible in the future.

<comment><USER5> Very near future. We're changing the installer for 2.4 (based on 11). Should be available later this year.

</comment></comment><comment><USER14> Great work. I have been using pfsense for 5.5 years now and it never disappoints.

</comment><comment><USER15> In my last round of my linux firewall comparison I liked opnsense very much:

[https://opnsense.org/](https://opnsense.org/)

not affiliated, I just liked it. However the GUI is made of many php scripts (if they did not change meanwhile) - so it would be great if many more eyes would like to take a look at the code...

<comment><USER6> They took those PHP scripts and rewrote / are rewriting them in a different PHP framework, for better, or worse.</comment></comment>


'Move Your Domain' Campaign Against GoDaddy Flops - <AUTHOR> http://www.pcworld.com/article/247134/move_your_domain_campaign_against_godaddy_flops.html

<comment><USER2> Those numbers refer to DNS changes, not registrar changes.</comment>


Announcing the NYC Startup Job Fair - <AUTHOR> http://www.metamorphblog.com/2010/03/announcing-the-nyc-startup-job-fair.html

<comment><USER2> "The NYC Startup Job Fair will bring students, _young_ alumni and startups together to help place bright motivated individuals in exciting New York based companies."

Emphasis on "young".

</comment><comment><USER3> Really looking forward to this.

</comment><comment><AUTHOR> this is a big deal and start of what I hope will be a trend!</comment>