Non-volatile Storage: CPUs no longer more performant than I/O devices - <AUTHOR> https://queue.acm.org/detail.cfm?id=2874238 <comment><USER2> I've made that point before on YC.[1] We need to view fast storage as something other than a disk accessed through the OS, and other than slow RAM accessed as raw memory. Access through the OS is too slow, and access as raw memory is too risky. What's probably needed is something like a GPU sitting between the CPU and the fast persistent storage. Call this an SPU, or "storage processing unit."

What would such a device do? Manage indices, do data transformations, and protect data. Database-type indices would be maintained by the SPU, so applications couldn't mess up the database structure. The SPU would manage locking, so that many non-conflicting requests could be serviced simultaneously. The SPU would have tools for doing searches. Regular expression hardware (this exists) would be useful. Record protection management (app can read/write part but not all of a record) would allow implementation of database type data access rules. Encryption and compression might be provided in the SPU.

There have been smart disk controllers before, but they haven't been that useful, since they couldn't make the disk go any faster. Now, it's time to look at that layer again. Some of the technology can be borrowed from GPUs, but existing GPU architecture isn't quite right for the job. An SPU will be doing many unrelated tasks simultaneously. GPUs usually aren't used that way.

[1] [https://news.ycombinator.com/item?id=9964319](https://news.ycombinator.com/item?id=9964319)

<comment><USER3> Being up front: this is what I work on for IBM Systems. A buddy wrote this blog ([https://www.ibm.com/developerworks/community/blogs/fe313521-...](https://www.ibm.com/developerworks/community/blogs/fe313521-2e95-46f2-817d-44a4f27eba32/entry/power8_capi_flash_in_memory_expansion_to_speed_data_access?lang=en)) with a little more info.

What we have is an IO offload accelerator that knows how to drive high bandwidth IOs to some external storage device. A user app doesn't interact with the device - they make shared library calls to read or write data from a particular buffer, and the accelerator (because it's cache coherent) can read / write from the virtual address space of the user space program to satisfy the request as needed. This means that the IOs bypass the entire OS driver stack, since everything is a shared library call from user space.

So yep! That exists. :-) There's other classes of accelerators out there too (and coming in the future as well). Adding additional function like compression or some form of indexing or search is stuff that we've talked about.

(edit) - [https://github.com/open-power/capiflash](https://github.com/open- power/capiflash) has the code for the shared libs, the APIs, and some examples.

<comment><AUTHOR> Interesting!

Incidentally (since this may be somewhat related), I'm wondering, what are your thoughts on the Persistent Memory Manager approach, as in the following:

Justin Meza, Yixin Luo, Samira Khan, Jishen Zhao, Yuan Xie, and Onur Mutlu: "A Case for Efficient Hardware/Software Cooperative Management of Storage and Memory." Workshop on Energy-Efficient Design, 2013.

Context: "emerging high-performance NVM technologies enable a renewed focus on the unification of storage and memory: a hardware-accelerated single-level store, or persistent memory, which exposes a large, persistent virtual address space supported by hardware-accelerated management of heterogeneous storage and memory devices. The implications of such an interface for system efficiency are immense: A persistent memory can provide a unified load/store- like interface to access all data in a system without the overhead of software-managed metadata storage and retrieval and with hardware-assisted data persistence guarantees."

The stated goals/benefits include eliminating operating system calls for file operations, eliminating file system operations, and efficient data mapping.

Paper: [http://justinmeza.com/bin/meza_weed13.pdf](http://justinmeza.com/bin/meza_weed13.pdf)

Presentation: [https://users.ece.cmu.edu/~omutlu/pub/mutlu_weed13_talk.pdf](https://users.ece.cmu.edu/~omutlu/pub/mutlu_weed13_talk.pdf)

<comment><USER2> One giant flat address space is not the answer. Hardware people tend to come up with approaches like that because flat address spaces and caching are well understood hardware. It's the same thinking that leads to "storing into device registers" as an approach to I/O control, even when the interface is really packets over a serial cable as in FireWire or USB or PCI Express.

File systems and databases are useful abstractions, from an ease of use, security, and robustness perspective. The challenge is to make them go faster. Pushing the machinery behind them out to special-purpose hardware can do that.

The straightforward thing to do first is to to take some FPGA part and use it to implement a large key/value store using non-volatile solid state memory. That's been done at Stanford[1], Berkeley[2], and MIT[3], and was suggested on YC about six years ago.[4] One could go further, and implement more of an SQL database back end. It's an interesting data structure problem; the optimal data structures are different when you don't have to wait for disk rotation, but do need persistence and reliability.

[1] [http://csl.stanford.edu/~christos/publications/2014.hwkvs.nv...](http://csl.stanford.edu/~christos/publications/2014.hwkvs.nvmw.slides.pdf) [2] [https://www.cs.berkeley.edu/~kubitron/courses/cs262a-F14/pro...](https://www.cs.berkeley.edu/~kubitron/courses/cs262a-F14/projects/reports/project13_report.pdf) [3] [https://dspace.mit.edu/handle/1721.1/91829](https://dspace.mit.edu/handle/1721.1/91829) [4] [https://news.ycombinator.com/item?id=1628550](https://news.ycombinator.com/item?id=1628550)

<comment><USER4> OK I find it easier to follow these ideas when thinking about how loads/stores to volatile memory are organized. Memory is not accessed via a syscall. Instead the OS sets up some data structures in the MMU and lets the application run. Some kind of fault happens when control must be transferred back to the OS.

Going back to non-volatile memory the question is what kind of abstraction should be implemented in hardware? Presumably something simple that the OS and applications can then use to implement higher level abstractions like file systems and databases. Pushing parts of a SQL database engine into the hardware does not intuitively seem like a right solution.

</comment></comment></comment></comment></comment><comment><USER5> This is so true, the world has changed greatly and not everyone has gotten the memo. I saw a really cool device made by Texas Memory systems which was a "ram disk" that was all ram with disk backing, and when you lost power it flushed to disk. I wanted something that worked better for a storage paradigm and designed/invented a network accessible memory appliance[1]. Basically using ethernet packets you could store 8K integrity protected chunks right there on the network. Initially I wanted to use a typical low power CPU with a bunch of DRAM attached but the CPU bottleneck got in the way, so we redesigned/rebuilt it out of FPGAs so that it had a couple of terabytes of RAID protected RAM in an appliance with a very simple network protocol for storing and fetching 8K blocks out of what was essentially a linear address space. Two of these on different power subsystems provided all of the fault tolerance you needed and you could have a terabyte of 'structured' data live from the moment your computer booted (made for very fast recovery from reboot).

[1] [https://www.google.com/patents/US8316074](https://www.google.com/patents/US8316074)

<comment><USER6> That is fascinating. Could this reliably reduce the hardware footprint on any device?

<comment><USER5> If I understand the question, then yes. When you consider the amount of cache memory in clustered systems which is all holding the same stuff in every independent machine. Using it simply as a victim cache for a block storage device penciled out to a pretty significant improvement.

It gets even better with 64 bit address spaces and a bit of kernel code to 'fault in' from the device.

</comment></comment></comment><comment><USER7> My sense is this is only true today because OS kernels are ridiculously slow relative to what the hardware can achieve.

Most of my recent designs treat RAM as if it were (what we used to considered to be) disks, i.e. all computation and in-process data is in cache exclusively, and "going to RAM" requires the use of a B-tree-like structure to amortize the cost.

For example, once you've opened a RAM page line on a normal four-channel Xeon server, you can read the entire 4KB page in about the same time it takes to read one byte, switch pages, and then read another byte. (Of course, you can't do that either since the entire cache line will be filled, but the overall point still stands.)

The situation we're in today with RAM is pretty much the identical situation with the disks of yore. Anyway…interesting article nonetheless.

<comment><USER8> Not _that_ is some interesting thinking. You got a blog post elaborating on this?

<comment><USER9> Not sure exactly what OP is referring to, but CSS-trees [1] are a classic example of cache-aware indexing structures that fetch entire pages into cache and arrange data so that most of the comparisons happen on cached data. In most cases, they significantly outperform binary trees. Masstree [2] is a more recent example of this.

[1] [http://www.vldb.org/conf/1999/P7.pdf](http://www.vldb.org/conf/1999/P7.pdf)

[2] [https://pdos.csail.mit.edu/papers/masstree:eurosys12.pdf](https://pdos.csail.mit.edu/papers/masstree:eurosys12.pdf)

</comment></comment></comment><comment><USER10> Yeah per packet processing at 40Gbps and higher is problematic on regular kernels, OS stack and CPUs. A lot of really can be cache hits -- hundreds of nanoseconds. Article mentions that too:

\--- To put these numbers in context, acquiring a single uncontested lock on today's systems takes approximately 20ns, while a non-blocking cache invalidation can cost up to 100ns, only 25x less than an I/O operation. \---

It also depends if workload is throughput sensitive or latency sensitive. If it is latency, can do things like tie processes and interrupts to cores, isolate those cores, etc. For throughput can processes more than one packet at a time perhaps.

Then there is dpdk and even unikernels.

> CPU has responsibilities beyond simply servicing a device—at the very least, > it must process a request and act as either a source or a sink for the data > linked to it. In the case of data parallel frameworks such as Hadoop and > Spark,7,17 the CPU

That's why you get more CPUs and explicitly isolate them if you can. But now depending on how they share data with other CPUs there will be invalidated cache lines so will pay that way as well.

In general if you run on RHEL / CentOS ( a lot banks, military and enterprise deployments do ), there is this helpful guide as an overview:

[https://access.redhat.com/documentation/en- US/Red_Hat_Enterp...](https://access.redhat.com/documentation/en- US/Red_Hat_Enterprise_Linux/7/html/Performance_Tuning_Guide/sect- Red_Hat_Enterprise_Linux-Performance_Tuning_Guide-CPU- Configuration_suggestions.html)

<comment><USER11> Cache miss latency was the first thing that popped into my mind as well when I saw the title.

It seems like they don't make a clear distinction between latency and bandwidth. From the little I know on SSDs(don't claim to be an expert here) the sequential reads are below or on-par with high spindle speed disks.

What seems to be a better take-away would be that sequencing of your reads isn't nearly as important as it used to be. Back in games we'd duplicate data across a DVD so that we could do "seekfree" loading where duplicating 5-10MB of data would mean just a single big call to read() and gain massive load time performance.

<comment><USER12> SSDs are still faster than hard disks even with sequential reads. 15,000 rpm enterprise spinning disks read at about 260 MB/s [1] while NVMe SSDs (like those in a 2015 MacBook Pro) reads at >1300 MB/s [2].

[1]: [http://www.tomshardware.com/charts/enterprise-hdd- charts/-02...](http://www.tomshardware.com/charts/enterprise-hdd- charts/-02-Read-Throughput-Maximum-h2benchw-3.16,3372.html)

[2]: [http://www.computerworld.com/article/2900330/apple- mac/holy-...](http://www.computerworld.com/article/2900330/apple-mac/holy- smoke-the-new-macbook-literally-is-twice-as-fast.html)

<comment><USER10> > NVMe SSDs (like those in a 2015 MacBook Pro) reads at >1300 MB/s [2].

No joke. Just got an new MBP for work. Before had a spinning disk (well hybrid). Was running some silly benchmarks that I ran before and clocked my disk throughput at about 100MB/s. On MBP got 800MB/s. I thought something broke (was hitting page cache or some trickery like that) or didn't compile things right. But no, tried other tools, looked online and it seemed correct. It really surprised me.

</comment></comment></comment></comment><comment><USER13> I like the article, but not your title. It implies that this trend of I/O becoming highly performant has occurred recently, when in fact has been observed and studied for quite some time [0, 1]. Even before SSDs Gigabit ethernet was saturating CPUs needing to do more than DMA a packet, and I'm sure this trend continued for some time. The original title seems more accurate: "Implications of the Datacenter's Shifting Center", and references the existing trends in an insightful article.

[0] [http://ucsdnews.ucsd.edu/archive/newsrel/supercomputer/11-09...](http://ucsdnews.ucsd.edu/archive/newsrel/supercomputer/11-09Gordon.asp)

[1] [http://nvsl.ucsd.edu/index.php?path=pubs](http://nvsl.ucsd.edu/index.php?path=pubs)

</comment><comment><USER14> So NVDIMM's... Is any one actually making those except Viking and is anyone actually supporting them in servers except SuperMicro?

These are basically DDR3/DDR4 DIMM's with onboard flash and a supercap/battery pack to provide persistence incase of system reboots and power failures.

They are also a bit odd as they would ignore various system event calls from the BIOS/UEFI and then have to be specifically managed by various software hacks that create RAM drives and access the memory directly rather than working with OS virtual memory. Since NVDIMM's are basically treated as system memory by both the server and the OS they pretty much only work for very very boutique applications I'ts a bit odd that these are presented as the next step in storage evolution while being effectively an overpriced hack. I've only seen them actually been use in weird server setups like the overclocked watercooled servers that are used for HFT where they strip everything even the OS as possible and bypass anything that adds even a few NS of latency and don't mind running their own code for everything from a bastardized TCP stack that isn't even remotely compliant but works to their own in memory custom database.

<comment><USER15> Intel will be pushing them hard starting with Skylake-EP, so some people are getting themselves ready.

<comment><USER14> Did Intel created a new interface for NVDIMM's? because to work with the ones Viking makes you pretty much need to hack your linux kernel to ensure that it doesn't access physical memory over a certain address range, and I don't even know if or how can you use them on Windows based applications.

<comment><USER15> Yes, Intel basically controls UEFI/ACPI and has been submitting patches to Linux for a while. [http://pmem.io/](http://pmem.io/)

</comment></comment></comment></comment><comment><USER16> This trend has been clear for awhile. Interestingly, this will put performance pressure back on programming and languages as they become the "new" bottleneck.

I'd expect an implicit migration away from slower languages toward faster ones.

<comment><USER17> To some extent, yes, but 'slow' languages usually delegate batch work over large datasets to optimized libraries.

Even more likely, as I see it, is this contributing to the increasing rise of tools like spark, hadoop, etc. Slow languages will continue to be popular as orchestration around these tools.

</comment></comment><comment><USER18> Both Tandem DP2 and IBM Coupling Facility on zSeries Sysplexes worked exactly like how you envisage an SPU would work. Therefore, when we developed RDMA- attached persistent memory at Tandem in 2002, we put it under the control of DP2/ADP process pairs. Later, we ported it to HP-UX and InfiniBand RDMA. There is one paper at IPDPS'04 and several published patents you can look up in my Google Scholar page. pmem.io crowd is reinventing some of this wheel. If any of you work at HPE, you can find much more detailed internal papers, source code, drivers, firmware, and other stuff that the outside world cannot get to.

</comment><comment><USER19> > and the performance of an SCM (hundreds of thousands of I/O operations per > second) is such that one or more entire many-core CPUs are required to > saturate it.

So, we are getting a lot of data, but latency is still killing. (Even more taking into account that this thing has a few stages of pipeline inside.)

Anyway, our CPU is getting distributed nearer IO and memory. We are going to get NUMA machines, everything points at it.

<comment><USER20> Knights Landing from Intel is already NUMA (I think). I'm not sure if it can be bought yet, but it should be very close to release.

<comment><USER8> Aren't AMD parts already (cache-coherent) NUMA?

<comment><USER19> Well, a bit, but not radical enough to be visible to its software.

</comment></comment></comment></comment><comment><USER21> Please excuse my ignorance on this matter but will this technology have any impact on the hierarchy levels below disk (i.e. RAM and CPU caches)? Compared to Register, L1 and L2 access RAM access is still really slow. Will non- volatile storage latencies rival or exceed those of standard RAM? From how I understand the article it's primarily disk IO speed that's affected, correct?

<comment><USER9> Yes. Even the persistent memories that attach to the memory bus are currently quite a bit slower than DRAM (5-7x from estimates I've seen), while the difference with PCIe-attached ones is even more.

I'm not sure what the future holds in terms of latencies for non-volatile storage but sub-DRAM levels aren't within reach yet.

<comment><AUTHOR> On a side note, it's interesting to me that emerging memory technologies currently seem to be mainly focused on addressing the "from-DRAM-to-disk" part of the memory hierarchy.

That is, as you mentioned, not directly competing with DRAM, and consistently on the same side of the 1 microsecond dividing line between memory and storage; as in:

[http://www.rambusblog.com/2015/10/15/mid-when-memory-and- sto...](http://www.rambusblog.com/2015/10/15/mid-when-memory-and-storage- converge/) (note SCM placed between DRAM and SSD)

[http://semiengineering.com/the-memory-and-storage- hierarchy/](http://semiengineering.com/the-memory-and-storage-hierarchy/)

As far as the other side of the line is concerned, I think I've only seen proposals for hybrid-cache architectures (HCA) -- other than [http://link.springer.com/chapter/10.1007%2F978-1-4419-9551-3...](http://link.springer.com/chapter/10.1007%2F978-1-4419-9551-3_7) \-- with a hybrid approach (e.g., combining SRAM/eDRAM/STT-RAM/PCRAM) probably making sense due to latency/endurance/bandwidth trade-offs.

If anything, there seems to be more development on the DRAM interface itself -- with multiple candidates for the (or a) DDR4's successor, so far involving Wide I/O (Samsung), Hybrid Memory Cube (Intel, Micron), High Bandwidth Memory (SK Hynix, AMD, Nvidia): [http://www.extremetech.com/computing/197720-beyond- ddr4-unde...](http://www.extremetech.com/computing/197720-beyond- ddr4-understand-the-differences-between-wide-io-hbm-and-hybrid-memory-cube)

(Latency and bandwidth improvements seem promising, [http://semiengineering.com/which-memory-type-should-you- use/](http://semiengineering.com/which-memory-type-should-you-use/))

One interesting development I've seen involves reducing SRAM's footprint, by moving from 6T (6-transistors) cell to a 1T (one-transistor) one: [http://www.eetimes.com/document.asp?doc_id=1328453](http://www.eetimes.com/document.asp?doc_id=1328453)

It's fairly recent development, though, and it remains to be seen how is it going to fare.

Other than the above, there doesn't really seem to be much progress around competing with/improving SRAM. However, this may become increasingly important, since some of the technological process scaling issues apply to SRAM, too.

</comment></comment></comment><comment><USER22> For anyone else who was momentarily confused: on figure 2, the y-axis scale is incorrectly labeled "ns" when it should be "ms".

<comment><USER23> Anyone old enough will remember the time hard drivers were correctly marketed with access time in milliseconds as the main speed indicator. This ended around 1994( _) when pretty much all the drives reached ~10ms access time.

[https://en.wikipedia.org/wiki/Hard_disk_drive_performance_ch...](https://en.wikipedia.org/wiki/Hard_disk_drive_performance_characteristics)

_I did a quick scan of old computer magazines (infoworld, pc mag etc).

</comment></comment><comment><USER24> HP was designed a "flat memory" OS based on vast amounts of cheap memrister memory [http://www.technologyreview.com/featuredstory/536786/machine...](http://www.technologyreview.com/featuredstory/536786/machine- dreams/). But when I googled gfor this article I saw the project has been delayed.

<comment><USER25> I'm very interested in working on a "flat memory" OS that doesn't use any RAM or file system but simply registers and a distributed database of key/value stores.

If you're interested in talking about this more (especially if you're in the SF Bay Area), my email is in my profile.

</comment></comment><comment><USER26> I'm seeing this on a new Postgres database server: my cache miss and cache hit queries take virtually the same time!

This is with a Skylake Xeon E3-1275, 64GB ECC UDIMM, and Intel 750 PCIe SSD (probably the fastest setup you can get).

It looks like I have to figure out something that tunes Postgres to account for the fact that disk lookups are no-cost.

<comment><USER27> > I'm seeing this on a new Postgres database server: my cache miss and cache > hit queries take virtually the same time!

That's likely because it's hitting the OS's page cache.

</comment></comment><comment><USER28> Serious but naive question: does a bottleneck curve trending toward CPU subsystems suggest micro kernel based approaches replacing spinning up virtual machines as a future trend due to the possibility of reduced overhead at the CPU?

tldr; Does increasing use of Storage Class Memory imply increasing use of microkernals?

</comment><comment><USER29> The numbers in this are daunting, but I personally believe massively multi- core systems make the problem a lot less daunting than the article makes out. Core counts in big servers can get up over 100 per server for Intel (see Amazon's new EC2 offerings for public evidence of this). Intel's Xeon Phi series of processors offer core counts approaching ~300. Going to 300x takes the required latencies per request from microseconds up to the millisecond range. POWER systems can go even higher. Moreover, for many workloads that actually leverage this sort of compute you can do something horrifying with the new DRAM-addressable persistent storage: DMA directly from the NIC into block storage. Some (many?) high performance network adapters offer the ability to filter packets to distinct Rx queues; buffers can be posted with addresses in the storage mapped region allowing direct NIC->storage transfer. If you bake more intelligence into the NIC, you can even do things like Mellanox's NVMe Fabrics:

[http://www.mellanox.com/blog/2015/04/mangstor-mellanox- show-...](http://www.mellanox.com/blog/2015/04/mangstor-mellanox-show-nvme- over-fabrics-solution-to-reduce-latency-tax/)

This is particularly relevant to the JBOD example.

Now, there's the question of what you're actually going to _do_ with all of that data, but in a lot of cases it's likely a durable read-mostly cache that's effectively a materialized view optimized of some (hopefully much slower write-rate) transactional store (say, product data on Amazon -- detail pages served up at some absurdly high rate, but a relatively low mutation rate).

Other workloads I can think of fall into a category I tend to think of as log processing -- a high-rate series of streaming writes which are slurped up and batch processed/reconciled to some (much smaller) state (which of course may then be exploded back out to large materialized views as above). In these scenarios, presuming the log entries have low contention over the underlying state, CPUs like those I called out above are more than up to the task of streaming over the input and optimistically updating the backing state.

Finally, in terms of real workloads, there is almost always going to be a bottleneck limiting your ability to fully utilize your resources. Either you're CPU bound and leaving network bandwidth on the table or you're network bound and are leaving CPUs/storage devices under-utilized. Massively improved storage performance local to a node is fantastic in terms of computation you can do locally, but if each network fabric upgrade costs you 10x what the previous one did to keep up with the storage/CPU available per-node, you're going to have a bad time. Amin Vahdat talked a bit about our (Google's) historical network fabric evolution: [https://www.youtube.com/watch?v=FaAZAII2x0w](https://www.youtube.com/watch?v=FaAZAII2x0w)

If I were betting on an annoying bottleneck to full resource utilization coming up in the near future, I'd put my money on network before CPU :)

</comment><comment><USER30> Increasing industry trend to expose CPU / GPU / SPU / NPU directly to applications for more efficient data handling.

</comment><comment><USER31> Performant: still not a word.

<comment><USER32> Seymour Cray never said "performant". Engineers say "fast" or "fast enough". Marketing types and nontechnical management seem to prefer this neologism. But it might also be a generational thing.

A new coinage that I noticed in the past year that also grates on my ears: "learning" as a substitute for "lesson", as in "what were your learnings from the hackathon?" Anyone else caught this one?

<comment><USER33> Past year? Past decade champ.

<comment><USER34> Maybe in Microsoft-land or certain circles, but anecdotally I've only started hearing "learnings" this year too.</comment></comment></comment></comment>


The North Korean Tablet - <AUTHOR> http://techcrunch.com/2012/06/28/north-korea-is-allegedly-building-its-own-tablet-the-samjiyeon/

<comment><AUTHOR> With no camera or internet, it'll be hard to overthrow a government with, unlike those pesky mobile devices the rest of the world uses.</comment>


Filezilla download goes to Ask.com-signed installer. Use 'show all' page instead - <AUTHOR> https://filezilla-project.org/download.php?type=client

<comment><AUTHOR> Title is too short. What I mean is that the normal download button uses some installer that is not from Filezilla. I have no idea what it does, but I'm quite sure that I don't want to use a proprietary installer when I am trying to download binaries from open source software.

It seems SourceForge became a hostile place for software.

If you want the official binaries straight from Filezilla's website, go to "additional downloads" and download it there instead. This one matches the checksums that they publish, although the checksums are published via SourceForge as well and could easily be modified.</comment>


Don't Waste Your 20s at Google or McKinsey - <AUTHOR> https://www.linkedin.com/today/post/article/20140727232741-15381-don-t-waste-your-20s-at-google-or-mckinsey?trk=mp-details-rc <comment><USER2> I agree on some sentiments, but you can learn at big companies also. You will likely learn a lot more tech at a startup but it is unlikely to teach you how large corporates handle various things, such as team work, testing, user requirements. I don't always think they do things the best way, but it can be useful to see how it is done in such places.

</comment><comment><USER3> "The perks seem attractive (free food and occasional visits by Hillary Clinton or Bono)." \- does anyone really want to meet them?

I agree that working for a start-up is good, as long as you are a member of the founding team, like the author.</comment>


Human body-on-chip platform could speed up drug testing - <AUTHOR> https://www.europeanscientist.com/en/research/human-body-on-chip-platform-could-speed-up-drug-testing/ <comment><USER2> If you can really, truly get liver-on-a-chip to work, it's going to positively change drug testing forever and will be the first huge win for tissue engineering.

You can throw away animal models for tox screens forever, and that's great news. It will save vast amount of time and money. By using a much better proxy model for humans (slices aren't organs, they have different rheological characteristics and morphology, but they're a damn sight better than mice), you'll wash out candidates earlier and faster, and hopefully enrich the pipeline.

Slowly but surely, and it's good to see steps in this direction.

<comment><USER3> Meh. There are plenty of human cell line derived assays for things like membrane permeability, liver toxicity, etc.

Those assays are super helpful, but don't replicate what happens in humans. They are good boxes to check, and can help weed out compounds early, but just because a compound passes the assay doesn't guarantee something weird won't pop up in humans.

<comment><USER4> Is it possible that these proxies you mention could weed out compounds incorrectly? For example, if we have a liver proxy that fails on drug A, is it possible that drug A will work fine in the human body provided that there's a lot of other processes that could contribute to the liver?

<comment><USER3> Absolutely. That's the flip side of in vitro and animal testing. You might see a negative signal and stop development, only to find out that it can't be replicated in humans.

The cell line assays replicate much of what happens in humans, but not all of it, in particular the interactions between different tissues. For example, a drug might be toxic in a liver assay because a reactive metabolite damages the cells. But in an actual human, you might never see those high of concentrations in the liver or there may be a process (like glutathione adducts) that deactivates the metabolite and prevents cell damage.

</comment></comment></comment></comment><comment><USER5> We need to clone decephalized humans and livestock.

With thousands of brainless bodies kept alive on life support, you have test subjects for a limitless number of experiments that would have never been possible before. You also create a never-ending O negative blood supply and organ harvesting program.

It the case of decephalized animals, you also get cruelty free meat. And that's how you bootstrap the program and port it to the human model.

There would be a lot of political pressure, but this would be a space-age jump in supporting fundamental biological research, supplying renewable body parts and tissues, and keeping us healthy and young.

We need to do it.

If I ever get Elon Musk money and power, I'm doing this instead of rockets.

<comment><USER6> I've seen a lot here, but this takes the cake. I'm not sure if that post is a troll, assuming it is not:

Half of the energy and time of any decent person with morals goes into stopping smart psychopaths like that poster from doing whatever the heck they want.

I'm not religious, but it makes me wonder if the church is actually needed to keep those people in check.

<comment><USER5> > I'm not sure if that post is a troll

It's not. I've had this idea for some time, and I'm absolutely serious.

We already grow human cells in culture. It's like growing an extremely delicate plant. The HeLa cell line is derived from Henrietta Lacks, and she's been dead for some time. There's nothing wrong with growing and manipulating cells like this. Eukaryotic cells are as brainless as prokaryotic ones. This is how we do basic research.

We create multicellular tissue cultures, often using the extracellular matrix from cadavers. Again, basic research. Nothing wrong, nobody hurt.

If you can grow a human body without a head and brain, how is that ethically different from a plant? Please explain it to me. As long as you guarantee the brain never develops, it's ethically cleaner than telling a white lie.

Personhood requires a brain. Period.

I don't believe in a soul. So that's not an issue either.

Next explain to me why we shouldn't do this, because headless clones will cut organ transplant waiting lists to zero and help us cure cancer, diabetes, and every other human disease that doesn't directly involve the brain.

<comment><USER7> > Personhood requires a brain. Period.

Well, it requires a complex nervous system.

How many neurons are there outside of the brain?

How many neurons create a sensing, feeling being?

It's an assumption to say that the only way for consciousness around organic feeling to exist is through a brain. We don't know that.

<comment><USER8> I was under the impression that the nervous system outside the brain is only propagating signals back and forth, and all the actual compute is localized in the brain. If that's true, then the GP's idea makes sense; if it isn't, then that's a can of worms we need to sift through eventually.

</comment></comment></comment></comment></comment><comment><USER9> Does anyone have a layman-accessible explanation for how exactly this works? My last biology class was freshman year of college, but I find the "body-on- chip" startups incredibly interesting. I take it this is different than "growing" organs via stem cells?

<comment><USER10> Basically, when you are growing organs or organoids (or 3d printing organs) with stem cells (which are used for organ on a chip too), you're actually trying to recreate the physical organ itself in it's original structure/function. You can imagine in the extreme the end result of trying to do this with a lung would be actually having a working lung that you could transplant into a person.

Organ on a chip is less about making a lung, amd more about making a very controllable device that that has all the relevant behaviors and characteristics of a lung, so you can experiment with it. The end result of this might be a bunch of chips all connected to eachother with all sorts of tubes, and different cells grown in each tube with exact stimuli of all sorts being applied by an automated system. The goal wouldn't be to make a long you could put into a person. Rather, the goal would be to make something that, when you put a drug in through one of the tubes, would tell you exactly what would happen to the drug and how the lung of a real person would react if you were to give them the drug.

The lines can get blurred, organoids on a chip, 3d printed organs used for research purposes, etc. But in general, (and you'll see this in how the different companies market themselves) growing organs is trying to get functional organs that could eliminate the need for a donor for an organ transplant, and organ on a chip is trying to get perfect research tools to eliminate the need for mice in research.

<comment><USER11> This reminds me very much of manufacturing quality testing, where you do initial DoE and validation builds to develop knowledge of the process. Eventually the knowledge of procc behavior at boundary conditions is known well enough that you can start to use alternative process monitoring approaches, like building a representative mockup of in-vitro conditions that represents the actual use conditions.

It doesnt have to be the same vector, just the same eigenvector.

</comment></comment></comment><comment><USER12> So they're basically trying to simulate humans? What a world we live in

<comment><USER13> Step by step, we are becoming obsolete...

<comment><USER14> Actually for drug research that's a good thing

</comment></comment></comment><comment><USER15> I've heard nearly any implant you put inside body has negative impact and this is also true for titanium implants which is seemingly inert and harmless metal.

How many people their body with some industrial chip, I'll never use anything like this unless my life depends on it.

<comment><USER16> This isn't a chip for implanting inside the body; this is a microfluidic chip that replicates some body systems in the laboratory.

<comment><USER17> OK why the need for a physical chip, why not replicate in software?

<comment><USER18> Because we can't effectively model the behavior of human tissue in software.

<comment><USER19> We can't even effectively model basic chemistry. We can't a priori tell you what color a chemical will be, much less many of its other properties.</comment></comment></comment></comment></comment>


California sets Covid-19 daily record, Bay Area cumulative total passes 20k - <AUTHOR> https://www.eastbaytimes.com/2020/06/23/coronavirus-california-sets-another-daily-record-as-bay-area-surges-past-20000-cases/ <comment><USER2> "The number of deaths per day and hospitalized patients, however, has remained flat for weeks, down from their peaks in mid-April. That is beginning to change, as hospitalization levels creep back up. Gov. Gavin Newsom said on Monday that hospitalizations had increased 16% in the past two weeks."

I always thought, based on what I've read/heard, that one of the big reasons, maybe the biggest reason we're doing all this is to flatten the curb so hospitals don't get overwhelmed. For some reason headlines always seem to be number of cases or positive test results, and maybe then the number in the hospital is mentioned. The article does say "hospitalization and deaths are also believed to be a lagging indicator." so maybe that's why, but I don't get why (from what I've seen) hospitalizations don't seem to be all that important when reporting on COVID numbers in general. Could it be the people getting the virus now are less likely to end up in the hospital so that number won't keep going up to the point of causing trouble? Another headline on the site says "17+ students test positive for COVID-19 after beach trip".

<comment><USER3> Great observation. I see the same pattern in other countries, too. One reason for this I believe is the lack of "goal clarity". What's the goal we're aiming for? I see several options:

1) we want to eradicate the virus from our population => total cases matter and should be 0 sometime soon, I think that was and is China's goal

2) we don't try to eradicate the virus, but we want to keep the stress on the health system manageable => total cases don't matter that much, bed capacity is the relevant indicator. This seems to be the goal of the "flatten the curve" thinking

3) we don't care that much, let the virus spread like wildfire so we will have herd immunity asap => I am not sure, but from outside it seems like Brazil is trying to do this

I think the reason you and I are confused is because it's seems to be unclear if we are trying to achieve 1) or 2). It's not communicated. Maybe it's unclear to policy makers, too.

<comment><USER4> I think a lot of countries do have clarity.

The USA definitely does not. I'm starting to wonder if "not knowing what we want" will be the root cause for the end of American hegemony.

We entered 2 wars without clarity on what it would mean to win. This was the #1 post-mortem on Iraq and Afghanistan from basically every general involved over 15 years: they never had mission clarity.

On the back of that tough lesson, we immediately entered a trade war with zero clarity on what it would mean to win.

And before we could learn the lesson a third time we're now up against a pandemic and apparently can't decide what it means for us to win.

I'm starting to wonder if "not knowing what you want" is even worse than wanting something dumb.

<comment><USER5> If you don't know what you want, you can't be disappointed you didn't get it. If you don't know what you'll get, you won't be beholden to citizens for not getting it or getting it done. It may seem careless, but it's also convenient to avoid accountability by avoiding responsibility, and avoiding responsibility by avoiding acknowledgement or awareness of an issue.

<comment><USER6> this seems true. but it kind of implies that there is a _secret_ agenda. it would kinda be more comforting if there was, but its not really clear.

<comment><USER5> I didn't mean to imply such an agenda exists in America. Sovereign democracy in Russia is one example of what such a secret (or even explicitly public) agenda would look like.

[https://en.wikipedia.org/wiki/Sovereign_democracy](https://en.wikipedia.org/wiki/Sovereign_democracy)

[https://en.wikipedia.org/wiki/Vladislav_Surkov](https://en.wikipedia.org/wiki/Vladislav_Surkov)

[https://www.economist.com/europe/2013/05/11/an-ideologues- ex...](https://www.economist.com/europe/2013/05/11/an-ideologues-exit)

</comment></comment></comment></comment></comment></comment><comment><USER7> We locked down _too soon_ in many places. Hear me out.

In my state, Oregon, we locked down when there were only a handful of cases, and not much community spread. Most spread was either in long-term care or familial. But we locked down _hard_ and had pretty good compliance. I am talking, standing in the middle of I-5 at noon compliance. Sure you'd have to keep an eye out for trucks from time to time. But still.

We set up surge capacity. Hospitals in Portland set up triage tents and cleared out other beds for a surge...... that never came. A few weeks in, the tents were taken down, the staff was laid off. We sat in our cocoon waiting for something, not sure what. Eventually the pain of being closed started to tick away, and we started reopening...

But nothing materially _changed_ between then and now. We just stopped community spread of an infection nobody really had. We've had less than 6k infections in Oregon. Now we are opening up and guess what? Cases are increasing! We didn't flatten the curve we crushed the curve. So there was no controlled spread, it was just us all feeling proud of ourselves for three months while the poor suffered.

The sneaky lie was that, in a country as big as the US, a flatten the curve strategy was never going to work at a national level. We need community level response.

We locked down Oregon over what was happening in NYC. That is akin to Northern Ireland locking down because of Moscow. That isn't an exaggeration. If anything, it is an understatement.

All of the European countries that we are saying "why can't we be more like them?" all had orders of magnitude more cases than Oregon. So if Oregon quadruples in cases over the next month we are still better off than a lot of nations.

To flatten the curve, there needs to be a damn curve to flatten. Had we waited until there was known, uncontrolled community spread and _then_ shut down, we would have a lot more political will. Now if it gets to that point after three months of being shut down, I doubt anyone will give a shit what the Governor says. Goodwill has been burned through and then some.

<comment><USER8> > We locked down _too soon_ in many places

Or maybe the real answer was a coordinated national response.

Why wasn't the NYC cluster confined to NYC or NY state? If the initial lockdown succeeded in killing community transmission in Oregon, how did it start growing again? Most likely from external sources.

Only a national strategy could have made these individual state-level lockdowns worthwhile. Since we didn't have that, the states implemented a haphazard, patchwork of lockdowns that weren't coordinated.

> I doubt anyone will give a shit what the Governor says. Goodwill has been > burned through and then some.

Unfortunately you're right about that.

<comment><USER9> you cannot eradicate this disease unless you turn the nation into China.

The viable model is the Swedish model, schools are open, restaurants are open, nobody wears a mask. What they do is voluntarily keep more distance between people. That's it. that will do it.

The sky has not fallen, they are exactly on the same track as the US.

<comment><USER10> This is not true. New Zealand has done an exemplary job. Taiwan has done a fine job. Hong Kong? Japan? South Korea? All democracies. All competent.

Sweden's seen way more deaths than its neighbors, and being on the same track as the US should be an embarrassment here.

Also, no, that will not do it. That will see a huge fraction of the population infected. Adding masks would help a lot, though.

<comment><USER7> NZ is an island with two big cities, doesn't count. Taiwan is also an island. So is Japan. HK is one technically, I guess. But not really. South Korea is basically an island, since they have a completely closed border with their only land neighbor.

Maybe the ability to control people coming in and out is more of a commonality than their responses? That is considering their responses looked _nothing alike_. NZ and Japan are basically as opposite as you can get in regards to a response.

<comment><USER11> Vietnam has 90M people, a massive land border and only 350 cases (no community spread for 6 weeks now). They are testing like crazy, so it's not because they aren't catching them.

How did they do it? All borders closed. Quarantine at home? Well, we'll post someone to keep an eye on you. Not following the rules? We have a quarantine camp that will work better.

Someone infected on the same floor of your apartment building? Everyone is locked down for 24 hrs (can't leave your unit). Floor is disinfected and everyone tested.

Vietnam just basically went full throttle using every public health measure they could.

<comment><USER12> This shows the power of an authoritarian system. Unfortunately we've seen other examples of authoritarian systems killing millions more than this disease ever would.

</comment></comment></comment></comment></comment></comment></comment><comment><USER13> It is spreading fast amongst the Latino population and more slowly amongst other populations. In SF, Latinos are over 50% of the cases now. In San Mateo, it is closing in on 50% of the cases as well.

A couple of weeks ago, the number of Latino cases in San Mateo was about the same as the Asian and white groups. It is now over triple.

[https://data.sfgov.org/stories/s/fjki-2fab](https://data.sfgov.org/stories/s/fjki-2fab)

[https://www.smchealth.org/san-mateo-county-covid-19-and- othe...](https://www.smchealth.org/san-mateo-county-covid-19-and-other-health- data)

<comment><USER14> why is the latino population seeing large number of cases?

<comment><USER15> I would venture, insular communities and bad luck. Maybe also something to do with cultural mores around visiting family when sick, or insufficient "how to mask/distance/etc" materials in Spanish.

I think it's like Gay-Related Immune Deficiency (GRID, before it was AIDS). There's no inherent reason that it showed up predominantly in the gay male population, except a) the first superspreader was a gay airline steward (so he had lots of contacts with the gay community all over the country), and b) gay men were enjoying the fruits of their sexual revolution.

If you think about it, monogamy is basically an extreme form of "sexual distancing", and so a non-monogamous community is much more likely to have an outbreak of an STD. Other non-monogamous communities heavily promote wearing masks (condoms), but as this had been viewed primarily as a way to prevent pregnancy, the gay community did not think it was necessary.

<comment><USER16> I think it's just because hispanic people are poorer and thus more likely to be essential workers.

</comment></comment></comment></comment><comment><USER17> The Johns Hopkins database implies that California's overall positivity rate (i.e. the percent of COVID-19 tests of Californians that are positive) remains roughly flat at just under 5%. [1] While that's still a concerning number (and it seems like LA in particular is not doing as well as the rest of CA), California overall is doing reasonably well, as it has been throughout the outbreak, despite our dense cities and massive population. More tests mean more positive results, but doesn't necessarily indicate worsening conditions. Cumulative totals are especially meaningless; of course an area with a large population (e.g. the Bay Area) will have large cumulative totals.

The focus on absolute numbers of positive cases (without normalizing by testing rate or population) pretty dramatically distorts the national discourse on COVID-19. California has been slow to reopen — although it is reopening — and has been criticized by the right as experiencing surging cases regardless. But that's largely inaccurate: CA is just much, much larger than any other state, and it's been reliably ramping up testing. As counterexamples, Florida and Arizona are places where the surge in cases is actually reflective of significantly worsening community spread: Arizona's positivity rate has tripled from a low of 6.5% in May to nearly 20% today [2], and Florida's positivity rate has grown 5x from a low of 2.3% in May to over 12% today [3]. Texas, another state that reopened quickly, had its positivity rate double from a low of 4.7% — approximately similar to the rate CA has managed to keep itself at — to 11% today. [4]

1: [https://coronavirus.jhu.edu/testing/individual- states/califo...](https://coronavirus.jhu.edu/testing/individual- states/california) 2: [https://coronavirus.jhu.edu/testing/individual- states/arizon...](https://coronavirus.jhu.edu/testing/individual- states/arizona) 3: [https://coronavirus.jhu.edu/testing/individual- states/florid...](https://coronavirus.jhu.edu/testing/individual- states/florida) 4: [https://coronavirus.jhu.edu/testing/individual- states/texas](https://coronavirus.jhu.edu/testing/individual-states/texas)

<comment><USER18> >despite our dense cities

Californian cities are actually not dense at all, which may explain why it isn't even worse. LA at 7.5k /mi2, San Diego 4.4k/mi2 etc.. compare that to New York at 27k/mi2, or even Paris 54k/mi2, Barcelona 41k/mi2

<comment><USER17> San Francisco is the fourth-most dense city in America (among cities with >75,000 residents). If you only count cities with >200,000 residents, it's the second-most dense city in America, behind NYC. Its overall density is 17k/mi2.

[https://en.wikipedia.org/wiki/List_of_United_States_cities_b...](https://en.wikipedia.org/wiki/List_of_United_States_cities_by_population_density)

And if you look at entire metropolitan areas, rather than just cities, LA is actually the second-most dense metropolitan area in the country.

Meanwhile, Phoenix, Arizona has a population density of 2k/mi2, and is having a massive outbreak.

Public policy appears to be the determining factor in flattening the curve of COVID outbreaks, not density.

</comment></comment></comment><comment><USER19> Only the US and Brazil, and some of the small Gulf states, seem totally unable to get their coronavirus epidemics under control.[1] Remember when Italy had a huge problem? That's over. China? Over.

[1] [https://ig.ft.com/coronavirus- chart/?areas=usa&areas=chn&are...](https://ig.ft.com/coronavirus- chart/?areas=usa&areas=chn&areas=ita&areasRegional=usny&areasRegional=usnj&cumulative=0&logScale=0&perMillion=0&values=cases)

<comment><USER9> Would you like to live like the Chinese? No.

Would you like to have the same death rates as Italy (much higher than US)? No.

Is the disease eradicated in either country? No.

The story of who did better remains to be written.

If anything California shut down way too soon. That's the problem that no one seems to talk about.

<comment><USER20> Countless lives were saved by California shutting down, that's WHY the death rate isn't like Italy.

<comment><USER21> Yet ...

</comment></comment></comment></comment><comment><USER22> The absolute fascinating thing about this is that for much of the country, they just got bored of Corona virus. Federal leadership is fading away to almost nothing, with getting the economy rolling prior to the election being the main focus. Then on top of that we have people who for whatever reason just want to go back to normal, thinking that its no big deal. There are many more who agree that the sickness is a big deal but it really only kills the elderly or weak and that's not a big deal, the sacrifice is worth it.

We decided that staying home and socially isolating was just to hard. The rest of the world must be looking at us in horror.

<comment><USER23> I think a lot of us inside our country are looking on in horror too. I for sure am - and terrified. I have lost all confidence and don't know if we can 'right the ship' (beyond just Covid I'm talking about basically everything all of our values)

<comment><USER22> Honestly have never seen anything like it. I am in shock that so many people are applauding whats going on here. If it happens again, I am not sure what to think.

</comment></comment></comment><comment><USER24> A lot of commenters here are stating that this is because of increased testing. Which is true compared to April. But I want to point out that at least in southern Callifornia the positive test percentage is actually going up now. [https://www.sfchronicle.com/bayarea/article/6-000-plus- Calif...](https://www.sfchronicle.com/bayarea/article/6-000-plus-California- shatters-its-single-day-15358857.php)

</comment><comment><USER25> I live in Palo Alto - last weekend I cycled past some sort of art festival in downtown Los Altos.

There were hundreds of people packed together looking at and touching art.

There was a big sign that 'we'd get through this together'.

They're also opening up farmers markets.

I think we're going to see the number of cases rise again.

<comment><USER26> >They're also opening up farmers markets.

The one near me in downtown San Jose just re-opened a couple of weeks ago. I walked through to see what it was like. Not too bad, they require masks for patrons and also the people working the vendor stalls. Also, customers aren't allowed to touch produce at all and contactless payment is preferred.

So all of that looked relatively decent. My issue was I would see patrons walking around or about to pay and they'd be lowering their mask to talk, not covering their nose with the mask, wearing their mask on their neck, etc.

I'm not sure how to correct issues like that.

<comment><USER27> Supposedly if it's outside it's not as bad: [https://www.youtube.com/watch?v=n6QwnzbRUyA](https://www.youtube.com/watch?v=n6QwnzbRUyA)

</comment></comment></comment><comment><USER28> Most Bay Area counties still aren't reporting recoveries distorting the Active Case number, unless you believe 0 people who have had COVID-19 in San Francisco have recovered since the SFDPH started tracking cases.

EDIT: Additional context: [https://www.worldometers.info/coronavirus/usa/california/](https://www.worldometers.info/coronavirus/usa/california/)

If a county's active case number equals Total Cases Minus Deaths, then they're not reporting recoveries. Los Angeles is the largest county not reporting recoveries. In just the Bay Area, Alameda, Santa Clara, San Francisco, San Mateo are the 4 of the 5 largest counties in the Bay Area by population and are not tracking recoveries. Contra Costa and all the North Bay counties appear to be tracking recoveries.

</comment><comment><USER29> Could this be due to an increase in testing or better testing strategy rather than an increase in cases? Daily COVID cases have been rising significantly over the last couple months in California, while deaths have been flat or slightly declining. Deaths are probably a more accurate indicator than positive tests, so it seems like positive test results do not match the actual case count very well.

<comment><USER30> It's both. Hospitalization is up, so it's really unlikely infections aren't also up.

</comment></comment><comment><USER31> People are getting quarantine fatigue.

I think at some point soon, people are just going to give up.

We could've bit the bullet, and did a hard lock down for 4 weeks. This would've killed the virus in its tracks. But instead, we didn't take the hard actions. And now, this virus is going to take its toll.

But, no worries. The stock market is doing all time highs.

<comment><USER32> Yep. At first it was SHOCKING to see the new cases rise. Then the death tally started rising fast. Then the videos from Italian and US hospitals.

But like most things, once you are accustomed to it, the amount of fear people feel declines. As fear declines, apathy increases.

It's like gang violence, child trafficking, or casualties of war. At first it is shocking, but then people get used to it and go on with their lives.

</comment></comment><comment><USER33> From a pandemic standpoint, it seems confusing to group together all of California. The article seems to be saying most of the outbreak is in LA and Riverside counties.

<comment><USER34> >The article seems to be saying most of the outbreak is in LA and Riverside counties.

LA and Riverside Counties are about 1/3 of California by population, so while it is somewhat disproportional for most of (whatever) to be there, it's not anywhere close to what it might seem when you name 2 out of 58 counties.

</comment></comment><comment><USER35> Paywall free article: [https://beta.trimread.com/articles/22708](https://beta.trimread.com/articles/22708).

Somewhat related to the OP's post, what really drives me crazy is focusing on cases since beginning of time. California has 174k cases in this case. What really should be looked at is week over week increases and number of new cases, and only of active cases, old cases shouldn't really be included but newspapers love sensationalism. In addition states are combining antibody test positives(person had covid, recovered) with active positive tests(nose swabs, person is infected with covid) [1]. Basically testing and presenting digestible results have been a disaster. In addition finding decent dashboards for this info is surprisingly hard. Local ones like santa clara counties one are good [2], but step up the state and national level and its hit or miss.

[1] - [https://www.beckershospitalreview.com/public-health/this- is-...](https://www.beckershospitalreview.com/public-health/this-is-a-mess- cdc-4-states-combine-viral-and-antibody-results-skewing-covid-19-data.html)

[2] - [https://www.sccgov.org/sites/covid19/Pages/dashboard.aspx](https://www.sccgov.org/sites/covid19/Pages/dashboard.aspx)

<comment><USER36> Agreed. The news functions like a monitoring system with an extremely bad alert configuration.

</comment></comment><comment><USER37> Does anyone know how many health care personnel in percentage of f.ex nyc hospitals that have had the illness. I've been wondering if institutional immunity in centers of spread would slow down resurgence as hospitals and health care centers get more immune workers.

<comment><USER38> From what I've read, it is still far from certain infection gives a usable immunity. So rather than that, it might as well kill off medical personal due to repeated exposure in huge amounts.

</comment></comment><comment><USER39> In other news .. [https://outkick.com/media-ignores-90-coronavirus-death- colla...](https://outkick.com/media-ignores-90-coronavirus-death-collapse-in- country/)

<comment><USER40> In other news social distancing caused a dip in deaths, so we should all go mingle again so we can cause a rise in deaths?

This article is ridiculous, the author is simply upset that many sports are closed for a season, and is looking for any justification to reopen. Yes the media blows things out of proportion, but the author even has a graph in the article that shows covid19 kills old people more effectively than flu and pneumonia!

The last few days have seen global upticks in infections, maybe they'll get their chance to write another article about the media being fearmongers once a few hundred thousand more people die and the death curve hits another flat spot.

Absolutely absurd.

<comment><USER39> How do you know what caused the dip?

Are you sure it's not any of these:

1\. The most likely to die have already died

2\. The virus has mutated to be less lethal

3\. We are learning how to treat it better

4\. Higher vitamin d levels in the summer

5\. A younger population is now getting it

6\. Some level of herd immunity especially in Europe and NY.

(Im probably missing some)

<comment><USER40> It's a straight forward leap from my proposed cause. This virus spreads from from person to person, so if you don't expose yourself to people, you are rather unlikely to get or transmit the virus. And if people don't get the virus, they can't die from the virus.

The leaps from your suggested causes are much more complicated.

If the second wave doesn't result in more deaths, then maybe it was one your proposed causes, and it'll be interesting to see how they contributed.

If deaths do spike again, then could you please stop spamming Hacker News with anti covid "skepticism"? Because if you continue to post and share these things in spite of a pile of dead bodies, you'd be complicit in the deaths from the third wave (if any).

</comment></comment></comment></comment><comment><USER41> California is still not doing enough. The original criteria for phase II were not met and yet the state decided to reopen anyway. Bay Area is thankfully staying stricter.

I was at a sit-down restaurant packed at capacity with live music a week ago about 100 miles from Bay Area. That should not be possible.

Close the state border to all interstate traffic. Test truck drivers at weigh stations near the state border. There should be sufficient capacity of the 5-minute Abbott test for this. Test other interstate travelers at mandatory checkpoints with a test that shows results no later than 60 minutes.

Close county borders. Mandatory testing at each county border. Issue a photo + ID card to each person tested. Mandatory verification of negative result before public transit boarding. Mandatory door to door testing in resisting areas. Deny all walk-in service to anyone not able to verify current test results with at-door scanner. Assume they are infected and serve them curbside. Have them scanned by a mobile dispatched unit at their if unable to travel to a testing location.

Annoying? That's what "essential" activity means. We are not making it inconvenient enough to not do non-essential activities.

Sounds extreme? That's what EU did to contain Italy and Spain. They closed internal borders and localized efforts.

Sounds expensive? Abbott will make a lot of money. Lives will be saved. The entire state could reopen and resume operations once strict testing protocols are in place and the curve is properly slammed.

We do not have an effective CDC to set national policy. Compare the real guidelines [1] leaked to press to the softened guidelines[2]. There is a reason why schools are rejecting[3] the official guidelines.

Finally. Want to host a church gathering? Your entire congregation is welcome to stay quarantined for 14 days on your premises.

P.S. The current CDC document has been updated. It was far less prescriptive earlier. Some sense prevailed. The entire section related to churches has been deleted.

Counties with zero infections should be allowed to open, but border protocols must be enforced. [1][https://www.documentcloud.org/documents/6889330-Guidance- for...](https://www.documentcloud.org/documents/6889330-Guidance-for-Opening- Up-America-Again-Framework.html) [2][https://www.cdc.gov/coronavirus/2019-ncov/downloads/php/CDC-...](https://www.cdc.gov/coronavirus/2019-ncov/downloads/php/CDC- Activities-Initiatives-for-COVID-19-Response.pdf) [3][https://www.usnews.com/news/education- news/articles/2020-05-...](https://www.usnews.com/news/education- news/articles/2020-05-15/school-superintendents-dismiss-white-house-will- follow-leaked-cdc-guidance-on-reopening-schools)

<comment><USER38> This is pretty much what was done here in Europe - country borders closed for everything except freight and on top of that towns and villages with exceptionally high infection rate were closed off and put under even stricter quarantine rules.

Even now with most of the wide area lockdown lifted, there are local clusters being isolated by locking down towns or whole regions, as at the moment with the German slaughter house cluster and the Czech and Polish coal mine clusters.

</comment></comment><comment><USER42> I'm wondering if this can be attributed to large gatherings associated with protests and demonstrations as well as the "reopening" that's currently on- going?

<comment><USER43> Re: protests, NYC has had large, extremely dense protests for weeks now. Mask compliance is near 100% at them, and testing is easy, ubiquitous and free. We have not seen a spike in cases.

<comment><USER44> Masks are only partially efficient. There is actually a spike all over US. The issue of protests is political, and no one wants to kick the wasps nest, because Republicans want unsafe Trump rallies to keep going.

<comment><USER45> Why specifically call out Republicans? Pretty sure that Democrats/liberals/BLM/antifa want the protests to keep going.

Or are you saying that the ones who would normally call that out are the Republicans, and they won't because they want Trump's rallies to happen? That's quite plausibly true. But putting it the way you did makes your view sound quite slanted.

<comment><USER44> Yes, the second statement.

</comment></comment></comment></comment></comment><comment><USER33> Is there any sign that summer in the northern hemisphere will help us out? UV, heat, outdoors, or other factors?

<comment><USER46> Major spread is happening in Florida, Arizona, and Texas.

<comment><USER9> yeah but in those states it is way too hot to be outside thus people spend even far more time inside than during the winter.

then Americans like their rooms numbingly cold so the AC goes full blast

so there, it actually makes perfect sense

these states got few cases in the winter because the weather was warm and sunny and they spent more time outside, now they are inside all the time in the cold, you get more cases

</comment></comment></comment><comment><USER47> I just want to point out that the government has not clearly outlined its strategy, and the reason is that the end-goal is to reach herd immunity in 12-18 month. Infections are OK, what's not OK is too-high rates of hospitalization which leads to death that can be prevented.

</comment><comment><USER33> Is there any evidence that the virus is evolving to change its deadliness?

</comment><comment><USER48> It seems that nobody dare to say this, so I will do, because I'm an idiot.

This is the expected outcome by the recent "black lives matter" events. Right actions but really bad time choice.

I wonder why people expected other results.

</comment><comment><USER49> How many are symptomatic?

<comment><USER50> asymptomatic =/= free of harm caused by the virus

<comment><USER51> The question was 'symptomatic'.

Lest you think it's a typo, the number of symptomatic tests is a critical number to know.

It's not about whether the individuals beings tested have been harmed, it's about whether testing shows us the infection rate in the population or not. If 100% of tests were of symptomatic patients, the numbers tell us almost nothing.

</comment></comment></comment><comment><USER52> We know now — definitively — those populations who are at risk from COVID, and those populations which are not.

In general the world is too connected, and populations too mobile, for a virus like SARS-CoV-2 to be contained, save for special case island nations that are willing to keep borders closed or visitors quarantined and ensure compliance until a vaccine becomes available.

For the rest of the world, what is abundantly clear in countries that are willing and able to perform the requisite testing, is that this is a virus that spreads easily and rapidly but thankfully in those under 50 is vanishingly unlikely to kill if it is symptomatic at all.

With over 10 million confirmed cases worldwide, likely in excess of 100 million actual infections, COVID could burn out in it's own before _widespread_ vaccination is available. Not due to containment measures, but due to enough people gaining immunity.

In my non-expert estimation, the public health response to COVID will ultimately result in about an order of magnitude more deaths, or roughly an order of magnitude more years of life lost, than due to COVID itself. The longer we continue lockdown-type policies, the greater the damage done.

COVID spread anyway, COVID continues to spread, although now it seems to be mostly in less risky populations as case counts surge and hospitalization and death rates remain constant. Some of this is of course simply sampling bias now that testing is more available for lower risk populations. We also see people who know they aren't at any real risk willing to be exposed in order to live their lives. Most inexcusably is that the most vulnerable populations (e.g. in nursing homes) were not adequately protected and even directly harmed by policies while the general masses bore the economic and social brunt of the involuntary enforcement measures.

In general every person who gets COVID and is asymptomatic or minimally symptomatic is basically someone getting an early access vaccine, and this can only be regarded as a positive in exerting a downward effect on _R_.

I think that speculating about long term effects in largely asymptomatic populations is an academic exercise which misses the forest for the trees. We need to reopen emergency rooms for the cardiac patients, reopen oncology departments for the chemo patients, reopen primary care clinics for health screenings, and reopen gyms, all to reduce/prevent well known and proven morbidities that cost millions of lives a year, not just continue to incur increase death in our most prevalent causes of death out of fear or politics.

</comment><comment><USER24> There are a lot of comments here essentially saying this is not a problem an we are properly reaching herd immunity.

I am wondering if these commenters are discounting the possibility of a vaccine? There is a strong possibility that we will have a vaccine before we can safely reach herd immunity.

<comment><USER53> Natural herd immunity is a non-starter anyway. If we reach herd immunity before a vaccine arrives, the economic damage would be far worse than any shutdown. If the R0 is 5.7, you need 82.5% (4.7/5.7) of the population either immune or quarantined/locked-down. If everyone stops the lockdown, that means 82.5% of the population infected. No matter how much we want to argue down the IFR, that's catastrophic.

Semi-related note, I think we haven't properly socialized the basic numbers behind the disease. I hear that Covid-19 is more contagious than first thought, with a "natural" R0 of 5.7. I also hear that it has a doubling time of around three days, originally thought to be more like 5-6. I also hear that the contagion period is 8-10 days. These three numbers do not make sense together. The R0 number needs to be reached during the contagion period. The average contagion period is likely far longer, or, it's way more contagious than 5.7 (which would mean the herd immunity threshold goes higher).

Also, if the illness is more contagious than we first thought, it means that the lockdown was even more effective than we first thought.

Here's a fun mental exercise for whoever's reading this. Say that you have a bubble of 100 people, and released a disease that infected 50 people and killed all of them; 100% fatality. Then say you discovered that it _actually_ infected all 100 people, but only 50% of them died. Is the disease less dangerous than you first thought?

<comment><USER54> The US is at 121K deaths atm. I made an estimate a while back assuming 60% of the population becomes infected and came up with 350K deaths. Bad, but not catastrophic. And I don't think we will get that high.

<comment><USER53> 60%/350k assumes an R0 of less than 2.5 (2.5 is to get to 60% herd immunity threshold, and then more infections continue to happen as it dies out), and also an IFR of less than 0.2%.

I think those are outside of the range of any recent estimates I've seen. Natural R0 is generally believed to be higher, and IFR is believed to be in the 0.5% - 1% range.

I also hope we won't hit 350k, but if we stay below that number, it will be because of a combination of continued lockdowns and finding an effective treatment/vaccine before we reach herd immunity.

<comment><USER55> The 1-1/R0 assumes a homogeneous and well-mixed population, and thus overestimates the fraction infected required for herd immunity. For real, people with lots of contacts will get infected and become immune first, with disproportionate benefit. That effect is hard to quantify, but it's near- certainly big, with many papers speculating that herd immunity may occur around 30% recovered:

[https://www.medrxiv.org/content/10.1101/2020.02.10.20021725v...](https://www.medrxiv.org/content/10.1101/2020.02.10.20021725v2.full.pdf)

This is before considering overshoot, which can occur just as in the homogeneous case (and is a reason to slow the spread as much as reasonable, even if you expect/fear that the end result will be herd immunity from recovered patients).

Reported IFRs are now calculated from serostudies testing for IgG. The best studies are on the high end of your 0.5-1%, but there's reason to suspect they're significant overestimates. Sensitivity for the blood tests was established from cases severe enough to seek medical help, and may be much lower for mild cases. There's also evidence of mucosal immunity in patients who test negative for IgG, and recently of T cell response in such patients too:

[https://www.medrxiv.org/content/10.1101/2020.06.21.20132449v...](https://www.medrxiv.org/content/10.1101/2020.06.21.20132449v1.full.pdf)

And the number of coronavirus deaths is greater than the actual number of excess deaths, since they're disproportionately elderly. All these second- order effects are uncertain, and difficult to incorporate into a final number; but they're real, and to ignore them completely risks a gross overestimate. Maybe people know that, and think of the overestimate as "conservative"; but considering the tremendous costs of both the coronavirus itself and society's response, I strongly disagree.

Finally, since you said "lockdowns"\--if you credit lockdowns (which are always vaguely defined; but government-imposed restrictions on movement and assembly, broadly) with all of benefit to the USA, how do you explain Japan? They're sitting at 8 deaths per million, they never significantly locked down, and their contact tracers are missing ~80% of cases. Maybe it's the masks? Even in California, use is waning. No one seems to have the political appetite to enforce the order coercively; NYC just stopped enforcing theirs entirely after an incident.

Maybe it's the contact tracing anyways, since missing 80% of cases might be fine if you catch the 20% who are super-spreaders (which you will disproportionately, since each person they infect is a chance to find them tracing back)? I could imagine big public campaigns to explain the importance of cooperating with contact tracers, or websites to help people anonymously advise their contacts (which have been tried for STDs, which have at least as much stigma). But I've seen almost nothing.

To be clear, I agree that restrictions on movement and assembly were and are a good idea in some cases, and New York and Lombardy surely wish they'd locked down earlier. I just don't like seeing them discussed as if they're responsible for all the difference between our current situation and a first- order model that we know is significantly wrong.

<comment><USER53> Therefore what? Are you arguing in favor of ending all lockdown-ish behavior, letting the chips fall where they may, and relying on herd immunity to save us? Because that's what I'm arguing against. I feel like I just read a template.

<comment><USER55> In my last paragraph, I said explicitly that I supported lockdown-type interventions in some cases. Further, even if the end result is herd immunity from recovered cases, slowing the spread will limit overshoot--so support for lockdowns and expectation the pandemic will end in herd immunity from recovered cases are not mutually exclusive.

So my suspicion is that you've skimmed my (long) comment, assigned it to some "template" that you've seen before, and made quick and inaccurate assumptions about my beliefs instead of reading what I wrote. I don't see how you could otherwise believe in good faith that I was "arguing in favor of ending all lockdown-ish behavior, letting the chips fall where they may, and relying on herd immunity to save us".

To start, I'm arguing that the first-order death calculations using 1-1/R0 and IFR from IgG are probably significant overestimates. Do you disagree? If yes, why do you think heterogeneity and evidence other than IgG of prior infection are negligible?

And again, how do you explain Japan? They found a way to continue mostly operating society, while American schoolchildren without attentive parents fall behind their classmates a day at a time. Wouldn't we be better off trying to understand and emulate their success instead of arguing about "lockdowns"?

<comment><USER53> I _don 't_ believe you are "arguing in favor of ending all lockdown-ish behavior, letting the chips fall where they may, and relying on herd immunity to save us". My point is, if you aren't, then why are you acting like you are disagreeing with me?

I've seen you ask about Japan in other threads, using much of the same words, without the various counterpoints seeming to register to you, and now you're making the same points here. It just seems weird. Why are you bringing Japan up as if it's some sort of counterpoint, when it's a counterpoint to something that you apparently believe in? Why are you bringing it up if you aren't in favor of ending mitigation and aren't in favor of acting more like... Japan? I don't care if (R0-1)/R0 is off by some _negligible_ percentage - it would still mean there's a huge difference in death depending on whether we do or don't stop mitigation. I'm arguing against the people that don't even seem to have a _rough_ mathematical sense about this, and also the people that seem content to be like Thanos and shrug their shoulders at a million, 350k, or 200k people in the US dying as long as it's not their friends.

I mean, back to your first comment, if you think herd immunity maxes out at 30%, and that IFR of 0.5 - 1% is a "significant overestimate", then you're pretty close to projecting our current death toll anyway. 0.25% IFR at 30% would be like 250k in the US, which we'll probably hit _with_ our half-hearted mitigation. But since you're in favor of mitigation efforts, you clearly don't believe the estimates to be that low. So... again, what is the relevance of what you're saying? Are you just being sophist? What's the syllogistic connective tissue in your points? What's your therefore? You believe the R0 and IFR estimates are overestimates; therefore what?

Why do I need to explain Japan? I mean obviously something must be happening in Japan that is consistent with what is happening elsewhere, it's not like they are an alternate universe. It's not like Japan is a _counterpoint_ to NYC or Lombardy; they're in the same reality _somehow_. How do _you_ explain Japan? It's probably some mixture of culture and bad stats in a way that doesn't really have any bearing on what _we_ should do - we already have sufficient data to know that mitigation saves lives, and we're stupid enough that we're about to learn it all over again for the first time over the next six weeks.

<comment><USER55> 1-1/R0 may be off by a factor of two or more due to heterogeneity, which I don't consider negligible. I listed that and other effects because I believe they are real and significant, and not because they fit cleanly into any argument for a preconceived position. I agree that a horrifying number of people have died (and will die) of coronavirus. I also believe that the societal costs of the lockdowns will be huge, and will be paid for years to come. Ten years from now, will social scientists find a significant and permanent divergence in educational attainment between rich and poor schoolchildren starting from the school closures? I don't know, but from other studies of interruptions in education I fear yes.

Both harms seem tremendous to me, neither so obviously larger than the other that we should disregard information that helps make the best estimate possible. Do you disagree? If yes, how did you convince yourself that the costs of the lockdowns were definitely smaller? I agree that some of the people arguing against lockdowns are ignorant of the likely death toll, or indifferent to human life; but some of them just think you're underestimating the cost of the lockdowns, just as you think they're underestimating the cost in human life.

In any case, we certainly both agree that the government should take actions to mitigate the coronavirus. In your reply above, you've switched freely between "lockdown" and "mitigation", when the latter is a broader category. Japan is surely undercounting some (like everywhere), but you can't miss NYC- scale mortality. I see basically no question that Japan has many fewer deaths, despite the near-absence of lockdown-type mitigations there. As you say, the Japanese aren't aliens--so if something in their culture is saving them, why can't we find out what and adopt it? Whatever they're doing, compared to the USA, it let them operate their society almost uninterrupted and avoided about 46,000 deaths. How can you dismiss whatever they did as non-actionable, when we don't even know what it was?

Was it all the masks? I don't know, but maybe. So why haven't we tried actually enforcing their use, and a big public education campaign on how to wear them properly? How many Americans have dismissed masks because they tried a fabric mask made from too many layers of high-thread-count fabric and found it hard to breathe through? Masks made from meltblown material are easier, and back close to normal prices. Do Americans know that? I didn't until recently.

Or maybe it's the contact tracing? I doubted that before because Japan is missing most of its cases, but I read a paper observing that heterogeneity meant the most important cases (super-spreaders) would be found preferentially. So maybe it's their cooperation with contact tracers? You could spend more than Coca Cola does in a year advertising the importance of that, and it would cost less than a day of strict lockdown. Why haven't we tried that? Or hygiene in general? Are Americans just bad at washing their hands? Major ad campaigns for basic public health measures have been successful in Africa. Why couldn't they here?

Or maybe it really is something non-replicable? I liked the humidity theory, though the trend in Florida goes against that (though Japan also tends to run the air conditioning hotter and more humid than American tastes; should we advise people to do that?). Something genetic? But even if it's not replicable, I'm amazed by the incuriosity most people--on either side of the "lockdown" debate--show at this massive unexplained difference.

I'm afraid you're trying to interpret my comments as supporting or opposing a small number of well-known, hardened, binary political positions. The result is confusing, because reality is a poor fit for either side of those positions. To the extent I have any goal in writing this beyond procrastinating, it's to show that, and to encourage people to spend their time exploring all the (messy, uncertain, self-contradictory) information available, instead of just arguing a position.

<comment><USER53> I don't really want to engage with most of this. I think you need to appreciate that there is very little daylight between your kind of points, which might very well be in good faith, and the approach of someone who just engages in what-about-isms with an intent to occlude debate and distract people from the urgency of implementing _blunt_ solutions that do a lot of good. The latter is a particularly cowardly form of debate, where people suggest various forms of possible counterpoints and allude to certain controversial conclusions that they just never quite say out loud.

Just to highlight one point that I think is often missed. Comparing the "additional cost of covid if lockdown is ended" to the "cost of lockdown" is a frequently abused argument - partly because of bad faith, but in other cases because people legitimately miss that "ending the lockdown" doesn't mean that people will all of a sudden leave home. There is a portion of economic cost, and I won't engage in trying to size it right now, because I tire of this, that will remain just from people being rational self-actors. Restaurants will still close, schools will be reduced attended, etc. Because of the _virus_. So the comparison is to compare "additional cost of covid if lockdown is ended" (and we know there are sufficient idiots out there to drive spread in this case; it doesn't take a lot to have large effects) to "saved _marginal_ cost if lockdown ends".

<comment><USER55> I understand the impulse to reveal truths selectively in order to guide people to the correct short-term action. I think it's incredibly dangerous, though. That's basically what policymakers did early in the pandemic when they advised Americans that masks were ineffective. Perhaps they got their short-term action, conserving scarce masks for medical workers; but now the shortage is over, and people (a) still aren't wearing masks, and (b) have lost considerable faith in the public health authorities in general. Hard to judge, but I suspect their noble lie saved hundreds of medical workers or patients at most, and will kill tens of thousands of Americans over the next year.

Those policymakers were trained professionals, so I doubt I'm any better at picking and choosing what truths should be spoken. So especially in a forum of moderately sophisticated readers, I'm not inclined to limit myself. I understand that may sometimes superficially look like things people often say in bad faith, but I hope readers expend the effort to distinguish.

In any case, I certainly agree that the coronavirus has tremendous societal and economic cost independent of the mitigations. That doesn't mean we still can't make it worse, though. I'm perhaps more optimistic than you are that Americans can do the right thing after exhausting all other possibilities, and actually wear masks properly and cooperate with other mitigations short of lockdown, with the right (perhaps coercive) guidance. I hope you weren't too close to the earthquake, continuing the apocalypse bingo.</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment>


After Facebook censors iconic photo, Norwegian newspaper pushes back - <AUTHOR> http://www.npr.org/sections/thetwo-way/2016/09/09/493267919/after-facebook-censors-iconic-photo-norwegian-newspaper-pushes-back <comment><USER2> [https://news.ycombinator.com/item?id=12466352](https://news.ycombinator.com/item?id=12466352)

You're late</comment>


Distributed Weakness Filing – CVEs for OSS Shuts Down - <AUTHOR> http://cve.mitre.org/news/archives/2019/news.html#March072019_CVE_Program_Root_CNA_to_Assume_DWFs_Open_Source_Product_Coverage_Responsibilities_Beginning_March_7 <comment><AUTHOR> Thread for the person leading the project: [https://twitter.com/kurtseifried/status/1103858442479910913](https://twitter.com/kurtseifried/status/1103858442479910913)

I couldn't even get most people to submit well formed CVE requests, let alone correct the badly formed ones, let alone actually help with CVE assignments. The market has spoken, nobody really cares about ensuring Open Source is broadly covered by CVE. 1/2

A few projects and vendors care about making sure what they ship or specifically research is covered (e.g. [https://cve.mitre.org/cve/request_id.html#cna_participants](https://cve.mitre.org/cve/request_id.html#cna_participants) …) but nobody cares about Open Source in general, e.g. npm? 11 million developers. 73 CVEs...</comment>


Checked C: Making C Safer by Extension - <AUTHOR> https://www.microsoft.com/en-us/research/publication/checkedc-making-c-safe-by-extension/ <comment><USER2> This does nothing for use-after-free or integer under/overflow, so "Making C Safe _R_ by Extension" would be a better title.

It's good to see people attempting this, though. I would think chances are good Microsoft will use it internally, and that already would be a win. Also, it's a clang/LLVM extension. That makes it easier for third parties to start using it.

On the other hand, their choice to use _Ptr <T>_ for familiarity with C++ requires users who want to keep their code compatible with C compilers that don't support this a bit harder (it will require wrapping that in some macro, say __ptrTo(T)_ ). That isn't ideal for adoption in open source programs.

<comment> ainar-g Re. syntax. I don't understand, why can't they just make _Ptr an attribute like _Atomic, and provide a header <stdchecked.h>. That is,

      #include <stdchecked.h>

      void read_next(int *b, int idx, checked int *out) {
          int tmp = *(b+idx);
          *out = tmp;
      }

</comment></comment><comment><USER3> The problem here is you're not writing C, you're writing a language that kind of resembles C but is nonportable. So you might as well just use a different, memory safe language at that point.

<comment><USER4> I think you're missing an important point:

 _Checked C's design is distinguished by its focus on backward-compatibility, incremental conversion, developer control, and enabling highly performant code_

The problem is that there is a huge body of decades-old C code out there that we all depend on.

A typical Docker container with a web app contains maybe 10K lines of your own code in Python/Ruby/JS, 100K lines of framework code, and 1M or 10M lines of C code (i.e. the interpreter itself, the web server, SSL, the base image, etc.)

So I welcome new languages focused on incremental conversion.

That's what I'm doing with Unix shell: [http://www.oilshell.org/blog/2018/01/28.html](http://www.oilshell.org/blog/2018/01/28.html)

I applaud projects like Corrode, a C to unsafe-Rust translator, although undoubtedly they would have an easier time if Rust was designed in the first place for conversion.

[https://github.com/jameysharp/corrode](https://github.com/jameysharp/corrode)

So Checked C seems like a great idea to me.

I've noticed that most programmers seem to wildly overestimate the rate at which code gets rewritten. I think it's more accurate to say that code piles up over time. And unfortunately when the foundations are unstable, what you build on top is also unstable.

<comment><USER5> There is also a new tool, c2rust, [https://c2rust.com](https://c2rust.com) that I believe was inspired by Corrode.

In addition, we also have the bindgen tool in Rust that will auto generate the Rust FFI from C header files. It works really well. At that point you wrap the FFI in more Rust idiomatic interfaces, and expose those as the preferred library for others to use.

I mention this, because it is a path toward allowing for the carrying forward of a C codebase, but using a safe language like Rust for all future work, perhaps going back and rewriting the C if it's deemed important.

There are many examples of this pattern being applied to projects.

In addition to that, there is a cbindgen tool, that does the inverse of the bindgen tool, which would allow you to write Rust code, and quickly generate C header files for the FFI, allowing for critical areas of C to be replaced with Rust, without converting the entire project.

Putting all of that together might be a reasonable thing for large projects to convert iteratively, should they decide it's an important thing for them to do.

<comment><USER6> That is nice for userspace apps, but no one is going to use c2rust on Aix, HP- UX, __*BSD, Linux, Zephyr, QNX, RTos, ....

I think something like Checked C is the only way to actually make it work on those use cases.

<comment><USER7> No one is going to use Rust on any of these projects, because of political reasons and portability concerns (because LLVM doesn't target all platforms). But those two issues also apply to Checked C and I doubt any project you listed will ever use it either.

<comment><USER8> Since BSD's system compiler is now LLVM, I think it's only a matter of time some enlightened BSD variant (OpenBSD maybe?) tries something like Checked C.

</comment></comment></comment></comment></comment></comment><comment><USER9> Checked C source code: [https://github.com/Microsoft/checkedc](https://github.com/Microsoft/checkedc)

</comment><comment><USER10> I wonder how this will work out. GCC had checked fat pointers for years, used by almost nobody. Microsoft can make it stick on their platform, because they own the libraries and can make them use fat pointers.

I hope this works out better than Microsoft Managed C++.

<comment><USER11> There has been dozens of "safe" additions, variants and extensions to C along the years (from commercial companies, from individuals, from university labs, ...). Not one of them ever took off. This one did not either.

We always just say "hmmm... right... this one may look a bit better than the previous one (or doesn't)" and we never bother to use them. After 20 years like this, we can conclude that we like C as it is and are not interested in extra layers.

<comment><USER12> > After 20 years like this, we can conclude that we like C as it is and are > not interested in extra layers.

This says a lot about how little the industry cares about software quality. You'd think a bunch of people who make a living out of automating tasks would happily use tools that automate the checking of an importnt source of bugs. Instead, many prefer to use tools that look simpler on the surface and then end up manually testing for the same kind of bugs over and over. So you have people writing in dynamically typed languages who rely on (manually written!) unit tests to check that their program is not mixing up types, and others using unsafe languages and then relying on tests to catch memory access violations. Inevitably, tests never catch all issues and you end up with type errors and segfaults in production.

<comment><USER13> > This says a lot about how little the industry cares about software quality. > You'd think a bunch of people who make a living out of automating tasks > would happily use tools that automate the checking of an importnt source of > bugs. Instead, many prefer to use tools that look simpler on the surface and > then end up manually testing for the same kind of bugs over and over. So you > have people writing in dynamically typed languages who rely on (manually > written!) unit tests to check that their program is not mixing up types, and > others using unsafe languages and then relying on tests to catch memory > access violations.

In the case of a "safer C" I suspect it's largely because it never materialized. C still has its deficiencies relative to Rust, but Rust is the first viable competitor.

<comment><USER6> Sure it did, all major systems languages outside UNIX universe were safer than C.

But the OSes they were tied to were killed by UNIX's adoption, just like the browser has helped JavaScript to take over areas of other languages.

</comment></comment></comment></comment></comment><comment><USER14> October 1, 2018 ? :O

<comment><USER15> That's likely when the paper will be presented at the [https://cybersecurity.ieee.org/blog/2018/01/31/ieee- secdev-2...](https://cybersecurity.ieee.org/blog/2018/01/31/ieee- secdev-2018-call-for-papers/)

</comment></comment><comment><USER16> There is some precedent for a new language that gradually improves safety of an existing language; TypeScript did this to JavaScript and is arguably successful.

I don't know if Checked C will have the same success story but if devs can easily add this to their existing projects and it catches bugs, that seems like a win to me.</comment>


Social distancing: Why every day matters for Covid-19 - <AUTHOR> https://miro.medium.com/max/8212/1*4kOJv8hmd5VFPcBL1mywsw.png <comment><AUTHOR> The most important chart from this incredible analysis: [https://medium.com/@tomaspueyo/coronavirus-act-today-or- peop...](https://medium.com/@tomaspueyo/coronavirus-act-today-or-people-will- die-f4d3d9cd99ca)</comment>


Collaborative Mind Map of Knowledge - <AUTHOR> https://github.com/nikitavoloboev/knowledge-map <comment><USER2> This is a wonderfully ambitious project, but mind maps are really, really the wrong thing for this. They are basically trees, but knowledge is a web. Linear Algebra, for example, shouldn't live only under math because it's heavily involved in machine learning, graphics, and many other areas.

It's an excellent venture, but it needs a _way_ better system for organisation and navigation.

<comment><AUTHOR> I understand your point. I try to implement it as a web of things though as indicated by arrows going through and between each of the nodes. Each node is a tree yes, but together they form a connected web of nodes and trees where each node is another graph that can go as deep as it wants.

And I do agree that linear algebra is not just a subset of math but is connected to all the varying disciplines and I try to indicate these connections with arrows. Most of the important ones at least, otherwise the graph will just get messy.

The idea is to expose knowledge in a visual way and attempt to solve the problem of discoverability on the web.

<comment><USER2> I can't see what to do, how to search, how to browse, or, in short, how to make use of this.

It needs something more - it feels like it's seriously lacking in discoverability. It's currently also failing to load:

[https://my.mindnode.com/83qmKgoATj8TyKzprzsPfoirxa2g9WmFGx3x...](https://my.mindnode.com/83qmKgoATj8TyKzprzsPfoirxa2g9WmFGx3xpr1J)

I've bookmarked it, but I don't know what to do with it.

<comment><AUTHOR> Thank you for your feedback, I really appreciate it.

I plan to solve this issue with providing a web search engine in the future. I have laid out the plan here ([https://github.com/nikitavoloboev/knowledge-map- search-engin...](https://github.com/nikitavoloboev/knowledge-map-search- engine))</comment></comment></comment></comment>