Ask HN: What is your favorite method of sending large files? - <AUTHOR> I just opened up a simple HTTP server to send someone a large file. Then I figured, I never gave this question proper thought.
But some of you have, and I figured they make for fun and interesting stories ;-)
So what's your favorite method to send large files, of at least 5GB or bigger? Though, I'm also curious on how you'd send 10TB or more. <comment><USER2> Bittorrent. No, really. Lots of nice behaviors when transferring large amounts of data between arbitrary endpoints.

Transferring the torrent metadata is pretty trivial and can be done via a wide range of methods, and having that flexibility can be nice.

Unlike HTTP, you get reasonable retry behavior on network hiccups. Also, more robust data integrity guarantees, though a manual hash test is probably a good idea either way.

At least among people I'm throwing TBs of data around with, torrent infra is common and it's nice to not have to deal with some special-purpose tool that, in practice, is probably a pain in the ass to get compatible versions deployed across a range of OSes. Basically every platform known to man can run a torrent client of some sort.

And obviously, no dependency on an intermediary. This is good if you're trying to avoid Google et al. That does, however, bring a potential con: if my side is fast and your side is slow, I'm seeing until you're done. If I'm uploading something to gdrive or whatever, I can disconnect one r the upload is done. If you control an intermediary like a seedbox, that's less of a problem.

In general, though, torrents are pretty great for this sort of thing. Just encrypt your stuff beforehand.

<comment><USER3> Just be careful - unless you mark that torrent private it'll get posted to the DHT and crawlers like BtDig will pick it up and list it publicly.

For this reason I prefer using something like Syncthing which is designed more with this purpose in mind.

<comment><USER4> I am working on a tool that will help with this. It is built with AlpineJS, Nim, Aria2 and Webview resulting in a 5MB download which doesn't include the torrent file as that varies. The idea is that it is a one click solution as the torrent is embedded with the binary.

The inspiration for this tool is for assist with LAN parties. The one thing I have against the private flag is that it also disables LAN peer discovery, which would be okay if you use a tracker behind the LAN (though the torrent would need to be modified if the hostname/IP changes). Since you can't configure other people's clients I found it simpler to use Aria2 that is preconfigured to disable PEX and DHT.

Combine this with Metalink or Web Seeds, so that you can have a initial seeder based on HTTP. I think using IPFS would be a great web seed as long as the their gateway and Cloudfare's continues to stay up. IPFS creates a permanent URL if you will, so no need to worry about dynamic IPs or domains. It would be great if IPFS would have a smaller binary (right now it is around 20MB compressed) and had a way to get a file and "seed" it like a torrent. But for now I think torrent is mature enough.

<comment><USER5> Why not use Syncthing or Resilio Sync for LAN parties?

<comment><USER4> I have considered it. The one thing about Syncthing is that there is no true readonly way to sync. For example, someone may accidentally extract an archive in the Syncthing folder and it will sync everywhere. If Syncthing gets this feature I think I would be 100% onboard with it for LAN parties.

Resilio I believe supports it but I would prefer an opensource alternative.

[https://github.com/syncthing/syncthing/issues/62](https://github.com/syncthing/syncthing/issues/62)

<comment><USER6> You can use the send only feature in Syncthing to make your folder read only.

[https://docs.syncthing.net/users/foldertypes.html#folder- sen...](https://docs.syncthing.net/users/foldertypes.html#folder-sendonly)

<comment><USER4> Sure, I can have for example my computer be send only. But unless everyone else configures it properly as receive only, they can start pushing data to other clients. It would be far easier to have Syncthing create a readonly share similar to Librevault and Resilio.

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> 5GB on my local network: Windows File Share, Rsync, or HTTP depending on source/destination

5GB on Internet: Upload to my OVH VPS and HTTP or Rsync it to its destination

10TB, local or Internet: Physical Hard Drive.

Never underestimate the bandwidth of a station wagon full of backup tapes!

[https://www.tidbitsfortechs.com/2013/09/never- underestimate-...](https://www.tidbitsfortechs.com/2013/09/never- underestimate-the-bandwidth-of-a-station-wagon-filled-with-backup-tapes/)

And since you now buy 1TB micro SD cards, so perhaps I'd split the file 11 ways (no way it'll fit exactly) and send them via carrier pigeon. Or heck, I could just tape them to a drone and hope they aren't damaged in the crash. There's lots of ways to move data around. Maybe you want to UUENCODE it and submit bad URL's to a servers log so that it can be exfiltrated later? It would probably take a very, very long time, but could be done. I call it "CURLyTP"

[https://miscdotgeek.com/curlytp-every-web-server-is-a- dead-d...](https://miscdotgeek.com/curlytp-every-web-server-is-a-dead-drop/)

<comment><USER8> _10TB, local or Internet: Physical Hard Drive.

Never underestimate the bandwidth of a station wagon full of backup tapes!_

In Poland there is an option to send package via train for 8$. Just go to station and give package to conductor/train guard, few hours later receiver can get it at destination station. Sending HDDs full of raw video footage this way is very popular among video editors here.

<comment><USER9> That's a really cool idea actually

<comment><USER10> Same thing happens in Greece with buses. It's a very popular way for moms to send home-made food to their single/university student sons.

<comment><USER11> In Croatia as well :)

<comment><USER12> In Laos the (mini)buses were the most reliable way to ship anything. We had 3 motorbikes strapped to the top and a sat cattycorner to a caged chicken on my trek through the north in a 15-seater minibus.

</comment></comment></comment></comment></comment></comment><comment><USER13> If the data is below 50G in size and on my personal computer, then I just drop it into the Google Drive folder and it syncs to Google over night. When it is done, I export it and then send a link to the person(s). I pay $2 a month for the 100G account and I usually have about 50G disk space unused, so this is not an issue for me.

If it is above 50G and on my personal computer, I encrypt the data and then physically mail a USB stick with the data on it. Trying to arrange the downloading/uploading of 50G of data from my personal computer to another personal computer is a real pain. The people that I would send/receive that much data to/from, that is stored on my personal computer are usually people who don't know much about ftp, scp or easily sharing files over the Internet. Sending a USB stick is just so much easier and in many cases, faster. I make sure the recipients know how to decrypt the data before sending the data.

If it is on a server (For example, a side project instance in AWS), then I drop it into S3 bucket, export it and send the URL to the recipient. I just eat the cost of them downloading it. Usually I am making money off the work anyways, so it is the cost of doing business.

<comment><USER14> >If it is on a server (For example, a side project instance in AWS), then I drop it into S3 bucket, export it and send the URL to the recipient. I just eat the cost of them downloading it. Usually I am making money off the work anyways, so it is the cost of doing business.

Consider BunnyCDN way cheaper than s3.

<comment><USER10> BunnyCDN is a CloudFront alternative, not an S3 alternative.

<comment><USER15> They also offer an object storage product, it seems: [https://bunnycdn.com/solutions/cdn-cloud- storage](https://bunnycdn.com/solutions/cdn-cloud-storage)

</comment></comment></comment></comment><comment><USER16> Magic Wormhole ([https://magic- wormhole.readthedocs.io/en/latest/](https://magic- wormhole.readthedocs.io/en/latest/)) if it's between OSes.

Between OSX, AirDrop works very well. I have sent >10GB files between Macs, quite quick as well.

I have never send a 10TB file so I wouldn't know. None of my drives are that large yet :)

<comment><USER17> Magic Wormhole is good. These days I use croc,[0] which I find even better.

[0]: [https://github.com/schollz/croc](https://github.com/schollz/croc)

<comment><USER18> I don't see an option to send text (browsed through README and Issues), is this not possible with croc? With Magic Wormhole it's the default to get a message prompt when running "wormhole send". Or you can do it as a oneliner with "wormhole send --text 'something'". Wormhole William can do this as well IIRC.

I use MWH often for sending stuff like URL's, passwords, API keys... having to create a file for that is quite annoying (and remembering to delete it afterwards).

"brew install magic-wormhole" or "sudo apt-get install magic-wormhole" is easy enough and I don't hang around with people who use Windows. ;)

Resumable file transfers sound like a great feature though. Not sure it's implementable with MWH the way it now works.

<comment><USER19> Just added it now. Send with croc send --text "hello, world" [1]

[1]: [https://github.com/schollz/croc/releases/tag/v8.2.0](https://github.com/schollz/croc/releases/tag/v8.2.0)

<comment><USER18> Nice! I just tried it out now. I also like the fact that "croc" defaults to receive, where with MWH you need to type "wormhole receive". Saves a lot of keystrokes.

Tab completion is something I'll miss from MWH (I'm already converted!) but I can live with it, and my non-technical friends aren't even aware of the concept.

</comment></comment></comment></comment></comment><comment><USER20> Netcat:

        $ nc -l 4242 > dest

And then on the sending end:

        $ nc hostname 4242 < file_to_send

This works great when you just need to get a file of any size from one machine to another and you're both on the same network. Are used this a lot at one of my offices to schlep the files around between a few of the machines we had.

<comment><USER21> This is especially fun when combined with piping through tar, and adding pv in the mix for a transfer speed "progress" bar.

Fastest way to transfer a collection of files on a local network and doesn't require temporary storage for the archive

<comment><USER22> piping through tar is my "power move" when I want to steer a new recruit. I just do it casually in front of them... depending on whether they are awed or horrified by that, they'll get more hacky or more formal work.

<comment><USER23> You can also pipe through gpg if you want encryption along the way!

<comment><USER24> And through gzip/zstd/etc if you want compression!

<comment><USER25> lzop if you gotta go fast :)

<comment><USER26> zstd easily outruns a gigabit link on most somewhat-compressible data.

</comment></comment></comment></comment></comment></comment></comment><comment><USER27> I remember using AIM (AOL (America Online) Instant Messenger) and other instant messaging applications for direct peer-to-peer file transfer back in 2002. On fast university connections, you could transfer movies in a few minutes. It's crazy to me that direct, fast file sharing was easier and more ubiquitous almost 20 years ago than it seems to be now. More context for my oft-misunderstood early feedback to dhouston about Dropbox.

</comment><comment><USER28> Private Nextcloud. Or some free tier OneDrive/alternative is enought for me.

However no-one mentioned a super simple service: [https://wetransfer.com/](https://wetransfer.com/) \- Simple as drag & drop, enter recipient address, SEND. Pretty simple if you want non techie to send you something.

<comment><USER29> +1 for WeTransfer.

A dead simple UI and a link is nicely emailed to the receiver.

<comment><USER30> Also emails the sender when the receiver downloads it.

</comment></comment></comment><comment><USER31> Syncthing. [https://syncthing.net/](https://syncthing.net/)

It's like a private dropbox.

For files on the order of <= 10GB, magic wormhole is lovely: [https://techcrunch.com/2017/06/27/magic-wormhole-is-a- clever...](https://techcrunch.com/2017/06/27/magic-wormhole-is-a-clever-way- to-send-files-easily-and-securely/)

`sudo pip3 install magic-wormhole` is an automatic "must run this command" on all my servers. Simple, secure way of sending files from A to B, with no bullshit. No account creation, even.

</comment><comment><USER32> I've been working on solutions in this space for a couple years now. IMO making data on your local device available to others via HTTP range requests is the sweet spot between storing your data in the cloud and going full p2p.

Here's a couple of my projects:

[https://patchbay.pub/](https://patchbay.pub/)

code[0]

Sender:

      curl https://patchbay.pub/random-channel/filename.bin --data-binary @filename.bin

Receiver:

      curl -O https://patchbay.pub/random-channel/filename.bin

[https://fbrg.xyz](https://fbrg.xyz)

code[1]

This one works in the browser. You select a file and it gives you a link you can share with others. The underlying tech[2] is more complicated (relies on WebSockets) than patchbay, but the server is currently more reliable. I'm in the process of improving the patchbay server to be more robust, and I think it will eventually replace fibridge completely.

My current project is building something along the lines of Google Drive, but much simpler and designed to be self-hosted from within your home.

[0]: [https://github.com/patchbay-pub/](https://github.com/patchbay-pub/)

[1]: [https://github.com/anderspitman/fibridge-proxy- rs](https://github.com/anderspitman/fibridge-proxy-rs)

[2]: [https://github.com/omnistreams/omnistreams- spec](https://github.com/omnistreams/omnistreams-spec)

</comment><comment><USER33> Sending a 10TB file on my internet connection would take 2.5 years of constant uploading. Shipping a hard drive is cheaper and quicker.

<comment><USER34> 8x10,240,000Mbits /2.5/365/86400 = 1.03Mbs?

<comment><USER35> That's slightly faster than my Silicon Valley upload speed.

</comment></comment></comment><comment><USER36> It doesn't have to be large files, file sharing sucks. Even copy/past across devices, even your own devices, is a pain. How many times did I email myself some links to share them across devices. And I'm not even speaking about sharing pictures with people not very accustomed to technology (in other words, don't try to get them install a cloud storage app).

I gave that question a few tries but I feel part of the problem is the market being saturated by giants doing half the job. Another part of it is the lack of interoperability regarding file sharing between operating systems. I mean - a native right click>send to>user - native notification to the user with a solution to direct p2p download from that person. No software needed, seemless integration. Why is that so hard?

I really wish the big boys would give that a try rather than giving us toy- like features and price bump.

<comment><USER10> You're gonna love magic-wormhole.

<comment><USER36> It's great, but we're very far from the os-agnostic seemless integrated files sharing.

You see, we have protocols like HTTP and the web relies on it, everyone agrees this is how the web works and all OSes have one or several browser able to access it. Something similar works with emails. Yet, we don't have anything to share files between computers, imagine something like bittorrent directly integrated to all OSes, letting you right click > send to.

<comment><USER37> Having a generic "Send to" would only solve a part of it though. There is currently no standard way of connecting the two ends of the same cable to two computers and expect them to exchange files without heavy configuration on both sides. Wireless goes the same way.

</comment></comment></comment></comment><comment><USER38> >I'm also curious on how you'd send 10TB or more.

For my 12TiB of data I use Syncthing when I need to sync them more often, rsync.

I used rsync several times for billions of smaller files totalling to 300GiB, but really all depending on how I connect nodes. I prefer syncthing, but when only ssh is available, then rsync is good too.

Currently largest synced directory by syncthing (that shares usage stats) is over 61384 GiB :)) [https://data.syncthing.net/](https://data.syncthing.net/)

<comment><USER39> Is there an iOS client for Syncthing?

<comment><USER40> No.

[https://docs.syncthing.net/users/faq.html#why-is-there-no- io...](https://docs.syncthing.net/users/faq.html#why-is-there-no-ios-client)

</comment></comment></comment><comment><USER41> I have run a internet reachable whatever.com webserver from my home desktop computer for 20 years. I just copy or softlink the file to a place in ~/www/ and give out the corresponding web link. I have a couple nginx locations prepared with pre-existing bandwith throttles so it's a matter of soft linking to the appropriate ~/www/dirwithassociatedrate/.

If it's for more than 1 person I upload it to a VPS if it's small (<20 GB) or make a torrent if it's not.

</comment><comment><USER42> You can create a simple python http server to expose files in a directory to the local web:

$ python -m http.server 8000

and then you can start ngrok to expose the file

$ ngrok http 8000

that will give you an URL to share with whoever wants it.

<comment><USER32> Does ngrok limit data at all or just connections?

<comment><USER42> I've sent a few GB over this method using their free service. Doesn't seem to have a transfer limit

[https://ngrok.com/pricing](https://ngrok.com/pricing)

</comment></comment></comment><comment><USER43> It depends in whether I want to send it to someone in the local network or through the internet.

For local network:

I use miniserve ( [https://github.com/svenstaro/miniserve](https://github.com/svenstaro/miniserve) ) which is just a simple http server. There are similar tools for when I want to share it from the smartphone.

Through the internet it really varies:

Sometimes it is Firefox send ( [https://send.firefox.com/](https://send.firefox.com/) )

For photos, I use a self hosted photo solution piwigo ( [https://github.com/Piwigo/Piwigo](https://github.com/Piwigo/Piwigo) )

In earlier days it has been a self hosted nextcloud ( [https://github.com/nextcloud/server](https://github.com/nextcloud/server) ) instance. I still use it when the files are getting too large for Firefox send.

I also tried the already mentioned wormhole but this works only with tec ppl.

<comment><USER44> Firefox Send doesn't work anymore

<comment><USER45> That's unfortunate. That was my go-to method.

From [https://send.firefox.com/](https://send.firefox.com/):

"Firefox Send is temporarily unavailable while we work on product improvements.

We appreciate your patience while we make the Firefox Send experience better."

<comment><USER46> I wonder how temporary it is because it's been saying that for a couple months already. I think that service is dead.

</comment></comment></comment></comment><comment><USER47> Related: If you need to transfer sensitive data over Bittorent, Age is a good tool for encrypting it before transmission.

[https://github.com/FiloSottile/age](https://github.com/FiloSottile/age)

<comment><USER23> Why a new tool? How is this better than gpg symmetric encryption, considering gpg is installed/available effectively everywhere?

Encrypt:

        gpg --symmetric file.dat
        (enter a password)

Decrypt:

        gpg --decrypt file.dat.gpg > file.dat
        (enter the password)

<comment><USER47> IMO, GPG is simply not a good tool and it should be replaced.

Others have said it better than I can. See e.g. [https://latacora.micro.blog/2019/07/16/the-pgp- problem.html](https://latacora.micro.blog/2019/07/16/the-pgp-problem.html)

<comment><USER48> That's said often about any tool which has been around long enough. People without experience come around and think they can replace an old tool with a better one, but it's usually only ignorance of either the complexity of the task, knowledge of using the tool properly, or both.

<comment><USER47> GPG is just not a very good tool. I think 'tptacek explained it quite well in the article I linked.

With that said, Magic Wormhole is also a very good tool for transferring files. It will encrypt in transit. So for many files, using a separate encryption tool is not necessary. (So far I haven't tried it for large files.)

<comment><USER49> What no one seems to mention is that Magic Wormhole depends on the maintainer's server to negotiate transmission between clients. I don't like that requirement. Better to GPG encrypt and use bittorrent for a direct transfer. At least then we're using public trackers instead of some private server.

<comment><USER50> How would public BitTorrent tracker servers be better than a single rendezvous-server run by the tool's author?

With trackers, you're revealing the fact-of-transmission, transmission-size, & endpoints to any number of unknown remote parties. Potentially, attackers not even on the privileged network-path from origin to destination could tee off a copy of your encrypted data for offline analysis.

With Magic Wormhole's rendezvous-server, only one server, run by the same person whose code you're trusting (& can audit), briefly relays encrypted control-messages. (It might even be limited in its ability to deduce the size of the transfer – I'm not sure.) And if that's still too much, you can run your own rendezvous server.

It seems to me the amount of information leaked in the BT Tracker approach is strictly (& perhaps massively) more, to more entities, than that leaked in using the Wormhole author's server.

</comment></comment></comment></comment></comment></comment></comment><comment> Paul-ish I use Dropbox to send and receive files that over the email/slack/etc attachment limit. Dropbox has a cool "request files" feature that let's people upload files directly to your Dropbox.

I've never needed to send TBs of data.

</comment><comment><USER51> I asked the same question on HN about 3 years ago[0]. I'm curious to see how the answers have changed and what might have stayed the same.

[0]: [https://news.ycombinator.com/item?id=15440571](https://news.ycombinator.com/item?id=15440571)

</comment><comment><USER52> SFTP Chroot server. lftp client using mirror subsystem + sftp. It is multi- threaded, even for a single file and supports rsync like behavior even in sftp chroot. I can max out any internet link using lftp (client) + sftp (protocol).

</comment><comment><USER53> I'm partial to [https://mega.nz/](https://mega.nz/)

</comment><comment><USER54> 10TB or more, not sure. Anecdotally I did have to send GBs of home directory locally recently. Fastest way I found out the hard way was mounting SMBFS and `tar` it there. But of course it is unreliable without resume etc.

For the thing you did, these could have worked.

[https://file.pizza](https://file.pizza)

[https://instant.io](https://instant.io)

[https://www.sharedrop.io](https://www.sharedrop.io)

</comment><comment><USER55> When I worked as a federal contractor in 1996-1997 we used password protected FTP servers to store giant text files for import to a database. Later on they moved it to a website you login and download the text file. Since it was so long it took almost all day and if it aborted had to start all over again.

I had to keep Netscape open because it showed the status of the file download. People asked why my web browser was open all day, it was for the giant download that is part of my job.

</comment><comment><USER56> Apparently not explicitly mentioned: rsync _over ssh_.

Windows now comes with real sshd - Mac has it, and linux/bsd of course has it.

For small files (less than 10gb?) generally _scp_ \- but trying to get in the habit of using rsync. Generally scp isn't a very good idea.

For larger _filesystems_ zfs send over ssh.

For streaming: dlna/upnp over zerotier vpn.

Shame Firefox send service imploded - for smaller files and where ssh isn't an option - it was a nice service. But a little too much hassle to self-host.

</comment><comment><USER57> Bike courier with checkered past, simultaneously on the run from organized crime clans and mercenaries working for retired spies, probably has mirror shade implants, lives in squat, punk haircut, etc.

Didn't realize there was another way.

</comment><comment><USER58> USPS

</comment><comment><USER59> Resilio Sync (formerly bittorent sync) is my go-to for any file larger than a few hundred megabytes.

<comment><USER60> Same here. I use this to deploy large files to my VPS

<comment><USER61> I third this suggestion, if you have to send the file to multiple parties they can all share among themselves (it's BitTorrent after all) speeding up the process even further.

</comment></comment></comment><comment><USER62> WDT from Facebook - [https://github.com/facebook/wdt/](https://github.com/facebook/wdt/)

We use it for copying hundreds of TiB's regularly, occasionally over a PiB.

</comment><comment><USER63> [https://Sendgb.com](https://Sendgb.com) is good way over the internet. Possible to send large files up to 20 gb. No needed registration or sign in.

</comment><comment><USER64> Not something I've used, and it's difficult to countenance anything called that, but what Globus (globus.org) has become seems to be quite popular for transferring large research datasets between academic sites which subscribe.

For ssh, there's [https://www.psc.edu/index.php/hpn- ssh](https://www.psc.edu/index.php/hpn-ssh) for improving wide-area transfer speeds, and something else, I think.

</comment><comment><USER65> Since 5GB would take quite a long time to upload on DSL: almost universally dropbox, simply because it resumes + it's accessible to people with a url. If the recipient is ssh-friendly and there's a shared machine we can both access, rsync is quite a lot faster and more controllable and doesn't make my CPU angry for hours.

Otherwise I liked Firefox Send while it was running since I mostly trust them to not be full of intrusive nonsense.

</comment><comment><USER66> I run FileBrowser [1] to share media with friends. I would suggest trying IPFS if you don't want to forward a port from the internet. You'll get similar download performance to FileBrowser once your node integrates with the network (this takes 30 minutes or so). Check it out!

[1]: [https://github.com/filebrowser/filebrowser](https://github.com/filebrowser/filebrowser)

</comment><comment><USER67> I and my customers all have symmetric FTTH. When I'm copying a file from me to me (working remotely), I use a VPN. These are usually .vhdx files, ranging from 6GB to 300GB. I copy them to here for troubleshooting, then I send them back.

When I want to make something large available to someone else, I post it on my local webserver. It points to an NAS on my LAN; anyone in the house can just drop whatever on it and hand out a link.

</comment><comment><USER68> Syncthing works pretty well in my experience, even across NAT. I recently shared 10GB worth of photos with a family member and ~30min later it was done.

</comment><comment><USER69> Worth noting that there's an expensive but very popular product from IBM called Aspera

I've never used it myself but I've heard but it's actually quite impressive but the business model makes it inaccessible to individuals

Apparently they do some sort of complicated Trace routing and open a ton of simultaneous routes. Probably also uses something other than TCP 4 correction.

Seems like a good place for some disruption

</comment><comment><USER70> \- Airdrop for immediate stuff. Gets dicey around 30-50GB tho.

\- iCloud drive for personal stuff I need to share between devices. I trust this to sync anything up to a TB reliably.

\- Google Drive when I need to share to someone else.

\- Occasionally external drives when I need to move data fast locally.

\- Some combination of S3 / AWS CLI / EC2 when things go beyond personal computer capacity depending on where the data is coming from and going to.

</comment><comment><USER71> For things smaller than around 10gb, I tend to use S3. Either with pre-signed links behind a little bespoke portal that people can log into and see what files I've "shared" with them, or just direct S3 links for those that what to automate it all.

For bigger things, there are two basic paths:

If it's to someone that's not primary a tech person, then the data goes on a portable hard drive, and I either drive it to them, or mail it and walk them through accessing the data. In both cases I encrypt the data, generally with Veracrypt.

If it's to someone that won't mind using AWS's toolchain, and has decent connectivity, I'll use Snowcone or Snowball to get the data to S3 and give them S3 URLs.

I tend to get the data to S3 sooner or later so I can migrate it to Glacier. More than once I've had to recover GBs or TBs of data because the customer lost it, so I'm prepared now.

</comment><comment><USER72> I have used Hightail for years, from when it was called YouSendIt. Here is info from their website on plans and pricing

[https://www.hightail.com/file-sharing](https://www.hightail.com/file-sharing)

Hightail offers four different subscription levels for file sharing, with varying restrictions on file upload limits and storage. With our free Lite plan, you can upload files up to 100MB each, and have a storage limit of 2GB. Pro plans, which start at $15/month, allow you to upload individual files up to 25GB each. Teams plans, which start at $30/user/month, allow you to upload individual files up to 50GB each. Business plans, which start at $45/user/month, allow you to upload individual files up to 500GB each. Pro, Teams and Business plans come with unlimited storage.

</comment><comment><USER73> magic-wormhole: [https://magic- wormhole.readthedocs.io/en/latest/](https://magic- wormhole.readthedocs.io/en/latest/)

      pipx install magic-wormhole
      wormhole send /some/big/stupid/thing.tar.gz

<comment><USER74> Also install microsoft visual c++ build tools which is like 1.2GB :(

</comment></comment><comment><USER75> Someone proposed WeTransfer but there is also Tresorit Send ([https://send.tresorit.com/](https://send.tresorit.com/)) who does the exact same job. Bonus point: they say they encrypt the data and their servers are hosted in Switzerland.

</comment><comment><USER76> Inside a company, I like transfer.sh, which is like an open source version of file.io - * [https://github.com/dutchcoders/transfer.sh](https://github.com/dutchcoders/transfer.sh)

That's good for the 5G +/\- file transfer.

</comment><comment><USER77> For files less than 1GB I tend to use Telegram. It obviously has the downside of uploading to server X and then downloading from server X, but usually if I am sending a file to one person, I'll likely be sending it to others too, and therefore the ability to forward the file to others arbitrarily after the fact actually proves to be a pretty useful quality. If I care about the datas security, an encrypted 7z container or something will do.

For files I need more control over that are less than say 5gb, I tend to scp them to a web server I control, so that I can delete them afterwards.

For files larger than that, I'll use a private Bittorrent file. It's very rare I need to transfer files this large, but I really like this solution.

</comment><comment><USER78> Personally I would use my own Nextcloud instance for up to 20-30GBs. Not sure about TBs.

What about using "Firefox Send"? (I never used it so far)

[https://support.mozilla.org/en-US/kb/send-files-anyone- secur...](https://support.mozilla.org/en-US/kb/send-files-anyone-securely- firefox-send)

I read that the limit is 1-2.5GBs => maybe you could break down the file and upload it in multiple pieces... .

EDIT: oopps, Firefox Send doesn seem to be available anymore - [https://support.mozilla.org/en-US/kb/what-happened- firefox-s...](https://support.mozilla.org/en-US/kb/what-happened-firefox-send)

<comment><USER79> People were using Firefox Send to "send" malware and other bad things to people. That's what happens if you let anyone anonymously host files on your domain for free.

<comment><USER80> And how other platforms, like wetransfer, are dealing with that?

</comment></comment></comment><comment><USER81> There are many ways of sending big files over the Internet, so I will go into the top three, save time. The first one is the shortest, when using Gmail. Click on the drive button instead, rather than just the paper clip button you will be using with regular attachments. It provides an outline of your drive files, from which you can pick the file to import. Just as easy.

You should go to another cloud computing provider if you don't like Google Drive.

My third alternative is even more easy, but it's a little sluggish because it cuts you at two gigs either way. It is a service called WeTransfer, which was almost your only choice 15 years ago if you had to submit a huge file.

</comment><comment><USER82> I like gofile.io, as it's private and has no limits.

But for something like 10 TB or more, I'd see a torrent as the only way. My uploads speeds are too slow for anything else, the connection would be reset. The torrent also helps prevent corruption.

</comment><comment><USER83> Magic wormhole - it traverses NATs, is encrypted, requires configuration of the source or destination.

[https://github.com/warner/magic-wormhole](https://github.com/warner/magic- wormhole)

<comment><USER16> Do you mean requires _no_ configuration on the source or destination? :)

<comment><USER74> Installing it on windows required me to install visual c++ build tools, which is a beast of 1.2GB. So yeah it requires a lot. I rather recommend croc, just dropped it on my path it Just works.

<comment><USER84> [https://github.com/psanford/wormhole- william/releases/](https://github.com/psanford/wormhole-william/releases/)

</comment></comment></comment></comment><comment><USER85> When I'm the sender, I use an http server in the current directory using http- Server (node) or http.server (python 3).

When I'm the receiver, I'm using a self-written CLI tool "send me a file" (smaf): [https://github.com/nikeee/send-me-a- file](https://github.com/nikeee/send-me-a-file)

It basically opens up an http server on my machine and let's a user upload a file via curl or a web interface. It also computes a sha256 hash while receiving.

These methods only work on a local network. I use my VPS and sftp/http for transfers descending the local network.

</comment><comment><USER86> S3. Sometimes I do that even when I'm transferring on my local network just because I know the flow so well. After all, my speed to the Internet is roughly the same as the speed on my LAN. They're both gigabit duplex.

<comment><USER87> You config IAM perms every time?

<comment><USER86> I actually just use [https://docs.aws.amazon.com/cli/latest/reference/s3/presign....](https://docs.aws.amazon.com/cli/latest/reference/s3/presign.html)

Then I copy-paste that into a self-Slack and `wget` it on the other side.

It's almost equivalent ergonomically to a `python3 -m http.server` and because it's the same whether I'm giving to someone else or to myself, I spend less thinking!

</comment></comment></comment><comment><USER88> For anything 5GB plus - external drive & cycle over

The only person I'm sharing with lives near me, so sneakernet is the most convenient.

I once hit a transfer speed of 30TBs/hour carrying a box of hard drives home from work.

<comment><USER89> "Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway."

– Andrew Tanenbaum, 1981

</comment></comment><comment><USER90> I used to do the HTTP server approach when I was a student at MIT and had a static IPv4 and symmetric gigabit in my dorm room and therefore the recipient could download at usually their full downlink speed.

Now I live in the heart of Silicon Valley, a couple km from Google's Headquarters, and have a crappy 15 mbps uplink with no other options available to me, so I typically throw the file on Amazon S3, Google Drive, or Dropbox before sending a link to the other person so that they don't have to put up with a 15 mbps download.

</comment><comment><USER91> For local networks I use the builtin python http server `python3 -m http.server` (or `python -m SimpleHTTPServer` in 2.x). By default it binds to all so anyone on your network can access it.

<comment><USER92> When send large files I often get broken pipe error using this method

</comment></comment><comment><USER93> For transfers on the smaller side, I'd have a look at [https://massive.io/](https://massive.io/) I've done a fair bit of looking into options for sending files online and I like their pricing model a lot because it is purely usage based and has no caps. It's $0.25/GB for downloads.

For larger transfers, I'd look at File Catalyst Spaces. If you have a bigger budget, you could look at IBM Aspera or Media Shuttle.

</comment><comment><USER94> >10Gb - Stick it on a bitlocker encrypted USB stick and chuck it in the post. Next day download if you send it 1st class here.

<10Gb - chuck it on S3 and send a link to the other person.

</comment><comment><USER95> Usually a local HTTP server. But I've wondered before if binary data could be encoded in to a video file and then uploaded to youtube or some other video service...

<comment><USER96> Any compression would ruin that.

Though it should be possible to do what you described with 16MP pictures and upload them to google drive. Those won't be compressed according to someone under that LTT video where they abuse 5 business accounts for their backup.

<comment><USER95> >Any compression would ruin that.

Well not really. You could have an entire frame as a solid color red to represent a '1' and a solid white frame to represent a '0'. At 24 fps, that is 3 bytes per second and would certainly survive compression. Just keep drilling down, half a frame that is red and another half that is white would represent a '10' pair, now you've doubled your throughput. You could keep cutting these blocks down until you get to a limit, you certainly couldn't get down to a single pixel, but some factor of that. Plus you could use an encoding with error correction instead of just raw binary data.

<comment><USER97> Sounds like you'll end up with a couple of qr codes per frame in the end, which would probably survive compression

</comment></comment></comment></comment><comment> Sean-Der [https://webwormhole.io/](https://webwormhole.io/) has web and native clients. You can build for Windows/Mac/Linux/FreeBSD/Mobile. It uses WebRTC so if you are in the same network it will establish the best path possible.

I see a lot of people mentioning magic-wormhole and NAT traversal. I can't find any docs that confirm this. I think it always runs through a relay server?

</comment><comment> Seb-C I have been using mega.nz to synchronize and backup my files for years, and so far I am very happy with it.

Everything works as expected, there is a lot of storage space and end-to-end encryption. There are clients for all platforms and it just works.

I can easily create secure share links whenever I want.

The only downsides are that the browser-based app is very slow to start once you have lots of files, and the android client does not have the same synchronization capabilities.

</comment><comment><USER98> In the enterprise space, there is dedicated software for this function. It uses an UDP protocol designed directly for efficiency (less overhead, vs. TCP), and builds in resilience such that it'll work over connections with lots of packet drops, or connections with high latency or low reliability.

[https://gojetstream.io/](https://gojetstream.io/)

<comment><USER99> Interesting. Is this implemented similiar to QUIC as some kind of connection oriented protocol on top of UDP?

</comment></comment><comment><USER100> I am actually transferring 400Gb of data across the world right now. I decided to use Apple iCloud and it works pretty good.

Too bad the laptop I am transferring from in a bad condition and with Windows so sometimes iCloud service is freezing and I need to restart it.

I tried to create torrent but it seems it takes forever on such a weak machine.

</comment><comment><USER101> I think the question for 10TB is more complicated than it seems at first

Is it one 10TB file? Multiple files? How do you handle file integrity? Error correction?

</comment><comment><USER102> I have used linux's rsync to send up to 100GiB at a time if the connection is reasonably stable. There are a couple challenges with this method: 1) the way it resumes takes a while to recheck the file before staring again; the larger the file, the longer this takes 2) you need to know a bit about writing a simple script to loop the rsync until the transfer is completed.

</comment><comment><USER103> If it's a file less than 10 GB I will use my own website: [https://pixeldrain.com](https://pixeldrain.com). For things slightly larger I will split it up using 7zip and use pixeldrain too.

For much larger files I always fall back to BitTorrent. I have also kept an eye on WebWormhole, but haven't been in a situation yet where I needed it.

</comment><comment><USER104> Large files, and I mean from hundreds of GB to terrabytes I use [https://www.filemail.com/](https://www.filemail.com/). Their desktop app uses UDP, so it is a lot faster than TCP based protocols like FTP and HTTP. Yeah you can still use their website to send stuff, and if small in size, free anyway for up to 50 GB.

<comment><USER105> UDP really makes the difference in speed! <3

</comment></comment><comment><USER106> There was some tool for doing this securely on the command line. You run it with the file name and it gives you back a token. You give the token to your friend, they run the same tool with the token, and it downloads the file. Can't for the life of me find it or remember what it was called. Wormhole?

Edit: I guess it was magic wormhole, discussed elsewhere in this thread.

</comment><comment><USER107> I have a home nextcloud with a few extra TB of storage. I would put it there and use a share link.

But HTTP is not a great protocol for really large transfers like 10TB. Ideally you'd want something that parallelizes and does hash checks.

Even then, at 10TB you need a 2 gigabit connection to be even competitive with SSD-and-overnight-shipping (about 12hrs).

</comment><comment><USER108> Apple Mail has a 30-day storage zone for large email attachments that it automatically offers when you attach big stuff, and I usually use that.

When I don't I usually just zip it and upload it to the media section of my Wordpress-managed website.

I've never generated files in the terabyte range, so I don't worry about that.

</comment><comment><USER109> Have a look at the "File Sharing and Synchronization" section [https://github.com/awesome-selfhosted/awesome- selfhosted#fil...](https://github.com/awesome-selfhosted/awesome- selfhosted#file-sharing-and-synchronization)

</comment><comment><USER23> I self-host a Nextcloud instance. If it's too big for a browser download, Bittorrent is the way to go.

<comment><USER32> This is interesting. What conditions would make a file too big for a browser download? If the server supports range requests (I'm almost certain Nextcloud does), then the browser can download in chunks and just retry any failed chunks.

<comment><USER23> It's technically possible, but most browsers don't auto-resume, and most webservers will timeout a request after a certain amount of time has elapsed. You can get around all this with browser extensions, but it's honestly easier to use a more robust protocol (this is in the context of, say, a 10TB file that'll take you a few days to download).

<comment><USER32> I don't see how anything would be much more robust than HTTP. Any network based protocol can fail at any time, so you're going to have to have some concept of tracking your progress and retrying failed requests. Both are easy to do with HTTP. Can you give a concrete example of something that would work better than HTTP?

<comment><USER23> Here's an exercise for you:

In your browser, start downloading a test file, like [1]. Turn off your network card for a few seconds, then turn it back on. See what your browser does.

Next, start downloading a large file over Bittorrent, like [2]. Turn off your network card for a few seconds, then turn it back on. See what the client does.

[1] [http://speedtest.tele2.net/100GB.zip](http://speedtest.tele2.net/100GB.zip)

[2] [https://cdimage.debian.org/debian-cd/current/amd64/bt- dvd/de...](https://cdimage.debian.org/debian-cd/current/amd64/bt- dvd/debian-10.5.0-amd64-DVD-1.iso.torrent)

<comment><USER32> This has nothing to do with HTTP. It's just poor behavior on the part of the browser.

</comment></comment></comment></comment></comment></comment><comment><USER110> Dropbox transfer supports up to 250GB files now. That's the way to go for most of my needs.

</comment><comment><USER111> Plain and simple: I use swisstransfer.com which is free (50Gb) and swiss- based.

I did it like you "in the past" (on my own fqdn / public-data) but I've given up on maintaining my website.

A torrent works really good if you've got multi-Tb files and many recipients.

</comment><comment><USER112> For 500Gb or more, aspera soft/ascp. Used by quite a few large bioinformatics institutes for allowing data uploads.

[1] [https://www.ibm.com/products/aspera](https://www.ibm.com/products/aspera)

</comment><comment><USER113> A station wagon full of tapes hurtling down the highway...

<comment><USER114> No one makes station wagons anymore, except luxury european makers :(

<comment><USER115> Aside from the already-mentioned Subaru Outback, there's the Buick Regal TourX, the Mini Cooper Clubman, and I'll contend that the Dodge Journey is more stationwagon than SUV; it's on the Avenger sedan platform.

</comment></comment></comment><comment><USER116> Physical media for anything over a few gigs. It's just easier, especially if the recipient isn't technologically inclined. A lot of the media industry ships hard drives around to one another because content can be so massive.

</comment><comment><USER117> "Never underestimate the bandwidth of a 747 fully loaded with backup tapes"

</comment><comment><USER118> I was asking myself exactly this a few days ago, to share ~500mb worth of files with a few people on a message board. [https://gofile.io/](https://gofile.io/) worked fine.

</comment><comment><USER119> I use [http://ifile.dk](http://ifile.dk)

Simple. Works.

</comment><comment><USER120> 5G from a kubernetes pod: netcat

Send help... no seriously... how do I upload a file from a server I have a shell on but can't easily open ports to or install software on (and where the built in cp tool fails on big files...)?

<comment><USER32> One of the main uses I made [https://patchbay.pub](https://patchbay.pub) for was transferring files to servers/VMs where I don't have easy rsync access. You just need curl or another HTTP client installed.

</comment></comment><comment><USER121> If the network bandwidth on both end allows for a transfer in less than 48 hours, most likely rsync with the --partial-dir=.rsync-partial flag. Otherwise SD card / USB key / hard drive in the mail.

</comment><comment><USER122> Telegram. Up to 2GB per file so just 7zip and split the file(s).

Never needed to send anyone 10TB nor do I have that much storage but if you do you probably have some kind of NAS/server where you can enable FTP access.

<comment><USER123> 10TB equals mail a HDD.

</comment></comment><comment><USER124> Locally: SMB. Over the internet: SendGB. Ridiculously sized: HDDs in a car.

</comment><comment> ha-ckernews Fram's Fast File Exchange ([https://fex.rus.uni- stuttgart.de/](https://fex.rus.uni-stuttgart.de/)) works well.

Free/open source

</comment><comment><USER125> I have Syncthing syncing to a "/shared" directory on my private web server. If the file is under the 20-ish GB I have free there, that's good enough.

</comment><comment><USER126> Dropbox was literally built for that sort of thing! And with 5GB you can probably even do it with the free account by getting yourself a tiny bit more bonus space.

</comment><comment><USER127> I send public links from the Seafile instance running on my home server. For 10+ TB, I think the only practical option would be to mail a hard drive or two.

</comment><comment><USER128> Given the quantity of data sent from the LHC to universities across the Atlantic, I'm curious how they (CERN et al) handle their data transfers.

<comment><USER112> Mostly gridftp [https://en.wikipedia.org/wiki/GridFTP](https://en.wikipedia.org/wiki/GridFTP)

</comment></comment><comment><USER129> Onedrive/sharrepoint online.

Technically, unlimited storage on OneDrive, can force the recipient to verify 2fa via their email address, force expiration link, etc.

</comment><comment><USER130> as a game of thrones fan:

a raven with a sufficient large micro-sd card ;)

</comment><comment><USER131> I use justbeamit if it's a direct send to someone.

[https://justbeamit.com/](https://justbeamit.com/)

</comment><comment><USER132> Interesting information; Thanks to all commentators;

</comment><comment><USER133> I am running Jumpshare ([https://jumpshare.com](https://jumpshare.com)) and our users regularly send large files anywhere from 2GB to 100GB and in some cases even more. My recommendation is to use cloud providers for sharing files that are maximum 50GB in size. Please note that although you can share even bigger files you have to consider the reliability of the internet provider of both the sender and receiver. For files bigger than 50GB in size, I would recommend a P2P solution.

</comment><comment><USER134> I like mega, it's encrypted, there's command line tools (megatools), it's very fast to upload from a server in my experience.

<comment><USER135> Not to mention the historical value: [https://youtube.com/watch?v=o0Wvn-9BXVc](https://youtube.com/watch?v=o0Wvn-9BXVc)

</comment></comment><comment><USER136> Firefox send works great for this sort of thing though there is an upper limit on file size (can't recall what it is right now)

</comment><comment><USER137> Maybe I'm old, but physically. Most of my large transfers are photographs, video and audio recordings and I like flash drives.

</comment><comment><USER138> upload to s3 bucket, make public, send link

</comment><comment><USER79> Upload to cloud storage then have the other party download it.

If it's too big for cloud storage, ship a hard drive in the mail.

</comment><comment><USER139> Up to a few GBs I'll host them on my server and send a link. More than that: I'll host a torrent.

</comment><comment><USER140> In person. If it's not possible to share in person I point them to the place where I myself got it.

</comment><comment><USER141> USPS Priority Mail.

You can send a 10TB hard drive faster by USPS than it would take to transmit across the internet.

</comment><comment><USER142> [https://transfer.pcloud.com](https://transfer.pcloud.com)

</comment><comment><USER143> OneDrive has served well enough for my purposes. I've only uploaded files smaller than 1tb though.

</comment><comment><USER144> I have a raspberry pi with a hard drive. I throw the files in that and then use dat/ipfs.

</comment><comment><USER145> Google Drive! Send a public link.

</comment><comment><USER146> I upload it to my VPS and send someone a link.

I think google drive is a decent solution as well.

</comment><comment><USER147> for non-technical folks to send me big files (5GB limit), I send them to [https://transfer.pcloud.com/](https://transfer.pcloud.com/). no registration needed.

</comment><comment><USER148> I use Transmit on macOS to upload to an S3 bucket and then send them a link.

</comment><comment><USER149> if i think the recipient is competent enough, i'll ask for or provide ssh access and copy it with rsync

otherwise, i've mailed a flash drive before, and for <20GB i'll just put it on a public S3 bucket.

</comment><comment><USER150> Uploading to Sia's Skynet, then emailing or texting the link.

<comment><USER151> Or you can use [https://defy.chat](https://defy.chat)

It's still under development, but it's working towards being a decentralized discord alternative.

</comment></comment><comment><USER152> At 10 TB I guess your best option is to physically mail an HDD.

</comment><comment><USER153> Google cloud storage. Ui is actually nice, permissions are flexible and has both a robust web uploader as well as a cli one. Not to mention it's cheap and only billed by the GB/hour or something.

</comment><comment><USER154> scp to one one of my unrelated websites and send http-link. Remove file afterwards... #notperfect.

</comment><comment><USER155> ngrok.io and similar reverse tunnels

</comment><comment><USER156> Free unused Google Drive

Some times I'll have to open the file in a hex editor and change the hash if it sets off copyright restrictions.

This is a 10 gig limit.

</comment><comment><USER157> I like to use transfer.sh

</comment><comment><USER158> Keybase

</comment><comment><USER159> Retroshare

<comment><USER75> I didn't know this software had users...

</comment></comment><comment><USER160> Either via Google Drive or iCloud Drive. If it doesn't fit on these, use physical media

</comment><comment><USER161> Backblaze/Torrent

</comment><comment><USER25> socat on both ends :)

</comment><comment><USER162> S3 and signed URLs.

</comment><comment><USER163> streaming json-lines (for large json array files)

</comment><comment><USER164> Magic Wormhole

</comment><comment><USER165> Git LFS ;)

</comment><comment><USER166> S3

</comment><comment><USER167> rsync over ssh with retries

</comment><comment><USER168> IPFS

</comment><comment><USER169> S3

</comment><comment><USER170> python -m http.server

</comment><comment><USER171> scp

</comment><comment><USER172> Google Drive seems a lot more practical than pretty much every other answer in this thread.</comment>


Google embraces, extends, extinguishes - <AUTHOR> http://sircmpwn.github.io/2018/05/03/Google-embraces-extends-extinguishes.html <comment><USER2> Previous discussion: [https://news.ycombinator.com/item?id=16993715](https://news.ycombinator.com/item?id=16993715)

</comment><comment><USER3> Google is truly scary, and I do believe now very evil. I used to be on their side, but that was long ago and I no longer have any faith or trust in Google. They are all about the dollar and are no longer get net citizens IMHO.

<comment><USER4> What's scary is the potential of their models and ad targeting infrastructure used to monitor populations and identify dissenters in other countries. Given how tightly connected Google is to the CIA and NSA I would assume this is already happening.</comment></comment>


GHC Haskell switches to an LLVM backend - <AUTHOR> http://www.haskell.org/pipermail/cvs-ghc/2010-February/052606.html

<comment><USER2> Although this is really cool stuff and it looks like it's going to get merged in, the title is a little misleading here. GHC hasn't switched to an LLVM backend -- they're currently reviewing this patch and possibly merging in this support for LLVM output (not necessarily switching over to it entirely).

<comment><AUTHOR> The title is accurate, as simultaneously, the GCC backend is being dropped. [1]

1\. [http://www.haskell.org/pipermail/glasgow-haskell- users/2010-...](http://www.haskell.org/pipermail/glasgow-haskell- users/2010-February/018425.html)

<comment><USER2> But that link only says that via-C is being dropped which, as I understand it, is not currently the default backend -- NGC is[1]. This means that (1) via-C is being dropped; (2) NCG (native code generator) has been, and is still the default; and (3) this LLVM backend support is currently being considered to be merged in.

[1] [http://hackage.haskell.org/trac/ghc/wiki/Commentary/Compiler...](http://hackage.haskell.org/trac/ghc/wiki/Commentary/Compiler/Backends)

<comment><AUTHOR> That's right. The GCC (-fvia-C) is being dropped in favor of LLVM. Work on the native codegen continues separately.

</comment></comment></comment></comment><comment><USER3> Is this a big deal? (not a rhetorical question!)

<comment><USER4> Yes; at present, adding code generation for new architectures is kind of a nightmare (google "haskell Evil Mangler"). LLVM will make this much simpler (or, at least, foist it on the LLVM guys :) )

<comment><USER5> What about GC? When I last looked at LLVM (the current stable release, in fact), it looks like it didn't have a very good story there. Plenty planned out, but little implemented and less solidly.

<comment><USER6> LLVM is the low-level virtual machine. (Basically portable assembly; except much smarter.)

GC is orthogonal. There is some work to add auto-gc to anything on LLVM, but while that's brewing, you can also do it in your runtime. GHC's runtime already does GC, so this is not a big deal.

<comment><USER5> Errr ... OK. So the GHC runtime that supports (supported) GCC (and that now will support LLVM) doesn't depend on any special cooperation by that level below it, and does the GC on top?

Hmmm, Gambit-C shows you can do this with good results, but ... uck. Still, I could well see how this new arrangement would be much better than using GCC as a back end.

<comment><USER6> You eventually have to have code to run on the CPU. LLVM lets your compiler output LLVM instructions instead of assembly, and then LLVM converts those instructions to assembly. The level of expressiveness is about the same.

Your CPU doesn't do GC, so LLVM is not necessarily the best place to do it, either. (LLVM does do other things your CPU doesn't do, which is why C compiled to LLVM runs faster than C compiled to native code in many cases. I imagine this will work well for GHC, as well, although GHC's native codegen can outperform gcc too.)

It's also worth noting that Apple (and others) have poured hundreds of thousands of dollars into LLVM. Taking advantage of that is always good, and the whole point of "free software", in fact.

<comment><USER5> Yes, I've looked hard at using LLVM for the backend of a project, so I'm familiar with the above.

However LLVM does at least one thing that assembly doesn't, which is implement a stack. It also obviously handles CPU registers (and opaquely). To do accurate garbage collection (<http://llvm.org/docs/GarbageCollection.html>; note, the whole site LLVM site is not loading for me right now) you have to work with it to find roots in the stack(s) and registers. And the state of the implementation of that is what I was talking about.

Does this GHC backend use a conservative collector such as the Boehm collector? As I recall VMKit does.

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> Without going in depth this looks like very impressive honors thesis work compared with some of the other work I have seen.

<comment><USER8> Especially given that the author writes that he had "no haskell knowledge at first."

</comment></comment><comment> loup-vaillant What happened to C--?

<comment><AUTHOR> GHC uses a modified form of C-- internally, which is fed to LLVM, GCC or the native codegen.

The C-- standalone compilers never reached the maturity of e.g. LLVM, to justify the port.

<comment><USER5> Yeah, as far as I can tell C-- is lying the street slowly bleeding. The group at Harvard has run out of money for the compiler they were working on, and as soon as the principle investigator can catch his breath (he had to assume the duties of a TA) he'll finish packaging it into something that others can easily work with (e.g. a Git repository as I recall).

As it is, it's supposed to be pretty hard to build, depending on a lot of old and/or exotic tools.

GHC forked off of this effort sometime before the current C-- standard.</comment></comment></comment>


Pipeline Design Patterns for Continuous Delivery - <AUTHOR> https://www.singlestoneconsulting.com/blog/7-pipeline-design-patterns-for-continuous-delivery/ <comment><USER2> I like the post. #1-#6 patters makes total sense. I'm not sure on how #7 should be implemented. How can I create the "button" in GitLab or GitHub?

<comment><USER3> One of the authors here. In GitLab CI, you can create that manual trigger button by including the `when: manual` parameter in your .gitlab-ci.yml. You can also restrict who can click that button for protected pipelines, such as those that deploy to production. ([https://docs.gitlab.com/ee/ci/yaml/#whenmanual](https://docs.gitlab.com/ee/ci/yaml/#whenmanual))

For GitHub Actions, try `workflow_dispatch` ([https://github.blog/changelog/2020-07-06-github-actions- manu...](https://github.blog/changelog/2020-07-06-github-actions-manual- triggers-with-workflow_dispatch/)).</comment></comment>


Show HN: My startup pivoted; here's the Sencha Touch/PhoneGap mobile app source - <AUTHOR> https://github.com/celwell/wesawit-st2-app

<comment><AUTHOR> The actual CoffeeScript source of the Sencha Touch 2 app is in this folder: [https://github.com/celwell/wesawit- st2-app/tree/master/web-s...](https://github.com/celwell/wesawit- st2-app/tree/master/web-src/src)

</comment><comment><AUTHOR> We moved from a mobile app to web based solutions for venues and artists. (wesawit.com)</comment>


Google, please hire me - <AUTHOR> http://googlepleasehire.me

<comment><USER2> FWIW this is a good example of how not to get hired by Google (at least in the technical space). The essence of his message was "I want to work for you, I work really hard." which, when you think about it, few people work at Google who don't already want to work there, and well if you don't work hard you don't stay. So the two pieces of information this guy tries to share with Google are not actionable or useful to someone evaluating them.

If you really want to work for Google, you need to have a good reason _why_ you want to work for Google.

The two types of folks I saw who regularly made it through the process into a hire were people who were pursuing some goal and doing so at Google would be good for Google, and people who were doing some thing Google thought would be useful for them to be doing at Google so they went out and recruited them. I don't recall a single person whose only goal was to "work at Google" actually got through the process.

</comment><comment><USER3> This was made in 2011. [http://techcrunch.com/2011/09/07/google-fails-to-hire- the-go...](http://techcrunch.com/2011/09/07/google-fails-to-hire-the- googlepleasehire-me-guy/)

Spoiler alert: He was not hired by Google.</comment>


We Are Living in a Failing State - <AUTHOR> https://www.theatlantic.com/magazine/archive/2020/06/underlying-conditions/610261/ <comment><USER2> _rolls eyes_. No you aren't. You're living in a state with systemic issues that perpetuate a growing inequality.

My family left Germany in the 20s. My grandma told me stories of people going to the bakery with wheelbarrows of cash because the inflation was so dramatic.

Stop it.

</comment><comment><USER3> A very good write up. The initial premise is something I agree with: that the virus has shown up existing weaknesses in the US political, economic (income inequality, taken to extremes), and cultural matters. The pandemic has been a magnifying glass.

</comment><comment><USER4> The more I read the Atlantic, the more I realize it's a nihilistic rag. Ok, there are shortcomings, but it seems they only like to trumpet inflammatory remarks-perhaps they think they are counterbalancing our govt.

</comment><comment><USER5> This is "narrative reporting" where you start with a premise (US is a "failed state") and then cherry-pick events to fit the narrative. It can be entertaining, but it's so full of holes it's Swiss cheese.

</comment><comment><USER6> From the article...

"Firefighters from Indiana drove 800 miles to help the rescue effort at Ground Zero. Our civic reflex was to mourn and mobilize together."

Headlines from DDG News search...

Fort Myers Nurses Travel to New York City to Help

Real-life heroes: 25 Cleveland Clinic workers travel to New York to help with coronavirus battle

Tusculum University nursing graduates travel to New York to join fight against virus

Minnesota nurse on battling coronavirus crisis in NYC: 'I stepped up to the plate and here I am'

</comment><comment><USER7> This kind of reporting is weird. Yes, the virus is showing up problems.

No, the US is not a failing state.

Europe as a whole is faring almost identically to the US.

<comment><USER8> No, they are not.

How many citizens have lost their health insurance in the US and the EU?

How many citizens, in the IS and the EU, will claim bankruptcy from medical bills in the next 12 months?

<comment><USER9> So having medical insurance or not determines whether or not a country is a failed state?

By that logic the US was a failed state before the epidemic. But all you need to do is think about it a bit to realize that's ridiculous.

<comment><USER10> The US was a failing state before the pandemic, unless you're upper class. It's just even more so now.

<comment><USER9> Interesting how so many people are so desperate to come here and live in this failed state they'll enter the country illegally!

Really remarkable.

<comment><USER10> Not so remarkable.

"The illegal immigrant population of the United States peaked by 2007, when it was at 12.2 million and 4% of the total U.S. population. Estimates in 2016 put the number of unauthorized immigrants at 10.7 million, representing 3.3% of the total U.S. population. Since the Great Recession, more illegal immigrants have left the United States than entered it, and illegal border crossings are at the lowest in decades. Since 2007, visa overstays have accounted for a larger share of the growth in the illegal immigrant population than illegal border crossings, which have declined considerably from 2000 to 2018. In 2012, 52% of unauthorized immigrants were from Mexico, 15% from Central America, 12% from Asia, 6% from South America, 5% from the Caribbean, and another 5% from Europe and Canada. As of 2016, approximately two-thirds of unauthorized adult immigrants had lived in the U.S. for at least a decade."

[https://en.wikipedia.org/wiki/Illegal_immigration_to_the_Uni...](https://en.wikipedia.org/wiki/Illegal_immigration_to_the_United_States)

<comment><USER11> Those use the Pew numbers, which are known to be bogus because their inflow numbers come from a study that counted only working-age males, and their outflow numbers count everybody, even infants. The true net (through 2016) has always increased, even during the Great Recession.

<comment><USER10> Please provide a citation backing this assertion.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> I didn't vote for Trump, but I always hoped from the day after the election in 2016 that he'd put the partisan showboating aside and turn to governing. But it doesn't seem like that ever happened. And we need that now more than ever.

I think in some way, too, people want to feel united. I know I do. But it just can't ever work when the President calls for unity one day and then reverts to us vs. them namecalling on Twitter the next.

I really wish there was some way to overcome all of it but I don't know how we do it.

<comment><USER13> > I always hoped from the day after the election in 2016 that he'd put the > partisan showboating aside and turn to governing.

What made you think he was even capable of that, let alone was interested?</comment></comment>


Hands-on: gesture, voice, and the many inputs of Samsung's smart TV - <AUTHOR> http://arstechnica.com/gadgets/news/2012/03/hands-on-gesture-voice-and-the-many-inputs-of-samsungs-smart-tv.ars?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+arstechnica%2Findex+%28Ars+Technica+-+Featured+Content%29 <comment><USER2> _The TV searches "basketball scores basketball scores"_

Aw, it messed up in the demo. That doesn't bode well for usability of the speech recognition software. I hope Apple's offering fixes this ;)</comment>


Nvidia calling gaming PC owners to put their systems to work fighting Covid-19 - <AUTHOR> https://www.gamesradar.com/nvidias-calling-on-gaming-pc-owners-to-put-their-systems-to-work-fighting-covid-19/ <comment><USER2> It would be nice if NVIDIA donated compute time on one of their large in-house machines the DGX SuperPOD [1] or the DGX SAturn V [2] (#20 and #67 on the last Top500, resp.) for COVID-19 research. Running simulations in a data-center is far more efficient.

[1] [https://www.top500.org/system/179691](https://www.top500.org/system/179691) [2] [https://www.top500.org/system/178928](https://www.top500.org/system/178928)

<comment><USER3> Sure, agreed. But Folding@Home is setup for home, and both of those machines put together add up to the equivalent of maybe 1k-2k gaming rigs. Putting aside looking for the maximum possible efficiency, crowd sourcing has the potential to scale to many orders of magnitude larger than what can be done in a data center.

<comment><USER2> > But Folding@Home is setup for home,

Nothing prevents running the fah client on nodes of a compute cluster -- in fact my colleagues did that (while running a local F@H server), though that was a number of years ago just because they wanted take advantage of the distributed computing facilities provided by the client-server setup and built-in algorithms.

> crowd sourcing has the potential to scale to many orders of magnitude larger > than what can be done in a data center.

Potential it does have, but I am skeptical of the "many orders of magnitude" claim ever having a chance to materialize. I'd love to see a cost / benefit analysis on the effective amount of useful work contributed vs the cost of the same in a data center.

<comment><USER3> > I am skeptical of the "many orders of magnitude" claim ever having a chance > to materialize.

The many orders of magnitude has already materialized [https://en.wikipedia.org/wiki/SETI@home#Statistics](https://en.wikipedia.org/wiki/SETI@home#Statistics)

"On September 26, 2001, SETI@home had performed a total of 1021 floating point operations. It was acknowledged by the 2008 edition of the Guinness World Records as the largest computation in history.[22] With over 145,000 active computers in the system (1.4 million total) in 233 countries, as of 23 June 2013, SETI@home had the ability to compute over 668 teraFLOPS.[23] For comparison, the Tianhe-2 computer, which as of 23 June 2013 was the world's fastest supercomputer, was able to compute 33.86 petaFLOPS (approximately 50 times greater)."

<comment><USER4> "On September 26, 2001, SETI@home had performed a total of 1021 floating point operations."

Just to clarify, I think HN formatting ate a caret here (now dang can see in the dark!) and it's supposed to be "10 to the 21"; either that or floating point math is much harder than I remember.

<comment><USER5> No caret was in the original so none was eaten - it's just a superscript.

</comment></comment></comment></comment></comment></comment><comment><USER6> It's a nice thought from a community manager but I just checked Nvidia's GeForce Now game streaming service where I pay $5 a month to get access to 6 hour sessions and a ray tracing GPU and it's not available. Seems like a better headline would be "Reddit user reminds PC gamers that Folding@Home exists. Nvidia's twitter account agrees."

<comment><USER7> Certainly Nvidia could just use the spare capacity themselves to do it, instead of asking paid subscribers to do so.

<comment><USER8> You would think. And where is AWS? Spare cycles for humanity?

Its like driving through some burger joint where they ask you to spare change for the needy... since you know, you have so much money, and the burger joint can not possibly afford it.

<comment><USER9> Is it likely that a large cloud company has more spare capacity than the sum of all home GPU owners? I'd have thought a cloud company would optimise for the maximum possible utilisation of their infrastructure, whereas most gaming GPUs are probably off 99% of the time.

</comment></comment></comment></comment><comment><USER10> Reddit's r/COVID19[1] has gathered big teams for Folding@Home and Rosetta@Home to crowd-source computing power against COVID19.

The F@H team COVID19@REDDIT[2] has ID 236269, if you are interested to join.

[1] [https://www.reddit.com/r/COVID19](https://www.reddit.com/r/COVID19)

[2] [https://folding.extremeoverclocking.com/team_summary.php?s=&...](https://folding.extremeoverclocking.com/team_summary.php?s=&t=236269)

<comment><USER11> Where does it say that they're actually using the compute for COVID-19?

<comment><USER10> They have an announcement on their web site[1] and on twitter[2].

Also the web client shows information about the executing tasks (e.g. [3][4]). The COVID19 tasks are prioritized and will be executed when you keep the target settings of the client at default ("Any").

[1] [https://foldingathome.org/2020/03/10/covid19-update](https://foldingathome.org/2020/03/10/covid19-update)

[2] [https://twitter.com/foldingathome/status/1238568504200253442](https://twitter.com/foldingathome/status/1238568504200253442)

[3] [https://apps.foldingathome.org/project?p=11749](https://apps.foldingathome.org/project?p=11749)

[4] [https://apps.foldingathome.org/project?p=14329](https://apps.foldingathome.org/project?p=14329)

</comment></comment></comment><comment><USER12> Any actual progress have been made from all that decades of distributed folding?

<comment><USER13> folding@home alone has 223 peer reviewed papers

[https://foldingathome.org/papers-results/](https://foldingathome.org/papers- results/)

<comment><USER12> I was asking about actual progress

<comment><USER14> How do you think science work? It's like losing weight and gaining muscle, you have to work hard at it and there is no moment in time where you can say that that's the moment you lost weight or gained muscle. It is all incremental progress.

<comment><USER15> Still, would be nice to be able to point to some tangible result, such as "knowledge obtained eventually led to developing of better treatment for bone cancer" etc.

<comment><USER16> Doesn't even have to be that. It would even be interesting to know what has been figured out based on the folding even if it hasn't led to any improvements in medical treatments.

</comment></comment></comment></comment></comment></comment><comment><USER17> I went here to get started:

[https://foldingathome.org/start-folding/](https://foldingathome.org/start- folding/)

and started running the client, but...

* The fahcontrol DEB package is not compatible with Debian 10 / Devuan 3.

* There is no APT repository.

I'm willing to do folding@home, but not take-care-of-their-devops-issues@home.

 __Edit: __Apparently, indeed, the folding@home work I would be doing doesn 't have much in particular to do with covid-19. I feel a bit cheated here.

<comment><USER18> Maybe someone's already done this and I haven't noticed, but having Docker and/or Flatpack images for this seems like an obvious way to include as many machines as possible.

<comment><USER10> docker images: [https://hub.docker.com/search?q=foldingathome&type=image](https://hub.docker.com/search?q=foldingathome&type=image)

kubernetes: [https://github.com/richstokes/k8s-fah](https://github.com/richstokes/k8s-fah)

<comment><USER19> Kubernetes chart which runs with StatefulSet so compute time would not be lost when node is going down and/or pods are being rescheduled:

[https://github.com/pcktdmp/charts](https://github.com/pcktdmp/charts)

This design pattern is extremely useful when running for instance on AWS Spot Instances.

</comment></comment></comment></comment><comment><USER20> There is also Rosetta@Home[1] and they have a related research too. They even provided some useful results[2] already.

[1] [https://boinc.bakerlab.org/](https://boinc.bakerlab.org/)

[2] [https://www.ipd.uw.edu/2020/02/rosettas-role-in-fighting- cor...](https://www.ipd.uw.edu/2020/02/rosettas-role-in-fighting-coronavirus/)

</comment><comment><USER21> Also take a look at Rosetta@home: [https://news.ycombinator.com/item?id=22528992](https://news.ycombinator.com/item?id=22528992).

<comment><USER22> Yap, running it right now on my pc with a rtx 2060.

<comment><USER23> Does Rosetta have GPU projects these days? I was under the impression that it only ran on CPUs.

<comment><USER24> Yeah, version I'm running (7.14.2) will use GPU and/or CPU

<comment><USER25> No, you are misinformed. For rosetta there is neither a version 7.14.2 (that would be BOINC) nor are there any GPU projects.

</comment></comment></comment></comment></comment><comment><USER26> Upate: Service still fails to start but I just ran FAHClient manually again and it connected and is now processing a job apparently, so must have been a temporary glitch. Cores and GPUs at full throttle according to htop. Don't see anything indicating whether what it's doing is related to COVID-19, so blindly hoping it's helping with something, but I'll leave it on. (Says: "project 14307"??)

\---- original post </comment><comment>

I was just trying to get it running on Ubuntu 18.04 in response to this post but I'm just getting a lot of errors with not very useful error messages. Any ideas?

When I run FAHClient manually I get, 21:04:10:WARNING:WU01:FS01:Failed to get assignment from 'X.X.X.X:8080': No WUs available for this configuration

(I replaced the IP address with Xs)

When I try starting the service I get an error and journalctl -xe gives me, e.g., (unedited, it actually says "result is RESULT")

\-- Unit FAHClient.service has begun starting up.

Mar 14 22:02:27 poole FAHClient[17956]: Starting fahclient ... FAIL

Mar 14 22:02:27 poole systemd[1]: FAHClient.service: Control process exited, code=exited st

Mar 14 22:02:27 poole systemd[1]: FAHClient.service: Failed with result 'exit- code'.

Mar 14 22:02:27 poole systemd[1]: Failed to start LSB: Folding@home Client.

\-- Subject: Unit FAHClient.service has failed

\-- Defined-By: systemd

\-- Support: [http://www.ubuntu.com/support](http://www.ubuntu.com/support)

\--

\-- Unit FAHClient.service has failed.

\--

\-- The result is RESULT.

<comment><USER27> [https://apps.foldingathome.org/project.py?p=14307](https://apps.foldingathome.org/project.py?p=14307)

For a more user-friendly UI, you can also go to [https://client.foldingathome.org/](https://client.foldingathome.org/), which will then attempt to communicate with your client on port 7396 and display the project description among other things. (This also lets you change some settings)

For those running on servers, you can telnet 127.0.0.1 36330 to control your client.

If you just want the project numbers quickly, you can do grep project /var/lib/fahclient/log.txt

</comment></comment><comment><USER28> I've taken my "gaming PC" to Ubuntu for running Tensorflow & Keras. Is there a Linux version of this?

<comment><USER29> Yes, the Folding@Home site has debian packages.

</comment></comment><comment><USER30> It's a shame there isn't a Folding app for PS4, considering you could run it on PS3

<comment><USER31> Agreed. I have an unused ps4 in a box that could be used. I already have a spare laptop doing some work right now.

</comment></comment><comment><USER32> You can of course just run folding@home on your CPU as well, no nvidia requirement... required.

<comment><USER33> At this time they are not sending covid CPU projects to clients, only GPU ones.

<comment><USER34> I only got a CPU project, but no GPU (GTX 1080).

> No WUs available for this configuration

<comment><USER35> Same problem on my GTX 1050 (just tried updating drivers, no change). However, my laptop's GTX460M has pulled down project 11745 and is folding away.

EDIT after fiddling with gpu-index, cuda-index, and opencl-index (had to manually set them to point to my GTX 1050 [also have a BARTS card that is unsupported]) I was able to get a WU to download :)

<comment><USER36> how do you find the correct index values? My 2080 Ti receives no work.

<comment><USER35> In my log window I see:

      GPUs: 2
      GPU 0: Bus:1 Slot:0 Func:0 AMD:4 Barts XT [Radeon HD 6800 Series]
      GPU 1: Bus:2 Slot:0 Func:0 NVIDIA:7 GP107 [GeForce GTX 1050 LP] 1862
      CUDA Device 0: Platform:0 Device:0 Bus:2 Slot:0 Compute:6.1 Driver:8.0
      OpenCL Device 0: Platform:0 Device:0 Bus:2 Slot:0 Compute:1.2 Driver:378.49
      OpenCL Device 1: Platform:1 Device:0 Bus:1 Slot:0 Compute:1.2 Driver:1800.8

I want to use the GTX1050, so I set gpu-index to 1. Only 1 CUDA device so I set cuda-index to 0. To use the OpenCL device for the GTX, I looked at the bus # (bus 2), found OpenCL device on bus 2 (device 0), so I set opengl-index to 0. Not sure if this is the right way, but it worked for me.

</comment></comment></comment></comment></comment></comment><comment><USER37> If it's crowdsourced GPU compute you want, why not hit up crypto miners too? Maybe slightly less altruistic than gamers, but still.

<comment><USER10> Many crypto miners are indeed doing F@H at the moment. Either via curecoin[1] or by lending/joining their rigs to one of the teams[2].

[1] [https://curecoin.net](https://curecoin.net)

[2] [https://folding.extremeoverclocking.com/team_list.php](https://folding.extremeoverclocking.com/team_list.php)

</comment></comment><comment><USER38> Are others able to download the fah client for Windows? I am getting a 502 when I try to visit [https://download.foldingathome.org/](https://download.foldingathome.org/).

</comment><comment><USER39> Is there a way to force foldit@home to work specifically on coronavirus related jobs?

I installed it few days ago but got so far random assortment of other tasks and nothing about coronavirus so far.

<comment><USER10> Explanation by the director of F@H on twitter: [https://twitter.com/drGregBowman/status/1238930030585143297](https://twitter.com/drGregBowman/status/1238930030585143297)

</comment></comment><comment><USER40> I'm so glad to see people stepping up to donate compute resources to one of the best (IMHO) uses of computers, helping humanity.

Thank-you everyone.

</comment><comment><USER41> In this day and age when one can spool up any number of GPUs on AWS, how much can crowdsourced GPUs add?

<comment><USER42> Assuming your question is in good faith, the answer is: when you rent something on AWS, you are paying for total cost of ownership sliced out. When you run something on your gaming rig GPU, you already paid for the machine/GPU - now you are "just" paying for electricity and depreciation. So if you have a gaming rig sitting around while you are writing a web app or whatever, let it do some science for just the cost of electricity.

</comment></comment><comment><USER43> For people heating their home with electricity this is an option, unfortunately you cannot force GPU only!

<comment><USER44> Not true. You can select CPU and/or GPU. You can even choose to use specific GPUs by index number. So it's quite flexible.

<comment><USER43> In that case it's a GUI problem, I could not figure out how to do any of that. I only want to use the GPU = preferably zero CPU cores.

</comment></comment></comment><comment><USER45> Out of curiosity, are Macs and AMD GPUs not powerful enough for this sort of thing?

<comment><USER46> Some AMD GPUs are supported (in beta test). Here's the 'requirements' page:

[https://foldingathome.org/support/faq/installation- guides/li...](https://foldingathome.org/support/faq/installation- guides/linux/requirements/)

</comment></comment><comment><USER47> I'm unable to view this page - it refuses to accept my ad blocker on iOS

<comment><USER48> Try using Reader mode.

</comment></comment><comment><USER49> This is cool because it can help quickly determine structures if the virus mutates. However, the obsession with structural biology isn't necessarily optimal, because we're talking about shapes of things (analog, constantly changing) instead of discrete states of molecular circuits. Check out the book "wetware: a computer in every living cell" —- cells figured out logic long before we did. Treatments which operate from a structural paradigm, like small molecules, antibodies, and vaccines, are analog. Treatments which operate from a sequence / switching / circuit paradigm are digital.

The digital solution is stronger: delete Coronavirus RNA. This isn't dependent on shape of some molecule and how some other molecule docks with that inside a molecular blender. Genome engineering targets sequences directly, works with novel viruses right away, tolerates mutations with wobble base pairs, works in immunocomprimised patients, has potentially zero side effects, etc etc

I applaud these efforts but implore you to consider: why throw a fancy molecular wrench into the virus machine when you can instead set the "virus switch" to "off?"

If anyone's gonna grok the benefit of digital over analog, it ought to be the Hacker News community. The hardest part is delivery, we need to affordably mass produce nano particles with the CRISPR DNA inside. That's the number one thing holding back biotech from curing many diseases including this one: AAV is the standard vector but it only works once before you're immune to the gene therapy.

here's a link to a repo (WIP) to target nCoV with CRISPR [https://github.com/bionicles/coronavirus](https://github.com/bionicles/coronavirus)

If you can help mass produce nanoparticles with microfluidics or self assembly please email me firstname.lastname@example.com

Folding At Home is cool. Is folding and shape-medicine the best way to cure human disease? How do we know the answer to that unless we try to program cells like we program computers?

</comment><comment><USER50> How does this compare with results published by DeepMind?

</comment><comment><USER51> Wouldn't this qualify as datacenter use and as such violate the consumer nvidia driver EULA?

</comment><comment><USER52> These are the victory gardens of our new home front.

</comment><comment><USER53> I find it ironic that this is behind a paywall. My default policy is to just find the article elsewhere.

</comment><comment><USER54> 82

</comment><comment><USER55> Imagine putting those bitcoin farms to work...

</comment><comment><USER54> Ppl pmllsiqqhq hlaz pp. plans

<comment><USER56> Are you all right?

</comment></comment><comment><USER57> Someone gets $$$$ from mining my CPU, great one! :(

<comment><USER58> What an amazing hot take.

</comment></comment><comment><USER59> It'd be interesting if they added a F@H button to GeForce Experience.

</comment><comment><USER60> Wouldn't AMD chips be far more powerful for this?</comment>


Biscuit: a multi-region key value store for your AWS infrastructure secrets - <AUTHOR> https://github.com/dcoker/biscuit <comment><USER2> The project is nice, but I'm gonna have to stick with Vault as I like the flexibility of storage backends and not locked into AWS for enterprise-y apps that can't go to AWS.

<comment><USER3> How does Vault verify the identity of the host requesting access to the credentials? I didn't find anything in the documentation that would give me information pertaining to that.

<comment><USER4> You can use the new AWS-EC2 auth backend in Vault 0.6

[https://www.vaultproject.io/docs/auth/aws- ec2.html](https://www.vaultproject.io/docs/auth/aws-ec2.html)

<comment><USER5> That backend is very nascent, so don't oversell it. It currently is AMI dependent, making it next to useless for a lot of teams.

</comment></comment></comment></comment><comment><USER6> This is also similiar to Sneaker ([https://github.com/codahale/sneaker](https://github.com/codahale/sneaker)), which is written in Go. It doesn't copy to other regions by default, but it's not hard to handle that on your own. This also uses KMS, but stores encrypted secrets in S3.

</comment><comment><USER7> I prefer credstash ([https://github.com/fugue/credstash](https://github.com/fugue/credstash)) which uses KMS and stores encrypted values in dynamodb. It has built in ansible support via lookups too!

</comment><comment><USER8> It looks like this is fairly similar to Mozilla sops[1].

[1][https://github.com/mozilla/sops](https://github.com/mozilla/sops)

<comment><USER9> I read the documentation but can't determine if sops supports credentials via IAM roles. Can anyone with experience chime in?

<comment><USER8> It does - [https://github.com/mozilla/sops/blob/master/sops/__init__.py...](https://github.com/mozilla/sops/blob/master/sops/__init__.py#L1127)

</comment></comment></comment><comment><USER10> At work we use chef to deploy our credentials via KML flat files on the servers that require them. Works rather well.

<comment><USER5> What are KML flat files? And how are you maintaining them without keeping the secrets in plaintext in source control?

</comment></comment><comment><USER11> Why no key rotation? I'd be very careful with something that doesn't rotate keys.

<comment><USER12> Keys: KMS handles rotation of the master key automatically ([http://docs.aws.amazon.com/kms/latest/developerguide/rotate-...](http://docs.aws.amazon.com/kms/latest/developerguide/rotate- keys.html)). The ephemeral key used to encrypt values is changed any time a value is set.

Values: [https://github.com/dcoker/biscuit#how-do-i-rotate-the- values](https://github.com/dcoker/biscuit#how-do-i-rotate-the-values)

</comment></comment><comment><USER13> I feel like this problem is already solved with iam ec2 instance roles

[http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam- roles...](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for- amazon-ec2.html)

<comment><USER14> You need to look at this tool and understand what it does.

<comment><USER15> Fair enough, but IAM roles does fit all of the requirements listed in the "is biscuit right for me?" section.

</comment></comment></comment><comment><USER16> Somewhat off-topic, but I read dcoker's username as docker at first and was fairly confused as to why docker was producing something like this just for AWS.</comment>


8 reasons why I think Nozbe is the best GTD to-do list app - <AUTHOR> https://koprowski.it/2020/nozbe-the-best-gtd-to-do-list-app/ <comment><AUTHOR> Hi this is my first attempt to affiliate marketing. I wanted to provide real value to my readers and not only sell some product.

I will appreciate any feedback :)</comment>


CISPA is not SOPA - <AUTHOR> http://overhack.wordpress.com/2012/04/09/cispa-is-not-sopa/

<comment><USER2> > I do not have an attorney's license, or even a law degree,

And therein lies the trouble with the analysis. It is factual, but performed from a perspective of an engineer. An engineer not looking for loopholes and not looking for how to stretch the meaning of the broad wording to attach it to something else entirely.

Attorneys are paid to find the loopholes, and to stretch the meaning of the statutes to fit the desires of their clients. Broad sweeping definitions just make it infinitely easier to stretch the bill's language to make it apply to unrelated areas.

If the bill was honestly and truly meant to be what it purports, then the language would be nice and tight, with no wiggle room to stretch it to fit something else.

For example, here's passage (B)(1)(i) of the bill:

> `(i) use cybersecurity systems to identify and obtain cyber threat > information to protect the rights and property of such protected entity; and

The bill does not define "cybersecurity systems" - leaving this term open to be just about anything. Nor does the bill define "cyber threat information" - leaving this term open to be just about anything. So long as they are "protect[ing] the rights and property" of a protected entity.

So, with this broad language, a deep-level packet inspection system sniffing every ISP's users data streams looking for purported copyright infringement would be perfectly legal, because it would be a "cybersecurity system" protecting against "cyber threats" in order to "protect the rights and property" of a "protected entity". Where "rights" is the copyright and the protected entity is a MPAA/RIAA member, and the "cyber threat" is that of supposed lost profits from supposed copyright infringement.

That's the trouble with broadly worded bills. They allow attorneys to stretch them to fit something they were never intended. Read up on "unintended consequences": <http://en.wikipedia.org/wiki/Unintended_consequences>.

<comment><AUTHOR> Except nothing in the bill obliges the ISP to monitor for copyright holders, which would run significant costs for them both in terms of systems and personnel. The ISP is not a cybersecurity provider for rights holders under the definitions of the proposed statute.

Please don't be condescending in actually linking a Wikipedia article on unintended consequences, by the way. That's not necessary for civil, rational debate.

<comment><USER2> [http://www.techdirt.com/articles/20120410/12180518442/cispa-...](http://www.techdirt.com/articles/20120410/12180518442/cispa- is-really-bad-bill-heres-why.shtml)

<comment><AUTHOR> Another flawed analysis. Again, while imperfect, this bill simply does not contain any provision that obliges anyone to do anything. I don't disagree that it can be improved, but CISPA isn't even close to SOPA: no blocking requirements, no facilities for intervening with advertisers or payment providers, etc.</comment></comment></comment></comment>


Forget the iPad 3 - Apple is Launching the iPad HD - <AUTHOR> http://mashable.com/2012/03/05/ipad-hd/

<comment><USER2> It does not really matter what the name is.

Having a Retina display iPad is a big achievement in itself. No matter what they call it, it is still going to beat all other tablets this year.

They launched iPhone 4S when everyone was saying that it is time for iPhone 5. Who knows what they are going to do.

</comment><comment><USER2> [http://thenextweb.com/apple/2012/03/06/why-the-next-ipad- wil...](http://thenextweb.com/apple/2012/03/06/why-the-next-ipad-will-and- wont-be-called-the-ipad-hd)</comment>


Latest advancements in online video streaming with deep learning - <AUTHOR> https://blog.peervadoo.com/2020-video-streaming-ai-p2p-technology-advancements.html <comment><AUTHOR> Hello, I am the author of this article

Netflix and Amazon had to reduce their quality from HD to SD amid pandemic

Covid19 has accelerated the consumption of online video

Video streaming platforms are facing pressure due to the enormous demand

There is a need for the streaming platforms to use the latest technologies to catch up with the continuous growth

In this article I discuss various techniques which can help in scaling the video streaming platforms, improving user experience and reducing latency with the help of AI, P2P, Multi-CDN, CMAF and Per-title encoding and the background of how these technologies work

Did I miss out any recent other technologies in video streaming. Let me know in comments</comment>