Build a fast deep learning machine for under $1K - <AUTHOR> https://www.oreilly.com/learning/build-a-super-fast-deep-learning-machine-for-under-1000 <comment><USER2> Contrary to the claims there, the CPU does matter: if the network is relatively small, it's easy to get bottle-necked feeding the GPU.

Same thing applies to storage, for the same reasons. There's a reason AMD is selling deep learning cards with Flash drives built in.

The choice of CPU cooler is silly. The CPU chosen is a 65W model, which will be cooled quietly even by the boxed cooler (and we don't need to care about overclocking with a non-K). By his own admission, the rig draws 250W, so the idea that 650W must be cooled is nonsense, and even then, only a part of that is due to the CPU.

It's probably worth looking more closely at the GPU, notably, if that cooler stays quiet when the card is under full load.

You do not need a monitor, keyboard or mouse to use such a machine. You can just ssh into it. You might need to borrow a set to set up the BIOS.

If you "run out of memory for your applications" on the GPU, have you considered simply lowering the mini-batch size? (See, this is why we got a good CPU, it removes the hurt from feeding smaller batches)

<comment><USER3> _There 's a reason AMD is selling deep learning cards with Flash drives built in._

For those not familiar with them, this is the Radeon Pro SSG[1]

This has a built in M2 drive, but this is used as video card memory, NOT main storage. It could possibly allow big (HUGE) models or batches, but doesn't prove fast IO is often a bottleneck.

Practically no one[2] is using these for deep learning. They are marketed to the oil and gas modelling and visualization market and notably the product page (linked above) makes no mention of neural networks or deep learning, and it has never been mentioned in AMD's publicity.

Yes, it _might_ be good for deep learning. And yet, modern deep models do want more memory. But there is a lot that needs to happen for this to be useful: AMD needs to release something like CuDNN and they need to make sure OpenCL is supported as well as CUDA is.

[1] [https://arstechnica.com/gadgets/2016/07/amd-radeon-pro- ssg-g...](https://arstechnica.com/gadgets/2016/07/amd-radeon-pro-ssg-graphics- card-specs-price-release-date/), [http://www.amd.com/en-us/press- releases/Pages/amd-radeon-pro...](http://www.amd.com/en-us/press- releases/Pages/amd-radeon-pro-2016jul25.aspx)

[2] I'm sure you'll find someone. But show me a published paper or any kind, or even benchmarks showing the use of the extra memory somewhere.

<comment><USER4> MIOpen is AMD's cuDNN alternative.

<comment><USER5> Which does not actually exist. It's announced but unreleased.

</comment></comment></comment></comment><comment><USER6> My home server has an i7 4790K, 32GB DDR3 and... a GTX 1080 Gamer Edition that's doing... well, bugger all. Why is the graphics card in there you might ask? Well, it was sitting on my desk collecting dust as I wait patiently for a Pascal driver for eGPUs on macOS but Nvidia had delayed these for long enough that I thought I'd chuck it in my home server, it is being used by ffmpeg for x265/HVEC transcoding with Plex media server but I'm really not even touching it's capabilities - not even close when transcoding a 120GB+ 4K 120FPS video and streaming it wirelessly to my TV that's running a Plex client. The server is running CentOS 7 and has 10x 8TB SATA drives in BTRFS RAID 10 for storage, two Intel NVMe 750 series 1.2 TB SSDs for the OS as well as a couple of 2TB Crucial MX300 SATA SSDs that aren't currently in use for much.

So, I have been sitting here wondering how I can make better use of it while it's practically parked in my server while I wait for some Pascal drivers to drop. I'm not so interested in robots / live video processing but it'd be neat if I could leverage its power for something fun or to assist the servers compute performance in some other ways that might be useful or interesting. I was running KVM with PCIe pass through passing the GPU to a guest VM running Steam as a steam streaming box but I got sick of having an OS on the network that I had to worry about Virus's, Malware and annoyingly slow and unreliable updates being installed when you least want it to happen and I got lazy and went back to playing PlayStation when I needed some time out from constructive work / research.

Open to ideas to any software I could tinker with if anyone has any interesting suggestions?

<comment><USER7> Can you ballpark today's price tag for such a setup? There seem to be several pricing tiers floating around in this discussion and I'd appreciate the chance to see how your setup fits in without having to shop for it.

<comment><USER6> Well it's all cheap kit with the exception of the disks, I guess you could just go online and have a look at what those parts cost you in your country etc...

</comment></comment></comment><comment><USER8> I think there should be a disclaimer here that while, yes, this will _work_ , it will quickly become suboptimal (perhaps even dysfunctional) for a lot of work as you scale your hobby into something resembling more professional work. I personally would not start with a thousand dollar machine if you have any intention of doing serious computation in the future. While you can trade out parts pretty easy on a custom built machine, you don't want to be in a position where you have to trade out most of the core parts 6 months in.

For reference, I built a home PC that I successfully do deep learning and data analysis on (mostly tensorflow and scipy stack) for about ~$10k. It's liquid cooled, has 15 fans, four radiators, an i7-6900K CPU, 128GB RAM, four GTX 1080 GPUs (controversial), four TBs of HDD space and 1TB of SSD space. I don't recommend you _start_ with this at all, but my point is that porting your hardware from point A to point B will be a pain if it comes to it.

I used the guide here as a reference about 8 months ago when I built it: [http://graphific.github.io/posts/building-a-deep-learning- dr...](http://graphific.github.io/posts/building-a-deep-learning-dream- machine/). My purpose in doing this was, essentially, to pay for electricity rather than AWS/GCP/Azure compute resources (and in that regard it's been very successful!).

I know I'm hijacking a thread here to talk about building home machines for professional deep learning work when this story is clearly not intended for that, but I wanted to throw in this perspective so that it's understood this is very different from just "build this machine to start out and upgrade it later." There's a law of diminishing returns here, but in general my point is that I do _not_ think this is a minimum for "start doing deep learning effectively at home." If you want to learn hands on deep learning cheaply, my opinion is that it would be more efficient to use compute resources from a cloud provider before diving into this with a home-based custom machine.

tl;dr: The demographic of folks who probably want/should/need to build a home deep learning machine probably has little overlap with the demographic of folks who want to do it non-professionally, or at least with only $1k in resources.

<comment><USER9> On the other hand, by the time anyone starting just now learn enough ML to be dangerous, the price of the $10,000 hardware would drop to $5,000 or less, so starting with the cheapest yet decent option seems a better approach...

<comment><USER8> Fair point. I'm just quibbling about where "decent" begins, and adding in the point about ease of upgrades. Replacing a GPU is pretty easy. Replacing a motherboard because you don't have enough PCIe slots will require tearing down the majority of the machine and rebuilding it.

The latter is more likely to happen if you start down this path at rock bottom prices.

<comment> lloyd-christmas I began building one a year and a half ago, starting with a 5930k, 1x980ti, 32GB. This put me in a position to incrementally upgrade it starting at ~$2.5k. That being said, at this point the smallest AWS p2 instances are equivalent to my current setup, and it would take 150 days worth of running my hardware to make it cost-effective vs. AWS.

<comment><USER10> When you build your first DL box you also get a moral boost. If you think every second is counted and penalized against you, you might use it less (such as when you're using cloud servers). But if it is already yours bought and paid for, then you can afford to run any crazy idea. A good compromise would be a local DL box and extra virtual machines on demand, for when you need to run extensive hyperparameter searches. If you make the leap and invest, at least you commit your money into the direction of ML, strengthening your resolve.

<comment> lloyd-christmas On the flip side, dropping $2.5k for a hobby you haven't investigated yet and then looking back 6 months later on the computer collecting dust will likely dissuade you from investing any time into potential future hobbies. I play piano, and I'd never encourage anyone to go drop $2.5k on an electric keyboard just to provide a motivation. A decision to pick up a new hobby/skill should be based on desire, not a handcuff.

</comment></comment></comment></comment></comment></comment><comment><USER11> If I were building a GPU box on the cheap, I'd start with a used Dell Precision T7xxx series off of Ebay for a <$300 including RAM and a Xeon or two. They have 1100 watt power supplies and lots of slots and the RAM is usually ECC. Odds are it will come with DVD burner and possibly a hard disk and a Windows license...again all for a couple of hundred dollars.

Leaving $700 for GPU's while providing reliable high end hardware pretty much designed to run GPU's for the base platform.

</comment><comment><USER12> I'm not sure why mITX was chosen - in my experience it's almost always more expensive and leaves no room for additional GPU expansion (Or network adapters/whatever card you might need) in the future. You can get 1151 mATX motherboards for more like $60-70 on Amazon instead of the $125 they paid.

Also, paying $125 for 16GB of ram but not spending the extra $20 to have your CPU be able to overclock? I'm not sure where OP is building but I can find that much RAM for $20 less than they paid, and overclocking isn't really that difficult or unreliable these days.

<comment><USER13> I was wondering that too. For something like this maximum room for future expansion seems good. In the past I've used mITX to build low-power, low-noise machines, rather than building a high-powered processing beast.

On the other hand, it made the article more interesting for me. I can't really justify a home machine learning rig, but I was thinking I might soon replace that old mITX machine.

(edit - just reached the pics - really not sure why you'd go with a mini form factor and then add a 6" heatsink/fan!)

<comment><USER12> I also prefer mITX and I have one I've tried to optimize for low noise and efficiency. Just seems ill-suited for this purpose.

</comment></comment></comment><comment><USER14> In Europe it's also very common to just choose the components in a web shop and have them assembled, also for about $1000. (Not looking at Dell)

Anyone interested in deep learning should go that path instead of burning their money on Amazon.

<comment><USER15> +1 on that. There are a number of online build-to-order shops here in the UK which offer a truly vast range of component combinations. Better still, the order process automatically checks for compatibility with, for example, your choice of motherboard.

Once you have the configuration you want, you just pay up and it arrives professionally assembled ready to rock. I don't think I'll ever by an off-the- shelf machine again.

<comment><USER16> I found the premium on these was quite high, although even just buying all the components from a single place added a fair amount to my recent build.

Compatibility, price checking and searching are all available with PcPartPicker (UK site: [https://uk.pcpartpicker.com/](https://uk.pcpartpicker.com/)) which saved me an astonishing amount of time.

The hour or two building the machine were worth the few hundred it saved me, but everyone has different priorities.

<comment><USER17> I've found that the premium depends on the store and model. Some seem to do relative pricing, others absolute. In the latter case, if you want a big rig it's best to let someone build it for you.

Often times in those cases you also get great service because a) if you build PCs for a living you're a computer geek and it's fun to build an insane PC and b) they often use their biggest systems as advertisement. At least, that's what I've seen.

+cable management is like black magic to me. If I were to take the money saved as payment for me to get it as nice as those places get it, I'd be below minimum wage.

</comment></comment></comment></comment><comment><USER18> I've been considering a "thing" that allows monetization of an AI-enabled machine by others who need the capacity to do larger scale testing. Anyone interested in this? My contact info is in my profile.

<comment><USER19> You won't be able to match price/expenses (fixed with hardware or variable with upkeep) of AWS/GCP due to their economy of scale. Even startups who are doing similar things sublicense from AWS.

</comment></comment><comment><USER20> But how do you scale cheaply to more GPUs?

<comment><USER21> Get a ATX instead of ITX mobo. It's likely is cheaper as well.

</comment></comment><comment><USER22> If by "super fast deep learning machine" you actually mean, mid tier gaming computer. Uh, sure. Don't spend more on an ITX board, when you can fit 4x to 7x the video cards in a full size ATX computer using pci-e risers, while paying less for the board.

Don't lose 5 percent of your cpu overclock or more by having an itx friendly cooler instead of a D15 noctua or better.

Choose your video card wisely, AMD does some things better than NVIDIA depending on what software you're running.

If you chose an ITX platform for deep learning fun, uh, you should really add the couple shoeboxes extra space to your platform and have 4x-7x the power available to upgrade into.

<comment><USER23> Try to bring good in the conversation.

What piece you would have use? Why? What tradeoffs you have considered?

You sound expert in the field, try to share your knowledge with the community in a constructive way so that we can all benefit from it.

I develop software for living and still I haven't any clue of what you said, while I followed quite well the article and I thought he was making reasonable choice.

Please show me better.

<comment><USER24> Not parent, but I've built my own computers for almost 20 years.

As the articles author himself questions, he should have gotten the marginally more expensive CPU, and definitely the GPU with more RAM.

* The overclockable CPU doesn't just mean that you very easily could get a 10%+ performance boost without much work, but also that you often (depending on your specific chip) can lower the voltage and make it run much cooler/quieter, which is something I increasingly care about if I'm using it a lot.

* He writes that fitting the model in RAM is basically the most important part, but then saves less than a dinner out by basically halving the RAM on the GPU he bought. The chip is otherwise the same though, so performance is only dependent on whether you fill that RAM or not.

* The Noctua D15 the parent mentioned is a CPU cooler where Noctua is a long standing high performant brand, and D15 is a specific model with a 15 inch fan and comparable sized heat sink. There are of course other brands, but I myself also usually end up with Noctuas. The reason it's important is that however fast you can dissipate heat from the cpu/case, the less chance of throttling, and the larger headroom for potential overclocks you get.

Airflow and room for larger heatsinks is also why he recommended not going for an ITX. A linked benefit is again the potential for a quieter system.

I haven't gone much into ML (yet), but I currently have a system with:

* I7 6700K (the difference to I5 6600K being higher base clock and hyperthreading, which is more important to computational work than to gaming, so if you have the money, definitely go for the I7)

* 32GB DDR4 (as author mentioned, RAM is cheap). The clock/timings on RAM isn't really as important, but try to find the best you can find for a given price point.

* An Nvidia GTX 1080: It's not Titan X or Z, but almost, at less price. It definitely blows the budget for a $1000 system, but I agree that the entire 10 __series is good.

If the limit is a firm $1000, I would get something like this:

[https://pcpartpicker.com/list/XHV9Fd](https://pcpartpicker.com/list/XHV9Fd)

And if more funds is available, I'd get more storage and RAM, then a better CPU, then a better GPU, then maybe bump the chassis up to an R5 (same brand), possibly another motherboard. In that order. There's always something better, so you compromise based on budget.

<comment><USER25> Was years ago when I build my last PC and I'm out of the game.

I'm tempted in build a hackintosh:

[https://www.tonymacx86.com/threads/hackintosh-cutting- edge-k...](https://www.tonymacx86.com/threads/hackintosh-cutting-edge-kaby- lake-nvme-ddr4-4k-viable.214249/#post-1433129)

\- Intel i7 Kaby Lake \- No decided on motherboard. The one that cause me less trouble (for hackintosh) is fine. \- GTI 750ti (have) or buy a pascal nvidia. \- NVMe drive if possible \- 32 GB RAM. \- Probably a Thermaltake CORE P3 case. Not decided.

I was thinking in use a Liquid Cooler but wonder if the Noctua could be better/less noise?

<comment><USER24> > I was thinking in use a Liquid Cooler but wonder if the Noctua could be > better/less noise?

Depends, is your option of water cooling an All-In-One-solution that have become popular in recent years? Their performance is on par or slightly better than a large heat sink + large low speed fan(s), but they're not generally quieter, as you still have fans for them, as well as a pump.

I considered those options when building mine too, and as I wans't too enthusiastic about assembling my own water cooling system, I went for a large air cooled heatsink instead (the D15). No risk of leakage or pump failure, and proven performance/low noise.

<comment><USER25> I was planning in use Corsair Hydro H60 ([http://amzn.to/2k9a9O1](http://amzn.to/2k9a9O1)).

However, I wish to have a quiet system, and my brother have it and it sound louder than I wish. I don't plan on overcloking.

<comment><USER24> Yeah, that's one of those AIO water coolers I was referring to.

[http://www.anandtech.com/show/5054/corsair-hydro- series-h60-...](http://www.anandtech.com/show/5054/corsair-hydro- series-h60-h80-and-h100-reviewed/6)

The "Silver Arrow" is an air cooler from Thermalright that is pretty equivalent to a Noctua or a Phantek or be Quiet! etc.

In that review it beats the H60 in both temperature and noise. The H60 is more than twice as loud.

So yes, unless you assemble your own water cooling system, I'd say definitely go for a regular heatsink+fan.

</comment></comment></comment></comment></comment></comment></comment><comment><USER26> If you're on a budget it makes little sense if any to buy things like a motherboard, CPU or memory new.</comment>


Angie's List Takes a Wallop on Wall Street - <AUTHOR> http://www.technewsworld.com/story/75912.html

<comment><USER2> May or may not be causal - but Angie's List painted a huge target on themselves for liberal/progressives when they refused to drop their ad spots on Rush Limbaugh's show after his harsh remarks about Sandra Fluke.

I see Angie's List as having done what App.net is attempting to do - a curated place that exists despite other venues/sites being free... just that recently they've been scaling heavily and have likely run into issues there.

</comment><comment><USER3> Good riddance. Angie's List was one of -- if not the -- first companies to resume advertising on Rush Limbaugh's show after the boycott.</comment>


ARM: Starter Kits & OEM Modules - <AUTHOR> http://dev.emcelettronica.com/arm-starter-kits-oem-modules <comment><AUTHOR> Starter Kit is very useful to learn and develop software on ARM. I merged Starter Kit and OEM module information, because all OEM modules supplier will offer corresponding Starter Kit or evaluation board for design startup.</comment>


Facebook Patent: Socioeconomic group classification based on user features [pdf] - <AUTHOR> http://pimg-faiw.uspto.gov/fdd/83/2018/28/003/0.pdf <comment><AUTHOR> Facebook patent application (US 20180032883) for "classifiers input information about a user and output a probability that the user belongs to a given socioeconomic group".

Relevant Twitter thread: [https://twitter.com/WolfieChristl/status/960630738256367617](https://twitter.com/WolfieChristl/status/960630738256367617)

Copy-and-paste friendly version of the patent application, but without images: [http://appft.uspto.gov/netacgi/nph- Parser?Sect1=PTO1&Sect2=H...](http://appft.uspto.gov/netacgi/nph- Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220180032883%22.PGNR.&OS=DN/20180032883&RS=DN/20180032883)

Limited news coverage, e.g., [http://expressnewsline.com/2018/02/05/facebook- to-develop-sy...](http://expressnewsline.com/2018/02/05/facebook-to-develop- system-to-track-users-socioeconomic.html)</comment>


Ruhoh another static blog generator. from the creators of Jekyll Bootstrap  - g-garron http://ruhoh.com/

<comment><AUTHOR> I maintain ruhoh and jekyllbootstrap, thanks for all the support guys. Will be monitoring this thread for any questions. =)

<comment><USER2> I must say _Ruhoh_ API is very well thought out. Kudos for the great work!

How about extending the Universal Blog API for storing user comments? That'd make it a complete out-of-the-box blogging solution (a user may opt for using a comments service provider like Disqus OR use the inbuilt comments API). The spec could look somewhat like so:

\- Directory Structure:

              _comments //folder with json files containing user comments

\- Comments file:

              {{post identifier}}.json //could be simply the post filename.

\- Comments structure:

           {
               "comments" : [{
                             "id" : "1123",  //unique comment id
                             "author" : "Chuck Norris",
                             "author_handle" : "firstname.lastname@example.com", //or a website
                             "date" : "yyyy-mm-dd", //comment date
                             "content" : "Hei! Chuck makes dumb comments."
                             },

                            {
                             "id" : "1124",
                             "author" : "Superman",
                             "author_handle" : "superman.com",
                             "date" : "yyyy-mm-dd",
                             "content" : "Superman is scared of Chuck."
                             },
                   ]
           }

We can leave the comment id and author details up to the login implementation.

</comment></comment><comment><USER3> What is the license for ruhoh?

I know it is none of my business but if you are looking for opinions I would kindly say "please do not use "Attribution- _NonCommercial_ -ShareAlike - BY- _NC_ -SA 3.0)." I wanted to use jeklybootstrap for my fledgling one man consulting shop but I could not because of the license....

EDIT/UPDATE:

I just saw this at the bottom of the ruhoh page:

 _"All my work is Unlicensed, Open and Free."_

Its a little confusing seeing that the jb license says otherwise...

<comment><AUTHOR> Yeah sorry, actually I updated JB license to Unlicense shortly after I launched it. You are right, it was only because I didn't realize that Creative Commons prohibited commercial use that I chose it in the first place.

Jekyll Bootstrap, Ruhoh are completely 100% Unlicensed, or MIT or whatever is the most permissive license available.

I want you to use my work to benefit as much as you can Douglas, I'll even email you personally, waiving all rights for your own records. Just shoot me over your email.

I love Open Source and I don't want to restrict any body in any way. Thanks for pointing out this confusion!

<comment> X-Istence The way you could go about this is by stating that it is in the public domain, and for those countries where the public domain doesn't exist that it is under an MIT license.

At that point you have covered your bases and I and many others would feel comfortable using/extending/developing with it.

Sqlite is public domain for example.

<comment><AUTHOR> I'll probably try that then. Initially I chose Unlicense because it addresses the "no license" means someone can go "over your head" and license it. The whole licensing thing is a pain in the ass. And now I see <http://unlicense.org> isn't even resolving anymore!

I like the idea of Public Domain with fallback MIT. The main thing is I want it to be un-copyright-able.

Thanks for the suggestion and do feel free to check out the development progress on github!

</comment></comment></comment></comment><comment><USER4> I find this Universal Blog API ( <http://ruhoh.com/universal-blog-api/> ) quite interesting. And some file names like '20012-10-10-my-draft.md' seem to point to the year 20012? :)</comment>


(Programming) Advent Calendars: Big in Japan - <AUTHOR> http://weblog.bulknews.net/post/13584639153/advent-calendars-big-in-japan <comment><AUTHOR> Not much to see at moment :) And especially if you don't speak Japanese!

NB. Google Translate does a pretty good job though it does often screw up the code a little bit.

Thought it was interesting to see the quantity and diversity. And I suspect there will be a lot of goodies to look back at come end of month.

</comment><comment><USER2> The article is quite wrong, and it's another example of the myth of "those wacky Japanese" - this is clearly a coding competition.</comment>


Start-Up Company Will Pay You to Read E-Mail - <AUTHOR> http://www.foxnews.com/story/0,2933,303636,00.html <comment><USER2> [http://www.rhyolite.com/anti-spam/you-might- be.html#e-postag...](http://www.rhyolite.com/anti-spam/you-might- be.html#e-postage)

</comment><comment><USER3> Oh shit the stupid 90s are back. Welcome bubble we greet you you are our friend.

</comment><comment><USER4> haha! my dad tried to start a company in the late 90s with this exact same model.</comment>


Ask HN: Internet during PG&E outages – ideas? - <AUTHOR> So, PG&E is cutting off my power again. And Comcast usually lasts only 30 mins after the power goes (despite me having generator power at the house).<p>I'd love to pick the community's brain on what I should look at for backup here. Cell phone coverage is crappy on a good day, and goes away when power goes. Thinking about satellite, LOS wireless broadband - anything else I should consider? <comment><USER2> Cell companies claim to be improving power back ups. If you aren't on verizon now i'd try one of their hotspots (better coverage in most areas).

Are there microwave providers i your area and do you have LOS to one? I'm one in Montana and it is great though it does go down in power outages but our power only ever goes out for a few minutes.

<comment><AUTHOR> Claim to, but lots of them seem to crap out where we are. Will check a couple when it goes down

</comment></comment><comment><USER3> Dial-up will work during a power outage, so long as you're not using VOIP.

<comment><USER4> It will only work as long as the phone company's backup batteries or generators last.

And while dial-up speeds are adequate for e-mail, trying to use today's bloated web sites over a dial-up connection could be rather frustrating.

But really, how many ISPs still support dial-up access today?</comment></comment>


The Mueller Report, Searchable and Accessible on the Archive - <AUTHOR> https://blog.archive.org/2019/04/26/the-mueller-report-searchable-and-accessible-on-the-archive/ <comment><USER2> > The government initially released the document in a PDF format which renders > it like an image, impossible to search. When PDF files are uploaded to the > Archive, we automatically run them through an Optical Character Recognition > (OCR) process. This turns those images into text, making it much easier to > move between sections and search for specific words or phrases. This allows > journalists and the public to more easily parse through volumes of > information contained within these massive documents.

Shouldn't an accessible, searchable and digital version of government documents be the default form in which they are released?

Why do citizens have to resort to the Internet Archive to see such a basic right fulfilled?

In my country this happens frequently, as well. Too often, documents that should be accessible and widely publicized are hidden somewhere on the internet with practically no incoming links and in a form which is the digital equivalent of a grainy fax.

Sometimes the intention of superficially fulfilling a transparency obligation, without actually communicating anything is palpable.

<comment><AUTHOR> One possibly is because they want to make sure the redacted sections are really redacted. You can do a web search and discover that others have tried to redact sections of PDF documents, only to later discover that people can uncover the redacted sections via software tools.

Scanning a redacted document is the simplest way of making sure redacted sections stay redacted as there is no underlying information to reveal.

<comment><USER3> Releasing the plain text is all that's needed. I fail to see how redacting by overwriting the relevant text is at all reversible - e.g. replace every redacted word with five X's. Replace multiple redactions with ten X's. That actually seems more secure than what's currently done, where you can guess at the words by their length.

<comment><USER4> > Releasing the plain text is all that's needed.

I doubt Mueller's team wrote the report in plain text. They probably used Microsoft Word and exported a PDF document.

When I'm president, all government agencies will be forced to use LaTeX. Then we can have both plaintext source, PDFs with actual text, and PDFs that don't leak redacted material.

<comment><USER5> LaTeX? That will effectively prevent most government agencies from writing reports.. oh, that was your goal, wasn't it? Well played.

</comment></comment></comment></comment></comment><comment><USER6> I really like this OCR version! I did a search for "Manafort" and it not only tells you where in the docs the keyword pops up with nice location markers, but also a snippet of surrounding text to give the keyword context. I do wish there was a low-level cache of commonly used keywords so that some searches would be faster, but I don't know what archive.org's position is on tracking user behavior (probably not a fan).

<comment><USER7> One can't be forced to divulge data on users one does not record in the first place.

</comment></comment><comment><USER8> Thanks to this OCR'ed version, I was able to put an NLP text summarizer of the document into production yesterday. Check it! [https://news.ycombinator.com/item?id=19815506](https://news.ycombinator.com/item?id=19815506)

</comment><comment><USER9> Here are few other options:

Web based and searchable

[https://www.nytimes.com/interactive/2019/04/18/us/politics/m...](https://www.nytimes.com/interactive/2019/04/18/us/politics/mueller- report-document.html)

Original document has been updated so it can be searched

[https://qz.com/1601873/the-pdf-of-the-mueller-report-has- bee...](https://qz.com/1601873/the-pdf-of-the-mueller-report-has-been-updated- to-be-more-accessible/)

</comment><comment><USER10> Only 7 views since April 22?

</comment><comment><USER11> At this point the whole thing feels like Iraq 2.0 and I wonder if I'm getting suckered by media.

What only worries me as a progressively-disenchanted but firmly and forever a supporter of Bernie Sanders is that even sanders seems to have been wrong on this one unlike Iraq, which I would shout from rooftops that he was the only politician right about. I would (and still do) quote his 'What are you afraid of' video to all who would listen...

The question is just why?

<comment><USER12> I'd be interested in what Chomsky would have to say about it. Manufacturing Consent seems more relevant than ever.

<comment><USER13> He's said several things about it, and has trashed the Russiagate conspiracy. He emphasizes both that the US interferes constantly in other elections, and also points out that Israeli collusion and interference in the US is orders of magnitude worse than Russian influence:

[https://www.realclearpolitics.com/video/2019/04/01/noam_chom...](https://www.realclearpolitics.com/video/2019/04/01/noam_chomsky_trump- russia_collusion_claims_a_joke.html)

<comment><USER14> I think you're misinterpreting him. He doesn't say that it's not a bad thing, or that it's a hoax. He simply says that the US does the same thing abroad and it's hypocritical to not care about that part of it.

<comment><USER13> No, I don't believe I'm misinterpreting him. He is clearly showing heavy disdain for the assertion that the Russians had any material effect on US elections. He is also pointing out that there is the much greater issue of Israeli collusion that everyone, including yourself, is ignoring when the Russia collusion narrative is a "joke" (his words). Everyone please watch the video for yourselves. It's short.

</comment></comment></comment></comment></comment><comment><USER15> The search function seems to be broken. I searched for instances of "Russian collusion" and waited for two years for the results and it came back with zero instances of Russian collusion found...

<comment><USER16> The word collusion has no meaning in a legal sense. Try searching for obstruction of justice.

<comment><USER17> [self-censored]

<comment><USER18> False.

"The Mueller report does establish that, in fact, members of the Trump campaign conspired or coordinated with the Russian government in its election interference activities."

[https://www.nytimes.com/2019/04/25/opinion/mueller-trump- cam...](https://www.nytimes.com/2019/04/25/opinion/mueller-trump-campaign- russia-conpiracy-.html)

Edit: Since the comment I am replying to is no longer present and I got downvotes for my comment: the person claimed there was "no evidence [anyone] conspired with Russia" so I linked to an article in the NYT by a law professor that contradicts this claim in detail. No evidence _is_ a false claim.

<comment><USER19> Note this link points to an OpEd.

At least I believe the NYT isn't so far gone as to publish such a claim outside of the editorial section, but I may be wrong.

For a different perspective, read what Glenn Greenwald has to say on the subject, in a piece titled "Robert Mueller Did Not Merely Reject the Trump- Russia Conspiracy Theories. He Obliterated Them." [1]

Edit: I actually took the time to read the editorial. It is a collection, in my opinion, of dubious claims about standards of evidence being too high as the reason Mueller and his massive team could not bring a single solitary charge against Trump or the campaign related to Russian interference in the election.

Please remember how this all started, with a dirty dossier obtained from the Russians by the DNC, with a counter-intelligence operation claiming without evidence the President could be a Russian patsy. And yet here we are, still entirely unable to expose truth to the lie.

[1] - [https://theintercept.com/2019/04/18/robert-mueller-did- not-m...](https://theintercept.com/2019/04/18/robert-mueller-did-not-merely- reject-the-trumprussia-conspiracy-theories-he-obliterated-them/)

<comment><USER20> >Please remember how this all started, with a dirty dossier

Nope, it began when some foreign diplomats gave us a heads up that George Papadopoulos was bragging abroad that the Russians were going to help Trump by releasing dirt on Clinton. That the investigation was predicated on the dossier is a common misconception. If you are getting news from sources that get this basic fact wrong, you may wish to downgrade those sources' credibility in your estimation.

<comment><USER19> That happened in "late May 2016" right around the time that Fusion GPS hired Steele.

Note that Steele and Bruce Ohr were already in cahoots for several months by then, and Steele was already established as an CI with the FBI (a cover he later blew).

Papadopoulos met Professor Mifsud in London in March, and later Misfud said he had ties to the Kremlin and he claimed Russia had dirt on Clinton. Papadopolous sent an email two weeks after the London bar meeting saying Misfud introduced him to the Russian Ambassador to England and that the Ambassador might be able to help setup a meeting between Trump and Putin.

It was what Mifsud told Papadopolous that he would later talk about with the Greek or Australian diplomat that would be reported back to the FBI.

Interestingly, there are reports that Mifsud is actually a western intelligence officer, and has no links to the Kremlin whatsoever.

<comment><USER20> Yes, Steele had good relationships with our intelligence community. I don't see how that bears on the beginning of the investigation.

I haven't seen anything actually credible that Mifsud was western intelligence.

<comment><USER19> Chris Steele (an agent of a foreign government) was dead-set on preventing Trump from becoming President.

He had high level contacts at the FBI and DOJ which he was in contact with throughout early 2016.

Hopefully one day it will become fully clear what Steele was telling Bruce and Nellie Ohr, up until the very day before the FBI opened their investigation, when Nellie met Chris at the Mayflower Hotel on July 29, 2016.

But we do know at least that it was lies, mostly (Russian?) lies, that Steele was feeding the FBI. And we know certainly those lies had a bearing on the character and direction of the investigation, and on the methods employed to attempt to gather evidence (FISA), even if it is not the publically admitted initial basis for starting the investigation.

Mifsud, for his part, has gone to ground and isn't taking interviews.</comment></comment></comment></comment></comment></comment></comment></comment></comment>


Modular Typechecking for Hierarchically Extensible Datatypes and Functions - <AUTHOR> http://www.cs.ucla.edu/~todd/research/eml-toplas.pdf

<comment><AUTHOR> This is an awesome paper on language design that demonstrates support for open algebraic types and extensible functions, all with modular type checking. This is kind of a thing that could perhaps open the algebraic types in Haskell.</comment>


A first look at Ubuntu desktop metrics - <AUTHOR> https://blog.ubuntu.com/2018/06/22/a-first-look-at-desktop-metrics <comment><USER2> _> Full HD (1080p) is the most popular screen resolution, followed by 1366 x 768, a common laptop resolution. HiDPI and 4k are not yet commonplace. 4k screens have been around for a while now but are often priced much higher than the full HD counterparts so the numbers are not entirely surprising._

There is a bit of a chicken-and-egg problem here, though: Ubuntu (or rather, Linux in general) doesn't have a great reputation for consistent, high-quality HiDPI support, so there's less incentive for its users to buy HiDPI displays.

<comment><USER3> Ubuntu and Gnome handles _doubled_ resolution just fine, much like the retina MacBooks.

It's the fractional scaling required for all the other resolutions in between which is troublesome, and I believe that applies to Macs too.

In this space, I think the only contender doing it _really well_ is Microsoft Windows.

That said, I prefer Linux. For me having a few warts here and there is an perfectly acceptable price to pay for having a full computing environment I can fully trust and adapt to my needs.

<comment><USER4> > Ubuntu and Gnome handles doubled resolution just fine, much like the retina > MacBooks.

Not if you have a secondary monitor that doesn't need doubled, e.g. laptop built-in 4k with a 1920x1080 external. As far as I know this is an X problem.

<comment><USER5> It is an X problem, there's (sorta) support for fractional scaling in Wayland

<comment><USER6> It's also an app issue. I run a 4K monitor on Ubuntu and the majority of apps take the entire screen when they start or open a window. And in many apps I have to tweak things so it becomes readable.

</comment></comment></comment></comment></comment><comment> DC-3 Worth noting that these days, Will Cooke is a host on the Late Night Linux podcast - so if this interests anyone, they might be interested to listen in as he will surely talk about it on an upcoming episode.

</comment><comment><USER6> I'm not too surprised by the relatively strong showing of 128GB memory. I've worked with scientific datasets where the primary constraint was always memory. So we would always be chasing motherboards with the most memory support, and with such a machine around why not have it as your desktop :-)

</comment><comment><USER7> Reading this makes me nostalgic for 'multiple CPU' PCs. Haven't seen a lot of them for a long time, but was it 'hyper-threading' or 'multi-core' that killed the 'multiple CPU' workstation? Having more than one CPU had cool value and exclusivity - everyone has multicore, whether faked by hyperthreading or not.

I was also surprised by the amount of people choosing automatic login. I thought there was a downside to that regarding the 'keychain' thing or has that been solved over the last decade or so?

Also

<comment><USER8> Dual and quad CPUs used to be defining features of PowerMac towers back in the PPC days. The competition might've had Pentium 4's with much higher clockspeeds, but the PowerMac towers had multiple PPC G4s or G5s where most of the competition only had a single P3 or P4.

The Intel Mac Pro's that followed had options for multiple Xeons, but the multi core nature of everything from the C2D forward put a major damper on the demand for multi CPU configs.

</comment></comment><comment><USER9> The popularity of manual install shouldn't be a surprise. If you want to keep your /home data intact when you upgrade to a new version of Ubuntu then it's best to put it on a separate partition from the system partition. Since that requires two partitions and two partitions requires a manual install a lot of users are doing this.

</comment><comment><USER10> Huh, what are these graphs taken from?

The single stat metrics look a lot like grafana single stat panels with sparklines and a light theme. The bar charts and the geographical view look a lot like kibana 5/6 though.

Either of these could point to an elasticsearch based storage? That would make sense, because linux installations with all of their annotated metrics are pretty event-based.

<comment><USER11> First is Grafana for sure

</comment></comment><comment><USER12> A question to those who have the full dataset: How many _unique_ records are there? In other words, how many bits of information is in each individual record? ("Record" being the set of data submitted by a single machine.)

</comment><comment><USER13> I wish they published total number of installs and active monthly users.

<comment><USER14> I wonder why they didn't. Either too few installs would look bad, or they want to keep data secret for evil purposes :-)</comment></comment>


PVS-Studio static code analyzer for C, C++ and C#: Free - c-plus-plus http://www.viva64.com/en/b/0457/ <comment><AUTHOR> had me until "you must put this comment at the top of each of your files".</comment>


How to live to be 100 – TED talk - <AUTHOR> https://www.ted.com/talks/dan_buettner_how_to_live_to_be_100/ <comment><USER2> "This the place where the oldest living female population is found. ... They live a long time and tend to die in their sleep very quickly, and often, I can tell you, after sex."

Is he admitting that he has shagged a number of those centenarians to death???</comment>


As cashless stores grow, so does the backlash - <AUTHOR> https://eu.detroitnews.com/story/business/2019/04/12/cashless-stores-grow-backlash/39336669/ <comment><USER2> On the one hand,

1.) I rarely use cash and

2.) I'm sympathetic to places being able to opt out of dealing with cash if they so choose but

3.) I don't really care for a trend that makes it increasingly difficult to conduct transactions anonymously (as well as making it more difficult for some people to participate)

There are certainly card systems that can be replenished with cash but you don't see them widely in the US--presumably because credit cards are so popular.

<comment><USER3> Are you talking about prepaid debit cards? Those have lots of fees and such; as someone who used to rely on them, they're helpful if you absolutely need it but not easy or painless.

Banks don't like establishing branches in low income areas, and low income people don't usually have a means of transportation or time to get to a bank.

<comment><USER2> No, I'm talking about something like Pasmo cards, which are widely used in Japan for all sorts of things not limited to transit. [https://www.pasmo.co.jp/en/buy/](https://www.pasmo.co.jp/en/buy/)

<comment><USER3> Pasmo and Octopus(Hong Kong)-type systems only gained massive adoption because they were a valid form of payment on widely used transit networks. The US doesn't have very many of those.

Most US transit agencies these days want to get out of the payments business (Pasmo and Octopus have to be managed as debit card systems and transit agencies can't afford the overhead), so they're adopting London's second- generation system, where you can pay with contactless bank cards. So we might just skip that intermediate phase entirely.

</comment></comment></comment></comment><comment><USER4> To me, the obvious solution here is to have a government-run, cash-free payment system that's explicitly designed to be accessible to all citizens. Let people deposit money at the DMV or some such, and provide debit cards to be used against their balance. Just provide what's strictly necessary—the goal isn't to replace private banks.

Sadly, I don't see any possibility of this happening in the current political climate.

<comment><USER5> Postal banks are a thing. They work fine, and exist next to private banks, all over the world.

[https://en.wikipedia.org/wiki/Postal_savings_system](https://en.wikipedia.org/wiki/Postal_savings_system)

<comment><USER4> > In the United States, the United States Postal Savings System was > established in 1911 under the Act of June 25, 1910 (36 Stat. 814). It was > discontinued by the Act of March 28, 1966 (80 Stat. 92).

Oh, wonderful, so we had it but we got rid of it. :\

</comment></comment></comment><comment><USER5> The problem here isn't that the store doesn't take cash, it's that these people don't have a bank. If you want to pass a regulation, it's offer fee- less checking accounts with no minimum balance. This isn't some crazy thing. It's literally a product every bank used to offer, until very recently.

It blows my mind that so many people in the US don't have bank accounts. The ads for alternative banks equally blow my mind. Their selling point is... direct deposit.

It's a whole alternative world out there of people that take physical checks to stores, pay $3 to cash a check, the. Turn around and pay their electric bill at the same place (probably for another $3). Just get direct deposit and autopay, and your credit will probably improve automagically as well.

<comment><USER6> True...ish.

Many illegal workers don't have accounts and either can't get them or don't trust banks.

Many of the poor get paid directly in cash for various reasons; you'd have to convince some of them that they need an account.

Then there's the whole criminal enterprise thing, a not inconsequential chunk of the economy.

<comment><USER5> I'm not really sure any of this actually explains a majority of the unbanked. Sure, it explains why people engaged in illicit trade don't use a bank, but I find it hard to believe that is a sufficient explanation, especially when compared to lack of identification, perceptions of banks, and perhaps lack viable products (eg A checking account with "$25 maintenance fee waved if you keep a balance of $500 or more" is out of reach for the paycheck to paycheck crowd.)

</comment></comment></comment><comment><USER7> During the 2017 Tubbs fire in Santa Rosa virtually all the stores in West Sonoma went cash only. I pulled out maybe 300 bucks when I drove up to check on my property and spent 3-4 days buying everything with cash. Came back to work with maybe 150$ in my pocket and was shocked how many of the places in downtown SF didn't accept cash. It was a real eye-opener.

</comment><comment><USER8> Cashless society will have real repercussions for the lower rungs of the economic ladder. As it is I ask myself how do I give money to homeless people when I'm not carrying cash? How do people without phones or cards interact with no cash merchants?

As cell phones and credit cards become ubiquitous the acquisition of these devices will need to be mandated by the gov. Aka, you can not be barred from getting a phone/bank account/credit card because you do not have a place to live.

<comment><USER9> Don't give money to homeless people, please. The homeless who need it get their money from services that help them get back on their feet. The degenerate drug addicts who plague the city need to stop getting cash from people.

<comment><USER10> I am very uncomfortable with your presumptive use of 'degenerate' here. Are you equating addicts with degeneracy, and what comprises degeneracy in your view?

And what is a drug - Does alcohol count? I know a guy who's drinking himself to death but has his own house.

Also what do you think of this [https://www.spectator.co.uk/2015/05/the-case- for-prescriptio...](https://www.spectator.co.uk/2015/05/the-case-for- prescription-heroin/)

<comment><USER9> I'm sorry that makes you feel uncomfortable. I view degeneracy as the moral decay that causes the societal and familial ties to dissipate, as well as the inability for a local or state government to manage its people, that results in feces and tents littering my street from broken humans unable to take care of themselves.

<comment><USER11> Crazy idea: maybe feces and tents are littering your street because no one's helping them, rather than 'degeneracy'.

<comment><USER9> Obviously it's both, right? Although "help" is an interesting term. It can span a lot of policies, some that make sense, others that don't. Meanwhile the degeneration of familial support networks and moral codes plays a part in a more atomized society, where falling into drug addiction with no non-state help becomes increasingly common.

</comment></comment></comment></comment></comment></comment><comment><USER12> What's the incentive for a business going cashless? I can see some upsides to it like eliminating the risk of employee theft but the intermediaries take a cut.

<comment><USER13> I imagine that the labor and security required to handle cash is costly. This is the best number I could find:

"The cost of cash handling can range on average from as low as 4.7% of cash to over 15% of cash depending on the retail segment."

[http://www.ihlservices.com/product/costofcash/](http://www.ihlservices.com/product/costofcash/)

<comment><USER6> I hate to cast aspersions on the source, but... " _IHL Group is a global research and advisory firm specializing in technologies for the retail and hospitality industries. The company, based in Franklin, Tenn., generates timely data reports, offers advisory services and serves as the leading retail technology spokesperson for industry and vendor events._ "

(Hey, Franklin, TN!)

Anyway, here's some more:

[https://www.cashmatters.org/blog/paying-with-cash-costs- less...](https://www.cashmatters.org/blog/paying-with-cash-costs-less-time- and-money-finds-bundesbank-study-feb-2019/)

" __ _Key findings_ __

" _Cash payments are about 7 seconds faster than card payments with PIN entry._

" _Cash payments are about 16 seconds faster than card payments with with signature._

" _Cash payments up to €50 are cheaper, since the fixed costs for cash payments are on average lower._ "

[https://www.researchgate.net/publication/321386231_Costs_and...](https://www.researchgate.net/publication/321386231_Costs_and_Benefits_of_Cash_and_Cashless_Payment_Instruments_in_Germany_Module_2_The_Benefits_of_Cash)

" _The assertion that cash is inefficient is often based exclusively on cost considerations and a purely partial-analytical or business perspective (eg Bloching, 2007; Guibourg & Segendorf, 2007). Benefit aspects are not taken into consideration. Moreover, the empirical evidence is by no means clear-cut. Even in the more business-oriented approaches with regard to costs, cash payments are not necessarily more expensive than cashless alternatives. In Germany, the cost per transaction of cash is even lower than that of debit and credit cards. In terms of cost per turnover, debit cards are cheaper than cash, whereas credit cards are more expensive (Krueger & Seitz, 2014, Chapter 3; Thiele, 2016). In a study commissioned by the ECB covering several countries (Schmiedel et al, 2012), the findings did not reveal a uniform picture: in some countries, cash is the cheapest means of payment, whereas in other countries, cashless payment instruments are the cheapest._"

[https://www.researchgate.net/publication/265496400_The_Econo...](https://www.researchgate.net/publication/265496400_The_Economics_of_a_Cashless_Society_An_Analysis_of_the_Costs_and_Benefits_of_Payment_Instruments)

" _We present three case studies that illustrate the welfare implications of substituting one kind of payment method for another. We find that when all key parties to a transaction are considered and benefits are added, cash and checks are more costly than many earlier studies suggest. In general, the shift toward a cashless society appears to be a beneficial one._ "

[https://www.bundesbank.de/resource/blob/710096/4f27e9114ce3a...](https://www.bundesbank.de/resource/blob/710096/4f27e9114ce3a08b74b6c6effee04f65/mL/costs- and-benefits-of-cash-2014-data.pdf)

[Ok, so I can't find a good summary quote. But it looks interesting. I think it's module 1 of the first link above.]

<comment><USER14> Most of the cashless places I see are also contactless-only, which is certainly going to be faster, if that is a concern.

</comment></comment></comment></comment><comment><USER15> Not every store or service has to serve everyone.

For most people who use credit cards a cashless store could be a huge benefit to them as it would likely increase efficiency and perhaps offer lower prices. The benefits to the city would likely be less tax avoidance. I don't see why we would limit innovation just because some people would not be able to participate.

</comment><comment> francis-io Why does it discriminate against poor people? Do bank accounts cost money in america?

<comment><USER16> What's the problem with a company "discriminating" poor people? The case could be made that any company providing services for money is discriminating against the poor.

<comment><USER17> > What's the problem with a company "discriminating" poor people?

From the point of view of a company, probably nothing in their eyes.

From a societal point of view poor people already have an uneven playing field when it comes to a fair shot in life so making that worse seems like a bad idea.

Unless for some reason you like a permanent underclass of people stuck where they are for some reason I guess.

I personally don't, social mobility is generally a good thing (not least for the economy as a whole).

<comment><USER16> Is Tesla evil for discriminating against the poor?

</comment></comment></comment></comment><comment><USER18> "Legal tender for all debts public and private" used to mean something...

<comment><USER19> Buying a taco isn't repaying a debt..

<comment><USER20> To get the fully nuanced picture: -If you wander around to a grocery store bakery, and eat one of the donuts before getting to checkout, you have the right to settle your debt with cash. -Otherwise, the store can force you to pay with their preferred method, or else will return the merchandise to the shelves.

<comment><USER19> > and eat one of the donuts before getting to checkout

I'm not sure you have a legal right to do this. I'm pretty sure consuming something that isn't your is actually theft.

<comment><USER20> Yeah, that's a good point. I mostly wanted to convey a payment after use of the product or service. A better example might be paying your mechanic after he finishes fixing your car.

</comment></comment></comment></comment></comment><comment><USER19> Can someone explain to me:

> It took another year to build up enough funds to use his debit card > regularly.

Once you have an account, is there some kind of barrier to getting a debit card?

<comment><USER21> Technically, there's no barrier. But practically speaking, yes, there is a huge barrier.

Having an account is not enough to safely use a debit card, you also have to build up enough emergency funding to use a debit card. Otherwise, you risk a ~$50/charge for overdrafting, if you happen to be a off (even by as little as one cent).

Note that you as a consumer have very little control over if/when you overdraft, as American banks can _legally reorder your transactions outside of real date /time_ to artificially generate higher overdraft fees than actually happened in the real world.

[https://www.forbes.com/sites/halahtouryalai/2013/06/11/yes-b...](https://www.forbes.com/sites/halahtouryalai/2013/06/11/yes- banks-are-reordering-your-transactions-and-charging-overdraft- fees/#e6b79136daa3)

<comment><USER19> Don't overdrafts have to be authorized? Can't you get a zero-overdraft card?

<comment><USER4> My bank, Chase, lets me turn off "Overdraft Protection". If I don't have enough money in my account, the debit card transaction gets denied.

But, you have to explicitly request this. The so-called "protection" is on by default.</comment></comment></comment></comment>


12-year-old suspended over toy gun seen in virtual class - just-juan-post https://kdvr.com/news/problem-solvers/12-year-old-suspended-over-toy-gun-seen-in-virtual-class/ <comment><AUTHOR> This is ridiculous. Not being able to take kids to school is already hard enough on parents. They're being forced to use their own home instead of taking them to a building their tax dollars pay for. It's already privacy evading enough that they have video cameras in their , but then they record the video and use it for BS like this. What a waste of resources.

<comment><USER2> Not only that but they called child welfare on the parents:

" The teacher notified the school principal who suspended Isaiah for five days and called the El Paso County Sheriff's Office to conduct a welfare check on the boy without calling his parents first."

We'll probably see push back from parents as they become aware of the state of education.

This pandemic is going to expose a lot of the education system to parents.

The hope would be the education system learn from this hybrid of state education + home schooling experiment and also for parents to get more involved and have a say —but likely there would be tremendous pushback from unions who would not want that.

<comment><USER3> Good point.

Interestingly, Black students in the United States are subject to disciplinary action at rates much higher than their white counterparts. These disciplinary actions put students at higher risk for negative life outcomes, including involvement in the criminal justice system.

QUOTE: "Black children do not misbehave more than their White peers, rather they are punished more. In fact, Black students are more likely than their White peers to receive a disciplinary action for a discretionary offense like talking back, violating a dress code, or being defiant. Black children are also more likely to be suspended out of school for their first offense."

SOURCES:

\- [https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2678799/](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2678799/)

\- [https://www.pnas.org/content/116/17/8255](https://www.pnas.org/content/116/17/8255)

\- [https://www.marshall.usc.edu/sites/default/files/slittle/int...](https://www.marshall.usc.edu/sites/default/files/slittle/intellcont/Little%20The%20Problem%20With%20Black%20Boys%20CEJ%202018-1.pdf)

\- [https://edtrust.org/the-equity-line/for-black-children- atten...](https://edtrust.org/the-equity-line/for-black-children-attending- school-is-an-act-of-racial-justice/)

\- [http://kirwaninstitute.osu.edu/racial-disproportionality- in-...](http://kirwaninstitute.osu.edu/racial-disproportionality-in-school- discipline-implicit-bias-is-heavily-implicated/)

\- [https://csgjusticecenter.org/youth/breaking-schools-rules- re...](https://csgjusticecenter.org/youth/breaking-schools-rules-report/)

\- [https://edtrust.org/the-equity-line/for-black-children- atten...](https://edtrust.org/the-equity-line/for-black-children-attending- school-is-an-act-of-racial-justice/)

<comment><USER4> Would it even be acceptable to publish a study that says otherwise? Could even a contrary theory be _presented_ in polite society?

<comment><USER3> This is going to seem rude but try reading more.

The Hoover Institution at Stanford has long supported the data-backed studies of conservative economists on disparities in schooling from Thomas Sowell, Mike Petrelli, Walter White and Chester E. Finn Jr among others.

The general conservative reaction to the mountain of data is that the studies are correct. The concern is in "what to do next?" or "Yes, the data is bad but it doesn't mean that racism lurks behind every tree."

A change in policy may result in even worse outcomes.

STUDIES & OPINION:

\- [https://fordhaminstitute.org/national/commentary/how- think-a...](https://fordhaminstitute.org/national/commentary/how-think-about- discipline-disparities)

\- [https://www.educationnext.org/disparate-impact-theory-bad- fi...](https://www.educationnext.org/disparate-impact-theory-bad-fit-school- discipline/)

\- [https://fordhaminstitute.org/national/commentary/disparities...](https://fordhaminstitute.org/national/commentary/disparities- disparate-impact#body)

<comment><USER5> To be fair, the fact that something happens to be true (which, I feel the need to point out, no one here is actually disputing in this case) doesn't have any direct bearing on the question of whether polite society, in the counterfactual case that it were not true, would allow studies to conclude accordingly.

<comment><USER3> Not understanding the point, unless your aim is to be conspiratorial in nature.

The Stanford/Hoover Institute studies are obviously the contrarian view so yes "society" allows an opposing viewpoint...backed by data.

Last I checked, you're free to conclude whatever you wish even when lacking verifiable data unless your actual goal is to be liked by some particular segment of "society."

</comment></comment></comment></comment></comment></comment></comment><comment><USER6> They are absolutely wrong in every way to suspend the kid if the facts in the story are true. That's absurd and harmful with no benefit.

That said, I think there has to be something more to this.

First, the gun is an airsoft gun that actually looks like a green handgun:

[https://nypost.com/2020/09/07/colorado-school-calls- sheriff-...](https://nypost.com/2020/09/07/colorado-school-calls-sheriff-on- boy-who-showed-toy-gun-in-virtual-class/)

For comparison, here is a real handgun that happens to be green:

[https://www.pinterest.com/pin/329185053986413778/](https://www.pinterest.com/pin/329185053986413778/)

Second, it's not clear that the teacher understood or believed it was a toy. And if it wasn't a toy, then their actions don't seem 100% unreasonable.

I love the second amendment, but if the teacher honestly believed a 12 year old was casually handling a firearm without supervision, I could see them wanting to let someone know. And maybe they just took it too far.

Maybe I have too much faith in humanity, but I really cannot image any reasonable person calling child protective services on a house because a child had a toy gun (nerf, airsoft, or otherwise). I really don't believe teachers are out to get students or to just waste the state's time.

But if that's what happened then it's reprehensible. And either way, the suspension makes no sense if the student is telling the truth about what happened.

<comment><USER7> > Second, it's not clear that the teacher understood or believed it was a toy. > And if it wasn't a toy, then their actions don't seem 100% unreasonable.

From the OP:

>> The Problem Solvers obtained the sheriff's report and it confirms the teacher "said she assumed it was a toy gun but was not certain."

</comment></comment><comment><USER8> School bureaucracy is insane and cares more about liability than common sense and empathy, story 8001.

</comment><comment><USER9> This is why teachers unions do not want you to have a choice where you send your kid.

Because no sane adult would send a kid to a school that did this.

<comment><USER10> I don't think this is remotely why and the premise is disingenuous as well. It seems pretty clear that they're against public funds going to private institution rather than they want you to have no choice of where to send your kid.

<comment><USER11> Why? I pay property tax which mostly goes to education. Why can I not have choice in how that is spent on the education of my children?

<comment><USER12> You presumably also pay federal income taxes - why can't you choose how money on soldiers gets spent?

Maybe because there's a societal benefit from people not getting a choice on how each one of their individual tax dollars gets spent... And maybe there's a societal benefit from people not being able to choose which public school they get put into. You're gonna deal with one of two evils: Tyranny of the majority or tragedy of the commons. Most places in the US seem to choose tyranny.

<comment><USER11> Federal expenditures are very different and removed from one's local community. Everyone should absolutely get some say in how their money is spent at the local level.

> Most places in the US seem to choose tyranny.

Very true and very sad as well.

<comment><USER10> This is why we have local elections including school boards. Good luck getting Middle school by AWS to offer similar accountability.

</comment></comment></comment></comment></comment></comment><comment><USER13> Are institutions eroding common senses or do people just lack common sense in the first place?

<comment><USER14> This is very common in American schools. At some point it just got easier to implement zero-tolerance policies than to deal with the actual issues.

<comment><USER15> I would say it has always been like this and now we are waking up to this fact. Actually dealing with issues would require a flexibility that would not go well with the rigidity of the educational system

<comment><USER16> It's been this way since at least 2003. My parents taught at a school for students expelled from their home-zoned schools, and some of the reasons were incredibly laughable. Zero-tolerance discipline is a plague on our society (mandatory minimums in sentencing is another example).

</comment></comment></comment></comment><comment><USER17> > The toy in question is a neon green and black handgun with an orange tip > with the words "Zombie Hunter" printed on the side.

Pretty sure that's a nurf gun I've seen in walmart before.

<comment> dane-pgp The article at [0] (which is not geo-restricted) does indeed describe it as a "neon green Nerf gun".

[0] [https://reason.com/2020/09/07/zoom-nerf-gun-school-cops- kid-...](https://reason.com/2020/09/07/zoom-nerf-gun-school-cops-kid-isaiah- elliott/)

</comment></comment><comment><USER4> > _The teacher notified the school principal who suspended Isaiah for five > days and called the El Paso County Sheriff's Office to conduct a welfare > check on the boy without calling his parents first._

This is the scary part to me. The amount of power CPS wields is incredible. Your children are taken away for weeks and kept with a stranger foster family. This process is a terror to them, and 15% of them are sexually abused while in the foster care system[1].

We learned with Pelosi Jr. that CPS is weaponized by the politically connected to threaten or silence people[2].

1\. [https://pdfs.semanticscholar.org/c495/52badb0c693eb6f7ed8960...](https://pdfs.semanticscholar.org/c495/52badb0c693eb6f7ed8960ba55720e71876b.pdf)

2\. [https://nationalfile.com/video-pelosis-sons-ex-girlfriend- cl...](https://nationalfile.com/video-pelosis-sons-ex-girlfriend-claims- forced-abortion-abuse-weaponizing-cps-massive-fraud/)

<comment> john-shaffer A "welfare check" is just police stopping by and asking if everything's okay. The CPS aren't involved.

</comment></comment><comment><USER18> School administration reaching into people's homes is _extremely_ offensive. I can't believe people tolerate this!

<comment><USER19> Given that teaching is happening inside people's homes, it has to.

That being said, this is stupid in the extreme. I can justify the welfare check, but the suspension is insane.

<comment><USER20> In the current environment (a significant percentage of Americans are afraid of almost any police interaction and are demonstrating to that effect), the welfare check seems ridiculous as well -- especially when a phone call to the parents would suffice.

</comment></comment></comment><comment><USER21> > "We will continue to support all families in our school to make sure they > feel safe, respected, and educated."

Wow, they _seriously_ fucked that one up. There's nothing in the district's actions that would make anyone feel safe or respected.

<comment><USER5> Technically they didn't fuck it up - they're just lying about trying in the first place.

<comment><USER21> I mean, yeah, you're right, although they managed to do the complete opposite of what they were lying about; if I went to that school I'd feel a hell of a lot _less_ safe or respected knowing that the school might send the police to talk to my children based on something they briefly see on camera, without even talking to me about it.

</comment></comment></comment><comment><USER22> "This site is currently unavailable to visitors from the European Economic Area while we work to ensure your data is protected in accordance with applicable EU laws."

Apparently tracking is more important than actually providing the news.

<comment><USER23> Apparently privacy is more important to the EU than being able to see the news.

<comment><USER24> You can guarantee absolute privacy by blocking the visitor entirely! Problem solved.

</comment></comment></comment><comment><USER25> Headline buried the lede: two boys were suspended, and one was threatened with imprisonment.

</comment><comment><USER26> Yep, I was waiting for this. There's a whole new world of stupid liability that we haven't even touched on yet. Similar stuff will happen in the workplace.

<comment><USER27> Ugh. Tell me about it. Workplaces are full of this bullshit.

Ergonomics training - CYA for lawsuits due to repeated motions, they want to make sure you signed off on the training. It has nothing to do with your well- being.

Diversity and Inclusion - I fucking hate this. It has not much to do with getting people of various backgrounds but CYA against discrimination lawsuits.

Office safety - I can't bring my own coffee machine because I might just cause a "trip hazard". Hmm... I see another CYA against workplace injury lawsuits.

</comment></comment><comment><USER28> Although a toy gun, doesn't the second amendment also apply to children? I don't remember seeing an age restriction in the constitution. Also the "gun" was at home and not taken on school property.

<comment><USER29> It's Texas. Zero Tolerance and minimum mandatory sentencing trumps your rights.

<comment><USER30> It happened in Colorado.

</comment></comment></comment><comment><USER31> We live in a "good" school district. I was thrilled when our home-school plan was accepted. Every time I see stories like this I take it as one more data point in support of our decision.

</comment><comment><USER4> A great reason to be in favor of "money follows the student" type of school funding[1]. This is one of those policies that I'm unexpectedly in favor of with the current admin. Especially with online education.

The reality is these parents are stuck. Parents in terrible districts are stuck. What incentive do schools have to change if they'll get the money for your child either way?

1\. [https://www.realclearpolitics.com/video/2020/08/12/trump_on_...](https://www.realclearpolitics.com/video/2020/08/12/trump_on_reopening_schools_we_want_money_to_follow_the_student_democrats_want_money_to_follow_the_union.html)

<comment><USER25> Money should follow the competence, not marketing campaigns.

<comment><USER4> If money should follow competence then the bulk of our public schools should be seriously defunded, especially in inner cities.

<comment><USER7> > If money should follow competence then the bulk of our public schools should > be seriously defunded, especially in inner cities.

I believe the inner city schools already have been defunded (or more precisely have been underfunded from the start), _which is part of the problem_.

</comment></comment></comment></comment><comment><USER32> That teacher and principal should be fired

</comment><comment><USER33> Shows how unaccountable government schools are for their actions.</comment>


Is this a DB inconsistency in HN? - <AUTHOR>

About a year ago there was a userid - mquaes - who kept posting simplistic mental arithmetic tricks, so many that HN was semi-flooded.
Now there's a new user - pcratio - who has posted something from that same site.  Fair enough, maybe it's genuine, maybe it's the same person back again.
But their submission is here:
https://news.ycombinator.com/item?id=5524269
... and yet it doesn't show up in pcratio's list of submissions:
https://news.ycombinator.com/submitted?id=pcratio
Is this a bug?  A fleeting inconsistency?  Or something else? <comment><USER2> "Works for me". I wonder if it's "transient" or "it does that". Anyway, good spotting, and let's keep killing that trash.

</comment><comment><USER3> Shows up now - for me.

I suspect that user submission lists are not updated in real time. Like average karma.

<comment><AUTHOR> Interesting, still doesn't show up for me ... I wondered if the fact that I'd flagged it then subsequently hid it from me, but that's not what's happening, I simply don't see the item listed in pcratio's submissions.

Very odd. <fx: shrug /> HN oddities.

<comment><USER3> The closure over your username is undoubtedly different from that over mine. Revel joyously in the personalized experience of HN.

</comment></comment></comment><comment><USER4> An odd one for me is from a user Arrington 1532 days ago, who seemed to post the same comment on hundreds of old threads - look at thread 1 onwards - it's all over the place but on threads that are far older than the user / post (ie post 1)</comment>


Are Real Names Required For Real Socializing? - <AUTHOR> http://www.avc.com/a_vc/2011/08/are-real-names-required-for-real-socializing.html

<comment><USER2> Real names enabled facebook users to quickly realize the "real world" social network within facebook, rather than building an entirely new network.

I don't know email addresses or commonly used online aliases for even 10% of the people whose names I know. I'm fairly confident that I could find just about any of my cousins or high-school classmates on facebook in minutes. Without them having put their real name on their facebook profile, I don't believe I would be able to find them at all. Surely it's clear how that leads to facebook having a much bigger network much faster than a service that doesn't use real names. Real names are the key my brain uses to identify people.

I also think there is a psychological impact of constantly looking at pages filled with names that you recognize. If you swapped out all the names on my facebook feed with random hn usernames, I think I would feel different while I read it.

All that said, I do agree with most of what Fred Wilson is saying. I hate the impulse to "civilize" or "sterilize" the web as well, and I'm especially concerned when politicians start with that sort of talk. Real names make sense _only_ for facebook or a facebook replacement (good luck with that), because in my mind the definition of facebook is "the thing you use to communicate online with people you know in real life."

</comment><comment><USER3> I know who my friends are. Their online nickname doesn't change that a bit.

Heck, when I lived in California, everyone there called me by a short form of my internet nick at the time. Most people didn't know my real name, and the few who did still called me what everyone else called me. It didn't change anything. My boss even wrote my check out to that name once, with my real last name. I had to have him correct it.

I think some people are scared of things being different, and socializing on the web is vastly different than socializing in your home or a bar. This 'real names' thing is an attempt to hold on to a little bit of the socializing they are used to. It will pass.</comment>


Blockchain-Enabled Kiosks - <AUTHOR> https://www.fastcompany.com/40405379/these-blockchain-enabled-kiosks-make-coffee-farmers-more-money-and-let-you-verify-your-beans <comment><AUTHOR> _" Bext360 has spent the last 18 months training its algorithms to identify dozens of cherry details. The unit makes an offer for the produce, and the grower decides whether to accept. If the answer is "yes," the money is wired there and then to their cell phone."_

Any coffee bean experts here? Q. What details do you grade the beans? Is this done chemical/analytical way? Do things like smell matter?</comment>


DeepCoder: Learning to Write Programs - <AUTHOR> https://arxiv.org/abs/1611.01989 <comment><USER2> Well, it's obvious that there will be an improvement in specifiying the syntax for a computer program as it has been for the last decades.

e.g. Assembler -> C -> C++

There recently has been a post @ HN about the missing programming paradigm ([http://wiki.c2.com/?ThereAreExactlyThreeParadigms](http://wiki.c2.com/?ThereAreExactlyThreeParadigms)). With the emerge of smarter tools, programming will get easier in one way or the other, releasing the coder from a lot of pain ( as C or C++ did realse us from tedious, painful assembler ). However, I am quite sure that it won't replace programmers since our job is actually not to code but more to solve a given problem with a range of tools. Smarter tools will probably boost productivity of a single person to handle bigger and more complex architectures or other kinds of new problem areas will come up. Research will go faster. Products will get developed faster. Everything will kind of speed up. Nevertheless, the problems to get solve / implement will remain until there's some kind of GAI. If there's an GAI smart enough to solve our problems probably most of the Jobs have been replaced.

<comment><USER3> >> our job is actually not to code but more to solve a given problem with a range of tools.

Actually, often the most valuable thing a programmer can do is build a tool/dsl that lets the domain experts(or cheaper labor) to solve problems.

So if a programmer just solves problems, he will probably have some serious competition.

<comment><USER4> Best programmers are domain experts. The line will just continue to blur.

<comment><USER5> spot on

</comment></comment></comment></comment><comment><USER6> It's intriguing, particularly in cases where the code written could be verified against a specification or at least comprehensively fuzz-tested against a 'known good' version of the program. A mildly terrifying thought for a performance programmer - the idea that, perhaps, someone could just write some sloppy Python code to solve a program "at some speed" and have a automated system iterate it into some tuned, SIMD-laden, C/C++ horror-show.

While of course we have optimizing compilers to do this sort of thing now, you could imagine automated systems that attempt to preserve clarity and simplicity as they do it - and such a system could work semi-supervised, iterating with a human in the loop to steer the system towards more comprehensible solutions.

<comment><USER3> Well, for math we're already here: [http://www.spiralgen.com](http://www.spiralgen.com)

It's optimizes math code, both at the math level and the implementation level, and Intel have used it to optimize their performance libraries.

<comment><USER7> > Well, for math we're already here

No, _they_ are here:

> SpiralGen holds the exclusive license to the Spiral software generation and > optimization technology.

But I wonder how that can be true if:

> Spiral was developed under the lead of Carnegie Mellon University

<comment><USER8> It's called Technology Transfer to private sector. Happens all the time. The CompCert certifying compiler for C was a landmark achievement of INRIA that lots of CompSci is building on. A company called AbsInt controls the I.P. now.

<comment><USER3> The thing i don't get: this seems like a useful but rarely used tech.

Even if there are patents involved , i'm sure it's not patented everywhere around the globe, and it's just software, so why isn't there no real competition ? something like a reasonably prices SAAS service ?

</comment></comment></comment></comment></comment><comment><USER9> So this is how the uprising happens.

First they came for the factory workers. Then they came for the accountants. Then they came for the drivers...

Then they came for me.

<comment><USER10> I mean that's how most circuit designers lost their jobs to CAD/EDA tools, why should software engineers be different?

<comment><USER11> Circuit designers didn't lose their jobs to EDA tools any more than software developers lost their jobs to compilers.

<comment><USER12> A programmer translates requirements into implementation. Which is translating natural language into a programming language.

Now, as automation increases, that layer gets thinner each time.

But it will be some time before we see AI capable of doing requirement analysis, negotiating requirements, etc.

</comment></comment></comment></comment><comment><USER13> For actual in-depth technical discussion / criticism of the paper, you can read its ICLR peer reviews (and other public comments): [https://openreview.net/forum?id=ByldLrqlx](https://openreview.net/forum?id=ByldLrqlx)

</comment><comment><USER14> Programming is easy if specs are frozen. I want to see that neural network discuss with my managers about inputs/outputs. Good luck.

<comment><USER15> Machine learning algorithms are actually quite good at solving poorly specified problems. They learn from examples and don't require any definite problem specification.

For example, recognizing people in a picture is almost impossible to specify but relatively easy to learn, given enough examples.

Next time your facing a poorly specified problem, don't ask for a clear spec, instead ask for a million examples and train a deep net to solve it :)

</comment></comment><comment><USER16> That's interesting that the paper was on arxiv since November ( [https://arxiv.org/abs/1611.01989](https://arxiv.org/abs/1611.01989) ) and people payed attention just now.

<comment><USER17> That's because, AFAIK, no any meaningful problems were solved by this algorithm yet. I'm not even sure it can make a Max(array) on its own.

<comment><USER18> Then why's it worth discussion?

Why is there no "general" deep learning algorithm?!

<comment><USER19> >Why is there no "general" deep learning algorithm?!

Because "deep learning" is almost always just neural networks with many layers and some tweaks to the unit types to alleviate vanishing-gradient issues, trained by stochastic gradient descent. In rather theoretical terms, "deep learning" involves using gradient descent, and gradient-increasing tweaks, to search for a specific continuous circuit in a given space of continuous circuits. It's only as general as the hypothesis class you search through, which here is, again, continuous "circuits" composed out of "neuron" units of specific types, numbers, and arrangements.

Now, in computability terms, given a large enough (continuous or discrete) circuit, possibly a recurrent one, you can represent any computable function. However, in learning terms, that doesn't make a _useful_ computable function at all easy to find in a very-high dimensional space.

</comment></comment></comment></comment><comment><USER20> Would a potential roadblock for such systems be automatic verifiability? Consider that a system provides a ranked list of possible code snippets someone would still need to pick from these choices and test it.

<comment><USER21> Just like humans program; you write code and have unit tests (if you are lucky). What is different about this? It has the same inputs / outputs so the unit tests will be there. Formal verification would be better; aka having a model for the original and having the computer prove that new version is mathematically identical to the original. But both the formal verification and the transformation proof are far off for almost all software projects in practice.

<comment><USER22> Who's writing the unit tests, the same bot as writes the code? The customer?

<comment><USER21> The customer. The bot cannot do that (and if it could, it would not need to I guess).

</comment></comment></comment></comment><comment><USER23> My two cents:

1) Write programs easy to understand at first glance

2) Write code easy to delete

3) Write considering costs: target zero -or as low as possible- maintenance costs

4) Write code for both fun and profit

</comment><comment><USER24> > The components of LIPS are (1) a DSL specification

LIPS with DSLs you say?

</comment><comment><USER25> surprised that Muggleton and ILP isn't referenced.</comment>