Attacking the Windows Nvidia  Driver - <AUTHOR> https://googleprojectzero.blogspot.com/2017/02/attacking-windows-nvidia-driver.html <comment><USER2> A lot of people hated the decision, but back when Microsoft refused to support WebGL, one of the reasons were that the GPU drivers were awful and exposing them to the internet was dangerous.

The proper solution, of course, i for GPU drivers not to suck but it was still a legitimate point and this article seems to validate that.

<comment><USER3> They were right that it was dangerous, but it became pretty clear that IE was using them to provide cover for not implementing a feature when they changed their mind later.

Whereas Chrome implemented a WebGL compiler that reduced the amount of attack surface WebGL could reach and audited a bunch of popular drivers to fix the exposed bits.

I think the fact that this blog shows that these vulnerabilities are not reachable from WebGL is a validation of Chrome's approach there, though it clearly shows the issues these drivers present for escaping Chrome's sandbox.

<comment><USER4> I think there's probably still a fair bit of attack surface behind glReadPixels() and the like. All it takes in a single backbuffer/texture/surface/etc to not be memzero'd properly and you can start looking at parts of the system's memory.

GPUs are both really complex and highly secretive about their implementations. The incentive for GPU vendors is to write fast drivers. Security is pretty far down on the list, esp when it competes directly against performance.

<comment><USER5> >fair bit of attack surface behind glReadPixels() and the like...

>...you can start looking at parts of the system's memory

I thought the whole point of using glReadPixels(), as opposed to just dereferencing a pointer in the system's address space, was that the framebuffer memory it accesses (whether backed by a texture or a surface or whatever) is GPU memory, and _not_ system memory?

<comment><USER4> That's true on split-memory architectures(desktop) but most mobile GPUs(and a couple consoles) use a unified memory model.

Also most browsers use the GPU to speed up rendering so you can pick out things from there too potentially.

<comment><USER6> I wonder, do you really have to write zero to all the memorycells? Or can you just stop the refresh cycle in hardware and let squares of memory drain and die in one or two cycles?

<comment><USER7> Even without refreshing the cells they actually last quite a lot longer than you would expect. This is an old research paper on the topic but unfortunately, it looks like all of the images and videos are broken which is a shame because it gave you a great mental picture of the "half-life" so to speak of RAM.

[https://citp.princeton.edu/research/memory/](https://citp.princeton.edu/research/memory/)

The bottom line is that you would need to stop refreshing it for minutes if not longer to be sure that there wasn't an information leak and if the memory is cooled down it'll last a great deal longer without being refreshed and even still keep the majority of the contents after hours of being removed from a running system if they are cooled using liquid nitrogen. Either way one or two cycles isn't going to really matter at all.

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> I cant prove this but my computer blue screened after a Nvidia driver update. It took my several hours to get everything working again because it wouldnt even launch in safe mode. Very frustrating. I wish they had a little more quality control on their drivers.

<comment><USER9> Oh trust me, they do tons of QA. If you can replicate what problem you have demonstrably with steps that QA can follow and you submit to the Nvforums or whatever, someone in the QA team will eventually try it out.

<comment><USER10> I dunno, I have had to live with this bug ([https://forums.geforce.com/default/topic/953432/geforce- mobi...](https://forums.geforce.com/default/topic/953432/geforce-mobile- gpus/dxgmms1-sys-bsod-crash-on-recent-windows-10-patches/) , [https://answers.microsoft.com/en- us/windows/forum/windows_10...](https://answers.microsoft.com/en- us/windows/forum/windows_10-update/kb3163018-causes-bluescreen-in-directx- dxgmms1sys/84db9eb6-0654-4716-809c-95fc66774985) ) for over half a year now.

<comment><USER11> Yeah, Nvidia rather sucks at cleaning up after themselves, for whatever reason.

The main reason I'm not buying another card from them is their stance towards Linux in general and their stance towards PCI virtualization in particular.

I'm currently not upgrading drivers due to the latter; they're trying to disable the capabilities (dedicating a card to a VM) that are the reason I bought a second card in the first place.

I can usually deal with either lazy or greedy, but both in conjunction is infuriating. Screw those guys.

<comment><USER12> If you're talking about GPU pass-through, they're just trying to make it hard, but they're not making it impossible.

If you're talking about virtual GPUs, where one card is split across multiple VMs, however, unfortunately that's Tesla only. That said, I worked (as the maintainer of KVM) with the nVidia driver people working on vGPU, and I was very impressed. They were very knowledgeable and professional, and they managed to contribute a generic Linux framework for virtualizing PCI devices rather than a one-off hack specific to nVidia. Intel is using the same framework now, in fact.

<comment><USER13> I think they want to limit it to Quadros to be more precise.

<comment><USER12> Yes, exactly. But they don't try _that_ hard.

</comment></comment></comment></comment></comment></comment></comment><comment><USER14> What I'd like to see is a "slimmed down" version of the graphics drivers.

Right now my graphics driver carries a boatload of "utilities" of questionable utility - especially "hand written artisanal shaders" to improve quality in AAA titles, where NV or ATI optimize the game's original shaders. NVIDIA ships stuff for 3D glasses.

How about that the driver packages only load these when asked to do so, e.g. when a game that can be optimized is installed / launched, or when a 3D glass is added?

<comment><USER15> Hear hear. Can they also please be less than 100megs in size? I think nVidia's clock in at 300 right now. I just want the driver and recording/streaming software. Maybe you could open up the required api to write a third party Shadow Play...

<comment><USER14> Yeah, the NV mobile driver clocks in at 292 MB (W7 x64) right now.

One thing that certainly blows up the size is that the NV driver installer bundles support for everything from the old NV 8600M - which IIRC was released in 2007.

If there's one thing I certainly can't whine about in times where phones carry less than 2 years updateability, then it is that NVidia still provides up-to- date drivers for a GPU chipset nearing a decade of life time.

<comment><USER16> I agree, but I recently bought a gtx1060, which meant I had to update to the latest drivers, which unfortunately don't support the other card I have in that machine (I forget what model, it's either a gt520 or a gt210). I only wanted to use that other card to connect extra monitors.

You can't run two Nvidia cards off different driver versions.

Anyway, the gtx1060 supports four monitors by itself which is enough but I needed to buy some cables.

Nvidia drivers are super bloated though.

</comment></comment></comment></comment><comment><USER17> I wonder if AMD drivers have the same attack surfaces as the nvidia drivers

Edit: a quick google search turns up CVEs for the old catalyst driver, but none for the newer crimson drivers.

<comment><USER18> Too bad my laptop can't use those crimson drivers, as AMD no longer support the APU part of the setup that make things like external displays work (and will you please stop trying to be helpful and silently "upgrading" my drivers, Microsoft!).

<comment><USER19> You can disable driver updates through group policy.

[https://technet.microsoft.com/en- us/library/cc730606(v=ws.10...](https://technet.microsoft.com/en- us/library/cc730606\(v=ws.10\).aspx)

<comment><USER18> Could have sworn that MS had disabled GP access on "consumer" grade installs.

</comment></comment></comment></comment><comment><USER20> Project Zero is awesome. I'm not normally a fan of the GOOG, but these guys are doing some seriously good work.

</comment><comment><USER21> Getting these issues fixed in the drivers is great, no doubt. But it's moot if nobody actually updates to those fixed versions. How hard are nVidia/Microsoft trying to actually push these out over Windows Update so that end users will actually benefit from all of this work?

<comment><USER22> Anyone who's into gaming probably has GeForce Experience installed, which manages drivers and gives notifications.

I want to say Microsoft will push them out a little later, but I can't be entirely sure since I've always used the nVidia path.

EDIT: Unfortunately the GeForce Experience is getting, as is typical, super invasive. Access to even basic settings requires an account (nVidia or Facebook account, etc).

<comment><USER23> Recently removed GeForce Experience. Graphics drivers should not require an account.

Considering that computer configuration is unique enough to enable fingerprinting across different browsers, I can't even see why it's required.

At least Experience tells you it's being creepy so you can remove it.

</comment></comment></comment><comment><USER24> Can someone give me a layman's definition of an "escape" and why they would be legitimately needed? Are they needed so callbacks can "escape" and be exposed to other classes? (does that make any sense?)

<comment><USER25> As the article indicates, they're similar to ioctl, which is a system call that, roughly speaking, allows arbitrary opaque blobs of data to be passed back and forth between a user-mode -process and a kernel-mode driver. This is intended as a generic mechanism allowing drivers to expose arbitrary functionality to user-space. This enables the implementation of functionality that would not otherwise be possible because it was not foreseen and enabled by the design of the operating system.

Bear in mind that the system call ABI changes slowly and with much difficulty: once a version of a kernel is in production, it can stay in use for a long time; it can take a long time for new functionality to be broadly available, and breaking back-compat with applications compiled against older ABIs is Not Done. Dynamically loadable kernel modules and ioctl-like system calls make it much easier to bring new functionality to all the various kernels running around in the real world.

Given the complexity and rate of change in graphics tech, it makes perfect sense for there to be a general-purpose arbitrary functional-call mechanism for interaction between user-mode and kernel-mode graphics driver components. Microsoft (or the linux graphics subsystem maintainers, etc) just doesn't know enough about Nvidia/AMD's current and future requirements to nail down a more rigorously defined API.

<comment><USER24> Thanks, I know nothing about this level of programming but this gives me a lot of good stuff to google.

</comment></comment></comment><comment><USER26> Slightly off topic but with all the video expertise in this thread, I want to ask anyway! :-)

I'm currently selecting parts for my next computer, to be used for continued development of the Windows .NET software for the Web site for my startup and also for my first Web server available to beta testers and then to the public on the Internet.

So, sure, I need a video card. Of course, I will do some routine Web browsing, maybe watch a movie at YouTube or Netflix. But I have never played a video game and, trying to get my business going, have no intention of playing a video game.

So, looking at information on video cards, it appears that maybe the card should support hardware acceleration of Microsoft's DirectX version 12 and also maybe some recent version of OpenGL.

Question 1: Why should I move from just VGA, that is, get just a VGA card and not even get a _graphics_ card? What will I get from a graphics card I really need and can't get from just VGA?

Question 2: If I get a graphics card, will DirectX 12 hardware acceleration on a graphics card help for some of Web browsing or movie watching?

Question 3: Same as Question 2 but for OpenGL?

Some people on this tread may have some good answers. As far as I can tell, good answers on the Internet are like hen's teeth -- it looks like everyone wants to sell graphics cards for the latest _gaming experience_.

Thanks!

<comment><USER27> Disclaimer: NVIDIA employee.

First of all, if your CPU has an integrated GPU, and you don't need more monitors than it supports (usually it's 3x1080p), that will be more than enough.

> Why should I move from just VGA, that is, get just a VGA card and not even > get a graphics card?

I don't quite understand what you mean by VGA card. You mean something that has a VGA adapter and framebuffer(s), but the rendering is done on the CPU?

I wasn't aware those still exist outside some niche markets. I'd guess it'd cost about as much as an entry level GPU, which will take the load off your CPU.

My advice, if you don't have any iGPU on your CPU, is to just get the lowest tier graphics card. Those are <$100 new for the latest generation. You don't need latest, and probably don't need new.

When it comes to web browsing and watching videos, any remotely recent card will work fine. You may have issues with some fancy WebGL pages (i.e. browser games) but that hardly counts as everyday browsing.

Be sure to read a review of the card before purchasing!

<comment><USER26> Thanks. I was slowly beginning to conclude much of that.

The CPU I plan is the AMD FX-8350 with 8 cores running at 4.0 GHz and 125 Watts. So, no it has no _integrated graphics_ support.

For a "VGA card", I just meant a video card supporting all the old VGA standards but without a _graphics processor_. So, there would be no "hardware acceleration" of OpenGL 4.5 (or some such) or DirectX 12 (some version of). Yes, there would be a standard VGA plug (socket, connector, etc.) for the signal connection to the monitor, but many high end graphics cards also have that.

Yes, looking, it's possible to find just a VGA card, that uses an old PCI slot, for about $20. But, a low end _graphics_ card can go for about $30 or $36 with 1 GB of memory of its own, a _graphics_ processor, and "support", likely _hardware acceleration_ , of OpenGL and DirectX.

Apparently by Windows 10, DirectX 12 is regarded as a _standard_ part of Windows.

In my old computer, I assembled in 2007, which apparently due to motherboard hardware problems, does _blue screen of death_ (BSOD), really, the screen goes black instead of blue, about five times a day, has an old nVIDIA GX 4000 with 64 MB of memory. As far as I know, the card has been fine. I never knew that the card had any _graphics_ capabilities until two weeks ago when I ran the standard Windows utility DXDIAG which showed that the card supports DirectX 9 and the card put up a nice rotating cube of the DirectX logo. Okay. So, maybe the graphics processor in the card can accept a gazillion triangles in 3D from the CPU, motherboard, and applications software, do rotations, hidden line removal, shading, maybe texturing, etc. Okay, but since 2007 that is the first time I ever saw such a thing!

I have been concerned about statements, e.g., that some graphics card needed for the PC's power supply to have capacity 300 Watts or more. Gads! That's a lot of power! Looking in more detail, apparently such graphics cards actually draw a maximum of only 25-40 Watts at 12 Volts, that is, <= 3.3 Amperes, which seems acceptable enough for the 650 Watt power supply I'm planning, the case cooling I'm planning, etc. I will be sure to use some of the standard ASUS software to monitor the 12 Volt lines from the power supply -- I doubt that the voltage will ever fall significantly below 12 Volts. The 12 Volts lines from the power supply are used for what, just the cooling fans, the hard disk drives, maybe the power on the USB ports, and, apparently, power to the PCI- Express slots? Gee, the pulse width modulation (PWM) of three of the cooling fans will put some fluctuation on the 12 Volt lines that will mess up a graphics card? I doubt that!

You are correct about WebGL -- I doubt I will be visiting Web sites that use that. I'm less clear about _scalable vector graphics_ (SVG). I don't see even from 50,000 feet up how ordinary Web browsing, say, displaying JPG or PNG still images or playing MPG4, YouTube or Netflix, or DVD videos could be helped by having a graphics processor -- tough to find such explanations. Do graphics processors routinely help display fonts faster?

I will have a 2 TB hard drive for bootable partitions. I will install Windows 7 Professional 64 bit on two boot drive partitions, say, drive letters C and D, and use one of those for my remaining software development for my Web site. Using likely the standard Windows utility NTBACKUP, which I like (e.g., it will backup a bootable partition while it is running, likely much like how relational database does a backup of a database while it is executing transactions and I can save it to any disk drive I want just by an ordinary copy operation) I will save both bootable partitions to a second hard drive. Then if, say, partiton D gets _sick_ and the usual Windows _restore_ is not good enough, I will boot partition C and restore the sick partition D from one of my NTBACKUPs on the second drive.

Some years ago when I was trying to install an Express (free) version of Microsoft's SQL Server, my boot partition contents were corrupted, really, destroyed, and I had to reinstall everything starting with an empty partition. Bummer. I want NEVER to have to do that again: Before I do any possibly dangerous _maintenance_ , installations, or _upgrades_ to a bootable partition, I will just save the whole partition with NTBACKUP. Then, if the partition gets messed up, I will just boot another bootable partition and restore the backup from NTBACKUP and try again.

Then I will install, again on two partitions, some version of Windows Server, likely 2012, and SQL Server of about the same vintage, and that will be the basis of my Web site as I go for beta testing and live on the Internet.

The Web site HTML sent to my users will be only just dirt simple HTML, say, up to date as of about 10 years ago, with just a little, simple CSS and nearly no JavaScript, no pop-ups, roll-overs, pull-downs, over-lays, or icons and no HTML <div> _elements_ ( _tags_?) -- dirt simple. I will have a simple logo graphics PNG I developed with just Microsoft's PhotoDraw, and that will be the only use of graphics. Net, for the Web site, I see no need for any _graphics hardware_ , for development, server, or clients.

I see from both nVIDIA and ATI graphics cards $30-$40 with 1 GB memory, OpenGL, DirectX 12 that use a PCI-Express x16 version 2.1 slot. The Asus motherboard I have in mind has a PCI Express x16 2.0 slot which I suspect one way or another will work well enough with a card that wants version 2.1. I suspect I will make a decision today.

I'm still not very clear on just why I need a _graphics_ card instead of just an old VGA card, but for just another $16 I'm going to spend the money, accept whatever system management mud wrestling I have to do to get an appropriate device driver working, quit worrying about the card, and get on with the more important work.

Thanks for the info.

</comment></comment></comment><comment><USER28> > Most of the vulnerabilities found ... were very basic mistakes, such as > writing to user provided pointers blindly, disclosing uninitialised kernel > memory to user mode, and incorrect bounds checking.

It's not just Windows drivers with these problems...

<comment><USER27> Some specific numbers:

The patches from Tuesday fixed a total of 16 CVEs. 11 are Windows only, 4 are Windows+Unix[1], 1 is Unix only. 3 of those (Windows) CVEs were reported externally.

[http://nvidia.custhelp.com/app/answers/detail/a_id/4398](http://nvidia.custhelp.com/app/answers/detail/a_id/4398)

Older bulletins: [http://www.nvidia.com/object/product- security.html](http://www.nvidia.com/object/product-security.html)

[1]: Unix in NVIDIA-speak generally means Linux+Solaris+FreeBSD</comment></comment>


Show HN: Amium 2.0 – Chat and collaborate with anyone in the world - <AUTHOR> https://blog.amium.com/the-new-version-of-amium-the-world-on-your-team-c601aec9edb6 <comment><USER2> Direct link: [https://www.amium.com/](https://www.amium.com/)</comment>


Raspbian Update Breaks Devices - <AUTHOR> https://mender.io/blog/raspbian-update-breaks-devices <comment><USER2> Is the device actually broken, or is the image merely corrupted, where burning a new SSD image will allow you to boot the not-actually-broken device?

I find it unlikely the device is actually broken. The file system, sure...

<comment><USER3> Yeah, bad title. It just mesesed up the install, device is still fine. Article even contains the fix.

<comment><USER2> The fix seems to be using some other software or whatever. I think a more important bit of advice that the article should provide is to ensure any data you need to keep should always be backed up.

In truth, this bug was embarrassing and probably annoying, but no worse than garden variety SSD card failures. No 3rd party rescue software is needed if you prepare for the inevitable SSD failures from the start.

<comment><USER4> Mender isn't a "rescue" program, it's a solid tool for firmware distributors to be able to update devices and avoid this problem. Article is marketing material, but good advice.</comment></comment></comment></comment>


Programmer Recreates London Tube Map in Code - <AUTHOR> http://www.tomshardware.com/news/John-Galantini-Tube-Map-London-Underground-Coded-Map-CSS,22709.html

<comment><USER2> That is pretty special. Looking at the code it would take me years to make that!</comment>


How to improve our Landing Page - <AUTHOR> http://heythere.de

<comment><USER2> If you can, try to pull in some of the reviews from App Store/Google Play. Your primary sell is to get people to give it a try and there's no better way to do that than by testimonies.

Can you perhaps give a better translation for the copy than what I'm getting from google translate?

\- With heythere you'll always know what's happening around you. Let me show you interesting places in your area and tell others what is happening at your place.

\- Look at how heythere works

Your copy should answer the following questions:

1\. What is this website for? (The picture of the mobile suggests it's some kind of app)

2\. What is this app and what do you use it for? -> Answer this question in terms of what your target market wants to hear. The copy will be different for business owners and for regular users.

3\. Why should I give this app a try? The video looks nice, but it should be optimized to answer question 2. What other options have you considered showing off the like/comment functionality?

You should also let people subscribe to a mailing list. You can setup one for free with Mailchimp if you stay within their free usage tier. This will allow you to communicate with your target audience things such as updates, offers, competititions, greetings or sale pitches.

It would also be good if you started thinking about how you could use the app as a tool to offer business services (ex. competitions, advertising).

Hope that is helpful. Best of luck.</comment>


ASK HN: Where can I find NSA's first open source project NiFi source code? - <AUTHOR> https://www.nsa.gov/public_info/press_room/2014/nifi_announcement.shtml

<comment><USER2> Looks like the only code they've published is here: [https://svn.apache.org/repos/asf/incubator/nifi/](https://svn.apache.org/repos/asf/incubator/nifi/)

So it's only the site code for now. We'll have to wait, then. NiFi looks interesting...

</comment><comment><USER3> Appears to be in Apache incubator: [http://incubator.apache.org/projects/nifi.html](http://incubator.apache.org/projects/nifi.html) (svn has only the site as of now.)

</comment><comment><USER4> In what sense would this be properly considered an "Ask HN" topic?

<comment><USER5> I imagine the tags "government" and "open source". And if the answer to the question is "nowhere", then another topic is spawned.

</comment></comment><comment><USER6> It'll be really fun when it becomes accessible - finding a vulnerability in it will be like getting a cheque from Knuth!</comment>


On Palm, Competition, and iTunes Sync - <AUTHOR> http://hunter.pairsite.com/blogs/20091004/

<comment><USER2> _Turns out it's a simple matter of reading the iTunes music library XML catalog file on a customer's computer, and using that to create a sync capability for the Pre._

The author seems to mistakenly believe that this XML file is a documented and supported format for third-party interoperability with iTunes. It's not. The DTD he links to is that of plist files in general, and does not describe how iTunes stores its application-specific information in the file. The quote from the support article is not a message to third-party developers that they should use this file, simply a statement to users that it _is_ used by other applications.

The _only_ documented and supported methods of interacting with iTunes are with visualisation plug-ins, AppleScript, and the COM API.

</comment><comment><USER3> "One reason that the iPod/iPhone and iTunes have been successful is that they work so well together."

So wouldn't it follow, therefore, that a competitor would be less successful if their product didn't work as well with iTunes? In this light, Palm's strategy makes perfect sense. All a user has to understand is "it works with iTunes." Forcing people to install a separate sync app makes for a more confusing experience for a good segment of customers, and regardless is an extra hassle that Palm has removed through their sync approach.

</comment><comment><USER4> While Palm is definitely skating on some thin ice with their approach, I can't blame them for wanting to use iTunes as their synchronization engine... Probably the majority of the market they are going after already uses iTunes. Why force those customers to install yet another application to handle the synchronization of their music and photos to their phone when they already have one that they are familiar with? Unfortunately, their customers are getting stuck in the middle of this cat and mouse game, but it may help them to get a bunch more early adopters on the platform and allow them to eventually build an better and more reliable solution.

</comment><comment><AUTHOR> Palm will be able to do this exactly as long as Apple chooses to let them. The problem Palm is facing is nowhere near as simple as just spoofing the right ID's.

<comment><USER5> Apple cannot go back an update every previous iPod in existence. So at some point, the Palm pre will respond exactly like some existing iPod model and Apple won't be able to do anything more.

<comment><AUTHOR> You're severely underestimating how hard it is to make one device behave exactly like another device. Start with Comer and Stevens back in 1993:

<http://portal.acm.org/citation.cfm?id=1267274></comment></comment></comment>


Notes for a younger programmer - <AUTHOR> http://jmduke.com/posts/notes-for-a-younger-programmer/ <comment><USER2> > whether it's adding documentation, cleaning up syntax, _or fixing an edge > case._

If you're a junior/younger programmer make sure you have your edge-case fix properly reviewed by someone else who knows that chunk of the codebase.

With the best will in the world, unsupervised "edge cases fixing" can wreak bloody mayhem by introducing new edge cases. Been there done that.

</comment><comment><USER3> What does "Decide what you're willing to leave at work. Stick to it." mean?

</comment><comment><USER4> I really enjoy your writing. Thanks for sharing!</comment>


Programmers Need To Learn Statistics Or I Will Kill Them All - <AUTHOR> http://zedshaw.com/essays/programmer_stats.html <comment><USER5> I strongly disagree with Zed on this one (at least part of his diatribe - <USER2> makes some good points about sample size/ramp-up/etc).

I don't care what your mean or standard deviation are. If your performance isn't symmetrically distributed about the mean (and it probably isn't), standard deviation probably doesn't mean what you expect.

I _only_ look at performance in 99th percentile (or sometimes, 99.9th percentile). If 99% of responses are being served in 100ms on my webapp, I'm happy. I don't care if the mean is 5ms with a 20ms standard deviation. As long as the performance at the 99th (or 99.9th) percentile meets a level of 'acceptability' that I have predetermined, I'm satisfied. And you should be too.

This is much easier to deal with than trying to do real math. Just make sure that no more than 1% (or 0.1%, as the case may be) of requests take longer than X.

Pre-emptive answer: I usually get someone asking me why I don't demand acceptable performance at the 99.99999th percentile or at the slowest request. It's simply an issue of resources. To get that kind of performance, you can't use garbage collection, context switches become an issue, disk buffering even comes into play. You have to basically write a hard real-time (or pseudo real- time) app, and that level of effort isn't worth it for a consumer facing webapp. If I was writing code for a pacemaker or something (and please, dear god, no one ever let me do that) it damn sure better be hard real-time though.

<comment><USER6> I can't really agree with you. If you care only about 99.9% of your requests being on time, it means that one in a 1000 can time-out and die. Not that bad? Ok - let's have an example:

Facebook - one random user-page view == 63 requests (reported by firebug). That means 16 people generate >1000 requests. Even with 99.9% requests within your "acceptable range", you get every 16'th person either missing an element on a page due to some DB timeout (for example), or seeing an incomplete page for a not "acceptable" time (with 1/63'th probability of it being the main document, if we don't know which element fails).

Is that really an acceptable behaviour?

I don't believe that a requirement saying "the complete website loads in less than 1s" is affected by GC or context switches at all, tbh. If you actually tested the website with load of 10x the average expected one (over a couple of hours of random scripted walk with a natural write/read balance - <USER3> you'd do on a service you really care about) and you still get random glitches with the expected load, then there's most likely something wrong on a higher level (disk seeks failing? shared hosting overloaded? network? etc. etc.)

<comment><USER7> Taking more than 100ms is not the same things as "time-out and die". Performance is focused on how long something takes and it normally assumes that the system is not broken. It would take a horribly broken system to timeout 1% of requests. Assuming that 1% of your requests taking long enough the user might notice, but still working is generally acceptable performance for a webapp. So a reasonable goal might be 99% within 150ms and 99.95% don't fail.

<comment><USER6> smanek wrote: "I don't care what your mean or standard deviation are.", which if taken seriously, means exactly that 1/1000th request can take an hour to finish and he doesn't care. Anything over 10s in real-life means a time-out in connection, or user refreshing, or going to the next page, or...

Going to 99.95% not-failing connections doesn't help you much IMHO. It means that you've gone from every 16'th person having a problem to every 32'th person. It's not acceptable at all.

<comment><USER7> Your also assuming that he / we measure per element not per page, which is just silly. Also, stating a minimum accepted value is not the same as ignoring the rest. It just places minimum reasonable performance that a context.

<comment><USER6> I read his "Just make sure that no more than 1% [...] of requests" as actual requests. If it meant whole-page-load-time then fair enough.

"stating a minimum accepted value is not the same as ignoring the rest" - <USER4> what is it if you don't care about mean and stddev? If you do care about other requests you care about the mean (or stddev, or median, or whatever other parameter) but just don't put it in a scientific way. If you say reasonable, it might mean - it doesn't time-out. That means it's below X seconds. That means with the other maximum load time constraints, you expect a mean time for Y requests to be less than Z. Saying "Assuming that 1% of your requests taking long enough the user might notice" you just put a constraint on that parameter.

(just to explain why I keep arguing this point - because that's what the post is about (kind-of), people say "reasonable time", but don't want to set the actual mean/stddev - which is what they do care about)

<comment><USER7> No, because taking less than the threshold is effectively meaningless. So, when you calculate the mean and standard deviation the calculation is altered by meaningless trivia. When you add in the latency and rendering times 1ms and 30ms might as well be the same number. What you really want is to round everything under your thresholds up to that threshold and then look at the breakout as x% take 100ms or less, y% take 1 second or less, and z% take less than 3 seconds etc.

You then look at that breakout based on what the page is, or the time of day ect. But, from a performance standpoint the the only really important number is the first number, once enough of your site is fast enough having a few slow areas, or some issues at peak times is reasonable, and once it's slow enough for a user to notice it's a problem.

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> In case you're interested in the (lengthy) original discussion: <http://news.ycombinator.com/item?id=626771>

EDIT: that wasn't the original, but it was the longest. This is the original (I think): <http://news.ycombinator.com/item?id=48006>

<comment><AUTHOR> It's a really good article - I came across it today on my twitter feed, and it surprised me that it still has the same impact it had in 2005.

</comment></comment><comment><USER9> I'm trying to teach myself statistics (for social sciences) right now. I got two books (Agresti & Finlay along with Knoke, Bohrnstedt & Mee) but both seem to be written in jargon instead of English.

Does anyone have recommendations for statistics books that are actually written in English?

<comment><USER10> For a long time, I wanted to teach myself statistics for the social sciences too. I decided the best way to do this was to find an interesting social science issue, and then learn statistics as part of tackling the issue. But there was one problem with this. There are almost no problems in the social science where it makes sense to use advanced statistics to analyze them. Using very simple regressions can be illuminating and illustrative (though not conclusive). But when you try to get more advanced, data quality, data coding, issues with choosing the right controls, too many variables, sensitivity to variable selection, inability to separate causation from correlation, etc. render advanced statistics utterly useless. Do you have a particular social science problem that you think would be a good fit for statistics?

<comment><USER11> I guess it depends by what you mean by "advanced statistics". Is correcting for heteroskedasticity, a fixed-effects regression, instrumental variable regression, etc advanced? Because if so, all of that is pretty common practice in say, economics. Simple linear regression is easy and often illuminating, but good luck getting a paper published if that's all you've done.

I agree that in practice, data munging is 90% of the work and can be pretty tedious and discouraging when working with real world datasets, but if you don't know about issues like the ones I mentioned above, how do you even know where to start?

For examples of how statistics can be used to solve interesting problems in the social sciences, just browse through Freakonomics.

<comment><USER10> Yes, that's what I mean by advanced statistics. I've read many papers in economics using such techniques. They are 99.9% utter garbage. The issues I mentioned above (data quality, data coding, issues with choosing the right controls, too many variables, sensitivity to variable selection) doom the project before you can even get to using the statistics. Garbage in, garbage out. If you data munge and get a result, you can never determine if it's a real result or a data mining effect.

And yes, I understand you cannot get published in the social sciences without using advanced statistics. All this means is that the economics academia is following into the trap of Schoclastism, and insular world that uses complicated and absurd methods of research, increasingly divorced from reality.

Social science needs to get over its science envy. As I've written here before, the proper tools to use as a student of policy and sociology are the tools of product management: <http://news.ycombinator.com/item?id=836196>

I have not read Freakonomics the book, but I read their blog occasionally. It's interesting when they find a simple statistic that raises an interesting point.

It might be easier to explain the problems with reference to a specific example. If there is some social science paper that uses the techniques you mentioned above, and you think is particularly good, send me a link, and I can explain why I think it's likely garbage.

<comment><USER9> I agree with your assessment of the 'scientism' of social sciences, and the rather absurd uses of statistical methods (that should be inapplicable because of data collective and basic assumption issues).

What I am to do is mostly descriptive statistics, but with a few simple tests (e.g. a test of variance with covariance). I've got three populations of civil war data and would like to compare features of those populations against each other (e.g. instance of civil war in period one vs. period two vs. period three). Controlling for # of states, or # of new states (say, within five years of their founding).

Most of the complicated stuff really can't be applied to stuff like data on civil wars, since the data doesn't meet basic assumptions of the models, but I would like to be able to say something more than just 'the data from period one looks different than the data from the other two periods.'

<comment><USER11> > Most of the complicated stuff really can't be applied to stuff like data on > civil wars, since the data doesn't meet basic assumptions of the models

I think this is precisely why we have more advanced statistical techniques. There are ways to correct for or at least detect serial correlation, homoskedasticity, etc, all of which are probably in your data. All real world data is fucked up in some way, having a large toolbox of statistical techniques helps you cope with this fact. Maybe not perfectly, but at least you'll know where/why your model is wrong.

<comment><USER9> Can you recommend a book on advanced statistical techniques or non-parametric statistics that's written mostly in English with a minimum of jargon?

</comment></comment></comment></comment></comment></comment></comment><comment><USER12> Statistics will save us all.

And for the HN folks that like to Lisp it up:

<http://incanter.org/>

</comment><comment><USER13> This is a great resource if you're studying / reviewing on your own:

<http://spartan.ac.brocku.ca/~jvrbik/MATH2P82/Statistics.PDF>

</comment><comment><USER14> Hacker News readers need to stop submitting linkbait titles, or I will kill them all.

<comment><USER15> Quoi? That is the verbatim title of the post.

<comment><USER14> That means you took the linkbait. Why reward articles like this with free views?

<comment><USER16> If the article delivers on the hype, why not reward the author for having the walk to match the talk?

<comment><USER14> Does this article deliver on the hype?

What I got out of it was: "Some people are dumber than me." This is a well- known problem both inside and outside the programming world, and unfortunately killing the people dumber than you is not the solution to the problem.

<comment><USER16> I took it as "some people are ignorant of basic statistics," which isn't quite as daunting as "some people are dumber than me." The cure is motivation to learn, and one can take the view that the hyperbole of the title broadens the post's reach and simultaneously provides a little negative motivation. I didn't take the title quite that literally.

I personally take the view that you catch more flies with honey than vinegar, but I suggest the article as a whole including the title provides a general good.

</comment></comment></comment></comment></comment></comment><comment><USER17> _This article is my call for all programmers to finally learn enough about statistics to at least know they don't know shit._

His call can be generalized: "This article is my call for all programmers to finally learn enough about [insert here] to at least know they don't know shit."

The power that we programmers have as little gods inside systems of our own making, tends to have us over-estimate how much we know. Thus our penchant for posting stuff that makes experts in various technical and scientific fields roll their eyes.

</comment><comment><USER18> Two great books for those who need to re/freshen up their statistics:

[http://www.amazon.com/How-Lie-Statistics-Darrell- Huff/dp/039...](http://www.amazon.com/How-Lie-Statistics-Darrell- Huff/dp/0393310728)

[http://www.amazon.com/Cartoon-Guide-Statistics-Larry- Gonick/...](http://www.amazon.com/Cartoon-Guide-Statistics-Larry- Gonick/dp/0062731025)

I haven't had a chance to check out the manga guide to statistics but that might be a decent introduction, as well.

</comment><comment><USER19> It's a great post, albeit, probably about 3 years old.

</comment><comment><USER20> Just thought I'd share an example where the graphs really help you to understand the performance profile.

[http://s3.amazonaws.com/four.livejournal/20090911/benchmark....](http://s3.amazonaws.com/four.livejournal/20090911/benchmark.html)

</comment><comment><USER21> once again slashdot posts hackernews as "new" news. would be nice if they linked HERE to gain from the new comments not the 2.5yr old comments. oh well.

[http://developers.slashdot.org/story/10/01/09/2154224/Why- Pr...](http://developers.slashdot.org/story/10/01/09/2154224/Why-Programmers- Need-To-Learn-Statistics)

</comment><comment><USER22> P = "Programmers Need To Learn Statistics"

Q = "Zed Will Kill All Programmers"

If P || Q must hold, go with P!

</comment><comment><USER23> Zed Shaw should stick to his area of expertise. His angry, opinionated rants on topics he clearly lacks deep knowledge of are perfect examples of the pseudo-intellectual junk that litters HN from time to time.

<comment><USER24> thanks for that argumentated non opinionated answer !

<comment><USER23> At least I don't write like a slightly retarded teenager, and I don't pretend to be an expert on Statistics. Thanks for the "kudos" anyways.

<comment><USER24> Sorry for the sarcastic answer, never a good thing. My point was, despite Zed's tone , that i found irritating, he does actually provide content in this article, that a newbie like me can use and put at profit (wich i did when the article first came out).

On the other hand, your post sounds angry, gives no insight as to why / how he lacks knowledge (which would have been interresting) and use condescending expressions like "pseudo intellectual garbage"

<comment><USER23> Look, I know it's none of my business, but if you're a newbie and you want to learn some Statistics, wouldn't you be better off checking the MIT OCW pages, or the countless lecture notes in PDF format that can be found on the web? This is an honest question.

Contrary to what many short-sighted "pure" mathematicians think, Statistics is hard. You can't explain statistics in a bunch of blog posts, not even if you are a true expert. You can't condense all that knowledge in a blog post, one must read the books, though painful that may seem.

<comment><USER25> To learn statistics, yes, of course. To get psyched up about learning statistics? That's what this article did for him.

Sometimes it's just fun to read someone go nuts about something like statistics, and it may give you motivation to go learn more about it so that you can understand the whole thing.

<comment><USER26> I agree. The last time I dabbled with statistics was at university and whilst I could see it was useful, it was boring as anything. I couldn't see any really application in my day to day job - after all, besides understanding the difference between mean and medium, what else would I need?

But Zed's article, juvenile as it was in tone, had undeniable passion about statistics. And it was enough to keep me reading through the whole article and pay attention to the examples. As it is, I will be doing further reading on statistics and so Zed's article has been incredibly useful to me.

"Maybe it's just me, but Statistics is a beautiful field, and such intellectual beauty should be all the "psyching up" one needed."

It's just you. Nine out of ten people think statistics are nothing more than boring numbers made up on the spot.

<comment><USER23> Nine out of ten people learned Statistics from teachers who knew jack of Statistics. The difference between Statistics and Math is that Statistics is data-oriented and somewhat "experimental". Yeah, I hated Statistics in high- school, but when I learned Information Theory and Statistical Signal Processing, then I understood what one could do with knowledge of Statistics, and what the field is all about.

If you can't see the beauty in it, you've probably been taught pseudo- Statistics. Don't feel bad. All the Math that engineering students learn is kind of pseudo-Math. All one learns in high-school is BS. If you want to learn something, here's my advice:

i) Don't use modern, over-designed textbooks.. you know... the thick expensive ones with many colors and boxes highlighting the formulas one must memorize.

ii) Instead read the classic books from the 1960s, many of which are published by Dover. They look boring at 1st sight, but their content is rich. Another option is to get the old Soviet books from the 1960s, which tend to be forgotten gems. The Russians are the best at marrying theory with application.</comment></comment></comment></comment></comment></comment></comment></comment>


Who has been running this classified ad for more than ten years? - <AUTHOR> http://workingwithwords.blogspot.com/2004/08/gee-i-wonder-if-this-will-attract-any.html <comment><USER2> I think the answer is in the comments: "6/4/08 - Just saw this and googled it, of course. If I had to guess on hedge funds, this sounds like something D.E. Shaw might try for recruitment. They're big on the whole "just be brilliant" bit in order to recruit talent. Additionally, I noticed in their own recruitment posts on CL the respond-to e-mail address is Craig- firstname.lastname@example.com while that one is firstname.lastname@example.com. Coincidence?"

Blog author's reply: "Aha! At long last, after nearly four years, I believe Jennifer may have finally cracked the mystery. This seems pretty plausible. I only wish her link carried me somewhere that I could thank her. Here's hoping you come back and take a bow, Jennifer. Thank you all for taking part in this bit of distributed intelligence exercise. I loved watching it all unfold. And of course, I'm not discounting the distinct possibility that it will continue to unfold in ways I couldn't have guessed."

[http://workingwithwords.blogspot.com/2004/08/gee-i-wonder- if...](http://workingwithwords.blogspot.com/2004/08/gee-i-wonder-if-this-will- attract-any.html#c7218485345738743166)

<comment><USER3> Very interesting and I wish to say thank you. Pat C. in Texas

</comment></comment><comment><USER4> For anyone else who was confused by this submission:

-The second part of the blog post ('Coming Across This Line [...] no better than its woods." Indeed... ') appears to have nothing to do with the first part (the quoted classified ad) and can be ignored.

-The blog post's comments section chronicles various sightings of this ad over the years. Unfortunately, the site only shows the _time_ the comment was posted, not the date. The first comment is from around 10 years ago.

</comment><comment><USER5> CTRL + F "Tue. Nov. 24th 2009" on that page. Apparently someone got into the Gmail account of one of the emails used for that ad and found various references to [http://deshaw.com](http://deshaw.com), as well as the various other people mentioning D.E Shaw in the comments.

I guess that's the end of the mystery? Or am I missing something?

<comment><USER6> >CTRL + F "Tue. Nov. 24th 2009"

Off topic, but is there a linking service where the reader can be directed to an arbitrary location within the page? In this case we could just click on the link and it'll directly take us to that particular comment?

I'm just surprised that there's still no way for me to link someone to a particular paragraph within a Wikipedia article in the year 2014.

<comment><USER7> Let's say you're reading the Land Rover (automobile) Wikipage and you want to share the 'Timeline' section with someone.

Here is the wiki [http://en.wikipedia.org/wiki/Land_Rover](http://en.wikipedia.org/wiki/Land_Rover)

If you click this one below, you will be brought straight to the 'Timeline' section [http://en.wikipedia.org/wiki/Land_Rover#Timeline](http://en.wikipedia.org/wiki/Land_Rover#Timeline)

Just append your URL with the selector for a DOM Element with a unique ID. In this case, the 'Timeline' div has a unique ID of 'Timeline', so I've simply appended that to the URL.

Welcome to 2014 :)

<comment><USER6> I'm trying to link to arbitrary paragraphs i.e. paragraphs without DOM selectors.

</comment></comment></comment></comment><comment><USER8> The answer is obvious and simple to anyone who has worked for "one of Wall Street's most successful entrepreneurs."

These assholes churn through assistants like you wouldn't believe. Most quit, in spite of the high pay, out of basic self-respect. The others are fired during a tantrum.

Most of these jagoffs have a few pending wrongful termination lawsuits at any given time. Just ask them, they'll probably brag about it.

<comment><USER9> As someone who has no idea what the working environment is like, I'd love to hear some stories. What's day-to-day life like in that sort of job? I guess I'm just curious what the owners do to make it so unbearable to work there. And why they don't care.

<comment><USER10> After reading these other comments, I Googled David E. Shaw and found this: [http://mathbabe.org/2012/04/05/it-sucks-to-be- rich/](http://mathbabe.org/2012/04/05/it-sucks-to-be-rich/). (WARNING: Not sure of the validity of these stories, but they're interesting.)

> First example: David hires a Ph.D. in English literature (he has a thing for > "geniuses", even in the mail room) to test mattresses for him. So that > person's job is to sleep on 15 different mattresses, for 8 nights each, and > draw up a report to tell him the pros and cons of each mattress. This is to > avoid him having an uncomfy night's sleep. That's what the risk was that we > were avoiding with that.

> Second example: David wants to be sure his trip to California goes smoothly, > so he hires a Ph.D. in Something to take the exact same trip – same car > service to the NY airport, same flight (same seat on plane!), same car > service upon arrival, same hotel, exactly a week before his trip (due to > understood seasonality issues of air travel) – to make sure there are no > snags, and to draw up the report that presumable explains how much leg room > there was in his plane.

As an aside, their research group encouraged me to apply for a position shortly before I graduated college. The application asked for things like SAT/ACT scores and (number of) lines of code written in various languages. I thought my application looked pretty good, but I guess my ACT score was too low for them to consider me (even potentially!) brilliant. I still feel kind of bitter about that.

<comment><USER11> I commented earlier that I thought about applying to that classified job in 2007 - as it happens, in 2008 I briefly worked as the _personal assistant to a personal assistant_ at D.E. Shaw. I was working alongside Harvard business school types, but doing the most trivial things. I literally had to run across midtown to deliver a six pack of Diet Dr. Pepper to one of Shaw's executives at one point. Half the people I met there were humanities PhD dropouts. I applied to humanities PhD programs while working there and sort of did the reverse.

</comment></comment></comment></comment><comment><AUTHOR> Here's an example, from the latest New York Review of Books: [http://www.nybooks.com/classifieds/](http://www.nybooks.com/classifieds/)

<comment><USER11> I thought that ad looked familiar! I'm almost positive that I thought about applying to it when I graduated college in 2007.

</comment></comment><comment><USER12> At 3:35 PM, Anonymous Anonymous said...

 _For me, the experience was just as weird and intriguing as the ad itself. First off was the reception area. The cathedral ceilings throw scale completely off, dwarfing the visitor...[http://tiny.cc/ne5pew](http://tiny.cc/ne5pew) It does convey a sense of power and high-tech savvy, but it also seemed dated. A young, very polished woman took my jacket. I looked her up later and she'd been a child actress in at least one film that I'd seen. Everyone was very nice in kind of a cult-y way. I was led to an office in the middle of a floor full of empty workspaces. The guy who interviewed me was a lawyer and said he still worked part-time on the side, and that that was an accepted part of the culture. Mr. Shaw had basically unlimited money thanks to the success of his hedge fund, but he did not have unlimited time. To create more time, they were staffing up for personal assistants who could handle everything from getting Knicks tickets to making dental appointments. We both figured out it wasn't a match early on, so we had a nice conversation instead. And somewhere in that building I guess my resume still sits..._

edit: The thread started by yesprocrast2 covers the topic further. I guess working among geniuses for one of Wall Street's winners isn't all it's cracked up to be.

</comment><comment><USER13> Perhaps this is a slightly more sophisticated version of the "Make $10,000 a month working from home" scheme used to recruit mules for money laundering.

<comment><USER14> _that_ is what those announcements are for? I always assumed they were scams of the "send us 100$ to start" kind.

</comment></comment><comment><USER15> Maybe we'll never know...

[http://m.huffpost.com/us/entry/166236](http://m.huffpost.com/us/entry/166236)

<comment><USER16> I saw the ad in hardcopy in The New Republic sometime in the 1990s. I replied but got no response. Sometime five or six years ago I applied directly to D. E. Shaw and came up empty. I believe that that these days Shaw himself spends his time on research in tool building for computational molecular biology. Has another company for that.

</comment></comment><comment><USER17> Well, if you go to spsfind.com, it 301 redirects to sps-app.com, which seems to be a social positioning app, in Hebrew. Doesn't seem to be related to D.E. Shaw, though.

</comment><comment><USER18> A return address for a spy ring?

</comment><comment> vezzy-fnord Well, given that it seems to be confined to the NYRB and has been consistently going for so long, could it perhaps be an in-joke of theirs?

EDIT: Well, I guess not. Looks like the resolution was far more anticlimactic.</comment>


Analyzing the Facebook ads placed by the US presidential candidates - <AUTHOR> https://medium.com/applied-data-science/56-070-165-facebook-ad-spend-of-us-presidential-candidates-broken-down-by-age-and-gender-2dcc32fe2c02 <comment><USER2> How exactly did they get access to the Facebook Ad Library API? You need special permission for that: [https://www.facebook.com/ads/library/api/](https://www.facebook.com/ads/library/api/)

Also, per those docs, ad spend is a range; I'm not sure how the article infers exact values.

<comment><USER3> No, you don't, at least not in the sense of "Facebook approves your use case based on some subjective criteria."

Per the page you linked, as long as you verify your ID/location, are from one of the countries where the Ad Library has launched, and agree to the Facebook Platform Policy, you can get access.

</comment></comment><comment><USER4> This is awesome. There's a vast amount of data being made available, and it's great that people are making that data more accessible through visualizations and pattern detection.

There's a site that does a similar kind of analysis and offers regular updates to their reports: [https://www.adspend2020.com/](https://www.adspend2020.com/)

</comment><comment><USER5> I'd like to see something similar on for YouTube and Twitter for the recent Hong Kong protests, I'm seeing lots of clearly political adverts on both platforms labeling protestors as "Black Terror" and the such.

[https://i.redd.it/00nqhz8ufnf31.jpg](https://i.redd.it/00nqhz8ufnf31.jpg)

</comment><comment><USER6> Gender and age are interesting. But elections are won or lost based on ad copy targeted at interest groups... neither appear to be available in this analysis(?)

<comment><USER7> Yeah this analysis is great, but we have actually been classifying Facebook Ad Spend by issue i.e. Environment, Economy, Guns, etc - check it out on our site www.adspend2020.com

<comment><USER8> The attack ad category is _very_ interesting but I suspect also very controversial.

It would also be interesting to see how many ads are identically shared between candidates of same party and how large percentage is unique for each candidate, and preferable being able to select them out (maybe a bit fuzzy so the difference is not just a few characters at the end that adds 2% uniqueness).

</comment></comment></comment><comment><USER9> Interestingly enough, it might be on purpose the Democrats are not targeting old white men. There could be something to be said that if the 60 year old football loving men of the world get bombarded with Elizabeth Warren ads they will turn out to vote AGAINST her when they otherwise would've stayed home.

<comment><USER10> 60 year old white men have voted in every election for the past 40 years and people generally don't switch ideology this late in life. The democrats strategy has always been to improve turnout from groups that dont vote in every election. There are more democrats in the US than republicans, but not everyone votes. Elections are determined on getting good turnout, not swaying people to jump ship to your side.

</comment></comment><comment><USER11> This is an interesting article show casing what kind of data is available through the API. There are many graphs, lots of data. But, little insight from my perspective. Democrats are rather targeting women and younger demographics than Republicans. Oh well...

</comment><comment><USER12> since when do you have to pay to read Medium posts? We have our clients use it for 'blog' content for ease and SEO, but wouldn't advise if the articles can't be read for free...

<comment><USER13> For many many months. it seems to be inconsistent like they are still figuring out their strategy.

</comment></comment><comment><USER14> But there is no input from the campaigns about how they choose their spending. I might guess they do their a/b testing and keep the ads that engage users more. this might be indicative of each candidate's audience. Assuming that this is true Sanders seems to be having the widest reach, judging from how close m/f ratio is to 50%

<comment><USER15> From the article:

"Facebook allows you to select the location, age range, gender and 'interests' of your audience. Only people that fall into your target audience will see the sponsored ad. Specifying 'interests' indirectly affects the distribution of age, gender and location."

</comment></comment><comment><USER16> This is great data, but missing one crucial piece: how much reach did each candidate achieve given their spend? Both what they directly paid for with the ads, and then what the ads achieved organically beyond the spend.

My suspicion is that Trump, by being outrageous, realizes far cheaper effective ad rates (i.e. when you factor in both paid & organic reach) than any other candidate.

</comment><comment><USER17> Clearly some candidates have learned from prev elections and some have not.

</comment><comment><USER18> This is super cool! I'd be really curious to see how the messaging to particular demographics compares as well. Wild that the Trump campaign is spending so much money...

<comment><USER19> Why? The D's as a group are outspending Trump, in fact the top 3 D's are outspending Trump. I'd be interested to see this type of analysis for the 2016 (and 2012 and 2008) elections.

<comment><USER20> Well it surprises me because Trump already has the nomination and it's not even time for the general election yet, the D's are essentially running right now, Trump is preparing to run.

Edit: actually looking at his ads, they're asking for fundraising so that makes sense if he's getting positive ROI.

<comment><USER21> Trump doesn't technically have the nomination yet - [https://fortune.com/2019/04/22/republican-presidential- candi...](https://fortune.com/2019/04/22/republican-presidential- candidates-2020/)

</comment></comment></comment></comment><comment><USER22> This link re-routes to Medium, can we get that fixed?

<comment><USER23> Sure thing! Updated from [https://adsp.ai/blog/us-presidential-candidates- facebook-ads...](https://adsp.ai/blog/us-presidential-candidates-facebook- ads/).

</comment></comment><comment><USER24> i don't have a facebook account so i get a warning at the top: "By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies"

this is so against gdpr it's crazy.

facebook, what the hell?

<comment><USER25> Are you an EU citizen?

<comment><USER26> I think the question is "Are you in Europe?"

</comment></comment></comment><comment><USER27> Clinton didn't spend anything on Facebook? The data is also a little screwed considering that it was Trump and Hillary in the General election. All of the other candidates were just in the primary. Ad spending will heavily increase for a candidate in the General election.

<comment><USER8> The article cover spending between May 2018 and July 2019.

Clinton announced On March 4, 2019 that she will not run for president in 2020. ([http://westchester.news12.com/story/40067049/im-not- running-...](http://westchester.news12.com/story/40067049/im-not-running- hillary-clinton-rules-out-2020-bid-for-first-time-on-camera-in-exclusive- interview-with-news-12))

The article is not about last the election cycle 3-5 years ago.

<comment><USER27> Fair enough - I scrolled over the dates. That doesn't dismiss the fact that you have a national party backing compared to individuals. In which case democrats are heavily outspending republicans.</comment></comment></comment>


Show HN: Space Invaders (more or less) - <AUTHOR> https://bitbucket.org/kennethrapp/spaceinvaders <comment><AUTHOR> This is a Space Invaders clone I've been working on for the better part of a year, as i've been teaching myself some basic game development stuff with C++ and SDL2, such as dealing with pointers, spritesheets, fonts, entity-component frameworks, states, etc. The actual game itself is rudimentary but I thought was far enough along to get feedback on. Keyboard and controller are supported, but my controller didn't return all of the button states and I don't know why (SO question here getting no love:[https://stackoverflow.com/questions/33974828/sdl- gamecontrol...](https://stackoverflow.com/questions/33974828/sdl- gamecontroller-cant-read-button-down-event))

It's set up to compile using Visual Studio through a batch file on Windows, because after seeing Casey Muratori set a project up that way on Handmade Hero, I tried it with an SDL project on Windows and it worked. There's no makefile, sorry. I should also apologize in advance for the quality of the code - it's not well documented or anything.

This is basically what it looks like: [https://www.youtube.com/watch?v=H9hwWUDT9f8](https://www.youtube.com/watch?v=H9hwWUDT9f8)

I hope someone has fun with it. Bad as it is, I certainly had fun making it.</comment>


Garry Shandling has died - <AUTHOR> http://variety.com/2016/tv/news/garry-shandling-dead-larry-sanders-show-show-1201738601/ <comment><USER2> Is it just me or has the frequency of celebrity deaths made an uptick recently?

I'm wondering if this is just observational bias or a complication of the dense baby boomer population aging into their mortality.

<comment><USER3> How many celebrities did you know who were born in the 1920's and 1930's? You're just observing more people who you've known. Television didn't get big until the 1970's and cable didn't really impact until the 1980's, which made a lot more people well-known.

<comment><USER2> Television was huge well before the 1970's. Movies have been big since before the 1930's.

Stars are not a new phenomenon, seeing one die more than once a week is.

<comment><USER3> There weren't nearly as many shows so there weren't nearly as many stars.

No one said movies weren't big before the 1930. Most of us weren't around to see them so it's irrelevant to the conversation.

<comment><USER2> Fair point.

</comment></comment></comment></comment></comment><comment><USER4> Very sad. Here's some gold:

[https://www.youtube.com/watch?v=pfNHX_gVKJ0](https://www.youtube.com/watch?v=pfNHX_gVKJ0)

Watch all seasons of The Larry Sanders Show!

<comment><USER5> It truly is the godfather of modern cringe comedy - today's landscape of comedies centered around fundamentally unlikable characters in uncomfortable situations is something I think wouldn't exist without Larry Sanders. RIP

</comment></comment><comment><USER6> reminiscing with Jerry Seinfeld

[http://comediansincarsgettingcoffee.com/garry-shandling- its-...](http://comediansincarsgettingcoffee.com/garry-shandling-its-great- that-garry-shandling-is-still-alive)

</comment><comment><USER7> Why in the world is this on hacker news? Did GS have some role in tech I didn't realize?

(Not trying to be too big of a dick, just confused)

<comment><USER8> HN is explicitly not just about startups and tech:

[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

</comment></comment><comment><USER9> Hail Hydra.</comment>


Diaspora Revealed: Sparse, But Clean; Source Code Released - <AUTHOR> http://techcrunch.com/2010/09/15/diaspora-revealed/ <comment><USER2> Other open source social networking projects:

Appleseed (<http://opensource.appleseedproject.org>) Elgg (<http://elgg.com>) OneSocialWeb (<http://onesocialweb.org>)

I realize everyone's hyped up about Diaspora, but we have a long haul before we have anything usable by the general public, and there are projects that are much farther along that deserve attention.

<comment><USER3> > that deserve attention.

Why do they deserve attention?

Every article about diaspora I see this being trotted out, a bunch of other projects that are claiming that they somehow deserve attention and probably more than the diaspora guys.

If they want that attention then they should do a better job of marketing themselves! That does not mean using articles about diaspora to claim their share of the eyeballs, but to independently create events and stuff worthy of mentioning in the press.

Most of these projects are old enough that they should be able to stand on their own legs by now instead of trying to piggy back on related PR like this.

Diaspora has a bunch of things going for it, and one of their major assets is that they seem to understand marketing and playing the press and the public _much_ better than their counterparts, no matter how far along. So copy a leaf from that book instead of claiming those projects 'deserve attention'.

<comment><USER2> Why do they deserve attention? Because they have better, more mature code- bases and more experienced programmers developing them. Other than publicity, what does Diaspora have going for it, and why do they deserve attention just because of their marketing? This is open source we're talking about, not startup companies, and open source works best when it's meritocratic. And if we're talking about which open source project to build a startup with, then we're talking about merit, nothing else.

<comment><USER3> > Because they have better, more mature code-bases and more experienced > programmers developing them.

So then maybe they should divert some of their attention to getting the word out.

> Other than publicity, what does Diaspora have going for it, and why do they > deserve attention just because of their marketing?

A great story, good timing and the ability to engage the community.

> This is open source we're talking about, not startup companies, and open > source works best when it's meritocratic.

In an ideal world, yes. But in this one, no, not really. Open source works best when there is momentum behind a project, not because it is 'the best project' or has 'the most qualified people' working on it. See PHP.

> And if we're talking about which open source project to build a startup > with, then we're talking about merit, nothing else.

Open source projects have to market themselves just as much as commercial entities do.

We wouldn't be having this conversation if that were not the case.

<comment><USER2> PHP caught on because it was the right hammer for all the nails people had. It wasn't marketing and hype, but momentum.

Open source projects may have to market themselves as well, but that doesn't mean the more savvy amongst us have to get swept up with the crowd.

</comment></comment></comment></comment></comment><comment> loup-vaillant > _The project also faces the hurdle of trying to popularize an open source > project — these projects often sound great on paper, but doesn't work too > well in practice._

If I read correctly, they are suggesting that if Diaspora were proprietary, it would have been easier to market. That sounds so impossible that I'm wondering if they really wanted to say that.

I'm also wondering about their second statement : are they saying that open source projects actually "don't work too well in practice", or is this a statement about the general _perception_ of open source projects?

</comment><comment><USER3> <http://news.ycombinator.com/item?id=1696396>

</comment><comment><USER4> i'm porting the protocol to couchdb <http://github.com/maxogden/couchappspora></comment>


Ask HN: Should I charge for my product (20k Users) - <AUTHOR> Hi HN,
I am looking for some advice, I am the founder of http://veed.io/ an online video editing platform. We have around 20K Monthly users and we have doubled that month over month for the last 3 with no marketing.
The site has been free to use while in beta and i believe we will be in beta for 2 more months or so. I want to start thinking about charging for the product in some way and this is where I need your help!
Our main competition adds a watermark to every video, you pay a subscription of $20 to have them removed. Simple. They are doing well and growing fast.
I don't want to add watermarks because I think they are ugly and ruins the user's video. I also believe it will give us a competitive advantage if we don't.
I thought about offering a PRO tools package to our users, but most of our users are entry level if they want more powerful editing tools they will go to some other product.
The other idea is to charge for cloud storage? But our users are not making complex videos. they are just cropping the video and adding text. Once they have posted the video on social, they are done.
I would really appreciate your thoughts and advice on this.
Thanks Sabba - @sab8a <comment><USER2> Can you allow free editing for videos under a certain length, and charge for longer ones? Or allow access to basic tools for free but charge for access to the more advanced ones?</comment>


Apple leaks new MacBook Pro with "touch strip" - <AUTHOR> http://www.theverge.com/2016/10/25/13408862/apple-new-macbook-pro-with-touch-strip-leaks <comment><USER2> Earlier discussion on referenced macrumors.com article: [https://news.ycombinator.com/item?id=12790840](https://news.ycombinator.com/item?id=12790840)</comment>


Inter-universal Teichmüller Theory, beautifully introduced by Shinichi Mochizuki - <AUTHOR> http://www.kurims.kyoto-u.ac.jp/~motizuki/sokkuri-hausu-link-english.pdf

<comment><AUTHOR> Related to yesterday's claim of proof of the ABC conjecture[1]. I found this while looking through Mochizuki's wonderful and whimsical website[3] for any hints about how to understand his approach.

In his words..

 _"IUTeich" lies in the image of a sequence of nested universes. The theta function corresponds to the gaze of the little girl into the "small house"_

[1] <http://news.ycombinator.com/item?id=4476367>

[2] <http://www.kurims.kyoto-u.ac.jp/~motizuki/top-english.html></comment>


I Created Micro CRM, the CRM for People Who Hate CRMs - <AUTHOR> Hello Community!
My name is Charles-Eugene Loubao, I am a software developer who recently turned into an Indie Maker and I am sharing my new product with you today.
Micro CRM is a Customer Relationship Managment web app built to be easy to use and intuitive. Most CRMs can be complicated to use and come with an expensive price tag. Micro CRM is built to fill that need for a much simpler and cheaper contact managment platform that offers compeling features without being overwhelming.
What can I do with it ?
- Keep all your contacts in one place - Timestamped notes can be used to keep track of events associated with your contacts, or as a call log.
By getting the Premium Plan you also can also:
- Import your existing contacts from Excel CSV files - Organize easily with tags - Create email reminders to help you remember follow-ups
What's next ?
Micro CRM is in it's early days and I am planning on adding the following features:
- Search - Sorting and Filtering - Custom fields - Contact attachments (files, links, images, etc) - Team Collaboration - Possible Integrations (email, calendar, Slack, etc)
How much does it cost ?
Micro CRM is free to use for manual entry and simple contact managment. The premium plan is $5/month and for a limited time I am offering a free 30 days trial with no credit card required when you create your account.
Head to https://microcrm.cc and create your free account today! <comment><USER2> Hey, I recommend you post this again as Show HN: for more visibility

<comment><AUTHOR> Done: [https://news.ycombinator.com/item?id=21403091](https://news.ycombinator.com/item?id=21403091)

<comment><USER2> Congrats on a front page :)

<comment><AUTHOR> Thank you!</comment></comment></comment></comment>


Email in the 18th century: the optical telegraph - <AUTHOR> https://www.lowtechmagazine.com/2007/12/email-in-the-18.html <comment><USER2> Sadly unreadable on mobile.</comment>


Show HN: Slackarma – Give Teammates Reputation Points in Slack - <AUTHOR> http://slackarma.gaberoeloffs.com

<comment><AUTHOR> Hey guys, I just launched Slackarma. To give someone a reputation point, all you have to do is type "++ @teammate" and to check reputation points, you just have to type "++ leaderboard." I would love to hear some feedback!

Also, the code is at [https://github.com/gdroel/slackarma](https://github.com/gdroel/slackarma)</comment>


Inside Hong Kong's Chaotic and Lawless Kowloon Walled City - <AUTHOR> http://www.businessinsider.com/infographic-inside-hong-kongs-lawless-kowloon-walled-city-2017-9 <comment><AUTHOR> Fascinating stuff:

[http://cityofdarkness.co.uk/category/the_people/](http://cityofdarkness.co.uk/category/the_people/)</comment>


Python Scientific Library - <AUTHOR> http://www.scipy.org/

<comment><USER2> Hello. Reading this new get back to my mind a debate I have with a friend of mine, maybe you can help us ; )

My friend works in the scientific research domain, in chemistry. In his lab he makes simulation of water particles and often he launches simulations that can take even a week to complete. He is using frontan.

I am using ruby since a few years and, as usual, I love its simplicity,etc,etc.

What I've been always wondering is if would be feasible to use a High Level Language (as Ruby, Python, etc) in a case like his.

I understand that fortran is language for the scientific community with loads of libraries but what matter me is the performance.

Saying Fortran and (maybe) Python with this library have the same functionalities, would my friend simulations take longer using Python rather than Fortran? how much longer? do you think it worths the jump to a nicer (from the programmer p.o.v.) programming language?

What do you think?

thanks ; )

<comment><USER3> The reason why fortran is used for simulations is twofold, first there is a large and very well tested set of software tools written in fortran for exactly that purpose, these codebases have been through hell and back with respect to having their output inspected. They are very close to being bug free. A re-write of such a package would cost considerable time and would probably not yield better results, and rewriting them in an interpreted language would yield those results considerably slower. Programming staff time is expensive, so is computer time.

Second, the hardware that these simulations run on ('supercomputers') is optimized for vector processing, which is an expensive way of saying that you are operating on whole arrays of data at a time.

Typical operations are multiplying two vectors of data with each other, multiply-and-sum operations and so on. Because simulations of real world scenarios almost always contain large amounts of these operations 'supercomputers' are one way to get good performance, and most numerical libraries in fortran are available for them, and sometimes even optimized for that particular hardware.

But even on commodity hardware (clusters of pc's) the general availability of tested code is the cause of inertia in the scientific world, these people are trying to solve a problem, they're not 'in love' with any language, the tool that suits them best (libraries available, co-workers versed in it, citations all over the place) is what they'll use.

If you want to get very high performance numerical work done on a pc I'd advise you to look into CUDA. Fortran bindings for CUDA exist so you can use all those libraries at absolutely amazing speeds for very little money.

<comment><USER4> FORTRAN bindings for CUDA simply allow you to call compiled CUDA libraries from FORTRAN, but they don't let you take libraries written in FORTRAN and compile them to run on CUDA. However, there is a company that nVidia has just partnered with that is writing a FORTRAN compiler for CUDA, which _would_ let you run those libraries on the GPU (though I can't say whether they will work without some modifications).

<comment><USER3> I must have misinterpreted some piece of text somewhere then, I remembered it that way, apologies.

<comment><USER5> Actually, initial CUDA trials can be quite disappointing, unless heavy effort is spent in understanding the architecture, so to be able to exploit it. Then you are back at square one: (almost?) a rewrite. GSI has nice examples about their studies on different architectures. Then again, the real danger when going with non mass-consumer items, is that the product lifecycle ends before you finish your project. A weird but teaching example, even in case of mass produced technology: a high-performance switch maker used a chip that was built for a mainstream game console. As the console switched generations, the part availability dried up...

I just hope that OpenCL takes off and becomes really open (read free as in free drivers and code). Once familiarized with the quirks of a new architecture (read non x86-alike) then it would require less investment to draw computing power (btw. Nvidia is a power-hog, significantly reducing its appeal in large data centres).

<comment><USER4> I've worked with CUDA a pretty good amount, so I feel obliged to point out that most of initial effort put into porting an algorithm to CUDA is not spent learning the architecture, but parallelizing the algorithm (which is something you would have to deal with even if just porting your code to multi-core CPU). Once you have figured out how to parallelize the algorithm, then you need to worry about the architecture, but only if you need to eek out every last bit of performance. In most viable cases, you should be able to get a pretty decent speedup without tearing your hair out.

In any case, OpenCL is based on CUDA (the driver API), and they are practically the same (compare the reference manuals and you'll see what I mean).

Also, GPU computing is much, much more efficient than CPU computing (in terms of FLOPS/Watt). A high-end GPU (say, a Tesla C1060) can probably pull between 200-300 watts (~2x what a high-end Xeon uses), but can do over 1TFLOP with sufficiently optimized code.

</comment></comment></comment></comment></comment></comment><comment><USER6> No new release or CFP. Why is this here again?</comment>


Ask HN: Bandit Algorithm and Postgresql - <AUTHOR> I have been thinking about using bandit algorithms to optimize headlines, sub-headlines, and colors on a set of pages.
I have wanted to track the score for each aspect in a Postgresql database, but I am still trying to work out a set of tables to represent this.
Has anyone done something like this and what type of table structure has you used? <comment><USER2> firstname.lastname@example.com</comment>


Show HN: datememe.com - <AUTHOR>

Hey HN users,  I wanted to get your feedback on a site i've been working on for a few months.  It's a free dating site that uses your likes and dislikes on various memes on the internet to match you up with a potential partner.
While still in the early stages of development I wanted to see if the HN community has any feedback for me in regards to the idea, the UX/UI and or comment on the technology choices i've made.
datememe is built as single page web app on top of node.js and its amazing modules: Backbone.js, connect.js, express, jade, mustache, socket.io to name a few.  After the first page load all client to server communication is done via websockets and views are rendered on the client.
The database is redis which is extremely fast and a pleasure to work with after years of working on RDBMS systems.
For me this was a learning project but i'm happy to say that these technologies together provide a great framework to deliver real time event driven user interfaces.  I can't wait to see how it scales.
Your feedback is greatly appreciated! <comment><USER2> This is possibly the worst - but funniest and most innovative - idea for a dating website I've ever heard of. The signup button didn't work in Chrome, so I had to get the signup URL from your HTML code. Just signed up now, will venture into the void.

</comment><comment><AUTHOR> <https://www.datememe.com/signup> here!</comment>


Crystallographer faked data - <AUTHOR> http://www.the-scientist.com/blog/display/56226/

<comment><USER2> Something like this happened/is happening to a professor that I worked for in grad school. He has retracted several papers and will probably be retracting at least one more from top journals (Nature, Science, etc). Fortunately for me I was only in the group for a very short time and never had any publications as a result. This can't be said for the other students I knew in the group, although it was one of the grad student co-authors who apparently "turned him in".

My assessment of the situation is that the prof is extremely capable, but the pressures of being a tenure-track assistant professor, probably combined with the arrogance of thinking he could get away with faking data, made it too tempting.

I think he will certainly end up shut out of academia, but at the same time I don't think you'll hear about it, because too many people stand to lose face if it is publicized. I think this is probably the most common outcome of "scientific misconduct" when it is actually caught and punished.

</comment><comment><USER3> What does this guy's future look like now? Nobody'll give him a job in science right?

Is there any way he can use his expertise outside of starting his own company that somehow monetizes it?

Would he be accepted as a high-school teacher?

</comment><comment><USER4> A timely article, given some higher profile accusations that have recently surfaced...

Newton appears to have faked experimental data in the optical research he published, in that he interpolated experimental results to accuracies he could not possibly have measured. Academic standards were a little different in those days.

</comment><comment><USER5> Were the journals that published his articles peer-reviewed or not?</comment>


Readyforce Acquires Umeqo – The MIT Campus Recruiting Event Management Platform - <AUTHOR> http://blog.readyforce.com/readyforce-acquires-umeqo-the-mit-campus-recruiting-event-management-platform/

<comment><USER2> We are very excited about this acquisition. Awesome to see what ideas are coming out of colleges.

The tech world should watch these students: Joshua Ma, Zachary Dearing, David Guo, Arkady Blyakher, Joshua Dhaliwal, Carolyn Zhang and Dmitrij Petters. Congrats guys.</comment>


WeWork: The 'hypothetical' company at the heart of the property market - <AUTHOR> https://www.ft.com/content/0e426c90-8c45-11e9-a1c1-51bf8f989972 <comment><USER2> Paywall.</comment>