Ubuntu 19.10: It's fast - <AUTHOR> https://arstechnica.com/?p=1623735 <comment><USER2> It's worth it to try KDE Plasma rather than GNOME. It's lighter and faster than GNOME by a fair margin, and it's far more customizable. GNOME is great for your OS X convert who isn't used to anything being customizable, but KDE is an utter delight for everyone else.

<comment><USER3> Unless something changed recently, it is weird to say that GNOME/KDE is heavier/lighter when they are both insane resource hogs. I use Xfce or MATE nowadays.

<comment><USER4> Plasma/KDE is mostly great, but a major thing that GNOME and Xfce still do better is GVFS and GVFS-FUSE. That means I can browse to smb://something or sftp://something, double click a file and it just works whatever the application is (e.g. play in VLC, edit in VS Code). This has never worked in KDE where only KDE apps can open remote files (and even that has bad performance).

<comment><USER5> GVFS/GIO is nice, but at least on LTS, it can be a bit rough (with i/o errors under load, at least during PhotoStructure imports). Also, somewhat frustratingly, the GIO system mounts network file systems with paths that include a colon character, which breaks standard PATH parsing.

</comment></comment></comment></comment><comment><USER6> A bit more information about the work done around performances and what's still being worked on: [https://discourse.ubuntu.com/t/boosting-the-real-time- perfor...](https://discourse.ubuntu.com/t/boosting-the-real-time-performance- of-gnome-shell-3-34-in-ubuntu-19-10/13095)

<comment><USER7> This is a _great_ read with extremely clear breakdowns of what are clearly complex bugs.

</comment></comment><comment><USER8> > _Most improvements in 19.10 can be attributed to the latest release of GNOME > 3.34, the default desktop for Ubuntu. However, GNOME 3.34 is faster largely > because of work Canonical engineers put in_

At least everyone can stop hating on Gnome now. Secondly, Fedora 31 has had Gnome 3.34 for more than a month now. This article does seem like a disservice.

<comment><USER3> >At least everyone can stop hating on Gnome now.

Performance isn't only reason for that. Mainly bizarre GNOME3 design decisions: lack of taskbar, tablet-like interface, atrocious file picker dialog, etc. Also maintainer attitude to themes and extensions, here's one rant from 2012: [https://igurublog.wordpress.com/2012/11/05/gnome-et-al- rotti...](https://igurublog.wordpress.com/2012/11/05/gnome-et-al-rotting-in- threes/)

<comment><USER9> Are you seriously claiming that Gnome hasn't changed at all since 2012?

<comment><USER3> No, but most critique is still valid. Weird tablet-like UI, missing features. (like that absurd file chooser which lacks prefix-search in current directory)

</comment></comment></comment></comment><comment><USER10> There is an article [1] by an Ubuntu engineer that describes the performance boost.

For Ubuntu 20.04, there will be extra work for performance boost on high-end computers.

For Ubuntu 20.10 there will be extra work for performance boost for low-end computers.

1: [https://discourse.ubuntu.com/t/boosting-the-real-time- perfor...](https://discourse.ubuntu.com/t/boosting-the-real-time-performance- of-gnome-shell-3-34-in-ubuntu-19-10/13095)

<comment><USER11> I'd like to see them quantify/qualify what they mean by low and high-end. I would have considered low-end to be a dual core i3-class CPU with no dedicated graphics but by the time 20.10 is out I'd be more inclined to say a quad core is low-end. Obviously a Threadripper is high-end today and will be even higher-end next year, with 64 cores perhaps.

<comment><USER12> When Canonical says low-end hardware, I'm inclined to believe they mean actual cheap low-end hardware. Ubuntu has supported the raspberry pi for years now. A raspberry pi is much much lower end then even a bottom of the barrel i3 processor

</comment></comment></comment><comment><USER13> Meanwhile, Gnome still can't manage to display a desktop wallpaper when running at 4k resolution and 200% scaling.

[https://gitlab.gnome.org/GNOME/gnome- shell/issues/1175](https://gitlab.gnome.org/GNOME/gnome-shell/issues/1175)

Kind of hard to get excited about perf improvements when relatively basic functionality manages to be broken for at least 7+ months and counting.

<comment><USER14> This is why I am on Mint. Turn off CPU vsync, turn on full-frame composition (which uses the vsync on the graphics card), and bam! silky smooth 60fps, with quick responses and no lag. I haven't tested it, but I imagine this new "fast" version of gnome will still feel slower and choppier than cinnamon when vsync is handled on the gpu. (I just wish it was the default setting.)

That and I'm not a fan of the side bar in Ubuntu. Most people agree, Mint's desktop UI and UX is just more pleasant.

</comment></comment><comment><USER15> I don't normally try non-LTS, I decided to give 19.10 a go, and I noticed the performance boost. What was a real pleasant surprise was that my NVidia GPU drivers were automatically installed, something I've always had to do myself before.

<comment><USER16> Glad to hear it. 18.04's NVidia on Wayland situation is so broken it's ridiculous. It's a major headache for any kind of machine learning workstation.

<comment><USER17> NVIDIA on Wayland is still ridiculously broken and NVIDIA has zero desire to fix it. The "solution" is to continue using Xorg.

<comment><USER18> Unless you need CUDA, the solution is to use AMD.

</comment></comment></comment></comment><comment><USER7> Currently running this on my XPS 13. Works beautifully and honestly battery life is insane. _Much_ better even than just 19.04. It's easily on par with my 2015 MBP. I'm getting almost a week out of a charge using it for a few hours a day running just Chrome, PyCharm, and Signal most of the time.

<comment><USER19> I think I am doing something seriously wrong then. My 2015 XPS 13 only lasts a couple of hours.

<comment><USER20> 4 years is long enough for lithium batteries to decline pretty noticeably.

<comment><USER7> Sorry, the MBP was my old work computer. I don't have it anymore. I was referring to its battery performance at the time.

</comment></comment></comment></comment><comment><USER21> Whenever I see claims like this "Old Hardware" seems to invariably mean "5 years old" and "fast" means "not as perceptibly slow as the bloated crap, but noticeably not as fast as actually fast things".

<comment><USER22> I'd give it a try yourself. It's really fast.

</comment></comment><comment><USER23> I've been running it for a while and it's great except it still takes a ridiculous amout of time the time to launch gnome calculator. Ever since they changed it to a 'snap' it's been awful.

Edit : It looks to be pretty easy to swap the snaps for regular debs: [https://blobfolio.com/2019/10/psa-upgrading-to-ubuntu- eoan-m...](https://blobfolio.com/2019/10/psa-upgrading-to-ubuntu-eoan-might- convert-some-local-apps-to-snap/)

</comment><comment><USER24> While Gnome 3.34 is great, there is still some basic (for me) functionality missing (talking about Wayland, because I can't stand screen tearing on Xorg):

\- no option to set mouse scroll speed

\- hidpi support

\- no tray icons (and extensions doesn't work great...)

\- notifications system

\- gestures/trackpad

I'm planning to move to MacBook to see how it is, in case I need more power, I will just use my workstation/gaming PC (with Nvidia, right now I'm using AMD with the only reason being Wayland support, because hardware is much worse).

</comment><comment><USER25> Based on my experience from using Ubuntu, it's perhaps more accurate to say that the Ubuntu 19.04 (and earlier) compositing window manager is slow enough to make new hardware feel old. It's extremely noticeable.

The blogpost they link to says as much:

> Gnome Shell 3.32 in Ubuntu 19.04 feels slower than Unity and other desktops. > If you have a slow machine then it won't run smoothly. Perhaps more > surprisingly even if you have a fast machine then it still would not run > completely smoothly.

([https://discourse.ubuntu.com/t/boosting-the-real-time- perfor...](https://discourse.ubuntu.com/t/boosting-the-real-time-performance- of-gnome-shell-3-34-in-ubuntu-19-10/13095))

<comment><USER26> Well, before 3.30 GNOME wasn't even remotely _usable_ , so I would still call that progress. Make it work, _then_ make it fast.

</comment></comment><comment><USER27> Wow, I've never had so much of the hard work myself, colleagues, and individual contributors have done misattributed to one company.

For those interested in what actually was improved, and some attribution for those involved, see [https://blogs.gnome.org/shell-dev/2019/11/22/a-review-of- gno...](https://blogs.gnome.org/shell-dev/2019/11/22/a-review-of-gnome-shell- mutter-3-34/)

<comment><USER28> Come on, Christian. Daniel van Vugt at Canonical did a huge number of performance fixes on mutter and gnome-shell this release. Some of those I had to personally vouch for because your colleagues wouldn't land them otherwise.

<comment><USER29> Can you elaborate on that a little? I've always gotten the impression that gnome is actively hostile to new contributors and anyone who isn't part of the "inner circle", so I'd be interested in hearing more about what that actually looks like, from someone who isn't very anti-gnome.

<comment><USER28> As someone who went through the pipeline, GNOME is not actively hostile to new contributors, and is actually really friendly, as long as you agree with the overall design and direction of the desktop. In one of the cases, I'd say that what happened was that there were two competing methods for how to fix a performance issue, and some of the maintainers involved were not graphics experts, so I was able to guide the team to determine which version to land, which happened to be the version by Canonical.

Canonical's version: [https://gitlab.gnome.org/GNOME/mutter/merge_requests/189](https://gitlab.gnome.org/GNOME/mutter/merge_requests/189)

Endless's version: [https://gitlab.gnome.org/GNOME/mutter/merge_requests/402](https://gitlab.gnome.org/GNOME/mutter/merge_requests/402)

This is much more well-understood now that open-source is popular, but it wasn't understood very often back in the day, which is that maintainers have the power to say "no", and do so frequently. If you are a new contributor and you try to change the design, you won't have a great experience.

<comment><USER29> I mean sure, but a lot of open source software is driven by the idea that developers "scratch their own itch".

It's a bit of a pain that so many patchsets need to be maintained separately for common "scratch my own itch" type features in GTK, instead of the feature just getting buried in a gconf setting somewhere. There are an increasing number of little things in GTK that require you to do things the "gnome way" or maintain separate patch-sets. Any attempt to figure out how we can satisfy both parties gets disregarded pretty quickly.

Obvious examples include thumbnails in the file-picker or type-ahead instead of relying on gnome's tracker or slow file-search.

I guess there's some subtle difference between "hostility" and "refusal to co- operate with other stakeholders" that I'm not getting.

<comment><USER28> Thumbnailing is one of those things that's difficult because it's a desktop service. Windows / OS X has one desktop, so there can be a tighter integration between the toolkit and the desktop. My recollection is that thumbnails in GTK+ work fine assuming you have the GNOME thumbnailer running, but lots of people use GTK+ outside of those environments, and it doesn't work there.

This is usually when someone says the word "standards" and expects everyone to go "oh, we haven't thought of that!", but standards take a lot of work, and if there are disagreements in the design and direction of the system, no standard is going to be made. My recollection is that thumbnailing was just one of those things.

<comment><USER29> Sorry, I think I misspoke. Thumbnails do generate properly and show up in GTK file picker, without needing any kind of hard dependency on gnome or any desktop services. That's not actually a hard problem, it was standardized way back.

The actual problem is this 15 year old bug: [https://gitlab.gnome.org/GNOME/gtk/issues/233](https://gitlab.gnome.org/GNOME/gtk/issues/233)

This right here has collected most of the patches I'd consider to be important, and the maintenance requirements do not _appear_ to be significant. Of course it is work that doesn't directly benefit the gnome DE.

[https://github.com/krumelmonster/gtk3-mushrooms](https://github.com/krumelmonster/gtk3-mushrooms)

But it's important to note that GTK is not gnome. It's really important to recognize that as gnome fights harder to become it's own platform, that other platforms do make use of GTK. The gnome-foundation being completely inflexible on the needs of other projects means that at the end of the day it is really hard for other people to collaborate on GTK.

Gnome can do whatever it wants, but the idea that GTK is only supposed to serve the gnome project does seem like a bit of a betrayal. That seems to be a pretty common thought: [https://en.wikipedia.org/wiki/GTK#Criticism](https://en.wikipedia.org/wiki/GTK#Criticism)

</comment></comment></comment></comment></comment></comment></comment><comment><USER30> I tried upgrading a fresh installation of 19.04 to 19.10 and it permanently broke itself during the upgrade process. Twice. So it seems like there's some issue with the upgrade process at the moment.

</comment><comment><USER31> A little off topic but mentioned in the article, does anyone know why macOS does feel slower than Windows and Linux on the same hardware? Does it have to do with the 2x scaling Apple is doing all the time?

<comment><USER14> I don't know what hardware you're using or what "slower" exactly means here, but OSX has always been silky smooth at 60fps for me, focusing on that nice feel, more than an instant responsiveness. To be fair, MacOS is still highly responsive, but it's polished, causing an intentional minor frame delay, which increases smoothness.

Windows, on the other hand, is responsive, but has incremental stutters from time to time. It happens in video playback, to the point when I'm out and about and at a mall a tv with a video is playing, I can tell if it is on Windows or Linux. It's more obvious in video games which has these microstutters all over the place, starting with DX9.

Linux has page tearing, and depending on the desktop can be highly responsive or not at all. Many linux desktops can be manually configured to turn on gpu acceleration on the desktop so the gpu handles the vsync and everything else making it butter smooth like OSX with more responsiveness, but you have a trade off at that point between how much responsiveness vs how much polish you want.

(When you drag your mouse across the screen, it skips pixels. When you're dragging a window, does the compositor follow the mouse flawlessly, skipping around, or does it smooth behind the mouse a frame or two making it look nicer?)

Maybe you're running a 4k60 external monitor out of a 2015 MBP. In that example, you'd be maxing out your GPU on the desktop, potentially causing a delay when dragging windows around that may or may not be noticeable. On this hardware setup, Linux and Windows are both lighter weight and will be more responsive. ie, ymmv depending on your setup.

<comment><USER31> I tested this with my iMac 5k with 32GB of memory and Radeon m295x (4gb of vram). The apps start slower than what I noramlly see on Windows but I don't have a lot of the same apps, so it's hard to compare. I agree that it has to do with responsiveness but I don't know, sometime I prefer the responsiveness. I do see stutter in Safari once in a while, it is not always 60fps for me.

The screen is set to default for display.

My previous 2012 rMBP was the same difference between Windows 10 (Bootcamp) and macOS.

iOS on my iPad Pro (first gen) is very responsive and smooth at the same time. I wish I could get the same experience on my Macs.

</comment></comment></comment><comment><USER32> GNOME 3.34, which the article attributes most of the speed increase to, has been available on Fedora for a while already.

But I wouldn't think that running GNOME on "old hardware" is a good idea anyway. Except for very generous interpretations of "old".

<comment><USER33> Yeah, I've been running the latest Gnome on Arch for awhile and it's really stellar performance upgrade. It's worth noting that later releases of Gnome are now targeting less resource usage now that they've worked out most of the critical performance issues for high end hardware.

<comment><USER34> Same here. Been running it on Debian and it looks and works great. I'm relatively new to Linux (moved from macOS last year) so don't know much about the past, but I really don't get all the negative comments about Gnome the way it is today. It's a perfect way for a macOS user to get introduced to Linux too.

<comment><USER32> Most of the negativity is on ideological grounds - they don't like that it is "opinionated" and offers minimal customisation compared to KDE. And there's some backlash against the fact that a lot of the UI uses Javascript. (Though recent benchmarks have shown that it doesn't cause any kind of performance hit.)

Frankly, I don't think that Gnome is particularly un-customizable, it just does it in a different way, using extensions that you can install. I do occasionally try out new versions of KDE to see what I'm missing, and I'm always surprised how buggy/crashy it is.

</comment></comment></comment></comment><comment><USER35> I upgraded my Intel Compute Stick to 19.10 Lubuntu last month from 16.04LTS. It's running great and I didn't have to mess with any of Ian Morrison's wonderful tools he made for running Linux on the Intel Atom platforms. Not saving my resolution settings across reboots is a slight issue, but that is something I can fix. After reading the article maybe I should try Gnome again.

<comment><USER36> Does that mean you didn't need to use a boot drive with a 32 bit boot loader?

I've been having trouble getting any of my sticks (one an ASUS Vivostick, the other a no-name) to even boot a Linux installer off USB or CD, but that could be because I've been trying to get Ubuntu Server, which may be part of the problem. If the Ubuntu desktop path is better, maybe I'll try that.

<comment><USER26> Fedora supports 32-bit boot out of the box, even on actual 64-bit OS's. The Debian multiarch image does as well, but the 32bit-boot/64-bit OS arrangement fails if secure boot is on, so make sure to turn that off beforehand in the firmware UI.

AIUI, Ubuntu doesn't support this.

<comment><USER36> After seeing @undersuit's comment, I decided to try the Ubuntu Server 19.10 ISO and it booted fine off a USB drive.

Good to know that no tricks are required (such as these: [http://www.linuxium.com.au/home](http://www.linuxium.com.au/home)). I had been banging my head against the wall trying to get older versions of Ubuntu Server installed on my stick PC.

<comment><USER35> Good to hear!

</comment></comment></comment></comment></comment><comment><USER37> Am I the only one who doesn't notice a huge difference between 19.04 and 19.10? I wouldn't have noticed if these articles didn't explicitly mention the performance improvements to be honest.

<comment><USER38> Using Gnome3? If not, it doesn't sound like the fixed performance regressions would apply.

</comment></comment><comment><USER38> Reading between the lines I gather this release fixed a performance regression in a previous Gnome3 release. So it is not a clear performance boost in the absolute sense.

I suppose those of us on Ubuntu Mate, KDE, XFCE, etc can disregard that aspect of the article. Unless there are some perf improvements in the kernel or X as well?

</comment><comment><USER39> The review doesn't touch my longest-running gripe with Ubuntu... does it still run tracker/nautilus/baloo in the background, sporadically choking on a developer's filesystem and hanging the entire UI? Because I'm really not impressed by a clean install being snappy.

<comment><USER40> Tracker does run in the background. I disabled some of the things it indexes. I don't experience any slowdowns.

Nautilus is there, it's the file manager. I installed Nemo because I want to keep using type ahead in the local folder. I still get Nautilus in the file dialogs and tracker in the full screen modal opened by the super key (is it the dash?)

Baloo, no idea what it is.

Overall 19.10 it's faster than 16.04 on the same laptop. It's 3 years of work and a clean install (dual booting for now, two different SSDs.)

<comment><USER41> Baloo appears to be the KDE equivalent of GNOME's tracker. Not sure why both would be running on the same system.

<comment><USER42> It wasn't implied that they were on the same system. The grandparent was either implying that they don't remember which is which, or that which one is running depends on which Ubuntu flavour (gnome or KDE) you're running.

<comment><USER39> Indeed, I've used multiple flavors over the years, and these have been culprits on various occasions. I'm not an expert on OS's so the uncertainty I conveyed was deliberate.

</comment></comment></comment></comment></comment><comment><USER43> I had to downgrade. There is a regression and you can't run it off a USB stick anymore.

</comment><comment><USER44> Or maybe (not so) current hardware is insanely fast, and windows really slow: Just to see if it could work I tried to install linux on a Chuwi Hi8 (windows/android atom based chiinesium tablet. Not enough space for windows now and really ancient android) Ubuntu was so slow, by the time I finally managed to begin copying files to the disk, the tablet ran out of battery. (And the thing eat more power than its charger can provide)

(as for the result of the experiment: it can work, but the amount of work needed was way beyond what I was ready to invest. And not exactly stable)

</comment><comment><USER45> The Raspberry Pi 4 info is out of date AFAICT. The initial release had non- working USB on my 4 GiB RPi4, but fixes came a few(?) weeks ago that apt update (over ssh) picked up. As best I can tell, it works flawlessly now. Mindbogglingly, I can comfortably do serious development work on $55 computer (albeit overclocking recommended).

At the other end of the spectrum, it's also running on my 5K iMac and that it just worked.

On the Dell XPS 15, the audio didn't work out of the box, but there's a common workaround for that.

</comment><comment><USER46> Imagine if that Unity experiment didn't suck up five years of time.

<comment><USER47> Imagine that they learned a few things working on Unity and they applied them to improve Gnome Shell.

</comment></comment><comment><USER48> Maybe it's time I give Ubuntu another shot. Unity was too bloated for my tastes and Gnome has always seemed kinda subpar. Lets see if the gnome changes can change my mind.

Does Ubuntu still have advertisements for Amazon search and whatever other crap they were pushing?

<comment><USER49> Unity2D was super fast. So sad that they didn't continue to develop it, and instead went for GNOME. If Ubuntu GNOME is now quick, good – it took a long while to get there. It's important for Linux and FOSS to have well known and widely used software. Without Ubuntu, we would probably have no Steam Linux client.

<comment><USER26> > So sad that they didn't continue to develop it

You can run MATE with the Mutiny desktop preset, if performance is an issue. It's essentially the same workflow as on the old Unity 2D, and it isn't even Ubuntu specific.

> Without Ubuntu, we would probably have no Steam Linux client.

SteamOS is actually based on Debian, and Steam is even in the Debian non-free repos.

<comment><USER50> > You can run MATE with the Mutiny desktop preset, if performance is an issue.

No. In comparison it is unstable in ways that make Gnome look like NASA software.

It often forgets it is Mutiny and no other of the presets.

Quality wise it is still way below Unity.

</comment></comment></comment></comment><comment><USER51> so i just installed it, and like, right away i notice every app always opens in the top left of my screen and their positions are not remembered?? :(

speed seems irrelevant if windows cant be arranged lol. google is coming up empty. is this a bug?

<comment><USER51> update for posterity: windows seem to open in some kind of a logical fashion, so they use all the available real estate of the screen dynamically. its pretty cool! generally i prefer it cuz i never have to move new windows out of the way. but it would be nice to have a setting to remember some windows in a fixed position. as is the case w gnome lately many such settings are hidden/removed for the sake of simplicity. cheerz!

</comment></comment><comment><USER52> Does wayland+gnome work with propietary nvidia drivers yet?

I've been using unity since 18.10 because gnome always blackscreens on my 1070. (I'd been using it since it became the default, for the sake of support. But preferred unity)

</comment><comment><USER53> It's fast and also magical, transmuting my laptop into an effective heater.

</comment><comment><USER54> If you like Ubuntu and Gnome services such as the account manager, install Ubuntu Budgie, it will run much faster. Try getting the future LTS release, it will be safer.

</comment><comment><USER55> I noticed absolutely no difference after upgrading from 19.04. Do I need to do anything?

<comment><USER14> Is your monitor running at 30fps? I imagine in that situation the changes wouldn't be noticeable.

Also, upgrading from 19.04 to 19.10 isn't super stable, so it could be it didn't upgrade properly.

<comment><USER55> It's a Dell latitude laptop, screen is at 60Hz

</comment></comment></comment><comment><USER56> TL;DR

Ubuntu 19.10: Gnome 3.34 is fast, like "make old hardware feel new" fast.

</comment><comment><USER57> Be honest, Ubuntu feels fast 'cause people compare it with bloatware/telemetry overflowed Windows 10.

If you wanna a really FAST distribution - try Clear Linux from Intel.</comment>


Ask HN: What advise would you give your T-10 year old self? - <AUTHOR> <comment><USER2> The richest man in the world is still bald.

</comment><comment><USER3> buy all the bitcoin you can scrounge up the cash for.

</comment><comment><USER4> Get therapy!

</comment><comment><USER5> lose weight.</comment>


Show HN: Getparked.io – Domain Bsckordering - <AUTHOR> Hi all! I had a little bit of time on my hands so I decided to put together https://www.GetParked.io, a domain backordering service. It allows you to search for a domain name, and backorder it if it's currently registered by someone else. GetParked will then try to acquire the domain for you when it expires. It costs nothing, except for if it actually is able to secure the domain for you, and is of course free to simply check the status and registration info of any domain name. I built it because as someone who enjoys building websites, I've always had trouble getting the domain names that I want. So I thought it would be fun to build something that will check availability for me without having to sit and hit refresh on a registrar. Please let me know what you think! <comment><USER2> Interesting, It's a confusing name though as [https://park.io](https://park.io) does the exact same thing, but has a bidding market place too

<comment><AUTHOR> Thanks for the feedback! Yup, I'm familiar with park.io. I haven't added auction functionality yet, but that's definitely in the backlog.</comment></comment>


CRAM: Efficient Hardware-Based Memory Compression for Bandwidth Enhancement - <AUTHOR> https://arxiv.org/abs/1807.07685 <comment><USER2> I wonder how it would fare compared to "software-only" solution, that is re- architecturing your algorithm to work with compressed data directly.

For example, consider an analytical engine going over values in a column of a database to sum it together. To increase the bandwidth, you can consider compressing the values (for example, with some form of Huffman encoding), and you have two options:

1\. Decompress the values before passing them onto calculation, then do the calculation. This is what the paper proposes (transparently in HW as the data are being used in the cache).

2\. Do the "addition" directly on the compressed values. This requires more logic. (But it doesn't have to be just implemented in software, it could be hardware assisted by some FPGA or something like that.)

So I wonder, instead of compressing the cache, wouldn't it be better to implement the 2nd solution?

The disadvantage of the 1st solution is that since it has to work in the general case, the compression is more likely to be bad in special cases, where the 2nd solution with a specialized compression/processing combo could be superior.

<comment><USER3> How does one work with compressed data directly? If you are dealing with the data in blocks, then your compression is only as good as the repetition in each block (perhaps the dictionary can be shared).

But even in this case, you don't deal with compressed data directly, but decompress only 1 block at a time.

<comment><USER2> No, the quality of the compression depends on what is the probability distribution of your data blocks. If some blocks are much more likely to occur, then you can compress them better (put them in a smaller number of bits).

What I mean by "directly" is instead of having to do

y = y + c^-1 (x_i)

where c(x) is the compression function (and c^-1 is its inverse, decompression), we could define a function f such that

z = f(z, x_i)

and final y = c^-1(z). This f would entail both the "+" (operation to be done on the data) and c^-1 (the decompression).

In general, it's non-trivial to come up with f, but I do believe that it would be in many cases a better approach than to use the first method.

Floating-point numbers can be seen as a simple case of this. If we represent numbers in fixed decimal point, then we have a lot of 0s in the numbers (and also lot of digits of very low significance) that we mostly don't need to have there. Therefore, in practice, we use compression - floating-point representation. The operations on FP are more complicated (you have to align the mantissas before you do arithmetic operations), but it's a good trade-off.

<comment><USER4> In most, if not all compression schemes, your "compression function" depends on all x_j with j<i. So it's really a new function for every i.

</comment></comment></comment></comment><comment><USER5> So they set aside 2 32-bit patterns; when these patterns appear at the end of a 64-byte cache line, they indicate a compressed line; otherwise, the line is literal.

This smells an awful lot like an impossible infinite compression claim; any real compressor must increase the length of at least some inputs due to the pigeonhole principle.

Their solution is to add a 16-entry table to track "lines that look like they have the compressed-line marker, but are not actually compressed" (and they also store such lines inverted in DRAM, but this seems inessential). They claim 16 entries is enough based on probabilistic arguments, but they should be thinking of storing data generated by an adversarial process; one such process would be to choose 60 incompressible bytes followed by the 4-byte compressed line indicator.

Including just one bit of metadata per cache line of memory is 32MiB, more than the 0.75 bits/line (24MiB) calculated on page 5 for the scheme they would like to replace.

Their next bit of handwavium is that they use "DES" to choose line markers in a way dependent on a secret key which can be changed in case of a LUT overflow. I wasn't immediately able to find a modern figure on how fast hardware "DES" is, but a 2003 paper gives a figure of "21 to 37 cycles of latency" [https://www.intopix.com/Ressources/WPs_and_Sc_Pub/intoPIX%20...](https://www.intopix.com/Ressources/WPs_and_Sc_Pub/intoPIX%20-%20efficient_uses_of_fpgas_for_implementati1520415691870.pdf) \-- will this fit into the time budget or will it slow memory accesses? (They first say this is off the critical path but then that the marker is per-line, I don't see how both are true)

When the limited table is exhausted, a new key is chosen and all of memory must be updated (so you really don't want to do that) They dismiss this by again appealing to the rarity as "1-in-2^512" but if that's true you might as well just design the chip to blow up when the condition arises. If any timing information about "line appears in LUT" is visible to a program, then it is just 16 * 2^32 instead for an adversarial program (you can tweak line 1 until it needs a LUT entry, which takes just 2^32 writes; then repeat for the next line)

<comment><USER6> > This smells an awful lot like an impossible infinite compression claim; any > real compressor must increase the length of at least some inputs due to the > pigeonhole principle.

Why even include this sentence when the rest of the post is about the extra state it uses to track collisions?

> Including just one bit of metadata per cache line of memory is 32MiB, more > than the 0.75 bits/line (24MiB) calculated on page 5 for the scheme they > would like to replace.

Sure? The point of this scheme is to not store _any_ metadata per line of main memory.

> Their next bit of handwavium is that they use "DES" to choose line markers > in a way dependent on a secret key which can be changed in case of a LUT > overflow. I wasn't immediately able to find a modern figure on how fast > hardware "DES" is, but a 2003 paper gives a figure of "21 to 37 cycles of > latency" > [https://www.intopix.com/Ressources/WPs_and_Sc_Pub/intoPIX%20...](https://www.intopix.com/Ressources/WPs_and_Sc_Pub/intoPIX%20..). > -- will this fit into the time budget or will it slow memory accesses? (They > first say this is off the critical path but then that the marker is per- > line, I don't see how both are true)

21 to 37 cycles is definitely lower latency than accessing memory. And you can easily pipeline it to match the rate of memory accesses.

> When the limited table is exhausted, a new key is chosen and all of memory > must be updated (so you really don't want to do that) They dismiss this by > again appealing to the rarity as "1-in-2^512" but if that's true you might > as well just design the chip to blow up when the condition arises. If any > timing information about "line appears in LUT" is visible to a program, then > it is just 16 * 2^32 instead for an adversarial program (you can tweak line > 1 until it needs a LUT entry, which takes just 2^32 writes; then repeat for > the next line)

So an adversarial program can waste CPU? It can already do that.

But more importantly, the LUT only needs to be involved when checking the marker on a memory access. It should be very possible to make it fixed-timing.

</comment></comment><comment><USER7> Sadly you have to remember that transparent compression is both defeated by transparent encryption (as more and more data is encrypted) and is also susceptible to timing attacks when that is not the case. I really wish this weren't the case, since I really think compression is awesome, but this is something you have to keep in mind whenever you introduce compression.

<comment><USER8> There are a couple of things that feel concerning about this from a security perspective, as it's optimizations like this that could create another spectre class vulnerability. Mixing cache lines arbitrarily seems bad when you think of virtualization and the dream of memory isolation, you would need a lot more holding data to maintain such an optimization. It is also hard not to be a crypto snob when the author writes 'secure hash' and indicates single DES as a remedy.

I also don't like that there is a claimed speedup figure but it is not clear in a cursory reading and looking at the available tables and graphs what the baseline is? It's all done in emulation (apparently) so it's hard to know if it is amortized because there is no actual time to access or something else, but that could just be my unfamiliarity with that emulator.

</comment></comment><comment><USER9> BLOSC already does this with software.

[http://blosc.org/](http://blosc.org/)

The sales pitch is "faster than memcpy." Fetch data from RAM, decompress it in L1,2,3 caches and run operations on the data. Some very nice benchmarks available.

<comment><USER10> I find it incredible how _slow_ modern RAM and databuses are. A far cry from the early days where you could upgrade the CPU cache by inserting some memory chips on the other side of the board

<comment><USER11> ram has gotten steadily faster since then. but cpus have gotten faster, faster.

<comment><USER12> RAM access time has not gotten much faster, only bandwidth increases.

</comment></comment></comment></comment><comment><USER13> For GPUs, memory compression isn't theoretical: they use it to compress frame, depth and stencil buffers to save bandwidth. Overall performance (i.e. frame rate) improvement is typically 10-20% AFAIK.

<comment><USER14> Yes it also only works well with GPUs. GPU-Ram communication is all about bandwidth.

General purpose CPUs are all about latency. Only very specialized problems would benefit from memory compression. On the other Hand compression will increase latency which is counterproductive to any traditional CPU tasks.

<comment><USER15> Even though the trend is toward GPUs, we'll likely be running large amounts of throughput workloads on CPUs for another decade. A 5% reduction in power consumption is worth exploring.

Is it bad that I'm looking forward to the end of Moore's law (a little)?

<comment><USER13> With my specialty in native code, I am already benefiting from the end of Moore's law. Now, a factor 2 in performance actually matters. I also expect to see more specialized hardware and software to, say, feed 8K screens, process many small independent transactions (high core count servers), etc. There is going to be a lot of interesting performance work :)

</comment></comment></comment></comment><comment><USER16> I wonder if this very clever approach is interesting in the energy consumption perspective. In this work ([http://hpc.pnl.gov/modsim/2014/Presentations/Kestor.pdf](http://hpc.pnl.gov/modsim/2014/Presentations/Kestor.pdf) ) the author shows (slide 11 ) that it cost 100 times more energy to move data from memory to register than to move data from L1 cache. So, this kind of mechanism could hugely improve the amount of data that could be stored in L1/L2/L3 cache, but what could be the energy cost of compression/decompression step ?

</comment><comment><USER17> If you're on a phone, here's an HTML version of the paper: [https://www.arxiv- vanity.com/papers/1807.07685/](https://www.arxiv- vanity.com/papers/1807.07685/)

</comment><comment><USER18> The "CRAM" name is already used by a NGS file format (for compressed sequences storage): [http://www.internationalgenome.org/faq/what-are-cram- files/](http://www.internationalgenome.org/faq/what-are-cram-files/) .

<comment><USER19> Is that supposed to be a dealbreaker? I imagine 4 letter acronyms, especially ones that are also words, are in short supply these days.

</comment></comment><comment><USER20> GPU vendors have been doing this with texture maps for years.

<comment><USER21> Texture map compression is lossy, and it's a very compute intensive process that is done at the game mastering time, and it's a fix ratio compression.

None of this is the case here.

You're probably thinking about delta color compression.

</comment></comment><comment><USER22> Their assumption of 64 bytes cacheline is unrealistic. Most CPU architecture use 64 _bit_ cacheline (1/8 of 64 bytes), for good reasons. 512 bit cacheline is insane.

Also, good compression requires large corpus / large model - thus page-based compression is much more useful in real life. For example Google's ChromeOS uses zram by default since 2013 [1].

[1] [https://en.wikipedia.org/wiki/Zram](https://en.wikipedia.org/wiki/Zram)

<comment><USER23> > Most CPU architecture use 64 bit cacheline (1/8 of 64 bytes), for good > reasons. 512 bit cacheline is insane.

No. 64 and 128 (to a lesser extent) bytes are pretty much the only extant cache line sizes these days.

64 bit cache lines would be insane.</comment></comment>


Ask HN: Chat Integration Software - <AUTHOR>

We don't want to stray too far away from perfecting our core competency so we're evaluating the integration of an outside chat software to compliment our existing product.<p>Can anybody point me to any other quality options outside of meebo? Thanks in advance! <comment><USER2> What do you mean by "chat integration software?" Are you trying to put chat in a website? Do you want a program that combines multiple chat services?</comment>


Firefox 12 now comes with good looking customize toolbar window - <AUTHOR> http://www.browsomatic.com/2012/01/firefox-12-now-sports-with-good-looking.html

<comment><USER2> v12?!. They really have gone crazy about versioning. Back when it was v3 - they used 3.6.24, 3.6.25 etc for minor updates. And then going up a version when major things changed.

<https://wiki.mozilla.org/Releases> \- 6 week release cycle incrementing the version number each time?! Firefox 12 beta is probably available soon too...

Apparently FF7 "uses as much as 50% less memory than Firefox 4" - Nope it doesn't, still the same ever increasing bloated browser.

<comment><USER3> In the bad old days of Firefox, I used to see it taking 500MB, 800MB, sometimes even a gigabyte of RAM.

It's gotten a lot better since then. I'm running Firefox 11 (on the Aurora channel) right now, and with a fairly typical tab load, about:memory tells me Firefox is using less than 300MB. I'm impressed.

<comment><AUTHOR> me too! and I am quite surprised to see his comment. Infact some blog posts came explaining why Chrome takes more memory than Firefox</comment></comment></comment>


Your app makes me fat - <AUTHOR> http://seriouspony.com/blog/2013/7/24/your-app-makes-me-fat <comment><USER2> [https://hn.algolia.com/?q=your+app+makes+me+fat#!/story/fore...](https://hn.algolia.com/?q=your+app+makes+me+fat#!/story/forever/0/your%20app%20makes%20me%20fat)

The cutoff for reposts of stories that have had significant attention on HN is about a year.</comment>


Ask HN: What is today's most important problem/opportunity? - <AUTHOR> I am unable to settle on a problem to solve. I keep realizing that the problem I'm working is just a subset of a larger one, and my focus shifts on that new one. Repeat.
For example, one starts by noticing that the school's schedule management system could be improved. Then you realize that the calendar hasn't changed in ages and that you could reinvent it. Then you generalize the problem and realize that you could improve how anything time-related is managed. Then you realize that time is just one particular axis/condition to what is a much larger logistics problem. Then you start re-designing how task management is conducted (with dependencies, conditions, deadlines, etc). Then you realize that you're not just dealing with tasks, you're dealing with contracts. Then you realize that contracts are the basis of a semantic marketplace for intents and promises. Then you realize that you need to create a new communication framework or language to facilitate the communication of unambiguous stateless descriptions and intents. Then you realize that it's pretty much a programming language you need to implement, but that's not accessible enough. You assume that this can be fixed by doing it right (unlike all other languages before this one) and that it will be easy for anyone to use daily. Then you realize the opportunity to leverage AI and autocompletion-like mechanics. Then you realize that text is an inferior medium, and that we have dynamic interfaces that allows us to do MUCH better. I'm now studying linguistics in order to properly understand how meaning works in order to design the perfect computer-assisted language. I honestly believe that it is achievable and that everyone will soon be using it. This will replace 80% of all apps and websites. It will elevate human consciousness to levels we can't even imagine. At least, that's my belief.
With that in mind, where do I stop? What's the most important thing I could dedicate my life to? <comment><USER2> That was the best thing I've read all day.

When I was younger and had time to spend on anything I wanted to, I did. I wanted to make a game. So I started making an OpenGL game, but I realized that any game worth a damn would support DirectX as well. So I set about abstracting out the functionality of the two graphics libraries into a standard interface. I didn't finish a game. I realized I didn't want to create a game. I could have used any game engine to do so, if I were sufficiently motivated. But, I did learn so much in the journey that I can't call the time wasted, simply spent.

So, I guess it comes down to whether you want to engage in thought experiments or create something. You can build castles in the sky all you want and there is a definite satisfaction in that. You can build a log cabin on the ground and live in it and see what it is like to live inside a cabin you built and maybe feel satisfaction that way. What does your heart tell you when you consider these paths?

Now that I don't have all the free time I once had, I am more focused on completion than conception, but every so often, I let my mind follow a path to a place that I will never go. But I am honest with myself about that.

<comment><AUTHOR> Implementing a vision is where it gets messy and compromises arise. I want to delay that as much as possible, in order for the vision to be as best as possible.

I can't be the only one that went through that process. It took me 12 years to get there. I have not written a single line of code for that project, only design. What I'm looking for is someone that went through that process and eventually reached the end. Someone that can tell me what matter, so that I can finally start getting my hands dirty.

As much as people here tend to claim that ideas are cheap and that only execution matters, clear visions or prescriptions of what the future should be are a very scarce resource.

</comment></comment><comment><USER3> I think the most important part is really just starting.

All in your brain is good but the real task is in the implementation, and it will be a lot more work than you think. Start where you feel you can make an easy in then build from there. More than likely will take you in directions you didn't expect, but you'll have a lot of fun along the way.

<comment><AUTHOR> I would regret if I had started "doing" at any point in the above thought process (which spanned over 12 years). I would have slowed down the evolution of the vision, and implemented something that's less than worthy of my dedication.

I have now reached the point where I believe that creating a new communication platform for human and AI seems like an important thing to implement, yet I'm not sure whether that will hold true when I discover the next better thing.</comment></comment>


Should Tesla receive government help? - <AUTHOR> http://dealbook.blogs.nytimes.com/2008/12/01/should-taxpayers-pay-to-back-tesla-motors/

<comment><USER2> A guy goes to his wife with a business idea. He has had them before, and his use of their savings has taken the couple's net worth to below zero when the debts that he has incurred are factored in. The thing is, he believes that this business will be different. This is the one that will turn things around for them. A bit skeptical, the wife hears the husband out. At length, the husband finishes his 'pitch'. Still skeptical, the wife tells him that while she would love to help him give this new business a try, they are out of money, and are obliged to start paying down debts at this point. Relieved at being able to blame their situation for the denial as opposed to her skepticism, the wife tries to move the conversation to a different topic. When the husband hits upon an idea.

They still have a sizable sum of money in the kid's college fund!!! Just enough to get him started!!!

\----

People we are broke. I would suggest everyone accommodate themselves to the new fiscal realities coagulating around us. To continue to spend in the hopes that some future generation will pick up the bill is morally unconscionable. It is also contrary to our self interests by the way. As it is highly likely that the most damaging consequences of our financial mismanagement will develop in a more immediate time frame. That is, it is likely that you only THINK your kids will be the ones saddled with the consequences. In our current situation, it is far more probable that you will.

If a business enterprise requires government assistance to make it economically feasible then alarm bells should be going off in our heads with regard to the economic efficiency of that enterprise.

</comment><comment><USER3> Any federal money or loans that goes to Tesla should be through some sort of grant project where other automakers and manufacturers can compete for the money.

The only way direct bailout money should go to them is if they have any IP worth salvaging. Personally, I don't think an all-electric car is "the future" of the passenger car. Fuel cells, PHEV, natural gas, etc. all seem like better options, so I'd rather see federal money invested into these technologies.

<comment><USER4> "Any federal money or loans that goes to Tesla should be through some sort of grant project where other automakers and manufacturers can compete for the money."

Some sort of grant project for clean energy? Some kind of grant project that has $25 billion to give away to _worthy_ projects?

 _Almost entirely lost in the subsequent discussion was the fact that Section 136 of EISA created a $25 billion fund known as the Advanced Technology Vehicle Manufacturing Incentive Program (ATVM). The ATVM specified that the Department of Energy (DOE) should provide loans, loan guarantees and grants to new and existing automakers and suppliers to encourage development and speed delivery of next-generation cars – vehicles that meet higher standards for fuel efficiency and stretch technology beyond the internal combustion engine. The program aimed to provide "grants and loans to eligible automobile makers and component suppliers for projects that re-equip, expand, and establish manufacturing facilities in the U.S. to produce light-duty vehicles and components that make meaningful improvements in fuel economy performance."_

[http://184.108.40.206/search?q=cache:iJGKVzIN1ikJ:www.courtho...](http://22.214.171.124/search?q=cache:iJGKVzIN1ikJ:www.courthousenews.com/2008/11/21/LeadRegNov2108GreenCars.doc+Advanced+Technology+Vehicle+Manufacturing+Incentive+Program&hl=en&ct=clnk&cd=1&gl=ca)

<comment><USER3> Right...

The debate is whether or not Tesla is more worthy of the money than another manufacturer. I'm not convinced they are.

</comment></comment></comment><comment><USER5> No.

No business should receive help from the government aside from a publically available grant. Bailouts violate the concept of a free market.

</comment><comment><USER6> No. Because they are not building consumer vehicles, but luxury cars.

<comment><USER7> The work they're doing on luxury cars could easily be applied to consumer vehicles.

<comment><USER4> You're right because they work on battery technology too. They _have_ to in order to make electric vehicles more appealing and they plan on selling their batteries to other automakers.</comment></comment></comment>


CopyCat: Protocol-agnostic implementation of the Raft consensus algorithm - <AUTHOR> http://github.com/kuujo/copycat

<comment><USER2> I am writing a program which has to durably store large number of small files. My current implementation stores these on the local disk as files in a particular folder structure. I am considering designing some approach to make this program run on a cluster of machines for high-availability, fault- tolerance & scalability reasons.

One approach I can think of is to run a distributed document database and use that for storage. I don't need most features of such database products in the foreseeable future, so I fear that they will add operational overhead for not much benefit.

Another approach I can think of is to run my processing nodes against a network file system, and rely on that to do the replication.

Yet another approach I am considering is to use something like CopyCat to implement the file replication in my application code. Is this a good use case for CopyCat?

<comment><USER3> CopyCat looks really good for HA and fault-tolerance. Not so sure about its scalability since all writes go through the single leader node. It's more appropriate for the use of maintaining metadata of a distributed system, rather than maintaining the data themselves.

For your requirement, can you use S3 or something similar?

<comment><USER2> Thanks! The software I am writing needs to be (easily) installed at the customer site. So, S3 won't work for me. Are there any open source S3 clones that are easy to setup?

<comment><USER4> If you mean that you need something that supports the S3 API, you could try S3Proxy[0]. It basically presents the S3 API over a variety of different storage backends.

[0] [https://github.com/andrewgaul/s3proxy](https://github.com/andrewgaul/s3proxy)

</comment></comment></comment></comment><comment><USER4> This looks fantastic! As a DB researcher, I've found in the past I need some type of consensus algorithm for a system I'm building, but I don't want to spend a lot of time implementing something myself. I can see this being very useful in academia for scenarios like this.</comment>


The Loyalty to AMD's GPU Product Among AMD CPU Buyers Is Decreasing - <AUTHOR> https://blog.parsec.tv/the-loyalty-to-amds-gpu-product-among-amd-cpu-buyers-is-decreasing-9647c70a6da4#.1te2mdi4i <comment><USER2> That's funny, I just built my first new home Linux desktop in maybe 7-8 years, and installed an AMD RX 480 GPU into it, because the increased openness in their devices is finally really allowing for a first class open source driver for Linux to be written for this generation of GPU. At this point it really seems to be just starting to pay off in the performance & stability departments. If they keep it up, there will be good reason (at least on Linux) to favor AMD video cards.

Here's to hoping the new AMD CPU does well too, it's good for the market that Intel has competition.

<comment><USER3> I'm getting ready to build a new Linux Desktop- and I am thinking about an AMD CPU, but a Nvidia GPU- from everything I have read, Nvidia's performance on Linux is far better than AMDs? Is that information out of date, or are you just forecasting future improvements by AMD? I do really appreciate the open source driver efforts by Nvidia, but I'm not sure I am willing to sacrifice that much performance at this point.

<comment><USER4> AMD was trying to push code into the linux kernel to speed up performance. However, it was recently rejected, publicly, and rather harshly.

With the push to the linux kernel AMD GPU's would be better than Nvidia... that being said, I wouldn't hold my breath.

<comment><USER5> AMD wasn't pushing code for better performance, they were pushing a janky abstraction layer for their devices to make porting their Windows driver easier, that basically did what NVidia does and skipped a lot of the existing DRM infrastructure in the kernel.

If you want to write a kernel driver that's effectively a Russian train toilet nobody will stop you, and you can feel free to maintain the engineering effort on it - but it's not going to be accepted into the mainline Kernel and you'll have to maintain it out of tree.

If AMD wants a shim to make it easier to port their Windows drivers over that's perfectly acceptable, but they need to work with the existing DRM infrastructure and the people that maintain it to get a kosher driver that everyone can be happy with.

</comment></comment></comment></comment><comment><USER6> I am still loyal to AMD CPU's, but I haven't been loyal to AMD GPU's since they were ATI. I always found their driver stack to be more cumbersome in the past, and I had a rash of bad cards that left a bad taste in my mouth. Not sure if it's changed since, but I still have faith in their CPU's. The way I see it, the benefit of supporting innovation that drives Intel to improve outweighs the benefit of having an arguably faster computer for arguably the same amount of money.

Without Chevrolet we'd be driving Model-T's. Without AMD we'd by running 8080's. Support the little guy, even though they've historically not always made the right choices.

</comment><comment><USER7> I was loyal to AMD's CPU from 1999 - 2013. Price and not nerfing advanced features (overclocking, VM extentions) kept me coming back. However, there aren't many recent server/workstation options. So I begrudgingly switched over to Intel Xeon chips.

I bought an AMD RX 480 this year. I hadn't bought a discrete GPU in decade. I looked at Nvidia, but saw you needed Quadro/Grid to use with VT-d. AMD's GPU work with VT-d out of the box.

It'd be a damn shame to lose a company like AMD that doesn't disable features for marketing reasons. I'll happily buy another AMD CPU if the zen line comes close to the hype.

</comment><comment><USER8> AMD's CPU offerings have been lackluster in recent years, Intel has taken the cake in that department. I'm not surprised people are building Intel more and more. Ryzen is going to change all that, though. AMD is squaring that up to be the next big competitor. On the GPU front, the 8GB RX480 is making quite a dent in the market. It's beating out NVidia's 6GB GTX1060 in DX12 games, creeping towards the 1070's benchmarks, while costing less than $250. They aren't catering to the high-end market that the GTX1080 latches onto, but the price for performance is hard to beat. I surmise Ryzen to play out the same way.

<comment><USER9> I recently bought a new 380X...

Being honest, AMD disappointed me, it was my first AMD GPU, that I purchased after all the pain of nVidia Optimus, and... AMD managed to outdo nVidia in how shitty they are.

* All cards and models have terrible power usage and heat.

* Drivers, both on Windows and Linux still aren't decently stable, on my Windows I had to switch drivers SEVERAL times depending on what game I wanted to play, because with each game it had a different serious bug.

* AMD software (not just drivers) crash a lot on my machine. No other software behave like that.

* AMD tried to "pretend" they don't have too much power usage, on the 380X case they just put a tiny TDP limit for a beastly GPU, so it all the time keeps throttling due to power limits (even when increasing them... I am looking now for information on how to edit the TDP beyond the card defaults), the 380X TDP is the same as the 380, despite it having double the RAM and having more transistors to power.

* AMD RX480 "tdp cheat" instead was to add only a single cable, pretend the card didn't had too much power usage, and let it melt people's PCI-e slots by pulling 7.7A from slots rated to 5.5A

* AMD distribution network is terrible, they make zero effort to sell around the world, meanwhile nVidia and Intel trounces them, not only in marketing, but by reaching local companies to do distribution deals, for example I paid for my 380X the same price as a GeForce 970... (this was one month before RX480 launch). When nVidia launched the 1080 they called the local media, and told them what price the retailers were supposed to sell the 1080 (even if they ran out of stock), a price that was slightly cheaper than AMD Fury cards...

* AMD and partners support just suck, I asked the SIZE of my card, and their support instead told me to "RMA" it, I tried to explain I wanted information, not an RMA, and they refused to help... when I asked about the TDP, then thigns got worse, they got even more staunch that I should just return the card (and eat the shipping costs to US myself!)

* AMD official forums has employees spouting bullshit (like claiming in a huge thread of people complainign about the 380X, that it was the hardware on the complainers machines that was defective, withotu realizing that he just accidentally painted the whole 380X product line as shitty, since there is lots of people complaining of common issues, and if lots of people have the same issues, and is a hardware issue, then the hardware that is crap... also, on the forums making ANY negative comment about AMD, get you attacked, people claim you are nVidia shill or worse, I even got banned from AMD chat after I asked how to circunvent a driver bug that was preventing me from setting my CRT resolution correctly, because they didn't wanted me talking in public about negative things).

<comment><USER10> Interesting, I am an nvidia owner but have read many times that AMD's software and drivers are much better now than they used to be. If that bit about the 480 is true that's appalling. It doesn't really surprise me the brand's product forums are defensive about the brand, I'd wager the Nvidia forums users respond to negativity about nvidia the same way.

<comment><USER9> It is true and AMD "fixed" it with a driver hack.

[https://www.custompcreview.com/news/amd-aims-reduce- rx480-po...](https://www.custompcreview.com/news/amd-aims-reduce-rx480-power- consumption-upcoming-graphics-driver/30719/)

The driver do two things to "fix" the problem:

1\. Draw more power from the cable (it is what the card should have done in first place).

2\. They put back on the card the behaviour the 380X have, that I am trying to get rid of, of clamping down hard on the TDP and throttling heavily.

And yes, the claims of this making the card faster ARE true, and also applies to older cards, AMD cards draw so much power and make so much heat, that if you UNDERCLOCK them, they can get faster, because sometimes the stable speeds with less throttling is a bigger benefit than the amount that you underclocked the card.

That said, last I checked (this was 2 months ago), the driver changes were Windows only, some guys mining bitcoin on Linux didn't noticed and happily melted their riser cables.

I really don't like nVidia behaviour as company, pulling shady tactics left and right, but AMD frankly shocked me with their shoddy engineering. (also see: recently Linux kernel devs REFUSED AMDGPU driver patch... AMD was warning in February that the patch they were attempting to do was crap, and they still went ahead with it, there was even some passive-agressive personal insults exchanged on official Linux kernel mailing list).

</comment></comment></comment></comment><comment><USER11> The really interesting "loyalty" aspect I've seen recently is AMD users (/r/amd, etc) buying into AMD stock, big time. It's not just loyalty on hardware purchases anymore, it's literally propping up their stock price.

Of course they seem to have been correct that AMD was way underpriced where it was before - but at some point it'll come back down too.

</comment><comment><USER12> The lack of a competitive consumer CPUs from AMD gave the year to Intel when it came to new gaming systems. With RYZEN and Vega GPUs coming Q1 2017, AMD's 80%+ rally in the market, and Intel's lack of innovation, I expect a comeback for AMD in the gaming PC market. Their move back into the Professional and HPC GPU market will also be a huge financial push for AMD to get back where they want to be.

</comment><comment><AUTHOR> We all know AMD is losing market share to Intel and Nvidia, but it's interesting to see that there was so much loyalty to AMD GPUs among people who bought AMD CPUs. This is starting to decrease as Nvidia wins more of the GPU market. AMD CPUs were only included in 11% of PC builds in the last six months on PCPartPicker.

<comment><USER13> I believe AMD's market share in consumer PC's is about to double or more with Zen, if their demo is to be believed. Their supposed < $500 CPU is on par in performance with Intel's $1100 CPU with a TDP of 95W instead of Intel's 140W.

<comment><USER14> Agreed. And with both lower power usage and ubiquitous support for ECC memory, I think we will be seeing a lot of them in server rooms.

<comment><USER5> If Zen ends up mopping the floor with the E3 and E5 Xeons I will be one of the first to replace gear in my homelab. The amount of money Intel wants for a second E5-2403v2 (a really weak cheap in comparison to my similarly priced i5-4570 in my desktop) to go in my ThinkServer TD430 is insane. Not to mention they're still gimping the memory capacity and PCI-E support of the low-end E3 chips, even though they can cost almost as much as a low-end E5/E7 with similar clock speeds (this part is extremely annoying for my FreeNAS box which is an HP ML10, I could really use support for more than 32GB of RAM).

</comment></comment></comment></comment><comment><USER15> Never had any loyalty to any manufacturer.

When I need a new CPU or a new GPU, I just open the latest benchmark and go for the best price/performance ratio for gaming.

<comment><USER16> In AMD's case, I think at least some of the loyalty to them is more than fanboyism. They're the only competitor to Intel in the desktop CPU space and Nvidia in the GPU space, and they're in a much more precarious financial position than either competitor. If AMD go away, both of those markets become completely monopolized. So in a way, it is to everyone's benefit to disproportionately support AMD.

</comment></comment><comment><USER17> I don't know how many of you here keep up with AMD but their drivers have lately been great, on par with Nvidia at least. The Crimson edition showed a change toward quality. I still think that for the price AMD cards are competitive.

</comment><comment><USER18> The data sample is for the past six months -- from my limited understanding, most big-ticket items have their prices raised prior to the Christmas holiday shopping frenzy so that grandiose claims of price cuts can be met with a straight face.

Perhaps most AMD GPU products have seen a higher price raise in the past six months in preparation for this shopping frenzy?

<comment><AUTHOR> The data is for Intel, Nvidia, and AMD, so you would expect the same from all, no? It's more about relative market share amongst the three.

</comment></comment><comment><USER19> It's pretty obvious why: Nvidia's top 5 GPUs wipe the floor with every current AMD offering.

The GTX 1070, GTX 1080, Titan XP, Quadro P6000, and Tesla P100 are all faster than every AMD GPU in similar price brackets.

Additionally, GloFo 14nm (used by AMD Polaris) is generally thought to be less efficient than TSMC 16nm FF+ (used by Nvidia Pascal).

</comment><comment><USER20> I've always supported AMD because they were the underdogs. However, overtime the cost advantage disappeared for it's GPU.

I will continue to buying AMD CPU but I've already switched to Nvidia for GPU. I will switch back if the next offering is something like Maxwell cards, silent, low power consumping GPUs.

</comment><comment><USER21> I still purchase AMD CPUs since the price difference between Intel is so huge. I have however given up on AMD GPUs, the price and performance are usually around Nvidia, but it's just always a little crappier or buggy.

I always want them to do well, since I don't want just on company running the x86 market.

</comment><comment><USER11> That's no surprise. AMD's CPUs are now objectively inferior except for a few narrow use-cases: highly threaded workloads, VM hosts that need lots of physical cores to pin to machines, maximum iGPU performance, ECC RAM support on a budget (note that i3 also supports this), or very cheap fileservers/media PCs (especially AM1). A high-end FX-8350 is going to significantly underperform an i3-6300 while gaming - especially in minimum frametimes. The single-threaded performance of AMD's construction cores (Bulldozer/Piledriver/Steamroller/Excavator) has always been abysmal.

The FM2-based products are alright, but they are glorified laptop processors and they do not really compete well in the desktop market overall. Nice if you want a decent iGPU but most people use discrete GPUs for any serious gaming, and without the iGPU all you have is a mediocre CPU.

AMD's future in the CPU market rests heavily with Zen. Right now they essentially do not compete for the vast majority of users (power-sensitive mobile/server market, productivity users, or gaming). They run the games, the averages are sometimes decent, but the minimum frametimes suffer pretty badly and they use a lot of power. Compare the 99th-percentile frametimes and the cumulative frametime charts here (both are "badness" metric for measuring stutter) and you can see that AMD's single-threaded performance really torpedoes some games far beyond expectations. The FX-8350's 99th-percentile frametimes are significantly worse than a Pentium G2130 and it even falls behind a dual-core Clarkdale (first-gen Core i5).

[http://techreport.com/review/23750/amd-fx-8350-processor- rev...](http://techreport.com/review/23750/amd-fx-8350-processor-reviewed/5)

AMD's GPU products, on the other hand, are still reasonably competitive - although their top product only competes with a GTX 1070 and is very low on VRAM capacity during a time when VRAM consumption is increasing rapidly and will continue to do so (particularly DX12 and Vulkan games). The RX 480 is a solid card though, especially for the price, and the Fury is due for a refresh soon with the new Vega series, which will undoubtedly have more VRAM.

In particular, there's a problem with combining AMD GPUs and AMD CPUs. AMD's GPU driver stack has a reputation for being single-threaded and somewhat inefficient - so you really need good single-threaded performance with AMD GPUs more than ever before, and Intel processors are very much the preferred pairing. Again, this difference is particularly pronounced when comparing _minimum_ frametimes rather than averages.

They have been working hard on cleaning this up with Crimson and they made another big driver refresh recently too - but AFAIK there's still a pretty significant quality-of-life improvement from using Intel processors with your AMD GPU due to minimum frametime improvements.

Conversely NVIDIA's driver stack has a reputation for being less dependent on good single-threaded CPU performance. So perversely, if you are running on an AMD CPU then you are best off getting an NVIDIA GPU.

Also, side note: AM1 is my favorite AMD CPU platform right now _by far_. The CPU supports ECC, most motherboards don't but the Asus AM1M-A does. It makes a nice little NAS box if you can forgive its paltry 2 onboard SATA channels and mATX footprint, and you can pick up a CPU+mobo for $45 from MicroCenter.

<comment><USER22> The surprise is that it is supposed to happening now, shrinking relative to 2015 when AMD CPUs where already just as far behind.

Maybe there was a pattern of brand-loyal, sufficiently rich gamers who made a hobby out of building a new system at the very top end of AMDs offerings whenever a new generation Radeons came out. Those would not find an excuse for an upgrade this year because the latest generation of Radeons tops out lower than previous generations.

But this explanation attempt breaks down completely when I try to back it up with the numbers from the article. The way I understand them, the average AMD/AMD system has actually become more expensive, suggesting that AMD lost more at the low end. But I have similar problems with the main conclusion of the article: what I see in those tables is that the Radeon fraction has dropped faster on Intel machines than on AMD CPUs. With this in mind I would rather conclude that brand loyalty is even more important for AMD than it used to be, it's just that with shrinking general popularity of Radeons, the CPUs get less of a helping hand.

<comment><USER23> The analysis in the article (and the PCPartPicker spreadsheet from which they source their data) excludes integrated GPUs.

This is a _significant_ difference, because Intel has a 17.48% marketshare on graphics per the Steam Hardware Survey [1] (the original source for the spreadsheet data), while AMD has a 23.46% share -- which includes both their integrated and discrete graphics.

The average price of AMD builds has risen because the lowest-end builds on the Intel side can rely on Intel's integrated graphics. Intel integrated GPUs attain one or two percent share of all GPUs for each generation, while AMD APUs are far behind.

[1] [http://store.steampowered.com/hwsurvey/](http://store.steampowered.com/hwsurvey/)

</comment></comment></comment><comment><USER24> I used to be quite loyal to AMD (and I'm typically not loyal to any companies), but their latest chips are meh. Their GPUs perform very good in benchmarks, but are buggy and poorly supported otherwise.

<comment><USER25> I often had a lot problems with the drivers on AMD GPUs. Sometimes it took 2 months to be fixed. Two months during which my laptop heated too much and I was unable to play games and had problems on Youtube.

</comment></comment><comment><USER26> My last 3 discrete GPUs for PC were AMD, and while AMD keep reasonable prices, Linux Kernel support, and supporting standards on Linux (OpenGL, OpenCL), I'll continue buying from them.

</comment><comment><USER27> Not counting Zen there are considerably more reasons to buy an AMD GPU than a CPU.

Everyone I know with an AMD GPU has an Intel CPU (Core i5/7) since buying AMD GPU's for gaming is an utter waste of money.

The CPU's are slower, and the platform is old and buggy, not modern PCIE, storage performance is garbage, and USB 3/3.1 support is laughable.

</comment><comment><USER28> My kids and I game on the family computer some, but nothing that requires bleeding edge technology. To me the performance of AMD GPUs is good enough, but they produce a ton more heat and require more active cooling than a comparable NVidia GPU. Even with very good fans there is no way to get a quiet PC that performs well using AMD GPUs.

</comment><comment><USER29> its funny its decreasing now... because now Vulkan is released which put amd back in the market tbh with power consumption and performance. the cpu's i've ditched after i couldn't get the octa core properly stable. but i'm learning towards their gpu's now as it's less than half price for same performance...

<comment><USER30> Vulkan is yet to be widely deployed.

</comment></comment><comment><USER31> Are there any deep learning libs that work ok AMD cards? Looks like CUDA totally owns that market.

</comment><comment> amiga-workbench After a AMD driver update removed audio pass through over HDMI on my HTPC build I'm not going back for more, Nvidia seem to support their gear for longer too.</comment>


NASA's X-59 supersonic jet is cleared for final assembly - <AUTHOR> https://www.engadget.com/2019/12/17/nasa-x59-supersonic-jet-assembly/ <comment><USER2> More flying garbage from lockheed martin which leans heavily on its wallstreet board connections to grant it government welfare contracts it doesn't deserve.</comment>


Ask HN: Service that sends 0-day security bulletins only for software we use? - <AUTHOR>

Staying up to date with vulnerabilities is proving to be a difficult task.  Not all software has security mailing lists, and some do not even have forums or any discussion thread thereof.  Lastly, operating system package updates are not always fast enough to protect against targeted attacks.
Is there any service that will keep track of the vulnerabilities of all popular software and will only tell us about the software we care about so that we can patch the software the instant a vulnerability becomes public?
If not, can you make one tptacek?
Thanks in advance,<AUTHOR> <comment><USER2> Securityfocus used to do this. Symantec bought them, and I'm not sure if they continue to offer the service, but they still create BIDs, so I'd assume so. Subscribers got an interface that let them choose what software/versions they care about, and they would get alerts based on that. I think Secunia either has or had a similar service.

A few more are mentioned in this article from 2005. I'm not sure how many are still up and running. It seems like a few businesses attempted this but it didn't work out well. Oddly, people were excited about this ~8 years ago and most companies got acquired and it seems like it died out.

[https://webcache.googleusercontent.com/search?q=cache:MVkOyW...](https://webcache.googleusercontent.com/search?q=cache:MVkOyWa_VXwJ:www.networkworld.com/reviews/2005/012405revvuln.html+&cd=2&hl=en&ct=clnk&gl=us&client=ubuntu)

</comment><comment><USER3> Would you consider paying, say, 2 or 3 dollars a month for a service that attempts to aggregate and curate almost all the 0 day mentions it can, alerting you when a particular keyword matches? Something like a targeted Google Alert?

I presume obscure pieces of software should be dealt with in a case-by-case basis. There must be a service that does this, I can't find any at the moment.

If you're allowed to say, how rare is the software you want to track?

This has piqued my interest because I've been in the same situation and it became a hassle to manage many mailing lists. My solution never went beyond a few mail filters.

Nowadays, Google Alert helps quite a bit but it is hard to keep the noise out.

I'm certainly no tptacek, but I'd be most willing to give it a shot!

<comment><USER4> Honestly, I think you'd get better traction making it something like $40+ a month, or alternately, allowing for company usage where there is one account with multiple email recipients, and charging $2-3 per recipient, plus a fixed fee for the account.

</comment></comment><comment><USER5> SecurityFocus maintains a list of vulnerabilities at <http://www.securityfocus.com/vulnerabilities>

You can use Feedity (our startup - <http://feedity.com>) to create RSS feeds (and even merge multiple feeds into a stream, per software in your case) for the vulnerabilities list.

</comment><comment><USER6> The Windows world has Secunia's Personal Software Inspector (PSI).

<http://secunia.com/vulnerability_scanning/personal/>

I can't really speak to it, personally. I tried it on one machine, some time ago, but I was leery of using it more widely including but not only for the reason of having yet one more thing scanning and bogging down the system drive.

The security-oriented press seemed, on balance, favorable towards the product.

</comment><comment><USER7> It would be interesting if that solution included a program that analyzes your machines installed software, and notified you on 0 days about the exact software-version you have.

Later, on the site, you would have your machines listed, with the software on these machines, and receive alerts per machine.</comment>


Show HN: Hivemind – Distributed jobs using AWS Lambda functions - <AUTHOR> https://github.com/littlstar/hivemind <comment><USER2> More mature python based solution.

[http://pywren.io/](http://pywren.io/)

</comment><comment><USER3> This would be an entertaining comment when read by a C programmer:

        // Overly clever way of saying: increment `currentChunk` then use the incremented value
        this.chunks[++currentChunk] = []

Pre- and post-increment operators are bread and butter in C, hardly overly clever. Are they a recent addition to JS?

<comment><USER4> Not only when read by a C programmer. I'm a JS programmer and I don't understand what's "overly clever" about something you learn in JS 101.

<comment> J-dawg Although the post-increment (x++) operator was part of 'JS 101' for me, I've never knowingly seen the pre-increment (++x) operator before. TIL.

</comment></comment></comment><comment><USER5> What do folks use for event-driven scatter-gather job control using Lambdas only? For example, imagine I launch 10K Lambda jobs from an orchestrator Lambda. I can't guarantee they'll finish in 300s, so I can't just poll for completion. I could wake up a gather Lambda as a scheduled event, but the schedule granularity is too coarse. Waking up a gather lambda every time a job completes is wasteful and racy.

I would really like AWS to have an event that fires when all the lambdas in a set have completed, but that doesn't seem to exist. Any suggestions?

<comment><USER6> Have you tried Step Functions? I've used them to orchestrate Lambdas with good success when I needed something that will run longer than Lambdas can individually.

<comment><USER7> Yeah, they're good, but more expensive than you'd expect given the relative pricing compared to other AWS services. $0.025/1000 transitions seems and order of magnitude or two high given it's just ultimately a central job store with some very, very simple logic for changing state.

<comment><USER8> Step Functions are convenient but for the reason you stated we stopped using them. Pricing seems off for what they do, especially when it's in volume.

</comment></comment></comment></comment><comment><USER9> Here's a not-even-half-baked idea I've had for a while: would it be useful to have partial application of AWS Lambdas? Such that if you don't "call" an AWS Lambda with all its "arguments" it will create a new AWS Lambda that's just waiting for the remaining arguments, and return an ARN for the new Lambda?

Again, I can't imagine why this would be useful, but it's a certain kind of fun to take Amazon's metaphor and run with it.

(It's also very hard to Google for "aws lambda partial application" and get results that match what I'm describing here.)

<comment><USER10> Why stop there. What if we implemented aws lambda calculus.

<comment><USER11> Distributed lambda calculus would be fun to implement. Differential distance- based reduction cost to start with?

</comment></comment></comment><comment><USER12> Thanks for sharing! I ended up building a similar featureset in Python but using invocation type Event instead of RequestResponse. This ended up being a pain in the ass because Event type has these awful baked in automatic retries that you cannot configure or disable. It ended up being worth it because we wanted to use up to the full 5 minute runtime limit.

How is this working for you? Where do you see it going?

<comment><AUTHOR> >How is this working for you? Where do you see it going?

It actually works quite well. I use it for processing large amounts of data quickly. I originally built this to process some data that would've taken ~14 days and shortened to a few minutes (so nice!).

I enjoy using it when I can find a way to fit it into a project. I've had it internally for a month so not a whole lot of time to see it in a variety of projects.

In terms of where I see it going, I'm not entirely sure yet. It is pretty capable for what I am using it for currently, it can use node.js libraries so it fits the current needs (which are pretty basic). I'll see what happens when more people try it out. :)

Do you have any tips on what should be added, since you've been down this road before?

<comment><USER12> Most of the things I've added are for logging and orchestration. The most important unlock for us was the ability for us to enable chaining of Lambdas that worked seamlessly with our orchestration. So Orchestrator kicks off 25 lambdas (A) that are reading files off S3. Each Lambda A reads 100 lines of that file and invokes a Lambda B to parse it. Lambda B parses and filters each line in its batch and sends it to Lambda C to be persisted or Lambda D if it needs to be discarded.

We end up with hundreds or thousands of Lambda invocations and the orchestrator is aware of all of them. If there's a failure in any lambda, I can stop the entire job and retry from scratch. The orchestrator knows exactly when the job is done so I can easily chain it together with some other non- Lambda job in my system.

Been working fine for us for ~200k invocations and 8k CPU minutes per day baseline with the ability to scale up to 10x that just by running more stuff. Our nightly job processes about 50gb of data in 3 min.

</comment></comment></comment><comment><USER13> good thing we changed our product name from Hivemind, a wicked trademark fight has just been averted:

[http://blog.codesolvent.com/2017/05/say-hello-to- solvent.htm...](http://blog.codesolvent.com/2017/05/say-hello-to-solvent.html)

:)

<comment><AUTHOR> Haha.

I originally wanted to name it unity-hivemind after I had watched Rick and Morty that weekend, but I figured the name was both a collision and a confusing due to the Unity graphics engine.

</comment></comment><comment><USER14> I'm sure this is great, but for some of us, being tied to a single vendor is a non-starter.

<comment><AUTHOR> Totally understand. :) Just thought I'd share something that made my life a little easier.

<comment><USER14> I definitely appreciate that, and I didn't mean to take anything away from your work. Cheers.</comment></comment></comment>