Estimate your Bitcoin taxes on BitTax - <AUTHOR> http://bittax.com/alpha

<comment><USER2> This looks cool, but [https://bitcointaxes.info](https://bitcointaxes.info) allows imports from all the major exchanges and has been a great tool for me.

<comment><USER3> I wasn't able to figure out how much I owe for taxes and I spent more than 10 minutes on the bitcointaxes.info. It looks more like an accounting software to me. A lot of tabular data and accounting lingo I don't understand.

<comment><USER2> Hmm, I didn't have trouble. It even calculates how much you owe with different approaches, e.g. first in first out (FIFO), last in first out (LIFO) etc. and can handle trades between coins like Bitcoin -> Dogecoin

</comment></comment></comment><comment><USER4> Great service, dead simple, accurate for me. I had already calculated my tax liability and this basically made me feel silly for going to the trouble.

I'm sure the final number is right for me, but that's because I had done the legwork beforehand. Right on.

Looking forward to upcoming features with other currencies (litecoin and peercoin, especially).

Keep with it, Bittax. Looking good.

</comment><comment><USER5> Your 2013 calculations are wrong. Really wrong. You said I am due a refund of about $50, but in reality I owe over $500 in taxes on short term gains.

0/10, would not use again.

<comment><USER6> Would you mind giving us the data at firstname.lastname@example.com? We will check these round numbers quickly. We do not keep any of your data for privacy reasons.

</comment></comment><comment><AUTHOR> I like the site, because they are the only ones who didn't launch just a landing page full of promises, but they actually calculate how much I owe to Uncle Sam.

<comment><USER7> Sorry, but this looks _exactly_ like a landing page full of promises unless you enter your Coinbase credentials. This is fallacious because Coinbase is not the only place where bitcoins will be stored.

Without giving up your Coinbase credentials, I was redirected to an email harvesting page to be on their invite list.

Maybe they changed the landing page since you last accessed it and removed that functionality?

<comment><USER6> Yes, the service currently works only for those with Coinbase accounts. We do not promise anything else at this time. I'd appreciate if you suggest which services should be added next.

<comment><USER7> I appreciate the difficulty of integrating with other services. It seems that the critical piece of information is figuring out a person's bitcoin buy/sell history based on the FIFO methodology as per IRS guidelines which you do a great job of clearly explaining in your FAQ.

I believe exchanges provide a means for downloading your personal transaction history, typically in CSV format. Thus, it would be great to be able to import/upload transaction histories for US bittax calculations.

This may boost exposure of your service since people could experience your core service functionality without exposing their broker/exchange credentials.

EDIT: If you're going to ask for credentials as the only way to access your service then you may want to also consider other popular wallet services or exchanges such as Blockchain.info, Bitstamp, BTC-E and/or Bitfinex. You may already be working on integrating your service with these suggestions, but I didn't see that listed on your site beyond the "this is an MVP" section and the October 15, 2014 date listed in the FAQ.

<comment><USER8> This is exactly what BitcoinTaxes provides [https://bitcointaxes.info/](https://bitcointaxes.info/)

<comment><USER7> Thanks for the alternative. Looks much more fleshed out and feature complete.

</comment></comment></comment></comment></comment></comment><comment><USER9> Surprised that Coinbase just doesn't offer this.

<comment><USER10> In the US the are more or less the only game in town and they probably have bigger fish to fry at the moment.</comment></comment>


Ask HN: Recommend Password Manager - <AUTHOR> Hello,
I have been postponing moving to a managed password service for a long time, the results have not been catastrophic but I've played the odds long enough.
Lastpass and 1password are often recommended. I am happy to pay an one-off fee but a monthly subscription service is a no-go for me. I would also prefer to have the option to switch off syncing and my hashed passwords going to the cloud. I don't have time to go in full homebrew mode as others seemed to have done.
So what do you guys recommend? Is there a site that compares these products? <comment><USER2> I'm using Bitwarden [1] and am happy with their product.

It's open source and you can self-host it quite easily if you don't want to use their cloud. They do have a free tier and a premium one for $10/year.

It's been a while since I set it up but if I remember correctly I didn't like the official implementation due to performance reasons and am now using bitwarden_rs [2] instead.

[1] - [https://bitwarden.com/](https://bitwarden.com/)

[2] - [https://github.com/dani- garcia/bitwarden_rs/blob/master/READ...](https://github.com/dani- garcia/bitwarden_rs/blob/master/README.md)

<comment><USER3> Seconding Bitwarden. I moved to it from LastPass a while ago and I'm glad I did. It's simple and easy to use, open source, and they keep improving it.</comment></comment>


Build a better Bookshelf - <AUTHOR> https://huytd.github.io/build-a-better-bookshelf.html <comment><USER2> I was actually hoping this was about actually building your own DIY bookshelf (for actual physical books)...

<comment><USER3> Ha! Me too. I was expecting some kind of analysis of the perfect shelf material and optimum depth and height of each shelf based on the number of books by genre and their dimensions.

<comment><USER4> Ha! I am making some shelves for myself and spent the weekend measuring the dimensions and weight of all of my books to later divide into different bins based on height, depth and weight (density).

<comment><USER5> I ended up with four different shelve heights based on my measurements (paperbacks, hardcovers, misc., and art-books), and used the classical technique of bird's beak shelving supports to make all shelves configurable.

Finished bookcase:

[https://twitter.com/jdhoek/status/885499829052485632](https://twitter.com/jdhoek/status/885499829052485632)

The four common shelve-heights I chose are 4, 5, 6, or 7 bird's beak 'units' spaced apart vertically.

You can see the configurable parts here:

[https://twitter.com/jdhoek/status/876126132734656514](https://twitter.com/jdhoek/status/876126132734656514)

<comment><USER6> Do you have a photo of the end of a shelf? I can't tell if you've got beaks at the back and front of the shelving, or if you're doing some kind of cantilevered setup.

<comment><USER5> I think you may be able to tell from the first photo here:

[https://twitter.com/jdhoek/status/881243775288389632](https://twitter.com/jdhoek/status/881243775288389632)

And the second photo here:

[https://twitter.com/jdhoek/status/876126132734656514](https://twitter.com/jdhoek/status/876126132734656514)

So beaks at the front and back, with trapezoidal blocks that fit in between. The shelves rest on those blocks.

</comment></comment></comment></comment></comment></comment><comment><USER7> Great idea. Someone please build a public database of tables of contents and indexes, keyed by ISBN, so we can skip the scanning and OCR steps. And dear publishers: when this database emerges please add to it as you publish new works.

Thanks.

<comment><USER8> Eat Your Books [0] is amazing for this, but is limited to the cookery world.

You tell it what books you own. It knows what's in them _but does not show you the contents_. You search for whatever you have in the fridge. It tells you to go to book x, page y, and there's the recipe.

Oh, and it includes magazines.

[0]: [https://www.eatyourbooks.com/](https://www.eatyourbooks.com/)

<comment><USER9> Hmm and to add more than 5 cookbooks you have to pay.. :-(

<comment><USER10> As long as the pricing is right I appreciate that alternative.

We complain a lot here (and to a certain degree, righly so) here on HN about annoying ads and creepy tracking so I will try to e positive when someone actually honestly makes a great product and asks for money instead of trying to extract it from profiling me and selling the profile.

<comment><USER11> Are you sure that their not doing both?

<comment><USER10> Definitely not, see an article I submitted the other day for proof: )

[https://news.ycombinator.com/item?id=17940617](https://news.ycombinator.com/item?id=17940617)

 _But_ : if something don't have an obvious way to monetize I fear one of three:

\- I'm not the user but the product

\- They're too idealistic and will disappear

\- They're looking to sell out to a bigger actor, -and then my data will be abused

With an open source self hosted solution I can be somewhat safe.

With a paid product there is a bigger chance they will be smart enough not to to kill the goose that lays the golden eggs by selling out their paying customers.

Example where this worked: Basecamp

Example where this failed: WhatsApp

</comment></comment></comment></comment></comment></comment><comment><USER12> Does anyone have any experience with book scanning in general? I've been eyeballing a unit from "CZUR" but am a bit skeptical of the product in general. I would prefer to buy something more generic/high end HW wise like a V-shaped scanner where you bring your own DSLRs but can't find out if there is a serious open source software platform for them.

<comment><USER13> What a shame that people have to use homebrew methods to duplicate the effort that someone (Google) has already done with vastly greater resources.

Google Books, a.k.a., Google Book Search, has already scanned and OCR'ed about 20% of all books in existence -- i.e., 25 million of an estimated 130 million books. So, "somewhere at Google there is a database containing 25 million books and nobody is allowed to read them."[1]

I understand the legal issues, authors not getting compensated, etc., but it's still a shame that it actually exists but it's inaccessible (except for snippets).

[1] [https://www.edsurge.com/news/2017-08-10-what-happened-to- goo...](https://www.edsurge.com/news/2017-08-10-what-happened-to-google-s- effort-to-scan-millions-of-university-library-books)

<comment><USER14> Well, at least Google has all the books indexed, even if we can't see them, so that you know what to go to your local library and look for even though you can't see it there on the internet.

</comment></comment></comment><comment><USER15> Offtopic: huydotnet, would you consider adding RSS to your blog? I'd like to subscribe, but couldn't find a link anywhere, including the source code of the page.

It seems like you're generating pages from Org mode. I've recently discovered ox-hugo, maybe it'll be of interest to you too. I wrote about my setup here [https://rakhim.org/2018/09/moved-from-jekyll-to-hugo-and- ox-...](https://rakhim.org/2018/09/moved-from-jekyll-to-hugo-and-ox-hugo/)

<comment><AUTHOR> Nice, I'm trying to figure out a way to generate RSS from my org notes. Reading your post now :D Thank you so much :+1:

</comment></comment><comment><USER16> CamScanner[1] on Android does a very nice job of this kind of work. I'm not associated with the product, just a longtime satisfied user.

[1] [https://www.camscanner.com/](https://www.camscanner.com/)

<comment><USER17> Hmmm, should I use a document scanner application, made by a company in China, after the recent discovery of numerous applications on iOS and OSX sending data to China.....

Sure, why not....

/sarcasm

</comment></comment><comment><USER18> I find it cool that this blog post was originally sketched by hand: [https://huytd.github.io/img/handwritten-build-a-better- books...](https://huytd.github.io/img/handwritten-build-a-better- bookshelf.jpg)

<comment><AUTHOR> Thanks :D if you go to the homepage, there are some other sketches as well

</comment></comment><comment><USER19> I've found books.google.com be alright for this as well.

<comment><USER20> The author specifically noted:

 _And the search result is limited to just my books, not the whole internet, that 's the point of scanning instead of Googling._

<comment><USER14> And the above commenter specifically noted

 _I've found books.google.com be alright for this as well._

Google Books doesn't index the internet. It indexes books. Including using OCR.

<comment><USER20> Consider for a moment the critical difference between _my books on my bookshelves_ and _Google-indexed books in the ether_.

<comment><USER21> You can only search books you own by adding it your library in Google Books. This should limit it to returning results only from books you have.

<comment><USER20> The problem with Google Books is search capability within "your library" really has no way to constrain itself to tables of contents and/or indices; even advanced search falls flat on its face by this measure and generally sucks hind tit by every other. I'm just imagining running a search query for "recursion" only to have Hofstadter and Wolfram comprise 90%+ of the return.

Even if full search capability is what you _really_ want, you're limited to ebooks for which digital rights are established via Google Play purchase. For everything else that isn't acknowledged public domain, you'll get a tease preview without even a hint of completeness...if you get anything at all. Invested in a Kindle digital library? I'm liable to suspect Google gave users vested in Amazon DRM a gratuitous GTFO-not-in-my-backyard finger.

Then there's the issue of specific revisions, e.g. it doesn't matter that Google Books shows skant preview of 4e Sedra/Smith when 5e is on my bookshelf --which, oh by the way, doesn't have preview.

Then there's the issue of copyright lockdown, e.g. only one of Carroll Smith's popular _...to Win_ series has any preview; the others are on lockdown. Same with Milliken on vehicle dynamics, Katz on aerodynamics, and many other titles published by SAE.

(Observe how the thematic high mark we've been striving for is a mere preview.)

Which brings up the issue of availability, e.g. good lucking trying to find the Institute of Navigation's canonical GPS red books in a Google Books query, or specific translations of the Bhagavad Gita, or other rare publications that an archivist would generally not hesitate to shoot you dead if caught attempting to adulterate in a scanner.

Surely I'm not the only person on HN who maintains a tangible library that's 1000+ large and growing. Like the author, I care about what's in _my_ library, not what Google Books superficially pretends to offer as a front to getting me to purchase books in the digital (when available) that I've already paid handsomely for. Amazon and eBay combined see roughly 70% of my bookshelves. Even if Google Books did a fair job by some objective measure, I neither need nor desire their service.

</comment></comment></comment></comment></comment></comment><comment><USER22> Are there multiple versions of the current OneNote? I can't OCR anything in my version for Windows 10. I'm stuck like this guy - [https://answers.microsoft.com/en- us/msoffice/forum/msoffice_...](https://answers.microsoft.com/en- us/msoffice/forum/msoffice_onenote-mso_win10-msoversion_other/onenote-ocr-not- working/e239c9ea-e1b3-46c3-a976-45a418231dbc)

<comment><USER23> AFAIK, OCR is only available in OneNote 2016 (i.e. through a licence or Office 365), but not in the OneNote UWP app, which you get for free in the Windows Store.

Edit: Though, if you imported the Image/Print in OneNote 2016 you can search the OCR result in OneNote UWP and access it using Right-Click -> Picture -> Alt Text

<comment><AUTHOR> I think the free version also has it, I'm using OneNote with my personal account, and I never pay for it.

<comment><USER22> Any chance you can tell me the steps you've used in the past to get OCR out of it? I just took like 30 photos of a book to test this idea out and can't OCR it.

<comment><AUTHOR> I scanned them by Office Lens, seems like OneNote itself does not have OCR.

</comment></comment></comment></comment></comment><comment><USER24> This reminds me of Bret Victor's Bookcase which displays the sections "highlighted" by projecting them on a wall and navigating to that page on an iPad.

</comment><comment><USER25> The fact that we have to scan any technical book published after 1980 is a distortion of capitalism. The obvious most efficient solution for everyone would be to have the darn fully searchable digital version of the book + the source code.

<comment><USER26> It's plausible that the fact we have so many amazing technical books is also a "distortion of capitalism". The obvious most efficient course of action is to not write a book.

<comment><USER27> But books predate capitalism as well... Dating back to times when writing or copying them was even less efficient...

<comment><USER27> (should have included this link...) [http://catandgirl.com/one-panel- every-250-years-since-30000-...](http://catandgirl.com/one-panel- every-250-years-since-30000-bc/)

</comment></comment></comment></comment><comment><USER28> Great idea !

</comment><comment><USER29> I'd rather scan the whole book and replace my library with an iPad. In fact, I've already done that. With over 1,000 books on my iPad it's the only reason I'm still married.</comment>


Ask HN: What email client do you use? - <AUTHOR>

Wondering what email client people use?<p>I have 3 or 4 different gmail accounts and it's a pain keeping up with all of them. I have an email address for support, one for feedback and a personal one. <comment><USER2> Apple's Mail.app, which is Good Enough. I have five accounts, all of them are checked with a single click, and I don't have to worry beyond that.

<comment><USER3> And it syncs all the accounts over to my iPhone:)

<comment><USER4> I just wish there was an Inbox that combined all incoming mail from my 4 accounts.

<comment><USER5> I don't know what you're missing, but I've got one Inbox folder with four accounts underneath it (3 imap, 1 pop) - the root Inbox contains all messages in the sub-inboxes.

<comment><USER3> Not on your iPhone you don't. Unless you've found something really cool:)

</comment></comment></comment></comment></comment><comment><USER6> straight up google apps gmail with gmail notifier. As far as multiple accounts:

1) set other accounts up to forward to your gmail/gapps email 2) setup a filter + auto label system for those emails 3) add it as an outgoing email that you can reply from (limited to 7 i THINK)

<comment><USER7> I use the Google Apps/Gmail interface also. I've found that as it has matured over time there are less reasons I would want a desktop client and more reasons for having the portability of a cross platform, location independent web interface.

Gmail is a class act when it comes to web apps.

<comment><USER8> I use IMAP on gmail with opera. I need a desktop client, for times when i'm not connected and quickly need to search an old email for some small little detail.

I just use it as an offline backup, and keep it synced.

<comment><USER9> Google has an offline gmail version in beta. Requires Google Gears (not available for FF3.5, last I checked) but works for Safari.

All my mail is synced locally but I can still use the Gmail interface.

<comment><USER10> it doesn't sync all your mail.

<comment><USER11> It is available for Firefox 3.5 and it does sync all your recent mail, including attachments. It uses an automatic, non-configurable heuristic to determine how much mail to sync. I believe mine syncs the past 1 year's.

</comment></comment></comment></comment></comment></comment><comment><USER12> Surprised there isn't any votes for Postbox app (<http://getpostbox.com>).

      * It integrates well with gmail
      * does great with multiple accounts
      * multiple outgoing addresses
      * great search functionality
      * GTD tools such as assigning emails a topic.
      * Integration with facebook/twitter to pull up contact info
      * Extensions such as calendar and contacts

Based on thunderbird, all around awesome client.

<comment><USER13> I, too, was surprised there weren't more. I use this; I like it quite a lot. The search feels quite superior to Thunderbird's native search.

</comment></comment><comment><USER14> Mail.app with MailActOn (<http://www.indev.ca/MailActOn.html>) which I am not affiliated with and is not free. Use it for 7 email accounts, lots of RSS, all filtered carefully into one combined inbox and organized manually but quickly into local folders with that MailActOn app.

In my experience heavy use of Mail.app results in accounts that do not get checked for email, and show no connection error or indication of a problem. No email for an hour and I restart the app. I wish refreshing my accounts actually displayed the status of checking each account, it is very obvious sometimes that the refresh button does basically nothing or at least doesn't inform me in any way what it is doing...

<comment><USER15> Turn on the Activity Viewer: Window->Activity Viewer or ⌘-0. Shows each account being accessed and all related activity. Also allows actions to be cancelled.

</comment></comment><comment><USER16> I can't believe I'm going to say this

don't laugh, but I use outlook, and .........AOL.

I've had the AOL account for years and I've never cared enough to switch it over (it handles all of my spam like e-mails anyway).

I've never really cared about my e-mail client in general. If I were motivated to switch - which is the most intuitive for organization?

</comment><comment><USER17> I use Gnus.

I've got it setup so all my incoming mail lands in Gmail. Then, Gnus checks that account via POP3. Gnus only checks mail when prompted, so anything that Gnus hasn't fetched yet shows up as unread when viewed on the web or via iPhone/IMAP. Gnus is setup to select the correct SMTP server based on the contents of the message being replied to or the view that initiated compose- mail.

</comment><comment><USER18> Thunderbird latest beta.

I also use the iPhone email client just about as much as Thunderbird.

<comment><AUTHOR> I also use my iPhone's client, but mostly for reading emails.

</comment></comment><comment><USER19> You might've made this a poll. From the HN FAQ:

> How do I submit a poll?

> <http://news.ycombinator.com/newpoll>

> This is an experimental feature so there is currently a fairly high karma > threshold for submitting new polls.

</comment><comment><USER20> VM, which (like gnus) is built on top of Emacs. I use a very old version (version 5) because I did not like the direction the project took after that. Heavily modified by me.

</comment><comment><USER21> Outlook at Work. Gmail and Google Apps gmail for my domain accounts.Also use gmail on my blackberry through the regular blackberry email client.

</comment><comment><USER22> Gmail and Mutt.

</comment><comment><USER8> Opera.

Mainly because it's open almost the entire day on my laptop.

</comment><comment><USER23> SquirrelMail, Thunderbird

</comment><comment><USER24> Gnus. (Local IMAP fed by offlineimap from my real IMAP server.)

I use K9 on my phone.

</comment><comment><USER25> I use mailnotify to check a few different accounts for new messages (<http://www.nongnu.org/mailnotify/>) then just log on to a webmail site to access the mail.

</comment><comment><USER26> Thunderbird, Usermin (our webmail client), GMail, and the Android mail application, in that order.

All of my @virtualmin.com mail comes to one mailbox via aliases, and I can access that box from home on Thunderbird, the web with Usermin, and my phone with the G1. GMail is for personal stuff, and for chat.

Our support is handled through a ticket tracker, which makes things dramatically easier to manage.

</comment><comment><USER27> fetchmail/procmail/mutt. All of my addresses forward to a single imap account (some via gmail for spam filtering).

<comment><USER28> I too use this. At work I use the same combination, but also Evolution, just because I get too many customer e-mails that are unreadable without an html reader. For personal e-mail, I get few enough of those that a lynx filter is sufficient.

</comment></comment><comment><USER29> GyazMail: <http://gyazsquare.com/gyazmail/>

I've been using this Mac OS X client for quite some time. It's everything I want in a desktop client. It has become rock soid and is slowly but steadily improving and adding features.

</comment><comment><USER30> You can solve the problem using only one gmail account. Just forward mails from the other accounts, and configure a filter to put different tags on them. You can send from different email addresses using only one gmail account, too.

</comment><comment><USER31> I have RoundCube [<http://roundcube.net/>] installed on my server and all of my other accounts forward to it.

I constantly hack it to add functionality where needed.

</comment><comment><USER32> Outlook with Xobni at work. I've now become the guy that can find "that email".

</comment><comment><USER33> I use Inbox2 (www.inbox2.com), but then again I am building that myself :-)

</comment><comment><USER34> fetchmail -> procmail -> mutt -> vim -> gpg -> ssmtp

<comment><USER35> This is similar to what I use, except offlineimap instead of fetchmail and esmtp instead of ssmtp.

I like offlineimap because it does a bidirectionaly sync, meaning I can keep a local backup of mail, but still use the web client when I need to (without a ton of old mail sitting around).

I started using esmpt because it handles local mail delivery when paired with procmail, allowing me to get the infrequent messages to root without having to run a full MTA on my laptop. There's also a script to allow mail queuing, but I haven't ever bothered with it.

</comment></comment><comment><USER36> Outlook on my windows laptop. Thunderbird on my linux desktoip. Managing multiple emails with these clients have never been a problem. My personal gmail account alone is web only.

</comment><comment><USER37> GMail on Mozilla Prism then Mail.app for everything else - I don't have a clue how Thunderbird performs on the Mac, but Mail.app works fine atm.

Lastly, Thunderbird on Windows

</comment><comment><USER38> Mail.app with Gmail / Google Apps, so I have backup in the cloud, access anywhere, and IMAP syncing with iPhone. Works great.

</comment><comment><USER39> Gmail on my desktop with the gears extension. Sometimes I use my blackberry message client to read emails.

</comment><comment><USER40> Thunderbird for Windows over SSL POP3.

</comment><comment><USER41> Google Apps GMail, mail app on the iPhone (and occasionally web app).

Mail sucks, GMail makes it ok.

</comment><comment><USER42> Gmail: all my accounts syndicate to there and I use multiple inboxes lab feature to sort it all.

</comment><comment><USER43> gmail for my main account and yahoo!mail for any sign ups, etc. that I don't really care about.

</comment><comment><USER44> Behind my main machine thunderbird, outside Horde.

Interface 'dynamic' is quite amazing.

</comment><comment><USER45> pine.

I kid, I kid. I use google apps all around (work & home).

<comment><USER46> I used Pine for a long time. It was great. I use Thunderbird now.

</comment></comment><comment><USER47> thunderbird for work, home and projects on the desktop. ipod mail.app and nokia n96 mailer the most rest of the time. If all those unavailable, squirrelmail.

</comment><comment><USER48> Gmail in a Fluid app, and Mail.app on iPod Touch.

</comment><comment><USER49> Kmail

</comment><comment><USER50> i switched from thunderbird to evolution and have been very pleased.

</comment><comment><USER51> TheBat for 10 years now.

</comment><comment><USER52> KMail

</comment><comment><USER53> Outlook for my exchange based email (at work), Thunderbird for IMAP (at home).

</comment><comment><USER54> gmail + thunderbird at work.

1 gmail account for all span (even if it looks like it might possibly have a slight chance of being span i use this email). Not forwarded. 1 gmail account (old) forwarded to main gmail account 1 gmail account main 1 work account

I use thunderbird at work to view work related emails, but gmail client to do everything else. Can't beat that damn gmail client!

</comment><comment><USER55> Mail.app in the office, mutt if I'm terminal bound.

</comment><comment><USER56> claws

<http://www.claws-mail.org>

<comment><USER57> Me too. Stable, fast, extensible, keyboard friendly, and tons of features.

</comment></comment><comment><USER58> Gmail forwarding all my accounts to my main account.</comment>


How misaligning data can increase performance 12x by reducing cache misses - <AUTHOR> http://danluu.com/3c-conflict/ <comment><USER2> I see many comments here acting like this means not aligning on word boundaries (e.g. using packed pragma, sandy bridge have unaligned access, etc.). This has nothing to do with word alignment. As the article states at the beginning, this is about aligning to page boundaries, which is on a very different level than word boundaries or structure packing. Let's not get these confused.

</comment><comment><USER3> It's important to at least know about the n in n-way set associative cache (i.e. that it exists) and this article is a good reminder. Next time you see data accesses that ought to be fast (ought to be hot in cache) but seem not to be, this is another thing you can look for.

It's easy, from a software engineer perspective, to know that your CPU has cache, and just think it's like any other cache you might implement in software. But the implementation details - the hashing by masking the address, and only having a few slots available for things that hash to the same bucket - are actually important, as shown here.

</comment><comment><USER4> One of the things to keep in mind here is that this trick won't work or work as well for every architecture or every processor family in that architecture.

Some architectures do not support unaligned memory access and will raise an exception. If you're using things like packed attribute with your structs your compiler will generate the correct code but that code will be slower. In almost all cases it will generate many more instructions and because of that your cache will be less effective (due to larger code size) your decoder cache will be less effective, etc...

The author has a more modern Intel processor. The x86 family always supported unaligned access, albeit it was always slower in terms of cycles. More recent Intel processor have made this penalty much shorter. I believe this was driven network applications many of which focus on efficiency of packing as many bytes down the channel and less on the alignment requirements. [http://www.agner.org/optimize/blog/read.php?i=142&v=t](http://www.agner.org/optimize/blog/read.php?i=142&v=t)

<comment><USER5> You're talking about word aligned boundries, which is absolutely an issue with certain architectures(I remember dealing with the issue with older SPARC processors). The article is talking about L2 and L3 processor cache hash collisions, which can result in lost performance as the cache's are overwritten.

This optimization doesn't preclude those architectures necessarily, it's saying that instead of allocating at address 512, 1024, etc., there might be a boost from allocating at off-page addresses.

<comment><USER4> You are correct.

</comment></comment></comment><comment><USER6> The usual term for this is "cache line colouring", and many allocators do this for you.

[http://en.wikipedia.org/wiki/Cache_coloring](http://en.wikipedia.org/wiki/Cache_coloring)

</comment><comment><USER7> I understand this only enough to understand that it may or may not be a surprising and paradoxical result.

As a not-systems c++ programmer, it seems to reinforce the usual lesson: don't optimize your structures initially for anything but readability, once you have your system running and can pinpoint the bottlenecks, then try things like aligning structures to various things. But naturally always have a real-world- like test-suite to verify you are improving things.

Sorry if this is boring.

</comment><comment><USER8> Interesting. One question. How would I take advantage of this ? Are there special flags for the compiler or special memory allocator that I would need ? Do libraries like BLAS already account for this ?

<comment><USER9> I'm not aware of any flags of C compilers that would "fix potential cache- slot-collisions" for you automatically, neither of any allocators. Intel processors have some profiling registers that can point you to this kind of problems, but typically you first have to know what you want to profile.

Compilers are actually built to align the structures you write as there are a lot of processors which have significantly slower access to the misaligned values. Allocators also have to return you aligned addresses for each "malloc." The newest Intel iX processors are actually an exception in being able to amortize misaligned accesses.

I've actually used hand-made unaligned "string" stores. As soon as you allocate some bigger memory block and store the character sequences of the variable size one after another, their starts won't be aligned unless you want that. For doubles and other fixed-size values it's still better to keep them aligned.

Moreover I wasn't able to extract some value out of the article. Something constructed can be constructed to be slow or fast, fine. But I don't see anything that would inspire me to improve my real code. Maybe somebody who reads the article manages to produce such examples?

<comment><USER10> You can always write your own incremental allocator to provide word aligned (or whatever alignment) blocks of memory (from a large block of memory you obtain via a usual malloc() call) for the individual structures such that their page offsets are spread evenly (and avoid other problems such as spanning pages).

[EDIT] although it is tricky to do optimally given that different processors will have different cache set characteristics (as the article shows).

<comment><USER9> "Spreading" which will produce optimal cache use is something that depends on dynamic and not static properties of the program, so you'd have to "spread" differently depending on the use patterns. I can't imagine any universal solution.

And most of the programmers make much bigger omissions than those mentioned in this topic. Like using wrong algorithms, wrong libraries, doing too many allocations, having bad structures of the data... So this topic effects are invisible unless you already fixed other issues.

<comment><USER10> True, but having every structure aligned to exactly the same offset into each page is extremely unlikely. Given a page size of 4KB then a whole bunch of 32 byte structures aligned in such a way would represent a huge waste of memory. No sane allocator will allocate things like this.

If your structures happen to be very close to the system's page size it could easily happen, then you'd need to avoid this yourself with your own incremental allocator (or other tricks).

Definitely agree with your last point, I've seen lots of code (in commercial applications) where people are optimising completely the wrong thing.

</comment></comment></comment></comment></comment><comment><USER11> Hi Dan --

Great example. I looked briefly at the source, and wasn't sure whether "pointer_chase" was on or off in your graphs. Or maybe it didn't make a difference?

 _Page-aligned accesses like these also make compulsory misses worse, because prefetchers won't prefetch beyond a page boundary. But if you have enough data that you're aligning things to page boundaries, you probably can't do much about that anyway._

To the contrary, I think this is one of the relatively rare cases that explicit prefetching can help you. But maybe this helps only once your sets are too large for L3?

I wrote a little a few months ago on my attempts to speed up a Stream benchmark for Sandy Bridge that might have some overlap with your post: [http://software.intel.com/en- us/forums/topic/480004#comment-...](http://software.intel.com/en- us/forums/topic/480004#comment-1763753)

</comment><comment><USER12> > _Well, I've now managed to blog about three of the areas where I have the > biggest comparative advantage. Three or four more blog posts and I'll be > able to write myself straight out of my job. I must be moving up in the > world, because I was able to automate myself out of my first job with a > couple shell scripts and a bit of Ruby. At least this requires some human > intervention._

This was a great explanation of caching architecture, but I really want to hear the story of how the OP automated himself out of his first job with some shells scripts and Ruby.

</comment><comment><USER13> Many modern processors hash some of the address bits before using them as a cache index to avoid these problems.

To avoid them in code, it's often sufficient to round to convenient decimal numbers when allocating arrays, instead of powers-of-2. That is, allocate an array of 1000, instead of an array of 1024.

</comment><comment><USER14> The linked usenet discussion is worth reading, partially because a student asks for help on their homework, and it eventually pulls in Linus Torvalds: [https://groups.google.com/d/msg/comp.arch/_uecSnSEQc4/jkdcQc...](https://groups.google.com/d/msg/comp.arch/_uecSnSEQc4/jkdcQcRatXoJ)

The actual discussion is good, too.

</comment><comment><USER15> Why don't caches store data based on a linear hash of the address, rather than simply the low-order bits? This would retain the property that an aligned block of data can be stored without collision, and would extend this benefit to page-strided data. Even a "hash" as simple as (low order bits) XOR (middle bits) would provide this benefit.

</comment><comment><USER16> Why would you want to make an in-memory data structure page-aligned instead of cache-line-aligned?

<comment><USER17> It's not that you would want to. It's that certain data structures (particularly in kernel land) naturally form around powers-of-2 sizing. Since the hashing in an n-way associative cache is done by masking away bits, you can get into this nasty situation where multiple elements of your data structure end up hashing to the same location set (the N in an N-way associative cache). You don't want that if you are going to walk the elements in your structure.

Either avoid walking the elements, or do whatever it takes to ensure that you don't end up getting stuck behind the N.

Hardware is inescapable.

It's also worth noting that you can use this to your advantage, ensuring that accesses to certain elements does _not_ push much else out of the associative cache.

</comment></comment><comment><USER18> Can someone explain those charts?

<comment><USER19> The OP is plotting the _ratio_ of page-aligned vs. unaligned access time against problem size. So, on the y-axis, a value of 2 means the page-aligned access took twice as long as the unaligned access.

Took me a few moments to grok the charts too, since I haven't had my coffee yet.

<comment><USER20> Ah good explanation. The use of "vs" in a graph title makes me assume it's describing X vs Y, which is confusing.

It also helped when I realized the graph starts at 1 for working set size of 8 and the author corresponding says:

>Except for very small working sets (1-8), the unaligned version is noticeably faster

</comment></comment></comment><comment><USER21> Darn, really unexpected. Learn something new everyday. Page-aligned access is common because I/O is usually aligned on page. Now CPU intense cases are different because of cache line usage.</comment>


On Demand Food Service, Sprig, to Shut Down - <AUTHOR> https://www.theinformation.com/on-demand-food-service-sprig-to-shut-down <comment><USER2> I used it a decent amount in its first years -- the UI design was great, the food was fine.

Goes to show how difficult it is to scrape a profit on ready to eat food prep, especially when their differentiation with existing kitchens kept getting smaller with the growing popularity of Caviar et al.</comment>


Intel axes Nervana just two months after launch - <AUTHOR> https://fuse.wikichip.org/news/3270/intel-axes-nervana-just-two-months-after-launch/ <comment><USER2> Could it be they bought Habana for US$2B?

But this is very intel: get psyched about a technology, dump billions into it for a few years, flake out. See: cloud services ca. 1997 and IoT ca. every 3 years since 2013.

<comment> y-c-o-m-b > But this is very intel: get psyched about a technology, dump billions into > it for a few years, flake out

Hit the nail on the head! I worked for Intel nearly a decade ago. I was a full time employee for a few years. They saw Android and the Apple app store capturing mobile users and decided to dump $$$ into an app store for the PC (Intel AppUp). The effort lasted a few years before it crashed and burned HARD. During this time they also tried to spin up Meego/Moblin for mobile and tablet, then a partnership with Samsung to work on Tizen, and bought up McAffee.

The amount of money Intel flushes down the toilet with fruitless endeavors is sad, especially considering the number of people that end up jobless (like many of my good friends did) because of it.

<comment><USER3> I noticed this recurring theme. Seems like that eventually all companies go down this path at some point if they become mega rich. At that point money alone doesn't matter as much as capturing new markets do so lots of money goes down the drain. It's a good thing for the companies that get acquired and get their paycheck.

I see as rich people who buy expensive things and let them rot because they forgot they bought them or so.

<comment> eric-hu Sometimes bets like these turn out to be huge wins, like Microsoft going into the gaming platform space with the Xbox or Amazon building out cloud services. The Intel app store idea doesn't seem like such an idea to me, but I can see how the board could have been pushing for "outside the box" big bets.

<comment><USER4> I'd argue that Microsoft and Amazon's branches were at least tangential to what they were doing in the back-end and that they already have most of the required talents to do it right.

I have no idea how Intel could work their way into running an app store beyond "we have tech and market dudes, good enough right?"

<comment><USER5> With their new fabric/modular system, the modules are effectively "apps". Though possibly sold by app developers, you could install the Adobe module or the Autodesk module and it would have hardware/ram/storage tuned to those applications. It will start with DL and Graphics accelerators but then morph into application specific.

</comment></comment></comment></comment></comment></comment><comment><USER6> This is unbelievably dumb. Folks at NVIDIA HQ are no doubt breaking out their best champagne over this news: their Tensor Cores got a few more years of life thanks to this decision, and they retain the pricing power as the only provider of practical deep learning acceleration that actually works and has good tooling. I was looking forward to their TPU-like "N" offering in particular. Roughly the throughput of Google TPUv3, better silicon expertise, _and_ I'd be able to buy one. As a an applied scientist, I'd pay good money for 100TFLOPS of bfloat16 throughput, better yet if I can stick 4 of those things into each machine and train things over lunch that'd normally take a week to train. And it was pretty simple: just a systolic array with some extras to efficiently do convolutions, nothing super complicated, which means that unlike the more DSP-like approaches like Habana you don't have to do spend years writing and debugging a specialized compiler. You get to have this throughput _now_. Dumb, dumb move on Intel's part.

<comment><USER7> AMD support for PyTorch is getting pretty good from what I understand. They're going to become serious competition soon.

<comment><USER6> Just in time for NVIDIA to blow them out of the water completely on both the included RAM (which they're increasing) and performance (+40% if rumors are to be believed) with Ampere. AMD has a looong way to go.

<comment><USER7> AMD might still be able to offer competitive price/performance. Even if Nvidia is faster, I'm glad there's competition. For a while, Intel was really dominant, and they really stagnated.

</comment></comment></comment></comment><comment><USER8> I'd bet _most successful hardware advances in the ML /AI will come from companies that also push the field's edge in software_ Bc all the others would be at at disadvantage - their "better" hardware will mismatch the software, and that mismatch will increase development costs A LOT.

 _Google 's TPUs, Tesla's whatever those are etc. Bet on them!_

NOT on whatever Intel or IBM or AMD or Arm are doing this field. Heck, even Nvidia will probably start loosing at this game in the long run. The field is too dynamic and volatile, companies need to "eat their own dogfood" by integrating vertically hardware + software, before pushing hardware to sell to others too. This is not the old game of general purpose hardware anymore...

<comment><USER9> I think NVIDIA is in a good spot with its strong software ecosystem, and an army of devtechs. Their architecture is pretty general, so that if the ML algorithms change they adapt better than say TPU. I can't find the video of the talk anymore, but late in the process of TPUv3 IIRC Google had to rebalance their hardware to account for new algorithm(change)s.

Second, most companies that want to deploy AI cannot afford to build their own custom hardware. Even most carmakers partner with Mobileye or NVIDIA.

These big companies also have research teams whose job it is to stay on the ball, research and develop AI techniques and influence how the hardware has to change.

As for eating your own dog food, I think NVIDIA does just that with their autonomous driving software stack, robotics kit, devtechs for customers and library optimization/porting.

With sufficient focus, funding and execution I think AMD and Intel can reach a similar spot.

That said, my hunch is that the legacy and compatibility that the general purpose hardware makers have to carry forward will become a problem. But until that happens (I.e. specialized HW delivers better bang/$) in a 2-10 years, they will likely figure out how to alleviate that and/or develop more specialized hardware.

<comment><USER10> Seems that if Tesla was able to build its own custom hardware and have it working in 2019, other carmakers which are much larger and profitable would easily be able to do it, if they cared to.

<comment><USER11> Domain expertise is a finite resource and couple that with the fact the automotive manufacturers are not adept at higher level software development (as opposed to embedded)

I'm not surprised companies like Ford have continued to push out dates

</comment></comment></comment></comment><comment><USER12> Just a quick reminder when considering Intel products... if it ain't Mama x86, it won't be around very long.

<comment><USER13> Counterpoint: Intel SSDs have been going strong since at least 2008.

<comment> mark-r Define "strong". I've never actually seen one.

<comment><USER14> They exist on Newegg. You can buy one. I have one.

<comment><USER15> I have a few. For certain applications they're excellent.

<comment> mark-r For which applications would they be better than any other brand?

<comment><USER15> Datacentre-type applications where you need a relatively inexpensive server- grade drive.

For example, the P4610 is a great deal for a mixed-use drive.

</comment></comment></comment></comment></comment></comment></comment><comment><USER16> Not surprised: 100GBe interconnects vs proprietary interconnect. Complex architecture for which a hell of a lot work has to be done in the compiler (anyone remember Itanium?) vs something that's barely more complex than NUMA. Lower TDP per node is relevant for large scale datacenters.

Yes, Nervana looks much sexier, but I'd expect Habana to be more useful/usable overall.

<comment><USER6> Hell of a lot has to be done for Habana, too. And hell of a lot had to be done only for their edge SKU. My understanding is, their "N" SKU was basically TPUv3 for the masses - it's not even a question there was massive product- market fit if they integrated it into TF or PyTorch, it'd take off like a rocket.. Their "I" SKU was more along the lines of Habana: more Movidius-like, but scaled up. That thing (Movidius) has a specialized VLIW architecture (SHAVE) for which they had to write a compiler and roll it into LLVM, and if you read the architecture docs you'll see why. 12 VLIW cores, specialized memory arch, it's a bear to program for. Nobody will ever touch this shit with a 10 foot pole if they can avoid it. Systolic TPU-like stuff is more constrained with what it can do, but it's much easier to use and write drivers for (not to mention shrink the litography with). And it's also nowhere near its peak potential either, from either the throughput or energy efficiency standpoint. _And_ you can scale it down to edge as well, as Google has shown. It's just a better approach at least in the foreseeable future.

<comment><USER16> > Hell of a lot has to be done for Habana, too.

Yeah "barely more complex than NUMA" was a slight exaggeration and only meant in comparison to the other ;-) Judging by the architecurial differences layed out in the link, Habana is still hard-as-hell to develop a toolchain for; but Nervana is the kind of stuff that can drive people insane.

</comment></comment></comment><comment><USER17> I didn't even realize they launched it.

I thought it was vapor ware. I went to their party where they talk about it and it was underwhelming. They did not give any number comparison to Nvidia and such.

</comment><comment><USER18> Probably couldn't find customers for nervana, so they bought Habana...</comment>


Ask HN: Is there anywhere to have civil technical political discussions? - <AUTHOR> I'm not talking about Phd level, but not the comments section Fox,CNN either. I like the tone of this community but, obviously it's off topic. Even though it is one of the most complex systems on the planet and could use innovation. <comment><USER2> related "Ask HN: Is There a Hacker News Equivalent for Politics?" from 4 days ago [https://news.ycombinator.com/item?id=17825121](https://news.ycombinator.com/item?id=17825121)

<comment><AUTHOR> missed that one thanks.</comment></comment>


A Simple Tour of the Ruby MRI Source Code with Pat Shaughnessy - <AUTHOR> http://www.rubyinside.com/ruby-mri-code-walk-tour-6020.html

<comment> throwaway54-762 Someone want to add a [video] tag to the title?

</comment><comment><USER2> This is useful only for two kind of people:

\- Looking to contribute to Ruby MRI or

\- want to understand certain behavior

For rest of us, I don't see much value.

<comment><USER3> I'm definitely not going to be hacking on MRI for a while, but I still get a ton of enjoyment & education off of stuff like this.

Similar to the Doom III source code review (<http://fabiensanglard.net/doom3/>) - I'm not writing 3D game engines, but there's still lots of things that can be learned from how others build things.</comment></comment>


UFO hacker loses appeal - <AUTHOR> http://www.thesun.co.uk/sol/homepage/news/expats/expats_news/article1489619.ece <comment><USER2> It amazes me that someone who is clearly talented at cracking networks (so by inference we could say he's quite intelligent) would believe that they would just store that kind of information on publicly connected networks - if it exists at all that is.

<comment><USER3> How talented do you have to be to crack a system with no password? The really alarming thing about this article is the (apparent) insecurity of our military networks.

<comment><USER2> One system without a password? not too talented....

97 systems across 4 different government departmental networks?... whole different ballgame right there.

</comment></comment></comment><comment><USER4> I'm glad this happened, if only so that we will secure our systems better.

<comment><USER5> People like you, make me cry

</comment></comment><comment><USER6> "In one of these lives you are a successful computer programmer. In the other you are a hacker known as Neo...One of these lives has a future, Mr. Anderson, the other does not."</comment>


Magic, Art, and Scanimation (2014) [video] - <AUTHOR> https://www.youtube.com/watch?v=JLfLSjbm4s8 <comment><USER2> Amazing documentary, it is surprising what people did in the past and how far they experimented with everything to achieve clever results. Thanks for sharing this fascinating and thought provoking video.

<comment><USER3> I second this. It is an almost thirty minute video that may transport you into a world where imagination meets innovation meets persistence. It is truly magic to see the inventor's process. Thank you for posting this.

<comment><USER4> I like how you characterized the salient aspects of this artist's endeavor. And yes, thank you for sharing this video.

</comment></comment></comment><comment><USER5> If you love this stuff, be sure to check out the documentary "Film before film" (1985) by Werner Nekes (1944-2017) [https://www.youtube.com/watch?v=3nQl9C3CUcc](https://www.youtube.com/watch?v=3nQl9C3CUcc) and the other films in the six-part series Media Magica [http://wernernekes.de/00_shop1/index.php?p=productsMore&iPro...](http://wernernekes.de/00_shop1/index.php?p=productsMore&iProduct=7)

</comment><comment><USER6> Here I thought Scanimate was an analog video effects generator: [https://vimeo.com/172607783](https://vimeo.com/172607783)</comment>


Colorado's internet of roads is a go. Soon cars will "talk" to signs - <AUTHOR> https://coloradosun.com/2018/12/26/colorado-internet-roads-x-technology/ <comment><USER2> Coming soon in Colorado: Traffic speeding tickets issued using timing calculated from highway sign radio signals.</comment>


Ask HN:Review My Startup Social Genie - <AUTHOR>

An app that grants your wishes relying on your Social Network.
Any feedback would be appreciated.
http://apps.facebook.com/socialgenie/splash.aspx
Thanks <comment><USER2> You have posted about this several times over the past few days. I know you're excited about it and probably work on it a lot, but if you're going to be making "review my startup" threads every day, then it may just be better to make your own blog about the progress of your application. Broadcast new updates to your blog using Twitter or Facebook. It's probably not a coincidence that you're not getting much feedback here - people have already seen your older posts. If I recall, you even deleted the first one, which had some good advice in it, I think.

</comment><comment><AUTHOR> Link <http://apps.facebook.com/socialgenie/splash.aspx></comment>


Ask HN: Should you discuss issues with manager if you aren't going to stay? - <AUTHOR> Noob engineer, been a part of the workforce recently. I've decided that my issues are too large for me to stay here anymore. I will move out of this job in a couple of months. While my manager is a nice person, he is not good at management. For a future cordial relationship, should I bother being honest and discuss my issues, what I think are a lack of skills or interest as manager?. <comment><USER2> If your manager asks for feedback, you should definitely give it.

If you feel like your manager can grow as a result of your statement + will listen to what you have to say, give your feedback.

If your manager asks you why you are leaving, you should give him the truth.

If none of those is true, don't bother.</comment>


Show HN: Top level domain HTTP://ai/ - <AUTHOR> <comment><USER2> From the Show HN rules: [https://news.ycombinator.com/showhn.html](https://news.ycombinator.com/showhn.html)

> _What to Submit_

> _Show HN is for something you 've made that other people can play with. HN > users can try it out, give you feedback, and ask questions in the thread._

Unless you are the owner or made a significant part of the linked site, it's not a good candidate for a Show HN.

</comment><comment><USER3> I often mention this in discussions of regular expressions to validate domain names. (As far as I know, it's the only DNS A or MX record that contains only a single label.)</comment>


Fruity Maps – Apple Maps in Your Browser - <AUTHOR> http://fruitymaps.com <comment><USER2> I wonder if it'll become public like CloudKit.

For independent projects I'd still use/recommend OSM (using something like leaflet.js) but if you have Mac/iOS/web apps this would be a good solution for showing maps</comment>


Ask HN: How does comment weighting work - <AUTHOR> Does the algorithm only consider the upvotes on a particular comment, or does it also account for any highly rated child comments?<p>I.e. if a parent comment is not high-scoring but someone made a valuable reply to it, that valuable reply would nevertheless get buried. <comment><USER2> I think it's considered "secret sauce". I think it mixes the upvotes and time with other factors (user karma?, average?, manual weights?, ???) (I think they dropped "average" a few years ago.) The source code that is available is very old and don't have a lot of the small details. I think you will have to wait until kens does a black box analysis.

Related: "How Hacker News ranking really works: scoring, controversy, and penalties" (2013) [http://www.righto.com/2013/11/how-hacker-news-ranking- really...](http://www.righto.com/2013/11/how-hacker-news-ranking-really- works.html)</comment>


Cache tables - <AUTHOR> https://fgiesen.wordpress.com/2019/02/11/cache-tables/ <comment><USER2> I use this technique often and I love the simplicity it has. If you want to see a real-world usage, there is one in [https://github.com/antirez/dump1090](https://github.com/antirez/dump1090):

        /* Add the specified entry to the cache of recently seen ICAO addresses.
         * Note that we also add a timestamp so that we can make sure that the
         * entry is only valid for MODES_ICAO_CACHE_TTL seconds. */
        void addRecentlySeenICAOAddr(uint32_t addr) {
            uint32_t h = ICAOCacheHashAddress(addr);
            Modes.icao_cache[h*2] = addr;
            Modes.icao_cache[h*2+1] = (uint32_t) time(NULL);
        }

        /* Returns 1 if the specified ICAO address was seen in a DF format with
         * proper checksum (not xored with address) no more than * MODES_ICAO_CACHE_TTL
         * seconds ago. Otherwise returns 0. */
        int ICAOAddressWasRecentlySeen(uint32_t addr) {
            uint32_t h = ICAOCacheHashAddress(addr);
            uint32_t a = Modes.icao_cache[h*2];
            uint32_t t = Modes.icao_cache[h*2+1];

            return a && a == addr && time(NULL)-t <= MODES_ICAO_CACHE_TTL;
        }

As you can see adding a timestamp you can give a TTL to the cached entry validity without any additional effort: in that case you don't need to purge the hash table of old entries eventually, because the cache table is fixed size and the entires will get replaced.

<comment><USER3> I don't really see how this applies to the post, or am I missing something?

The linked article is about having multiple entries per open indexed hash entry, and sizing them to take advantage of cache line sizes.

Your code seems like it basically turns each open index entry into a struct of hash code and timestamp and then compares the timestamp on lookup. This just isn't clear because of how you store the timestamp in odd table offsets after shifting the bottom bit off the hash.

You could take your hash code / timestamp pair and use the articles cache line packing together really (at the cost of half the line).

<comment><USER2> I think that the main point of the blog post, is that instead of an exact match in O(N) worst time, you change the algorithm in order to replace the entry when adding, so that the lookup time also is always O(1) _and the size is bound_. The cacheline thing is just an optimization in order get the best out of the single cache miss you incurred. The table offset I use is the manual version of what you would do with a struct, no differences in the generated code more or less: the entry and the timestamp are stored at sequential addresses.

<comment><USER3> I can see that a little. It is a good article regardless what you drew from it. I found it more of a lossy set membership test that used packed cache lines to help in performance (over chaining), but I don't really know what his use case was. Good write up none the less.

</comment></comment></comment></comment><comment><USER4> The comparison between the classes of "Las Vegas" and "Monte Carlo" algorithms is a useful distinction, along with the author's suggestion of using a cache table as a lookup table for compression match finding.

If anyone is interested, I wrote @ronomon/hash-table as a fast, scalable cuckoo hash table for Node.js.

It has high load factor, constant lookup time in the worst-case and uses a non-recursive variant of the cuckoo probing scheme, together with a bloom filter to reduce cache misses to the alternative cuckoo bucket by an order of magnitude.

Last but not least, caching is supported according to the CLOCK LRU algorithm, which is elegant and effective.

Benchmarks and design decisions are in the README:

[https://github.com/ronomon/hash-table](https://github.com/ronomon/hash-table)

</comment><comment><USER5> If the number of searches for each key isn't uniformly random, then instead of random replacement, you can bubble sort the bucket values into an LRU eviction scheme. Each time you do a lookup that succceds and the element in the hash entry isn't index 0, swap it with index - 1. Replacement should always be done on the last element in the bucket vector.

This has 2 advantages:

\- scanning the bucket aggregates to theta(1) if your searches follow the Pareto law

\- popular keys are less likely to be evicted

Because your buckets are more relevant, you can also make them smaller, reducing the k time in an O(nk) complexity over n searches and decreasing the amount of cache thrashing.

<comment><USER4> If I understand you correctly, CLOCK LRU works like this, except without the memcpy swapping cost.

[https://en.wikipedia.org/wiki/Page_replacement_algorithm#Clo...](https://en.wikipedia.org/wiki/Page_replacement_algorithm#Clock)

<comment><USER5> In order for the scan to be most efficient in cache, you need to store key- reference pairs in the buckets, so the swap cost should only be a couple instructions rather than a memcpy.

</comment></comment></comment><comment><USER3> I've written a couple cache-aware hashes for work (fintech), including an always correct version of this that rehashed when buckets filled. I always call them multi-buckets for lack of a better term.

Most data structures that rely on a chain of items can be improved by bucketing for cache lines (even linked lists).

I've also implemented 32 byte multi-buckets for a cuckoo hash scheme to try and balance search time and memory usages (you can get extremely high load factors using this approach). I don't think this approach is any better than a few other similar multiple-hashing schemes though.

I have tried keeping the hashs together in memory and using SSE compare instructions to do the comparison/mask of the entire bucket in a couple instructions, but have found the technique hit/miss.</comment>


I commissioned an oil painting of Barbra Streisand's cloned dogs - <AUTHOR> https://simonwillison.net/2019/Mar/7/oil-painting/ <comment><AUTHOR> I used Instapainting (YC W11) for the commission.

<comment><USER2> >I used Instapainting (YC W11) for the commission.

How much did the painting cost?

And (if you know it) how much of the total sum went to the actual artist that painted it?

<comment><AUTHOR> Around $140 - and I don't think Instapainting have shared that information.

<comment><USER2> Thanks.

Ir sounds like a very "fair" price (to the customer), whenever there are these kinds of "middlemen" the question is always how much of that goes to the actual people that actually do the work.</comment></comment></comment></comment>


Ask HN: What's my best option to get into the IT field with my BA degree? - <AUTHOR> I was considered a non-traditional college student when I started for my BA in Computer Information Systems when I was 25 years old. I graduated this past December (2018) at the age of 30.
Since then I have been actively applying for jobs in the IT field such as "Software Developer, IT Help Desk, Desktop Support, Programmer" etc. Some of them are mostly the same thing just different titles.
As of right now (May 2019) I have applied to at minimum 50 different jobs in the surrounding cities near me. The two largest cities near me are an hour away in opposite directions. I live in a relatively small town with no computing industry at all so the other two larger cities are my best bet.
Unfortunately, just about all of the 50 applications have been rejected due to lack of professional experience. No one seems to care about my personal experience even though I have it listed on my resume. I'm becoming frustrated because I have to start paying my student loan back in a couple months.
I'm married with kids, my wife has a good career in the next county over, and we own our home so I cannot relocate. I don't have a lot of free time to due to my part-time job to pay some small bills while my wife covers the rest.
We don't have a lot of free money so after looking at the cost of study materials and exam fees for certifications I have decided I'll have to wait to find a better job to save up some money.
As a side note one thing I've noticed is that not a lot of companies are posting "Entry-Level" positions for people like me to apply to. Most of the positions assume you already have some professional experience.
This all being said (sorry it's so long), what advice do you have to help me figure out how to get started? <comment><USER2> I would steer clear of "IT", you need to get some projects on a blog or fill in your GitHub with things that aren't necessarily over the top complex but somewhat novel and creative.

Employers see you as a risk because they aren't sure how much effort it's going to take to get you to a point where you can not only write code but understand how to work as an engineer in a larger org and be extensible.

</comment><comment><USER3> One possibility is that, because there are a lot of people with various certifications and degrees that cannot, in fact, actually program, they are suspicious of your lack of experience. They may have been burned before by people who supposedly had training, but cannot program.

So, make a software project, that you can show. Best if it is something you can actually use somehow, like a database-backed website for your personal collection of something, or your hobby, or what-have-you. Something real, so that you encounter real-life tradeoffs and design decisions, that you can talk about when it comes up in an interview. But, not so big that you can't actually finish it in your available time.

As a side benefit, this will keep your skills from eroding with the passing of time since Dec 2018, which is also good, both in an of itself, and also to help you answer interview questions.

<comment><AUTHOR> Thank you for your feedback. I agree I should work on some side projects.

</comment></comment><comment><USER4> Contribute to open source.

Create a project and post it to github to show your skills.

Get certifications in particular technologies [https://www.pluralsight.com/blog/software- development/best-d...](https://www.pluralsight.com/blog/software- development/best-developer-certifications)

Do jobs on upwork to build a resume and examples

Apply on sites like Trilobyte that give you tests

<comment><AUTHOR> Could you provide a URL for Trilobyte? I tried searching but only got results for a gaming company.

<comment><USER5> Triplebyte

</comment></comment></comment><comment><USER6> List of remote work platforms:

[https://docs.google.com/spreadsheets/d/1JfNAbUX_lN9K3MCNHO15...](https://docs.google.com/spreadsheets/d/1JfNAbUX_lN9K3MCNHO15GJtJ5qpk7H9Cl3xTBwv2FR8/htmlview)

</comment><comment><USER7> You'll find my email address on my profile, reach out to me. I've mentored 4 people in your same position into jobs (and I was in your position a few years ago). Totally free, just paying it forward.

<comment><AUTHOR> Noted. Thank you very much.</comment></comment>


Show HN: acewidget - widget to make ace/bespin even easier to embed - <AUTHOR> http://jeromeetienne.github.com/acewidget/

<comment><AUTHOR> This is a little tool i did this weekend for another project of mine. It may be usefull to quickly embed ace if your project without worrying about namespace collissions.</comment>