Will calling "free" or "delete" in C/C++ release the memory to the system? - <AUTHOR> https://lemire.me/blog/2020/03/03/calling-free-or-delete/ <comment><USER2> There are usually very sound reasons not to release memory back to the OS, particularly in a multithreaded program. Each such release causes a "TLB shootdown", in which threads on other cores are blocked while the cores' "translation lookaside buffers", caches of page mappings, are cleared, and further stalls as their entries are re-filled.

This is another reason to prefer single-threaded processes, which are less subject to such shootdowns, and less-coupled forms of parallelism.

Besides the TLB potholes, releasing memory means that the next time memory is requested, the OS is obliged to zero it before the process gets to see it again. Furthermore, each page will be marked read-only, causing a trap the first time it is touched, and then zeroed lazily.

As a result, freeing memory to the OS should only be essayed with the support of a great deal of measurement of the consequences.

</comment><comment><USER3> Always appreciate your posts. Perhaps good to explain the sbrk/mmap threshold for this and maybe also why the memset() is required to fault in pages :-)

</comment><comment><USER4> Interesting. It calls to mind garbage collection, actually, where the "collection" of memory happens at some indeterminate time in the future based on heuristics about how much there is. Of course it doesn't have the same performance impact, probably because the collection process consists of just freeing known chunks instead of iteratively searching for chunks that can be freed.

<comment><USER5> GC doesn't search for chunks that can be freed. Also "iteratively" - what do you mean by this?

There is no known algorithm for GC that searches for garbage to delete, the way GCs work is exactly opposite - find all live objects (reachable from roots - registers,stack, statics/globals) and the rest of the heap by definition is garbage.

<comment><USER4> [https://en.m.wikipedia.org/wiki/Tracing_garbage_collection](https://en.m.wikipedia.org/wiki/Tracing_garbage_collection)

> tracing garbage collection is a form of automatic memory management that > consists of determining which objects should be deallocated ("garbage > collected") by tracing which objects are reachable by a chain of references > from certain "root" objects, and considering the rest as "garbage" and > collecting them

So it starts at the root and, in a sense, "iterates" down through the reference tree to see which allocations are reachable, and then the inverse of that is freed. In the OP case, by contrast, "which allocations to free" is known at "collection" time, but there's still an extra step of actually doing the collection.

<comment><USER5> The point is that tracing GC searches for things to keep and assumes the rest is free.

Which I believe is the important distinction, and quite often it doesn't know or care "which allocations to free" because it does it in bulk. That allows it to be efficient.

<comment><USER4> No, the important distinction is that typically GC iterates over _something_ (incurring a cost of O(N)), whereas the C++ case iterates over _nothing_ (incurring a cost of O(1)).

But my original point was just that it's interesting that there _is_ an extra step at all in the C++ case, where we tend to think of collection as happening "for free". It has a constant-time overhead, which is not the same as zero overhead.

<comment><USER5> > important distinction is that typically GC iterates over something > (incurring a cost of O(N))

During _collection_ and N is size of live set, so divided by allocation the cost is amortized and can safely considered as O(1) much like appending a dynamic array where costly O(N) resize is amortized over N appends.

Typically GCs allocate via bump a pointer allocation which is actually faster then what pretty much all of libc malloc implementations would do.

> C++ case iterates over nothing (incurring a cost of O(1)).

Not quite, take a look at jemalloc paper for instance:

[https://www.facebook.com/notes/facebook- engineering/scalable...](https://www.facebook.com/notes/facebook- engineering/scalable-memory-allocation-using-jemalloc/480222803919/)

A lookup to get the metadata for this specific allocation could be anything from O(1) to O(lgN) depending on size and malloc strategy.

</comment></comment></comment></comment></comment></comment><comment> Iwan-Zotow You could trim memory as much as you want, but next malloc call will result in (a lot of) syscalls, and they are NOT free and NOT cheap

</comment><comment><USER6> All else being equal, could the efficiency of malloc_trim depend on heap fragmentation? Could the heap end up with "gaps" of free'd memory that couldn't be returned to the OS independently?

<comment><USER7> Not an expert in the matter, but: In my simple mental model of how memory management works under the hood (fixed-size memory pages, and a virtual- to physical-address indirection through a page table), only "clean" pages could be returned to the OS which don't have any allocations left in them. So fragmentation might prevent a 'mostly clean' page from being returned to the OS because there's a single tiny allocation left which pins it into the process' address space.

However: Most (all?) general memory allocators have small-allocation buckets for minimizing this effect, which basically groups allocations of specific sizes into different memory pages.

Despite that, virtual address space fragmentation in 32-bit applications is definitely a thing. Memory might still become fragmented over time so that "big" allocations (bigger than a memory page) are failing because there's no big-enough gap left in the processes virtual address space, despite having enough free physical memory available for mapping.

This usually isn't a problem in 64-bit processes, because there's always enough room at the "front" (new memory allocations would basically move like a tidal wave through the 64-bit address space, leaving a mess of fragmented address space behind).

The best way to prevent memory fragmentation of course is to minimize allocation frequency in long-running applications (ideally only allocate memory when the program starts up).

</comment></comment><comment><USER8> I'm guessing this is because the runtime keeps a few pages around for free- lists and the like? Interesting that there's a hump in the middle there.

</comment><comment><USER9> TLDR: no more so than standard mallocs, all of which for obvious reasons cache allocated pages.</comment>


What is a coder's worst nightmare? - <AUTHOR> https://www.quora.com/What-is-a-coders-worst-nightmare/answer/Mick-Stute?share=1 <comment><USER2> Understanding unfamiliar legacy code with a bunch of kludges and "domain knowledge" or "business logic" baked into it, when the original authors are uninterested/unavailable and management has _zero_ respect or understanding for the fact that it often takes an order of magnitude longer to modify/maintain this stuff than it does to write it.

Oh, and there's no test suite.

That is my nightmare. I've been there a few times and it can be literally impossible to shine in that role.

"What do you mean, it will take you ten hours to add a button? They coded this whole module in one week and there are 100 buttons!"

aarrrrgh

Now, as an engineer, I understand and embrace the challenge of explaining this sort of thing to management. I get it. That's the type of skill that separates code monkeys from senior developers and architects. But, no matter how good you are, you can't _force_ management to listen nor understand.

<comment><USER3> If you are on a legacy project, it's hard to know what to prioritise in terms of improving code. However, getting requests to take somewhere within an order of magnitude of how long people think it _should_ take is usually my guide. I never tell people I'm going to refactor something. I just do it. Then after the fact I say, "Remember when this used to take a week? Now it takes an hour". Once you do that a couple of times, management gets the point (usually). The only problem I've come across is when you get on a team where they actively discourage code churn. On a legacy project, that's death, so it's best to find another job in that case.

I really enjoy working on legacy code. Usually everybody else leaves me alone because nobody wants to touch that code. I get a tonne of freedom. As long as I don't break things (requires a fair amount of experience) I can quietly improve things under the hood -- targeting areas where we get a lot of requests. Legacy code is also great because usually nobody wants new features just for the hell of it. You almost always get real requests from real users that have real pain points. The biggest downside (as you imply) is that it's hard to shine because the best you can do is make things acceptable. It's difficult to hype the legacy project so you don't get a lot of recognition for your work.

<comment><USER2> I liked reading this! I'm glad you found happiness doing this sort of work!

One thing I'd say is that it's awfully tough to do this sort of work as a contractor when there's a (direct) hourly cost for your time. It's tough when you're on salary working on an internal project... but doubly tough (not impossible) when you're a contractor! Theoretically, it would be in clients' best interests to improve efficiencies, but it's _really_ hard to get a client to sign off on hundreds or many thousands of dollars of work for a future gain that's hard for them to grasp.

(Getting them to grasp it is part of our job, of course)

       Then after the fact I say, "Remember when this used to take a week? Now it takes an hour". Once you do that a couple of times, management gets the point (usually)

This is great advice. It's all about those metrics and demonstrating the value of your work!

<comment><USER4> Please don't use codeblocks for quoting. This quote is unreadable on most screens (esp. on mobile) because the line doesn't break.

Just put a > in front of a normal paragraph to indicate a quote.

<comment><USER5> I stopped leaving comments like this because I encountered some surprisingly vitriolic resistance to making comments readable.

<comment><USER2> Yep, I've been yelled at for both ways. Not putting any effort toward it at this point.

</comment></comment></comment></comment></comment></comment><comment><USER6> > _I suddenly realize it 's in the compiler. It was the compiler. And every > time you compile the original code and run it puts in the subliminal message > code into the source code. I'd heard of this before._

Ken Thompson, "Reflections on trusting trust", 1984. [https://dl.acm.org/citation.cfm?id=358210](https://dl.acm.org/citation.cfm?id=358210)

<comment><USER7> However, I imagine that Ken Thompson hardly had in mind somem low-brow text manipulation hacks that go as far as to clobber the textual _inputs_ to the compiler.

</comment></comment><comment><USER8> Coders worst nightmare is working at a company that could desperately use good technology but whos management completely devalues it. The existentialist feeling of expending hard mental labor on something that no one cares about

</comment><comment><USER9> Great story, but I don't think it qualifies as a nightmare. It's just a really interesting and challenging puzzle. The fact that the author elected to work on it for free ("This is nerd war") clearly places it in that category. If you can exit a nightmare, you exit the nightmare. Speaking for myself the only coder's nightmare I have is the one I think every coder has. It's the universal programmer nightmare: you don't really _know_ your stuff works, the way you can know that turning a key in a lock will open a door. Maybe you missed something. Maybe you didn't test something. Maybe it's going to break over the coming holiday weekend.

<comment><USER10> For some reason, I don't find that story believable at all. It sounds like somebody read "Reflections on Trusting Trust" and decided to exercise their creative writing muscle by passing it off as their own experience.

<comment><USER11> Yeah it's just fan fiction.

</comment></comment></comment><comment><USER12> My worst nightmare is I publish a paper, post the code on github, and then someone contacts me pointing to a bug in my code that invalidates the main result of the paper.

<comment><USER13> > invalidates the main result of the paper.

Something like 30% of papers can't be reproduced, regardless of bugs.

<comment><USER12> I'm fine with that as long as those are not my papers! :)

</comment></comment></comment><comment><USER14> Being described as "_my_ developer" by a manager or designer.

It says a lot about how these people see themselves within a team, and it's the kind of team I don't want to work in.

<comment><USER15> Being called a coder.

It's one of my pet peeves.

<comment><USER16> I understand that negative feeling. Coming from a non technical person or role, what wording would be better ? Developer ? Software architect (its different but let's assume non technical people don't understand the difference), code monkey ? What would be the best term we can find for our profession?

<comment><USER17> You have multiple options, this is how to choose one:

1\. "software engineer" (might drop the "software" for specific roles like "front-end engineer" or "machine learning engineer" etc.) - _use this for people who value their education /fancy-degree and/or who pride on code- quality, low bug/defects rate, reliability etc. and they'll love it!_

2\. "software developer" \- _use this for people priding on craftmanship and "ability to ship the right stuff on time" no-respect-for degrees etc. and they'll love it!_

2b. feel free to mix up (1) and (2) and nothing bad will happen :) also using (1) or (2) instead of others will tend not offend, just show that the person using it is clueless a bit.

3\. "programmer" \- _this is the most direct / no-bullshit term for people who hate fancy extra words and pride on practicality and getting stuff done_ (but it might offend ppl with fancy degrees or make them feel devalued - never use it for a PhD unless you know beforehand he/she would approve)

4\. "software architect" (this is a dangerous one) - _can be used to boost the self-confidence of smart but terribly insecure people_ (use it sparingly bc some might be pushed tot he other extreme and end up with over inflated egos bc of this... better add "senior" before "engineer", it's more honest, what we call "architecture" in software is closer to "structural engineering planning" or smth, very far off from what building architects do) ...also practical senior no-bullshit people might be offended if someone junior to them gets this title!

5\. "code monkey" ...the '99 bubble is over man, like 20 years ago, forget about this, unless you're in SV or other hip place and some club/group uses it for nostalgic-value _...otherwise it 's offensive, and God have mercy on your soul if you mistakenly use it in a place where ppl are unused to it and at the same time to refer to someone with non-white skin color!_

6\. "coder" \- might be acceptable in some places, but it's also cunningly devaluing (see sister comment for details)... unless you have the malevolent intent of "placing a hint of their personal irrelevance in someone's mind while at the same time avoiding to offend them" please _avoid it!_ (might be the worst of all actually, bc nobody can complain but at the same time you spray them with a depressing hint of their inferiority while mildly boosting your superiority and subtly lowering everyone's morale at the same time... and then as a manager you end up asking yourself in the end why it all turned to shit despite you being "such an awesome and nice guy")

<comment><USER18> Ugh, code monkey my boss is heavy into that lingua

<comment><USER16> This one was definitely tong in cheek, as a clear not to use example of course.

</comment></comment></comment></comment></comment></comment><comment><USER19> When you realize your code has caused grave injury to someone, physically, cost their life, ruined them financially or ruined a relationship.

Everyone pays attention to plane or space craft crash. But bad software in cars, air bags, trains, medical equipment have cost lives. Bad software at financial institution have occasionally ruined people financially. Bad social media software have ruined relationships. Bad localhost software that has crashed and taken data could ruin people in ways many people can't imagine.

Someone out there is using something as simple as a text editor to write their favorite book or keep track of their finances and run their businesses. if the editor screws up their data, you can't imagine the hardship.

You never know how your users will use your software, don't take it for granted.

</comment><comment><USER20> Deleting the production database on your first day:

[https://www.reddit.com/r/cscareerquestions/comments/6ez8ag/a...](https://www.reddit.com/r/cscareerquestions/comments/6ez8ag/accidentally_destroyed_production_database_on/)

Obligatory (poor guy): [https://github.com/MrMEEE/bumblebee-Old-and- abbandoned/commi...](https://github.com/MrMEEE/bumblebee-Old-and- abbandoned/commit/a047be85247755cdbe0acce6f1dafc8beb84f2ac)

<comment><USER21> "Today was my first day on the job as a Junior Software Developer..."

And he has permissions to delete the production database? Heads much higher up the tree need to roll.

<comment><USER22> This was the general consensous on r/sysadmin when the post made it over there.

If you can destroy a buisness on your first day, or even severly hobble it, you have larger issues than a green dev.

</comment></comment></comment><comment><USER23> Surprised to see no one has mentioned 3rd party APIs yet. Especially half- baked, poorly documented SAAS APis where you're left wondering whether is was designed by malice - to protect some vague business interest - or simply by incompetence / lack of care / lack of time.

I've found the best mental defence in such situations is to assume the worst from the outset, by expecting to have to burn untold hours figuring out how this black box _really_ works and then be pleasantly surprised some parts actually work OK. Better that than assuming "this can't be that that hard" and ending up frustrated.

<comment><USER24> Exactly, also a frequent API changes for no reason. Tensorflow is a good example of this. Looks like every month they schedule a meeting on what they gonna break next. They seem to enjoy thousands of github issues about cryptic messages on renamed nodes in the neural network models.

<comment><USER25> Struggling through their newly released tensorflow-graphics library right now. They wrote it, published some simple, single google-colab tutorials and left it at that. No documentation anywhere else

</comment></comment></comment><comment><USER7> What is the basis for the belief that there was a single person, who wrote the program for the psychologist, and perpetrated the compiler hack?

This could be a case that person B was innocently working on the curses-based questionnaire program for the psychologist. Person A was aware of this work and hacked the compiler to recognize and doctor that program (including squashing it down to one line), in addition to /sbin/login and possibly others.

I'm curious how the program managed to include various headers, like <curses.h>. That requires multiple lines. I'm thinking that the hacked compiler took the _preprocessed_ original innocent program and then output it as one line not requiring any preprocessing directives. Not even necessarily to obfuscate it, but because the division into lines was gone at the point in the compiler where this was done; i.e. this was a manipulation of the token stream of the program, and the tokens were just converted back to text and output as one line.

Could even be that this was all done in the C preprocessor rather than the compiler proper, by recognition and replacement of token sequences. Though modern compilers integrate the preprocessing phases of the language, this sounds like it was in a classic Unix environment, in which cpp was still a separate program.

<comment><USER26> It is strange to think that someone with this level of skills was in a graduate program in psychology (as opposed to working in industry as an engineer).

<comment><USER7> Better that than making another NewLisp, I suppose.

</comment></comment></comment><comment><USER27> Spending ages debugging javascript with alert boxes, publishing the final code, and then hearing that the homepage has an alert box that says "hi" on it.

<comment><USER28> Not "hi" it usually says "this should never happen" :)

</comment></comment><comment><USER29> Reminds me of this story: [https://www.teamten.com/lawrence/writings/coding- machines/](https://www.teamten.com/lawrence/writings/coding-machines/)

</comment><comment><USER30> Realizing that "coder" is largely a pejorative term and most people have no idea and appreciation for what you do.

<comment><USER31> Is this a regional thing? I haven't ever perceived it that way over the internet or here in Europe, but I've seen several comments who agree with you on HN.

<comment><USER32> I'm in NA and also don't understand why coder is negative. I remember it was very common in the old demoscene days. I've never even seen someone non- technical use the term. Usually non-technical people use programmer.

The one that irks me is engineer. I've met very few developers who have the sort of professionalism that the term engineer requires. Most developers are more like cowboys whose primary goals are making other developers and themselves happy. An engineer needs to instead feel responsible for serving the users of the application, which includes maintenance by others as a subgoal. No doubt my definition of engineering is not popular here.

</comment></comment></comment><comment><USER33> Carpal tunnel or losing ability to use my arms/hands.

<comment><USER34> Yeah, my first thought was of the guy who blogged the whole time an inoperable brain tumor was killing him. Over the course of multiple years I watched as it robbed him of his intellect. The cleverness faded. The subtlety faded. His posts started getting shorter, and then started getting sloppier. His grasp of grammar was failing. Shortly before he died his final post was "love you kids. love you wife."

That's my worst nightmare.

<comment><USER35> I'm not sure there's objectively worse things than that.

But I think one could make a good argument his final post indicates he had people who probably cared for him during his deterioriation, in part because love for them had been a north star for him.

Maybe suffering such an end without that would be worse.

</comment></comment></comment><comment><USER36> Mine: Being forced into management. Which is happening.

<comment><USER37> It happened to me about a year ago. I hated it at first but began to like some parts of it eventually. I'll never like the "paperwork" and the chores navigating inscrutable internal applications and processes, but having real influence over the direction of products is very satisfying.

</comment></comment><comment><USER38> Data loss. Behavioral bugs can be fixed, but if you hose your customers' data, it's gone forever.

</comment><comment><USER39> Having a project transfered to you that is "nearly finished"

</comment><comment><USER40> Low iq MBA manager with insecurities charged with delivery. Usually vindictive and on a power trip, and has social leverage on some senior exec.

<comment><USER41> I'm living that right now, except he has no MBA, is a consultant, has the dogged determination of an elderly employee on his last ever career project and used his leverage to bring in a bunch of unqualified architect yes-men.

</comment></comment><comment><USER42> Bugs that you can reproduce perfectly in production but not at all in any of 20+ non-production environments is my usual one... and it usually winds up being some updated Oracle binary...

<comment><USER43> Better or worse than a bug that appears once every few months in Prod, but only when it's raining at the customer's location, and never on Sundays?

(This is an actual issue I've encountered. It had to do with code scraping weather data for a horse racetrack, specifically if the page was updated while the scraper ran. It was buried in a PHP monolith with no tests. The track didn't run races on Sunday.)

</comment></comment><comment><USER44> Working as a programmer for Amazon (at least that was my experience)

<comment><USER45> Strange thing to hear. Can you expand a bit?

<comment><USER44> Search for "Median tenure at Amazon" (they say that is 13 months)

</comment></comment></comment><comment><USER46> Code reviews complaining that your code isn't cargo-cult enough.

<comment><USER47> I would add 'unforgiving use of linters' to that, where 'unforgiving' generally means that straying even a hair's breadth from the default settings is forbidden.

Especially so when you can't adjust those defaults, thus making your post- listed code substantially more abstract and less readable than it would have been at first.

Case in point: forcing your simple 11-line function to become something more complicated that has to maintain state through each call.

<comment><USER48> Been there. Other dev on my team INSISTED on limiting to 80 char lines in a codebase with many long strings. It just wasn't appropriate.

</comment></comment></comment><comment><USER49> How about infringing on a patent for inventing something independently without knowing it?

</comment><comment><USER50> Being asked to "improve" a piece of internal software with not a single hint of what actually needed doing. After two meetings where nothing more concrete than "make it better" came up, I quietly continued working on actually achievable stuff until I left a few months later. When I checked up on my colleagues a few years later the damn thing was still in limbo.

</comment><comment><USER51> Writing a stupid bug in a critical system that causes people to get injured or killed.

</comment><comment><USER52> The rest of the answers to this question can be found here, for the curious:

[https://www.quora.com/What-is-a-coders-worst- nightmare/](https://www.quora.com/What-is-a-coders-worst-nightmare/)

</comment><comment><USER53> Obtaining poor requirements, then being accountable for maintaing the feature and codebase having had little influence on decisions, especially if the people behind the requirements have now left the organisation.

</comment><comment><USER54> >What is a coder's worst nightmare?

Hired by a company and realize afterwards that you must work on a project that's designed and built on potatoes using potato languages and potato tools

</comment><comment><USER55> My personal nightmare:

Working in a company where I have only one monitor and a PC without SSDs. And I am not allowed to bring my own mouse/keyboard. And I don't have admin rights for the OS.

<comment> Const-me I totally agree with the general idea. Especially the mouse.

However, since I've bought my 27" 4k monitor I pretty much stopped using extra ones. Unless I have to for something very specific, like debugging an app which output to multiple monitors.

</comment></comment><comment><USER56> Interesting hacking task and free time is a nightmare?

</comment><comment><USER57> This story of course makes obligatory a link to Reflections on Trusting Trust, Ken Thompson's Turing award lecture, which first popularized this concept.

[https://www.archive.ece.cmu.edu/~ganger/712.fall02/papers/p7...](https://www.archive.ece.cmu.edu/~ganger/712.fall02/papers/p761-thompson.pdf)

</comment><comment><USER58> Having to refactor the IRS.

[https://youtu.be/qL5ut8o5pfs](https://youtu.be/qL5ut8o5pfs)

</comment><comment><USER59> A manager who engages in circular reasoning is a nightmare that never ends.

</comment><comment><USER60> Going viral on the internet for an embarrassing situation .. if that ever happens I will finally realize my dream of becoming a hermit in the wilderness

</comment><comment><USER61> Changing jobs is very stressful for me. I have put up with a lot of BS to avoid that stress, can seem to get better but worse actually.

</comment><comment><USER62> Mine is being drugged and hit with a $5 wrench in order to get me to give up my password

[https://xkcd.com/538/](https://xkcd.com/538/)

<comment><USER63> This plus TSA demanding passwords to corporate laptop. Mountains of confidential client data compromised...or in custody. hmm...which to pick

</comment></comment><comment><USER64> Deploying something on friday and figuring it out its broken production monday morning.

</comment><comment><USER65> We're always finding something tainted about memories taken for granted. A fine story.

</comment><comment><USER66> Ah, this old chestnut. Has anyone analysed the cycle time for stories on HN?

</comment><comment><USER67> Being forced into a management job with no direction or mentoring.

</comment><comment><USER68> "What crash? It is working fine on my machine."

</comment><comment><USER69> "Hey X, mind if I push this quick hotfix to live?"

</comment><comment><USER70> I will loose my job because of AI epidemic :D

</comment><comment><USER71> Asked to reverse binary tree.

</comment><comment><USER72> PHP, period

</comment><comment><USER73> obfuscated Perl?

</comment><comment><USER74> Stuff that doesn't work, leading to days of problem solving.</comment>


Bullet Hole Statistics - <AUTHOR> http://jada.ms/bullet-hole-statistics

<comment><USER2> Neither strategy is sufficient. Putting armor on areas that had statistically more bullet holes based on examining planes that survived, is obviously not a good strategy as explained in the article. But neither is the strategy of armoring up all the areas that did not have bullet holes. May be not all of those areas are critical. You just don't know unless you examine the planes that did not survive.

In the absence of data on the planes that did not survive, you just have to follow sound structural/mechanical/electrical/mathematical (flight trajectory) principles to determine which of the exposed surface areas of the plane are likely to be hit under various trajectory fly overs over anti-aircraft gun batteries, and among those areas , which are critical (engine, pilot cockpit, weapon storage, fuel etc), and armor those areas up.</comment>


How to beat the CAP theorem - <AUTHOR> http://nathanmarz.com/blog/how-to-beat-the-cap-theorem.html <comment><AUTHOR> How is this different than Event Sourcing? CR is basically Command Query segregation.</comment>


Google Says Android, Chrome OS Will Eventually Converge - <AUTHOR> http://www.tomsguide.com/us/Google-Android-Chrome-OS-Chromebook-Chromebox,news-15400.html

<comment><USER2> Anyone that's surprised has not been paying attention.

<comment><USER3> I've been following both projects closely, and I'm very surprised. They're two very different project, and I don't see how they possibly could merge.

ChromeOS is open-source through and through, and is based on the web as a platform. Android is tied to a Google account, is mostly but not fully open- source, and has a very different eco-system. It's API-driven.

Will we be seeing web intents and Android intents merge? Will Dalvik be rewritten on a web technology? Will Java be used at all in this new merged platform?

I can't even begin to imagine how these could merge... User interfaces, maybe, but not the underlying technologies. But maybe that's what is meant by this line: "in time there will be a seamless user experience across all the devices"

<comment><USER4> Is Chromium OS up to date with Chrome OS? Their site was too too quite...

<comment><USER3> As far as I know it's the latest source. It doesn't have everything from Chrome OS though, for instance plugins and the auto-updater [1]. Browsing the commits I see an update from 8 minutes ago [2].

[1] <http://www.chromium.org/chromium-os/chromium-os-faq>

[2] <http://git.chromium.org/gitweb/?p=chromium.git;a=summary></comment></comment></comment></comment>


Holograms, 3-D said to be on verge of new era - <AUTHOR> http://www.bostonglobe.com/business/technology/2012/06/24/mit-hosts-conference-focused-future-holograms-art-entertainment/m5ypbtQBRYxMdNcVRth2tN/story.html

<comment><AUTHOR> I personally think this is pretty exciting and part of the trend towards very real interactivity in our lifetime.

<comment><USER2> How unfortunate, then, that the article doesn't actually describe any progress or innovations.</comment></comment>


AES-based Synthetic IDs: deterministic AE for 64-bit integers - <AUTHOR> https://github.com/iqlusioninc/aes-sid <comment><USER2> Similar ideas go back quite a ways. Here's notes from a Perl library based the Skipjack cipher, this wound up inspiring some of my own work and I wound up porting it to node.js.

> One example where Crypt::Skip32 has been useful: You have numeric database > record ids which increment sequentially. You would like to use them in URLs, > but you don't want to make it obvious how many X's you have in the database > by putting the ids directly in the URLs.

> You can use Crypt::Skip32 to scramble ids and put the resulting 32-bit value > in URLs (perhaps as 8 hex digits or some other shorter encoding). When a > user requests a URL, you can unscramble the id to retrieve the object from > the database.

[https://metacpan.org/pod/release/ESH/Crypt- Skip32-0.17/lib/C...](https://metacpan.org/pod/release/ESH/Crypt- Skip32-0.17/lib/Crypt/Skip32.pm)

</comment><comment><USER3> I'm curious what properties this has that aes(0||id) doesn't.

<comment><USER4> It's reversible. That's the whole point. You don't need to keep a separate table or column for the hashed id.

<comment><USER5> Isn't AES encryption also reversible? The use-case for AES-SID appears to be mapping 64-bit ID values to 128-bit ciphertexts and reversibly producing an authenticated 64-bit ID value from the 128-bit ciphertext. Zero padded AES does both of these things as far as I can tell.

</comment></comment></comment><comment><USER6> i see tptacek is stalking this thread so i'm going to post a semi-off topic question.

with AES-GCM is it safe to generate nonces randomly? i see some recommendations that after a certain number of encryption operations you should rotate keys.

i haven't really looked at AES GCM SIV but looking at the description of the algorithm on this github page it doesn't look like it fixes 'misuse' if your misuse is generating random nonces and generating huge numbers of ciphertexts.

i see AWS has a KMS solution that encrypts stuff server side using AES-GCM that don't mention anything about IV generation and only support rotating keys once a year. is it safe to assume this scheme is safe if you are encrypting lots of ciphertexts with a single key?

<comment><USER7> Search for the birthday paradox. Basically if you assume an attacker that can store every messages, as used nonces accumulate the chance of a collision increases. It does not remain constant, as naive intuition would suggest.

The maximum suggested number of uses per key can be calculated based on the desired security bound (chance of a collision with past messages) and the size of the nonce/IV.

If you want a rule of thumb, (N/2)-1 the number of bits in the nonce is not too bad. So if you have 64-bit nonces, you should send no more than 2^31 messages.

This only applies to random nonces. If you are incrementing the nonce sequentially, you get as many nonces as the range of your integer. But here be dragons: if you lose track, you will _definitely_ repeat nonces! This is why schemes that save and restore the nonce value are dangerous. If you lose power and restore an old value, all the nonces from then until the latest real position will repeat.

Usually it's safe to use simple incrementing nonces with ephemeral keys that live only as long as a session and a specific instance of the app/OS/whatever. You can restart the nonce with a new key no problem. Just be sure your increment is protected by a mutex or something like std::atomic<> if there are multiple threads or two threads could send the same nonce.

<comment><USER6> yer. i've always used N/2 as a rule of thumb and checked if that is a big enough number. i saw this quote from NIST on crypto exchange in regards to AES-GCM. i would usually assume 96 bit -> 48 bit would be enough but they have what I think is some pessimistic advice:

[https://crypto.stackexchange.com/questions/42982/safety- of-r...](https://crypto.stackexchange.com/questions/42982/safety-of-random- nonce-with-aes-gcm)

>> AES-GCM takes a 96-bit nonce and NIST says that you can only encrypt 2^32 messages under a single key if using random nonces. This is because if you throw 2^32 balls at 2^96 buckets then you have roughly a 2^-33 chance of getting two in the same bucket and NIST drew the line there. That probability might seem either high or low to you. It's pretty small in absolute terms and, unlike a work factor, an attacker can't spend resources against it, but it's a very long way from the safety margins that we usually use in cryptography.

i notice libsodium uses xchacha20 which has a larger nonce which is 192 bits size. and in there docs they say:

>> Unlike plain ChaCha20, the nonce is 192 bits long, so that generating a random nonce for every message is safe.

Not sure if they are referring to both libsodium ChaCha20 implementations. One of them is obviously very unsafe with only 64 bit nonce, but one is 96 bit the same as AES-GCM.

<comment><USER8> > i would usually assume 96 bit -> 48 bit would be enough but they have what I > think is some pessimistic advice:

The "You have n Bits, then you need 2^n/2 tries for a birthday attack, so your real 'security level' is n/2 bits" rule captures the expected value of tries to get a collision. That's the 50 % point. You don't want to design your crypto system in a way that results in a 50 % chance of at least one nonce repetition.

</comment></comment></comment></comment><comment><USER9> This is an interesting approach for primary key obfuscation.

I recently stumbled upon something that I feel might be even more powerful - Hashing complex types into 256-bit keys. E.g. If you had some type representing the composite keys of a Customer in your system (email address, phone number, etc.), you could serialize this instance, run it through SHA256, and as long as the same process is used for lookups, you can get everything back out as expected.

Essentially, you can compress your entire scope of composite key data into a single 256 bit value. Just like with GUID keys, this can be pre-computed on each client (whereas autoincrement cannot). This approach is very clever IMO because it can be used directly on top of any universal byte[]/byte[] key- value store. Your keys are all 256 bit values corresponding to the SHA256 of a serialized complex key instance. The type information can be encoded into the key itself (e.g. hash the fully-qualified type name as well).

<comment><USER10> If you use an unkeyed hash (as opposed to a PRF) on low-entropy inputs, they can be preimaged by an attacker.

This is especially problematic in the case of PII like email address/phone number</comment></comment>


Ask HN: Considering a bootcamp. Anyone familiar with Bitmaker? - <AUTHOR> I've been self-teaching myself to code for a little while now and I feel like I need something more to make myself employable in the industry. The best code bootcamp near me seems to be Bitmaker in Toronto.<p>Does anyone have any experience with then or their alumni? <comment><USER2> I've known a few people involved in Bitmaker.

We had someone apply who completed Bitmaker, and she applied for a development role. I work in developer relations. She was initially rejected for a developer position because she wasn't experienced enough. She caught my eye though, and I hired her on as a developer relations intern. She quickly became my team's dedicated developer, and built us a lot of stuff to make our lives easier. I think she was really green coming out of Bitmaker, but we were able to provide her an environment where she could get acquainted with working in a larger organization and get some real world experience coding professionally while not in a developer role. She quickly became incredible, and moved off my team to be a junior developer. She's now a senior developer after a few years, and is absolutely killing it here. I'm super proud of her, and thrilled we took a chance on her.

Sample size of one, I know. And she really is exceptional, so I'm sure she would have been successful with or without Bitmaker, so take this with a large heap of salt.</comment>


On Jack's One Year Anniversary, Twitter Remains a Mess - <AUTHOR> https://www.wired.com/2016/10/jacks-one-year-anniversary-twitter-still-mess/ <comment> at-fates-hands Why does every story now, regardless of media outlet, have to attach race and politics to everything?

Does it really matter if Louie CK is a white guy who quit Twitter?

Does it matter why or how Donald Trump uses Twitter when Hilary Clinton does and says the same stupid things he does?

Why did the article single out Milo Yiannopoulos (conservative) as someone who got banned and then praise how Black Lives Matter is using the platform?

I really thought some of the industry media I consumed would be above bringing race and politics into every story, but it's every where now and really disconcerting.

It's true. . . .journalism is dead.

<comment><USER2> There's a great anecdote in Victor Klemperer's diary of life as a Jew in Nazi Germany. He was a longtime subscriber to a cat magazine. In 1932 his cat magazine was all about cats. By 1935 it couldn't shut up about the virtues of the "German Cat."

Not that our society is Nazi -- it's pretty much the opposite. Nonetheless, the feeling of everything becoming poisoned by politics is inescapable. If you prefer you can go over to Breitbart and read about how Twitter is a mess because it doesn't respect free speech and banned Milo. Somehow I don't think this will leave you feeling any better.</comment></comment>


AirDropper Requests Files via Email and Saves to Your Dropbox - <AUTHOR> http://lifehacker.com/5615351/airdropper-requests-files-via-email-and-saves-to-your-dropbox?skyline=true&s=i

<comment><USER2> We made a FAQ page for AirDropper based on all the feedback everyone provided earlier in the week. (<http://news.ycombinator.com/item?id=1613574>) You really helped us start to understand which benefits were important to emphasize. We'll definitely be keeping the feedback in mind as we work to improve AirDropper.

The FAQ is available here: <http://www.airdropper.com/faq>

Thanks everyone!

<comment><USER3> From the FAQ: _AirDropper is best suited for professionals who need to make sending files securely easy for the sender, including designers, lawyers, accountants, or consultants._

I don't know if it came up in the earlier discussion, but these types of people are likely to have several clients and projects going on at the same time. Rather than dump everything into a common AirDropper folder, would probably be useful organizationally if the person requesting the file could indicate a subfolder somehow, so when the client uploaded the file it would get dropped into that subfolder within the parent AirDropper folder.

Of course as soon as you implement that, people might want support for an infinite hierarchy, but I'd think one level of subfolders would cover most needs.

<comment><USER2> Yeah, that's a big issue. You can actually specify a subfolder when you create the request. It's the last field on the page. We probably need to make that more obvious.

Thanks!</comment></comment></comment>


Unit 731 - <AUTHOR> https://en.wikipedia.org/wiki/Unit_731 <comment><USER2> My first exposure to this was a fictional book I read on a whim. I looked into it and came up with the actual history; evil stuff.</comment>


AI to escape conference calls - <AUTHOR> http://money.cnn.com/2016/05/19/technology/watson-conference-call/index.html?iid=SF_LN <comment><USER2> Fairly certain that this could be a commercial product!</comment>


Why economic crisis is affecting women more than men - <AUTHOR> https://www.theguardian.com/business/2020/aug/04/shecession-coronavirus-pandemic-economic-fallout-women <comment><USER2> The titles of posts in HN are so terrible: poor grammar, poor spelling, poor wording. But you would think a _newspaper_ would get things right. It should be " _the_ economic crisis" not simply "economic crisis".</comment>


Freedom of Information Rallies Tomorrow (Sat Jan 15) - <AUTHOR> http://www.whyweprotest.net/en/

<comment><USER2> Not going to lie, pretty surprised to see this reach the front page. Seems like astroturfing to me, but whatever.

Good luck guys, keep fighting the good fight.</comment>


BTC mining pool just reached 50% global mining power - <AUTHOR> https://blockchain.info/pools?timespan=24hrs

<comment><USER2> Why does this matter?

One of the most notorious threats to blockchain-powered networks is allowing a malicious actor to obtain control over more than half of the network's computing resources. This situation is commonly referred to as the elusive "fifty-one percent attack." It is difficult, but not impossible to pull off —[http://motherboard.vice.com/blog/bitcoins-fatal-flaw-was- nea...](http://motherboard.vice.com/blog/bitcoins-fatal-flaw-was-nearly- exposed) — the aggregation of this kind of hashing power. And it looks like we've nearly made it there again, but there is a difference between a pool getting > 50% of resources, and an "attack."

A successful attack using this method would allow the attacker to exclude, or "orphan" any new blocks from the valid chain causing all newly-minted coins go to the attacker. He may also execute a "double-spend" and reverse any of his own transactions during the window of time that his sham blockchain is considered authoritative by the network.

A successful 51% attack on Feathercoin — [http://www.coindesk.com/feathercoin- hit-by-massive-attack/](http://www.coindesk.com/feathercoin-hit-by-massive- attack/) — was stopped in its tracks by the network's natural uptick in difficulty in response to an increase in network hash rate. The Feathercoin attacker was likely executing a price pump in parallel to compromising the network, which caused coin-switching pools to mine feathercoin and increase the difficulty to a level that stopped the attack.

</comment><comment> sampson-san Interesting data point, thanks! If all the compute power that together forms the GHash pool were controlled by a single entity, this would be an issue. Thankfully that's not the case.

<comment><AUTHOR> Please see my comment above. For all practical purposes, GHash pool owner controls the whole hashing power - he becomes the owner of each mined block and can do whatever he want's with it without users' consent.

</comment></comment><comment><USER3> What % of HN knows what this means? Maybe this should be a blog post instead and explain what a "mining pool" is. Or what is the significance of one pool taking over the market.

<comment> sampson-san Anyone familiar with Bitcoin is likely aware of the 51% attack. A blog post is hardly warranted, because this isn't significant. GHash.io does not control all of that compute power. It is a metalayer that coordinates distinct and unrelated compute resources controlled by (supposedly) 180,000 seperate entities[0]. In order for a 50% attack to take place the vast majority (at this point) would have to agree to attack, and that simply won't happen.

[0] GHash.io homepage

<comment><AUTHOR> I don't think you're right. The mining machines are pointed towards Ghash pool owner and he becomes the owner of the mined block - that's how he can distribute the BTC to everyone who contributed. So in practice, he controls the whole hashing power and can perform the attack without users' consent.

</comment></comment></comment><comment><USER4> as a long time proponent of BTC, this is the first time I'm truly worried. Not that I think GH will do anything 'dodgy', just that it's _actually happened_ and that's not a good thing at all.

GH just did a great job, I think when other competitors step up and provide as good a service - this will become a distant memory of the past - a blip on the radar (I hope).. There'll be a handful of decent pools, thereby never letting this happen again. (I hope again)

</comment><comment><USER5> Bitcoin was supposed to be a decentralized system, not one with a centralized benevolent authority. In GHash We Trust...

<comment><USER6> Well, nobody is forcing anyone to participate in the pool. People can just switch to other pools. But here again greed will win, because those who participate in biggest pool can expect most reward.

</comment></comment><comment><USER7> and how much of the "unknown" is owned by GHash ...</comment>


China Warns World Leaders: Don't Talk About the Hong Kong Protests - <AUTHOR> https://news.vice.com/en_us/article/d3njzv/china-warns-world-leaders-dont-talk-about-the-hong-kong-protests <comment><USER2> Wait, is China putting world leaders under a gag order? Under what authority?

Are there mechanisms by which they are allowed to do this (stipulated in bilateral or international agreements of any type?). Or is this just the height of arrogance and delusion on the CCP side?

I don't think even Putin goes to these extents to quash dissent.

I can even understand internal (domestic censorship, it makes some sense from their perspective), but this is attempting to control what sovereigns of other nations may say... What?

<comment><USER3> Money and power drives leaders mad. It's not delusional if you can retaliate against people who don't toe the party line. Just look at the US destroying whole countries for not upholding "freedom and democracy" mythology... well that's different you say... but is it, really?

<comment><USER4> > Money and power drives leaders mad. It's not delusional if you can retaliate > against people who don't tow the party line. Just look at the US destroying > whole countries for not uploading "freedom and democracy" mythology... well > that's different you say... but is it, really?

What do you mean by, "Uploading the 'freedom and democracy' mythology?"

<comment><USER2> It's referring to thd Middle East meddling we got involved during Bush and to a somewhat lesser extent Obama presidencies (though North AfricanNear East and Middle East Arab spring was a debacle as well).

</comment></comment></comment></comment><comment><USER5> Does the Streisand effect work with politicians?

China already would censor anything related to it, so isn't this simply China trying to see who will fall in line?

Re: Tiananmen Square Massacre

Secretary of State Mike Pompeo issued a statement commemorating the 30th anniversary of the Tiananmen Square massacre - [https://www.newsweek.com/tiananmen-square-massacre-china- say...](https://www.newsweek.com/tiananmen-square-massacre-china-says-us- violating-international-law-honoring-30th-anniversary-1442014)

China's response - [http://www.china- embassy.org/eng/zmgxss/t1669295.htm](http://www.china- embassy.org/eng/zmgxss/t1669295.htm) -

"On June 3, 2019, the US side issued a statement on China in the name of Secretary of State Mike Pompeo out of prejudice and arrogance. Under the pretext of human rights, the statement grossly intervenes in China's internal affairs, attacks its system, and smears its domestic and foreign policies. This is an affront to the Chinese people and a serious violation of international law and basic norms governing international relations. The Chinese side expresses its strong dissatisfaction and firm opposition to it.

The Chinese government and people reached the verdict on the political incident of the late 1980s long ago. Over the past four decades of reform and opening-up, China has enjoyed rapid economic and social development, continuous progress in democracy and the rule of law, flourishing culture and significantly improved standards of living. China's human rights are in the best period ever. Socialism with Chinese characteristics, a choice of history and the people, has been proved a right path in line with China's national conditions and supported by the whole population. China is firmly committed to the path of peaceful development and to the development of human rights in China and the whole world, and has made significant contribution to the international human rights governance. This is a fact that is recognized by all unbiased people.

The Chinese people have the best say on China. Their pursuit of a better life cannot be stopped by any force. Whoever attempt to patronize and bully the Chinese people in any name, or preach a "clash of civilizations" to resist the trend of times will never succeed. They will only end up in the ash heap of history."

Talk about our past and we'll leave you in an ash heap? Sounds like they didn't learn anything in whatever "verdict" the government and people supposedly came to.

<comment><USER6> From the totalitarian perspective, ultimately the whole globe, and the inside of any mind, is an "internal affair".

That they talk of "resisting the trend of the times" is just par of the course:

> _More specific in totalitarian propaganda, however, than direct threats and > crimes against individuals is the use of indirect, veiled, and menacing > hints against all who will not heed its teachings and, later, mass murder > perpetrated on "guilty" and "innocent" alike. People are threatened by > Communist propaganda with missing the train of history, with remaining > hopelessly behind their time, with spending their lives uselessly, just as > they were threatened by the Nazis with living against the eternal laws of > nature and life, with an irreparable and mysterious deterioration of their > blood._

[..]

> _The scientificality of totalitarian propaganda is characterized by its > almost exclusive insistence on scientific prophecy as distinguished from the > more old-fashioned appeal to the past. Nowhere does the ideological origin, > of socialism in one instance and racism in the other, show more clearly than > when their spokesmen pretend that they have discovered the hidden forces > that will bring them good fortune in the chain of fatality._

[..]

> _Totalitarian propaganda raised ideological scientificality and its > technique of making statements in the form of predictions to a height of > efficiency of method and absurdity of content because, demagogically > speaking, there is hardly a better way to avoid discussion than by releasing > an argument from the control of the present and by saying that only the > future can reveal its merits. However, totalitarian ideologies did not > invent this procedure, and were not the only ones to use it. Scientificality > of mass propaganda has indeed been so universally employed in modern > politics that it has been interpreted as a more general sign of that > obsession with science which has characterized the Western world since the > rise of mathematics and physics in the sixteenth century; thus > totalitarianism appears to be only the last stage in a process during which > "science has become an idol that will magically cure the evils of existence > and transform the nature of man." And there was, indeed, an early connection > between scientificality and the rise of the masses. The "collectivism" of > masses was welcomed by those who hoped for the appearance of "natural laws > of historical development" which would eliminate the unpredictability of the > individual's actions and behavior. There has been cited the example of > Enfantin who could already "see the time approaching when the 'art of moving > the masses' will be so perfectly developed that the painter, the musician, > and the poet-will possess the power to please and to move with the same > certainty as the mathematician solves a geometrical problem or the chemist > analyses any substance," and it has been concluded that modern propaganda > was born then and there._

\-- Hannah Arendt, "The Origins of Totalitarianism"

Same book, but I'm note sure if this passage comes before or after the above:

> _[The method of infallible prediction] is foolproof only after the movements > have seized power. Then all debate about the truth or falsity of a > totalitarian dictator's prediction is as weird as arguing with a potential > murderer about whether his future victim is dead or alive – since by killing > the person in question the murderer can promptly provide proof of the > correctness of his statement. The only valid argument under such conditions > is promptly to rescue the person whose death is predicted. Before mass > leaders seize the power to fit reality to their lies, their propaganda is > marked by its extreme contempt for facts as such, for in their opinion fact > depends entirely on the power of man who can fabricate it. The assertion > that the Moscow subway is the only one in the world is a lie only so long as > the Bolsheviks have not the power to destroy all the others. In other words, > the method of infallible prediction, more than any other totalitarian > propaganda device, betrays its ultimate goal of world conquest, since only > in a world completely under his control could the totalitarian ruler > possibly realize all his lies and make true all his prophecies._

same thing for this quote (since this discussion was flagged like a brick anyway I don't feel bad for taking up that much space -- if it's an additional incentive for even just one person to read that book, it's worth it)

> _We don 't know a perfected totalitarian power structure, because it would > require the control of the whole planet. But we know enough about the the > still preliminary experiments of total organization to realize that the very > well possible perfection of this apparatus would get rid of human agency in > the sense as we know it. To act would turn out to be superfluous for people > living together, when all people have become an example of their species, > when all doing has become an acceleration of the movement mechanism of > history or nature following a set pattern, and all deeds have become the > execution of death sentences which history and nature have given anyway._

edit:

> _The really frightening thing about totalitarianism is not that it commits > 'atrocities' but that it attacks the concept of objective truth; it claims > to control the past as well as the future._

and

> _Already history has in a sense ceased to exist, ie. there is no such thing > as a history of our own times which could be universally accepted, and the > exact sciences are endangered as soon as military necessity ceases to keep > people up to the mark. Hitler can say that the Jews started the war, and if > he survives that will become official history. He can't say that two and two > are five, because for the purposes of, say, ballistics they have to make > four. But if the sort of world that I am afraid of arrives, a world of two > or three great superstates which are unable to conquer one another, two and > two could become five if the fuhrer wished it. That, so far as I can see, is > the direction in which we are actually moving, though, of course, the > process is reversible._

\-- George Orwell

If you got this far, read this speech, it's really good, and I want this ray of hope to be your takeaway more than anything:

[https://news.ycombinator.com/item?id=20262265](https://news.ycombinator.com/item?id=20262265)

<comment><USER5> Thank you for sharing all of this.

</comment></comment></comment><comment><USER7> and I ask again, how are American tech giants defending their production in this country. Very quick to toot their human rights horns yet show no shame in being in China.

<comment><USER8> this

No issue bashing collaboration with DoD, but damning silence when it's the business in China.

</comment></comment><comment><USER9> First thought when reading this headline is what is Trump about to say

<comment><USER3> Whatever you wanna say about Trump, in the end silencing dissent all over the world is not something good. It's good that someone stands up to this before we can't do that anymore.

<comment><USER10> I was thinking something similar. Trump might not be the brightest person, but he sure can't keep his mouth shut... Which is exactly what we need in this instance.</comment></comment></comment>


Show HN: Listify – Turn a group of lines into an HTML list - <AUTHOR> http://enrmarc.github.io/listify/

<comment><AUTHOR> I'm the author. I've been doing several demo web sites during the last month and one of the things I did a lot is turn paragraphs or something else into lists. I have not found anything to do that with some kind of flexible customization (adding classes and incremental ids, nested elements, etc.)

I know that it's only a few lines of JavaScript but some of my colleagues found it useful, so I have decided to publish here.

All feedback about the code or the grammar (English is not my first language) would be greatly appreciated.</comment>