Ask HN: How have you handled existential crisis in your life? - <AUTHOR> From time to time, more often than I would like, I have a full blown existential crisis.  It can be triggered by something as simple as a YouTube video, suggested to me by YouTube, that is a cancer patient talking about their final days.
That's what it was today.
Given we're (largely) engineers and science minded folks here, I ask HN:  How do you handle deep existential questions?
More specifically, here are some questions I really struggle with:
-  How do you deal with the fact that we don't know why anything at all exists?  Why, for instance, do atoms, subatomic particles, etc, exist AT ALL?  Why not NOTHING?
-  How do you deal with the fact that your life is meaningless.  You will die and - at best - leave behind a legacy for history books.  More likely, you will die and it will be completely insignificant.  To put it another way:  How do you deal with the fact that your life is MEANINGLESS?
-  How do you deal with the likelihood that you will suffer greatly at some point in the future, whether it be from cancer, or some other disease?  How does a day of 9-5 work, capped off with a horrible prolonged cancer treatment at the end of your life, not corrupt your mind? <comment><USER2> Nobody is going to hand you meaning on a silver platter, and anyone that claims to have all the answers is lying to you. You have to find your own meaning in life, and that means you have to choose what matters and choose what you spend your time on. There's nothing inherently meaningful in being alive, true, but our nature as thinking beings lies in giving meaning to things that are meaningless.

If you don't have a purpose, find a purpose for yourself. Everyone is insignificant, so why try to be significant to everyone? Just be significant to your friends. Show up in the world. And do things that matter to you.

Forget about the unanswerable questions. They're unanswerable, and they'll only trip you up. Go be around people, and let them remind you why you're alive. Go whistle in the dark for a while.

</comment><comment><USER3> \- How do you deal with the fact that we don't know why anything at all exists? Why, for instance, do atoms, subatomic particles, etc, exist AT ALL? Why not NOTHING?

A: Realize we're all made of the same "stuff" what ever that is... and we'll never know "why", but it matters not, as it has no effect upon our lives if this is merely a hallucination or sim, ignore it, enjoy it, or not, it's always your choice.

\- How do you deal with the fact that your life is meaningless. You will die and - at best - leave behind a legacy for history books. More likely, you will die and it will be completely insignificant. To put it another way: How do you deal with the fact that your life is MEANINGLESS?

A: If you seek a lasting public legacy, I feel for you, as darn few achieve that. But your life is meaningless only if you remain selfish and don't give back to society, your community, and friends, seek out ways to help others and share your knowledge, your time is a gift worth giving.

\- How do you deal with the likelihood that you will suffer greatly at some point in the future, whether it be from cancer, or some other disease? How does a day of 9-5 work, capped off with a horrible prolonged cancer treatment at the end of your life, not corrupt your mind?

A: I never concerned myself with that possibility, until age 60 when I awoke in a hospital, and lucky to be alive, then I went through a year of hellish pain and depression. What "saved" me was honestly asking myself what would truly make me happier, it doesn't have to be a material object, but in my case it meant getting rid of my car and getting another motorbike as my only transportation.

I ride in all weather, and will ride until I don't have the strength to hold my bike up anymore and it's huge, it saved my life.

Whatever you come up with, I wish you the best in figuring it out.

</comment><comment><USER4> When my body dies I believe I will go to heaven and continue the journey I am currently on.

Like yourself I have my existential moments. I have been kept awake at night thinking about the many events which take place in the universe at any one point in time.

My understanding of existence, my own significance and life's purpose come from the teachings of Jesus. There is a book in the Bible's Old Testament, Ecclesiastes, which asks the very same questions you are asking. You may find it interesting to read. [https://www.biblegateway.com/passage/?search=Ecclesiastes+1&...](https://www.biblegateway.com/passage/?search=Ecclesiastes+1&version=NLT)

Mark's Gospel in the New Testament is the shortest of the Gospels and you may find it interesting to read as well if you are interested in a perspective that life is not meaningless. [https://www.biblegateway.com/passage/?search=Mark+1&version=...](https://www.biblegateway.com/passage/?search=Mark+1&version=NLT)

I am an old HNer in my 40s and have seen a number of people close to me die. I was with a friend while he died of HIV and lost another good friend to suicide. I said goodbye to my Father several years ago. It is good you are asking these questions now as many don't.

</comment><comment><USER5> It's interesting to consider another question: what kind of rational "meaning" we would be ready to accept as an explanation? For example: somebody tells you that we were created by a higher being, or civilization, to serve some _rational_ purpose for them. E.g., they use us as "food". Or they need us to solve some other practical problem in their civilization (not unlike the tools we are area creating for ourselves) - like, help them migrate across the Universe (because they are too small to create the necessary tools by themselves). Would you accept this as a decent meaning? Would your existential problems become easier to bear based on this knowledge?

</comment><comment><USER6> Remember, it's also an existential opportunity. Granted, most of us seem to rely on Professional Sports Leagues to create meaning and significance out of pure air, but you have the chance to find something that both helps others and is highly enjoyable to you by wrestling with some of that angst.

Bertrand Russell's answer was that the more you involved yourself in helping young people and future generations, and shifted meaning beyond your singular mortal self (Conquest of Happiness, book) by doing that, the happier you'd be. (Since things obviously aren't going to end well for any of us, as individuals.)

It is a way of fooling yourself (because heat death of the universe) but ya get to help others and enjoy yourself at the same time. Win-win. A kid's laugh is pure enjoyment, worth adding to the world by and despite any logic. My version of Utilitarianism holds children laughing as the highest good worth working for. Kid's enjoyment is totally real.

Beyond that, meaning or purpose ain't much: hammers have a purpose 'cause they're mere tools. Slaves can have a purpose and meaning for their (continued) existence, similarly. But that's not a good thing. A free human being doesn't come with a purpose 'cause we're the thing that confers meaning on other objects. That's a good thing.

As the wise old ones say: "Enjoy the show."

Or join the Mormons. Or get hired by SpaceX. Meaning can be a contact high.

</comment><comment><USER7> I don't know you, but based purely on what you wrote here, it sounds like you could knock out half of your troubles by simply _not_ being an engineer working a 9-5. :-)

There are infinite possibilities for life. Would you choose to sit at a desk all day, if you had to do it all again? Or is that just what you happened to get from picking the default option, every step of the way?

As to the unanswerable questions, we've invented philosophy and art and religion and meditation and others to help tackle those. Don't expect to find answers in engineering or science. They're great but that's not what they're for.

I've found _my_ answers to these questions, but if they were the sort of questions which could be answered in a few words, we'd just have a "MEANING OF EXISTENCE" pamphlet to give to everyone in first grade. We can't even write a single document on how to write good software, and that is much more concrete and focused.

I'm in the middle of making a big career change myself. There's no secret trick to finding what you're meant to do. You just have to keep trying new things until you find it. Go for it.

</comment><comment><USER8> > How do you deal with the fact that we don't know why anything at all exists?

I'm occasionally troubled by this late at night (often after too much to drink). At some point you just have to accept "it is what it is" and move on to less troubling thoughts.

> How do you deal with the fact that your life is meaningless.

Meaningfulness is a feeling/emotion, probably unique to humans. I don't think my dog wakes up wondering what his purpose is for instance. As an emotion, it's easiest to feel like you are doing something meaningful by helping your fellow humans in some way. You will feel good about it, even if it ultimately doesn't matter during the heat death of the universe.

> How do you deal with the likelihood that you will suffer greatly at some > point in the future.

I take solace in the fact that I live in state with a "death with dignity act" if the disease is terminal. Not that you couldn't take things into your own hands if need be.

</comment><comment><USER9> > we don't know why anything exists Yeah, neat. Maybe one day we will figure > it out. Maybe it is an accident. Maybe it is on purpose. Maybe we don't get > to know.

> life is meaningless Depends on your definitions. Will I be remembered when > the sun burns out? Nope. But I can and do enjoy existing. I enjoy my family > and their existing. I can improve the existence of others. There is meaning > in that at the micro level, if not the macro level.

> you will suffer greatly It is the journey, not the destination. I enjoy my > existing now. I might not enjoy it at the end. My grandma enjoyed over 90 > years and had a rough last 4-6 months. Who knows what the future holds, but > I hope for it to keep getting better for me and those I care about.

I really like this: > "If you have a problem that can be fixed, then there is no use in worrying. If you have a problem that cannot be fixed, then there is no use in worrying." – Buddhist proverb

I pretty much live by that. I don't worry. I just do what I can and don't do what I can't. This is not to say I don't have plans or dreams. I can influence those, so I do.

My life has gone from hard to very blessed. It could become hard again. I'll worry about that if that is the case again.

</comment><comment><USER10> I think I only really have an "answer" (in that I think I know how I would handle this personally) for the last one. The second one doesn't really bother me at all (when I realize the second one I actually just feel kind of free, at least _most_ of the time), but the first one haunts me on an intermittent basis. I don't have the answer to that one.

> \- How do you deal with the likelihood that you will suffer greatly at some > point in the future, whether it be from cancer, or some other disease? How > does a day of 9-5 work, capped off with a horrible prolonged cancer > treatment at the end of your life, not corrupt your mind?

When I feel that the great suffering has begun and there is nothing more to look forward to (hopefully when I'm very old, but can be at any point), I will opt for assisted suicide. I don't think I'm afraid of death, but of dying in pain. If I have the opportunity to make that choice, I plan on doing it. On the other hand I do not _want_ to die in general and am also open to trying whatever methods of life extension are available to me as long as the suffering does not outweigh the state of being alive.

</comment><comment><USER11> Rest assured you're not the only one with such thoughts/questions. Here are my 2 cents...

As you correctly state, most of us being engineers, we have to accept some facts, initial conditions and axioms for the system/life we're living in and unfortunately work with these constraints. \- Just accept that the universe around us exists. If it's real, a simulation, a dream or whatever else suits you is irrelevant. \- That's the difficult part, some (most?) humans understand our place in the universe and how meaningless our existence is in the vast time scales of the cosmos. Try to live your life everyday. \- We're just like any other animal or complex machine or a PC. Eventually we'll "break down" and will suffer or be unfortunate. Our habits, environment, genes, whatever other factor, affects this but at the end of the day we'll all die.

Try not to overthink it, try to keep your mind occupied with other (more relaxing/fun) activities, educate yourself so as to better understand the system you're part of and spent some quality time with other humans, otherwise you might fall into one of the many mental medical conditions.

Just keep tight, we're all into this.

</comment><comment><USER12> The meaning of life is survival. Perhaps you lack eudaimonic happiness. Introduce struggle towards goals that are meaningful to you. If nothing is meaningful, you have depression. If only a very small number of things are meaningful, struggle at those. Meaning is built by you, no one can give it to you and it doesn't happen by default once you graduate from extrinsic rewards.

<comment><AUTHOR> > If nothing is meaningful, you have depression

I almost certainly do have depression then. Not the kind where I am suicidal, or lack the ability to get out of bed... but rather, the kind where I would rather not exist because I do not enjoy life.

I struggle very much with meaning, obviously. That's part of this existential crisis... Nothing is meaningful. We are all going to die and I ask what will it mean then? What will some service or app that I made mean when I am dead?

Has anyone yet lived for some purpose, some greater meaning? It appears not, and I am not special.

<comment><USER13> Imagine tomorrow was your last day. Would you have any regrets?

I personally ask myself that whenever I'm not happy and it usually straightens me out. It's not an easy path but if you want to find meaning in your life, it's a good question to ask.

Most people find meaning through the things they care about which tends to be a small circle of friends and family.

Coming to terms with your death is one thing but coming to terms with the death of everyone you love is another.

In that sense I prefer the idea of make the best out of the limited time you have because it is limited. Once time is limited, your own and that of others, maybe you can find a way to use the little time you have meaningfully.

Think about how you want to die. Do you want to die alone or would you like to die surrounded by people whose lives you made a positive impact on? How do you feel about dying alone and if no one cared whether you lived or died? If you think that no one would care whether you lived or died, you are depressed and should seek help.

</comment></comment></comment><comment><USER14> * Read 'The Human Evasion' by Celia green [1]

* Watch 'Neurons to Nirvana' [2]

Instead of avoiding the issue or running head-first into escapism (like most of the responses here are urging you to do), I advise you fully embrace it by exploring the 'inner space':

\+ Do psychedelic drugs (mushrooms, lsd, DMT, ketamine) Lots of them. When peak experiences manifest and your reality is in shambles, give it some time to absorb and integrate what happened.

      Don't burn yourself out like some street freak, the objective here
      is not to escape but to SEE. When integration takes place, go back
      to more psychedelics. You're an explorer.

\+ Meditation is good too but psychedelics are better and work faster.

[1] [http://www.theabsolute.net/minefield/humevas.html](http://www.theabsolute.net/minefield/humevas.html)

[2] [https://www.youtube.com/watch?v=_UlAMocYXQw](https://www.youtube.com/watch?v=_UlAMocYXQw)

</comment><comment><USER15> I believe the way most people operate is by avoiding thinking about these kind of subjects.

Another coping mechanism (a bit more healthy IMHO) is to try and accept death and pain as an inevitable part of the human experience and of what it means to be human. "Memento mori" and all of that.

</comment><comment><USER16> > _How do you deal with the fact that we don 't know why anything at all > exists? Why, for instance, do atoms, subatomic particles, etc, exist AT ALL? > Why not NOTHING?_

The "why something instead of nothing" question is one of those universal philosophical questions, but it's interesting to consider the fact that we have no concept or example of the alternative (true nothingness). If you think about it this question is kind of odd in that it carries a hidden assumption that nothingness is the normal state of things and existence (of anything, not just biological life) is somehow abnormal. Why should we have reason to believe that?

</comment><comment><USER17> > \- How do you deal with the fact that we don't know why anything at all > exists? Why, for instance, do atoms, subatomic particles, etc, exist AT ALL? > Why not NOTHING?

We might actually answer that question. First, there is no such thing such as "atoms". The universe (or reality) is not physical. You might want to read more about quantum mechanics to get a glimpse. It is also very possible that the universe created itself out of nothing. It is also possible that we are living in a loop. It is also possible that we live in a simulation, but that doesn't answer the existential question. As our gods have started somewhere.

As far as I'm concerned, I think the consciousness loop is what it is. Your life is never going to end. You are looping through it and probably through the same life infinitely.

> \- How do you deal with the fact that your life is meaningless. You will die > and - at best - leave behind a legacy for history books. More likely, you > will die and it will be completely insignificant. To put it another way: How > do you deal with the fact that your life is MEANINGLESS?

I changed my career. Broke the regular social norms. It was simple for me: You can just end it if you want. Might as well try to "see" things that are interesting to me. I did that way before my discovery of the quantum and uncertain reality of reality. It was Steve Jobs talk that pushed to it. If you were to die tomorrow, what will you be doing today? Just go ahead and do it NOW.

> \- How do you deal with the likelihood that you will suffer greatly at some > point in the future, whether it be from cancer, or some other disease? How > does a day of 9-5 work, capped off with a horrible prolonged cancer > treatment at the end of your life, not corrupt your mind?

Good riddance life! But seriously, I don't think it matters or it is about cancer. You can die from a meteor too. Stop the 9-5 job if that is what you don't want to do. Go back to 2. Answer that question. And don't be afraid to take life changing measures. Remember, you are insignificant on the larger scale of the universe. You only fear because of the social environment around you. Might as well change it.

</comment><comment><USER18> > How do you deal with the fact that your life is MEANINGLESS?

I do not agree with the philosophical idea of "Life is absurd because it's meaningless". Collective human constructs are what gives meaning to life. This meaning depends on what your society projects on you as valuable.

> How do you deal with the likelihood that you will suffer greatly at some > point in the future, whether it be from cancer, or some other disease?

If you can't change it, don't worry about it.

> How does a day of 9-5 work, capped off with a horrible prolonged cancer > treatment at the end of your life, not corrupt your mind?

Everybody dies. You may as well enjoy life, otherwise it's a complete waste.

</comment><comment><USER19> Acceptance is how I deal with the fact that I don't know why things exist, that ultimately my life is meaningless, and that I may end up with cancer or some other horrible disease. I accept it and move on because ultimately existential questions and crisis don't have answers.

They're like playing the 'what if' game. What if this happens? What if that happens? What if it does? It hasn't yet so why worry about it?

'Worry is like a rocking chair, it gives you something to do but doesn't get you anywhere.' \- Van Wilder (or at least that's where I heard it)

</comment><comment><USER20> Let's set some ground rules. There is a mind-body problem. This problem becomes exponentiated once emotions become involved. Whether things exists or not are not important, though they might exist. The ego is the most tenacious self -one has acquired through existence. It's such that stages of the unveiling of the true self, are pegged with such trivialities, such incomprehensible nonsensible non-sense that an existential crisis becomes rampant upon one's life journey. The highs and lows of life, after all the pain the true emotions they are all part of a regimen that an individual goes through. Which includes the complete mental and physical breakdown of you, the world around you, and everything you hold to be true. There are reasons on top of reasons for this. From the eyes of a guru, you are exactly treading down a path tracked by those that came before you. The only difference is that communication is more open ( as we're "(largely) engineers and science minded folks here"). Hehe, dude as you break down the many layers of ego, and as the ego is about to show it's true face, it will falsify and destroy everything else before it even lets itself be known -- and then right there at that very end of that first step your ego conforms to the self, and makes you think that your ego is the true self, that it was always the One, that it was and is the truest self. People find it easier to believe that their entire world doesn't exist rather than that their ego's don't exist. So how do you deal? Well if it's this process of self, this unopened rose that exists inside you, forget the new age bullshit, the religiosity of the masses, forget all that you know about anything, and if you can still hold onto nothingness. That's how you deal. You're at the cusp of your life, where you need to make sense out of meaninglessness and turn it into meaningness. It is the most decisive battle in one's entire life, the battle of the self, against the self. And if you already know who you are and what you want to be, then you start finding meaning in everything b/c well. I wish it was a gift I could give, something I could let you download off of me, heck I could tell you the same things in a million billion different ways --but it's better you come it yourself. That's the problem with free will right? It's a Catch-22 --No one else can do it for you.

</comment><comment><USER21> Absurdism for the truly existential stuff.

For the more practical matters understand that a bunch of people before you shaped how you live - and you will do the same to others. Whether thats negatively or positively, because you were active or passive. The engineer in you can look at the "wage-slave til' you die of cancer" problem and take it from there.

PS the lack of meaning and transient life isn't a bad thing - its the ultimate freedom! Go figure out some shit to pass the time, if you can try and make it fun/beneficial for the rest of us..

</comment><comment><USER22> 1\. The only thing that really comforts me deeply is the fact that we were here and we lived. I've had a really great life overall. I've gotten to enjoy a lot of what makes human life great. Much more than most people and that luck is never lost on me. Nothing can take away the life I've had. It happened. It's been great. It's a beautiful thing. The fact that it will end in tragedy is unfortunate but it doesn't alter the fact that it happened. I was here. You were here. We lived.

2\. As for meaning, the only thing that makes sense to me is to help yourself and others live as great a life as possible (See #1). In the best case you can not only help yourself but you can also help other people have better lives too. Any contribution is great. If you can't contribute at all, that's okay too. We should all just do the best we can, with no judgement or negativity for people that can't contribute. Bill Gates has spared millions of people from terrible suffering, that's an example of a huge contribution. Ideally everyone would live a great and long life. Anything you can do to help is meaningful, however small or indirect.

3\. Humans in the future won't suffer like this. We will evolve, using technology, to be superior beings that are likely immortal and beyond all forms of suffering we're currently aware of. Star Trek and beyond is inevitable if we don't blow ourselves up. This is comforting because it means this terrible situation won't afflict our descendants and it's upsetting because we won't personally make it to the promise land and we got pretty close.

4\. Humans are very much alike. It's reasonable to think of humanity as a single entity in some ways. So humanity itself is kind of immortal already, and that means all humans are kind of immortal. Almost everything that makes you special exists in other humans too, in different combinations but it's all there in the DNA and culture. You will die but humanity will live and what makes you special will live on within it.

5\. On a practical level, the best thing to do is ignore our ultimate fate and focus on having as good a life as possible. Living in this "ignorant bliss" is a necessary coping mechanism and nothing to be ashamed of. The luckiest people live in this ignorant bliss most of their life. Focusing on being mentally and physically healthy is the first step towards having the best life possible.

</comment><comment><USER23> It is nice to see that you are asking what I think are the truly meaningful questions. All too often we get caught up in the rat race we call this material life.

My humble suggestion would be to put an intention out into the world that you would like to try and find an answer to your questions.

The universe, I hope you will find, is all too willing to share the gift of timeless wisdom to a sincere seeker!

</comment><comment><USER24> Faith. I am an engineer and science minded person, and I've never seen that being opposed to having faith in god and the afterlife. My faith is not necessarity based on a purely logical belief system, but I find it at least reasonable in explaining why things exists, and rendering my life meaningful.

</comment><comment><USER25> 1\. Accept that you don't currently know. Things might change.

2\. Per (1), this might not be true. You don't know the meaning now, but things might change.

3\. Personally, I plan to (I hope) check out once things get too bad.

I second p0d's mention of Ecclesiastes. Beyond that, "life is but a dream".

</comment><comment><USER26> These matters are, strictly speaking, outside the purview of science.

I would say that what you're looking for is impossible to describe, the best I can do is point you towards what is called "spirituality", which is different for each person.

<comment><AUTHOR> I struggle with spirituality. For starters, nobody in my 'community' is deeply spiritual, really spiritual at all.

Add the fact that I do not believe in a god in any traditional sense, and I am stuck wondering what exactly I can be 'spiritual' of?

<comment><USER7> For almost any attribute, if you have nobody at all in your community like that, it would probably be worthwhile to expand your community to include such people.

It's all too easy for "engineers and science minded folks" to become a rather insular community, if you're not careful.

</comment></comment></comment><comment><USER27> What you are describing is essentially nihilism. And it's hard to talk yourself out of.

Your first statement is the crux of the argument. It's a tremendous question, we just don't understand, but somehow we __are __here. Is there even anything here if there wasn 't our consciousness to witness it?

But if we don't know the why, we also don't know if it matters or not. And perhaps individually you feel your impact is moot, but you affect those around you, in small ways, and that ripples out to the way they affect the people around them, and so forth.

And you __will __suffer at some point, but you can also contribute to or ease the suffering of others, that is an active choice on your part. And I believe it is meaningful.

Jordan Peterson touches on this with a nice though experiment. If we collectively all try to contribute to suffering tomorrow, every human on the planet, consciously acting to make things worse. You would have 'hell' on earth in no time.

Alternatively what if everyone tried to make it better?

You are just a cog in that wheel, as we all are, but we add to it's direction.

</comment><comment><USER28> [https://www.youtube.com/watch?v=7tWsTQR6_2s](https://www.youtube.com/watch?v=7tWsTQR6_2s)

</comment><comment><USER29> Vipassana Meditation really helps.

</comment><comment><USER30> I became a Buddhist, lol.

</comment><comment><USER31> I guess mine might be an unpopular opinion, I believe in a supreme being and am religious. Usually when people say thing like "Given we're (largely engineers and science minded folks here" they aren't looking for religious folks to step in and say anything. Nevertheless, I am and it brings me peace and joy.

My beliefs help to answer a lot of the first question.

When it comes to the second, I find a lot of meaning through life. That doesn't mean that I will leave some great legacy but I hope I help quite a few people along the way. I do that by changing my daily routine from what you describe. I don't work 9 - 5 and cap off my day with a prolonged cancer treatment, nor do I cap it off by thinking about my future need for a cancer treatment, heart surgery, or my impending death in anywhere from a few days to a few decades (for clarity, I am not currently dying any faster than a normal American middle ages male, but I will eventually die).

So what do I do? Well I spend a huge amount of my spare time helping others with my family and friends. I volunteer evening in refugee shelters, homeless shelters, drug addiction treatment groups, food banks, and low cost clinics. Sometimes I just help people around my neighborhood who need it. This week on Monday I went and had a long conversation with an older woman in my neighborhood and her boyfriend (both in their 80's) before she was going in for a major heart surgery. We talked about her life, about her thoughts and feelings, about her fears, we prayed together. At the end of the evening she felt better, I felt amazing, and she had her surgery (she's doing very well).

On Wednesday I went to another neighbors and my kids and I cleaned up her yard. She just lost her husband. She didn't ask for the help and I didn't know her real well, but we had a great time and now my family and I have a new friend. It was hard, hot, and dirty work, but who cares? At least I didn't sit up worrying about my mortality all night.

I spent quite a few of the other nights at the refugee shelter. They always need a ton of help! My wife and kids love the people they meet. We have a great time and meet new people and learn new things. And I believe we make a difference.

Is my life meaningless? I don't think so. I believe there is more after this life. However, even if there is nothing and we are just an extremely short blip on an extremely small blue dot around an unremarkable sun on one of the little spiral arms of a little insignificant galaxy, I still don't believe that life is meaningless. I enjoy the moments I have with the other insignificant blips that I come into contact with.

My point in posting is not to brag, I don't really care for accolades. My point is to show meaning. So my answer to your major question is: find meaning in your life. I've given a few suggestions. Start right now on one. If you walk out of your house right now and look around, I bet you're find someone who needs you.

</comment><comment><USER32> More beer.

</comment><comment><USER33> 1\. Some people view life as a puzzle to be solved. It's a game created by a higher being, maybe even myself, to be played.

2\. I don't believe in other minds. So therefore not only is my life not meaningless, it is the only life that ever had meaning.

3\. People are not guaranteed to suffer.

</comment><comment><USER34> Drugs.</comment>


Facebook Spells Its Name with IPV6 Address - <AUTHOR> http://www.theregister.co.uk/2015/07/08/facebook_casts_a_hex_on_dns/

<comment><USER2> Clever, but will likely result in lawsuits over randomly assigned IP addresses which just happen to spell prominent trademark holder names. So, a "funny once" joke.

<comment><AUTHOR> I'm actually curious given the massive range of possible IPv6 addresses whether this will ever be an issue? Would the chance of randomly getting anything resembling a trademark not be so low that it is a non-issue?

</comment></comment><comment><USER3> This isn't news

<comment><USER4> This was the first phrase that came to my mind.</comment></comment>


The Big Data Brain Drain: Why Science is in Trouble - <AUTHOR> http://jakevdp.github.io/blog/2013/10/26/big-data-brain-drain/

<comment><USER2> this should have landed on the front page...</comment>


DTrace and Python - <AUTHOR> https://github.com/pyconsk/2016-slides/blob/master/Python_and_DTrace/DTrace%20-%20pyconSK.pdf <comment><USER2> Great deck! In particular, for the incredible magic on slide 14, a debt of thanks is owed to John Levon[1] and to whomever has maintained that work and brought it forward.

On slide 28, the presentation asks "What is 'Speculative Tracing'?" It's unclear if that's a rhetorical question, but just to answer it here: speculative tracing is a DTrace facility that allows for data to be traced speculatively, and only committed to the trace buffer if and when some other (later) condition is met.[2] My original inspiration for this was a case that we had back in the day at Sun on the Performance and Application Engineering (PAE) team, when Yufei Zhu (now at Facebook) described a case she had in which one out of every 10,000 mmap()'s was failing with EINVAL -- and it was really tough to use DTrace when she was only interested in its output 0.01% of the time. For Yufei's (motivating) example, speculative tracing offered a way of capturing all of the necessary data on every mmap request, but only emitting that data when the entire operation was found to have failed. Speculative tracing is one of those you-don't-need-it-until-you-need-it features of DTrace (to which I would certainly add anonymous tracing) -- but when you need it, it's a lifesaver, and I have it used it as recently as last week to nail a particularly nasty bug that very much needed it.[3]

[1] [https://blogs.oracle.com/levon/entry/python_and_dtrace_in_bu...](https://blogs.oracle.com/levon/entry/python_and_dtrace_in_build)

[2] [http://dtrace.org/guide/chp-spec.html](http://dtrace.org/guide/chp- spec.html)

[3] [https://twitter.com/bcantrill/status/769225926726918144](https://twitter.com/bcantrill/status/769225926726918144)

</comment><comment><USER3> Shameless plug: if you are looking for something similar to DTrace but for Windows that can be also used in Python or any other COM capable programming language... you can take a look at our Deviare Hooking Engine: [https://github.com/nektra/Deviare2](https://github.com/nektra/Deviare2)

Additionally, we have other open source instrumentation engines like [https://github.com/nektra/Deviare-InProc](https://github.com/nektra/Deviare- InProc) (better and more secure than Detours, check [1]), RemoteBridge: [https://github.com/nektra/RemoteBridge](https://github.com/nektra/RemoteBridge). SpyStudio will also be open sourced very soon: [http://www.nektra.com/products/spystudio-api- monitor/](http://www.nektra.com/products/spystudio-api-monitor/)

[1] [https://www.blackhat.com/docs/us-16/materials/us-16-Yavo- Cap...](https://www.blackhat.com/docs/us-16/materials/us-16-Yavo-Captain-Hook- Pirating-AVs-To-Bypass-Exploit-Mitigations.pdf) and [https://www.blackhat.com/docs/us-16/materials/us-16-Yavo- Cap...](https://www.blackhat.com/docs/us-16/materials/us-16-Yavo-Captain-Hook- Pirating-AVs-To-Bypass-Exploit-Mitigations-wp.pdf)

<comment><USER4> interesting.

by the way, i got a google security warning on chrome trying to get to one of the nektra blog pages: Google Safe Browsing recently found harmful programs on blog.nektra.com.

also, have you considered an open source anti-virus program at nektra ? One based on stuff like what's available here: [http://www.nsrl.nist.gov/nsrl- faqs.html](http://www.nsrl.nist.gov/nsrl-faqs.html)

<comment><USER5> Hi @naveen99.

We upload precompiled samples but time to time Google marks us as harmful when they aren't. Feel free to download the source code from [https://github.com/nektra/](https://github.com/nektra/). Visual Studio projects are provided too.

<comment><USER4> Thanks. Will do.

</comment></comment></comment></comment><comment><USER6> Does anyone have any inside knowledge of when, if ever, DTrace might become a standard/official part of Linux? Seems like right now you need to compile a kernel module which will taint your standard kernel in order to use it.

<comment><USER7> > Does anyone have any inside knowledge of when, if ever, DTrace might become > a standard/official part of Linux?

Never. Due to dtrace being licensed under the CDDL it will never be mainlined into the upstream kernel. I would be keen to point out this does _not_ mean there is a conflict of licenses from a legal point of view. Before anyone starts making claims like "you're not allowed to do this", see the use of the ZFS kernel module in recent releases of Ubuntu. I shall say no more on the matter as a refuse to be drawn into a licensing debate...

Getting back on topic, those seeking more on user space providers should check out this video from Bryan Cantrill.

[https://www.youtube.com/watch?v=6chLw2aodYQ](https://www.youtube.com/watch?v=6chLw2aodYQ)

Full disclosure I never had to run Solaris in production so I can look back on Sun with rose tinted glasses. That said I still long for many of the features it pioneered. Seriously ZFS would eliminate half of my production problems and dtrace would make the remaining half solvable.

<comment><USER8> > I would be keen to point out this does not mean there is a conflict of > licenses from a legal point of view. Before anyone starts making claims like > "you're not allowed to do this", see the use of the ZFS kernel module in > recent releases of Ubuntu.

AFAIK it's only packaged as a dkms-package:

      Package: zfs-dkms
      Version: 0.6.5.6-0ubuntu8
      Priority: extra
      Section: universe/kernel
      Source: zfs-linux
      Origin: Ubuntu
      Maintainer: Ubuntu Developers...
      Original-Maintainer: Darik Horn ...
      Bugs: https://bugs.launchpad...
      Installed-Size: 8 325 kB
      Provides: lustre-backend-fs,
        lzfs, lzfs-dkms
      Pre-Depends: spl-dkms (>= 0.6.5.6)
      Depends: dkms (>= 18.104.22.168), zfs-doc (= 0.6.5.6-0ubuntu8)
      Conflicts: lzfs, lzfs-dkms
      Replaces: lzfs, lzfs-dkms
      (...)
      xenial/universe amd64 Packages

Note that this is in "universe" _not_ Ubuntu proper.

From DKMS(8): dkms is a framework which allows kernel modules to be dynamically built for each kernel on your system in a simplified and organized fashion.

> I shall say no more on the matter as a refuse to be drawn into a licensing > debate...

I really don't want to start that debate, but I also think the "Ubuntu distributes (binary) ZFS" isn't quite accurate.

Now, how this all shakes out in terms of whether or not ZFS is bundled with Ubuntu ("on the ISO so to speak"), and if it then really makes a difference that it's distributed as source-code that can be automatically compiled -- I don't know. I wouldn't think shipping pythons source (to be later compiled to byte code, pyc) is enough to dodge GPL restrictions on bundling other code/binaries as part of a distribution.

But, yeah, whichever it is, CDDL code isn't likely to ever be upstreamed to the kernel proper.

<comment><USER9> That's actually only half the story and mostly not relevant... the dkms package is still there, but ZFS is now shipped with the kernel package:

lathiat@ubuntu:~$ dpkg -S zfs.ko linux-image-4.4.0-35-generic: /lib/modules/4.4.0-35-generic/kernel/zfs/zfs/zfs.ko linux- image-4.4.0-36-generic: /lib/modules/4.4.0-36-generic/kernel/zfs/zfs/zfs.ko

Announcement is here: [https://insights.ubuntu.com/2016/02/16/zfs-is-the-fs- for-con...](https://insights.ubuntu.com/2016/02/16/zfs-is-the-fs-for- containers-in-ubuntu-16-04/)

There is a lot of debate about the licensing legality of it, and there are opposing positions from various parties. Here's the official post summarising it: [http://blog.dustinkirkland.com/2016/02/zfs-licensing-and- lin...](http://blog.dustinkirkland.com/2016/02/zfs-licensing-and-linux.html)

<comment><USER8> Thank you for taking the time to reply with those two links. Very interesting (and directly helpful as one of my use-cases is led+zfs. Not needing the dkms module makes things simpler).

</comment></comment></comment></comment></comment><comment><USER10> Video: [https://www.youtube.com/watch?v=HwqvHUGyGTE](https://www.youtube.com/watch?v=HwqvHUGyGTE)</comment>


Show HN: StrelkiJS - small library to index and SQL-join in-memory collections - <AUTHOR> https://github.com/amaksr/Strelki.js <comment><USER2> Does anyone know of data structures better suited to solve the same goal? (ie fast in memory joins with multiple dimensions, star schema OLAP queries, etc.) ?

<comment><USER3> If you're interested in defining dimensions using tabular data in JS, check out crossfilter.js.

[https://github.com/square/crossfilter](https://github.com/square/crossfilter)

<comment><USER4> The updated link seems to be [https://github.com/crossfilter/crossfilter](https://github.com/crossfilter/crossfilter). Crossfilter (the organization) is now actively maintaining and extending Square's original crossfilter implementation.

</comment></comment></comment><comment><USER5> This is pretty cool. If you don't mind me asking, did you have a specific application in mind when you decided to write this? Is the idea that your data would be transient (user session?) and that you need to manipulate it, do some things, and then toss it when the work's done?

<comment><AUTHOR> Hi there,

Thanks for your feedback.

I am working on adding paid subscription to KidsTrack app ([https://www.izhforum.info/forum/izhevsk/tracker_live_map.php...](https://www.izhforum.info/forum/izhevsk/tracker_live_map.php?demo=1)): it is going to be a one-page app with map div and a bunch of dialogs, so user could access and manage all his info. There is a bunch of related tables, that are being displayed on different dialogs in different combinations. For example "Transactions" dialog displays data from linked TRANSACTIONS, ACCOUNT, PAYMENTS, DEVICES and CURRENCIES table; "Devices" dialog displays DEVICES, TRANSACTIONS and CURRENCIES table, "Accounts" table displays data from ACCOUNTS and CURRENCIES tables, etc.

It quickly became a mess of API calls that are querying the same tables but with different joinings. Also, to open each dialog it has to query server and receive the same data many times, which leads to wasting traffic and CPU time. Also, the length of column names from queries becoming bigger, which also wastes traffic. These objects with long prefixed column names need later to be converted on the client back into original objects with unprefixed column names, which also adds complexity to the code.

I tried to optimize it and do some of these joins on client side, but quickly got tired of repeating nested loops and excessive usage of find() and indexOf() functions.

So, I put my main project on hold and decided to write something to do joining more easily on a client.

It has following limitations:

* Objects in array have to have unique "id" field. But, since many server-side frameworks already use "id" field for similar purposes, this should not be a problem.

* Joining can be only done into target table with index on target field. Otherwise it will throw an error.

* Joining currently can only be done by direct value matching, no arbitrary expressions or functions.

* IndexedArray does not store copies of the objects, only references. Therefore it is possible to make indexes inconsistent by updating original object after it has been put into IndexedArray.

To answer your questions - it should be handy for one-page apps or for node.js code, as it may help to reduce asynchronous calls.</comment></comment>


A Virtual Weimar: Hyperinflation in a Video Game World - <AUTHOR> http://mises.org/daily/6435/A-Virtual-Weimar-Hyperinflation-in-a-Video-Game-World <comment><USER3> "Furthermore, inflation is not simply an increase in the supply of money within an economy; it is the increase in that portion (if any) not backed by a commensurate increase in specie: most common in history, market commodities like gold or silver. "

Either the author of that article is misenterpreting, I am misinterpreting or Austrian school of Economics is tad stupid. I'm hoping that the one at fault here is me.

Basically even with increase of the amount of gold or silver you'd still get inflation, check what happened with Spain at 1500 or so when they got a huge influx of gold from the new world.

You can also imagine what would happen if suddenly we'd get huge influx of new gold. As the amount of stuff in the world does not increase the amount you can buy with that new gold must decrease.

<comment><USER4> "Austrian economics is tad stupid" is the more correct choice. Austrian economics does not correspond with reality and is rightfully marginalized and ridiculed by the vast majority of economists.

In the 19th century we thought of inflation as the change in ratio of money to goods. Inject more money in the economy without a commensurate increase in the amount of goods and you get inflation. Take money out of an economy and deflation occurs. Now we know this isn't really accurate. Inflation happens not when there is more money in the economy but when more money is competing for the same goods. If money just sits in a bank account (or virtual bank account in the case of Diablo III) it doesn't really affect the economy and no inflation occurs. There is the same economic activity but a few rows in a database somewhere contain larger numbers.

This is also why in the real world the fed can triple the size of the money supply without causing inflation. Of course the austrians predict, over and over again, that runaway inflation is going to happen Any Time Now. But it doesn't and it won't.

Of course in a completely healthy economy you _would_ expect an increase in the money supply to lead to an increase in economic activity and inflation, because in a healthy economy people spend and invest the money they have.

Austrian economics is about digging gold up from the ground, then smelting it into ingots and then burying those ingots again in a vault somewhere. Only through this mystical process can you get Real Money. Otherwise the money is "fiat money". It's so silly.

<comment><USER5> As an engineer with no background in economics, do you have any sources that discuss the pitfalls of Austrian economics from a layman's perspective?

It seems nowadays that the internet is so rife with neo-Libertarians that it's very difficult to find content discussing Austrian economics that is not biased towards it. I'd like to see a more balanced perspective, or at least a rundown of reasons why Austrian economics has failed to correspond with reality.

<comment><USER6> I have a Master's in Applied Math, and a Master's in Economics (educated by neo-Classicals).

Keep in mind the Austrians are not a monolithic movement, this article was published on the Von Mises Institute site, which has more to do with the Rothbardian strain of Austrians. Core to Austrian-Rothbardian theory is a system of a priori axioms and a repulsion to mathematic formulations and empirical validation, so the argumentation can end up being along the lines of "because I said so!"

The objection to mathematization comes from the belief that preference relations lead to a discontinuous utility curve, which the tools of differential calculus cannot crack (this is wrong, as a graduate-level mathematical analysis covering Lebesgue integration shows).

A more detailed neoclassical analysis of what's wrong with Austrian economics is Bryan Caplan's [Why I Am Not an Austrian Economist](<http://econfaculty.gmu.edu/bcaplan/whyaust.htm>).

</comment></comment></comment></comment><comment><USER7> Bear in mind that inflation of the money supply is not the only possible cause for price bubbles. I used to think so too (because hey, I used to read mises.org religiously). But it's not, as they say, that simple.

Here's are some good pieces on bubbles generally, and how money supply by itself can't explain real-world phenomena:

[http://skepticlawyer.com.au/2013/05/15/bubble-trouble-not- an...](http://skepticlawyer.com.au/2013/05/15/bubble-trouble-not-an-easy- money-problem/)

[http://skepticlawyer.com.au/2013/04/19/bubble-trouble- about-...](http://skepticlawyer.com.au/2013/04/19/bubble-trouble-about-asset- booms-and-busts/)

<comment><USER8> Also, the focus on "the money supply" is a red herring. What matter for real world prices is supply and demand of real world goods.

When you seriously study hyperinflationary periods, you realize that - <USER2> the story of this Diablo auction house bug - it's basically never a case of governments going crazy with printing money. There's always an underlying cause that came _before_ the hyperinflation - usually a combination of a collapse of production capacity (in Zimbabwe, caused by drastic and ill thought-out economic reforms) and crushing _foreign-curency denominated_ debt. See this paper, for example: <http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1799102>

It is true that excessive spending by _anyone_ can lead to inflation (not just by government, since most money is actually "created" by private banks). However, what Austrian-types like the Mises people blind themselves to is the fact that the economy has two possible reactions to an increase in demand: There can be price increases, and there can be increases in production.

To end up at their stated position, Austrians have to completely deny the possibility of production increases. It's no wonder they got basically all of their inflation-related predictions wrong for many years. They are unfortunately closer to religious dogma than serious economic analysis.

<comment><USER9> > the economy has two possible reactions to an increase in demand: There can > be price increases, and there can be increases in production.

armchair economist here, but i woudl say that when the increase in money supply outstrip the increase in production (becase there is a limit to the rate of increase of production, but there is no real limit to the rate of increase of printing more money - just put an extra zero at the end, you don't even need more paper/plastic!).

<comment><USER8> Absolutely. It is a matter of quantity, and it would be nice to understand how much and how fast production can increase. But since Austrians usually outright deny the possibility that production increases at all, the path to a fruitful discussion is usually closed.

In practice, even the more ambitious types of policies that are debated do not actually increase spending that much.

<comment><USER7> Rothbard-school Austrians make the same error as classical Keynsians. They freeze the structure of production, treat it as a homogenous gas, and then modify some exogenous variable.

The Austrians modify money supply; the Keynsians modify the aggregate demand.

It's the same error of over-simplification. Economies are not smooth curves, they are lumpy and have heterogeneous networks of heterogeneous agents.

</comment></comment></comment></comment></comment><comment><USER10> Good to see the Austrian School weirdos at the Ludwig von Mises Institute have given up real-world economics and now just play video games. Hopefully in the virtual world their incredible track record of dead-wrong predictions will cause less harm.

<comment><USER11> > incredible track record of dead-wrong predictions

The Austrian School economics methodology does not allow for making predictions. See e.g. the note at the beginning of <http://wiki.mises.org/wiki/Austrian_predictions> It provides certain tools and models that one can use for explaining events and reasoning about policy options. These reasoning can be used as a basis of predictions (along with the other, non-economics-theoretic assumptions), but the predictions themselves will fall outside of the economics science in the Austrian School sense.

Basically, if your analysis manages to reason away uncertainty of the future, you're not following Austrian methodology. And no, the Austrian methodology won't even let you quantify that future uncertainty in probabilistic terms: that would only be possible if it used _causal_ explanations of human action; instead, it relies on _teleological_ , goal-oriented explanations.

Now, questioning and disputing the value, coherence and real-world applicability of the Austrian methodology is perfectly OK. (Personally, I find the Austrian economists' disregard for formal notations very unfortunate, as I've mentioned in another discussion: <https://news.ycombinator.com/item?id=1647747>) Your comment, however, seems to be simply misguided, based on a flawed understanding of the issue, and the sarcastic tone is unlikely to facilitate a constructive discussion.

(Edited: spelling.)

<comment><USER12> You probably know more about economics than me (I'm just a mathematician), so I won't argue against you, but if I understand your two posts correctly, it seems like what you're describing is completely useless. If a model doesn't have predictive power then what is it good for?

I understand from your other post that the justification of Austrian "models"/"explanations" comes from their simplicity. Which I guess is some application of Occam's razor? But that makes no sense - the simplicity of a model on it's own is no indication of correctness. You always need some notion of the likelihood of the observed evidence under that given model. Otherwise, you can just say "all people make all decisions completely randomly" and that's the simplest model of all.

A completely non-predictive model has no real-world meaning, because it can't be used to effect reality. I suspect that Austrian models do effectively have some small predictive power - there is an implicit causal analysis that comes from the small element of qualitative evaluation of uncertainty, no doubt used instinctively by practitioners to decide whether an argument sounds plausible.

Anyway, it all sounds completely ridiculous. How is their method of choosing explanations any better than a witchdoctor deciding that thunder is a sign of the gods being angry?

<comment><USER13> For me, the major idea that the Austrians bring to the table is that the study of economics is not a science and never can be. Attempts to create and support models by quantitive analysis of data are flawed because human economies are so astronomically complex that it's impossible to isolate variables, and therefore impossible to design valid economic experiments or draw sound conclusions from economic data.

There are many parallels to algorithms analysis. If we want to know how fast an algorithm runs, our approach isn't to try to run it on every processor in existence and compare the results. Instead we break down the algorithm logically to deduce its theoretical running time.

Austrian economics--in its good parts anyway--is an attempt to build a framework for economics that is more like the asymptotic analysis of algorithms. But this very compelling goal is frequently derailed by politics and polemics.

<comment><USER12> Algorithmic analysis is absolutely scientific. Asymptotic analysis is based on models of computer behaviour that are testable (and people do test them and do improve on them). Moreover, the asymptotic models make predictions and those predictions are testable (and people do test them and do improve on them).

The asymptotic analysis of algorithms would not be useful if it didn't make accurate predictions about real world phenomenon. I've written peer reviewed computer science papers where I have devised an algorithm, predicted its asymptotic behaviour and then validated through empirical testing.

What you're describing sounds completely different. If the major idea is that human economies cannot be scientifically analysed, then surely all analysis is a waste of time? An analysis that isn't based on the scientific method isn't more likely to be correct.

</comment></comment></comment></comment></comment><comment><USER14> The problem is that in order to make the experience fun for new players, the faucets have to be relatively open and the sinks not too punitive. I'm not sure how you can successfully balance a truly working virtual economy with making it fun for any but the most hardcore trader types, especially once real money comes officially into the equation.

<comment><USER15> > successfully balance

You could try to balance faucets with omnipresent unavoidable sinks aka. taxes. But how fun would it be to pay taxes in a game? "You loot 100 gold. Minus 30% gold drop tax. 10% dungeon upkeep tax and 15% financial transaction tax." :)

So the thing is: As soon as you try to model a "real" economy (possibly with real money exchanges) you can't have artificial faucets and sinks. Even Eve Online - which has the most advanced game economy - struggles with inflation as there are too many faucets. For such an economy to work you need to have a steady amount of money circulating the economy. And the only source (and destination) for money for players has to be other players.

But then you get a lot of other problems. Like what if a player hoards all the gold? Or if players just deactivate their accounts with significant gold sums on them? How do new players get their first gold? etc.

<comment><USER16> At least one problem with gold in games is that the management of large sums of gold is unrealistic. If you really had a huge portion of gold available in the game the storage and transportation of said gold would be both costly and risky. These concepts work mostly for fantasy themed MMOs though, not as much in something like Eve Online.

<comment><USER15> Heh, yeah ... I remember Ultima Online where gold had weight and you had to carry it in your back pack. If you wanted to move large sums you had to get a pack horse. :)

But most MMOs nowadays treat gold as something ethereal that you even can't lose when you get killed.

</comment></comment></comment></comment><comment><USER17> Nicely written article, but it doesn't seem to factor in the fact that the item supply is also increasing over time. Since players are only equiping stuff only once in a while, the items have nowhere to go, thus decreasing in value.

While players (or bots) farm for gold, they're also farming for items. It would be interesting to see how item prices evolved over time...it might be that only prices for high end items have skyrocketed, while decent low end items are almost free.

<comment><USER18> The experience of a friend and myself when we returned to D3 about three months ago was that for a relatively small amount of money (250k gold perhaps) you could get gear good enough to play on some of the lower monster power levels of Inferno. Inferno is the hardest difficulty level, and there's 10 settings of monster power to tailor the difficulty to your preference / gear. I think that he played on MP3 or 4, but I usually stuck around at MP 1-2.

However, I stopped playing because I couldn't even begin to afford the gear to go higher. I figure if I wait a few more months then the gear that I'd need for MP5 shouldn't be so expensive anymore. As such, even though my gold is technically becoming worth less in terms of top tier items, it's becoming worth more in terms of decent low-mid level items. As such, I think that the D3 situation is more complex than just hyperinflation.

</comment></comment><comment><USER19> There's a delicious irony to dollars being the hedge against virtual gold hyperinflation while real gold is the canonical hedge against dollar (hyper-) inflation...

</comment><comment><USER20> This makes me wonder to what extent online games could be used to model/explore/study economic and social theories. Here we have an example of hyperinflation happening. Could you reliably reproduce the results? Could you reliably adjust conditions to lead to expected results? If so how accurately? I think you could get results much better analyzing surveys because of the seriousness people put into games.

<comment><USER21> valve hired an economist do do something like that. [http://blogs.valvesoftware.com/economics/it-all-began- with-a...](http://blogs.valvesoftware.com/economics/it-all-began-with-a- strange-email/)

<comment><USER22> And EVE online has had economist onboard since 2007

[http://www.scientificamerican.com/article.cfm?id=virtual- wor...](http://www.scientificamerican.com/article.cfm?id=virtual-world- economists-on-real-economies)

</comment></comment></comment><comment><USER23> While it's true there was a significant upset in the Diablo 3 economy after the 1.0.8 patch came out, things are settling down again, and we're certainly not seeing the kind of delta that would make me call out a hyperinflation. If anything, the prices remind of me a bubble forming and consequently popping.

Worse, the article is not just inaccurate, it's dishonest: it hides the data that contradicts it, despite being publicly available on a day-by-day basis. [1] is a wonderful resource.

The eye-catching table near the end stops at peak prices. The prices since that peak are more-or-less hidden in the second half of an otherwise fairly unremarkable paragraph. We've seen a downward tendency, with prices moving back to pre-patch prices. We can illustrate that by looking up more recent data for the same examples as listed in the article:

Radiant star amethyst: 20m before patch (april 29), high of 60m, 22m now.

Radiant square ruby: 200k before patch, high of 528k, 266k now.

Star emerald: 760k before patch, high of 1.6m, 770k now.

Flawless square topaz: 3k before patch, high of 16k, 4.5k now.

Tome of jewelcrafting: 1.2k before patch, high of 6.5k, 1.5k now.[1]

All these items have a downward price trend, with the exception of the (relatively inconsequential) tome of jewelcrafting, which is stable. I would not be surprised to see these and other (harder to track) items reach month's end at roughly pre-patch prices.

It's also worth mentioning at this point that the prices of each rank of gem is linked directly to the price of the rank below it. For example, a star gem costs 80k gold plus 3 radiant square gems of the same type to craft. This is an ingame feature. All gem prices are therefore functions of the price of flawless square gems, the highest rank that monsters can drop. The article mixes the prices of different gem ranks, which is either ignorance of the way the game's economy and mechanics works, or an intentionally and artificially pretense of diversity. It's a bad sign either way.

Finally, and most important of all: _the European market has been fairly stable throughout this whole debacle_. The only difference between the European and the North American markets is that the latter suffered the (disastrous) bug described by the article, while the European market did not: Europe gets its patches 6-9 hours after North America does and the bug that it introduced was fixed before the patch hit Europe.

This lends credit to the idea that all we're seeing here is the aftershocks of massive destabilization due to a simple bug. To me, the fact that prices seem to be returning to normal testifies to the relative stability of the Diablo 3 market, and long term price trends show the same. Huzzah, Blizzard!

All in all, while interesting in a general sense, the article is flawed when it comes to the specific case it uses to promote its message, the turbulence of Diablo 3's economy over the last few weeks. This is not hyperinflation.

[1] <http://www.diablohub.com/price-tracker/>

<comment><USER24> >> Worse, the article is not just inaccurate, it's dishonest: it hides the data that contradicts it, despite being publicly available on a day-by-day basis. [1] is a wonderful resource.

What do you expect from mises.org?

It has an agenda to push, and is relentless in doing so.

</comment></comment><comment><USER25> I remember when the money in Phantasy Star Online became worthless.</comment>


Introducing FreeNAS Corral - <AUTHOR> http://www.freenas.org/blog/introducing-freenas-corral-an-open-source-hyper-converged-storage-platform/ <comment><USER3> Wow, I didn't realise that FreeNAS had come so far. Featureset looks amazing and better than most of enterprise solutions.

<comment><USER4> Bear in mind, ixSystems HAS an enterprise solution based on FreeNAS called TrueNAS - we bought two appliances with 72TB if usable storage to handle image storage for our ECM suite which has BILLIONS of small images. The things run like a champ, and outside some minor issues with the AD integration we've taken care of they've been a great storage solution.

<comment><USER5> Same experience with FreeNAS 9.x and AD.

But... I just upgraded to FreeNAS 10/Corral and the AD integration just worked!

Previously there was a `generate_smb4_conf.py` script that translated the UI selections into a `smb4.conf` file, but our AD had some weird schema issues that required a custom config. I ended up hacking parts of the generator script to get it to produce the right thing. In FreeNAS 10 it seems they switched to samba's Registry Configuration [1].

[1] [https://www.samba.org/~obnox/presentations/linux- kongress-20...](https://www.samba.org/~obnox/presentations/linux- kongress-2008/lk2008-obnox.pdf)

<comment><USER4> My issue was actually the way their HA setup attempted to configure itself with duplicate SPN's - I ended up needing to manually export a key tab with ktpass to use it instead of getting it via winbind.

</comment></comment></comment></comment><comment><USER6> I've been using FreeNAS 9.10 for just under a year now having originally moved from Ubuntu Server. Its feature set, ease of use and stability really is incredible. Can't wait to try Corral out.

</comment><comment><USER7> I'm excited to see FreeNAS "Corral" come out and the user interface looks great, however I'm still disappointed that jails are not able to be used.

I'm not sad to see the plugins system gone, it was terrible and I always set up my own custom jails on FreeNAS anyway, but It would have been nice to be able to use jails at least from the command line. I guess they don't want to have to worry about supporting both.

I just wonder about the extra overhead of having to run a virtual machine for every container as opposed to just running a FreeBSD jail. bhyve is also not as established as FreeBSD jails and I wonder about stability. Now you're having to worry about the stability of bhyve and docker. For just running services like emby, syncthing, SABnzbd or the like I honestly think having the option to just run a FreeBSD jail is superior. Considering it is FreeBSD under the hood I don't see why the option is gone.

It's nice to be able to run Docker containers don't get me wrong, but I just wonder if it is really the superior technology.

</comment><comment><USER8> Well done iXsystems and FreeNAS developers! You blow the doors literally off everything else.

</comment><comment><USER9> FreeNAS is awesome.

Does anyone know of a way to configure FreeNAS (or an alternative FS, such LizardFS or Tahoe LAFS) that can provide the feature set of Isilon OneFS (not the performance) with an open source implementation?

That is,

1\. to add more storage, just add another machine (everything looks like one big volume, irrespective of machine/disk boundaries).

2\. replication / error correction / raid works across the entire fleet of machines; that is, you can configure it so that up to 1-disk per node, and up to 2-nodes overall can die without damage to data

3\. snapshots

<comment><USER10> If you dont know, take a look at [https://infinit.sh](https://infinit.sh)

<comment><USER9> Interesting. The features I asked for are almost all in "enterprise" and there's no price quote yes (or even a mention of whether enterprise is open source - I am willing to pay for a good system, but not for one I can't support myself)

</comment></comment></comment><comment><USER11> What is the status of ipfs integration? I know it was part of the v10 roadmap, but I don't see any mention of it in Corral and have been unable to figure out if it features in the Corral roadmap. Corral looks like it would be a great UX for farming disk space for ipfs/filecoin in the future. Thanks.

<comment><USER12> Dropped from the 10.0 release. No idea if/when it will make it in. It was working in ALPHA, but only as an experiment.

</comment></comment><comment><USER10> Is it possible to run Windows 10 as guest in Bhyve? I was planning to build my workstation on Ubuntu as host OS. Running few VMs inside - two windows, two linux and managing my ZFS pool (host os directly)... but if FreeNAS can run Windows VMs, it would be ideal...

</comment><comment><USER13> I'm wondering, what's the usecase for Docker there?

<comment><USER14> They've always had the functionality to run software like Transmission inside a Jail. I guess they just moved that functionality to docker to be more "hip".

<comment><USER8> Reminds me of the joke, "How do you get docker to be secure? Run it inside a (FreeBSD) jail."

Having not looked at it that's my guess what they are doing. Everything related to it is in a jail. Making it secure regardless of dockers questionable security.

<comment><USER15> Of course jails are the underlying mechanism of that Docker port on FreeBSD that someone did once… But I'm not sure they're using that. Linuxulator is not perfect, and they announced "Docker support" not "imperfect Docker support" :D I bet they're using something like boot2docker in a bhyve VM.

<comment><USER8> I'm trying to find out. Oddly enough searching the FreeNAS docs on the website for "docker" returns nothing.

I've seen reference to jails mostly regarding running docker and to a lesser extent bhyve.

Anyone running running corral or a recent snapshot that can say what secure tech docker is running in?

<comment><USER16> It's described a bit in the video [1] they seem to use VMs as docker hosts.

[1] [https://youtu.be/x4IBKUmC5ns?t=1374](https://youtu.be/x4IBKUmC5ns?t=1374)

</comment></comment></comment></comment></comment></comment><comment><USER17> What's FreeNAS like from a security point-of-view nowadays? How is it with security updates being pushed out? How about default configuration?

<comment><USER18> You can set up email notifications to have your server email you when new updates are available. They come out semi-frequently; maybe monthly. You can install them from the web interface pretty easily; major updates usually require a reboot.

By default I think the only way to access FreeNAS is through the web interface (which seems fairly secure, though I'm not really all that familiar with FreeNAS's codebase). No SSL by default, but you can set it up with self-signed certs almost trivially using FreeNAS's GUI. You can enable SSH access and other services manually, and those services come with their own set of security concerns you need to be aware of.

</comment></comment><comment><USER19> Does this version still require a tiny slice of every data-storage drive be used as swap?

I've got hardware built up but am still debating NAS4FREE vs FreeNAS.

<comment><USER20> In FreeNAS 9.10, that's configurable via the webUI.

Haven't tried the released version of Corral though, so unsure if it's carried through.

</comment></comment><comment><USER21> The only thing stopping me using FreeNAS was that I can't run CrashPlan on BSD. What are others doing for offsite backups?

<comment><USER5> You can now run CrashPlan in a docker container on Corral. (In 9.x it's available as a plugin.)

<comment><USER21> Awesome! Thanks.

</comment></comment></comment><comment><USER22> Looks like I lost all my backup configs (from windows via iSCSI and from my Macs via time machine) :/

</comment><comment><USER23> Does it support heterogeneous drives?

<comment><USER12> Yes, you can put drives of similar sizes in raid groups ("vdevs") together and then stripe together dissimilar groups. You can also put them all together in one big group, but in that case the group will act as though all the drives are the size of the smallest one and waste the rest of the space.

Example disks:

* 2x 500GB

* 4x 1TB

Example of first option:

* Mirror of 500GB: 500GB

* RAID-Z of 1TB: 3TB

* Total Usable Storage: 3.5TB

* Total Overhead (Redundancy): 1.5TB

* Pool fails if any two drives of the same size are lost

Example of second option:

* RAID-Z1 of all: 2.5TB (Effectively RAID-Z1 of 6x500TB)

* Total Usable Storage: 2.5TB

* Total Overhead (Redundancy): 500TB

* Total Wasted: 2TB

* Pool fails if any two drives are lost

Terminology/facts in case any reader doesn't know:

* A mirror is what you'd think, basically RAID 1.

* A RAID-Z(1) is a RAID group with a single disk worth of redundancy, like RAID 5.

* RAID-Z2 and RAID-Z3 are two- and three-disk redundancy groups respectively.

* ZFS can stripe together any number of similar or dissimilar groups of drives, but if any one such group is lost the entire pool is corrupted.

Edited for formatting

Edit Edit: These behaviors are due to ZFS, not choices made by the FreeNAS developers.

</comment></comment><comment><USER24> Awesome.. I moved away from FreeNAS to Ubuntu 16.04 ZFS.. Now, back to FreeNAS..

</comment><comment><USER25> Damn nice. Can it import btrfs volumes?

<comment><USER26> You might want to check out Rockstor - it's basically the same thing except with Linux and btrfs.

[http://rockstor.com/](http://rockstor.com/)

<comment><USER25> I was on Rockstor before - but the number of "rock-ons" (approved docker images) was too low, and raid5 btrfs comes with a giant warning, so I swithed to FreeNAS Corral. It's brilliant. More a beta than the name suggests, but a really impressive piece of work.

<comment><USER26> Hah, that's funny, I tried FreeNAS initially (an earlier version, though) and gave up because it didn't like some of my hardware.

I'll grant you that raid5 is unstable and the "official" selection of rock-ons is pretty limited. FWIW, adding your own rock-ons isn't very hard - <USER2> drop a json file in a folder and refresh the page. They also accept PRs, so I've added a couple to the official list myself: [https://github.com/rockstor/rockon- registry](https://github.com/rockstor/rockon-registry)

</comment></comment></comment></comment><comment><USER27> oh, shiny. I wonder if there is a way to make this run on my Synology...

</comment><comment><USER28> For any future readers / searchers, it looks like the link was changed [0].

[0] [http://www.freenas.org/blog/introducing-freenas-corral-an- op...](http://www.freenas.org/blog/introducing-freenas-corral-an-open-source- hyper-converged-storage-platform/)

<comment><USER29> Thanks. Updated from [http://www.freenas.org/blog/introducing-freenas-corral- world...](http://www.freenas.org/blog/introducing-freenas-corral-worlds-first- open-source-hyper-converged-storage-platform/).

</comment></comment><comment><USER30> According to the release notes, you can update to Corral and roll back to 9.10 via the GUI.

The system requirements are still 8gb RAM and 8gb disk minimum.

There are no details of the performance.

Release notes are here.

[https://download.freenas.org/Corral/RELEASE/ReleaseNotes.txt](https://download.freenas.org/Corral/RELEASE/ReleaseNotes.txt)</comment>


$100k to move your startup to Austin, TX - <AUTHOR> http://movetoaustin.capitalfactory.com <comment><USER2> The real headline of the article is "Five entrepreneurs get free plane tickets to pitch for $100,000 in seed funding from Austin's top angel investors"

</comment><comment><USER3> Also terrible traffic!

</comment><comment><USER4> *a chance at 100k.</comment>


5D chess game where pieces time travel - <AUTHOR> https://www.5dchesswithmultiversetimetravel.com/ <comment><USER3> Previously discussed: [https://news.ycombinator.com/item?id=24054313](https://news.ycombinator.com/item?id=24054313)

</comment><comment><USER4> It's fun and was worth buying, but I'm pretty sure it's 4D. In the in game manual, it even says that there is an unused spacial dimension in the in-game rulebook.

First, you have the 2 dimensions that pieces can move in in regular chess. The third dimension is the fact that pieces can move back in time a certain amount of boards depending on the move, creating a different timeline. The fourth dimension is how pieces can move into different timelines using one their movement dimensions.

<comment><USER5> I think it's 5D because it just skips the "our" 3rd dimension. Just because Chess movements are in a 2D space, doesn't make the 3rd dimension temporal. You can move in 2 spatial dimensions and 1 temporal one (the 4th), but when you move through the temporal one you create a split in the 4th one...

"5 - a split" [https://www.youtube.com/watch?v=0ca4miMMaCE](https://www.youtube.com/watch?v=0ca4miMMaCE)

Of course I'm assuming you're all playing the game in the same universe I am!

In other words - the game USES 4 dimensions, but is PLAYED in 5D.

<comment><USER6> One could argue that traditional Chess still has 3 dimensions since knights wouldn't be able to "jump" in true 2-space.

<comment><USER7> You can even play chess without any physical or digital 2D or 3D board. Blindfold Chess is a thing. The chess board in my mind is 3D, but this may vary from person to person. Game position and moves can be stored in 1D, just text. [https://en.m.wikipedia.org/wiki/Portable_Game_Notation](https://en.m.wikipedia.org/wiki/Portable_Game_Notation)

<comment> munk-a Eeeh I'm not buying 1D - by the same logic all shapes can be stored in 1D by writing their descriptive formulae in a linear series of text.

<comment><USER7> Well you cannot play football, because this requires physical activity. But I think, yes, for games that just require 1D information exchange and thought, that's all it needs. Any 2D, 3D, 4D representation of that 1D information is just optional e.g. for convenience reasons, because people like to move around physical 3D chess pieces instead of having to remember chess positions and playing blindfold.

<comment> munk-a I can precisely record football in one dimension - in fact some folks over at EA have been making a whole bunch of money simulating football using a linear data stream for quite a number of years. If you dislike the simulation then bear in mind that most people who "observe football" are observing a 1D data stream that's being read in to cause some photons to bounce around.

Saying that you can describe something in X dimensions doesn't make that thing an X dimensional object - or well it might depending on how you define dimensionality, but if you do it that way then everything is 1 dimensional.

<comment><USER7> I mean the difference is that I can actually play (not only observe) chess with just 1D representation of the game. Whereas football requires you to act physically in 3D space. But you are right, I would also consider "virtual football" e.g. simulated by a computer as 1D.

<comment> munk-a I disagree - you can play a game with the same rules as chess in 1D but you can't move your pawn forward on the board in 1D. Chess is a cerebral game so it's easier to abstract it down to entirely be about the will of the movements but those movements aren't committed (in a normal game of chess) until the pieces are actually moved.

So I think you can tactically play football and chess equivalents in 1D, but the actual movements required to execute the game still require three dimensions and Chess remains a game that is pretty naturally aligned with the requirements of two dimensional space.

This is very pedantic at this point but I think the naive and natural assumptions about game dimensionality are probably the most correct to stick with. They can be argued up and down (absolutely no game can actually be played in less dimensions than we exist in since we only exist in so many dimensions - we might not need to exercise them to play the game but a chess piece, a computer or any existing encoding of data that we have today requires representation in 3 (or however many you think we have) dimensions - <USER2> pretty much everything (everything, I think?) can be described in data that can be expressed linearly so all games and the entire world can be said to be 1D. I think the fair middle ground is to try and adhere to the meaningful definition of dimensions in the middle that actually lets us classify things differently instead of saying that everything in existence is both 1D and multi-dimensional.

<comment><USER7> Hmm. okay I understand that you still disagree but I don't get your point.

> So I think you can tactically play football and chess equivalents in 1D, but > the actual movements required to execute the game still require three > dimensions

I don't know what you mean by tactically. I never played football in 1D in my life and I think no one ever has, except if you are referring to a computer game or computer simulation. But I have played blindfold games without a chessboard and without a computer.

> but you can't move your pawn forward on the board in 1D

you don't need a board to play chess. A chess board be it in 2D representation on a computer screen or in 3D in real life is just a optional representation of the chess game, but you don't need it.

> a computer or any existing encoding of data that we have today requires > representation in 3 (or however many you think we have)

I think we most likely have (according to theories that are mathematically sound) four non-compact dimension and seven that are curled up, so 11, but obviously there's no proof for that.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> I've enjoyed Alice Chess[1], which is played with two boards and on each move a piece on one board goes "through the looking glass" to wind up on the corresponding position on the other board.

Also fun was Suicide Chess[2], where the winner is the first to get all their own pieces captured.

About 20 years ago, I found a book in The Strand which had collected something like 300 chess variants... I'm sure by now there are many more.

[1] - [https://en.wikipedia.org/wiki/Alice_chess](https://en.wikipedia.org/wiki/Alice_chess)

[2] - [https://en.wikipedia.org/wiki/Losing_chess](https://en.wikipedia.org/wiki/Losing_chess)

<comment><USER9> Within 5d chess, there's a mode called "Timeline invasion". In this mode, there are 2x 5x5 boards. On one board, you have BRKBN and five pawns, and I only have five pawns. On the other board, I have BRKBN and five pawns and you only have five pawns. These two boards are parallel dimensions that then follow the rest of the rules.

</comment></comment><comment><USER10> This is a weird little game - regular checkmates are often not, because the king can just escape back in time or across timelines, but threatening a king's past self is pretty easy - it can't escape having been where it was. And you only need one checkmate to win, because you have to make a move on all boards where it's your turn (even if you have no pieces there - you have to move one there!).

You can also give check with a king, because of the quirk that only "the present" can threaten check. At least, I think that's why it works.

</comment><comment><USER11> Quite an interesting puzzle game, but it's not all that similar to chess given the extra dimension. This means that a knight might move two spaces forward and one back in time: a lot of patterns you may be used to seeing & responding to in normal chess will need to be rethought or thrown out.

With that in mind it is a lot of fun to figure out new strategies and patterns!

</comment><comment><USER12> The alpha go team is working with Chess grandmasters to determine which variations to the rules of chess result in a larger state space with more complexity.

They believe this will allow them to find a game that has more space for human creativity (since most games now are very constrained in the first 40 moves or so).

Would be cool to submit this game to see where it falls :)

</comment><comment><USER13> Damnit, now Neal Stephenson is gonna roll a new crypto standard based on this

</comment><comment><AUTHOR> [https://en.m.wikipedia.org/wiki/5D_Chess_with_Multiverse_Tim...](https://en.m.wikipedia.org/wiki/5D_Chess_with_Multiverse_Time_Travel)

<comment><USER14> Non-mobile link:

[https://en.wikipedia.org/wiki/5D_Chess_with_Multiverse_Time_...](https://en.wikipedia.org/wiki/5D_Chess_with_Multiverse_Time_Travel)

</comment></comment><comment><USER15> The press kit section of the page says it has AI opponents. At first this didn't seem remarkable to me since it's a standard feature of most chess games.

But, how would that work? Would they have to create a new AI implementation? An off-the-shelf chess AI presumably doesn't understand the best strategies for multiverses. Unless there is some trick to apply an off-the-shelf AI to this.

<comment><USER16> Off the shelf techniques for zero sum games of perfect knowledge would work though. Minimax and Monte Carlo Tree Search and reinforcement learning from self-play. Leela has been adopted to play both go and chess, I suspect it would be practical to adopt it to play any game.

</comment></comment><comment><USER17> I guess that's the Time Lords' chess variant. /jk

This is interesting but it will be nice if a short description was on site.

<comment><USER18> No, apparently timelords just play the Towers of Hanoi.

[https://www.bbc.co.uk/programmes/articles/3DHlYgfSdY2RlXkS2K...](https://www.bbc.co.uk/programmes/articles/3DHlYgfSdY2RlXkS2KwjtKK/the- fourth- dimension#:~:text=THE%20CURSED%20GAME,the%20completion%20of%20the%20serial).

</comment></comment><comment><USER19> Yeah, I is totally smart enough to play this.

</comment><comment><USER20> This marks the first repost I've ever seen on HN, having been using it for nearly seven years. Not bad going!

</comment><comment><USER21> It's only 5D if you are allowed +5 to -5 moves each move.

Your result in 3D would always be an approximate

</comment><comment><USER22> Y'all are not going to grok this, but time is not a dimension.

<comment><USER23> In this context it is, it's defined as the bi-directional line between game state transitions, and can be measured quantitively like any other spatial dimension.

I'm talking specifically about time in the context of this game, which is what this thread is about, not time in the universe.

</comment></comment><comment><USER24> This is just the same as playing the computer and hitting "Undo" when you get screwed until you find where it all went wrong. Then you play that new timeline and can "Undo" again to create another timeline.

This is a cool way to visualize "Undo" in chess.

<comment><USER9> "Undo chess" is a single timeline that a (human) player can rewind to a previous board state. Movements in time are constrained to moving the board state. Pieces cannot move independently of the board state in time. Additionally, you still play all the timelines that have been created.

In 5D chess, the knight moves two spaces in one dimension and one space in another. In a 2d board, this is the description of how a knight moves. In 5d chess, a knight can move one space back in time and two spaces in the Y dimension on the board two one space back in time. That is fundamentally different than Undo chess.

Additionally, this puts a new knight on the board timeline that is forked off of the previous move - so there are now three knights.

Lastly, as pieces can move through time as well as space, one of the standard wins is a piece that is attacking the king in the past. I had a game where I moved the queen to the same file as the king (which was currently protected) - it was five spaces away. I moved another piece, and then won as the queen was now attacking the king five turns in the past. The timeline for that board is [https://slack-files.com/T0VD96U9G-F01A45YRLTG-cf2d776bd3](https://slack- files.com/T0VD96U9G-F01A45YRLTG-cf2d776bd3) . You can see that there is no piece at [T-1, Y+1], [T-2, Y+2], [T-3, Y+3], ... until the king at [T-5, Y+5].</comment></comment>


Understanding Disk Usage in Linux - <AUTHOR> https://ownyourbits.com/2018/05/02/understanding-disk-usage-in-linux/ <comment><USER2> Website seems to be hugged to death, here's an archived copy: [https://web.archive.org/web/20190429214402/https://ownyourbi...](https://web.archive.org/web/20190429214402/https://ownyourbits.com/2018/05/02/understanding- disk-usage-in-linux/)

The article is well worth reading as it's reasonably comprehensive, including a small foray into COW filesystem.

<comment><USER3> Is that a reference to Hug Bot? [https://pbfcomics.com/comics/hug- bot/](https://pbfcomics.com/comics/hug-bot/)

<comment><USER4> Not sure, but I always thought "hug of death" came from reddit:

[https://en.wikipedia.org/wiki/Slashdot_effect](https://en.wikipedia.org/wiki/Slashdot_effect)

<comment><USER5> Previously, that was called "slashdotting".

(Knowing such things makes me feel kind of old).

</comment></comment></comment></comment><comment><USER6> One feature not mentioned in this article is that BTRFS (and some other file systems) supports transparently compressing files. It's another (very important) reason why the actual number of bits the file contains might not be how much of your disk space the file is actually using.

</comment><comment><USER7> To show extents - including sharing - of a file you can use the tool _filefrag -v_ , it's more general than _btrfs filesystem du_ since it uses the fiemap and fibmap ioctls that are supported by multiple, but not all, filesystems.

</comment><comment><USER8> Related: is it possible to reliably maintain physical disk space quotas in Linux (similar to cgroups)?

Furthermore, is it possible to say how much "space" you would use if you were to create a file with a given size, accounting for block-size, fragmentation, and metadata? Matters such as block-size, inode usage, and metadata seem to make this very difficult even if you add special integration to the userspace application, for example by using stat or statfs. This could help prevent quota overruns for example.

These seem like hard problems unfortunately, and I suspect the best solution is to just create separate disk partitions for each quota group.

<comment><USER9> First question: Quotas have been supported on Linux for a very long time. All major (and native) file systems support them.

Secondly: disk usage accounting for metadata as well as regular file data may or may not be tricky. ZFS always tells you how much data+metadata is used by a file, helped by metadata itself being dynamically allocated on ZFS like everything else is. File systems like ext4 that have fixed metadata locations on disk don't report back metadata allocation with the file; it wouldn't really be useful to see this information since removing the file doesn't free any metadata in the ext4 case.

<comment><USER10> Project quotas appear to be more similar to cgroups. They are available in xfs and ext4 [https://lwn.net/Articles/623835/](https://lwn.net/Articles/623835/)

<comment><USER11> The traditional 4.2BSD-style quotas (1983!) on linux also support quotas on unix groups. Not sure if you had this in mind, but anyway.

I suppose project quotas as outlined here would allow multi-group support though.

Another option could be sparsely-provisioned COW LVM volumes.

</comment></comment></comment></comment><comment><USER12> Questions about "cp --reflink" (I have never used that option so far, but it sounds useful).

Quoting "man":

 _" When --reflink[=always] is specified, perform a lightweight copy, where the data blocks are copied only when modified. If this is not possible the copy fails"_

Q1: this is copy-on-write, right?

Q2: once/if the command completes successfully, are there any (potential) dangers (e.g. if I then immediately delete the original file) or can the new file be treated 100% as if I had copied it the classical way (without that option)?

Thx

<comment><USER13> 1) Yes, it's copy on write, and requires such support from the underlying filesystem (Btrfs, maybe XFS not sure of what others support it)

2) It's intended to be treated 100% as if it had been copied the classical way. It's not a hardlink or a symlink and can be treated as a completely new file

<comment><USER14> Support for reflinks in XFS for a while now, although it requires a mkfs time option to enable it. That option was just enabled by default in xfsprogs-5.1.0 which is now in Fedora 31 (still prerelease).

<comment><USER12> Cool => I'll then probably reformat on my root server the partition hosting the images of my VMs from EXT4 to XFS => then to make 100% backups from time to time I would just have to shut them down for a minute, do a "cp --reflink" of their image-file on the host/dom0, then I could start them up again and do anytime later the slow download of their "copied" file.

Any recommendations about a utility (working on a layer higher than the filesystem) that can synchronize only the chunks of a specific file that has changed since the last sync and which generates as final result a "normal" file (no behind-the-curtains database - just copy the identical data from the local/old file and the changes/new data from the remote location to get a normal up-to-date local file)?

<comment><USER15> I believe rsync is the utility you are looking for.

<comment><USER12> Damn, now I'm ashamed of myself - it's since forever that I'm using rsync... :)

I think that when I tested this with rsync a long time back I did a similar mistake like this one [https://stackoverflow.com/questions/28819379/block- level-cop...](https://stackoverflow.com/questions/28819379/block-level- copying-and-rsync) (doing a sync not over a network) and/or this one [https://www.reddit.com/r/linux/comments/3nhx0p/rsync_block_l...](https://www.reddit.com/r/linux/comments/3nhx0p/rsync_block_level_incremental_backup/) (doing a sync by adding/removing some bytes instead of just changing their values, as it would happen in the preallocated img-file of the VMs).

I'll test it again - thanks a lot for the info!!

</comment></comment></comment></comment></comment></comment><comment><USER16> add (2018)</comment>