Show HN: CoinMarketBook – CoinMarketCap, but with a different metric - <AUTHOR> https://coinmarketbook.cc/ <comment><USER3> One of the problems with this metric is it's easy to manipulate. Exchanges can easily lie about their book depth and/or people can put in lowball buy orders they have no intention of ever filling. To find out if those buy orders are real you would have to purchase them _and all the orders above them_ , which would be very expensive and move the price a lot.

On the other hand, the instantaneous price of a coin is easy to verify. Everyone who buys or sells at the exchange can verify it, so exchanges can't easily lie about it.

Here's an interesting solution for ranking exchanges by their order book depth in a way that's more difficult to manipulate. Perhaps it could be adapted to rank coins. [https://bitcoinity.org/markets/rank_explanation](https://bitcoinity.org/markets/rank_explanation)

<comment><AUTHOR> Amazing idea - will implement their algorithm for all coins.

</comment></comment><comment><USER4> Unfortunately as a metric this is far too easy to game and is heavily influenced by algorithmic order regimes. Maybe decent as signal in trading strategies but not useful for real fundamental analysis. Market makers create phantom interest all the time to try and scare the market through stop loss orders.

Also why not focus on the selling interest too? Any one sided metric is even less trustworthy.

<comment><AUTHOR> Yes, you're right - it's possible to place fake orders and cancel them as soon as the actual trades start happening near them. However, from my experience, it only happens during the pump (when the whales are pushing the price higher).

You can see the explanation for including all buy orders here: [https://news.ycombinator.com/item?id=18162456](https://news.ycombinator.com/item?id=18162456)

Sell pressure is a bit different: while the lowest price is zero, the highest price is unlimited (technically, there is a limit enforced by exchange engine, but it's very high anyway). So actually, sell pressure should be calculated in base currency (in coin itself) instead of USD.

</comment></comment><comment><USER5> There is $1.5T of outstanding buy orders for BTC right now? That seems inflated.

<comment><AUTHOR> Last time I checked, the orders were actually there (based on API responses).

Please note that some markets may allow leverage - we'll mark them with asterisk on next deploy.

<comment><USER6> What is one really to do with the metrics? Right now, HOT has the highest support/cap % for a lesser known coin. Does it mean to buy HOT?...

<comment><AUTHOR> In crypto, it's always better to do your own research instead of following buy recommendations. That said, you can use the numbers in your research: for example, if two coins have a similar market cap, but one has much higher buy support, then it might be a better investment.

Buy support shows demand for this coin.

</comment></comment></comment></comment><comment><USER7> As someone who is not very financially orientated, I find this difficult. Firstly because I don't really have a feeling for what that % _means_ \- presumably the ratio of market cap to book depth gives an indication of how sensitive the market cap is to large orders. So it combines liquidity and total volume - but I don't know why you would combine them into one statistic? Either I want a single number which means something clear, which this fails on. Or I want more information, in which case isn't it just better to tell the market cap AND book depth?

Also whilst market cap is easy to measure, book depth seems more difficult. How far away from the touch do you count? A million dollar offer which values bitcoin at 3cents represents almost no information. Firstly because by the time the price drops there's no way that offer will still be good and secondly because even if that offer were still good that change in price would represent a total shift in the market.

Also, this seems to compare apples to oranges, because some coins will contain market makers who are likely to leave orders in the book and cancel in response to signals, thereby creating phantom liquidity whilst others won't have enough volume to attract market makers and so all the depth is real.

<comment><AUTHOR> Ratio (%) is calculated as "Buy support / Market cap". You can also see separate metrics for both "Buy support" and "Market cap" in the same row.

We count all buy orders (on all prices). It's true that low-hanging orders might be cancelled before they are executed. We've decided to count them anyway, because even low orders provide support during flash crashes.

Flash crash horror movie: [https://youtu.be/R7WfSTePuA0?t=171](https://youtu.be/R7WfSTePuA0?t=171)

This was possible because there wasn't much buy support. And that's ETH - imagine what can happen with lowcap altcoins.

P.S. The fact that some altcoins don't even have market makers is a little... unsettling, don't you think?

</comment></comment><comment><AUTHOR> With CoinMarketBook, you can evaluate cryptocurrencies by a different metric: "buy support".

Buy support is the sum of buy orders in USD equivalent from top 20 exchanges (planning to include more in future). It shows the actual amount that people want to buy.

One might say that it's easy to place a large buy order at very low price & increase buy support artificially. However, because of low price, the increase will be negligible.

What do you think? Tell us in comments.

<comment><USER8> So do you suggest that buy (and sell) orders on Bitfinex, for example, are genuine and there's no wash trading, no other tricks to make people believe market will go one way or another? LOL.

<comment><AUTHOR> _Real_ wash trading happens without orders.

As for manipulative orders - you're right, they flash in the orderbook on and off. However, when I needed to exit, I actually filled a couple of them - <USER2> they were real.

</comment></comment></comment><comment><USER9> There aren't many fiat trading pairs for smaller coins. That means, if people are holding fiat, they most probably put it on low buy order for Bitcoin. I'd like to see exactly which markets are calculated in the results.

<comment><AUTHOR> Sure, we'll add the markets in a tooltip on next deploy.

</comment></comment><comment><USER10> > Buy support is the sum of buy orders from top 20 exchanges. > This is the > actual amount that people want to buy.

Curious how you have arrived at $1.5T of buy support for BTC?

Given an example, a $1 bid on 100 BTC equates to $100 of buy support. In this example, would you count this as 100 * current price?

For reference, my custom tool reports BTCUSD pairs across major exchanges and shows approx 40kBTC of liquidity 10% either side of the current price. This is approximately $0.25B.

<comment><AUTHOR> You're right, $1 bid on 100 BTC = $100 of buy support. We include it into the metric as "100" (without multiplying by current price).

So your tool counts the closest 10%? Would be interesting to expand it to 100% and compare the metric.

</comment></comment><comment><USER11> I look at Daily Issuance (last column in [https://onchainfx.com/v/SMT45r](https://onchainfx.com/v/SMT45r)) as a measure of how well a (mineable) coin is doing, as this reflects the cost of energy spent on mining that coin every day.

This is one of the very few measures that is not easily faked...

<comment><AUTHOR> Interesting metric - noted.

</comment></comment><comment><USER12> \- What does support/cap tell us? I see that it's a "gamble vs investment" metric but I don't understand the logic behind that statement.

\- Is buy support at any level? As in, there's hug buy support for bitcoin, but is this counting people who have buy orders at the $6500 range all the way down to the $1 range?

<comment><AUTHOR> \- It shows you how much of market cap is backed by actual buy orders (~ whether it's vapor or not). The more buy support, the higher the probability that current price will hold (discounting for manipulative orders, of course).

\- Yes, we sum buy support at all levels. You can see the explanation here: [https://news.ycombinator.com/item?id=18162456](https://news.ycombinator.com/item?id=18162456)

</comment></comment><comment><USER13> Whatever that validity of the metric itself, it's an interesting idea that if a market is different "enough", standard market metrics fall short of their goal.

On the website design, a fixed-width font would be helpful for the numbers. Currently, ETH at $22,898,033,476 looks one digit larger than BTC at $113,768,712,510.

<comment><AUTHOR> Thanks for the comment.

Good idea, we'll switch to fixed width.

</comment></comment><comment><AUTHOR> Just fixed an important bug in calculation: [https://medium.com/@dengorbachev/coinmarketbook-bugfix- new-m...](https://medium.com/@dengorbachev/coinmarketbook-bugfix-new- metric-665a786b1dbd)

</comment><comment><USER14> Market cap isn't perfect by any means, but if there was a better metric that could be used to get a quick valuation of something then the stock markets would already use it.

</comment><comment><USER15> How did you get the data from which exchanges?

<comment><AUTHOR> Currently, it's `'bitmex', 'binance', 'bithumb', 'bitfinex', 'okex', 'huobipro', 'bittrex', 'poloniex', 'kucoin', 'cryptopia'` (planned to include top 20, just fixed the message on home page)

<comment><USER16> The general idea is awesome. Excited about ranking using the bitcoinity metric mentioned by the top comment.

Any plans to include the top decentralized exchanges? ([https://etherscan.io/stat/dextracker](https://etherscan.io/stat/dextracker))

</comment></comment></comment><comment><USER17> What does the "Tell Friends" column mean in the table? Was it supposed to be a metric or just a placeholder?

<comment><AUTHOR> Just a placeholder. We were looking to reach wider audience with this message (difference between market cap & buy support).

Do you have any advice on how to encourage sharing?

</comment></comment><comment><USER18> Can the list not be sorted on other columns or are my mobile browsers failing me? (Firefox and Chrome on Android)

<comment><AUTHOR> Added sorting, thanks for suggesting it.

<comment><USER18> Thanks!

</comment></comment></comment><comment><USER19> An alternative to the 24 hour price change would be a price change since a specific date.

<comment><AUTHOR> Interesting. Could you please explain why you needed that?

<comment><USER19> I'm just interested in the value movement relative to my investment - and as I did not invest 24 hours ago, the information does not help me much.</comment></comment></comment>


Ask Hackers: How to solve the Wifi dilemma? - <AUTHOR>

With all the urban WiFi projects getting shelved: Chicago Drops Expensive WiFi Project http://www.tmcnet.com/wifirevolution/articles/10205-chicago-...
S.F. citywide Wi-Fi plan fizzles as provider backs off http://www.sfgate.com/cgi-bin/article.cgi?f=/c/a/2007/08/30/...
Anybody have any ideas how to create a breakthrough solution? Seems there are numerous supply of wifi connections, but they all need passcodes.
I want to be able log on to any WiFi in San Francisco even if I don't have the passcode... I don't want to use a hacker tool, and I want it to be fair to everyone.  What's the problem with doing that? <comment><USER2> The problem is that it doesnt work. I installed wifi in large NYC apt buildings for a long time. WiFi just doesnt work as reliably as wireline solutions.

Here in NYC, I cannot even use my own wifi network from more than 15 feet away because, the entire spectrum is getting stomped on from everyone else around me.

If your main goal is blowing away tax dollars to consultants, municipal wifi is awesome. If you just want to connect to the internet get an EVDO card.

<comment><USER3> Have you tried 802.11a ? Same speed, more bandwidth, less interference.

</comment></comment><comment><USER4> WiFi isnt the right technology. Wifi is for homes and offices. Wait for WiMax which is perfect for Towns and Cities.

If you cant wait for wimax get a 3G phone and use it as a modem.

</comment><comment><USER5> We could get everyone to sign up to <http://www.fon.com/en/> or some similar service.

<comment><AUTHOR> hmm, buy new hardware :-/

<comment><USER5> If you look around on the site there is free software to turn a linux or mac box with a free wifi card into a hotspot (with simple TUN/TAP).

If the new linksys routers weren't so crippled compared to the good old WRT54G V4, you could probably work out a firmware solution, which would open it up to anyone able to flash.

There is a startup idea for someone that wants it. Create a firmware solution for FON... or create your own better solution.

<comment><USER6> Linksys actually made a special model, the WRT54GL, that remains flashable.</comment></comment></comment></comment>


Airports would be 300% less soul-crushing if they had libraries - <AUTHOR> https://twitter.com/AstroKatie/status/1118543895502082048 <comment><USER2> Amsterdam-Schiphol has a library[0] and it's pretty awesome!

[0] - [https://www.pri.org/stories/2011-02-16/library-amsterdam- air...](https://www.pri.org/stories/2011-02-16/library-amsterdam-airport- schiphol)

</comment><comment><USER3> Taipei Taoyuan International Airport (TPE) has libraries.

</comment><comment><USER4> I actually had the same thought but with movie theatres. I think movie theatres would be highly effective as they could tightly pack many people into a room that they could not have otherwise. I feel as if they could generate profit and easily prevent boredom in the airport.

I sure wouldn't say "no" to a library either though.

</comment><comment><USER5> I remember airports before

The first round of hijackings in the early 70's. About 1000% less soul crushing

And before deregulation. About 500% less soul crushing

And before the TSA. A slaughter house is less soul crushing even if you are the cow.

</comment><comment><USER6> Why not carry a Kindle or a book that you want to read?

<comment><USER7> The entire thread was full of reasonable people reasonably suggesting things like this and the OP being sarcastic and dismissive in response, guess she _really_ wants those libraries

<comment><USER8> It's not just about the books, it's about the experience.

"Why take your kids to Disney World when you can just put Frozen on".

</comment></comment></comment><comment><USER9> I have an ereader</comment>


Code-Switching to Improve Your Writing and Productivity - <AUTHOR> https://chroniclevitae.com/news/1242-code-switching-to-improve-your-writing-and-productivity <comment><USER2> I have found that a similar technique helps with coding as well. If you are debugging some issue and it getting bogged down in the details, take a break by working on a side project that is less technically intense, maybe in a different language. Or do some devops work and write shell scripts or develop a Dockerfile. Or write an automated testing tool, or add unit testing for Bash scripts or refactor an overly complex SQL query by improving the joins and using 'with' to reduce the use of select sub queries.

The point is that you are still doing useful work, and therefore are "productive" but your conscious mind is now focused on a completely different task. While you do that, the subconscious is still working on the thorny debugging problem and when you get back to it, you will find it easier to choose the next steps that break through the chaos and lead to the root cause of the problem.

This is important even if you are in crunch mode because of a deadline. Usually what happens is that insane focus on fixing an important problem leads you into a long chain of insane fixes that mask the real problem. Eventually the scotch tape and rubber band fixes collapse anyway. Better to manage your mind so that you have a chance to reach full understanding of the problem so that you can make the RIGHT fix that is stable and longterm.

<comment><USER3> This is an approach that many are known to use - for example, in the autobiography "Surely you must be joking, Mr. Feynman!", Feynman notes that he keeps about a dozen problems in the back of his head, and sometimes he'll get a flash of inspiration on one & solve it, or come back to it and investigate further & make more progress.

This approach transcends disciplines I've found, although I've rarely had to apply it to coding.

</comment></comment><comment><USER4> John Carmack said something similar on another HN post a few days ago: [https://news.ycombinator.com/item?id=10845832](https://news.ycombinator.com/item?id=10845832)

 _" Whatever the grand strategy for success is, it gets broken down into lots of smaller tasks. When you hit a wall on one task, you could say "that's it, I'm done for the day" and head home, or you could switch over to something else that has a different rhythm and get more accomplished."_

</comment><comment><USER5> I thought this was going to be about linguistics code-switching:

[https://en.wikipedia.org/wiki/Code- switching](https://en.wikipedia.org/wiki/Code-switching)

I also love this kind of code-switching, y me gusta poder hacerlo all the frickin' time.

<comment><USER6> haha, pensaba lo mismo. Agree with you too, its just so easy cambiar entre los dos idiomas, but I do wonder sometimes q piensa la gente when they hear code- switching conversations.

</comment></comment><comment><USER7> When I take breaks from writing, I look at pictures of art or cities (in books). I find reading text isn't different enough for a true break.

I'm going to try this switching tactic generally. I had a project last year where I got stuck because I kept working at it while hitting too many tough points.

</comment><comment><USER8> This is exactly what we do in school. Maybe that's why it work. First you do math, then languages, then religion etc. You never work on the same thing for a whole day in school.</comment>


Ask HN: a standard microblogging format? - <AUTHOR>

What does HN think about a standard microblogging format, that both clients (like Twitter apps) and providers (like Twitter and Facebook) could implement? It could include some mechanism of universally qualified usernames (like @aplusk would translate into twitter://aplusk), and maybe some provisions for realtime search.<p>This way, I could get use one application for receiving both Facebook and Twitter updates, and have an improved user experience. This could really be a good move for Facebook; their status updates would become a push technology, instead of pull as it is now. It would also fight against Twitter's hegemony. <comment><USER2> Check out <http://identi.ca/doc/openmublog></comment>


Comments on female-presented HP review: "Dumb whore", "Make me a sammich", etc. - <AUTHOR> http://www.youtube.com/watch?v=m1SzN_q3r40&feature=player_embedded

<comment><USER2> Welcome to the wonderful world of youtube comments, where kids and inbreeds meet to discuss their thoughts.

<comment><USER3> I find it somewhat scary by the amount of people that don't know how to express their feelings. "Dumb whore" - how does this relate to the video? The review is spot on and one of the few that focus on the weaknesses of the TouchPad instead of glorifying it's potential. HP (like Microsoft with Windows Phone) has a lot of catching up to do.</comment></comment>


In a Blog Post Google Recognizes Jerusalem as Part of Israel, Do No Evil?  - <AUTHOR> http://arabcrunch.com/2012/04/google-recognizes-jerusalem-as-part-of-israel-adds-parts-of-the-old-city-to-street-view.html

<comment><AUTHOR> Do you agree with Google or should it have taken a neutral view?</comment>


Launch HN: Tech Moral Compass, a community for techies who care about morals - <AUTHOR> https://www.soapboxproject.org/tech-moral-compass <comment><USER2> Talks about "moral compass" and then tries to load facebook tracking and google analytics ...</comment>


US Eyes Building Nuclear Power Plants for Moon and Mars - <AUTHOR> https://www.usnews.com/news/politics/articles/2020-07-24/us-eyes-building-nuclear-power-plants-for-moon-and-mars <comment><USER2> The need for nuclear energy on moon is desperate.

On the Earth it is a big annoyance that solar energy is available half of the time, but because the Earth rotates in 24 hours and has an atmosphere, it easy to cope with the effects.

On the moon the night lasts two weeks so you need vastly more storage to ride out the night than you do on the Earth. Probably people will live under a few meters of dirt which will provide insulation, but it seems likely that a lunar factory won't run continuously. It would be nice to have a nuclear reactor at a moon base so there is a comfortable margin of energy supply in the night.

(A counter to that is that there are some spots near the poles which see the sun always and also are near possible volatile CHON deposits that could be game-changing)

Contrast that to full in-space manufacturing, say a solar sail factory based at an asteroid. There your solar array points at the sun 24-7 so potentially you can run the factory at more than twice the throughput and pay back the capital investment more quickly.

</comment><comment><USER3> So can we expect to live on moon or mars in very near future? Honestly, how long will it take? Is it really possible?

<comment><USER2> If people were interested in colonizing the Moon that could be done by 2030 or earlier.

If, for instance, Musk got his starships to work you would probably land one on the moon with a bulldozer, turn it on its side and bury it, and that is the beginning of a moon habitat. It seems crazy to me that one would want to go to mars before getting boots on the ground highly comfortable on the moon. Martians would be so vulnerable to tech failures that they should have confidence that people live a boring life with the same kit they use on the moon.

The problem though is the motivation. If Starship was working, for instance, you could ship hydrogen fuel up to earth but produce the oxidizer on the moon and ship it up with a mass driver. This is not the miles-long device that Heinlein and O'Neill dreamed about but something like a scaled-up battleship gun. (e.g. the US Navy can buy a lunar escape velocity railgun today)

You could catch lunar oxygen near the Earth-Sun L2 point if you have a 2.5 km/s gun, with a 3.5 km/s gun and active guidance you could graze oxygen carriers against the Earth's atmosphere to land liquid O2 at LEO where it could refuel starships.

The byproduct of that lunar oxygen is likely to be iron. Astronauts dug up hematite ore as good as anything you would find on Earth -- in Science Fiction space economies are often aluminum based until iron asteroids are tapped, but the path to lunar iron+oxygen is short compared to Al.

Studies on the economics of Lunar LOX vs LOX from Earth don't show a consistent benefit from moon mining if you want the fuel in order to go to Mars.

Personally I would skip Mars and aim for exploitation of asteroids. As you move away from the sun you get more tarry gunk mixed in with your sand; it is like Saudi Arabia but instead of a little tar in your sand it is a little sand in your tar. Great for making food, plastics, trees, etc. Get out to the orbit of jupiter and you will be looking at 50% ice, methane, carbon monoxide and you'll need to use the Fischer-Tropsch process if you want tarry gunk.</comment></comment>


IBM to pay more than $30m in compensation for Australian census fail - <AUTHOR> http://www.abc.net.au/news/2016-11-25/ibm-to-pay-over-$30m-in-compensation-for-census-fail/8057240 <comment><USER2> Before census day they were predicting two thirds of the 10 million Australian households to complete the census online. They stated their site had been tested with half a million concurrent users and was able to sustain double that.

Unfortunately no one seems to have considered that the majority of users would do it after dinner (say 6-10pm). Take the 6.6 million users divide it by the time frame and it was never going to work with a 1 million per hour capacity.

What is interesting is with IBM paying out they must have failed to deliver the required capacity. I thought it more likely the government had not tendered it properly and requested a lower capacity than needed but obviously IBM just tried to cut costs and under resource the operation.

<comment><USER3> I often wonder why companies and governments even trust large organisations like IBM over individuals they've interviewed themselves. I mean, it's not like IBM's consultant's competence is any secret. IBM is also trying to make it worse, or at least, not making the competence factor any priority. Rather, it's cost over everything:

[http://www.businessinsider.com.au/ibm-layoffs-1-month- severa...](http://www.businessinsider.com.au/ibm-layoffs-1-month- severance-2016-3)

I mean how do you get away with hiring IBM (or accenture, PwC, didata, deloitte or, may God help us, Tata or something like that, where cost of paycheck is literally the only factor in hiring).

<comment><USER4> Such individuals are usually unable to detect real signal in the thing they're hiring for, so interviewing in and of itself doesn't provide them any qualitative information.

They also know the people judging them are in the same boat.

Thus they fall back to the only thing understandable to themselves and people who will judge them: big brand and prestige/shininess. If the project succeeds, you take credit. If it fails, you can't be blamed for "buying the biggest and the best" (even if the biggest and the best are just a bunch of stary-eyed ignorant grads with a salesman/project manager at the head).

/that's the good version. The bad version points out how many of these firms gets business through networks, personal connections and "you scratch my back I'll scratch yours" type dealings. //it's not a rant if it's true.

</comment></comment></comment><comment><USER5> That's a pretty hefty fine, but I wonder what the original billings were for developing the system in the first place?

I've heard reports that they spent >AUD$5Million on just the load testing aspect, but I hope someone can verify that. And I trust that they have improved comprehensive testing methodologies that actually work to improve reliability under duress. (<sarcasm>Tip: Perhaps don't build it using Lotus Notes??</sarcasm>)

I guess we can be thankful that is was 'merely' a census and not something like electronic voting!

<comment><USER6> In the US, Census data is used for redistricting purposes, which CAN influence election outcomes in the future - so the implications of a census may be wider than you think. I don't know exactly the way it works in Australia, but some googling has led me to believe they use a system that is similar.

[https://en.wikipedia.org/wiki/Redistribution_(Australia)](https://en.wikipedia.org/wiki/Redistribution_\(Australia\))

<comment><USER7> Another major problem with the census was the Australian Bureau of Statistics holding onto the data and changing their privacy policies which meant that in an act of protest a ton of people entered false information or ommitted details. A really poor census overall.

</comment></comment></comment><comment><USER8> Just wait until IBM's Phoenix Pay system and the scandal here in Canada comes to a head. With 300,000 civil servants waiting six months to be paid and it's still not fixed.

There must be a big penalty looming for such a massive level of incompetence.

Phoenix seems like a rather awkward name for an application.

[http://www.nytimes.com/2016/11/18/world/canada/government- wo...](http://www.nytimes.com/2016/11/18/world/canada/government-workers- pay.html)

>Though many government employees work in 9-to-5 office jobs, the public payroll also includes a wide array of people like Mr. Ryan with complicated work schedules and pay rules.

My father used to be in the Coast Guard he worked 28 days on and had 28 days off. If you were injured that meant the days you were off were really two days?? Try explain that to some bureaucrat in Ottawa who can thinks they are speaking English (really French), they're 2,000 miles away and doesn't understand the system either.

<comment><USER9> Oh God, that sounds an awful lot like two more Australian systems: the Queensland Health payroll debacle, where a bunch of health workers were paid incorrectly or not at all. Or the National Disability Insurance Scheme debacle, where healthcare providers haven't been paid, resulting in small business owners having to take out huge amounts of personal debt just to pay their employees, and sole traders unable to provide services their clients need to live day to day while still keeping a roof over their own heads. (They're both IBM projects too.)

<comment><USER10> Do you have a link for IBM's mishandling of the NDIS? Interested, but can't find a link.

</comment></comment></comment><comment><USER5> Availability and load balancing aside, did any other Australian citizen (who actually managed to complete the census at all) notice that the UX was quite ordinary? Reminded me of a web form circa 1990.

For instance, IIRC the census asked for my and my wife's country of birth and nationality, but later on when filling in the info for our children, it asked for the nationality of the both parents again. Simple logic checking back through the data should have shown that our (the parent's) information was already supplied, and this information for our children could be pre-filled or skipped and auto filled in the database.

Just a few small things that would have made the filling out of the forms a lot smoother, faster, and less complicated for lay people who were already nervous about doing the data entry.

<comment><USER11> Except you may have step-children, adopted children, foster children, children in the house not part of your family (friends of your child, children of your friends), etc. Keep in mind that the census needs to include everyone who was in the house at the time it was filled out, not everyone who is part of your regular abode/family situation.

You might argue that it would work for the "most common scenario". In which case why not pre-fill all the data to the statistically most common values? (Hint: the census's purpose is to collect those statistics.)

Also, pre-filling might cause confusion as to which fields have already been filled out by the user; if a value is present, they might wrongly assume no action is required on their part.

(Having said all that, I haven't seen the online census.)

<comment><USER5> Sure. I did consider the scenario if the kids had a friend sleeping over that night or something, or if the respondents had adoptive or foster children etc., but I think when I traced back the line of questioning in my case there seemed to be a logical thread link there that could/should have negated asking for the same information again.

I am relying on my memory here from few months ago, so I could be wrong, but I did feel at several points during the session that quite a few questions could quite possibly be chained together in better fashion. Perhaps not so much a failing of the actual site UX designer, but the census analysts who designed the questions in the first place?

<comment><USER4> Not responsible for this census, but I've done work with it in the past. Two observations:

The questions are largely derivative and the same each year: cost to develop and test while maintaining statistical validity and capture of all possible scenarios, while still being interpreted correctly by the population, is enormous.

Secondly, the number of permutations and combinations of answers is almost universally underestimated by everyone I've ever met outside of this process. Nationality is not the same as birthplace is not the same as parent is not the same as adults in the household. The amount of diversity is staggering.

Additionally, although those two are the primary reasons for the question designs (and I assure you, people will be doing deductions and logical inferences on the back-end once the data is collected), being exhaustive at the point of data entry rather than deductive allows for further meta analysis on the data entry process and survey responding experience itself, whereas if you decide to attempt to deduce, you miss out on a dimension of typo, misskey, misinterpretation, and other analysis data that you would forgo in the former case.

<comment><USER5> Thanks for your viewpoint from the 'sausage factory', so to speak. I am beginning to understand the enormity of trying to combine ease of use, with the effort to gather statistically valid data.

</comment></comment></comment></comment></comment><comment><USER12> My only disappointment is that they didn't have to pay more for their incompetence. IBM has been on a mission to replace their workforce with cheap labour from India for quite a while. I'm glad the work they deliver to their clients is finally being exposed for what it is - junk code with no quality control.

</comment><comment><USER13> I wonder what the infrastructure provided by IBM consisted of.

Based on non-cloud setups like [http://nickcraver.com/blog/2016/02/17/stack- overflow-the-arc...](http://nickcraver.com/blog/2016/02/17/stack-overflow-the- architecture-2016-edition/)

A rack or two would have been able to handle the load.

<comment><USER14> You have to understand that what StackOverflow gets away with in terms of hardware is not at all reflective of what other shops will be able to do.

SO can work on such a small amount of hardware because they've got people who either know the entire infrastructure from ground up, or have the skillset and interest to figure it out. Plus they have very tight coordination between the folks doing the development, and running it day to day (possibly because it's the same people).

Your average developer working on a government contract through $BIGNAMESERVICESPROVIDER is almost certainly not going to have that skillset or motivation to learn it. Even in the unlikely scenario that they did, they're not going to be in a situation where they can act on it. It'll be a case of developing to specification, throwing stuff over a wall to some sort of QA, and then throwing the result of that over to someone to run it. The whole thing will be run by project managers who have opex budgets to meet, on infrastructure that was specified well before the thing was ever built.

</comment></comment><comment><USER15> We requested paper forms long before census night, because we were pretty sure this was going to happen. Didn't put our names or any other personally identifying info on them, either. (Never received a fine.)

</comment><comment><USER16> The Prime Minister is correct in saying that it was a predictable debacle.

Because it's bloody hard to DDoS paper forms. Widely distributed atoms are curiously disinterested in IP packets.

</comment><comment><USER17> Ummm the article isn't very clear.

It says the damage was estimated at 30 million. It says IBM paid something. It didn't explicitly say IBM paid $30 million.

<comment><USER18> First line of the article:

"Computer giant IBM will pay more than $30 million in compensation for its role in the bungled census"</comment></comment>


Linux Storage, Filesystem, and Memory-Management Summit - <AUTHOR> https://lwn.net/Articles/lsfmm2019/ <comment><USER2> If you appreciate this kind of reporting, please consider subscribing to LWN. Subscriber support is the only thing that allows us to do this kind of work.

</comment><comment><USER3> Favorite quote from the articles so far about BPF:

Gregg started with a demonstration tool that he had just written: it's immediate manifestation was in the creation of a high-pitched tone that varied in frequency as he walked around the lectern. It was, it turns out, a BPF- based tool that extracts the signal strength of the laptop's WiFi connection from the kernel and creates a noise in response. As he interfered with that signal with his body, the strength (and thus the pitch of the tone) varied. By tethering the laptop to his phone, he used the tool to measure how close he was to the laptop. It may not be the most practical tool, but it did demonstrate how BPF can be used to do unexpected things.

<comment><USER4> Brendan Gregg is also the guy who shouted at a rack in a datacenter to prove that some hard drives were sensitive performance-wise to vibrations. The video is on youtube and it's hilarious.

</comment></comment><comment><USER5> I was surprised to see discussion of NFS. NFS certainly was a big deal "back in the day" but it had its own quirks and headaches. I haven't seen NFS in 20 years, but that could simply be because of the particular worlds I live in.

Is it still widely used and I just happen never to see it because the environments in which I work?

Or is it only used for a small number of sites (or certain applications) but they happen to be extremely important ones?

<comment><USER6> VMware supports NFS as a storage backend so it ends up being used in a lot of storage arrays in Enterprise shops.

Gitlab.com was also using it as of last year, but I'm not sure if that's still the case.

<comment><USER7> Yes, it has now been replaced by Gitaly: [https://about.gitlab.com/2018/09/12/the-road-to- gitaly-1-0/](https://about.gitlab.com/2018/09/12/the-road-to-gitaly-1-0/)

<comment><USER6> From the drawing, it looks like gitaly decouples git storage from the workers, but still uses NFS on the backend.

</comment></comment></comment></comment><comment><USER8> Btrfs isn't even on the agenda.

<comment><USER9> is btrfs as a project even healthy? a startup I worked at looked hard at btrfs since the COW model and better integrity verification were both very, very useful from our POV. But btrfs in practice was just simply unsuitable; not only were there basic reliability issues (corrupted filesystems) and really bad corner case behavior (full filesystem in particular), we noted that it was not actually fully endian neutral, at least at the time, which caused filesystems between x86 and a BE ISA to appear to work, and then to be horribly corrupted if taken from a BE embedded system and mounted on x86.

Given the volume of btrfs negative experiences that are going to need to be overcome (for example, many of the posts in [1]), maybe people have just given up? If ZoL licensing wasn't a problem, would anyone even be interested?

[1] [https://news.ycombinator.com/item?id=15087754](https://news.ycombinator.com/item?id=15087754)

<comment><USER10> I routinely hear about how btrfs is unreliable, unstable, and corrupts your data.

But it's also been the default filesystem for SUSE Linux and Synology NAS products for a long time now, and they don't seem to be having any problems.

I don't know what to believe.

<comment><USER11> Anecdotally, I've found btrfs unreliable. Here is a comment I made a week ago:

\</comment></comment></comment></comment><comment></comment><comment></comment><comment></comment><comment>----

Personally, BTRFS feels like it has a ways to go before it's ready for prime- time.

I've had two major and one minor BTRFS-related issues that have scared me away from it.

1) One of my computers got its BTRFS filesystem into a state where it would hang when trying to mount read/write. What I suspect is that there was some filesystem thing happening in the background when I rebooted the machine. I rebooted via the GUI and there was no sign that something was happening in the background, so this was really a normal thing that a user would do. No amount of fixing was able to get it back, but I was able to boot from the installation media, mount it read-only, and copy the data elsewhere.

2) Virtually all of the Linux servers at work will randomly hang for between minutes and hours. This was eventually traced to a BTRFS-scrub process that the OS vendor schedules to run weekly. The length and impact of the hang seemed to be based on how much write activity happens - servers where all the heavy activity happens on NFS mounts saw no impact, but servers that write a lot of logs to the local filesystem would get severely crippled on a weekly basis. We've moved a bunch of our write-heavy filesystems to non-BTRFS options as a result of this.

3) This is a more minor issue, but still speaks to my experience. I had a VM that was basically a remote desktop for me to use. Generally speaking it would hang hard after a few days of uptime with no actual usage. When I reinstalled it on a non-BTRFS (sorry, can't remember which filesystem I used) filesystem it was rock solid. I have no proof that this had anything to do with BTRFS.

All of these were things that happened around a year ago, they may not be a true representation of the current state of BTRFS. But they've burned me badly, so now any use of BTRFS will be evaluated very carefully.

In contrast, I've been running ZFS on a couple of FreeBSD servers, with fairly write-heavy loads, and have had no issues that were filesystem-related. Even under severe space and memory constraints ZFS has been rock solid.

\</comment><comment></comment><comment></comment><comment></comment><comment>----

The first problem is directly attributable to BTRFS. There is no way a filesystem should get corrupted by a simple user-initiated reboot, regardless of what the system is doing in the background.

The second problem is a combination of BTRFS and the distribution. The distribution added a weekly job which did a BTRFS scrub (IIRC), under certain workloads that would completely hang machines for minutes to hours. The time this ran seems to be based on when the OS was installed, so as luck would have it these brought production systems down during business hours.

The third problem is something I have no idea about. It could be BTRFS, it could be something completely different, I honestly have no idea.

<comment><USER12> >One of my computers got its BTRFS filesystem into a state where it would hang when trying to mount read/write. What I suspect is that there was some filesystem thing happening in the background when I rebooted the machine.

>The first problem is directly attributable to BTRFS. There is no way a filesystem should get corrupted by a simple user-initiated reboot, regardless of what the system is doing in the background.

It doesn't seem like you have any direct evidence that this has something to do with rebooting. In fact, Btrfs is much less susceptible to these sorts of problems than previous file systems. That's because writes in Btrfs are atomic, so a file is never "partially" written to disk. Either it's written, or it isn't. You can't get disk corruption from write failures.

What I suppose might potentially happen is that you rebooted the system while in the middle of installing a bootloader or kernel update. I don't know if you tried mounting the partition r/w from another system or not, but assuming you didn't, it's probably more likely that something broke on your system that prevented it from remounting the partition during the boot process.

<comment><USER11> I know that updates were not installing when I rebooted. It may have been checking for updates, but it definitely wasn't installing them.

To recover, I booted from an installer image on a USB drive and tried to mount the partition R/W and it hung. Since the installer is a known-good environment, this rules out breaking my system the way you describe. This was a corrupt BTRFS filesystem.

One possibility is that because I was using a rolling-release distribution, I may have gotten a version of BTRFS with a bug in it or minor changes in the BTRFS code as the system got updated eventually got the filesystem into a state that rendered it unable to mount R/W. Either scenario doesn't inspire confidence.

</comment></comment></comment><comment><USER13> > System observability with BPF

Initially I thought this was about recent Spectre vulnerability variants related to BPF. Then I found it is actually discussed in BPF: what's good, what's coming, and what's needed ([https://lwn.net/Articles/787856/](https://lwn.net/Articles/787856/))

</comment><comment><USER14> Are videos of this conference available or is going to be available?

<comment><USER2> No video recording was done, so no.</comment></comment>


Rabbit Hole - <AUTHOR> https://cryptic.io/rabbit-hole/

<comment><USER2> I was thinking of missing/incorrect header when the post started talking about setting IP Address from an optional header. As someone has already said, 'Never trust a client'. But while its very easy to say, its not so hard to remember in practice. I find it even harder to remember when the client is essentially some other part of your application.

</comment><comment><USER3> Reminds me of this hack using the x-forwarded-for header:

[http://blog.ircmaxell.com/2012/11/anatomy-of-attack-how-i- ha...](http://blog.ircmaxell.com/2012/11/anatomy-of-attack-how-i-hacked.html)

(Anatomy of an Attack: How I Hacked StackOverflow)

and this:

[http://xkcd.com/327/](http://xkcd.com/327/)

(Exploits of a Mom)

Never trust the client....</comment>


Common Enemies - <AUTHOR> https://www.collaborativefund.com/blog/common-enemies/ <comment><USER2> This is an instance where on the one hand, a globally coordinated response doesn't do all that much. Korea doesn't seem to need any special help from Italy, for example.

On the other hand, global coordination by medical researchers is both critical and, I expect, frantic. The speed we've progressed from identification through sequencing and then on to Phase 3 antivirals is nothing short of jaw-dropping; this is the most amazing response to a disease we've seen in however many thousand years f recorded history we have.

We may also have to wind back the easy availability of travel after this too. This is probably going to become a 1 in 30 years style event if the travel networks don't change. Good thing to coordinate on, but an issue for the future.

<comment><USER3> >wind back the easy availability of travel

What do you mean by that? I'm assuming it's not just a roundabout way of saying "more expensive", and thus more exclusive, like in the (bad) old days.

<comment><USER2> I do mean the bad old days. Unless COVID turns out to be radically overblown I'm hopeful we don't have the stomach to do this again. Our fast and well connected airline system was a massive contributing factor to this spiraling out of control.

If I had the choice; permanent 2 week quarantine for all air travelers on arrival. That'd kill a lot of travel-related industries.

</comment></comment></comment><comment><USER4> I don't buy it. A crisis that causes the world's countries to isolate themselves from each other does not stimulate cooperation. In fact I'd argue the opposite - it highlights the downsides of a cooperative world with open borders and open markets. I'm not making a value judgement here, just that this will probably result in the opposite of what the article implies.

<comment><USER5> Physical separation doesn't suggest a lack of mutual cooperation. I'm interacting with my friends as much as if not more than ever despite never seeing them in person.

<comment><USER6> I would say that this pandemic would argue more for better digital interconnectedness.

Global access to the internet is still not great. We need better networks!

<comment><USER7> I just had a random thought how ironic it would be if we all displaced physical connections with digital ones, only to be thwarted by a global, digital supervirus in lieu of a physical one.

<comment><USER6> Yes that could happen too. But physical and digital connections need not be mutually exclusive.

I only argue about digital connectedness because it still needs improvement.

Meeting people in person...not much can be improved there. You just need to turn up.

</comment></comment></comment></comment></comment><comment><USER8> Much less confident in our leaders in Washington. FDR seems almost biblical in comparison.

</comment><comment><USER9> This is a beautiful analysis, no less so because it overlooks that, in World War II, the enemy was tangible and knowable in ways that a virus is not.

</comment><comment><USER10> An encouraging essay, but things were not quite as rosy as described.

For examples in the article blood donations were required to be segregated by "race" (to their credit, the Navy just aggregated it and labeled outgoing bottles as needed). Crime _rose_ in blackout Britain as the chance of being caught for theft and robbery fell dramatically.

I say this not to pooh-pooh the article. In fact on international cooperation we have a nice A/B test running right now: Europe sent emergency supplies to China and is now getting some in return. Countries that jumped at international standards reaped the benefits (Senegal has had and continues to have better testing capability and medical response than the USA). Those that chose to "go it alone" or practice censorship have done worse than those who have been more holistic.

It remains to be seen if any useful lessons are learnt from this experiment.

</comment><comment><USER11> I do not expect this outcome under current leadership.

</comment><comment><USER12> "field of race relations"....

</comment><comment><USER13> Did he invent that Voltaire quote? I can't find a source for it anywhere.

</comment><comment><USER14> The problem with this virus is not that there is no vaccine, but that it has asymptomatic carriers.

Hence since there is no way to prove if one is infected, the government assumes that everyone is infected, thus halting economic activity.

The solution will probably be:

1) Massive testing - I could see "at home" test kit. 2) Some sort of certificate of the day of the test and the results. 3) Some sort of division between clean areas (where only non-infected people reside). For example, a flight must certify that all the passengers are clean.

The other option is to have the virus return every X month until there is a vaccine, or until 90-80% of the population is infected.

</comment><comment><USER15> _> Pandemics kill people and recessions ruin people. Saying they have silver linings is a step too far. But I wonder if the best map we have that tells us what to expect next is the kind of extreme cooperation, solidarity, and empathy we last saw in the 1940s. And I wonder if we'll look back at COVID-19 as one of the worst things to happen to us, yet triggering something positive that couldn't be achieved any other way._

I really -- _really_ \-- hope the author is right. His arguments are persuasive and his logic is sound, but I'm less certain than he is of such a positive outcome.

Another historical precedent to keep in mind is the Little Ice Age in Europe, which brought about _both_ sustained economic depression (due to poor crop growth and poor livestock survival) and widespread disease (due to increased activity of pathogens and disease vectors), including the Black Death. During this horrendous period, economic depression and widespread disease fueled each other, generating a lethal feedback loop. Quoting from Wikipedia: "Communities often lashed out via violent crimes, including robbery and murder; sexual offense accusations increased as well, such as adultery, bestiality, and rape. Europeans sought explanations for the famine, disease, and social unrest that they were experiencing, leading to the act of placing blame upon the innocent" (think Jews, women accused of witchcraft, people accused of blasphemy, and so on). "Evidence from several studies indicate that increases in violent actions against marginalized groups that were held responsible for the Little Ice Age overlap with years of particularly cold, dry weather."[a]

Back to the present: If most societies on earth blame _an impersonal pathogen_ for this disastrous shock, and if governments everywhere manage to keep populations in reasonably good living conditions throughout the crisis, we may indeed see more unity against a common enemy. But if most societies resort to _scapegoating_ , Heaven help us.

This isn't idle talk: As I write this, Trump is actively blaming "the Chinese" and "foreigners" for the virus,[b] while some Chinese officials blame "the Americans."[c] Russian media, by instinct or calculation, is blaming "America" too.[d] Others (e.g., Turkey's Erdogan, Iran's leaders) blame "a Zionist plot"[e] or other crazy conspiracies.[f]

The shock of COVID-19 could give demagogues everywhere fuel for re-lighting a fire of old prejudices and hatreds that had remained subdued for a generation or longer.[g]

As I wrote above, I _really_ HOPE the author is right. I hope "the better angels of our nature" prevail.

\--

[a] [https://en.wikipedia.org/wiki/Little_Ice_Age#Cultural_respon...](https://en.wikipedia.org/wiki/Little_Ice_Age#Cultural_responses)

[b] [https://www.cnbc.com/2020/03/19/coronavirus-outbreak- trump-b...](https://www.cnbc.com/2020/03/19/coronavirus-outbreak-trump-blames- china-for-virus-again.html)

[c] [https://foreignpolicy.com/2020/03/02/china-blames-united- sta...](https://foreignpolicy.com/2020/03/02/china-blames-united-states- coronavirus/)

[d] [https://foreignpolicy.com/2020/02/14/russia-blame-america- co...](https://foreignpolicy.com/2020/02/14/russia-blame-america-coronavirus- conspiracy-theories-disinformation/)

[e] [https://www.jpost.com/Diaspora/Antisemitism/Coronavirus- is-a...](https://www.jpost.com/Diaspora/Antisemitism/Coronavirus-is-a-Zionist- plot-say-Turkish-politicians-media-public-621393)

[f] [https://abcnews.go.com/International/wireStory/iran- leader-r...](https://abcnews.go.com/International/wireStory/iran-leader- refuses-us-citing-virus-conspiracy-theory-69733905)

[g] [https://en.wikipedia.org/wiki/Demagogue#Methods](https://en.wikipedia.org/wiki/Demagogue#Methods)

</comment><comment><USER16> In the meantime, countries are closing their borders and isolating more than even the most regressive and backwards-looking people could have dreamed of a year ago. Almost no country has a response plan that has anything to do with any other country at all. As an American abroad, it honestly feels like I'm trapped between waiting on uncertain visa extensions and being forced headfirst into the pandemic by returning home.

<comment><USER17> > anything to do with any other country at all

Virus spread is inherently a local problem. Are you imaging Xi should order American restaurants closed? The best China can do is watch out for themselves, warn others, and send masks once supply allows.

Closed borders is more about ensuring people do not flee with the virus in- tow.

<comment><USER16> Wow, we have come so far from the Ebola response that it has literally become inconceivable there would be more active international cooperation. This doesn't bode well.</comment></comment></comment>


Ask HN: Is it a waste of time to code internal tools? - <AUTHOR> I see a lot of companies (especially startups & small businesses) with dedicated teams/engineers to build internal tools for their support and other internal teams. It seems like it is really time-consuming & engineering heavy depending on the use case.
I think I just developed a prototype that can build internal tools automatically (to a great extent), for any company, and has inherent support for dealing with certain entities such as users, cards, payments etc., As easy to set up as Stripe, and as easy to use as Zapier. All of this instantly, with minimal coding (only if you want to change the visual appearance of the tools generated & add custom logic using JS & Python). Also, there are efforts to integrate this product with other tools like Salesforce, ZenDesk, Kissmetrics etc.,
This product will reduce the time spent in developing internal tools, and the engineers in these teams could be used to build something even better.
So the question here is: Is it a waste of time to code internal tools manually? What are the issues you frequently encounter in your companies w.r.t internal tools (development & usage).
Also, is anyone interested to be an early adapter and use our product to generate internal tools for your companies/products (for free)? <comment><USER3> What exactly does "internal tools" mean to you? Maybe my definition is overly broad, but I've worked on teams that developed and maintained internal tools such as compilers, renderers, visualization tools, signal analysis tools, payload decoders, packaging and provisioning tools, etc.

I think it's highly unlikely that we're talking about the same thing here.

<comment><USER4> Exactly. A program that solves this problem could probably also program the external tools too.

<comment><AUTHOR> Can't agree more!

</comment></comment></comment><comment><USER5> It's not a waste of time to develop in-house tools, sometimes in-house tools become the future of a company.

For small businesses there's definitely a market for automation platforms but there's a lot of problems in actually selling these products. You either focus on a key area where they already have a solution they don't question or you go too broad and confuse the people you're selling to. Like let's be honest, your tool can't build everything.

I think the question you gotta ask is why is your tool builder better than the frameworks and resources that devs already have available to them.

<comment><USER6> > sometimes in-house tools become the future of a company.

e.g. Slack started as an internal messaging tool at a game development company

<comment><USER7> Docker was created as an internal tool at dotCloud.

</comment></comment></comment><comment><USER8> Depends on the job. For example, I write super low-level embedded code, and it is advantageous for us to control all allocation of code space and memory space. The reason is so that patches can be very small. To help with this, we have very specific tools that help us with visual allocation maps, custom diff tools and patch generators, etc. This may never become mainstream - <USER2> resources are just too small for widespread adoption. Just my $0.02

</comment><comment><USER9> _" General-purpose tools are not enough, however. Both specialized needs and personal preferences dictate the need for specialized tools as well; so in discussing programming teams I have populated one toolmaker per team."_

\- "The Mythical Man Month" p 128, "Sharp Tools".

</comment><comment><USER10> Selling a magical pill without providing pharmacology is no different than selling snakeoil. Share your solution please, unless that part in your question is just a digression; (addendum) does not seem to be the case from the last paragraph.

To answer your question, it depends. Anecdotally, we created custom solution for analytics for our Robot, and then moved it in as a solution. You can think of it like 'internal-pivot'.

Additionally, in-house tools, if designed correctly, are usually small pieces of software that do just what a company needs. Tailored solutions, if effectively created, are sometimes better than something out of the box. It also lets the developers stay stimulated when there is not 'actual work' to be done and adds potential for features to be implemented for their product; if not providing with something they can pivot and sell separately.

<comment><AUTHOR> > Selling a magical pill without providing pharmacology is no different than > selling snakeoil. Agreed. I was just not sure if it was a good idea to share > my solution as the answers would be mostly anlayzing the solution. I wanted > to know more about the problem - and see if such a problem really exists.

But briefly, here is how it works. 1\. You give the schema of your DB to the tool. 2\. The tool analyzes and build backend flows. 3\. The user can now select which fields indicate the username, card or payment amount etc., and the flows will be adjusted for the same. 4\. User can add access control to tables and fields. 5\. Preview and publish, and the tool is ready to be used.

Since a single tool like this cannot build tools for specific usecase, there is an option to define own actions and layouts.

<comment><USER10> That kind of sounds like an ORM no?

</comment></comment></comment><comment><USER11> You will be competing with the likes of salesforce and an army of salesforce "integration experts". I guess I'm saying your target market might not be who you think it is and you're probably better off talking to the salesforce integration experts and figuring out how to solve their pain points.

</comment><comment><USER12> In my opinion you probably know the answer to this question, that's why you're asking, that is you know it's probably a waste of time.

The best advice I could give is ensure you're doing it for the right reasons, that everyone around you will truly profit and make sure you've _honestly_ evaluated existing tools.

I often see people failing to correctly evaluate an existing technology/tool and then rush off to build their own inferior product. Yes it can be fun to just code, and there are cases where it's truly a good / beneficial thing to do, but they're not common from my experience.

Good luck!

</comment><comment><USER13> What else is "scripting" but that act of making a very small automation of a business function? Is a customized spreadsheet a tool?

There's always going to be demand for tools of many kinds, so whatever you're offering has to differentiate better than "internal tool". The abstract you give is kind of like a software framework, possibly with some form of high level scripting, configurator, or wizard to guide the user.

At the end of the day always evaluate your solution against a short bash or Python script. Did your solution gain anything? If not, why?

<comment><AUTHOR> Its more like a wizard, and seems very similar to [https://developers.google.com/appmaker/](https://developers.google.com/appmaker/).

</comment></comment><comment><USER14> This doesn't make any sense unless you specify what you mean by "internal tools"

</comment><comment><USER15> Standard cost benefit analysis: does the time spent coding and maintenaning the tool offset by time savings or value improvements? Your pitch is whether you can come in and present a build or buy choice...I think the internal problem space is pretty vast, but if you think you've got something to offer and you can present useful functionality in ten lines of code, you might have a chance. You'll need to get some people raving about how much time and headache saved, I don't think any amount of conventional marketing will convince your target audience.

</comment><comment><USER16> No, because they're a lot funner to code than customer-facing tools.

<comment><AUTHOR> I agree! :D

</comment></comment><comment><USER17> This reminds me a bit of The Last One, a program for generating business software on demand. It was released in 1981 and supposed to be the last program you'd ever need to buy. You can guess how that (and every subsequent such product) turned out in the end.

That's not to say don't try, but be aware that this is harder than it looks!

<comment><USER18> Some of those are still around (here in Uruguay we have a tool called Genexus that's been around for like 20 years and generates code in almost any language, the OP might want to look into that [https://www.genexus.com/global/home?en](https://www.genexus.com/global/home?en) ).

The problem is, they're still programming languages, but very opinionated They look like magic if you go the way the tool allows you to go, but when you go off the rails you end up wasting more time than if you had coded it in a standard language.

<comment><AUTHOR> GeneXus seems interesting, though my tool isn't exactly like GeneXus, it does overlap a bit in the sense of not requiring to code and easy to develop UI etc.

</comment></comment></comment><comment><USER19> Companies build internal tools, _usually_ when it makes financial sense to do so. Us developers do the same thing on a smaller scale when we automate tasks we find ourselves doing repeatedly. Teams/companies that find themselves similarly doing something repeatedly, and costing lots of time will do the same. Obviously to be successful it needs to be something that makes sense and where the time saved by it will outweigh the time spent on it.

And excuse me, I dont know what your tool is or does, but there is no way that it could replace even a fraction of this development (though that doesnt mean it wouldnt be useful or profitable). Otherwise it would be intelligent enough to replace developers entirely.

</comment><comment><USER20> I suspect there are already a lot of tools like the one you've built. Two examples are knack.com and ragic.com. Now, maybe your's is better, but it's good to be aware of the competition.

Sadly, companies often end up going with something like salesforce, which I hear also has the ability to be extended like these more general tools (though they try to sell consulting services to do so). The reason companies go with something like salesforce is that 80% of what they need is already built and done in a pretty targeted way. For the rest they can have an in-house dev or a consultant extend it and integrate it with whatever other tools the company is already using.

<comment><AUTHOR> I just realized I built something similar to [https://developers.google.com/appmaker/](https://developers.google.com/appmaker/)

Irony, I wrote it for AppEngine, didn't host it though.

</comment></comment><comment><USER4> @Sreyanth: Internal tools of a given type? Perhaps. Sounds payment orientated. You need to put this in the title.

I would also like to point out that you're asking people whether they want to be an early "adapter" of a product you haven't described or referenced. "I think I just developed" doesn't install any confidence into startup wanting to adopt your mystery software, either you have or you haven't.

Opinion: What you're describing sounds like a large API. I think you may struggle to get adoption if you're looking for money from this software.

<comment><AUTHOR> Thanks for pointing out @bArray.

I intentionally avoided describing the product as I wanted to know more about the problem space, so I can come up with the right solution using this prototype.

But I just realised that Google is working on something similar: [https://developers.google.com/appmaker/](https://developers.google.com/appmaker/)

<comment><AUTHOR> Also, just realized. It is adopter, and not adapter. My bad. Was too sleepy to realize what I typed.

</comment></comment></comment><comment><USER21> Sounds somewhat like the "App Maker" tool Google has in limited beta.

[https://developers.google.com/appmaker/](https://developers.google.com/appmaker/)

<comment><AUTHOR> Ah, shit. Yes. :(

Should I be giving up?

<comment><USER21> There is one big problem with Google's implementation. Only users in the Gsuite domain where it was created can access it. So, for example, if you wanted to create an applicant tracking system, the job applicants wouldn't be able to sign into the app to uploade their resumes or see status.

If you think they won't ever fix that, there's an opening there. It's an odd limitation to me. I suppose they are trying to sell GSuite licenses.

<comment><AUTHOR> I think they will. Or probably they want to use this to get more GSuite users? I am not sure. Most of the startups that I know of which take tools seriously are already using GSuite. Will need to see how my tool will add any value to them.

<comment><USER21> Well, offering the ability to run on premise would be a difference. Most of the tools in this space don't do that. Ragic, Knack, Zoho Creator, Quickbase, etc.

Or maybe tie it to AWS with a pre built AMI? There are at least some companies where a point-and-click AWS instance would have value.

<comment><AUTHOR> My tool enables the users to deploy the tools generated wherever they want. It will anyways be deployable to AWS, Azure, GCloud, Digital Ocean and Heroku with a single click - using their own credentials.

Do you think pre-built AMIs might be a good idea? I am not sure about it though.

</comment></comment></comment></comment></comment></comment><comment><USER22> Internal tools are not a waste of time. I have built them for previous employers and for my company. Those tools solved critical issues while saving time and money. It made more sense to build than to use current solutions.

There are plenty of cases where it doesn't make sense. I am not surprised companies build tools even when it isn't logical.

I would like to try your product, if I find it useful I might be willing to pay.

<comment><AUTHOR> Sure. At this point its just a really weird prototype. I was thinking of getting a few early adopters, build a product to serve their needs and then extend it. Should I contact you on your email?

</comment></comment><comment><USER23> IMHO, crafting something useful, useable and making sense isn't waste of time, especially when you are paid to do that and your product is considered as your KPI.

You are paid to do that, at least it means that worth your compensation, right?

<comment><AUTHOR> Yes, but when it can be automated and the engineering resources could be used for building something even more useful, I think that's a clear decision for the business?

</comment></comment><comment><USER24> What type of internal tooling are you generating?

<comment><AUTHOR> Not the in-house tooling for devs, but for non-devs in the company to support their daily tasks.

</comment></comment><comment><USER25> > I think I just developed a prototype that can build internal tools > automatically

So, a new programming language?

<comment><AUTHOR> No, a web based UI which basically does the work of providing an interface to the DB (think of phpMyAdmin), but also adds meaning to the data based on the entities and teams that need the data.

For example, the tool processes the DB schema, builds a UI. Learns which field is username, and then shows in the front end the way user management softwares work. For the sales guy, its just all the information from Salesforce, different forms, the value of their converted leads. For the finance team, its the payments, fraud disputes etc.,

All this data is in the DB. And engineering efforts go in to build these tools for the non-engineers and keep re-iterating. I am trying to solve that problem of re-inventing the wheel. Any functions specific to the business can always be added using JS or Python (just like Zapier).

<comment><USER18> It sounds really useful, but are you sure non-programmers will be able to do stuff with it?

There's a continuum of automation software, non-programmers often find themselves comfortable writing stuff in Excel, and some become kind-of self- taught programmers using something like VBA, Access, or whatever is used today, but they are still not programmers and don't understand how your "magic" tool works.

If your tool has the ease of use for non-programmers as Excel you might have something big.

If not, you still might have something useful :) but understand your audience :) .

<comment><USER21> It is an already crowded space. Quickbase, Fusioo, Ragic, Knack, Zoho Creator, Google App Maker, etc.

I've used most of them, and they end up being frustrating. Quickbase was the most flexible of the lot, but it's expensive. I haven't tried Google's. It sounds like it has a problem many of these tools share though. Per user pricing, with no notion of "external users". Think something like job applicant tracking. You couldn't offer an external user portal to upload a resume without paying a license for every applicant.

<comment><AUTHOR> @tyingq. That's a brilliant point, and am glad that you pointed it out.

We were thinking of per-user pricing only for the employees. The tools generated will have an "Allow external user signups" option that would allow external people to signup, and then the external users' traffic would be charged by the hosting company (or by us if the user chooses us to host their tools for them). I think this is a normal way of doing it, and I believe these tools will soon switch into this mode, if not, I am glad :D

</comment></comment></comment></comment></comment><comment><USER26> Like everyone else is saying: it depends. If there's a tool that will save your employees a few minutes a day and it'll take one of your engineers a few days to write, it might be worth it. There's even a handy XKCD about it: [https://xkcd.com/1205/](https://xkcd.com/1205/)

<comment><AUTHOR> This is exactly what the tool does! It takes a few minutes to set it up or modify. Engineers would've otherwise spends days writing, another week testing and deploying.

</comment></comment><comment><USER27> I'd be interested in taking a look; email is in profile.

</comment><comment><USER28> I'm interested in taking a look. josh[at]josh[dot]mn

<comment><AUTHOR> Sure. At this point its just a really weird prototype. I was thinking of getting a few early adopters, build a product to serve their needs and then extend it. Should I still contact you over email?</comment></comment>


Ask HN: Now that my site is live, how do I make changes to the source code? - <AUTHOR>

	As a newbie web programmer, I've been editing the source files directly through an SSH connection. Now that the site has traffic, I'm not sure how to make new updates and furthermore test new features that are guaranteed to be broken upon their release.
Am I supposed to be using some form of revision control system, or is this a game of hidden subfolders and permissions?
Thanks <comment><USER2> Here are a few thoughts, some people may disagree with me, but you're new, so you don't get to be one of them.

1) Use git. Go create an account on github and use that. If you don't know how to use git, there's a tutorial at [http://try.github.io/](http://try.github.io/)

2) Do all of your development on your local machine. That means that your local machine should be able to run your site.

3) Try to make your deployment process as simple as possible. Since it sounds like your site is small, you can probably get away with pulling the new code and restarting your web server. Total downtime should only be a few seconds. You can make it more complex as you get your feet under you.

<comment><AUTHOR> What happens if I open up a live .php file and just paste new working code and hit "Save"? What happens on the end user-side if I do that? Furthermore, I'm with DO and I don't think I've ever rebooted my virtual server.

<comment><USER2> Ah, I was unclear. When I said restart your web server, I didn't mean to reboot the machine. Just restart the web server process. Nginx or Apache or whichever you're using.

<comment><AUTHOR> Why does it need to be restarted ? It seems that when I save a file, the changes reflect on the site automatically

</comment></comment></comment></comment><comment><USER3> I work for an ecommerce site and we generally have a diff. sub domain which will have the "beta" or "test" version of code in live. Users are welcome to use it, but with fair warning. We also trickle, small percentage of users to the newer experience to gather feedback before enabling it site wide (A/B testing).

<comment><AUTHOR> So you're sayign what I can do it duplicate my site onto a subdomain, test the features there, and when they're bug-free, I can just copy paste the new code into the source files on the live site?</comment></comment>


Hedge funds turn out not to work: average fund down 17.6% for the year - <AUTHOR> http://www.nytimes.com/2008/10/23/business/23hedge.html?em <comment><USER2> The recent financial reporting has been pretty atrocious.

"This unregulated, at times volatile corner of finance which is supposed to make money in bull and bear markets lost $180 billion during the last three months."

Our financial systems are heavily regulated. The specific regulations that the author would like to see may not exist, but this does not justify claiming that hedge funds are unregulated. Many people seem to believe that all risk should be regulated away and actually believe that this is possible. More information on current regulations that hedge funds are subject to:

[http://www.forbes.com/opinions/2007/04/13/hedge-fund- regulat...](http://www.forbes.com/opinions/2007/04/13/hedge-fund-regulation- op-ed-cx_pw_0413hedge.html)

"Many hedge fund investors, particularly the wealthy individuals, are flabbergasted by their losses this year. The average fund was down 17.6 percent through Tuesday, according to Hedge Fund Research."

The S&P 500 is down 40% for the year.

<comment><USER3> Agreed, I would have like to be down 17.6% this year instead of whatever I am actually down. Luckily, I pulled most money out last spring to travel and start a company - best investment I have ever made!

<comment><USER4> I'm only down 2%. You can give me all your money and I'll invest it. Or rather, I won't, but that will save you even more money!

(And then you can curse me when we get hyperinflation, the stock market shoots up, and all that cash becomes useless.)

<comment><USER5> Did you cash out because you suspected the market was due for a big correction?

Or were you just lucky to have assets in cash at the right time?

<comment><USER4> A little of both.

When I was first starting my startup, I figured I ought to rebalance my portfolio just so I could forget about it while I worked. Tallied everything up, divided them up by asset classes, and figured out what dollar value I wanted in each fund. And I moved some of it around, into bonds and out of individual stocks and such.

But when it came to putting money _into_ an S&P 500 index fund, I just couldn't do it. I had the Vanguard website open, everything ready to invest, form filled out, but my gut told me that there was no possible way that the stock market was worth what the ticker said it was worth. I'd previously worked at a financial software startup, so I had all the stats on market P/E, record earnings, leverage levels, etc. So I closed the window and forgot about it.

This was on Oct 5. The S&P 500 peaked on Oct 11.

So yeah, a whole lot of luck, and maybe a little knowledge thrown in. I tell this story because everyone here's coming up with excuses for hedge funds: they're "only" down 17.6% when the market is down 40%, or they got those outlandish returns with less risk, or they're really really smart and so should be compensated for those sophisticated quant models. But excuses don't matter in finance - only your return. Ultimately, it's a whole lot of luck and a little knowledge - _for everybody_. I predict we'll see many, many more hedge funds do a lot worse in the coming year.

<comment><USER6> So, at what point are you going to reverse that decision, or what do you think are currently good investments for a retirement beyond 20 years from now?

Especially passive management investments?

<comment><USER4> I think the market is pretty close to fairly-valued now, by conventional valuation measures. Maybe still a little overpriced, but you can't call the exact bottom.

I'm waiting to put money in because I don't yet have a job, and so may need that cash to fund another startup (there's that luck aspect again; had I not been starting a startup last year, I likely would've had way more in the market). But if I had a normal employment situation, I'd be gradually money into stocks, towards a more normal asset allocation.

As for a retirement 20 years out - I wouldn't even try to predict that. I haven't touched my Roth IRA this crisis, nor do I count it in overall returns (haven't even looked, really). The one thing I _would_ say is to have some international exposure. Prices rise or fall based on how reality compares to the market's expectations, and I think it's unlikely that America in 20+ years will have the hegemonic position it does now.

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> That's not the full picture, really.

1) Since a large chunk of the money on financial markets flows between funds, and this is pretty much a zero sum game, the average hedge fund performance is bound to be around 0%, other things being equal. It's your fault if you pick the wrong hedge fund, or you don't know the risks.

2) A lot of hedge funds do not target an absolute return, but rather a return on top of the market's return (and they pick an index or several which they consider to be "the market"). So if the market is down 20%, but funds are down 17.6% on average, then they have actually made a 2.4% average return compared to the index they follow. As weird as this may sound, this is actually a success for them. They would usually get a cut of the 2.4% gains as their commission.

One of the main reasons is that absolute return funds are mostly quantitative and/or derivatives-based, and those are much harder to do than buying stocks by your gut feeling (which is what a large number of funds do, really). The other reason is that when a fund is big ($5bn+) it is very hard to move your positions around quickly (as would be required for an absolute return strategy), without shaking the market, and losing money in the process.

<comment><USER4> That's some pretty funny accounting.

By your first point, hedge funds _are_ the market, and so between all of them, ought to average exactly the market returns. By your second point, they measure their returns against the market, and so on average should return 0%.

Why should anyone ever invest in hedge funds, under these premises?

A much simpler (and probably truer) explanation is "What goes up, must come down." Hedge funds booked large paper profits during the boom; now it's time for them to book large paper losses in the bust. Except that if their customers get scared, those large paper losses turn into large actual losses.

<comment><USER7> Ah, but you missed the "other things being equal" part. ;) If the markets move as a whole and hedge funds are heavily long on the market, but others are market-neutral, or even short, things are not equal.

<comment><USER4> Right, but on the whole, you'd expect those all to average out. If the markets move up and hedge funds are long, they're as likely to move down while hedge funds are long, or move up while hedge funds are short.

<comment><USER7> That is why better hedge funds know when to be long and when to be short. My explanation makes a lot of assumptions for simplification, because it is impossible to know what every hedge fund is doing at every point in time.

</comment></comment></comment></comment></comment><comment><USER8> This article is pretty bad along with most NYT financial news coverage.

A lot of critics blame everything on deregulation. But the big banks were the most regulated part of the financial system, and they have imploded. Meanwhile hedge funds have the least regulation, and while they haven't been making money, they at least are not threatening to destroy our financial system.

The Economist had a much better article on hedge funds in the aftermath of the financial crisis.

[http://www.economist.com/finance/displaystory.cfm?story_id=1...](http://www.economist.com/finance/displaystory.cfm?story_id=12381701)

Particularly, hedge funds have not been allowed to hedge! If the government bans short-selling in the hardest-hit stocks, don't be surprised if a hedging strategy doesn't work any more.

<comment><USER5> "But the big banks were the most regulated part of the financial system, and they have imploded."

I think this is not true.

It is my understanding that it was the big, less regulated investment houses that got into a lot of trouble with mortgage securities. The heavily regulated retail/commercial banks haven't suffered nearly as much.

Unless we are defining "banks" differently...?

<comment><USER8> Yeah we are defining "banks" differently - I was considering both "investment banks" and "retail banks" as types of banks. You are right that retail banks haven't suffered as much, but considering Washington Mutual and Wachovia collapsing and several others signing on to the government equity purchase program, they're definitely costing the taxpayer more than hedge funds are. So far, at least....

</comment></comment></comment><comment><USER9> Seems like that's still a pretty good investment if the market as a whole is down 40%.

<comment><AUTHOR> Stocks are down. Gold is up.

Hedge funds don't just invest in stocks. They can bet on anything, including stocks going down. This is supposed to allow them to make money in down markets as well.

<comment><USER10> Gold is actually down, trading near its 52 week low

<http://www.marketwatch.com/quotes/gc08z?sid=1633395>

I agree that hedge funds can invest in anything (unlike MFs, which are regulated) but if you look at the short term (4-week) T-bills their interest rates are down by as much as 62% since 2002, so the general consensus is that, that is where most money is getting drained.

Why? They are the most conservative, safe investment choices available. If I may digress, this is not a good thing for the economy as we are not using that money to produce anything.

Considering the Dow is down over 30% and counting for the year, 17.6% down seems impressive to me :-)

<comment><AUTHOR> _Gold is actually down, trading near its 52 week low_

Oops, sorry, was relying on a news story I saw a couple weeks ago. It was doing better than the stock market a couple weeks ago, though, which was what I was getting at.

</comment></comment></comment></comment><comment><USER11> obviously they DO work then, since most indices are down 40% on the year.

Hedgefunds are designed to outperform benchmark indices. Outperforming by 23% is phenomenal.

By the logic of the poster, a year where the SP500 returned 50% and a fund returned 10% would be proof that hedge funds DO work. That fund manager would be immediately fired.

Hedge funds have outperformed for the last 10 years. Taking a one year performance that loses money doesn't mean anything.

</comment><comment><USER12> It happens because there is also a silent run on hedge funds that are forced then to close their positions, bringing, even sound, assets down.

The other reason is highly leveraged funds when the margin requirements went up after interest rates raise.

<comment><USER13> From The Economist article:

 _All this means that hedge funds have been unable to ride to the rescue of global markets. According to the IMF, the average cash balance of hedge funds has risen from 14% last year to 22%, while the amount of leverage (borrowed money) they use has fallen from 70% of capital to 40%. In theory, that gives them the firepower to buy now that prices have fallen; in practice, they may need their cash to repay clients that want to redeem their holdings._

You are only as smart as your dumbest investor is.

<comment><USER12> Means that hedge funds should hedge against their own investors, too.

</comment></comment></comment><comment><USER2> BTW Paul, why did you post this? Do you believe this drivel or were you trying to incite discussion?

<comment><USER12> Hedge funds exploit the system, as we, hackers, do.

</comment></comment><comment><USER14> This does not prove hedge funds do not work. It only proves what we already knew: active investing is a zero sum game.

This is tantamount to claiming that poker doesn't "work" because the "average" player breaks even.

<comment><USER15> I assume what was meant was that they failed in their job of hedging. Hedging is supposed to involve taking a very conservative position to protect against losses.

This, I assume, is a bit of a joke since hedge funds have largely turned into speculation funds which take risky positions in the hopes of large returns.

<comment><USER7> They actually succeeded in hedging, because they halved their losses compared to the market.

Hedging doesn't mean you always make money. It means that if your main instrument falls, instrument B is going to go up and cover some of the losses, but not all of them.

<comment><USER16> The term "hedge fund" is a leftover from times when it wasn't a catch-all for investment firms for qualified investors. As the other kind of firms had their ups and downs, hedge funds expanded into their space. By now htey are so well established many firms that aren't technically hedge funds (because they do so many other things and may not do any hedging at all) are called hedge funds.

From <http://en.wikipedia.org/wiki/Hedge_fund#History>:

Sociologist, author, and financial journalist Alfred W. Jones is credited with the creation of the first hedge fund in 1949.[2] Jones believed that price movements of an individual asset could be seen as having a component due to the overall market and a component due to the performance of the asset itself. In order to neutralise the effect of overall market movement he balanced his portfolio by buying assets whose price he expected to rise and selling short assets he expected to fall. He saw that price movements due to the overall market would be cancelled out because if the overall market rose the loss on shorted assets would be cancelled by the additional gain on assets bought and vice-versa. Because the effect is to 'hedge' that part of the risk due to overall market movements this became known as a hedge fund.

</comment></comment></comment></comment><comment><USER17> Circle this date in your calendars: an argumentative headline with "gratuitous editorial spin" from PG himself!

The S&P is down ~38% year-to-date; would it also follow that stocks, stock markets, or shareholder capitalism "turn out not to work"?

Wouldn't anyone in an S&P index fund rather be in one of these awful, awful "average" hedge funds? I know I would -- even just for 2008, never mind the many up years!

</comment><comment><USER18> So when does someone cry 'But PG, it's not hacker news!'

</comment><comment><USER19> Why are these things even called hedge funds? What they do is the opposite of hedging.

<comment><USER20> From the LTCM history When Genius Fails, I gather they wait for prices of equivalent bonds,stocks etc, to converge to the same value.

Okay, so a US treasury bond is equivalent to a UK treasury bond, except it's off by 0.0005%. So you buy billions/trillions of each and the price converges, and you make millions.

<comment><USER19> Well, yes. That's what arbitrage is and in what you would _expect_ hedge funds to engage. Only they do much riskier things than that. Hence my comment.</comment></comment></comment>