Boost Your Data Munging with R - <AUTHOR> http://jangorecki.github.io/blog/2016-06-30/Boost-Your-Data-Munging-with-R.html <comment><USER2> While data.table does have impressive benchmarks, the rise of dplyr in the years since with its _significantly_ more readable and concise syntax makes life a lot easier for data sets that are not super large. ([https://cran.rstudio.com/web/packages/dplyr/vignettes/introd...](https://cran.rstudio.com/web/packages/dplyr/vignettes/introduction.html))

The vignette linked above describes some compatability between dplyr syntax and data.table, but I admit I have not tested that.

Nowadays, if you are working with multigigabyte datasets, it may be worth looking into Spark/SparkR ([https://spark.apache.org/docs/latest/sparkr.html](https://spark.apache.org/docs/latest/sparkr.html)) for manipulating big data in a scalable manner instead, as there is feature parity with data.table + analytical tools [GLM] as a bonus. (The syntax is still messy, to my annoyance)

<comment><USER3> You can rent an r3.8xlarge instance with 344GB RAM on AWS for $2.66/hr. If that isn't enough, an x1.32xlarge instance with ~2TB RAM is $13.34/hr. Assuming your data is already in AWS, of course. Of course those machines have a crap ton of CPU power (I'm explicitly not saying cores because I'm mystified by what vCPU really means) as well which you will be hard pressed to take advantage of but if it's RAM you need they're super-cheap.

If you prefer Azure, a G5 instance with 448GB RAM is available for $9.65/hr.

Not a lot of need for Spark/SparkR for the vast majority of data sets given the cheapness of compute these days.

<comment><AUTHOR> The CPU power would not be too hard to utilize, if one's task is amenable to fairly coarse-grained parallelization, which is rather easy to implement in R scripts with the help of foreach[1] and doFuture[2].

Such an approach worked to great effect for me recently, when I needed to perform a `zoo::rollapply`[3] across a time series with tens of millions of rows. The speedup when throwing more cores (7, on my laptop) at it is roughly linear. If I ever need to scale up the analysis to hundreds of millions of rows, the 128 vCPUs of an x1 EC2 instance would be well worth the $$/hour.

[1] [https://cran.r-project.org/web/packages/foreach/index.html](https://cran.r-project.org/web/packages/foreach/index.html)

[2] [https://cran.r-project.org/web/packages/doFuture/index.html](https://cran.r-project.org/web/packages/doFuture/index.html)

[3] [http://www.rdocumentation.org/topics/1505094](http://www.rdocumentation.org/topics/1505094)

<comment><USER4> Shameless self promotion: my `slide_apply` function shows 5-10x improvement over `zoo::rollapply` (despite no Rcpp call, somehow...)

[https://gist.github.com/stillmatic/fadfd3269b900e1fd7ee](https://gist.github.com/stillmatic/fadfd3269b900e1fd7ee)

if your function has a well-defined rolling form, e.g. rolling mean or standard deviation, you should use a pre-optimized function for that. package `catools` has a bunch of useful ones.

<comment><AUTHOR> Thanks! As it happens, my function doesn't conform to any pre-optimized facilities, though I wish it did. At some point, I may look into writing an optimized variation, with the help of Rcpp, or even using the (inspiring!) fortran approach of quantmod/xts.

</comment></comment></comment></comment></comment><comment><USER5> Does any of these support out of memory data? IIRC dplyr can use a db as backend but I could be wrong. Anyway, I wonder why not just use a db & sql for more demanding data munging?

<comment><USER6> Presumably that depends at least on the type of data and whether you have a suitable database to use. As another approach, for R there's an interface to parallel NetCDF in the pbdR system that you might use for distributed processing on an HPC system. There doesn't seem to be anything like pdbR for Python, for instance.

</comment></comment><comment><USER7> How to convert Wide format data to Long format data (process called 'reshaping')is very very important to understand. Certain statistical tests cannot be performed if not in Long format.

The article mentions Reshape2. I also love TidyR. It's much easier to use than Reshape2 when dealing with a large number of variables (i.e., columns).

<comment><USER8> I find Tidyr more frustrating than useful a lot of the time.

<comment><USER7> Interesting, I'll have to watch out for it then. I found it more helpful than Reshape2 when trying to convert a dataset with 55 variables from wide to long format. 8 of those variables needed to be collapsed into 1 variable. With Reshape2, I had to identify all the ones I didn't want in the final result, with TidyR I just needed to identify the ones I wanted. Hence it was easier with TidyR.

</comment></comment></comment><comment><USER9> I went from barely using data.table to only using data.table for basically everything in less than a few years. I think this is the trend given it's faster than basically everything: [https://github.com/Rdatatable/data.table/wiki/Benchmarks-%3A...](https://github.com/Rdatatable/data.table/wiki/Benchmarks-%3A-Grouping)

</comment><comment><USER10> In Python, you have pandas and then Python is great for text processing in general. Is there anything that R provides more for data munging?

<comment><USER11> It's a bit lame, but I am so in love with rstudio... I can't think of a python IDE that's near for data science. Simple things like being able to see all the entities you've created in your session and being able to jump back and forth through plots are really nice.

<comment><USER12> Rodeo is a good start:

[http://blog.yhat.com/posts/rodeo- native.html](http://blog.yhat.com/posts/rodeo-native.html)

<comment><USER13> Also a silly reason: for me the text rendering in Rodeo on a Mac without a retina display is a lot worse than the text rendering in RStudio.

</comment></comment></comment></comment><comment><USER14> The only good solution to ETL/munging/wrangling is streaming. Loading everything into memory is not a good idea.</comment>


Why I am (not) getting into Y Combinator - <AUTHOR> http://www.rorywalker.com/why-i-am-not-getting-into-y-combinator

<comment><USER2> I feel you Rory. Sometimes I find myself spending more time building a plan than building a product. Good luck with Robmenot. Neat concept...

<comment><AUTHOR> Hey Geoffrey, Thanks for the kinds words. Even launching RobMeNot into open beta was a feat in a itself. But it all gets easier once you do it.</comment></comment>


Diamonds Suck (2006) - <AUTHOR> http://diamondssuck.com/ <comment><USER2> If you're using moissanite to "pass" as a diamond, I would tentatively recommend against it, or recommend that you practice some caution. I own some moissanite jewelry, and it does have a yellowish hue and sparkles like CZ. It looks like a bad quality diamond under most lights, and looks very "rainbowey" under certain other lights. I love it, and it's a great stone, but I love it for reasons other than it being a "diamond substitute"

</comment><comment><USER3> It's still several hundred bucks for a rock.</comment>


How Employers Track Their Workers - <AUTHOR> https://www.theatlantic.com/business/archive/2018/10/employee-surveillance/568159/?single_page=true <comment><USER3> I've never seen an employee monitoring system that's built to the benefit of the employee. But nothing a company does it to the benefit of the employee. HR isn't there to help employees, HR is primarily a risk mitigation and compliance department. Who would assume employee monitoring systems are any different?

"They try to never speak up, never stick out, do nothing that might get noticed by management. This leads to a vicious cycle, whereby management grows more suspicious and feels justified in ratcheting up the surveillance."

This is the most troubling thing about surveillance, it changes peoples behavior. The people who implement these systems don't understand the psychology of surveillance, so they use the change in behavior as a indicator of increased risk. Its a vicious cycle.

<comment><USER4> > But nothing a company does it to the benefit of the employee

This isn't a feature intrinsic to companies, but rather a relatively recent social norm. There are still a few companies that do look after their employees, and do things to the benefit of their employees. They aren't very common though, usually it's smaller (often family run) businesses, and often a physical trade rather than an office job.

<comment><USER5> " but rather a relatively recent social norm."

Ask your great-grandfather working in factory, or a mine, or as a tenant farmer (basically 99% of all work) - how he was treated by 'today's standards'. Where I did my undergrad, the main office building was built by a logger baron who literally had strikers _shot_. It's almost universally better today.

" usually it's smaller (often family run) businesses" \- are usually lower paying and often less stable. It's really, really easy for a contractor to not get paid for a job, or to make a mistake costing him $$$ - and then to be in a position wherein he can't pay his workers the aloted amount. This is common.

Any entity that provides fairly steady, predictable work and guaranteed pay is doing a good thing for the most part.

<comment><USER6> >Any entity that provides fairly steady, predictable work and guaranteed pay is doing a good thing for the most part.

I'm not really sure about that, any big busines wouldn't hesitate to fire you and as many people needed to close the numbers or to cut your hours-wages for the same reason let's not talk about benefits, there's not "safe job" anymore.

<comment><USER5> " there's not "safe job" anymore."

Most jobs are safe actually.

First - government, which is a massive employer.

Second - healthcare - generally low turnover, and a huge portion of the economy.

Energy - parts are up and down, but anything closer to the core is very safe.

Core jobs at any classic company. They are generally safe.

Large layoffs are not common, and this idea that companies will drop huge numbers of people 'just to save a buck' is simply not true: most layoffs only happen during economic calamity, when the company is starting to face trouble. In decent times, firings is rare.

Now factory wok, hourly workers, retail workers - that's a whole other thing.

Remember that companies make money by expanding and hiring, not by laying off. CEO's are 100x more likely to want to 'grow' than 'shrink' and there's a lot of political cost with layoffs.

Rubin's startup is flailing, that's why he's laying off, that's to be expected.

<comment><USER7> >Now factory work, hourly workers, retail workers - that's a whole other thing.

These _are_ most of the jobs, not government, energy, healthcare and whatever "core jobs at any classic company" means. Those are also overlapping sets.

<comment><USER5> No, you can see the breakdown here from the BLS [1]

Most jobs are relatively safe from calamity. Though there is ugly turnover, the same applies - in retail in particular, companies do not want to layoff - this would be very bad. They want to hire, it's just that the jobs suck. As for manufacturing - that's generally the one where 'laying off' might be good for the company as they would move the jobs overseas.

But this idea that companies are always looking to layoff and drop people is not right, it happens in specific places.

[1] [https://www.bls.gov/emp/tables/employment-by-major- industry-...](https://www.bls.gov/emp/tables/employment-by-major-industry- sector.htm)

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> When our company first began implementing employee tracking the goal was to help our staff understand their own performance, co-educate and to attempt to support a culture of experimenting with every aspect of our work.

Our initial systems ended up being decent at keeping people "moving fast" but in our first few feedback sessions we found it stressed people out, encouraged competition instead of co-education and locked people into workflows they "knew worked" rather than fostering experimentation.

We quickly killed all but the most basic performance tracking aspects of the project and we found better ways to achieve our goals.

We now have a sort of "opt-in" tracking for staff who are interested in testing the strengths and weaknesses of a novel approach, for example. This puts the data in _their_ hands as a tool to convince _us_ of the validity of their approach and the empowerment is not trivial.

It's far more friendly and humane but it's also something that benefits everyone at the company.

<comment><USER9> I'm curious; in the first version, where was the data stored and who had access to it?

<comment><USER8> There was an intranet portal available to all the staff and much of the day- to-day data was presented directly to the staff as they completed their work.

However, only management was really "engaging" with the data, the staff themselves only interacted with it enough to outrun it.

Now, employees feel excited about the opportunity to dig into datasets that they created with some specific intent and it's the staff themselves that engage directly with management.

It's just much better all around.

</comment></comment></comment><comment><USER10> Pretty disgusting stuff. There are plenty of places in the world where a video camera provides relief and safety, but work is rarely one. The engineers building this tech should be ashamed, they grew up to be the bad guys.

<comment><USER11> A few years back a guy I knew from college wanted to have a meeting with me regarding a "business opportunity".

He pitched me an idea for a system that would essentially spy on IT contractors through the webcam during work. Fortunately his budget was laughably small and my participation out of the question so he never went through with this.

He is known for being utterly shameless but I guess what other kind of person would come up with such an idea?

<comment><USER12> Doesn't upwork and other freelance style sites monitor by webcam/screencapture as an option?

<comment><USER11> This man is not known for having original ideas either.

</comment></comment></comment></comment><comment><USER13> as someone who works a blue collar job in an auto repair shop, these articles always make me pause for a moment and ask if i really want to keep learning python in the hopes of pursuing an office job. The entire environment just sounds like high school all over again. Why does anyone in an office put up with it?? is this somehow different than time keeping? do you have a punch card?

If you asked my boss where i was, shed probably just point at the garage and wave at the 'authorized personnel only' sign. She has no reason to care what size wrench im holding or why im beating the living shit out of a siezed idler pitman assembly. I once drove a car with no doors and a missing windshield out of the lot and down the street, and the only thing she wanted from me was to know if id run to leroys donut and pick up a frozen coffee.

Why is this different in an office? are office jobs just not trustworthy?

<comment><USER14> No one in an office job knows if anyone else is doing anything useful. Or even if _they 're_ doing anything useful.

<comment><USER15> Au contrair. mon ami. Office workers are busy keeping themselves in the office, which means that the vacation spots, and such are riffraff-free for their betters until the next major holiday break. That's _quite_ useful.

<comment><USER15> I see some people lack a sense of humor and/or that a 'truth' perhaps cuts too close to the bone.

</comment></comment></comment></comment><comment><USER16> It should be titled: How employers lose their best employees.

There are certain jobs that do require extra security and monitoring. But if a manager actually thinks their white collar underling is going to perform better, they should not be a manager.

In the case of IT people, it will probably just get hacked and disabled. I've done it at past jobs. I've also flat out refused to implement this sort of crap.

</comment><comment><USER17> "It's possible that almost any change—even changing the lighting—would have prompted a similar increase in productivity."

Hawthorne effect: [https://en.m.wikipedia.org/wiki/Hawthorne_effect](https://en.m.wikipedia.org/wiki/Hawthorne_effect)

</comment><comment><USER18> Also relevant here is panopticon/panopticism

[https://en.m.wikipedia.org/wiki/Panopticism](https://en.m.wikipedia.org/wiki/Panopticism)

</comment><comment><USER15> > _"a culture where … people more often alter their behavior to suit machines > and work with them, rather than the other way around," and that this tends > to erode their sense of "agency." That is, the constant surveillance of > employees diminishes their capacity to operate as independent thinkers and > actors._

Isn't this the point? At least until the _people can be replaced_? Personally, can't wait to see the autonomous robots attempts to subvert the system. That should be interesting.

</comment><comment><USER19> How ubiquitous is this for TSA? I hear people on hackernews complain about theft from the TSA every so often, but according to this, they're being recorded so it shouldn't happen, or if it does, they should be able to prove if it was stolen by TSA.

Are people just blaming TSA for their lost things, is TSA covering up thefts that they know are happening, or what?

<comment><USER20> It said they were told that's why they were being recorded, it didn't say it was true.

It's very hard to be certain what's going on unless we install tools that monitor management. On the plus side studies show this would improve their productivity.

</comment></comment><comment><USER21> These type of "tools" are used by low-lives to control people so that they feel that they have some sense of control over people. In terms of getting good hard-working motivated people to work for you, this sort of technology does exactly the opposite.

<comment><USER22> It makes me think of a piece from Peopleware (1987):

> Historians long ago formed an abstraction about different theories of value: > The Spanish Theory, for one, held that only a fixed amount of value existed > on earth, and therefore the path to the accumulation of wealth was to learn > to extract it more efficiently from the soil or from people's backs. Then > there was the English Theory that held that value could be created through > ingenuity and technology. [...]

> The Spanish Theory of Value is alive and well among managers everywhere. You > see that whenever they talk about productivity. Productivity ought to mean > achieving more in an hour of work, but all too often it has come to mean > extracting more for an hour of pay. There is a large difference. The Spanish > Theory managers dream of attaining new productivity levels through the > simple mechanism of unpaid overtime.

</comment></comment><comment><USER23> The bank I was working at tried installing monitoring software on programmers' computers one day with no monitoring.

I sent my MD an email saying I wouldn't be working that day because of it, and left. The software disappeared the next day.

<comment><USER24> Most companies I've worked for (including a bank) you'd be gone the next day. Monitoring is so rampant right now. It's disgusting.

<comment><USER25> Indeed, I'm working at a bank at the moment. There are half a dozen "endpoint protection" agents installed on my laptop, and don't even mention the network filtering - the Docker, Nginx and Vagrant documentation websites are all blocked for example.

I try to work mostly on my own laptop using Citrix in a web browser to access the corporate network so that the personal data I generate in a day (for example, browsing HN during downtime) doesn't get caught up in all the monitoring but this has some obvious limitations. With the security regime in place they will actually ask you about every piece of software you have installed on your corporate laptop (on which you'll need to request temporary admin access to install most software, of course) or about why you visited x, y or z website or why you searched this or that on Google. Like the article says it's best to keep your head down and try not to get noticed - <USER2> collect so much information but nobody looks at it until they have a reason to do so.

</comment></comment></comment><comment><USER26> IMHO having a camera over your head is just one (small) step worse than working in open-plan offices. It got popular only for one reason, for just like cameras it gives the management that false sense of control.

</comment><comment><USER27> Workers who are being surveiled should do the bare minimum while developing skills during free time that benefit future employers and not the current one.

<comment><USER28> Sadly, I think this almost always should be the case. That is, unless the employer actually is benevolent and puts effort into nurturing the futures of its employees. But how often do we see that?

People should basically work no more than 4 hours a day, and the remaining 4 hours should either be spent developing skills or running a side-hustle on a personal laptop. Better that than bullshitting, which is what the vast majority of people do after they're burned out working 3 hours.

</comment></comment><comment><USER29> Completely off topic but The Atlantic's GDPR notice is the least intrusive and most compliant I've seen so far. Good job.

<comment><USER30> Try [https://wikia.com](https://wikia.com). No "manage my preferences" or "are you sure" etc, just a simple yes/no.

As a side note, basically no one complies with Article 7.3.4: "It shall be as easy to withdraw as to give consent." ([https://gdpr- info.eu/art-7-gdpr/](https://gdpr-info.eu/art-7-gdpr/))

<comment><USER29> Personally I prefer the finer grained controls of The Atlantic, but that is still better than 99% of the rest. Though I'm not sure about the wording, it only says 'advertising cookies'.

</comment></comment></comment><comment><USER31> so yeah, if a company considers surveillance of employees' performance to be documentation of their work for the company then all employees will do is just be surveilled.</comment>


Reed Solomon codes are cool - <AUTHOR> https://djhworld.github.io/post/2019/02/24/reed-solomon-codes-are-cool/ <comment><USER2> Wow. I love explaining Reed Solomon codes and this didn't do it justice.

The basic concept is that if I give you five points all on the same line, you only need any two of them to reconstruct the line.

Reed Solomon doesn't use lines (it isn't optimal) and the geometry they use isn't the Cartesian plane (this requires infinite precision) - but this is what the codes do!

Just like it requires two points to reconstruct a line, three points are required to reconstruct a simple polynomial (degree 2). Reed Solomon uses "high degree polynomials" which require n+1 points to reconstruct the degree-n polynomial.

The codes take data, make a polynomial out of it, and then send points from the polynomial to the other side, which can interpolate it as soon as enough points are gathered!

All of this looks like impenetrable discretization and matrix operations if you just get a description of the algorithms, which makes it seem a lot less approachable than it is.

<comment><USER3> I remember playing a computer game in math class where you were presented with a XY plane and had to "hit" points (really circles, didn't need to be exact) by plotting equations through them. With some trial and error you could make a polynomial go through basically as many as you wanted, it's neat to see this applied to a real problem!

<comment><USER4> Doing this kind of thing was my first introduction to Lagrange interpolation, too, but it turns out that you can do Lagrange interpolation without trial and error; you can just use linear algebra, since the points are a linear function of the coefficients, regardless of the degree of the polynomial.

This was in fact the first decoding algorithm for Reed–Solomon codes, but it's not very efficient when you don't know which of your points are the ones with the corrupted data; you kind of have to guess, which gets expensive fast when you're trying to tolerate more than one or two errors.

Although it turns out that there are better algorithms for that, the most commonly used RS codes _don 't_ actually encode a series of points on a polynomial, as subjoriented's comment at the root of this thread suggests; instead they are so-called "BCH codes", where you consider the data you transmit (both the original data and the "parity" symbols) to be actually a sequence of _coefficients_. So where does the redundancy come from? After all, any set of numbers is valid as the coefficients of a polynomial.

BCH codes require the polynomial to be _divisible_ by a known _generator polynomial_ , and that's where you get the redundancy you need to correct errors. Gorenstein and Zierler published an efficient error-correction ("decoding") algorithm for BCH codes in 1960; the first efficient decoding algorithm for original-view RS codes (where you transmit a series of points) wasn't found until 1986, and even today, BCH-code decoding algorithms are more efficient than original-view decoding algorithms.

The Gorenstein–Zierler algorithm works by evaluating the received polynomial at the roots of the generator polynomial. Since it's supposed to be a multiple of the generator, it should be zero at those roots, as the generator is; if it's nonzero, the values at those roots are due to some "error polynomial" that's been conceptually _added_ to your message in transit. If you suppose that it has only a few nonzero coefficients, there's a clever way to compute the error polynomial from those values that should have been zero. This allows you to subtract it from the received message to correct the errors.

At least, I think that's how it works. I haven't implemented an RS decoder yet, so I might be getting some of this wrong.

But something like that was what I was actually hoping to read at the above link.

<comment><USER5> I explained all the linear algebra involved in the Peterson-Gorenstein-Zierler decoder and offer a working implementation in [https://www.nayuki.io/page/reed-solomon-error-correcting- cod...](https://www.nayuki.io/page/reed-solomon-error-correcting-code-decoder) ; [https://news.ycombinator.com/item?id=13565939](https://news.ycombinator.com/item?id=13565939)

It runs in O(n^3) time, and unfortunately is not the state of the art. What actual Reed-Solomon implementations use are the Berlekamp-Massey and Forney algorithms, which run in O(n^2) time.

<comment><USER6> Incredible. This is the sort of thing my non-programmer friends think that my work entails. Thank god it doesn't.

</comment></comment></comment></comment></comment><comment><USER7> If this interests you, make sure to look into Golay codes: [https://en.wikipedia.org/wiki/Binary_Golay_code](https://en.wikipedia.org/wiki/Binary_Golay_code) This code is a mathematical miracle. Apparently, number theory comes together exactly such as to allow you to code 12 bits into 23 bits. Such that you can correct all possible 3 bit errors, but none of the possible 4 bit errors (which means it is efficient). AND it detects all of the possible 7 bit errors, but none of the possible 8 bit errors.

That is very efficient. So in exchange for storing all your data about twice, you can detect all errors where ~30% of the bits are flipped. You can even _correct_ all errors where only 12% of the bits were flipped!

I still consider it magic that such an efficient and 'perfect' code exists. There is unfortunately a proof that is the best one, and that there is no bigger perfect one.

<comment><USER8> Everything related to the sporadic groups is crazy [https://en.wikipedia.org/wiki/Sporadic_group](https://en.wikipedia.org/wiki/Sporadic_group)

</comment></comment><comment><USER9> This post should be titled "Error correcting codes are cool", as it contains zero bits of information that's specific to Reed-Solomon codes.

If you're interested in Reed-Solomon codes (and other error correcting codes), I recommend reading Huffman's (NB: _not_ the same Huffman as in Huffman codes!) textbook on the subject:

Fundamentals of Error Correcting Codes

[https://www.amazon.com/Fundamentals-Error-Correcting- Codes-C...](https://www.amazon.com/Fundamentals-Error-Correcting-Codes-Cary- Huffman/dp/0521131707)

<comment><AUTHOR> Fair comment, I'm new to this subject so wanted to write about what I was reading about, but looking forward to reading about this topic further!

</comment></comment><comment><USER10> I got excited about erasure codes a while ago and tried my best to give an accessible intro at [https://www.akalin.com/intro-erasure- codes](https://www.akalin.com/intro-erasure-codes) (with HN discussion at [https://news.ycombinator.com/item?id=18702843](https://news.ycombinator.com/item?id=18702843) ).

Glad to see other people getting excited by them too!

<comment><USER4> This looks great! I haven't finished reading it, but does it explain Gallager decoding or RS decoding? At a first skim, it doesn't seem to, but that might be my lack of knowledge of the topic.

<comment><USER10> Thanks! It doesn't explain Gallager codes since I don't know much about them (but will look into them now!). It only briefly mentions RS decoding, since the RS erasure code uses Vandermonde matrices, and Cauchy matrices seem more foolproof for an intro article. I was planning on writing a follow-up article on PAR1 (which uses RS erasure codes, but slightly incorrectly) but alas, my free time isn't what it used to be...

</comment></comment></comment><comment><USER11> One of the things that I miss about analog electronics is how the signal degrades nicely. For example, with FM, if I drive under a bridge the radio gets a little fuzzy or switches to mono. With newer digital satellite radio, the playback just stops.

Evan back with old analog TV broadcasts, if there was a signal issue, the picture would just get fuzzy. Now with HDTV, the picture goes away completely.

I know a lot of this has to do with analog formats being highly inefficient and thus having lots of redundancy.

<comment><USER12> I distinctly remember that one of the marketing features of DVB-T was "perfect picture or none at all" while I haven't seen any consumer grade DVB-T receiver that actually behaves that way and instead marginal signal results in various ugly MPEG artifacts.

<comment><USER13> I've not used DVB-T, but ATSC (which on Linux uses the DVB-T subsystem); the range of conditions where you can get perfect picture is much larger than from analog broadcast TV, but the experience with marginal signals is indeed much less watchable. If you've got nothing else to watch, you can watch an analog broadcast that's more snow than picture, as long as you're getting the sync signals well enough.

</comment></comment></comment><comment><USER14> After learning about Hamming code (the predecessor of Reed-Solomon), I wrote a tutorial that explains Hamming code and how to implement it in a simple simulator. [https://manuelfi.com/blog/hamming-code- simulator/](https://manuelfi.com/blog/hamming-code-simulator/)

<comment><USER15> Hi major, I just read your post and its great, thank you, it really has inspired me to learn more on erasure codes. I only have one doubt: in the Example Problem, I understand the 4th parity bit should be the 8th position (starting the first at 1), however you placed it in the 9th position, and I think this is the reason why (according to me) in the 2nd parity check the bit should be 1 instead of 0, however your answer was still right, so could you tell me what is the correct place for the 4th parity bit?

</comment></comment><comment><USER16> Computerphile did a video recently giving a 10,000 foot overview of Reed Solomon codes [https://www.youtube.com/watch?v=fBRMaEAFLE0](https://www.youtube.com/watch?v=fBRMaEAFLE0)

</comment><comment><USER17> My favorite class of error correcting codes is one I haven't seen mentioned here: rateless / "fountain" codes. [https://en.wikipedia.org/wiki/Fountain_code](https://en.wikipedia.org/wiki/Fountain_code)

You take any message and can generate any amount of coding symbols from the message, and a receiver that gathers _any_ K of those coding symbols can decode the message.

The mental image I have is using these to send messages into space -- a receiver could pick up at any point and just wait until they have any K coding symbols and they'll be able to decode the message.

My understanding of all this stuff comes from USENET and par/par2/yenc (where, IIRC, the message gets packaged up in plaintext blocks and if your local server is missing N blocks you have to collect N parity blocks and you can recover from there, but my recollection is probably fuzzy), and I imagine rateless codes would be quite useful for USENET binaries.

</comment><comment><USER18> You know what? Not just Reed-Solomon codes, but Information Theory is cool.

People don't realise how ubiquitous different forms of encoding schemes are (I am leaving out compression/decompression schemes altogether here, but it has always amazed me as to what profound impact Information Theory has, and yet it is so out of the way):

Forward error-correction codes (FEC) at L2: IBM's elegant 8b/10b encoding [https://en.wikipedia.org/wiki/8b/10b_encoding](https://en.wikipedia.org/wiki/8b/10b_encoding) used in pre-10g era Ethernet, and its successor 64b/66b [https://en.wikipedia.org/wiki/64b/66b_encoding](https://en.wikipedia.org/wiki/64b/66b_encoding) used in post-10g era Ethernet / Infiniband / RoCE, and the sister 128b/132b encoding used by PCI-e 3 and USB 3 standards.

Linear error-correcting codes (LEC) for Volatile Memory: [https://en.wikipedia.org/wiki/Hamming_code](https://en.wikipedia.org/wiki/Hamming_code)

FEC for HD Voice Transmission over Cellular Networks: AMR used in 3G and VoIP [https://en.wikipedia.org/wiki/Adaptive_Multi- Rate_audio_code...](https://en.wikipedia.org/wiki/Adaptive_Multi- Rate_audio_codec) and AMR WB for VoLTE aka HD Voice: [https://en.wikipedia.org/wiki/Adaptive_Multi- Rate_Wideband](https://en.wikipedia.org/wiki/Adaptive_Multi-Rate_Wideband)

Fountain Codes for Digital TV on Mobile (DVB-H): Qualcomm's Raptors [https://en.wikipedia.org/wiki/Raptor_code](https://en.wikipedia.org/wiki/Raptor_code)

Turbo Codes for 3G/4G Connectivity (OFDM), Satellite communication: [https://en.wikipedia.org/wiki/Turbo_code](https://en.wikipedia.org/wiki/Turbo_code)

Optimal Erasure Coding (for data at rest, and for Mobile TV, and for 3G/4G): Reed-Solomon and its variants are a clear winner here (used by dropbox [https://blogs.dropbox.com/tech/2016/07/pocket- watch/](https://blogs.dropbox.com/tech/2016/07/pocket-watch/) and rumored to have been used by S3 [https://perspectives.mvdirona.com/2009/06/erasure- coding-and...](https://perspectives.mvdirona.com/2009/06/erasure-coding-and- cold-storage/) )

Upcoming: [https://en.wikipedia.org/wiki/Low-density_parity- check_code](https://en.wikipedia.org/wiki/Low-density_parity-check_code) LDPC codes are recommended for use by data-channel in 5G replacing Reed-Solomon whilst Polar Codes are adopted for the control-channel in 5G [https://en.wikipedia.org/wiki/Polar_code_(coding_theory)](https://en.wikipedia.org/wiki/Polar_code_\(coding_theory\))

\---

Ref: [https://en.wikipedia.org/wiki/Error_correction_code](https://en.wikipedia.org/wiki/Error_correction_code)

<comment><USER19> Are any of the ones that are better than Reed-Solomon either open-source or otherwise out of patent protection yet? A library I could play with, perhaps? (would love to get my hands on a Raptor or Turbo implementation)

<comment><USER20> libraptorq (lgpl3+): [https://fenrirproject.org/Luker/libRaptorQ/wikis/home](https://fenrirproject.org/Luker/libRaptorQ/wikis/home)

zfec (gpl2+): [https://github.com/tahoe-lafs/zfec](https://github.com/tahoe- lafs/zfec) (discussion: [https://news.ycombinator.com/item?id=12976168](https://news.ycombinator.com/item?id=12976168))

liberasurecode (bsdesque): [https://github.com/openstack/liberasurecode](https://github.com/openstack/liberasurecode)

cm256 (3 clause bsd): [https://github.com/catid/cm256](https://github.com/catid/cm256)

openrq (apache2.0): [http://openrq-team.github.io/openrq/](http://openrq- team.github.io/openrq/)

openfec: [http://openfec.org/](http://openfec.org/)

lt-codes-python: [https://github.com/Spriteware/lt-codes- python](https://github.com/Spriteware/lt-codes-python)

<comment><USER19> The libraptorq link isn't loading, but I will try to Google.

Regardless, thanks for these!!

</comment></comment></comment></comment><comment><USER21> Would love to read popular explanations as well as opinions on Mojette Transform[1] as and alternative (superior?) technique for erasure coding.

1\. [https://en.wikipedia.org/wiki/Mojette_Transform](https://en.wikipedia.org/wiki/Mojette_Transform)

</comment><comment><USER22> This is a very practical explanation of how to actually implement RS codes with a focus on QR codes, and one of the best ones I've come across that doesn't immediately go into the deep theoretical maths of it: [https://en.wikiversity.org/wiki/Reed%E2%80%93Solomon_codes_f...](https://en.wikiversity.org/wiki/Reed%E2%80%93Solomon_codes_for_coders)

It's very enlightening to see finite-field arithmetic implemented in a few lines of Python --- it gives something concrete to look at and think about.

</comment><comment><USER23> Last month I tried to use Reed Solomon ECC in order to do error correction. I stopped using it, the moment I understand that error correction does not necessarily mean "error detection". If one does not exactly know the number of possible errors within encoded data, RS-ECC may "correct" to a wrong result (if the data contains more errors than the ECC is able to correct). So checksums might be the better approach (in certain scenarios) if data integrity has the highest priority... I wish, Reed-Solomon would handle that.

<comment><USER24> Or both. Much like encryption, to avoid various attacks you need to add a checksum and then encrypt.

</comment></comment><comment><USER25> RS codes give great reliability but galois polynomial solvers are slow. Parity (xor) bits are fast, but they can't fix more than one error, which means it's really not reliable to fix an error at all. Use parity bits to check for an error, and then RS to fix it.

That backblaze speed quoted seems really slow.

<comment><USER24> Yes, but it's Java. Read the comments for a version in go that's "an order of magnitude faster". For their use case though I suspect that handling GigE per server is plenty.

</comment></comment><comment><USER26> So is the idea basically that you can recover any missing n% of a dataset by using an extra n% of disk space to store specially calculated data, rather than 100% extra disk space (because you don't know which pieces are going to go missing)?

</comment><comment><USER27> I do use it in LibreDWG, but I don't find it cool or useful enough. For CD's or WiFi with lossy transports yes, but for harddiscs (bitflips don't happen) not needed and not so easy to plug in.

<comment><USER28> Consumer HDDs get bit flips all the time. I have plenty of backup files over 100 GB. Put the file on the drive, compute the MD5SUM of it, check the MD5SUM, and come back to check again in 6 months. On many magnetic HDDs, you will get a different value and no SMART errors.

<comment><USER27> Whow, thanks

</comment></comment></comment><comment><USER29> I'm quite disappointed that nobody mentioned Viterbi algorithm, as it's often used together with RS and really interesting.

</comment><comment><USER30> Obligatory computerphile video: [https://www.youtube.com/watch?v=fBRMaEAFLE0](https://www.youtube.com/watch?v=fBRMaEAFLE0)</comment>


Poynt – Smart payment terminal - <AUTHOR> http://www.getpoynt.com/

<comment><USER2> Interesting merchant terminal -- but it seems to be little more than a pretty rendering and physical mock-up right now.

I like how the first guy puts his card in, takes it out AND THEN punches in his PIN, which is exactly how PIN & Chip doesn't work. I would have hoped they'd at least be familiar with the process.

It looks to me like a mobile computer strapped to a terminal. I'm not sure why this is better than having a terminal + iPad or something similar but I'm not the target demographic.

I see many other issues with it:

\- Hardware development is hard and long. This looks to be little more than a physical mock-up. For example datasheet lists "Ethernet" is connectivity options but there is no ethernet port visible on the body.

\- BLE & GSM antennas are exactly where you wouldn't put either of those. Here is tip: You can't put an antenna behind an LCD. They have metal backings.

\- Shouldn't QR & Barcode Scanner be pointing UP? You have very limited field of view when it is pointing down.

\- Getting EMV and FIPS certified are going to take a long long time. Granted they could buy that firmware off the shelf to accelerate the process.

<comment><USER3> Some ATMs require you to take your card out before entering your PIN. Or are they using the magnetic strip?

<comment><USER2> If you are using magnetic strip you're correct. For PIN & chip the PIN is sent to the card and verified by the card and the resulting "signature" is sent to the bank. The card has to be in for PIN entry.

An employee, above, said that it is for entering tip. Which would make more sense, but then the card would have to be inserted after the total amount I imagine but I can see how they would have a slightly different flow. Personally, I would want any terminal that I use to show me the final amount that I am going to pay before I put in my card. Lest I pay, walk away and the cashier adds her own tip.

</comment></comment></comment><comment><USER4> This is the first EMV payments terminal I've seen that allows PIN entry using a touch screen. Is that really allowed by EMV, as a tamper-evident PIN pad?

<comment><USER5> I've seen a few merchants using a touch screen, including a few Gap stores in Montreal.

<comment><USER6> Yeah, the touchscreen terminal that GAP uses is pretty common. Canada Post uses it too, and i know i've seen it in some other places.

</comment></comment></comment><comment><USER7> If someone from Poynt is on, what are the features that make this better than competing products? I've used Shopify for doing some PoS setups, but I'm having a hard time looking at where Poynt fits in. Other than being cheaper, it lacks some of the things that make Shopify great, namely cash handling and connecting to a back office system.

<comment><USER8> Hi, I work for Poynt on the PoyntOS and applications. First thing first, Poynt Smart Terminal is a Secure Payment Terminal at the core that provides credit card payment processing functionality to any merchant that needs to accept payments in their stores. This is analogous to the traditional payment terminals that you might probably notice every day at a lot of merchant stores (cafes, salons, grocery stores, etc.). Poynt Smart Terminal enhances the credit card payment processing functionality by supporting more payment methods (MSR/NFC/EMV/QR Code/etc.) so the merchant doesn't need to worry about carrying multiple devices in the store and be able to serve more customers. In addition to that, Poynt Smart Terminal runs on Secure PoyntOS (powered by Android) to provide an application platform and framework for developers to build apps and solutions for the merchants and distribute through Poynt. Developers can take advantage of the PoyntOS to build unified solutions for the merchants that run on the same device where they are used to process payments (unlike various solutions today in the market that require additional hardware like ipads, dongles, etc.). Shopify is an online platform that allows developers to build solutions for their merchants hosted online or on devices. Now with Poynt Smart Terminal, developers can build PoS apps using Shopify platform that run on Poynt Smart Terminal too. So I would probably say they are complementary, not competing.

Cheers!

<comment><USER9> Should definitely highlight (at least to the HN crowd) that this is an Android device and you're exposing some APIs that can be built on. This is _really_ cool.

</comment></comment></comment><comment><USER10> This device is insecure. It is going to take a long time to get the PCI/EMV certifications. Besides that how do you handle the certification of new applications running side by side with your payment application? Everytime you deploy a new application you should re certificate the entire stack, by the PCI standards. Nice concept, but you have a long road ahead before competing with VeriFone, Ingenico, PAX, Miura shuttle and others.

<comment><USER8> Hi - I work for Poynt on the PoyntOS and Payment interfaces - so maybe I can provide some clarity without going into too much of our IP. As mentioned on our site ([https://getpoynt.com/specs](https://getpoynt.com/specs)), we have two separate subsystems - one for Android and the other for secure payment processing.

All the payments (EMV/NFC/MSR), secure key (including acquirer keys) management, P2PE encryption, EMV/PCI, etc. are handled by the secure processor. There are no other applications that can run on this secure processor other than the signed and certified applications.

On the Android side, Poynt's Secure service is the only service that's capable of communicating with the Payment Processor to initiate card reading (EMV/NFC/MSR/others) and pass through the encrypted data it receives to the merchant's acquirer. All the 3rd party applications run independent of the Poynt's Secure Service and when they need to collect a payment, they do so through our Poynt Payment Fragments to facilitate the Payment flows. (See here for information on how it works: [https://getpoynt.com/developers/terminal#2.3](https://getpoynt.com/developers/terminal#2.3) Poynt Payment Fragments).

So as you can see, we are able to keep the security domains separate and thereby able to handle PCI certification in a much more graceful way. Obviously they are some complexities but choosing a certifiable payment processor board was one of many ways we are able to deliver a secure solution.

Cheers!

<comment><USER6> How are you securing the PIN entry? It looks like that happens on the same screen as the random 3rd-party apps get to run on, leaving open the potential for an app to intercept the PIN. As i understand the PCI stuff, anything that the PIN hits is fully in-scope.

<comment><USER10> The same question here. Anyone can develop and 3rd-party app to capture the PIN on the same screen from the payment app.

<comment><USER8> A rogue app asking for PIN on the merchant facing screen ? not sure there's anything much we can do about that other than making sure we catch that during the review process. Whenever there is a need for the consumer PIN entry, it's driven by the second payment processor - not from the android side.

<comment><USER11> Should be able to prevent PIN information from getting accepted by any means other than your locked-down PIN entry screen. So, any app that wants to grab people's PIN entry would either require them to enter their PIN twice, or block the transaction from going through, which should be _very_ visible.

</comment></comment></comment></comment></comment></comment><comment><USER2> Interesting to see Osama Bedier listed as one of the people. He is the guy who ran PayPal's merchant terminal integration efforts, left for Google Wallet, and then left Wallet.

I think there was a lawsuit filed against him and Google by PayPal the day of Wallet's launch, claiming Google stole their secrets. No idea how that turned out; though I imagine it was a PR move on PayPal's part.

</comment><comment><USER12> Apple Pay is supported.

> Does the Smart Terminal accept payments by Apple Pay?

> Yes, the terminal accepts Apple Pay since the iPhone securely communicates > with the terminal through NFC. We here at Poynt made our first Apple Pay > payment with our Smart Terminal on the morning the software was released to > the world. It was a very exciting moment for us!

</comment><comment><USER13> And here I thought the other Poynt was pivoting or reinventing itself. [http://www.poynt.com/](http://www.poynt.com/)

Probably not the best idea to launch with a name identical to another start-up that also has apps on all major platforms...

</comment><comment><USER14> Poynt already exists. Can I use Poynt to find retailers who are using Poynt?

[https://play.google.com/store/apps/details?id=com.poynt.andr...](https://play.google.com/store/apps/details?id=com.poynt.android)

<comment><USER8> Hi - they are not related to each other in any way. Poynt.co or GetPoynt.com is the new Smart Payment Terminal that was announced today.

[https://getpoynt.com/about](https://getpoynt.com/about)

</comment></comment><comment><USER15> One of my concerns is that it appears the door to change the receipt roll is on the bottom of the device. Does this mean I have to shut the device off and flip it over if it runs out of receipt paper to print a receipt for a customer?

<comment><USER8> it's actually in the front of the device as you see on the website ([https://getpoynt.com/](https://getpoynt.com/)) and the button to open the printer door is on the side. The door opens forward and you can load the paper from the front (no need to turn it upside down). We will try to post videos of the paper loading as soon as we can.

</comment></comment><comment><USER16> It seems like a good time for this product. Payments feel like the wild west right now. A reader that can potentially handle all protocols is a safe investment in the future.

</comment><comment><USER17> What's to stop somebody from stealing this right off the counter and gaining access to customers' data?

<comment><USER18> A very valid concern.

To begin with, while our terminal is Android based we have taken numerous steps to lock this device down. Side loading apks is not possible nor is arbitrary access via adb. On top of that, we take great lengths to protect consumer data. In addition to full PCI compliance data is fully encrypted on the device. And if that's not enough, there are several anti-tamper mechanisms that will trigger and lock down the device even further upon physical instrusion.

In terms of physical theft we are actively looking into an option to physically secure the device (think kensington). Our plan is to have a good solution for this before our merchants go live.

<comment><USER2> Well, here is my big question: _WHY ARE YOU STORING CARD DATA AT ALL_? (sorry for the caps). You are a pass-through entity, merchant terminals do not store card data. They keep the authorization number from upstream provider to allow void/refunds but there is no need for them to store the number.

With respect to anti-tamper mechanism, are you FIPS-140-2 certified or plan to be?

<comment><USER18> We aren't storing actual card data encrypted or otherwise. As you said, we are a passthrough as far as the payment portion is concerned. We do store a hashed representation of the card for things like refunds (referenced credits).

Our security subsystem is being built to be FIPS 140-2 Level 3. Complete with tampers seals, switches and a security mesh that will destroy sensitive keys when triggered.

</comment></comment></comment></comment><comment><USER19> Is Poynt classified as a 'handheld' terminal to address the privacy shield requirement in PCI PTS?

</comment><comment><USER20> I would love to have one to code a nice bitcoin payment gateway on it :) anyone interested to collaborate?

<comment><USER21> If you decide to build something on our platform, please fill out the form at [http://goo.gl/forms/dgwMwDysAv](http://goo.gl/forms/dgwMwDysAv) to start a conversation with us. Good luck!

</comment></comment><comment><USER15> I'm unable to watch the video from my Android phone.

<comment><USER22> Hi, thanks for letting us know -- are you still having problems? We enabled the embedded video on mobile devices a little while ago.

<comment><USER15> It works fine when I open the page with my phone held vertically. When I hold the phone horizontally the button doesn't seem to work though.

Opening the video with the phone held vertically and then rotating the phone once the video loads works fine, however.

</comment></comment></comment><comment><USER23> Looks cool to me.

</comment><comment><USER24> No wifi?

<comment><USER25> [https://getpoynt.com/specs](https://getpoynt.com/specs)

Yes wifi.

</comment></comment><comment><USER26> Anything to keep the mass from ditching the plastic, cause when that happens, VISA, MC and co are in jeopardy.

<comment><USER27> The credit card processors do far more than just issue magic pieces of plastic. The plastic itself is probably the least significant aspect of their business model.

<comment><USER26> The business model relies on a monopoly, an information monopoly. If the mass's interaction at POS is with a software or a hardware that potentially allows id-ing the patron, preferential pricing becomes possible, security is anon issue -- it seriously damages the value of the middleman.

So that the secure element(yeah right) that's on your plastic or your phone only communicates with OSes and HW that limits the ID-ing of the cx and provides the strict minimum of info to your POS is the critical part, in my view.

<comment><USER27> If privacy were really the concern, people would pay with cash. They always have, and always will. It is accepted everywhere, is the definition of liquidity, and is virtually untraceable. At best, in-person swiping of physical credit cards is a compromise that people make that gives away some of that privacy (your card provides your name, and can be used to verify your address) in exchange for something more valuable:

1) near immediate payment resolution

2) fraud protection

3) insurance against bad products and vendors - you can almost always get your money back if something goes wrong

4) credit accounts

5) points/miles/cash-back, at the expense of the vendor

<comment><USER26> How many different merchants to do you deal with on a regular basis?

What is the risk of fraud or of bad product and or bad vendors when both ends know each other? Why introduce a 3rd party in those transactions that gives back a fraction of its fee to the payer? (5) Instead why not reward repeat business with preferential pricing? (not directly possible but attempted via loyalty programs that cost a lot to deploy)

The credit is provided by the issuing bank (4). It can exist independently of the credit card network.

And with any electronic means, 1 is pretty much a given.

Credit 3rd parties still have a reason d'être if a solution that shortcuts VISA and MC when both ends know each other catches on. A Credit 3rd party is needed as an insurance policy when two unknown parties do a transaction; that insurance is bound to cost more than today given less volume; but it could be efficient if pricing would be market driven instead of diluted in 100-1 day to day transactions.

Thanks to the down voters, BTW. Again HN is showing openness to look at things from a totally non conventional angle. I think I'll log off for good. So long, and thanks for all the fish!</comment></comment></comment></comment></comment>


Candy crush Maker King valued at $7.1B - <AUTHOR> http://www.reuters.com/article/2014/03/26/kingdigital-ipo-idUSL4N0MM4PD20140326

<comment><USER2> Somehow wishing FB had acquired King instead of Oculus Rift.

</comment><comment><USER3> How on earth?

<comment><USER4> Don't worry it's already down 13% and the day isn't even over yet. I've got some faith in the market on this one.</comment></comment>


A typical day at Intel Finland - <AUTHOR> http://www.intel.com/en_UK/CannonBells/index.htm

<comment><USER2> Is this an "ad" for Intel's processors' capacities for graphics or physics?

I still don't know, but this reminds me of this German DIY sore ad (which is a fake): <http://www.youtube.com/watch?v=Qyv3q_yr8tI></comment>


Show HN: Aqua – A website and user system (Hapi/React/Flux) - <AUTHOR> http://jedireza.github.io/aqua/

<comment><USER2> I don't understand what does it do. A "website" for what? A user system? It makes a website for me in which users can sign up and login, but then what?

<comment><AUTHOR> It's a starter/boilerplate.

</comment></comment><comment><USER3> When I move between the pages its noticeably slow, almost like you're actually loading a new page. I thought React was supposed to be fast?

<comment><AUTHOR> Yes. It's more like a few single page apps. The account and admin areas feel much faster.</comment></comment>


Strfriend: visualize regular expressions simply - <AUTHOR> http://strfriend.com/

<comment><USER2> I pasted my favourite regex in the box and it errored.</comment>


Is Guy Fieri in Your Node.js Packages? (2016) - <AUTHOR> https://nodesource.com/blog/is-guy-fieri-in-your-node-js-packages/ <comment><USER2> Needs a [2016]. Some info is relating to npm@2 package structure which is long out of date.

<comment><USER3> Ha, I was wondering why I couldn't recreate and then I saw in their changelog they removed Guy Fieri from the source code. I bet that was a sad day. The end of flavortown.</comment></comment>


Slingshot: Enhanced try and throw for Clojure - <AUTHOR> https://github.com/scgilardi/slingshot

<comment><USER2> On a related note, Michael Drogalis has done great work developing Dire, which now works with Slingshot. Slingshot allows for a great deal of information to be passed along with what would be exception messages, and Dire brings something like Erlang style supervisor trees to Clojure:

[https://github.com/MichaelDrogalis/dire](https://github.com/MichaelDrogalis/dire)

I do think the combination of Slingshot and Dire gives Clojure something similar to the condition handling system of Common Lisp. If you don't know about Common Lisp's condition system, you really should read this:

[http://www.gigamonkeys.com/book/beyond-exception-handling- co...](http://www.gigamonkeys.com/book/beyond-exception-handling-conditions- and-restarts.html)

<comment><USER3> Thanks for this. Dire looks like a fantastic addition to the Clojure eco- system!</comment></comment>


Why Do Obese Patients Get Worse Care? Many Doctors Don't See Past the Fat - <AUTHOR> http://www.nytimes.com/2016/09/26/health/obese-patients-health-care.html?hp&action=click&pgtype=Homepage&clickSource=story-heading&module=second-column-region&region=top-news&WT.nav=top-news&_r=0 <comment><USER2> > _Be civil. Don 't say things you wouldn't say in a face-to-face > conversation. Avoid gratuitous negativity._

Much of the discussion in this thread does not meet this standard of discourse that we ask for on Hacker News. It's all too easy to respond reflexively (and thus with less likelihood of reaching this bar) when commenting on controversial issues, and while that might be OK elsewhere, it isn't here.

</comment><comment><USER3> I'm disappointed by the lack of empathy in these comments.

While I agree that yes, an obese person needs to lose weight, telling them that and offering no other help is like telling someone with depression "why don't you just cheer up?" It's totally unuseful advice that incorporate all sorts of gross value judgments about the person. It doesn't solve problems.

I've been overweight - not obese, but close at times - all my life. I'm now on a weight loss program and seeing significant success. It took years of mental and physical issues to get me to that point and it is a struggle every day. It's hard work. That's why people are proud of losing weight: it's an achievement. It represents a permanent change to lifestyle and behavior that takes weeks, months, even years to effect.

Please don't discount people's experiences by saying things like "oh, just lose weight." Consider the human being in there, and how to reach them. Don't assume that because it's easy for you to be fit and skinny, it's easy for everyone.

<comment><USER4> It's infuriating. HN commenters will fall over themselves to rail against discrimination, real or imagined, against women and minorities, but it's perfectly OK to insult, demean, and minimize the problems of big people.

They know nothing about their lives, why they are big, what their health problems come from, what they've tried to lose weight. It's easy to dismiss them as weak willed disgusting blobs of fat, ignoring the human being underneath that is trying to live their life as best they can.

<comment><USER5> The whole point of the majority of the conversation here is that there _isn 't_ systemic discrimination happening. People like to feel victimized and are claiming discrimination when it's overwhelmingly not the case, and dishonest articles like this do a huge disservice to the real, legitimate discrimination that _does_ happen.

<comment><USER4> Yes, the majority has investigated the majority and found the majority not guilty of discrimination. Please tell me more about how you know that big people aren't discriminated against.

<comment><USER5> I would if it wasn't the equivalent of talking to a child with hands over their ears yelling "LALALALALLALA"

<comment><USER3> Insulting parent commenter for asking you to support your arguments makes me feel that the child here is you.

</comment></comment></comment></comment></comment></comment><comment><USER5> Let me save you the time: it's an article pandering to fat people without any substance and really dishonest arguments.

The complaint is that doctors tell fat people to lose weight when they come in with symptoms that match the symptoms of being very overweight. If you go to the doctor with knee pain and you are drastically overweight, the most likely answer is that the knee pain is from being overweight. The doctor is going to tell you to lose weight. This doesn't mean they're _dismissing_ that it might be something else, which is evident by the example in the article where the root cause was also found.

I had to stop reading at the point where the quoted a doctor as saying

>doctors and hospitals have become risk-averse because they fear their ratings will fall if too many patients have complications

followed by

>A recent survey of more than 700 hip and knee surgeons confirmed Dr. Yates's impressions. Sixty-two percent said they used body mass index scores as cutoffs for requiring weight loss before offering surgery.

This survey doesn't confirm that surgeries are refused for the fear of lower ratings. It confirms that doctors have a BMI cuttoff for surgery. Saying that there shouldn't be a limit _somewhere_ is beyond stupid, meaning this survey is pointless. And it's frankly tremendously disrespectful towards doctors - they have BMI cutoffs for the sake of the patient's life. The risk of complications is extremely real and doctors don't want a high risk of literally killing someone to resolve a knee pain that, while crappy, is better than dying. And they're not even asking patients to live the rest of their life with that knee pain - only long enough they can lose weight and not have such drastically dangerous odds from the surgery.

This article is garbage.

<comment><USER6> This seems like an unfair characterization of the article. Denial of care, or dangerously mismeasured care, is not acceptable simply based on BMI (a non- scientific measure invented in the early 1800s). If you look at the statistics on weight loss success rates, asking a 500lbs person to lose 200+lbs before you're willing to treat their disease with actual evidence-based medicine—looking at symptoms, pathology, etc—is tantamount to outright denial of care. If doctors, after examining the patient, have reasons for postponing surgery that can be explained medically, fine. But just saying "we have a BMI cutoff" is pretty much discrimination masquerading as medicine.

<comment><USER7> > Denial of care, or dangerously mismeasured care, is not acceptable simply > based on BMI (a non-scientific measure invented in the early 1800s).

How did you conclude that BMI is non-scientific? Did you know that Centigrade was invented in the early 1800s, and the first realization of the metric system was in 1799?

BMI definitely isn't accurate for people with high lean body mass, but the patients in this article aren't bodybuilders whose muscle mass drives up their BMI--people with high lean body mass are less likely to need hip and knee surgery. Sure, BMI is an approximation with some well-known cases where it's inaccurate, but doctors are well aware of this fact.

> If you look at the statistics on weight loss success rates, asking a 500lbs > person to lose 200+lbs before you're willing to treat their disease with > actual evidence-based medicine—looking at symptoms, pathology, etc—is > tantamount to outright denial of care.

I'd like to see those statistics.

<comment><USER8> When I was in the best shape of my adult life, I weighed 175, with 10% body fat (measured with calipers) and a size 30 waist. I wasn't overly into lifting weights but did a lot of cardio. My BMI was 30 - obese.

5 years later. I do about 3 hours of cardio a week, I weigh 185 and my waist is about 32. My BMI is 31.8. No doctor has ever looked at me and said I need to lose weight.

BMI is completely bunk in my case.

<comment><USER9> You're 5'3" and weighed 175? You must be built like a tank

<comment><USER8> 5"4\. But like I said. I don't lift weights anymore at all. But most people assume I do.

As far as the caliper body fat test. It was done using 6 parts of the body.

</comment></comment></comment></comment></comment></comment><comment><USER10> Obesity is deeply repulsive on two levels. People who get to these extreme levels of obesity are basically suicidal. Aside from the obvious revolting physical appearance, there's an underlying message of "might as well die". Their chosen method of suicide is death by uninhibited indulgence in food.

<comment><USER4> How is it acceptable to call big people revolting and wave off their condition as "suicidal"? You know nothing of these people, or why they are big. They aren't here to look pleasing to you, they just want to live their lives. People are not always big by choice. Do you blame all alcoholics for their condition?

<comment><USER11> > They aren't here to look pleasing to you

So?

GP is not here to indulge them either. To each his own. They can choose to indulge on food, it's their freedom, and he can choose to ignore them, it's his freedom.

<comment><USER12> Another person who thinks obesity is caused by solely overindulgence.

Maybe next you would claim that people with movement disorders (e.g. in wheelchairs) are overweight because of overindulgence. Or maybe you will shift the blame to lack of exercise, not considering they would need multiple times more. Our maybe you will start designing diets for everyone that are trivial to adhere to and make millions as a life coach.

GP is there to treat, not indulge. If they are unable to treat obesity, they are to blame, not the patient. Obesity is extremely hard to treat even when there is will to change. Heck, the causes of it are not yet well understood. Something goes wrong with the feedback system of satiety and energy expenditure, which is evolutionarily stacked to accumulate calories.

<comment><USER11> I used to be obese.

I know what causes obesity.

It basically does boil down to over indulgence.

If you're in denial about that, I can see why you would be. It's no different than religious people denying science to protect their egos.

However, your sensitivity does not warrant that others should indulge your delusions and fantasies.

<comment><USER13> In light of your past experience overcoming your indulgence, you may wish to take some time to recognize that a community of informed, helpful people (basically HN) is pointing out that you're indulging in arrogance and self righteousness, and you might consider applying the same self control that you used to overcome obesity to your other character flaws.

<comment><USER11> HN is not pointing out anything to me. There's a couple of members here who seem to be involved in the "fat acceptance movement" who are defending the article.

<comment><USER13> Hi! I'm a fellow hn user.

You're being a jerk.

That work?

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER4> Irrespective of the article, the comments in here are appalling. Ask yourself if it's truly acceptable to paint big people with such a broad brush. HN commenters are generally liberal, and liberals (I am one as well, this is not an insult) will fall over themselves to decry real or perceived discrimination against women, minorities, or disabled people - unless that disability happens to be obesity. Would these comments be acceptable if the subject was some other class of people receiving inadequate care?

I'm a big guy. I've been big for 2/3 of my life. It fluctuates up and down as I've tried everything in the book to lose weight. I would _love_ to lose weight. I hate being fat. It causes numerous physical problems and complicates everything I do. Most big people I know are in the same boat. It's not fun carrying the extra weight around with you, or trying to fit in airline seats (I've given up), or being uncomfortable in your own skin, or looking in the mirror every day and hating what you see. I know my weight is unhealthy. I promise you the vast, vast majority of big people are similarly aware. Do you think if we could lose the weight as you say that we would not? That we'd willingly stay this way?

Calling big people weak willed, revolting, or undeserving of medical care is unacceptable. I am not a Healthy At Every Size activist. You know nothing of why these people are big. You don't know what they've tried or why they stay this way. You don't know their struggles and what they've gone through. Please stop making value judgements on other human beings based on their physical dimensions.

<comment><USER13> The Venn diagram of HN and liberal overlaps a lot, but more than that it also tends to skew towards the "young, hyper-confident (perhaps arrogant) male who thinks they can fix every problem and basically knows better than everyone else" set. I've known plenty of liberals who are aware that being an asshole isn't magically OK because you're being an asshole to fat people. I am one.

<comment><USER4> Me too. I realize that I also painted with a broad brush there. I was admittedly getting pissed off.

</comment></comment></comment><comment><USER14> Life is hard when you have worked yourself into a health hole. Getting out will require treating a cluster of issues. Obesity is trivial to diagnose. So the rest get little attention until that one is dealt with. Occams razor says don't spend a moment more figuring out probable cause than is necessary.

<comment><USER12> Except medicine works on different principles than Ockham razor. Such as the Hippocrates oath some doctors selectively forget. (2nd and 4th verse of modern version)

A doctor is not supposed to stop or not attempt treatment because a person is fat... which is a kind of sickness, not really a choice. Not should they stop at the most obvious issue, especially when it takes years to fix.

<comment><USER14> Diagnosis works on Occam's razor most of the time. Yes it could be a tumor or degenerative bone disease, but its probably just a sprain (because you twisted it hard while playing basketball). In fact Occam's razor describes medicine perfectly.

As for the 2nd part - my back hurts and I'm fat - weight-related for 99% of the patients. Fixing weight fixes the back (and the feet and the knees and on and on). Any other treatment is palliative but doesn't address the fundamental cause. It would be irresponsible to keep giving obese patients 'crutches' (treating the complaint) when the cure is weight control.

<comment><USER12> This applies only for movement disorders (e.g. joint damage) directly caused by obesity, maybe. Which the doctor gets to prove by excluding other reasonable causes existing in non-obese people. No shortcuts.

<comment><USER14> And blood pressure? Circulation issues? Cholesterol? Diabetes? Arthritis? Sleep apnea? Depression?

All of those cluster around obesity. The simplest, most direct and most effective cure is controlling weight.

</comment></comment></comment></comment></comment><comment><USER13> My experience, having been both a very obese person (100 lbs overweight) an an almost normal-weight person (10ish pounds overweight) is that obese people get worse _everything_.

I expected to have an easier time talking to the opposite sex when I lost weight, and this ended up being true. I had no idea how much better talking to everyone else would be. In customer-business interactions, job interviews, and really every interaction you have with other people it made a remarkable difference. I was an avid cyclist before I lost weight, and I still am now, but people are nowhere near as dismissive when I tell them I rode a half- century the previous weekend, even though I did so before as well (riding a recumbent helps a lot).

This is anecdotal inasmuch as it was just me, but over the course of my life thousands and thousands of data points suggested the idea that people are just extremely prejudiced against fat people.

<comment><USER15> _riding a recumbent helps a lot_

Don't tell anyone _that_ , or they'll be just as dismissive as before! b^)

</comment></comment><comment><USER16> It reminds me of the Anna Karenia (sp?) principle.

Happy families are all alike, every unhappy family is unhappy in its own way.

Most fit people are fit the same way. They have the same body proportions, respond the same to anesthesia, and can be maneuvered into the same positions that doctors train for and are used to.

If you're obese, you accumulate fat in different ways, you respond differently to medication and anesthesia, because you just accumulate in different patterns.

Doctors don't train on obese patients because they're all different, so it's hard to get a standard operating procedure. They also don't train on obese patients because until now, they've been a vanishingly small proportional of the population.

Take a look at vintage photographs of "fat men" in carnivals in the beginning of the 1900s. You see people like that all over the place these days. Cheap abundant food and obesity has only recently been a problem.

<comment><USER12> The real question then is why. It is certainly not just access to abundant cheap food - otherwise you would see more rich fat people in the past. There is probably more than factor and actual epidemiologists should be able to disentangle and find most of them, especially as records are available.

It is either something tied to methods of agriculture and farming or appearance of some sort of endocrine disruptor that is nowadays abundant.

<comment><USER16> I'm pretty sure it's access to abundant cheap food. With a few exceptions (ahem, Mexico) most obesity crises are in well developed countries.

Even so, cheap abundant food is definitely much more of a thing in Central/Latin/South America than it used to be.

Also, a lot of corporation have spent a lot of time and energy dressing up cheap ingredients to directly stimulate our pleasure sensors.

In the past sweet, salt, and fatty food were hard to come by and rather expensive, so you'd gorge when you could. These days, the cheapest procesed foods are saltier, sweeter, and fattier than anything in the 1800s.

It's doubtful that it's tied to methods of farming or an endocrine disruptor. Unless you're talking about food subsidies for cheap grains. We just like salty, greasy, sweet food cause it used to be rare, and now it's not.

<comment><USER12> I am pretty sure access to food is a prerequisite, but not a direct causative agent. Otherwise everyone in the West would be obese today, not 1 in 5.

And yes, manufacturing practices may be truly causing this, coupled with more hectic but sedentary lifestyle preventing people from cooking properly (it takes time and care), compounded by lower quality of basic ingredients and higher levels of stress.

It is highly unfair and unjust to throw it also onto a single person to invert, especially a sufferer.

<comment><USER16> > And yes, manufacturing practices may be truly causing this

How? Are you talking about pharmaceuticals used in factory farming, trans fats, what?

> coupled with more hectic but sedentary lifestyle preventing people from > cooking properly (it takes time and care)

It does not take that much time or care. The choice is yours. You can either watch an episode of Seinfeld on the couch, or spend time on your feet and cook dinner and lose weight and feel healthier.

> It is highly unfair and unjust to throw it also onto a single person to > invert, especially a sufferer.

Fairness and justice are two concepts the universe does not care about.

And how is it unfair and unjust? Once you accept that your suffering, in this case, is based on decisions you've made (remember 4/5 people are not obese in the west, placing you in the lowest 20 percentile.) you can start changing that.

Also, these things don't just happen to you. You still have a choice in the great majority of situations.

</comment></comment></comment></comment></comment><comment><USER17> There is something similar in IT operations/support. Many companies refuse to investigate an issue until you are on the latest versions of software and firmware.

There are good reasons for each of these checkpoints: there may be a real issue not caused by obesity/old firmware, but it is really hard to tell until you get past that.

<comment><USER18> Great analogy

<comment><USER12> If an upgrade took years to apply, this wouldn't be a standard operating practice.

Such as in high risk environments with certification requirements. You wouldn't tell a bank or military "upgrade it".

</comment></comment></comment><comment><USER19> The problem is that obesity is truly the primary cause of many medical problems. You can treat other symptoms but sometimes things won't get better. This article is garbage.

<comment><USER20> > The problem is that obesity is truly the primary cause of many medical > problems.

Obesity makes lots of medical conditions worse (or more likely), but it's the primary cause of almost none of them. It's rare that it's even the biggest risk factor.

</comment></comment><comment><USER21> Coming from a family of medical doctors, medicine stands to gain a lot in terms of quality of service by increasingly automating doctors out of the loop. These people (my family members and relatives included) typically go into the profession for money and social status instead of helping patients, to the point where it borders on being exploitative.

<comment><USER19> Comments like this are incredibly uninformed and in my experience not really true. If you can get into medical school, you could do another job that is a lot less of a slog and make more money. There are slides going around somewhere where if you account for hours worked, doctors make about the same as UPS drives on average.

<comment><USER22> There's a saying that we can't block the sun with the palm of our hands. What gp says is definitely a fact. A lot of people just want a decent life for themselves. I would like to deny this because I know when I throw a brick in the mud, it can come back to my shirt as well. However, we shouldn't run away from the truth because it is unpleasant. A lot of people go to school to become doctors, lawyers, (and lately software programming) because they think this is a good way to make a good salary. If it is not about absolute value of salary, it is about opportunity cost. "What would I do if I don't go to medical school?" Military recruiters know that there is something similar going on in potential enlistees as well.

Lets not pretend that it is not going on. Lets face the facts. Many people (if not most) are not motivated primarily by their need to make the world a better place for everyone else. We are too primitive for that. I'd argue we aren't even programmed for that yet. In a world where people think it is acceptable for people to starve if they can't or won't work, it is too much to think that our doctors, lawyers, and corporate executives are a different breed or species.

Sorry I have tremendous respect for our doctors as I have for our soldiers but that doesn't mean everyone who goes in to medical school or military training went in with perfectly altruistic purposes.

</comment></comment></comment><comment><USER23> Curious...have any individuals ever found the reverse to be true? Doctor takes a look at patient (who presumably appears to be in very good shape) and quickly draws the conclusion, "You're working/training too hard. Just take a break."

No real diagnosis, just a quick assumption that, "Hey, this person looks like they're in great shape. They'll get better one way of the other. I have bigger fish to fry."

Just curious.

<comment><USER3> Oh, yeah, that definitely happens. It happens to young people, too. They come in with X symptom and doctors are like "nah, you're so young, you're just fine." Then it turns out you are not fine. This also negatively effects people who have unusual or hard-to-diagnose conditions that exhibit symptoms similar to those of other conditions.

Doctors apply 90%-suitable heuristics to 100% of candidates too often, in my opinion.

<comment><USER12> If it were 90% applicable if would be good. These are not validated and are likely much less accurate.

</comment></comment></comment><comment><USER24> Constantly ignored is that medical problems are a huge source of weight gain. The arrow of causality is frequently diabetes-> weight or sleep apnea-> weight or sciatica-> weight or asthma -> weight or joint problems -> weight

Not the other way around. A constant thread in the dismissal here and across culture is that the only way you could possibly weigh more that 200lbs is pure weakness of character.

</comment><comment><USER25> Most doctors will also tell an alchoholic to stop drinking.

<comment><USER26> Actually they won't.

You can very quickly kill a dependent alcoholic without special treatments. Going cold turkey has a risk of sudden death.

Quickly losing a massive amount of weight has risks too. That said, if a doctor says you have a condition that is being exacerbated by obesity, the patient should be sent to a weight specialist.

<comment><USER27> > You can very quickly kill a dependent alcoholic without special treatments. > Going cold turkey has a risk of sudden death.

My girlfriend became profoundly psychotic when she ran out of alcohol. She was taken to the hospital, who didn't care that she had been drinking, and didn't include on their 'petition for court-ordered evaluation' that their initial blood work found cocaine metabolites. The psychiatrists at the county's behavioral health center also didn't care that their patient told them she'd been drinking heavily.

Modern Medicine does some things quite well. Weight loss and addiction are two areas where conventional medicine is inadequate.

</comment></comment></comment><comment><USER28> To be a bit blunt, the doctors are probably thinking "if you won't take care of your body, why should I?"

If someone is so massively overweight that they need to use equipment intended for large animals or industrial scrap, they are putting themselves in an extremely precarious health situation.

<comment><USER19> Doctors take care of people who don't take care of themselves all the time, everyday. It's part of being a doctor.

<comment><USER29> And when those people go to the doctor with complaints related to not taking care of themselves, the doctors tell them they need to stop drinking, smoking, or whatever. Why would obesity be any different?

<comment><USER4> Telling them to correct the initial cause is wholly different than not treating the symptoms or not mitigating damage because "they don't deserve it" or "they can't even take care of themselves".

<comment><USER29> I'm not sure it has anything to do with the patient "deserving it" or not. Nobody wants to put effort into something when somebody else is actively working against it.

<comment><USER4> Big people do not want to be obese, and doctors are paid for their efforts. Nobody sets out with the life goal of being fat.

</comment></comment></comment></comment></comment></comment><comment><USER30> I didn't read the article yet, since I opened the comments first... and was shocked by them.

I have a thyroid problem, the first symptons started to show up when I was 13 or 14, including my hair starting to get white.

I didn't knew, it was a thyroid problem, my mom, for some reason (that back then offended me, I told her I didn't wanted to go to a "fat people doctor") took me to a endocrinologist, suspecting something was wrong.

The endocrinologist made no tests, only said I was fat, and dismissed me.

The thing is: according to BMI, I was only slightly overweight, not even really "fat" yet.

But I worsened over the years, and became obese, no matter how hard I tried to not to (including having strict diet and going to the gym, and having medical help from cardiologist, physiotherapist and nutrition).

A random friend of my mother then commented I had obvious thyroid problems symptons, including a obviously enlarged neck.

My mother started to take me to endrocrinologists again... and again, they just kept telling me I was fat, and refusing to help.

I ended researching my own problem, figuring on my own what I needed to have tested, and spending lots of money and time looking for a decent endocrionologist.

I found one that is half-decent, and started at 25 years old my treatment finally... and only then, the treatment is kinda half-assed, my current endocrinologist mostly don't believe me, and don't really want to help, in fact I ended mostly treating myself by myself, buying whatever medicine I wanted, and informing the medic after the fact (where if I made the right choice, the medic would inform me that I made the right choice, so far I always made the right choice, according to improvement in symptons and blood tests results).

I am currently trying to drop my weight until I get obviously "not fat" so I can then save some money, and go to a expensive endocrionologist and hope he will treat me correctly, instead of telling me I am fat.

I even tried to go to the most famous endocrinologist in my country, I spent a entire month salary in one single visit, and the guy just told me I was fat and refused to ask for any tests, despite tests being kinda straightforward (I have a autoimmune disease, a test to see if I have anti-thyroid antibodies would already be enough to diagnose me, yet not a single doctor ever wanted to test that, the single one that did, was a doctor that was going to retire, and that I was very "persuasive" in convincing her to ask for the test, that indeed proved I was correct, and indeed I had a huge amount of anti-thyroid antibodies destroying my thyroid).

<comment><USER11> Whatever condition you had didn't cause you to gain weight. You can't defy physics. If you control your caloric intake, your body /will/ lose fat. Your story is anecdotal so I'll response with an anecdote as well: I've seen more than one person with a "thyroid condition" who were in good shape.

<comment><USER30> You just forget the body is made of more than fat.

Yes, eating less calories than I use will make me lose weight, but where that weight will come from? Nothing guarantees it will be from fat.

For example right now my hair, nails and skin is breaking, and I am losing muscle. But when I raise my food intake, I get an increase in fat, and my nails, hair, muscle and so on continue fucked up. But when I cut calories, they get worse faster, but the fat remains.

<comment><USER11> There's hardly any calories in your nails.

You can supplement with specific vitamins and minerals for your nails and skin. You can supplement with protein and perform basic exercises to protect muscles (pushups, beginner level).

Protecting hair is not a good excuse to not lose weight.

<comment><USER30> Are proteins calorieless?

Right now my diet is already protein heavy.

There is a way to force all my protein eaten to do protein job without a single bit of it being turned into calories for other purposes?

EDIT: my questions were rethorical. I know proteins have calories, that is my whole point. even with a diet where I get only the macronutrients I need, my calorie intake is too high. Right now my lifestyle consists of switching periods of losing weight woth periods of fixing the damage. so far lost 30kg from my peak weight, still need to lose more 30kg to doctors stop telling me I am fat and start helping me.

<comment><USER11> Combine it with healthy fats and healthy carbs.

Muscle loss is unavoidable. But you can't use it as an excuse to not lose weight. You just try to minimize it by having a balanced diet and performing some basic exercises.

<comment><USER12> Another platitude of a "balanced" diet. Not even dieteticians know what that means for purposes of weight loss. Muscle loss is a major problem for people who fail diets and bounce back. Muscle once lost is very hard to regain. Lack thereof will also reduce energy output making further attempts harder.

Even a vegan or near vegan diet easily exceeds required caloric intake for weight loss. Going for healthy foods is no good if caloric input is too high.

Protein has a star quality in that it suppresses hunger directly. The trouble is finding sources not also laden in fat and also making it palatable.

What you need is a highly unbalanced diet, but one that will not damage your health. It is very tricky to set up.

</comment></comment></comment></comment></comment></comment></comment><comment><USER31> Yes, I know/experience that since my BMI increased from 23 to >30.

Even worst, majority of doctors fixate on reducing weight (as if that would solve everything else)

</comment><comment><USER32> The article starts out talking about a woman who is 502 lbs. Why wouldn't "lose weight" be the best suggestion? You're 500 lbs!

<comment><USER33> >The doctor's scale went up to 350 pounds, and she was heavier than that. If she did not know the number, how would she know if the diet was working?

The important bit of that anecdote that you skipped over. Every doctor will tell overweight people to lose weight, but they also tell them not to lose it too fast, or it can lead to liver and kidney problems.

If a doctor says "lose weight", but can't actually assist you in doing that in a safe, controlled and sustainable fashion because he doesn't have the equipment, he's actually not very useful. My grandmother could tell a 500lb woman to lose weight, but I wouldn't pay her $200,000 a year to do so.

However, the key allegation of this article isn't just that doctors don't have the equipment to give first-world-quality health care to overweight people, it's that they have biases that cause them to not want to rectify that problem, even though that's as simple as buying a sturdier scale.

That's a much more serious issue. It's one of the key reasons AIDS was such a problem 30 years ago. No one wanted to treat ~~the gays~~ in the 70's, and so HIV was allowed to run rampant rather than be treated. Within a handful of years of our society realizing it didn't actually give a shit about whether people like butt stuff or not, HIV went from a terminal diagnosis to a manageable condition, because /doctors actually started taking the people it affected seriously/.

<comment><USER34> There was about a 11 year period (first US cases in 1981, "Gay-Related Immune Deficiency" in 1982, Magic Johnson got HIV in 1992) where an HIV/AIDS diagnosis was a certain death sentence. I think there was more of a stigma against gays from politicians and their constituents than doctors, which contributed to it's spread too. It was recognised as early as 1983 that GRID wasn't a gay disease, hence AIDS.

If you would like to read more I believe the definitive history of the early days of the AIDS epidemic is "And the band played on" by Randy Shilts.

I honestly think that if the governments of the world stopped allowing corporations to put sugar in _everything_ then the problem of obesity will fix itself. I'm not going to promote a low-carb diet or whatever, but it is not a disputed fact that sugar makes anything taste better and is addictive, which is why companies add it to everything. People will complain, but it's not really "personal choice" when you are literally choosing food spiked with a drug to make it more addictive.

A lot of people who overeat do it for a number of reasons. I know I like eating a lot now because I was poor once and had to eat as much as possible of whatever was available. I lift 3-4 times a week and bike daily, but am still obese according to my BMI and was recommended against having surgery for appendicitis last year (it fixed itself with a hospital stay and IV antibiotics which was lucky).

I think that for a lot of people counselling would be a good start instead of "eat less". If it were that simple then there wouldn't be fat people, would there? One problem is that some people are just bullies and like to put down others to make themselves feel superior. Then there are things like the "fat acceptance movement". I will never understand that either.

<comment><USER35> Yeah, it was for sure recognized as a public health problem in 1981.

[https://www.aids.gov/hiv-aids-basics/hiv-aids-101/aids- timel...](https://www.aids.gov/hiv-aids-basics/hiv-aids-101/aids-timeline/)

AZT was approved by the FDA in 1987, with some federal funding to pay for it. Which is the real story of why people survive it today, drugs were developed to treat it. But the attitudes of front line doctors in 1990 wouldn't have much to do with the development of those drugs.

One strategy I've used against my poor eating habits is to eat something like broccoli when I want a large serving. A huge serving with a modest amount of butter is ~200 calories.

<comment><USER34> > One strategy I've used against my poor eating habits is to eat something > like broccoli when I want a large serving. A huge serving with a modest > amount of butter is ~200 calories.

My wife had a good idea and she has got me eating a big bowl of high-fiber vegetable (cabbage etc) soup before whatever silly main I want to eat. It's very filling and I believe the same strategy as yours.

</comment></comment></comment></comment></comment><comment><USER36> Veterinary medicine always does dosing by weight. Why doesn't human medicine?

<comment><USER37> They do. Weight dosing is way more common in pediatric medicine, though, since children cover a pretty wide range of weights.</comment></comment>