Can an extreme low carb diet be used as medicine? - <AUTHOR> https://sciencenorway.no/diseases-food-and-diet/can-an-extreme-low-carb-diet-be-used-as-medicine/1708291 <comment><USER2> I am 58, on a (mostly) strict ketogenic diet since 2013. It has worked wonders for me and I expect to eat this way the rest of my life. I eat a lot of meat and eggs as I have found that combination (1) is something I enjoy very much, and (2) makes me feel the best.

I didn't have much weight to lose but I went from 175 to 160 (high school weight) and under 15% body fat. My energy level is consistently very good.

Diet is a subjective thing, of course, but at a minimum getting rid of sugars and processed food might be a good starting point for anyone.

Anecdote: my brother-in-law reversed his T2 diabetes and lost quite a bit of weight doing keto. His doctor had advised meds to "manage" the disease.

<comment><USER3> Weight loss comes from having a calorie deficit, not from keto. You can gain weight on keto if you eat more calories. You can achieve the same weight loss by just eating fewer total calories, without cutting carbs as well.

<comment><USER4> This is well understood by now. The issue is that most people aren't actually going to track their calories- they eat when they're hungry. If you adjust your diet- you can satiate your appetite on a lower number of calories.

The idea is that meats, vegetables, and fats help you meet your nutritional goals on a fewer amount of calories when compared to carbs- because carbs are empty calories.

<comment><USER3> Agreed that satiation is a major aspect. How satiating a food is, is not simply about carbs/proteins/fats. Most carb rich foods people eat today are high in sugar and low in fiber. That causes people to overeat. You can also overeat with high fat foods because fat is much more energy dense.

You can just as easy have a bad keto diet as you can have a bad high carb diet. Replacing processed carbs with processed meat will not do any good. It's a group 1 carcinogen and will also lead to high cholesterol level causing heart disease.

My point is that carbs aren't the enemy. Neither is fat. Eating unprocessed/low processed food should be the goal.

<comment><USER5> The evidence that high cholesterol "causes" heart disease isn't very good. There appears to be an association, but that's really all we can say.

<comment><USER4> in my experience- the more I learn about nutrition, the more I realize that we're just talking about everybody's best guess. Hard facts are scarce in this space.

</comment></comment></comment></comment></comment></comment><comment><USER6> Before the advent of anti-seizure drugs, the ketogenic diet [1] was an effective and widespread intervention used to treat epilepsy. The downside is the discipline required to keep intake of carbohydrates below 15 grams per day, especially in children.

[1] [https://en.wikipedia.org/wiki/Ketogenic_diet](https://en.wikipedia.org/wiki/Ketogenic_diet)

<comment><USER7> additional downsides include kidney stones[1,2,3,4], bone mineral loss[5] and a long list of other complications (including death):

[https://www.thepaleomom.com/adverse-reactions-to- ketogenic-d...](https://www.thepaleomom.com/adverse-reactions-to-ketogenic- diets-caution-advised/)

1\. [https://pubmed.ncbi.nlm.nih.gov/11095028/](https://pubmed.ncbi.nlm.nih.gov/11095028/)

2\. [https://pubmed.ncbi.nlm.nih.gov/10893623/](https://pubmed.ncbi.nlm.nih.gov/10893623/)

3\. [https://journals.sagepub.com/doi/pdf/10.1177/088307380730192...](https://journals.sagepub.com/doi/pdf/10.1177/0883073807301926)

4\. [https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2852806/](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2852806/)

5\. [https://pubmed.ncbi.nlm.nih.gov/19064531/](https://pubmed.ncbi.nlm.nih.gov/19064531/)

<comment><USER8> Firstly those studies are all in children..

Study 1 doesn't actually show increases in kidney stones.

Study 2 has N of 4...

Study 3 looks reasonable, though they did note "use of oral potassium citrate significantly decreased the prevalence of stones" which is good news!

Study 4 has N of 25 & no info about what they actually ate other than macronutrient ratio.

<comment><USER7> > Firstly those studies are all in children

Your point? the OP ended his sentence with "especially in children" so I thought I'd share that which was relevant to children. If you are looking for information on adults, here are some possible clues:

[https://www.nature.com/articles/pr19971830](https://www.nature.com/articles/pr19971830)

It makes sense considering the increase in uric acid, but might only be a problem in children and those susceptible to kidney stones (although when it presents in children, that is some pretty important information in terms of holistic, long-term diet considerations). Still definitely worth mentioning and I'm sure we will get more information as more studies are done on adults (most are on children due to it being used as an epilepsy treatment)

</comment></comment></comment></comment><comment><USER9> I'm going to go with a corollary to Murphy's law. Since anything that can go wrong will, every activity or substance is going to be the problem for somebody.

As far as I can tell, the only thing that dieticians are consistently against is sugar in large doses.

<comment><USER10> They are also for the hard to market, varied diet but largely fruit and vegetables, moderate exercise, little or no alcohol, and regular checkups.

I'm failing to some degree on these, I admit.

<comment><USER11> They are also for 7-8 bite sized 250 calorie meals every other waking hour. A feat that can only be feasibly achieved via living off ready-made readily- purchasable health snacks™.

There is no money in fasting, intermittent or otherwise.

<comment><USER12> > They are also for 7-8 bite sized 250 calorie meals every other waking hour.

No, I wouldn't say this is modern consistent dietary advice. So many people that make it there business to talk about diet are hyping intermittent fasting now.

</comment></comment></comment></comment><comment><USER13> I've been on keto for probably a full year of my life at this point, broken up every few months. I usually can't maintain it for longer than 2-3 months at a time because it really is a strict diet - but it's always been worth it once I get into the "mode".

I absolutely love it because it turns the daily roller coaster of hunger pains for me caused by carbs into a steadier, more predictable climb. I basically feel no hunger at all during the day, and it's fantastic.

The big downside, obviously: little to no carbs. As a home baker I miss making bread when on keto, but I get that energy out by making different keto treats (usually almond/coconut flour based) instead.

<comment> blaser-waffle Same here. Love me some homemade bread, gotten really good at ciabatta.

It makes it hard to do anything more than 2-3 months on keto. I found that it works as advertised -- skin gets better, weight drops, generally more energy -- but it's very hard to get away from carbs. Like, you need to be DILIGENT, and even if you stay on top of it it ends up limiting your life in many ways. I could live without a lot of carbs but giving up on pizza was hard...

</comment></comment><comment><USER14> It's been used to treat diabetes and seizure victims for a while so, yes. Personally I lost over 70 pounds (from 295 to 225 -- I'm 6'5") at one point using keto. My main concern is whether one who is on the diet for weight loss will become overly sensitive to carbs if/when they start eating them again causing the body to immediately store them away as far instead of burning them (and if so, how long would it take to "return to status quo ante" with regard to carbohydrate metabolism).

<comment><USER15> Yes, in my personal experience. However, I suspect my body is just desperate to hold on to carbs and pack those bastards around my liver anyway, regardless of previous Keto periods or not. When I stick to Keto, I feel better, think better, poop better, I don't get headaches and I don't have acne. Is it magic? I dunno. The chemistry seems to support it.

Years ago we called it Atkins (yes, I am older) and it worked great for me, but chips and salsa are my kryptonite. Unfortunately, as I age, I must rely more and more on my diet for control I am not but a humble nerd who now sits behind a glowing screen toiling away as a PHB, but I can offer this: Keto works for me and I would recommend it to anyone.

<comment><USER16> I think the Atkins diet was low carbohydrate but not low enough to induce ketogenesis.

<comment><USER17> Of course, eating high-carb meat like hot-dogs and chicken nuggets won't work.

I tested my urine during Atkins and I was either high or medium in Ketones. I did eat a lot of meat high in fat and little to no carbs.

I was able to keep being in ketosis while still eating about 50g carbs/day (Through meat products), while I was doing exercises daily.

<comment><USER18> Note for others tha urine isn't accurate. You need blood test.

<comment><USER17> Time of the test matters a lot, I dismissed morning, because it was always too high.

Tested during mid-day and late afternoon and made an average.

</comment></comment></comment></comment></comment></comment><comment><USER19> Yes. And it has been used to cure some things you wouldn't believe. Here is the shortlist of the more serious illness that the PaleoMedicina group in Hungary have shown to effectively treat using their low-carb PKD protocol

[https://justmeat.co/wiki/pkd/](https://justmeat.co/wiki/pkd/)

[https://www.paleomedicina.com/en/paleolithic_ketogenic_diet_...](https://www.paleomedicina.com/en/paleolithic_ketogenic_diet_PKD_in_chronic_diseases_clinical_and_research_data)

Here are testimonies of even more people healing their serious diseases on a purely carnivore and zero-carb diet

[https://meatrx.com/category/success- stories/](https://meatrx.com/category/success-stories/)

<comment><USER20> With URLs like that, it makes me wonder if they are telling the whole story.

I don't understand why people make science "pointed"? Do people get extra credit for not being the status quo?

Heck all I want in Nutrition is some objective data. Someone be that scientist.

<comment><USER21> Unfortunately almost no human nutrition studies meet evidence-based medicine standards. And in fairness to the scientists, the constraints imposed by funding and ethics make it impossible to do really meaningful research. All we get are observational studies which show some correlation, often mixed up with multiple uncontrolled confounding factors.

So if you want to try a different diet like extreme low carb or whatever then go ahead. Maybe it will work for you, maybe it won't. In the worst case it probably won't kill you.

<comment><USER7> > In the worst case it probably won't kill you

Unfortunately, that is also not necessarily true:

[https://pubmed.ncbi.nlm.nih.gov/19027591/](https://pubmed.ncbi.nlm.nih.gov/19027591/)

[https://pubmed.ncbi.nlm.nih.gov/15679508/](https://pubmed.ncbi.nlm.nih.gov/15679508/)

[https://pubmed.ncbi.nlm.nih.gov/15329077/](https://pubmed.ncbi.nlm.nih.gov/15329077/)

[https://pubmed.ncbi.nlm.nih.gov/11575609/](https://pubmed.ncbi.nlm.nih.gov/11575609/)

<comment><USER8> > > [https://pubmed.ncbi.nlm.nih.gov/23273808/](https://pubmed.ncbi.nlm.nih.gov/23273808/)

> Conclusions: The KD is a safe and efficacious therapy for intractable > childhood epilepsy in Chinese children. The influence of age on efficacy is > worth further investigation.

?

<comment><USER7> edited. Thanks for catching that. I have a lot of different links in my notes haha

</comment></comment></comment></comment></comment></comment><comment><USER22> One thing many overlook with low carb is that you can still enjoy your favorites sauces and similar, you will be amazed how far a tablespoon of one will go. This can apply to mixing in what otherwise are considered carb no nos like bread crumbs. A little goes a long way and the carb impact can be minimized if the base is large.

Plus there is so much variety in the produce department that fits the diet that many overlook. Low carb/keto does not mean having to eat what is on their list but looking on the lists for items you have never tried to see new opportunities.

<comment><USER23> YMMV but my favourite sauces were not compatible with Keto. Brown sauce for example - and no one is making a sugar free version of it AFAIK

</comment></comment><comment><USER24> Fasting has excellent results for treating many illnesses including mental ones.

There is a documentary that covers two clinics that offer fasting as a treatment and scientific research in this area.

[https://www.youtube.com/watch?v=t1b08X-GvRs](https://www.youtube.com/watch?v=t1b08X-GvRs)

</comment><comment><USER25> Can someone explain why ketosis is the secondary energy system in the body and not the primary?

<comment><USER26> Our bodies are primate bodies. Primates are frugavorian omnivores (get most calories from fruit, and eat very little animal tissue (mostly insects)). Fruit contain mostly carbs: short carbs a.k.a. sugars. We're optimized for eating lots of sugar.

Primates have a hard time producing fat from carbs (a process called lipogenesis), just a few calorie% per day. This, combined with the tiny bits of fat in fruits/insects, we may store as fat tissue.

Here comes ketosis: when we cannot find adequate food supply (starve) we can live off our fat storage, this is what ketosis is for (biologically/evolutionary). It's our starvation mode.

Primates are not made to be in ketosis for long times. We're also not made to eat high fat and/or high protein diets. We're made for a high fruit diet.

Is long term ketosis bad? I believe so, but we still need to do research on this. Many people are trying it on themselves lately, so we will soon know.

Is short term keto dieting bad? Nope. And it may help you to burn that body fat fast. I'd prefer to get into keto by mimicking starvation, a.k.a. (intermittent) fasting, and not by eating lots of fat/protein.

Is eating lots of animal products bad for you? Yes. It's been shown again and again in large studies that humans do better on predominantly whole plant foods.

<comment><USER8> > We're optimized for eating lots of sugar.

Not even sure how to respond to that one. American diabetes rates?

> Primates have a hard time producing fat from carbs

??? carbs -> glucose -> insulin -> increased lipogenesis. Carbs are _better_ than protein or fat at producing bodyfat.

> Is eating lots of animal products bad for you? Yes. It's been shown again > and again in large studies that humans do better on predominantly whole > plant foods.

In large, observational studies with innumerable co-founding factors, including lumping regular meat (a steak) in with highly processed meat (mcdonalds burger).

I agree with the whole food part, and plants can be good, but exclusion of unprocessed meat is not proven to be any more healthy.

You also self describe as vegetarian in your profile, and ketogenic diets often include meat.

<comment><USER27> I was vegetarian for 10 years and tried a beef only diet for my immune problems.

I was told that I would have trouble digesting meat again but to my surprise, it was much easier than usual, and I felt great the whole day (also actually felt full for like 6-8 hours after a meal).

Then gradually reintroducing foods one by one helped me pinpoint the culprits.

</comment></comment></comment></comment><comment><USER28> I did it before for weight loss, it worked amazingly, but the weight came back on just as easily. It would have to be a permanent lifestyle change, and I'm not convinced that's safe.

<comment><USER29> What makes you think that it isn't safe? I guess the other question is what makes you sure that another diet is safe?

<comment><USER12> It's a really extreme diet. I would be concerned with the fat intake, the cholesterol intake, and limits on certain fruits and vegetables.

<comment><USER8> Your body makes more cholesterol than you get from diet, it's more complicated than just "cholesterol high = bad".

Also would encourage you to read up on recent literature about fats. The "low fat = healthy" fad hasn't held up for some time.

[https://www.sciencedirect.com/science/article/pii/S073510972...](https://www.sciencedirect.com/science/article/pii/S0735109720356874)

> The recommendation to limit dietary saturated fatty acid (SFA) intake has > persisted despite mounting evidence to the contrary. Most recent meta- > analyses of randomized trials and observational studies found no beneficial > effects of reducing SFA intake on cardiovascular disease (CVD) and total > mortality, and instead found protective effects against stroke.

</comment></comment></comment></comment><comment><USER30> Well the ketogenic diet was designed specifically to treat intractable epilepsy ... so yeah ...

</comment><comment><USER31> So my daughter had epilepsy soon after birth and was hospitalized for 20 days @ 1 month old. By the time we left they had her on 4 different drugs, but the seizures continued. At 4 months old she went to the local children's keto clinic. The seizure frequency slowly tappered off until a month or so later she was seizure free! That lasted for a few months until an ear infection brought the sickness back and then revealed the true cause of the seizure, a brain abnormality.

That was when my eyes were truly opened to how much diet can effect the body.

</comment><comment><USER32> I have been on keto for three months now, for me it's mostly about the mental clarity, free from the ups and downs you experience when eating a lunch high in carbs.

</comment><comment><USER33> There is The Autoimmune Protocol diet (AIP), a stricter version of Paleo, and there is keto diet. They both seem to help with autoimmune deceases and inflammations.

Did anyone try them, and can share their experience? Do they both have comparable effects in a long term?

</comment><comment><USER34> No.

</comment><comment><USER35> My dog Toby was recently diagnosed with stage 2 splenic hemangiosarcoma and given 30-60 days. Opting against doxorubicin chemo we are working with the Keto Pet Sanctuary to make a new diet and hopefully start HBOT soon.

</comment><comment><USER36> I love how we're starting to focus on our diets to prevent or cure a huge amount of diseases. You don't need drugs, expensive treatments, nothing, only a good diet. (read the comments below for an clarification for this statement)

I personally can't work on a very low carb diet. Now I'm eating carbs again but NOT refined carbs. I make sure everything I eat is nutrient dense. Well, not everything, I'm not a robot but I always try to eat stuff with a low glycemic index.

And I eat everything in a span of 8 hours. The rest of the day I only drink water.

Y recommend listening to Dr. Rhonda Patrick. She can explain all this way better than me. She's been on the Joe Rogan podcast (she talks about this in her first time there) and she also has a podcast and a lot of very interesting videos.

<comment><USER37> >You don't need drugs, expensive treatments, nothing, only a good diet.

Moms on facebook keep telling me this but my physician disagree! Do you have an essential oil for this?

<comment><USER36> I realized that that statement can be misinterpreted. I'm not talking about not using drugs or going to a doctor when it's needed. I'm saying a good diet can prevent a lot of bad stuff and sometimes even reverse things like diabetes.

<comment><USER7> I agree with the responder that your original statement sounds a bit hyperbolized, but I also am an advocate that diet can treat a LOT more conditions than most people think</comment></comment></comment></comment>


Instagram worth a half billion dollars? Sure, and I'm Mick Jagger - <AUTHOR> http://thenextweb.com/insider/2012/03/08/instagram-worth-a-half-billion-sure-and-im-mick-jagger/

<comment><USER2> According to Crunchbase[1], they have 12 employees. I applaud their team for their user and product growth on such a small team, but come on, have they sold anything? Ever? Let's assume they now have 20 employees, that's $2.5M/ employee.

Do these investors know something that I don't? Is the Fed opening up a program for app developers that make no money? Because if they are investing at half a billion dollar valuation, they are looking for an exit north of a billion dollars. I mean, come on, Pandora generated $75M last quarter and as of today their market cap is 1.7B[2] and that's after their stock tanked yesterday.

I wish the best for the instagram team, but a round like this cannot be good for them, the product or the users in the long run.

[1]- <http://www.crunchbase.com/company/instagram> [2]- <https://www.google.com/finance?q=NYSE:P>

</comment><comment><USER3> I am tired to death of this phase of the web economy. I was downvoted in another thread for characterizing this dance as "charlatanism," but I will repeat it here. I hope to God I never need to monetize my HN karma because that would mean I have failed in the actual real economy outside the Peddling of Startups to...actually who are the end investors in these VC funds? Facebook and Google employees and maybe some hedge fund managers who think tech is cool?

Downvote me now to show that you are smarter. To send a message that you will aggressively monetize your Javascript skills, and naysayers will be pushed outside the circle of trust.

<comment><USER4> I feel you my friend. Up vote for you :) When the heck did it become to have a business that does not make money, does not focus on making money and gets a valuation to half a billion. When? Is there something that I just don't understand that the investors do?

I think its simple. People are investing hoping that hype will continue and the next round will cash them out. With that in mind, when IPO hits, the people cash these types of business in and everyone loses once they find out that the company is still trying to find out a business model.

</comment></comment><comment><USER5> Let's assume they have some kind of master plan that will eventually result in them being profitable.

The question coming to mind is how exactly are they going to spend $40m. I mean it's a simple image taking app - not exactly nuclear physics.

</comment><comment><USER4> wow, I could not agree more with the writer. Some where along the way, the basic concept of making a profit, and making it fast, and making it within a certain period has simply been erased in the mirage and conveniently accepted excuse of "money will come later." Really, why can't money come now, and how later is later. It seems like, the investors are simply putting money in to reap cash in the next round and walk out while the last person or entity to take on the investment burden (public if it goes ipo) will deal with the massive losses of a business that has yet to figure out it's revenue model. Unbelievable.

<comment><USER6> Doesn't this sounds like 1998 all over again? I hear all the common refrains about "this time is different! we're not in a bubble!".

Yes, this time _is_ somewhat different (huge number of people on the net - entire generation has grown up wired, etc), but the same irrationality about first movers capturing markets and money coming later that we saw in 1998 seems to be happening again.

<comment><USER4> Well I was very young in 1998 - I was too interested in getting my maxpages in top rankings in the maxpage domain :) so i don't know much. But man, this is just ridiculous. Really, $500 Million! $500 Million! for a photo sharing app that has a huge list of competitors and easy to knock off?</comment></comment></comment>


Face.com opens facial recognition API to developers - <AUTHOR> http://developers.face.com/

<comment><USER2> Who will be the first one to come out with an iPhone app that allows you to take a picture of someone and then automagically find the persons name using facial recognition?

It really seems like a pretty simple idea, especially with an API, which is already attached to Facebook, that does all of the work for you.

The API seems a bit limiting, though. For example, you must specify a list of possible users "you're looking for" in order it to find a match. The only way around this seems to be creating a private namespace that includes your own index of all Facebook users, etc.

<comment><USER3> I've implemented something like this before. I ran into a lot of performance issues though. A match against a known person seems quite fast, but when you multiply this against even a medium sized network of people the speed becomes problematic.

This was in the 90s though, I'm sure it's much faster now - I just dont know if it's fast enough to be useful.

<comment><USER4> Hey, they do it all the time on CSI and in real-time - get with the program, man!

</comment></comment></comment><comment><USER5> I've been waiting for this to happen for a while and I'm both excited and scared.

No longer are photos anonymous just because they are in a place that doesn't have you name attached. This is just the beginning of this technology.

<comment><USER6> Suddenly blocking out faces on google steet view makes even more sense than before. I wonder if we ever get to the situation where there's a law requirement to remove faces from crowd photos in newspapers / passer-bys behind street interviews.

<comment><USER7> no reasonable expectation of privacy, not gon happen

<comment><USER6> Are you implying that there was a reasonable expectation of privacy in google steet view case?

<comment><USER8> Google did it to be nice, not because they were legally required to.

</comment></comment></comment></comment></comment><comment><USER9> Similar API's are available all over the web. the best seems to be www.pittpatt.com but it's kinda pricey. A big advantage is that PittPatt can be installed locally on a server to provide a secure app. Privacy could be a concern as the images are stored on a CDN.

A big question is what is the confidence score of accurately matching images? The smaller the dataset the better but this was a big concern with the app that i was building. General uses like celebrity sites wont have this concern.

<comment><USER10> Interesting. How pricey?

</comment></comment><comment><USER7> one thing that makes me a bit nervous about this is how frequently it's wrong on its finer points. if you look at the attributes of the tagged people you'll see they are wrong about 50% of the time (for instance a smiling male is auto tagged as a non-smiling female). this makes me nervous because we are in a place now where the layman takes the absolute, digitally measured attribute as fact (ie: this is a woman, this person is smiling), instead of using their own facilities.

it's a minor thing, with this example, i know, but how long before the precision of our digital measurement ends up hurting us because the layman doesn't understand its inaccuracy?

edit: this isn't a critique of face.com, but those who may end up using it. face.com = awesome tech, very cool.

<comment><USER6> The demo page gives a lot of fun and space for experimenting... It seems popular fastfood mascots don't get recognised properly either re. sex and smiling (even though 2 of them have beards!) (tried these links: [http://upload.wikimedia.org/wikipedia/en/thumb/b/bf/KFC_logo...](http://upload.wikimedia.org/wikipedia/en/thumb/b/bf/KFC_logo.svg/160px- KFC_logo.svg.png,http://upload.wikimedia.org/wikipedia/en/4/46/Ronald_McDonald.jpg,http://upload.wikimedia.org/wikipedia/en/6/6f/Creepy_King_Bed.png))

<comment><USER7> yes, although i was testing their own demo pictures.

for those who downvoted: its pretty dangerous to take something that is traditionally done by humans or in an analog fashion and do it digitally when you are wrong around 50% of the time because people tend to trust technical things that look "safe enough." i understand the tech will get better, but i think its pretty bad to have attributes like "smiling?" and "male/female" even present in the API at all if i can flip a coin blindly and have roughly the same outcome.

of course its complicated, technically impressive regardless. but when a consumer sees "facial recognition technology" from the tech community they aren't about to question it. all im saying is if you are going to offer that kind of analysis, it should be right before people can start using it in their apps via an API and end consumers start depending on the data. if it's not accurate, or not a big enough a deal to care about, it shouldn't be in the API.

here come more down votes...sigh

</comment></comment></comment><comment><USER11> Anyone know where to find a "developer code" to sign up?

<comment><USER12> Trying filling in everything else and leaving that field blank.

I tried a few different combos and that's that one that I think worked.

<comment><USER13> Yep. No asterisk next to the "developer code" box so it's not actually required.

</comment></comment></comment><comment><USER14> Wow. The algo is really awesome. I detected every single face here: <http://chir.ag/gallery/md/nature-coast-kayaking/047.jpg> and got smile + glasses right for everyone except one. Nice domain too.

</comment><comment><USER15> Does anyone know how mature algorithms that use video for facial recognition are? It seems one could use multiple consecutive frames to draw better inferences about 3D geometry, and perhaps resolve various occlusions.

</comment><comment><USER16> uow, this is really good! Finally i can throw all my crappy opencv integrations with java :)</comment>


A Metric Learning Reality Check - <AUTHOR> https://arxiv.org/abs/2003.08505 <comment><AUTHOR> "Our results show that when hyperparameters are properly tuned via cross- validation, most methods perform similarly to one another"

"...this brings into question the results of other cutting edge papers not covered in our experiments. It also raises doubts about the value of the hand- wavy theoretical explanations in metric learning papers."</comment>


Who else is behind SOPA? - <AUTHOR> http://blog.dentcat.com/2011/11/who-else-is-behind-sopa-other-than.html

<comment><USER2> It can be effective to tweet, blog, and contact these companies and complain about their endorsement of SOPA, as reflected in their Business Software Alliance (BSA) membership. Associations like the BSA exist, in part, so that companies like Apple and Microsoft can have someone do their political wet- work while pretending to be above the fray. Call them out on it!

<comment><USER3> I didn't even know our industry had an RIAA/MPAA type body acting on behalf of so many companies. I feel a bit dirty now.

<comment><USER4> They even have a mass-settlement-extraction campaign, though it gets less press because it's not targeted at average home users; they're smart enough not to send threatening letters to grandmothers whose grandkids might've installed a pirated copy of Windows on their machine. If they suspect a company is using unlicensed software from a member, they'll demand the company undergo a "BSA audit", which will work to correct the problem and reach a settlement. Of course, they aren't a law-enforcement organization, but they try to pressure companies to agree to a BSA audit through a mixture of claims that the company agreed to such an audit via their licensing terms, and through threats of lawsuits if they don't agree. There's a whole "BSA audit defense/compliance" specialist-lawyer industry around that.

To be fair, many of the companies targeted are not exactly innocent, but many businessmen who _do_ have valid licenses are also worried about the expense/disruption of the process. Disorganized shops who don't have their licenses nicely indexed can end up in trouble, too, if they can't manage to rummage through enough cabinets to locate copies of their licenses (one reason some have resorted to physically attaching the Windows license to each machine, so it moves with it). It's sort of like being audited by the IRS; sucks even if you weren't cheating on your taxes, except it's weird that it can happen from someone who isn't even the government!

There were also some allegations (I believe settled out of court) in the 1990s that the BSA was using its enforcement threats to coerce companies into buying site licenses of its members, because having a site license would exempt you from the possibility of an audit.

<comment><USER5> What happens if you say no to an audit? Do they ban you from buying software?

<comment><USER6> They sue you. And then you have a long lengthy lawsuit ahead of you!

</comment></comment></comment></comment></comment><comment><USER7> Interesting how the antivirus software companies are so on board (I mean, AVG gives it stuff away for free). Any thoughts as to why this is?

<comment><USER8> On one hand there is a lot of pirating of antivirus software, and yet to some extent P2P does increase the need to have it installed on one's system (which may increase demand/business).

<comment><USER7> I feel like antivirus software is less piratable than a lot of things though - you need to constantly update through a centralized server which checks your credentials (which are inevitably false if you've pirated the software).

</comment></comment></comment><comment><USER9> I hope Android will become a more full blown OS in the next few years, so I can replace my Windows computers for good. I don't want to support in any way a company that abuses so much the patent system, and is also supporting Internet censorship. I think that day will come sooner rather than later.

<comment><USER10> You can replace your Windows computers now if you really want to.</comment></comment>


China has its own distinctive version of the maker movement - <AUTHOR> http://www.economist.com/news/technology-quarterly/21590756-technology-and-society-china-has-its-own-distinctive-version-maker-movement

<comment><USER2> The only thing I don't like about the "Maker Movement" is the way so many talk about themselves as if they were the first people ever to do this sort of thing. Like handmade craft and technologies was something invented by Internet hipsters.</comment>


My parents didn't tell me they skipped my vaccines. Then I got sick - <AUTHOR> https://www.washingtonpost.com/outlook/my-parents-didnt-tell-me-they-skipped-my-vaccines-then-i-got-sick/2019/04/25/2434feec-66a6-11e9-82ba-fcfeff232e8f_story.html <comment><USER2> I don't understand how skipping vaccines is still legal. If you go to public school, then you need to be vaccinated. Otherwise, you go to jail.

<comment><USER3> The article explains that the author did not go to public school:

> My parents held all kinds of alternative beliefs. When someone in our family > got sick, they turned to home remedies. They home-schooled my seven siblings > and me, and after that, I took college classes remotely. I never passed > through any of the usual institutional checkpoints, where some authority > would ask for health records, so the subject never came up.</comment></comment>


How much CO2 will there be? - <AUTHOR> https://scienceofdoom.com/2018/12/29/opinions-and-perspectives-3-how-much-co2-will-there-be-and-activists-in-disguise/ <comment><USER2> "burn more natural gas instead of coal" \- says the author.

sciencing hard bro...

where are your morals and sense of proportion? does the word "methane hydrates" or "permafrost" ring a bell? let's focus on attacking particular models because surely the world needs more people attacking climate science.</comment>


Python AdminUI: web UI development with Python (Flask + Ant Design) - <AUTHOR> https://github.com/bigeyex/python-adminui <comment><USER2> Looks interesting... though the very basic "getting started" code has an error in it.

Also... messages are in Chinese so perhaps it's not intended for the west.

</comment><comment><AUTHOR> Author's blog post [0]

      [0] https://medium.com/@bigeyex/introducing-python-adminui-68b9f55153e3</comment>


New Encryption System Protects Data from Quantum Computers - <AUTHOR> https://www.scientificamerican.com/article/new-encryption-system-protects-data-from-quantum-computers/ <comment><USER2> dgb gives talks about the current status of NISTPQC often, 3 months ago he claimed they were only ~0% finished checking the submissions for even basic proof errors like unproven theorems/conjecture claimed in the submission papers to avoid a repeat of Asiacrypt 2004 where "OCB2" was standardized in 2009 then completely broken in 2018 due to finding a proof error, let alone ready to begin any kind of cryptanalysis on the risks left by the various proofs. Round 3 is already next year too with still a mountain of work to do. Kind of early for IBM to declare they're going to run a post-quantum encrypted cloud with their CRYSTAL suite.</comment>


Even low earners have money to spend - <AUTHOR> https://www.bloomberg.com/opinion/articles/2019-11-29/inequality-doesn-t-show-up-in-u-s-spending-statistics <comment><USER2> Paywall...</comment>


EEVblog's new YouTube channel inexplicably banned - <AUTHOR> https://www.youtube.com/watch?v=YWo9xDKXKq4 <comment><USER2> Damn shame. EEVblog is one of the shining examples of the sort of program that could only come about on a platform like youtube. I've often wondered why netflix or amazon haven't picked it up or made similar programming, and yet I have a hard time seeing them producing content of such quality and depth.

<comment><USER3> > I've often wondered why netflix or amazon haven't picked it up or made > similar programming,

Because due to how the economics works, once you add a corporation behind a genuinely informative project, it becomes an advertising machine losing about all of its value. EEVBlog works because it uses Youtube and not the other way around; if it was produced by Youtube things would be way different.

</comment></comment><comment><USER4> Per the stickied comment from Dave/EEVblog on YT: "The EEVdiscover channel is now back up! [https://www.youtube.com/channel/UCkGvUEt8iQLmq3aJIMjT2qQ](https://www.youtube.com/channel/UCkGvUEt8iQLmq3aJIMjT2qQ) Still no idea why though it was taken down though"

</comment><comment><USER5> Since they partnered with the 'Discovery' network, that could not possibly be related... /s

<comment><USER6> That is really sad that basically using a term like discovery is banned. Not only that, but it will let you create them, then ban them, and not tell you why.

If there were a legal case, I'd be excited about the _discovery_ phase of it, to figure this one out!

</comment></comment><comment><AUTHOR> Someone in the comments pointed out that Youtube's new structure is designed so that only famous people can make money. I have no idea whether this is true, but it would fit it with the constant drumbeat of threats, pressure and lawsuits from the large media companies. The democratized model of Youtube and other video sites is a threat to their business model. Even though they've gotten their way with pirated content, they now need to crack down on anything original, because the independent content takes attention away from the dross on television, and to them, the viewers themselves are their property.

<comment><USER7> So YouTube is the new TV, they're trying to compete with Netflix, and all the big names on YT just pulled out of advertising... and Google is culturally an ad company.

Yay. YT is too big to fail or be disrupted, but this will be a fun blip to watch.

</comment></comment><comment><USER8> What a shame. EEVblog and AVE have been some of my favorite engineering channels on YouTube for some time now, hopefully Louis Rossman isn't next.

</comment><comment><USER9> I love this guy's stuff! He puts out some great information!

</comment><comment><USER3> GoogleServicesWeBetterFindAlternativesForBeforeItsTooLate++;

<comment><USER7> Big local hard disks, and caching web proxies?

:(</comment></comment>


Can "Indie" Social Media Save Us? - <AUTHOR> https://www.newyorker.com/tech/annals-of-technology/can-indie-social-media-save-us <comment><USER2> The article says the indie web won't succeed because it lacks the engineered addictivity of the regular, 'big tobacco' web.

In reality, it doesn't even come to that. The indie web is, as of now, inferior to the point that no honest product comparison could make it seem an alternative. It requires you to learn concepts (like Mastodon's federation and servers) that does not benefit you in any real way in your day-to-day use, adds additional complexity, and makes things less practical on the user end in the real world, even if in abstract it does improve the ideas underlying.

P2P is still trying to win based on the goodness of people's hearts. That does not work - I'm saying this as someone who built and is running a P2P platform myself (Aether, [https://getaether.net](https://getaether.net)) so I struggle with the same thing. You have to be decentralised _and_ a better product. The only way to make this is to play to the strengths of P2P networks.

If you try to copy a centralised service to make it P2P you will always be at a huge disadvantage, since centralised networks will be always better at running centralised systems. If you do something that's only really possible on a P2P system though, then you can offer the centralised internet an actual alternative. I don't think anyone does that yet. (Though for whatever it's worth, I'm trying.)

<comment><USER3> I see all of your points about 'success' and 'inferiority' if I look at this from the perspective of Facebook being an 'upgrade' to the Web - ease of use, friends at your fingertips, a landslide of short simple updates.

However, I don't actually see it that way. I would have preferred to fork the Web back in 2007 or so. This is going to sound out of touch, perhaps, but the future of the Web for me, personally, consists of entirely different things. Something like: a place to store my writings/conversations that's mine and can be organized and designed like I prefer, a good way of finding new people (random little out-of-the-way folks of any stripe), and I don't want a big feed of random garbage - I want to visit other sites now and then, like I would visit friends IRL - I don't need everyone in my life everyday.

I don't even care about decentralization or P2P - I actually don't see that as a central part of the Indieweb. The above is what is important to me. And the Indieweb has been surprisingly satisfying in this way. And it's actually growing. Not much, but I'm happy with the friends I've made. I'm happy with HN as well, for learning things and staying out of my face - but it gets to feel a bit faceless at times because it's just a bunch of gray comments with often very little to distinguish the people here.

<comment><USER4> > However, I don't actually see it that way. I would have preferred to fork > the Web back in 2007 or so. This is going to sound out of touch, perhaps, > but the future of the Web for me, personally, consists of entirely different > things. Something like: a place to store my writings/conversations that's > mine and can be organized and designed like I prefer, a good way of finding > new people (random little out-of-the-way folks of any stripe), and I don't > want a big feed of random garbage - I want to visit other sites now and > then, like I would visit friends IRL - I don't need everyone in my life > everyday.

that's nice but it's not what prints money

<comment><USER5> The web should be a library, a collective knowledge. F money. We're in the friggin 21st century it should be time to look beyond.

<comment><USER6> The web already _is_ the biggest, most comprehensive library and store of collective knowledge in human history.

But it shouldn't _just_ be that.

<comment><USER5> > But it shouldn't just be that.

No, it shouldn't, true, it's also a media and a connectivity platform.

However, in my opinion, the web always should have been strictly non-profit. (Note: non profit means employees are paid properly but the "profit" goes back straight into development, infrastructure, and future proofing). Money involved is only destroying and splitting it.

</comment></comment></comment></comment></comment></comment><comment><USER7> I get the impression that nobody has really mentioned the elephant in the room, which is that Facebook and Twitter are so popular precisely because they are so big, and the IndieWeb is unattractive mainly because it isn't.

Like probably most of you here, I don't care for walled gardens. I host my own blog, avoid Google as best as possible, and am regularly one step away from deleting Facebook. But I don't. Why? Because many old friends are there, and I have no other realistic way of keeping up with them. There's no earthly chance I'll ever get them all onto Mastodon, so for better or worse I'm going to stay where they are. In the end, friendship is more important than philosophy.

On the contrary, after years of avoiding it, I recently started using Twitter. Again, the reason is that that is where the people are. I'm still a bit iffy about it, but I have realised that in my field, almost everybody is on Twitter. It is simply the most efficient way to keep up with what is going on and what the important people are doing. In the end, practicality beats purity.

So yes, I'd love to use micro.blog, or host my own Mastodon instance. But until they can offer me what the elephants of Facebook and Twitter can - i.e., a global community - they don't stand a fighting chance.

<comment><USER8> On the other hand, exclusivity has its appeal.

I come to HN precisely because not everyone knows about this place (and even if everyone knew, not everyone would come).

I don't go much on Reddit anymore (except for some pretty niche subs), because I know that I'll read the same dumb comments over and over again.

<comment><USER6> >I don't go much on Reddit anymore (except for some pretty niche subs), because I know that I'll read the same dumb comments over and over again.

Not to slag on HN, but you'll often read the same dumb comments over and over again here as well.

</comment></comment></comment><comment><USER9> Mastodon may not "save" us but I find it to be enjoyable. Smaller communities can actually be moderated _properly_ , and not everything is always about collecting data, pleasing advertisers, etc. It feels more like a real internet community again.

<comment> Funes- I've really tried to like Mastodon, but I see too many toots accusing seemingly innocent people of being "fascists" and being complicit in the killing of minorities every time I step foot in almost any instance (that was an extreme case to illustrate the actual situation, but I hope you know what I'm saying). As well as other types of content you can already find on Twitter: rants, uninspired one liners, ego masturbation, etc.

I don't like these kind of broadcast platforms because they tend to incite abuse; only in Mastodon it leans more toward the left most of the time, given the strict rules most instances enforce. I also don't like not owning my data.

<comment> phoe-krk > I also don't like not owning my data.

In molochs such as Twitter or Facebook, you already do not own your data, so the only option is to not use them. When it comes to federated networks, the resolution is simple: do not federate with the instances whom you do not trust.

Regardless of the choice you make, if you make it, then please do not complain that your content has no visibility. If you want other people to see the stuff you post, you naturally give them a right to save, repost, screenshot, print, memorize or otherwise make a copy of it. If you do not want that to happen, either do not post it online, or invest in getting a social network to run via Widevine.

<comment> Funes- >In molochs such as Twitter or Facebook, you already do not own your data, so the only option is to not use them. When it comes to federated networks, the resolution is simple: do not federate with the instances whom you do not trust.

I do not have an account on either Facebook nor Twitter. Federated and centralized networks are not that different in terms of how your data is handled: they both rely on trust. Indeed, in centralized platforms you only have one possible group of people in which you can place your trust, but it comes down the very same thing in both cases: hosting your data somewhere else and hoping for the best. Peer to peer networks come with other shortcomings, but that is not one of them.

>If you want other people to see the stuff you post, you naturally give them a right to save, repost, screenshot, print, memorize or otherwise make a copy of it

I agree. Nonetheless, microblogging platforms like Twitter or Mastodon incentivize abusive content with their design. That's why Jack and others at Twitter are testing new ways of presenting and interacting with tweets by means of projects like twttr, and also why some instances on Mastodon let you disable boosts and so on. I won't complain about anything because that's not something I generally intend to do and because I don't think I will ever sign up for Mastodon.

</comment></comment></comment></comment><comment><USER10> Is HN "indie" enough for this definition?

There are plenty of non-facebook social media platforms. From HN to minecraft forums. Eve-online and hundreds of associated websites is essentially one large social media network. And there are plenty of online communities catering to all the stuff you cannot talk about on facebook (piracy, hacking etc). Teens gather in these places specifically to avoid facebook.

Each of these network nodes is run by an independent person or very small group. I think they together get at least as much traffic as the larger networks do individually. We just don't talk about them as much because they don't collect the statistics that facebook does. They don't market themselves to advertisers like facebook does. They don't seek to spy on their users like facebook does.

</comment><comment><USER11> Calling it the indieweb is spot on imo. I was running IRC servers and phpbb message boards throughout the 2000s and it seems to me that the only thing changed with the web is the massive influx of novice users on big social media.

The rest of us have always been, and likely always will be, a sub-culture of sorts. We used to have message boards, now we have federated social media instances.

If you look at a list of mastodon instances it reads like a list of privately hosted message boards in the 90s/2000s. Or a list of privately hosted BBS' in the 80s.

The major difference being that these islands are now all federated.

In my youth I had a dozen IRC accounts setup in my .irssi/config (or BitchX;), today this would have to be federated. (iWish IRC was fedarated too but that's another story)

So not much has changed on the social aspect. And that's where I believe we're headed. A big shallow part of the internet will be used by people who have better things to do than engage in deep conversations on the computers about their special interests.

While a much smaller, sub-culture, will use specialized forums for their special interests.

So I guess the sad conclusion is that you shouldn't expect the IndieWeb to go mainstream.

</comment><comment><USER12> Imagine if the internet was owned by one company. Or email was owned by one company.

Instead we have a non profit set of standards which divide the internet into different layers. So you can have your ISPs, your web hosting providers, your websites etc. There is a healthy demarcation which prevents consolidating of power.

Maybe the way to breakup twitter/facebook etc is to create a set of protocols and divide them into layers.

For instance I should be able to register a username for tweeting like how you register a website name for DNS. Once registered you have a choice to store your tweets in any data hosting service.

Then a number of aggregate readers handle the front end user interface.

Each reader can have its own content filter and discovery algorithms. So even if your content is found offensive in one reader, you can just switch to another. If one is really addictive in a negative way, switch to a less intrusive useful version.

<comment><USER13> Email is an instructive example.

In theory, anyone can implement a standards-compliant system and play on the same level as the major providers. In practice, the email ecosystem has had to evolve a series of defenses against bad actors who took advantage of precisely this standardized openness.

There are, in practice, a relatively small number of email providers that users are likely to interact with. They all find ways to do it profitably. Power is very much consolidated.

<comment><USER14> > There are, in practice, a relatively small number of email providers that > users are likely to interact with. They all find ways to do it profitably. > Power is very much consolidated.

For now.

It wouldn't surprise me a bit if EU takes time to look into this after they've finished what they've already started.

In fact I'd recommend we all send complaints to local authorities and point out if big actors are blocking our servers without reason (and no, I guess "didn't care to verify" won't fly.)

</comment></comment></comment><comment><USER15> I've only dabbled, but my impression is that mastodon and micro.blog are oriented towards building "online" communities like Twitter, ie communities of interest with people you likely do not know in real life.

Facebook and to some extent Instagram seem to be most often used to share within pre-existing offline networks; ie you might use it to follow life updates from someone that you also see in person on occasion. This works because everybody is on it — you don't have to hope that your acquaintance from college (or whatever) has picked the same instance as you.

Is there an "indie" real-life-network alternative? Is that what Diaspora is intended to be?

<comment><USER16> Friendica works with Mastodon and Diaspora and intends to do exactly that.

</comment></comment><comment> Funes- Yes, but only social media that is actually "social"; that is, platforms which enhance our offline lives instead of keeping them online.

<comment><USER17> I don't buy this.

I've met a ton of people online that I've never met in person and I've learned a ton from them. In some cases I've met these people in person and wonderful friendships developed as a result even though most of the interaction was online.

There's nothing wrong with online communication. And I'm tired of the narrative that something is inherently more meaningful just because it's in person. It's not. There is far more nuance than that.

The internet is a tool and it can be wonderfully used as well as abused. We need to make it easier to use in productive, healthy ways. But to spread this false-narrative of "online vs offline" is mostly people trying to resist change in my estimation.

<comment><USER18> You're replying to a lot of things, none of which the person you replied to actually said.

<comment><USER17> It's absolutely possible that I mis-interpreted OP. If that's the case, I apologize.

</comment></comment></comment></comment><comment><USER19> > I was both pleased and chagrined by the irony of the fact that my anti- > social-media talk had found such a large audience on social media.

This immediately reminds me of the "Fifteen Million Merits" episode of Black Mirror.

</comment><comment><USER20> This data that the tech Giants hold close to their chest belongs to the people and should be released for competitors to use in their products.

I envision companies competing on UX/privacy/ect while all using the same data set. People should be able to use Myspace to talk to their grandparents on Facebook.

How it is now is like if Ma Bell was never broken up, and we were only able to use our AT&T phone and data plan to talk to other AT&T customers only. How could any other phone company start with such barriers in place.

But who am I? Just the last human.

<comment><USER14> Personally I really don't want the data I shared with one company to be shared with others without my explicit content.

I don't particularily like Google, but unlike a bunch of others they haven't shared my data voluntarily AFAIK.

In a way they already compete on privacy.

If my data gets shared among a bunch of them my privacy will be decided by the most sloppy one.

</comment></comment><comment><USER21> Back in the early 2000's, I became part of a large group of friends. Someone in the group hosted a private web site where friends could post pictures and messages. When MySpace became popular, people started migrating over there and then to Facebook. You could say this was the private or indy social network of the day.

I was disappointed to see the site fall by the wayside, It's just easier, and that's were people are.

</comment><comment><USER22> There is still usenet groups, forums and email lists around. Those technologies have existed for decades and are optimised for user happiness/utility, not user engagement (drastically different and all the fancy facebook measurement only gives you engagement).

<comment><USER23> > optimised for user happiness/utility,

obviously not. They were optimised for the conditions that existed at the time, which included limited bandwidth and intermittent connections, which are no longer realistic limits.

Usenet was wonderful and still has many pockets of wonderful in it; but even I who cut my teeth hacking UUCP to WWIV gateways and such want nothing to do with setting up a modern NNTP impementation.

I don't pretend to have the answers to "how do we make a usenet analog happen again?" but the remnants of antiquated forums don't effectively argue against the need for trying new forms of forum.

</comment></comment><comment><USER24> Not unless it comes bundled with an indie society.

</comment><comment><USER25> I am willing to bet that someday, we will return to our roots: classic communities forming organically around topics without algorithms guiding us along. But until then, let us remain hopeful and vigilant.

</comment><comment><USER26> What's there to stop any of the big sites from buying them out before they become a threat?

<comment><USER27> Mastodon is AGPLv3 so there is nothing to buy.

<comment><USER28> It is possible to use the embrace-extend-extinguish pattern. Has been done with XMPP.

<comment><USER29> I sort of welcome that...in a way...Because to me that signals that the federation protocols (not unlike xmpp) are worthy of attention (and fear by) the big boys of the web like FB, Twitter, etc.

</comment></comment></comment></comment><comment><USER30> I doubt indie social media will save us from facebook, no more than indie journalism will save us from the new yorker.

I hope one day the likes of facebook and the new yorker doesn't exist, but I'm not holding my breath.

The only thing we can do is stop consuming divisive toxic garbage that is facebook and the new yorker.</comment>


Watch TEDxCMU LIVE - <AUTHOR> http://tedxcmu.com/watch

<comment><AUTHOR> RF Culbertson just rapped about being fearless...wow RF is the man</comment>


Matlab, R, and Julia: Languages for data analysis - <AUTHOR> http://strata.oreilly.com/2012/10/matlab-r-julia-languages-for-data-analysis.html <comment><USER2> You know what will be popular? whatever runs reasonably fast and helps you import and clean data quickly from a variety of sources.

Because the analysis is often the quickest part of being a data scientist. Coursera, as I recall, apparently cleans it's data, and also lets you easily import it.

In real life, data is messy, and messed up. You looking at birthdays from some website? expect a spike for whatever the default is... but that doesn't mean you can eliminate that data completely, because some people were presumably born Jan 1st.

You looking at birth years? I recall dealing with them in SAS... remember if it's four digit that you check for births occurring in the current and past century.

And hey... do you have two or more elements of data for an individual? 2% to 5% will probably be missing some element, and some will have wrong data. a zip code off by one, an address not in the city you are looking to geocode for, whatever. If you are lucky, it will be obvious stuff like that.

The life if the data scientist is mostly cleaning, formatting, and transferring data, with the occasional sweeeet analysis. Of course your analysis will probably give you nothing useful, because despite several thousand usable records, it's not clear if any element has a significant effect on the dependent variable you are looking at. If you are smart, maybe you can finagle an analysis based on a non parametric distribution or logistic regression.

Oh, and often the speed of your analysis running is inversely correlated with how easy it is to code and enter your data. There is a reason people use SAS, and it's not because of it's amazing IDE.

</comment><comment><USER3> I wouldn't be surprised if Octave (open source version of Matlab) doen't become very popular because a lot of Coursera classes use it for homework assignments.

I thought that Octave was an ugly little language at first, now I really like it - a great tool for doing linear algebra, data visualization, machine learning, neural networks, etc.

<comment><USER4> Many use Matlab because of its amazing IDE, a great collection of toolboxes and remarkable speed, not so much because of its language features. Last I checked Octave was still missing all of that. If you can afford it, Matlab is usually well worth paying for. If not, other alternatives (Python, R) are much better in my opinion.

<comment><USER5> When I used Matlab daily, I never used the ide. The command line and editor were good.

Matlab is awesome above all else because the design is coherent. Both the syntax and the standard libraries.

It is extremely quick to whip up anything and then turn that into a script and then into a software with functions (since functions can return many variables, and they also have zero overhead, you don't need any includes or requires, you just call them). Type conversions are practically never a problem, since they are sane and automatic. None of this 1+1.5 giving syntax error. Real booleans. Data input and output libraries just simply work like you would expect them to. ( A=imread('/home/gravityloss/abc.png') creates a width x height x 3 matrix with all the rgb values. No requires, includes, plugins, hunting and compiling libraries.). You don't need libraries to do a huge amount of stuff, but if you need them for something experimental, they work extremely easily.

You also rarely need stuff like loops since mass operations on data are native. If you as a newbie create a custom function for a scalar, there's good chance it will work for vectors or n-matrices automatically. This reduces the amount of error-prone housekeeping code for indices and lengths immensely. It's also much much faster than some looping in another scripting language. As a result, the code is often very readable as well.

There's help which actually returns something sensible when you type help, you can type help help or help command or search this or that, the help texts are actually very thoughtful and helpful too and not at all like Linux man pages... I could go on for hours on features that don't really exist anywhere else, even though everything's been in plain sight for decades in Matlab.

Julia's an awesome thing though, I hope it gets more traction...

<comment><USER6> Virtually none of that is in fact unique to MATLAB or even a strength in the first place.

      > Matlab is awesome above all else because the design is
      > coherent. Both the syntax and the standard libraries.

It certainly is coherent, and also consistent, but only in the weakest and least interesting sense. Namely, everything's about equally messy. Namespaces are non-existent in the standard library and clumsily realised otherwise. OOP remains rudimentary and feels as tacked-on as it happens to be. The one- function-per-file system ruins everyone's day. No standard arguments (and checking _nargin_ loses its appeal rather fast), and shitty inlined pseudo- lambdas.

      > It is extremely quick to whip up anything and then turn
      > that into a script and then into a software with
      > functions

No different from R or Python, and most of the time a genuine weakness; it's a key reason for scientific/engineering code being as ad hoc and convoluted as it is.

      > If you as a newbie create a custom function for a
      > scalar, there's good chance it will work for vectors or
      > n-matrices automatically.

Vectorising functions in Python (that is, _NumPy_ ) is about as straightforward.

      > It's also much much faster than some looping in another
      > scripting language.

Nah, MATLAB is now fairly good at unrolling and optimising such loops. Don't worry too much about vectorising every single bit of your algorithm.

      > There's help which actually returns something sensible
      > when you type help

That's also true in the cases of both Python and R. Long story short: Most of your perceived advantages aren't unique, and that comes on top of MATLAB's exorbitant pricing schemes and _extremely_ dubious language design. Trust me, if you think MATLAB's a particularly well-designed language for anything other than linear algebra, you owe it to yourself to check out alternatives and other languages.

<comment><USER5> True, Matlab has its limitations, but those are partly unavoidable. If you want to build a large object oriented program, you often use something more heavyweight anyway. But that heavyweight language (or framework) is usually not so quick to build something in anymore, because your heavyweight structures are just in the way in the earlier phase.

I tried Python and Numpy, and the vectors, matrices and all that felt just tacked on and the syntax was much more complex compared to Matlab. Maybe it's changed since. Also in Scilab the type conversions and function overhead are a nuisance. Every time you edit a script or function, you have to specifically reload it before running it. Makes rapid prototyping about three fold as time consuming. Would it be hard to make the software notice I actually edited something?

Many people actually want to solve problems, and they just end up creating a program as a side product. They do not set out to study libraries and do not want to actually write any code that is not directly related to the problem they are solving.

It's why Matlab is able to charge the price. It sometimes saves time. Some of the users are not primarily software developers but are quite educated and intelligent and their salary is not small.

<comment><USER7> So you're implying anyone who is a software developer is not educated or intelligent, nor has a large salary. I think a lot of people would beg to differ.

In reality, matlab only exists because of inertia. It's the same reason why microsoft windows is still around. There's no substance behind it.

<comment><USER8> He wasn't implying that. You've inverted his statement. Many economists and engineers I know care less about how they code and care more about getting a solution to the model at hand--this seems to be the poster's point. The implication is exactly what he exposited, whereas the logical inverse is what you've mistakenly deduced as the implication.

<comment><USER7> His last paragraph is not clearly worded then. It comes of as thinly veiled insult to software developers.

<comment><USER9> Perhaps it comes off that way _to you_ but I'd be willing to bet it doesn't to _the great majority of readers_ , because it simply isn't saying what you say it's saying.

<comment><USER7> OK, but he's worded it wrong, it gives the impression of a snarkiness. Maybe he's saying that matlab users can't program well but are still intelligent/well paid (but that doesn't really make sense since numpy is equally easy and an intelligent/educated person wouldn't find programming hard). Anyway, maybe I misread it and you're right.

edit: great, I was getting points before, and then you come along with your italics.

<comment><USER8> It's all about the illiquid karma ;). No worries.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER4> Here is a nice 4-year old still active discussion on pros and cons of different data analysis technologies: [http://brenocon.com/blog/2009/02/comparison-of-data- analysis...](http://brenocon.com/blog/2009/02/comparison-of-data-analysis- packages-r-matlab-scipy-excel-sas-spss-stata/)

<comment><USER8> HellMcFly--you're hellbanned so I couldn't reply directly. What is MPlus?

<comment><USER4> Mplus is quite popular in social sciences. From what I understand its main functionality is fitting Latent Variable Models and structural equation modelling. I've never used it myself, but it can in fact do things for which it is hard to find R packages at this point.

</comment></comment></comment><comment><USER10> I think the article tagline would be better "Domain Specific Languages for data analysis". Fortunately, the article does mention Python which is critical because new people might not recognize just _how_ prevalent Python is for solving data analysis problems after reading this. The great work of the SciPy community has enabled Python to be used for _all_ of the things that Matlab, R, and Julia can do. In addition, Python can integrate easily with these languages, so if you are a data analyst you need to learn Python.

<comment><USER11> > The great work of the SciPy community has enabled Python to be used for all > of the things that Matlab, R, and Julia can do.

As much as I hate R and love Python, this is not entirely true (unless you count rpy2 as part of "Python"). R has many more statistical models and better plotting capability compared with Python. It also has a lot of domain-specific packages (for example, Bioconductor) that are not available in Python.

<comment><USER12> Though Python doesn't have the library support that R has, it far exceeds what's available in Julie (and,depending what you are looking for, in Matlab as well)

<comment><USER8> Sure. But Julia is brand new and already supports C/fortran libraries.

<comment><USER13> .. which are supported by all the environments in question

</comment></comment></comment></comment></comment><comment><USER14> Personally I'm in love with R's data.frame. It allows very concise, robust and elegant manipulation and subsetting of a data set.

I wish every language would have such a built-in object type, I definitely feel its loss when I manipulate data in other languages such as Javascript or Mathematica.

<comment><USER15> > Personally I'm in love with R's data.frame. It allows very concise, robust > and elegant manipulation and subsetting of a data set.

The performance is terrible though. For data of more than ~10,000 observations SQL is much better performance wise, is more robust, and is as good at subsetting. Although it's maybe not as elegant for everyone's definition of elegant.

<comment><USER16> What dataframe operations do you find to be slow? Usually I'm able to get huge performance wins by rewriting my slow R code in a loop free way (*apply and friends).

</comment></comment></comment><comment><USER17> I wonder if there is room for some smaller languages optimized specifically for data analysis. In particular, I wonder how a carefully designed non- Turing-complete language would fare.

That would be a really cool project to work on: design a minimal language for expressing most types of data analysis at a higher level. If the language is sufficiently small and simple, I could see some very powerful tooling being possible for it.

Perhaps it might make sense to go even more specific: have a small language designed not just for data analysis but for analysis in a very specific vertical (say finance or bioinformatics). It would be awesome to let people express their ideas in terms of the domain and not worry about low-level details like loops.

<comment><USER4> It seems like a good idea, but I wonder how actually useful highly specialized programming languages would be. Why?

1) Most data analysis tasks boil down to roughly the same things: accessing the data source --> data cleaning -->simple transformations --> (optional)stats/fitting/ML/specialized procedures-->pretty pictures and reporting.

2) Not everyone wants programming to be the main component of their job.

People who can take advantage of the flexibility that programming offers can usually take advantage of existing technologies. People who don't enjoy coding will always look for of-the-shelf solutions that have pretty GUI's with magic buttons that solve all their problems. I just don't think there is a huge market in between to be filled... in the domains that i've been exposed to anyway.

<comment><USER18> I disagree with your supposition. I think highly specialized languages exist and are highly useful to non-programming communities. I think there is plenty of proof of their usefulness and room for growth.

For instance, consider illustrator products or d3? Both of these are specialized ("deep") tools for creating pictures that I've used extensively in the "pretty pictures and reporting" stage you outlined.

Also of serious note are BUGS[1], JAGS[2] and (recently) Stan[3] as small semi-declarative languages for MCMC model building, fitting, and checking.

SQL is an obvious example of a component of the "simple transformations" step.

[1] BUGS <http://www.mrc-bsu.cam.ac.uk/bugs/> [2] JAGS <http://mcmc- jags.sourceforge.net/> [3] Stan <http://mc-stan.org/>

</comment></comment></comment><comment><USER12> When introducing python, the author writes "Despite the obvious advantages of MATLAB, R, and Julia, it's also always worth considering what a general- purpose language can bring to the table."

Even with thousands of hours of experience in Matlab, R and Python... I'm not sure what "obvious advantage" Matlab and R share over Python.

<comment><USER8> For me, depends on how new a person is to the language. Numpy certainly has a learning curve after spending a long time in Matlab.

<comment><USER12> I hope the "obvious advantage" the author is speaking of isn't "it takes a while to learn numpy if you are used to Matlab."

That would be a pretty weak argument in my opinion.

<comment><USER8> I agree :D

</comment></comment></comment></comment><comment><USER19> python fanboy here: "[python is] not as tuned to numerics as MATLAB": if you build numpy with ATLAS there is, in my experience, hardly ever any noticeable speed difference between numpy and MATLAB

<comment><USER20> " Python a compelling alternative: not as tuned to numerics as MATLAB, or to stats as R, or as fast or elegant as Julia "

The part about python not being as fast as Julia jumped at me. Wes McKinney's benchmarks show that python is faster than Julia for numerics: <http://wesmckinney.com/blog/?p=475>

EDIT: should not have said "python faster than Julia". They are comparable because the slow bits get done in BLAS anyway.

<comment><USER21> A couple of nits...

Cython is actually what is faster than Julia in Wes' comparison, not Python. Cython looks kinda, sorta like Python, but it is actually a static language with C-like types (but quite different syntax for those types), no polymorphism, and, afaict, ill-defined semantics. The best answer I seem to get about Cython's semantics is that Cython's semantics are whatever it does. I'm not alone in this complaint – Travis Oliphant expressed a similar concern at this year's SciPy (in this panel [<http://www.youtube.com/watch?v=7i2vhoQY-K4>], if I recall correctly), which is part of his motivation to work on Numba [<https://github.com/numba/numba>].

If you look at the comments on Wes' post, when I used the dot(x,y) function, which ships with Julia and uses a BLAS to compute the inner product just like the fastest "Python" version does, Julia is equally fast. That stands to reason – they're both just calling a BLAS.

Finally, that blog post is months old – since then Julia passed the milestone of being no slower than 2x C++ on its microbenchmarks suite [<http://julialang.org/>]. That's not a guarantee that all code is that fast, but most things we see can be pretty easily tweaked to get there (counterintuitively for those coming from Matlab, Python or R, usually by _devectorizing_ the code rather than vectorizing it). And of course, there's a lot of room for improving Julia's performance, the compiler is still quite young and there are many optimizations that we haven't implemented. Basically, there's nothing but work standing in the way of reaching C or Fortran's speed across the board.

<comment><USER22> I just ran Wes's benchmarks (not the BLAS call versions) on my machine with a Julia I built on 10/13 (17c3c13), and the timings have indeed improved. For the details, see this gist: <https://gist.github.com/3901139> (including the comment I posted on it).

The highlight is

numpy: (x * y).sum() => 41.1 ms

julia: inner(x,y) => 37.4 ms

julia: x*y => 19.5 ms

cython: inner(x,y) => 13.8 ms

The numpy and Julia versions are much easier to write and run.

Disclaimers: I've never written or built cython code before just now, and I think Julia is the coolest.

EDIT: whoops, missed the most important one (inner() written in pure Julia). Added it. Any thoughts on why inner() in Julia isn't faster?

<comment><USER21> Nice. Thanks for running those. The reason inner isn't faster is probably that we do bounds checks on every array access. This is surprisingly inexpensive on modern hardware but it still takes some time. We're working on a couple of things to address this: generating code so that llvm can more easily hoist bounds checks out of loops, and allowing turning bounds checking off entirely for blocks of Julia code.

</comment></comment></comment></comment></comment><comment><USER23> I have to create figures with Matlab and that's a pain in the ass. Changing XTickLabels, kills another part of the figure, and in general it's very hard to do a little more with figures.

But the basic data analysis is fine. The IDE has awful code completion and lacks more refinement in the editor.

<comment><USER24> Python has a wrapper for matlab-style graphs called matplotlib, if you are interested in something else.

</comment></comment><comment><USER21> This is a really excellent and well-balanced article. Very much captures the pluses and minuses of these various systems for data analysis.

</comment><comment><USER25> One of my bioinformatics courses "required" MATLAB because the class project was based on a simulation framework called the COBRA Toolbox which was developed in MATLAB[1]. I didn't know who to ask about obtaining a MATLAB license, so instead I just got it to work in Octave and used that. I was pleasantly surprised at how little I had to tweak before the framework just worked in Octave, given that as far as I know everyone in the lab that develops the framework just uses MATLAB.

[1] <http://opencobra.sourceforge.net/openCOBRA/Welcome.html>

</comment><comment><USER26> Perl was mentioned in the article, but PDL (Perl Data Language) wasn't.

<https://metacpan.org/module/PDL>

 _PDL is the Perl Data Language, a perl extension that [...] includes fully vectorized, multidimensional array handling, plus several paths for device- independent graphics output._

 _PDL is fast, comparable and often outperforming IDL and MATLAB in real world applications. PDL allows large N-dimensional data sets such as large images, spectra, etc to be stored efficiently and manipulated quickly._

For integration with R, there are Statistics::R (<https://metacpan.org/module/Statistics::R>) and Statistics::useR (<https://metacpan.org/module/Statistics::useR>)

</comment><comment><USER27> Everyone loves to shit all over Java, but Mahout, RapidMiner, Weka, Hive, HBase are all written in it.

<comment><USER2> I've used Weka, and RapidMiner once. As I recall, RapidMiner seemed to be general purpose, but lots of posts were for using it for data mining stock data to build a model.

I think it would be interesting to see breakdowns of different software, and where they are used. Often times it seems to me that people just use the tools their peers and co-workers use, and people tend learn to like whatever they use most.

</comment></comment><comment><USER28> Thought I'd ask since I'm learning Clojure - are there experiences worth sharing re: using Incanter in these types of settings?

<comment><USER29> I'm also learning Clojure and have been playing with Incanter. Seems quite a decent statistical library/environment. I had a few issues with lazy evaluation with the dynamic charting functions, but I think that has more to do with my inexperience with Clojure than a problem with Incanter. Also, I'm not to sure how active the project is?

</comment></comment><comment><USER30> no love for J?

<comment><USER31> J is more of a general-purpose array language than one specialised for numerical work.

</comment></comment><comment><USER31> surprising omission at the end - any mention of scipy should at least include a pointer to sage as well.</comment>


Investigating the Galileo Satellite Navigation System Outage with a LimeSDR - <AUTHOR> https://destevez.net/2019/07/galileo-constellation-outage/ <comment><USER2> How often has GPS, excluding intentional military operations, been unavailable?

Edit: Went looking around and it seem GPS has never suffered an outage. They have had issues with some satellites, even two days ago it seem, but never a true outage. [http://navigationservices.agi.com/SatelliteOutageCalendar/GP...](http://navigationservices.agi.com/SatelliteOutageCalendar/GPSHistoricalOutages.aspx)

<comment><USER3> Galileo has not declared Full Operational Capability (scheduled for 2020), it's still in the Initial Services stage. It's a bit like complaining that a beta has bugs.

<comment><USER4> The testing phase ended in 2016 - Galileo is a live GNSS network intended for actual use in navigation. The reason they haven't declared Full Operational Capacity is that they still haven't managed to deploy a full constellation of satellites. All of the official announcements and navigation messages from the satellites say that it's meant to be providing actual service, just with slightly degraded accuracy due to the missing satellites.

Not only that, the EU's controversial eCall initiative, which requires cars that go on the market after April 2018 to automatically call the emergency services with their locations and a live mic to the interior if they detect a crash, mandates the use of Galileo for positioning. Manufacturers can (anbd probably will) use other GNSS systems in parallel, but Galileo is the only one that's guaranteed to be supported by cars. This is for a system whose life- saving benefits are supposedly so important as to outweigh the privacy issues.

</comment></comment></comment><comment><USER5> The author mentions how they're _not_ going to talk about rumors several times over the course of the article. I'd love to hear what the rumor mill is saying about the possible causes of the outage.

</comment><comment><USER6> I'm no expert but according to this site, linked in the article, it appears the system has started to come back online and that the issue was limited to the ephemerides: [http://www.navsas.eu/node/611](http://www.navsas.eu/node/611)

<comment><USER3> Service was restored yesterday shortly before 9pm UTC: [https://www.gsc- europa.eu/notice-advisory-to-galileo-users-n...](https://www.gsc- europa.eu/notice-advisory-to-galileo-users-nagu-2019027)

</comment></comment><comment><USER7> It is a massive oversight that the satellites don't mark their own data as bad if not commanded to from earth.

There are lots of reasons ground stations could go offline, and having the satellites broadcast no data is far far better than broadcasting incorrect data.

</comment><comment><USER8> Anyone in here have a LimeSDR? They look great on paper but I get a sense from what I've read that they can be a bit finicky to run.

Also I'm curious about the 1-bit recording. Is that a thing?

<comment><USER9> 1-bit recording is in fact a common thing for very noisy signals. You're actually aggressively over-sampling and using correlations over many samples (in time) to get the signal out from under the noise. I've only glanced over it, but this looks like a fun related article, but driving the opposite way (oversampling to get more effective bits) [https://thecavepearlproject.org/2017/02/27/enhancing- arduino...](https://thecavepearlproject.org/2017/02/27/enhancing-arduinos-adc- resolution-by-dithering-oversampling/)

<comment><USER10> Thanks! There is a lot on the site, I really liked:

[https://thecavepearlproject.org/2015/01/29/a-simple-diy- unde...](https://thecavepearlproject.org/2015/01/29/a-simple-diy-underwater- connector-system/)</comment></comment></comment>