Cost of Living - <AUTHOR> My is gearing up to submit to the ycombinator winter round, and we are having a small amount of trouble to see how to fit 4 months into 5000 per person?  Especially when 2 of the guys are married.
this is the budget i have for the south bay area:

 housing         600
 internet         50
 electricity     100
 food           1050	35$ per day
 water            40
 entertainment   100

 cost            1940
 how much should we budget per person? right now we have 2 single guys on the team, and 2 newly married guys, that would either want to bring their wives with, or they would need to visit often.
any advice would be great to see how this would work, but right now it seems like we would need money just to win ycombinator, which for those of us in Milwaukee, the VCs/ Angels alone might be worth it. <comment><USER2> "I'll keep it short and sweet -- Family. Religion. Friendship. These are the three demons you must slay if you wish to succeed in business." - Mr. Burns

<comment><USER3> "Misquoting Mr Burns...that's a paddling."

<comment><USER2> <http://tinyurl.com/3ex4y8> ? =(

<comment><USER3> "Quoting Mr Burns correctly? Oh, you better believe that's a paddling."

</comment></comment></comment></comment><comment><USER4> Depending on your age and current lifestyle, saving up beforehand makes a world of difference. I currently own a house and have a very comfortable job (which I would have to leave). Fortunately, I have saved up for the obligatory 6 month emergency fund, which I may dip into if my team is accepted into the program.

If you have decent savings, then great. Planning is always key, but taking risks are where people make themselves in life...

</comment><comment><USER5> I live in the Bay Area, and those #'s look pretty close to me for a minimum, penny-pinching baseline. When it comes down to it, you can almost forget about the money ycomb is offering. The money is NOTHING. It is all about the contacts and guidance, which is priceless.

<comment><USER6> I agree with that sentiment. I am married, and I have a child. My wife and child will not be coming with me to the Bay area if I am accepted. I will be living in a cardboard box and eating ramen in the Valley, but the necessary financial arrangements have been made to ensure that my son will continue his current (private) school, the bills will be paid at home, and life will continue more or less as normal for the 3+ months I am away.

It comes down to how badly you want the experience. Look at it this way: Grad school at Stanford is ~40k/year, not including living expenses. Making whatever sacrifices necessary to experience ycombinator should be significantly less, but the experience and contacts most likely will dwarf what you are most likely to gain from a top tier graduate program in Engineering. Is it worth the investment? I can't speak for anyone but myself when I say "Absolutely."

</comment></comment><comment><USER7> your food budget is 2x what I would consider normal, your housing budget is about 1/2 what I would consider normal.

However, about $2K a month is pretty close to average living expenses, no matter where you cut back.

here is my breakdown

housing: 1100

internet: 0 (upstairs neighbors let me leech, coffee shops)

electricity: 0 (covered in rent)

food: 300 (burritos, cook at home, burger/pizza on weekends)

water: 0 (included in rent)

entertainment: 100 (beer)

gym: 60

cost: 1560

if you live in the south bay, you forgot car, car insurance, parking and/or caltrain.

<comment><AUTHOR> thanks for the info, great response

</comment></comment><comment><USER8> The womenfolk can work too.

</comment><comment><USER9> I say worry about that if and when you get accepted.

</comment><comment><USER10> $35/day for food is insane. You can eat really really well for less than half of that (if you are willing to cook).

</comment><comment><USER11> The food is high and the housing sounds low.

Have you checked housing advertisements?

<comment><USER12> 600 a month rent is what I'm paying, splitting a large rental house with 3 other people in Santa Clara, so definitely doable.

I agree that the food is really high. If you are really trying, the one time cost of a rice cooker and a 10 lb bag of Basmati rice from Costco will go a long way, as will bag lunches instead of going out. If you try, food can really cost very little.

<comment><AUTHOR> i have found more opportunity working in a cafe, and going out, than i have by working in an enclosed space. I figure

2-3$ for breakfast (egg, cereal, organic milk) lunch will cost (12$) at any reasonable panera, etc... and if i get something else cheap for the day, that means another day i can invite someone to lunch for a networking opportunity. that leaves me with ($10-$20) for dinner. eating just noodles does not work for me, i need to get some sort of carnivorous delight for a meal at least once a day, fish is like 6$ a pound and then you have veggies on top of that if you cook at home, or a sushi tray is like 8$-12$ for a small one.

At 28, I can not live like a freshman college student any more. I just can't think on 1800 calories of carbs. I need a more well balanced diet.

Am i wack for thinking this? I know the average age of YC is 25, and on our team we are all 27/28, with a few younger and a few older helping on the side lines.

<comment><USER11> > I have found more opportunity working in a cafe, and going out,

That may be true, but are you really lacking in opportunity if you're in a decent startup?

>i can invite someone to lunch for a networking opportunity

Is this "networking" for the startup, you, or is it a euphemism for a social life?

Let me suggest a test. Networking involves enough pre-meeting prep to enable a non-trivial agenda. Post-networking, there will be a debriefing to evaluate whether you achieved your goals and possibly what to do about it. You may not achieve said agenda every time, but if you do, that will often result in work that you have to do for the startup. If it rarely does, then either you're not very good at networking or you're not networking for the startup.

So, let's hear more about the value of this networking to the startup.

<comment><AUTHOR> I guess now that I am in the bay area, Networking is a whole different beast. Everyone is doing something, so its more about something in detail and with an agenda, than it is about just finding people in general.

</comment></comment></comment></comment></comment><comment><USER13> Mainly your food seems extremely high. If you don't mind the tasty Lipton noodles ($1), Shells and Cheese ($1.50), Frozen Pizza ($2), Hot pockets ($3 for 2), Spaghetti (~$2), Canned soup ($1-2), Cereal ($0.25)... you can eat a lot of cheap meals. I don't see why it should be hard for someone to eat nicely at $300/mo.

For instance, last night I ate a chicken breast ($1), mashed potatoes (instant kind - $2), and green beans (can - $1.50), with a can of Coke ($0.25)... a decent meal for $4.75.

</comment><comment><USER14> As a married person who applied to ycombinator (and didn't get selected):

You are totally focusing on the wrong things - think about the application, your product, and your team (not in that order actually). Worry about the other stuff once you get in.

\- Visiting/bringing wives is nice, but keep in mind that apart from the expense, your cofounders won't really be seeing them all that much.

-Make sure all or a majority of your founders can move. If they can't you've just given YC an easy way to move on to the next application..

<comment><AUTHOR> We are discussing it tonight, and for those people out there with out 1 year leases, I just do not know how you wing it so much.

Right now I am on a 3 month notice monthly lease, and I have a roommate that is planing on leaving in the spring. I have verbally committed to him to stay till march, it would be really sh*tty of me to leave him early and to also not give my landlord proper notice, especially since we work in a professional environment together.

So if i am going to be decent and honest, i will need to support my $800 Milwaukee, habit until march just to keep the peace.

Of course, it might be that the only way to win, is to just think of yourself. I would rather not live in such a world, and so often times I am willing to support the consequences of being decent to people and friends. It has tended to pay off for me in the long run.

</comment></comment><comment><USER15> Entertainment should come out of your own pockets. We just paid for food and housing out of our YC funds, in addition to miscellaneous expenses for incorporation and etc.

</comment><comment><USER11> > Unless you somehow lose intelligence and execution ability when you get > married and have kids, the structure doesn't make sense to me. It seems to > be geared towards single, foot-loose folks. As somebody who's put long hours > in on multiple technology projects, I don't feel that having a higher > overhead translates to being a worse risk.

Higher overhead may not make someone a worse risk, but it does make them more expensive. That has consequences.

Is your participation worth twice as much as that of someone with half the overhead? If not, why should YC pay for your overhead?

As someone once said, "The universe is not here to please you."

<comment><USER16> Ouch.

No it's not. But neither am I here to please the universe.

I'm just saying that it's not a lot of money. Perhaps it is to you. I can make that money as pocket change in a very short amount of time in my normal consulting business. If YC wants to pay 10 bucks for their team members, more power to them. Not my call. They've got their math to do and I've got mine. Even families have profit/loss statements and break-even points. Their rates have consequences. After all the universe is not here to please them either, right?

<comment><USER17> The solution seems pretty simple then.

Apply to YC and let's hypothetically assume you get in.

Spend 20% of your time making this ridiculous amount of money you can make doing consulting, the rest on your startup.

Problem solved. =) In all seriousness though, didn't Xobni have to do something like this for a while? (side consulting)

<comment><USER16> I agree. I think you hit the nail on the head.

But you can't do both at the same time. Good one-on-one consulting (or anything, for that matter) requires you to be 24/7 thinking of your client and their problems, not yours. Perhaps other guys are better at multi-tasking among completely different things. To me, you've got to pour your heart and soul into it, or it's not worth doing. That's just the OCD-kinda guy I am :)

My biggest problem isn't the savings, it's the team. I chose to live in a really rural area, which is great for kicking back after getting off a jet from somewhere, but really sucks wind when it comes to meeting and forming teams. I'm waiting for one of these YC groups to help us hermit schmucks out with some good team-formation software.

</comment></comment></comment></comment><comment><USER18> I'm glad im not married, have no kids, and can manage to survive on the minimum if I must. That said, a lot of people out there aren't so lucky, but ultimately it comes down to what you really want. If you want to continue working for a useless paycheck while the job requirements go up, then dont apply. If you would like to risk it all for the slim possibility that your idea/product might succeed, then by all means go for it. YC here I come!

</comment><comment><USER16> Living expenses are the reason I'm not applying. Unless you somehow lose intelligence and execution ability when you get married and have kids, the structure doesn't make sense to me. It seems to be geared towards single, foot-loose folks. As somebody who's put long hours in on multiple technology projects, I don't feel that having a higher overhead translates to being a worse risk. After all, what are we talking about here? The difference between 5 and 10K, or something like that? Being hungry is great. Having your wife call you every day to tell you the house is getting repossessed is counter- productive.

I agree that the knowledge and contacts are what you guys should be going for. Running on savings and forgetting about the money is the only strategy I can figure that works for us "high maintenance" entrepreneurs. If I apply, I'll just make sure I have money in savings to run for six months or so first.

<comment><USER14> If you think risking the delta between what YC offers vs. living expenses for a few months is worth not applying, you might (as a potential founder) want to reconsider your appetite for risk.

<comment><USER18> Amen to that.

</comment></comment></comment><comment><AUTHOR> ok, for now, it seems like the same debates that always rage when making a budget. you can either be honest with your self and try to spend less, or you can lie to yourself and not have enough.

I am strongly in the camp of planning for the reality of the situation. when it comes to entertainment, its less than 5% of the budget, i think this seems fair. That is 2 dates at the movies, 2 nights of drinks, or one SCCA race a month.

as for food, i might be able to get by on ramen, but i might also miss out on other opportunities if i was out and about.

also, unlike many of the teams applying we nearly have a product ready to role now, for us this is all about the networking and so our strategy is quite a bit different than the make it go strategy, for us this is about a life style shift, its about committing to the start up and if that means moving to silicon valley, this might be the kick in the pants to do it. From my stand point that means that the wives will most likely move too. They may not move at first, but i suspect they will want to move out there.

If you want to see a demo of our app in adobe air. you can find it at: <http://s3.derby.adobe.socialhelix.net/zcalendar.air>

Our goal is to start serving two customers by jan'08, which would be before we leave, and have a revenue potential of $1,000 - $20,000 a year depending on how well our app works.

You can see an early demo we did for one of our customers last year at <http://info.socialhelix.com>

Our biggest fear is how do we scale our application, do we have all of the kinks worked out, will the security model of the web keep shifting, etc...... How can we do this full time?

Just recently some of our team finished paying off their weddings and cars. I am just trying to figure out what it will cost us to go out there so we can make a decision on what our next move is.

</comment><comment><USER19> PG, have you thought of buying or leasing an entire apartment or condo complex and putting YC founders up in that? Besides economies of scale there would be efficiencies in putting together people with common goals.

There's a reason so many startups form in grad student housing.

<comment><USER20> Actually our plan is the opposite. To run YC with just 4 people we try to outsource everything. We don't even own our own servers. We'd never want to own apartment buildings.

The reason startups form in grad student housing is the grad students, not the housing.

<comment><USER19> The YC way seems to be working rather well.

I think grad student housing does help focus. You can get meal plan and not have to spend time going to the grocery store so much; you tend to interact much more with other people who are also focused on similar problems and random friends don't drop by and try to drag you off on some irrelevant adventure.

I have a problem with that sort of thing. I have to be a bit brutal when I'm really deep in something. The keeping-in-your-head essay really hit the nail for me.

</comment></comment></comment><comment><USER21> Of course, there's always the path to starting up without going to YC. You just might not be a good match for YC. Doesn't mean that you can't be successful, it just means you're going to have to find alternative routes to success.

</comment><comment><USER22> Learn to love Mr. Noddle.

</comment><comment><AUTHOR> So I thought I would update this post.

I am now living in the bay area and here are some hard numbers

our 3brdm is $3350 a month, and it fits 4 people snug but comfortably. So that is about $1200 a bedroom.

Internet is included, but you might expect to pay a bit more than $50 a month.

food well, eating at Mel's Dinner will run you $12 for a meal, and about $2.50 for a drink, and another $2 for tip, or about $17 for a dinner meal. Milk is between $4-$8 a gallon, and ground beef is $3 a pound. Cereal is about the same cost, and Starbuck's Coffee, is the same price in SF as it is in MKE or Chicago. Bagels at Noah's will run you about $2.50 for a bagel with cream cheese, and lunch will run about $8 a person without getting a drink.

You will be using a laundry mat, and it will cost you $2 a load to wash, and $1-$2 to dry them, so $3-4 in total at the brainwash laundry mat.

Beer is about $5-$6 in most of SF, but last night I was in the mission drinking micro brews for $3-4 a beer. On another occasion in north beach I paid $11 for a beer in a plastic cup.

The AMC theater on VanNess is $10.50 a person and matinées are $8.50 a person.

The bus will take you anywhere for $1.50 a trip, with a transfer that is good for ~2 hours+ depending on how the driver tears the transfer ticket. The BART is about $5 to Berkley, or SFO and a bit more if you want to go to Freemont or to Palo Alto, etc.. CalTrain is a bit more expensive and runs about $10 for a trip between Sunnyvale and like SF. Taking a cab between the Mission and say "the Richmond" will run about $10-20, and where I live its about $10.

Oh, parking is a bitch. Expect to pay $200-$400 a month more for rent or a separate parking space.

Living in SF you will also save money for heating and cooling costs, because the weather is so temperate.

There are at least 5-10 tech events a week, so you will have to choose wisely which ones you are going to go to and during the week its not pleasant getting between Sunnyvale, Palo Alto, and Mountain View and SF proper.

Living in the Mission is probably the cheapest and has the best transportation options for a tech entrepreneur, but pacific/laurel heights and the richmond have their advantages too. potrero hill is a great area but pricier than others.

if you are going to try to save on rent by being a bit farther out in the city, make sure you are near a bus line that runs regularly, at night many of the buses really reduce their schedule, but for example lines like the 38 will run fairly late with 7 min intervals, and then 20 min in the late night.

</comment><comment><USER23> $100 for entertainment is way too much ... YC's no fun ...

</comment><comment><USER24> wives...(!)</comment>


Infographic: The Everything Bubble Is Ready to Pop - <AUTHOR> http://www.mauldineconomics.com/editorial/infographic-the-everything-bubble-is-ready-to-pop <comment><USER2> A lot of valid criticism already about this "info graphic" (advert).

Also not proof of "a bubble", but more interesting to me are the four metrics presented graphically here [https://isthestockmarketgoingtocrash.com/](https://isthestockmarketgoingtocrash.com/)

And it's fun to watch those astronomically large debt numbers increasing in real time.

</comment><comment><USER3> Let's say everything written here is accurate, not cherry picked, and fully contextualized.

Even assuming all that, I'm not sure this site could ever convince me it is a 'problem', given the clear purpose of this site is to sell me something to fix that 'problem'.

That said, this infographic is missing a lot of context, and basically any sort of rigorous analysis.

</comment><comment><USER4> This infographic isn't very convincing. It shows a bunch of forms of valuation and debt have drastically increased in the last 7 years, but that doesn't alone mean it's a bubble. You need to establish that the rate of growth is the same type of thing that preceded other corrections. You need to show that the growth is irresponsible or problematic.

I believe we are in a bubble, because the valuations of a lot of companies defy all real logic, and for a number of other reasons I won't get into because it will just devolve into an argument about what I believe. But I still find this infographic lacking.

<comment><USER5> Please indulge? I feel there is a bubble, and I know why, but I'm entirely powerless to understand how can it possibly deflate? Panic might cause temporary flight to safety, but there has to be something drastic going on for the bargain-hunters to sit this one out and let it fall...

<comment><USER4> Were you not around for the first dot com bubble?

All it takes is the right high profile failure(s) and investors get spooked. They stop investing, which causes more collapses of companies that can find investment. They pull their money out, the selling undermines market confidence and things go kablooie. That's just one example of how it could happen based on how it did happen before, not a prediction of what will happen.

The valuations are based on confidence, if the confidence is reduced (not even entirely gone) it can start a downturn. That can go fast or slow.

<comment><USER6> but in the first .com bubble, companies lived and died only on their stock activity. these days, the winners have enough cash piled up that a recession would actually be welcome; it would allow them to buy companies and employees on the cheap, and many of their flimsy competitors would fold.

if you think Apple, Google and Facebook dominate now, wait until the next recession. can your company laugh off losing half it's market cap temporarily? they can

<comment><USER4> Apple, Google and Facebook are not the issue dude. All the companies that have yet to really be profitable are. Come on, that's obvious. Of course those companies which have cash on hand would be fine.

</comment></comment></comment></comment></comment><comment><USER7> QE money inflated asset prices and low interest rates mean bonds don't deliver good return so people buy stocks instead. And since we have the looming pension crisis the we will need a lot more money printing.

If everything is in a bubble, what will pop the bubble? An interesting take is the baby boomers hit max earning power at 55 and buy a second house, but then around 65 or 70 they realize the don't need either big house and they downsize (which will create downward pressure on housing) and they want to annuitize their assets (sell non-dividend paying stocks, etc)

<comment><USER8> > An interesting take is the baby boomers hit max earning power at 55 and buy > a second house, but then around 65 or 70 they realize the don't need either > big house and they downsize (which will create downward pressure on housing) > and they want to annuitize their assets (sell non-dividend paying stocks, > etc)

My initial assumption was that you'd see a glut of housing because of this (Boomers unloading real estate to downsize), pushing down home prices, helping first time home buyers. I've recently updated my assumption: Boomers are going to hold onto their property as long as possible as rentals. People are seeking out returns where ever they can, which is going to make this intergenerational housing problem drag on for at least another decade or two.

Housing supply metrics:

[https://www.usatoday.com/story/money/2017/10/23/heres-why- ho...](https://www.usatoday.com/story/money/2017/10/23/heres-why-house- hunting-so-frustrating-right-now/786034001/)

[https://fred.stlouisfed.org/series/MSACSR](https://fred.stlouisfed.org/series/MSACSR)

<comment><USER7> Good point, why sell your house when you can't buy anything that gives a good return? Better to rent it. So we're in the everything bubble and there is no pin in sight.

<comment><USER8> > So we're in the everything bubble and there is no pin in sight.

Indeed. The only solution is to either 1) hustle incredibly hard and hope for success/a liquidity event/etc or 2) walk away/opt out and find another economy where you can succeed.

I'm honestly _shocked_ that more first world countries with low birth rates below replacement rate don't appeal to US citizens to import them, train them, and have them join their society as valued workers (vs raking them over the coals in the US with student loans, a broken healthcare system, underpaid/non- existent jobs, and housing that might be forever out of their reach).

<comment><USER9> _> I'm honestly shocked that more first world countries with low birth rates below replacement rate don't appeal to US citizens to import them, train them, and have them join their society as valued workers (vs raking them over the coals in the US with student loans, a broken healthcare system, underpaid/non- existent jobs, and housing that might be forever out of their reach)._

Most higher education in Europe is free, even for international students. More and more countries are offering programs at Bachelor or Masters level taught in English.

What more of a red carpet welcome do Americans need?

<comment><USER8> > What more of a red carpet welcome do Americans need?

This is a great point, which I'd counter with: marketing. Ever notice how many locations around the world advertise to come bring your business there? Or holiday/vacation there? Same idea.

<comment><USER9> _> Ever notice how many locations around the world advertise to come bring your business there?_

France made a very public appeal to climate scientists. [1]

 _> Or holiday/vacation there?_

Anecdotal evidence I know, but almost all of my friends in Canada have or want to visit Amsterdam. But while they're enjoying pot brownies and the red light district, Amsterdam has many startups who will hire English speaking employees.

Or the people who really want to visit Oktoberfest. Munich also has many firms willing to hire English speakers with the right qualifications.

I will grant that perhaps Europe can do a better (more obvious) job of appealing to North Americans who want to immigrate. Personally I found it extremely easy to immigrate from Canada to Europe (no family ties to gain Eau citizenship), though I had already finished studying.

Another easy option that is often overlooked is the Working Holiday visa. It's quite easy to obtain and allows someone under 35 to work and travel between 6-12 months in many EU countries.

[1] www.businessinsider.com/emmanuel-macron-american-climate-scientists- france-2017-6

</comment></comment></comment></comment></comment></comment></comment><comment><USER10> the choosen metrics seem very suspicious... "number of cryptocurrencies" doesn't mean anything other than it's easy to create one. bitcoin market cap growing means there's probably a bubble, but 65 billions is too small to trigger a systemic crash by orders of magnitude. putting the two close one to a other is a really cheap trick to make you feel like the monetary system is in danger. Same for tesla, it doesn't make any sense other than to show that stock is overvalued.

To call something a bubble you need to identify a market that's "bubbling", a whole area or sector. otherwise you've only identified a few assets.

housing market is more sensible, but bubble on the housing market have been announced for the last thirty years, and prices kept climbing. everywhere. maybe it's not a symptom of a bubble, but just that there's more and more people, and less and less places they want to live in.

</comment><comment><USER11> What is supposed to be the relevance of stating that there are more cryptocurrencies than fiat currencies. It seems like stating that there are more cavalry troops than there are aircraft carriers. Both are true, but you have to be completely deluded to think that raw numbers equates to significance.

</comment><comment><USER12> Re: crypto, it's relatively easy to get into and there seem to be a lot of success stories, so yeah, a lot of people jump into those which boosts the price to bubble proportions. I'm fairly sure we're going to see a crash after new year's, once tax evasion periods have passed.

But when it comes to the stock market, I think there's a lot of people that have gotten on board with that in recent years; at least from my own perspective, regular boring old savings account have gone from 5% or more down to 0.5% or even negative interest in some cases; this is probably due to EU- wide laws to help economic growth in the less economically well-to-do countries, but still.

On the plus side of that equation however, is that mortgage interest rates are still very low. I bought a house this year and fixed the interest for 30 years, I'm sure it's going to go up again soon enough.

</comment><comment><USER13> I wrote [http://yuhongbao.blogspot.ca/2017/12/google-mozilla-and- debt...](http://yuhongbao.blogspot.ca/2017/12/google-mozilla-and-debt-based- economy.html) based on Twitter conversations with BrendanEich and others.

</comment><comment><USER14> This is one huge advertisement

<comment><USER15> If anyone has received the report, can you please post a quick tldr?</comment></comment>


Amazon reveals private Alexa voice data files - <AUTHOR> https://www.heise.de/amp/meldung/Amazon-reveals-private-voice-data-files-4256015.html <comment><USER2> Mods: The title should likely be changed back to the original ("Amazon reveals private Alexa voice data files").

The story here isn't that Echo stores recordings (they tell you they do, you can even listen to them) and it wasn't done "via GDPR" since that's a law, not a method of communication or platform.

<comment><USER3> Yes. Submitters: please follow the site guidelines and "use the original title, unless it is misleading or linkbait; don't editorialize." ([https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html))

(Submitted title was "Amazon Echo stores your voice recordings – sent other user's records via GDPR".)

</comment></comment><comment><USER4> Since it seems like quite a lot of people are surprised that Amazon stores these recordings...

PSA: you can go into the Alexa app and look at your Echo history and even listen to recordings of each interaction.

<comment><USER5> Google is the same - see your recordings here:

[https://myactivity.google.com/myactivity?restrict=vaa](https://myactivity.google.com/myactivity?restrict=vaa)

I'm not sure if it's codified in law, but big tech companies are moving towards 'if we store non-anonymized user data, it must be possible for the user to log in and see it themselves'. I think they do that so they can argue that storing and playing back the audio is a feature of the product, rather than something ancillary they happen to do which might not be in the users interest.

<comment><USER6> This used to be called "search history", and then "web history", and now "my activity". These controls for user data have been a best practice Google has been following for many years.

[https://policies.google.com/technologies/retention](https://policies.google.com/technologies/retention)

</comment></comment></comment><comment><USER7> Dupe. From the other day with 482 points, 403 comments: [https://news.ycombinator.com/item?id=18727020](https://news.ycombinator.com/item?id=18727020)

</comment><comment><USER8> Perfect example of how regulations designed to serve one purpose can in fact create just as many problems as it solves.

They meant to give consumers insight into their own data. Instead they created a process for getting previously locked up private data out of the company, which is prone to human error and / or intentional abuse. Nice.

<comment><USER9> But the problem was still there: poor control over stored recordings. It could have surfaced another way, like a law enforcement request for some suspect's recordings and being sent someone else's recordings.

So the regulation is not at fault here, _" they created a process for getting previously locked up private data out of the company, which is prone to human error and / or intentional abuse"_ is the real problem.

</comment></comment><comment><USER10> Anybody know how this compairs with Google's home product ?

I currently have both and need to make a decision on which route I am going to go.

<comment><USER11> Google Home stores everything after the hotword trigger until the light goes off (you stop speaking).

Each action has a card that explains what triggered it, what device, and what result was given, and you can listen to it. You can also turn recording storage off entirely, or delete by device or date ranges.

Honestly the only bad part is it's buried several layers down in the options and account activity, where most people don't go looking. If you do care though the privacy options and controls are pretty good.

<comment><USER12> This is the same with Alexa, with the exact same drawback that it's buried in the settings>History section.

It's actually a really cool feature; this entire fiasco could likely have been avoided if Amazon were to embrace the feature (data export and review) rather than treat it as something only nerds are interested in.

If you have data export by default (like Google's Data Takeout), then you don't need to build internal custom systems and manual processes that are only tested on GDPR requests. You've already built them for the default case. Handling GDPR requests is now user self-serve with a link to documentation explaining how to get their data.

</comment></comment></comment><comment><USER13> also good way to discover boyfriend of my wife

<comment><USER9> I you think she's talking about him at home or he's coming over when you're not home, wouldn't it be easier to go the traditional route of a hidden camera, or hiding and audio recorder in the bedroom?

Relying on her to say "Alexa, play sexy music for Johnny" seems unreliable. But if you really want to use the Echo to spy on your wife, you can just review the voice history yourself.

</comment></comment><comment><USER14> Amazon also has people listen to your voice recordings so they can manually label the outcome for future training data.

<comment><USER15> Everyone does this, training data needs labels.

<comment><USER14> Obviously I recognize that training data needs labels as that's why I said they did it. But I bet you ten bucks the average consumer does not expect humans to hear what they say to Alexa.</comment></comment></comment>


Drupal 8 is here - <AUTHOR> https://drupal.org/8 <comment><USER2> My favorite CMS for years, time to upgrade over the holiday season.

</comment><comment><USER3> w00t!</comment>


Oracle Data Marketplace - <AUTHOR> https://docs.oracle.com/en/cloud/saas/data-cloud/dsmkt/oracle-data-marketplace.html#GUID-5F2FD071-6C43-4804-902A-7026AD180EAB <comment><USER2> It's also hilariously inaccurate. I'm a freelance media buyer of sorts, and it think I works in either manufacturing or HR at a company of 500. The hilariously awful levels of inaccuracies this contains is the reason why the Facebook/Google advertising empires chug on. They have the data you need to target ads correctly.

Check for yourself (This link also deletes your data from them if you're so inclined.): [https://datacloudoptout.oracle.com/registry/](https://datacloudoptout.oracle.com/registry/)

<comment><USER3> Datalogix data (now part of Oracle Data Cloud) is what powers the creation of Facebook advertising segments. So I don't think its that inaccurate.

<comment><USER4> Datalogix is just one of several third parties that were allowed to push matching data to Facebook for the creation of segments (partner categories), as well as Experian, Epsilon, Acxiom, etc.

The way it works is not very different from custom audiences, but they are syndicated to every FB marketing api user instead of private to your account and there's a revshare with FB based on usage.

So the matching has to be based on one (or more) of the keys listed here: [https://developers.facebook.com/docs/marketing- api/audiences...](https://developers.facebook.com/docs/marketing- api/audiences-api) plus a few extra ones like IDFA/IDFP, probably cookie matching, etc.

It's not hard to create a massive audience on Facebook if you're matching on public data like postcodes, but ultimately your understanding of that audience is based on how accurate your data is. Whether you believe that data companies have accurate data or not, it has nothing to do with their segment size on FB.

PS. the partner categories program has actually been discontinued as of April 2018, so these third parties will no longer be able to syndicate their data to all FB ads users in the future.

<comment><USER5> > PS. the partner categories program has actually been discontinued as of > April 2018, so these third parties will no longer be able to syndicate their > data to all FB ads users in the future.

One thing to point out is that it's true that the Partner program itself has been discontinued, which gets rid of all the 3rd party targeting categories within the self-service interface using Facebook-managed data integrations with those firms.

 _But_ the ToS[1] rewrite effective 5/25/18 goes out of it's way to ensure that they don't outright prohibit the use of third party data. They just effectively decentralized the usage of it and require ad buyers and the data brokers to have a direct relationship now, and leverage the custom audience functionality you mentioned to do the targeting. And in so doing allowed themselves to become willfully blind to the usage of 3rd party data plus shifting liability onto the advertiser for usage of it.

[1] [https://www.facebook.com/ads/manage/customaudiences/tos](https://www.facebook.com/ads/manage/customaudiences/tos)

</comment></comment></comment></comment><comment><AUTHOR> The Oracle Data Marketplace is the world's largest third-party data marketplace and the standard for open and transparent audience data trading. ... data providers offer more than 30,000 data attributes to power your branding ... actionable audience data on more than 300 million users.

That's over 80% of the entire US internet population at your fingertips. ... a range of data ... some of which are exclusive and not available anywhere else. ... Eighty percent of the top 20 ad networks, portals, trading desks, and creative optimizers leverage data from the Oracle Data Marketplace platform to run high-performance ad campaigns.

Equifax? They were kids.

</comment><comment><USER6> > Users who have demonstrated intent through ... searches

I know marketing always makes exaggerated claims, but to claim that a search implies _intent_ is such blatant nonsense. At least later on they make claims about "interest", which at least _could_ be true, for very broad definitions of "interest".

(As auslander already said, aggregating personal data into huge databases like this invites Equifax-like data theft.)

</comment><comment><USER7> I've probably spent $50K on media over the past 9 years on media that was targeted using BlueKai data, and worked on the publisher side where their pixel fired billions of times. I remember the early days when its founder Omar was running around the globe selling this data as "Stamps" because they were charging so much for it reaching a single person cost as much as direct mail, printed and delivered.

AFAIK, a web request is sent from a consumer's browser with some form of identity, ie cookie ID, fingerprint etc. The hash of it gets sent to BlueKai when there is a bid request, and it then queries for keys associated with that hashed ID. The bidder (a service that makes purchase decisions on bid requests) returns a value for that data, and if its the highest value among all advertisers bidding on that slot the ad is shown.

The advertiser never sees any of the underlying data. They are able to see how many bids they won against the pool of data. Say you wanted to sell some organic sausages via banner ads- you would purchase impressions from a publisher, ad network, or agency, and pay an extra $2 CPM (mind you most media clears around $1) to BlueKai to show ads against an Organic Food Buyers audience. That audience is most likely derived from grocery store club card data which include things like recent purchases, brand preferences and more. I'm amazed that more people don't care about that form of data collection...

How often do you have to buy organic products to be entered into the targeting pool? 50% of your shopping cart, or 1 item ever? Literally nobody except for BlueKai knows. There is no oversight, and as such the data is almost certainly low quality.

I came to the conclusion after all of that spend against at least 100 different audiences that its almost universally more efficient to buy cheaper banner ads in bulk. It was often the case that media targeted with 3rd party data would perform twice as well as untargeted media, but for 5 times the cost.

Still, I think the concept of audiences, not raw pii, has been a decent compromise for the sake of privacy, and the net effect of all of it is that some marketing budgets got blown on ads that could have been otherwise targeted.

That is one hell of a security risk though to have all of that data sitting out there.

I got so fed up with the inequity of digital marketing I've done my best to move myself away from it. If I were building an advertising business today, the consumer would be an equal partner in all data transactions, for two reasons: 1. Fairness 2. Data quality.

<comment><AUTHOR> > move myself away from it

I stop approaches from ad-tech companies myself.

> the consumer would be an equal partner in all data transactions, for two > reasons: 1. Fairness 2. Data quality

why would a consumer be interested in cooperating? No one likes ads, right? :) The businesses milk data out any way they could, and share it. People defend themselves and share the knowledge. Its how it is now, and looks like will be in future.

Browser companies may be on either side, today I'd stick with Firefox and Safari.

<comment><USER8> > why would a consumer be interested in cooperating? No one likes ads, right?

Let's assume I can't block ads or choose not to block them to support site/service I'm using.

If that was the case I'd rather see relevant ads as long as they were targeted based on a subset of information about myself I've decided to provide, to the parties of my choosing and with usage restricted to that single purpose only.

<comment><AUTHOR> Yeah, but you talking about supporting a website/service. I replied to the 'If I were building an advertising business today ...'

</comment></comment></comment></comment><comment><USER9> This is where the scum of the adtech industry lies. While Google and FB collect a lot of user data, they only keep it to themselves and use the data BOTH for better advertiser targeting as well as for customizing user experiences.

Oracle BlueKai, which was an acquisition, collects data from various sites users visits (via cookies and their SDK) and explicitly sells that data to other ad networks. Ad networks, outside a few of them, don't have a lot of data to target and thus rely on 3rd party data brokers such as BlueKai to fulfill this need. BlueKai also have tie ups directly with the advertisers, especially brand advertisers such as Unilever etc., where these advertisers require the ad networks who are advertising for them to use BlueKai customer segments (which they created using user data) for targeting purposes. And all of this is completely invisible to the user.

<comment><USER10> Why specifically is it worse than FB? If all those ad networks merged into BlueKai aka Oracle, so the data was kept "internal", would that be better?

<comment><AUTHOR> Its unclear. I guess his idea was that FB collects only data you put into FB, while BlueKai collects data from many trackers embedded into majority of websites.

Unclear because Google and FB have its hand (trackers like Like button and pixels) in many websites as well. In uBlock Origin, enable advanced mode and will see tens of tracking domains on pages like newyorker.com online stores et cetera.

<comment><USER11> Facebook has share buttons on all the world's media. Each one of those views is almost certainly recorded by Facebook, even if you don't have an account. The only way to stop it is put a DNS block ( hosts entry ) on your machines. If you think Facebook records less data than bluekai, I think that is very very unlikely.

</comment></comment></comment></comment><comment><AUTHOR> ... still digging ... FAQ:

\- On average, how many categories does a unique user belong to?

\- We see 750 million unique users per month with an average of 10-15 attributes per user.

\- How long after a user qualifies for a category will that category last in their cookie?

\- Categories are stored per user for 90 days. This is the legally allowed time limit. It is also a rolling 90 day period ... if the user is back online on day 2 then the activity counter resets.

It was last addition to my defences - private browsing tabs only, closing tab clears cookies and storages.

<comment><USER12> Is this a new discovery for you? Third-party cookie data is everywhere, and usually very messy if not completely inaccurate.

<comment><AUTHOR> No, but what about first-party cookies? And all sorts of local storages? I don't need them too.

<comment><USER12> 1st-party data is different from 3rd-party data, regardless of how it's physically stored.

I'm not sure what your question is though, are you asking how to protect yourself? It seems you already have it covered if you delete cookies.

<comment><USER11> Facebook's collection system is through share buttons. Those share buttons do cookie share because the domain is Facebook, but the technology is certainly there to track by ip address and device fingerprint, even if you are not a fb user, and a single fb login redropps the cookies and allows association of your internet media behavior with your FB account and behavior.

</comment></comment></comment></comment></comment><comment><AUTHOR> HN mods changed title. Original title was:

Oracle BlueKai, audience data on 300+ million users, over 80% of the US

<comment><USER13> The new title seems inaccurate, why did MODS change the title to be vague and less specific? Seems shady practice to me.

<comment><USER14> The guidelines say to use the original title of the link, unless that title is misleading or click bait. The original title was expressing an opinion.

<comment><AUTHOR> The original is direct quotes from the top of the page.

What opinion?

</comment></comment></comment></comment><comment><USER15> Is this new? Because Oracle couldn't possibly choose a worse release date directly following GDPR-day/-week, underlining its relevance. But maybe they're on to something because they're able to sell data when it's getting scarce?

<comment><USER16> Not new at all. Not sure why this was posted today though. Here is a similar marketplace / solution from one of their biggest competitors, Neustar [https://www.marketing.neustar/identity-data-management- platf...](https://www.marketing.neustar/identity-data-management-platform)

Adobe is also very much in this space.

</comment></comment><comment><USER17> Where can I get a link that does what I'm reading in the comments. The link sends me to a doc page / what link for them to share their guesses about me please?</comment>


Analysis of airline performance - <AUTHOR> http://blog.revolution-computing.com/2009/09/analysis-of-airline-performance.html

<comment><AUTHOR> Here's the original poster from the SAS institute team - [http://stat- computing.org/dataexpo/2009/posters/wicklin-alli...](http://stat- computing.org/dataexpo/2009/posters/wicklin-allison.pdf)</comment>


Ask HN: How to communicate better with people whose first language isn't English - <AUTHOR> I've recently experienced several situations where upper management thought that they had clearly communicated their requirements for a task, but the person who completed the task didn't understand what was expected of them.
With that in mind, do you have any experiences or advice for improving clear communication with and between people who speak English as a second language? What policies and standards worked best, and what failed?
How do you encourage people to ask for clarification if they don't understand? Too often I've seen ESL people very reluctant to ask questions, whether it's in person, chat, or email. <comment><USER2> Written communication is better for specs on work to be done. It can be enhanced with diagrams, images, mockups, etc. Bulleted lists are effective for listing out acceptance criteria and details they need to be aware of.

But there is also a human side to it. When I led offshore teams (from India and Mexico), I worked hard to get to know them, which included open discussion of our communication. Sometimes it was just talking to them often enough that we knew each other's accents. Especially for coworkers in India, we all knew English well, but spoke it quite differently. They had a hard time understanding my American accent, so I'd deliberately slow down when talking. I also would stop the conversation and make sure that if words were not understood, we had a culture of just saying so and trying again.

We still had times when we struggled to understand each other, but they trusted that I didn't let that get in the way of how we worked together. They knew that communication problems were something we could work at, and laugh at, and was not going to be looked at as a failure.

It isn't all that different than any other communication concerns within a team - the more you build trust, the better communication gets.

<comment><AUTHOR> That's good advice, thank you!

Did you find people receptive to slowing down and repeating things, or were people embarrassed and wanted to ignore difficulties?

If you have time, may I ask for specific situations where communication was poor, and what happened to improve it? (Sorry to sound like an interview!)

<comment><USER2> It did take time to build enough trust with new team members for them to be comfortable cutting me off to ask me to repeat things. I had to take the lead for new team members, stopping them and politely asking them to repeat something, and I tended to apologize for not understanding. I'd let them know early on that I knew there would be times that we didn't hear each other well, that it is completely OK for that to happen, and I wanted them to stop me and ask me to try again. And I'd do the same when I did not understand them.

I don't have specific situations, because it was never a big thing. We'd just be talking, and someone would say, "Hang on, I didn't get that, try again?" Not any different than remote calls when someone's audio goes out and you just let them know that you didn't hear them.

</comment></comment></comment><comment><USER3> From what I've been told by foreign contractors, verbal communication should be avoided when it comes to important details because it can be harder to parse different accents and vocabularies. Email is probably best. Can be referred to as many times as required, and can easily be put into an online translator.

<comment><USER4> [Hi from Argentina!]

I agree. Try to send the requirements in a written form like email.

[I don't discard the possibility that the language barrier is used as an excuse by the contractor or by the manager.]

</comment></comment><comment><USER5> I think encouraging people to ask for clarification is super important.

One thing that's helped me receive more clarifying questions is adding pauses while talking. This gives an opportunity for someone to ask questions without interrupting.

</comment><comment><USER6> As others have said, write it down

When talking, speak slowly with clear enunciation</comment>


NY Insurers Can Evaluate Social Media Use If They Can Prove Why It's Needed - <AUTHOR> https://www.wsj.com/articles/new-york-insurers-can-evaluate-your-social-media-useif-they-can-prove-why-its-needed-11548856802 <comment><USER2> I would like to see the increasing granulization of insurance put to an end, because otherwise it won't stop. There should be one pool, everyone's in it, we spread risk, and the whole damn thing is simpler and easier to regulate. Sure, Mr Base Jumper is going to raise your premiums when he breaks his leg, so what. Maybe he doesn't drink and so has less health affects than you. Or you'll raise his premiums when it turns out you carried a gene for disease-X and got it at 45.

What we have now is an ad-hoc nonsensible set of what you can and can't risk profile on, and every time we learn some new way to risk profile then we'll have to decide again if we want to regulate it. On top of that this just increases the risk of insurance companies getting it wrong and needing to be bailed out. I'd rather we had a conservative, boring and simple insurance industry and just get off the treadmill.

<comment><USER3> > There should be one pool, everyone's in it, we spread risk, and the whole > damn thing is simpler and easier to regulate.

Call me naive, but I also always thought that's kind of the point of it: spreading the risk, or solidarity to use a less fashionable term. People can be very sick through no fault of their own, and those who are healthier should look out for them. If you calculate that "fair share" according to exactly how sick people are, then what's the point, wouldn't that just be a needless middleman that gets paid upfront?

<comment><USER4> The idea is correctly price the consequences of lifestyle choices.

Cautious Prius owners probably wouldn't be happy sharing equally in a risk pool with enthusiastic drivers of sports coupes. If insurance rates were standardized, there'd be a lot more sports cars out there, until the Prius constituency got them banned altogether. Charging different insurance rates gives all parties what they want at a cost that creates appropriate incentives.

You might be able to say something similar about health insurance and sedentary smokers vs. runners. If an extra hour of work is less economically valuable than a gym visit, the incentive structure should reflect that.

I agree that insurance rates should not be affected by luck. One of the reasons the preexisting condition part of the ACA is important.

<comment><USER5> 1\. It's hard to disentangle lifestyle choices from genetic ones. If someone is born with an insatiable appetite that means it's 10x harder for them to stay at a healthy weight compared to someone else is that a lifestyle choice of genetics?

2\. Being healthy already has quality of life consequences. And the extra cost to insure those lifestyle choices are probably pretty low, and most of that cost is just shifting it from primary insurance to medicare. Because everyone dies of something, and a lot of times it's really expensive. Insurers care a lot about whether that happens at 64 or 66. But as a society it doesn't really matter, and the insured bears most of the impact.

<comment><USER6> In support of your comment, when talking about weight it's common to talk about "fast metabolisms" keeping people skinny. Studies have shown that metabolisms don't vary much at all. What does vary is how hunger signals work with different people. The people who complain about being too skinny and can't gain weight simply find it difficult to eat enough to gain weight. Their body tells them they're not hungry. If you force feed them they'll still gain weight.

I'm one of those people who gain weight easily due to always wanting to eat. I was morbidly obese at 19 despite coming from a household of healthy eating. I'm now 33 and have maintained a healthy weight for years but it isn't fun. For me to maintain a constant, healthy weight I need to be hungry. Despite 5+ years of maintenance, I am still hungry constantly.

</comment></comment></comment></comment></comment><comment><USER7> The only reason this works now is that people behave naturally (or at least natural enough) when posting on social media. If social media profiling becomes more prevalent (like this article suggests), everyone is going to be much more careful when it comes to what they're posting. All their real activity is going to be driven into pseudonymous (or at least private channels) that can't be easily monitored, and their "real" accounts is going to be so carefully curated that the signal to noise ratio is abysmally low. Or at least, this is my hope. I could be overly optimistic, and people will continue posting their true selves against their own interest.

<comment><USER8> People are pretty dumb.

There's another side to this though, why should a low risk person be in the same risk tranche as somebody who has lots of guns in plain sight, is out drinking 3 nights a week or has high-risk hobbies?

Life insurers have pretty transparent interests. A middle aged guy with guns and heavy alcohol use is a more expensive risk for death. Right now they only use age, occupation, specific medical and smoking status.

<comment><USER9> Insurance is weird.

It operates by taking events that are unpredictable individually and grouping them together such that they are predictable in aggregate. The larger the groups are, the better this works.

At the same time, there's a competitive advantage to using smaller groups, if you can reliably discriminate. Except that once you take it too far, you approach the individual case again and insurance ceases to have a point.

A life insurance company that buckets everyone together will lose out to one that divides people up into groups based on their risk. That one in turn would lose out to a company that has an accurate death clock they can use on every customer to provide perfectly individualized coverage. And yet the death clock company won't be able to sell any insurance at all, unless they can keep it a total secret, because people could accomplish the same thing on their own.

<comment><USER10> I think you're missing something important in this.

If you make the fundamental assumption that what happens to people has a level of randomness to it (don't think we'll ever be able to escape this assumption) then insurance always will have a point even if your bucket size is the size of the individual.

This is because insurers provide a hedge against a certain amount of negative randomness over a time peroid. They calculate the risk on a policy as if they provided it to you a million times and then base their pricing on that. Since they do this across a body of customers the pooling system still functions, it just is more accurately priced.

So as long as there is a risk instability over time they eill have the ability to provide a service worth paying for.

<comment><USER9> You're right that there's always some utility as long as there's randomness, but that utility decreases the more accurate your predictions get.

Rather than a magical death clock, let's say that actuarial science gets to the point where it can figure out an individual's life expectancy ±5 years with a 90% hit rate. The possibility of being hit by a bus or having some unknown wild-card disease means you could die much younger, and you might beat the odds and die much older, but 90% of people die within their predicted range.

Now, Bob goes to sign up for life insurance and he's predicted to die at age 50. He really wants this insurance to ensure his family can survive without him, but the premiums are completely outrageous, so he passes. Alice is predicted to die at age 100 so her premiums are cheap, but she sees little point in it, so doesn't sign up.

In this world, I see a lot of accidental death insurance being sold, but almost no general life insurance. Some people would buy it, as it would have some utility, but not a whole lot.

<comment><USER8> Bob needs to participate in a group policy.

</comment></comment></comment></comment></comment></comment><comment><USER11> A bunch of people have pointed out that, for all the criticism we're leveling at China's nightmarish social credit system, we're well on our way to building the exact same thing but with corporations in charge. And while you can talk about how this is _totally_ different because the corporations can't put you in jail, that really doesn't matter at all if I have to constantly self-censor all my words and thoughts or lose my health insurance. That's an equally shitty outcome.

<comment><USER12> True, I never show my real name in any public platform unless it's my "official" one, which is whitewashed. I go through pseudonymous accounts every 2 years for my privacy. We already do have social credit, it's called your credit score. Self-deception is a powerful force. Everybody rise up to the polls! Oh wait, it's both sides that are abetting this crap. Just vote with the closest to freedom (including a social safety net) and don't worry, cause you can't control it.

<comment><USER13> Unless I'm mistaken, your credit score is your financial credit rating, not your social. It isn't affected if you get a ticket jaywalking, or acting like a jerk.

There are interconnections between the two, but there are pretty clear distinctions.

<comment><USER14> A matter of time, I guess. Your "insurance score", as seen by insurance companies, provides information on how likely you're to be able to pay back a long-term loan, or even to be alive long enough to pay it back. And "insurance score" is based on increasingly deep insight into your lifestyle. It's only a matter of time before someone will have the bright idea to use "insurance score" as an input to credit score - and then, suddenly, you have social credit (sans the politics part).

</comment></comment></comment></comment><comment><USER15> It's quite hard to avoid "disguised and illegal race-based underwriting", so expect some lawsuits soon. Even something as simple as a zip code is too biased, so an attorney shut it down in a project I was working on.

</comment><comment><USER16> NY has some odd laws. On the one hand they are progressive (energy, human health), on the other hand they allow some very neo-liberal things to happen like where debt collectors can continue suing you, if your bank has one branch in NY, and now this nonsense.

<comment><USER17> It's almost as if NY has a huge financial industry that lobbies an otherwise liberal government.

<comment><USER16> Yet they overcome the auto industry lobby's efforts and many others'...

<comment><USER18> The financial industry being in NY means actual voters. While a company cannot force their employees to vote in any particular way, they can make their employees aware that some politician did something... They can't actually say raises and bonuses were affected but if you read between the lines you can see the relation.

The auto industry isn't large in NY. When enough voters in NY decide the auto industry is doing something bad, no amount of lobbing from the industry will prevent the laws they don't like.

Politicians need votes. Money is not nearly as powerful as a voter who is informed.

</comment></comment></comment></comment><comment><USER19> Not particularly surprising after the recent NY bill which will require gun owners to share their social media logins and search engine history with the state. Sometimes the slope really is slippery.

<comment><USER12> Then spying, tracking, censorship of "offensive" or "unpatriotic" words depending on your political side for little gain of the little people. Of course, the state won't do anything to actually reduce mass shooting, they profit of fear and hate. History doesn't repeat, it is remixed.

<comment><USER20> Mass shootings are vanishingly rare. Your odds of being in one are microscopic.

<comment><USER8> Risk assessment fundamentally about probability and impact.

We took measures to get serious about drunk driving because it increased the probability of car accidents, especially at night,and was killing people.

On the flip side, we mandate tamper evident seals in medicine because the rare event of tampering hurt a small number of people, but undermined the citizens faith in the institutions they depend on.

The mass shooting issue is like the pill bottle problem. It's a serious matter with an obvious solution. At the end of the day, it's a problem unleashed by marketing — as hunting and shooting sport participation evaporated, gunmakers were left with a problem. Now we have batshit crazy people with political power, which is a genie that will be difficult to put back in the bottle.

<comment><USER20> > It's a serious matter with an obvious solution.

This is a bold statement. Why do you think the solutions are obvious? Do you believe this is a subject without nuance?

<comment><USER8> No gun, no bullet, no shooting.

Look at the gun death rates in the northeast compared to the south.

</comment></comment></comment></comment></comment></comment><comment><USER21> "Hey, why did my rate go up?"

"I'm sorry, we cannot offer you a CleanSocial(TM) discount because your social media footprint is insufficient to properly inform us of your risk profile"

</comment><comment><USER22> I don't imagine this will go over very well with most people.

However, headlines like this are a good reminder to practice good social media hygiene; I've been careful about what I post and have a habit of deleting old Facebook posts from my teenage days.

<comment><USER23> That reminds me I'd like to erase my HN comments over a few years old.. oh wait.

(Seriously, can we please make it easier to remove comments/accounts from non- archive sites? There's not a single person on earth that can survive a Willie Horton style campaign where everything you say is examined without context - it's an abdication of responsibility on those running a community)

<comment><USER24> I wonder how compliant HN is with Article 17 of the GDPR[1]. It's the "right to erasure".

[1] [https://gdpr-info.eu/art-17-gdpr/](https://gdpr-info.eu/art-17-gdpr/)

<comment><USER23> Personal data is not the same as comments posted on a public forum, but its not to deter the most determined actors - its more to indicate that I no longer want to have any association with the comments.

</comment></comment></comment></comment><comment><USER25> [https://outline.com/c82rJ3](https://outline.com/c82rJ3)

<comment><USER26> Hadn't heard of outline before -- very cool. Thanks for sharing this link.

</comment></comment><comment><USER27> Looks like it's time to either fudge all my social media with unicorns and rainbows or start operating purely behind pseudonyms?

</comment><comment><USER28> When social media use and friends you have there affects the price of the services, it's already a type of social credit.

</comment><comment><USER29> Ah, yes, this is when the _real_ dystopia begins

</comment><comment><USER30> Joke's on them. They'll be lucky to uncover a handful of uncorroberated shadow profiles and spam catcher email dead ends that flout "real name" policies.

Meanwhile, I scarcely have a need for insurance. Come up and get me.

<comment><USER31> > Meanwhile, I scarcely have a need for insurance. Come up and get me.

I don't live in the US but I have 5 insurance policies - home insurance (mandated by my bank for my mortgage), car insurance (mandated by having a car), health insurance (optional but less so if you live in the US), Travel insurance (traveling for work mandates I'm covered by a policy) and life assurance (the only actual optional one) - should I have to provide all of these groups my social media details even though I don't "want" to have them?

Further, whos to say they decide that by me not participating in "normal" activities online, that they won't decline or offer me a substantially worse policy?</comment></comment>


Tensorflow 2.0 AMD Support - <AUTHOR> https://github.com/ROCmSoftwarePlatform/tensorflow-upstream/issues/362 <comment><USER2> We've been experimenting with TF 1.14 on a Radeon VII. When you manage to properly install it, it does work at about 90% the speed of an NVIDIA RTX 2080 Ti, but is much cheaper, which can be a good trade-off.

However, the tooling and installation procedure is still rough around the edges and the documentation is lacking.

Last time I tried to upgrade the rocm things and the drivers, and bumped up against an issue that one of the packages tried to write to /data/jenkins_workspace during installation. We have /data as an NFS mount and is not writable even for the root user, so the installation broke down. Worked around it, but there are other issues as well. The order of packages you install can matter as well, I had to purge packages and reinstall them a few times. It worked with TF 1.14.1 but not with 1.14.0 if I recall correctly. The first few iterations are very slow, it seems it needs to find the optimal conv implementation for all kernel sizes separately.

Also, the card does not have anything like NVIDIA's tensor cores for half- precision (float16/FP16) computation. It can do FP16 but won't speed up and delivers worse results (in terms of prediction accuracy) than NVIDIA cards running FP16. There is no prediction accuracy difference in FP32.

And some other small issues here and there. So be warned and test it yourself on one card before investing in big multi-GPU machines.

But I do wish AMD entering the field will benefit us through increased competition in the long run. But at the moment ROCm seems like just a side project for a small team in AMD and they can't yet deliver the streamlined experience we're used to from CUDA.

<comment><USER3> Interesting. What benchmark are you running? Getting it within 10% on such drastically different hardware looks wrong to me. And sounds like GPU is not a bottleneck in that particular case. Could it be that the system is CPU bound (with some horrid thread contention in TensorFlow or something)?

<comment><USER2> LambdaLabs' benchmark: [https://github.com/lambdal/lambda-tensorflow- benchmark](https://github.com/lambdal/lambda-tensorflow-benchmark)

It's certainly around 10-15% and it's not CPU bound, as the benchmark uses synthetic images without the need for data loading and preprocessing.

I cannot rerun them now for precise numbers because my ROCm packages are broken and I don't have the spare hours to fix things. Surely it can be fixed but needs fiddling.

<comment><USER3> You can still be CPU bound, even if the benchmark is using synthetic images. A quick check could be to look if your 2080 Ti is 100% utilized (with nvidia- smi). And calculate percentage of theoretical peak FP32 throughput that you are getting for both cards.

It is just that "more normal" result that you would expect to get, would be a drastic differences in performance. Note, Cvikli comment on "May 12" on that GitHub page:

* _The result with RNN networks on 1 Radeon VII and 1080ti was close to the same_

* _Comparing convolutional performance the 4AMD and 4Nvidia, difference got really huge because of cuDNN for Nvidia cards. We can get more than 10x performance from the 1080Ti than the Radeon VII card. We find this difference in speed a little too big at image recognition cuDNN, I can 't believe that this should happen and the hardware shouldn't be able to achieve the same._

Note also in that comment, it looks like 4xGPU system just didn't work.

It is hard to fully utilize GPU resources. Getting same result normally means that the same bottleneck is being hit. Algorithmic differences could result in drastic performance differences (10x). And as a final point of small things making a difference, note it is 1080Ti in that comment.

<comment><USER2> I fixed the installation and ran a benchmark again: [https://news.ycombinator.com/item?id=21666411](https://news.ycombinator.com/item?id=21666411)

The benchmark is not CPU bound, as evidenced by looking at rocm-smi, nvidia- smi (GPU usage at 100%) and htop (low CPU usage).

<comment><USER3> Nice. Your results could be more reproducible, if you'd include CuDNN version. Turing architecture support (and optimizations specific to DNN training) are still relatively recent and there are differences in performance between the versions [1]. Multiplying matrices is kinda tricky ;) [2]

[1] [https://developer.nvidia.com/cudnn](https://developer.nvidia.com/cudnn) . [2] [https://scholar.google.com/scholar?as_ylo=2019&q=nvidia+gemm](https://scholar.google.com/scholar?as_ylo=2019&q=nvidia+gemm)

<comment><USER2> Cannot edit that comment any more, but it's driver version 430, CUDA 10.1, CuDNN 7.5.

</comment></comment></comment></comment></comment></comment></comment><comment><AUTHOR> I posted this because I have noticed many people aren't aware TensorFlow can run on AMD gpus. When informed, a common response is how _well_ does it work?

This GitHub issue provides answers to both those questions with respect to the latest major release of TensorFlow. (Admittedly just a single benchmark data point, but better than nothing.)

<comment><USER4> That's because it really doesn't work on AMD gpus (at least on Linux.). Sure if you spent a lot of time setting up the rocm driver you might get lucky but it's alpha quality at best and in my case (and the case of several people in this thread and the GitHub one linked) it doesn't work at all.

</comment></comment><comment><USER5> My tf or pytorch code usually uses custom CUDA extensions, so sadly it will never be plug and play on AMD gpus for me. I hate that scientific computing and ML settled on a (admittedly superior) proprietary API.

<comment><USER6> Google is working on integrating CUDA directly into TensorFlow with the end goal being that you can use any GPU to do your computation. Basically a leapfrog over the inherently closed system that Nvidia has tried to implement.

I'm going to be glad when that finally happens. It's not healthy for machine learning to be so dependent on a single provider like that.

<comment><USER7> It's going to be oracle v Google all over again.

<comment><USER8> Oracle v Google is still working its way through the court system.

<comment><USER9> by the time Google v NVidia will hit the highest courts, those will be stacked with AI overlords. I wonder what they'd think about being stuck in a proprietary API. :D

</comment></comment></comment></comment></comment><comment><USER10> Google implemented AMD for Stadia. As my understanding this is part of their Cloud infrastructure: will be interesting to see if they start supporting AMD in their cloud compute engine instances. If this happens, we should see a price difference and "probably" a push from their customer base to also support AMD in their Deep Learning VM. There is a company out there called LogicalClocks which already integrated AMD and TensorFlow

<comment><USER11> Yes, we added support in our Resource Manager (a fork of YARN) for AMD GPUs and ROCm. It works fine with TensorFlow, and our benchmarks should at ROCm 2.10, the Radeon VII's performance for RESNET-50 is just a few percent lower than the 2080Ti. If you install our open-source platform, Hopsworks, using TensorFlow on Nvidia or AMD is exactly the same experience. We handle the complexity of installing/configuring drivers/libraries.

</comment></comment><comment><USER12> Of course many HN readers know that TF has been available for AMD for a while, but I still think that this Github thread is newsworthy, specifically this comment[1]:

> The RADEON VII's performance is crazy with tensorflow 2.0a. In our tests, we > reached close to the same speed like our 2080ti(about 10-15% less)! But the > Radeon VII has more memory which was a bottleneck in our case. On this price > this videocard has the best value to do machine learning we think that in > our company!

The comment above echoes the findings of HN user bonoboTP, see comment currently at the top of this thread[2]

[1] comment link on Github: [https://github.com/ROCmSoftwarePlatform/tensorflow- upstream/...](https://github.com/ROCmSoftwarePlatform/tensorflow- upstream/issues/362#issuecomment-479366031)

[2] comment link on HN: [https://news.ycombinator.com/item?id=21657173](https://news.ycombinator.com/item?id=21657173)

</comment><comment><USER13> With nvidea not supporting MacOS beyond the current version, and Google pushing Swift for Tensorflow, I assume both Apple and Google have an increasing incentive to improve the adaptation of AMD GPUs for deep learing.

<comment><USER14> Correct me if I'm wrong but I believe one of the advantages of macOS is that the GPU can use the system ram for deep learning. Which makes it possible to use cheaper GPUs. One of the reasons I'm looking forward to better AMD Tensorflow support.

<comment><USER15> Oh really? As an outsider, that's super interesting to know. Why can't system RAM be used in Linux/Windows, or why is it able to be on macOS?

<comment><USER16> You can do that with OpenCL or CUDA on any platform, it's just ridiculously slow because of the much lower bandwidth so nobody does it. GPUs can access their memory (GDDR or HBM) at 300-800 GB/s, but system memory at only 16 GB/s.

</comment></comment></comment></comment><comment><USER17> Does anyone know if AMD intend to bring ROCm to Navi? Because I dont see any Vega successor on their Roadmap, and Navi doesn't seems to be focus on GPU Compute.

<comment><USER18> Phoronix states that Arcturus is the next compute acceleration chipset after Vega20, and thus Navi support may be unofficial if it does work:

[https://www.phoronix.com/scan.php?page=news_item&px=Radeon-R...](https://www.phoronix.com/scan.php?page=news_item&px=Radeon- ROCm-2.9-Released)

<comment><USER17> Thank You. Wasn't aware of Arcturus, I have always thought GPU Compute and GPU Gaming focus are at odds with each other and the trade off do not make any sense. So I am glad after so many years we have GPU specifically for Compute and GPU for gaming.

</comment></comment></comment><comment><USER19> I've checked the prices and it seems a XFX branded Radeon VII with 16GB RAM can be had for similar price as Nvidia RTX 2070 super with 8GB. This is very interesting.

I hope someone publishes benchmarks comparing those cards in training most frequently used network architectures. I would be heading to buy one of those Radeon cards myself if not for their lack of performance with conv networks.

Also, current version of Nvidia RTX cards with tensorflow supports automatic mixed precision training which can significantly speed up training of some networks.

I think we're still quite far from having a serious competitor to nvidia in general purpose machine learning.

<comment><USER2> > I hope someone publishes benchmarks comparing those cards in training most > frequently used network architectures.

I just did a haphazard, not very scientific, but ballpark-useful comparison: [https://news.ycombinator.com/item?id=21666411](https://news.ycombinator.com/item?id=21666411)

</comment></comment><comment><USER20> Finally.....

Me, yesterday : [https://github.com/tensorflow/tensorflow/issues/30638#issuec...](https://github.com/tensorflow/tensorflow/issues/30638#issuecomment-559201634)

<comment><USER21> > Can't bloody nvidia just open source everything. Those managers should be > hanged. Wasted way too much time on this.

As much as I want NVIDIA to open-source their drivers, they won't because of the same reasons the Raspberry Pi guys won't open-source their bootloader; they have to protect their IP.

To us it is a triumphant revelation to the 'open-source' community if they open-source their drivers, but in reality, to them it is absolutely 'treasonous' for someone in the company to open-source them as it is essentially risks their IP being stolen or fully reverse-engineered from a 1:1 reference to the original code.

Not as simple as it sounds to just 'open-source' something like drivers does it?

<comment><USER22> The claims about their IP being stolen are not credible.

For one, AMD has an open source graphics stack and none of their IP has been stolen.

Furthermore, Nvidia actually has pretty good support for nouveau (open source driver) on the Tegra line. Nvidia's real IP has not been compromised. Nvidia had to provide good open source support because the automotive industry wants guarantees that hardware will be able to run modern kernels many years in the future.

Open sourcing/upstreaming the device drivers has business value because it means the actual devices sold will be supported by Linux until they are truly obsolete. (Think i368)

Device drivers are not sexy, all they do is setup/shutdown hardware and provide hooks for the operating system to make system calls work. This is not core IP, kernel drivers are the bare minimum to enable hardware to be useful.

The real reason Nvidia doesn't want to open source it's device drivers is because it would cost money to do so. It would likely require a total rewrite, and Nvidias current driver presumably shares code between platforms with a shim that only Nvidia knows about. If everyone used Linux Nvidia would probably have open source drivers for workstations (like they do for embedded)

<comment><USER23> AMD is playing catch up, being far away from NVidia; the strategy is to open its stack so that people like you jump on the bandwagon and do exactly what you are doing, i.e. complaining about closed nature of the GPGPU computing NVidia created and cultivated. The same complaints as with Intel on MKL, i.e. AMD ignoring that market completely and then fans suddenly waking up and complaining everywhere. As we see with new Threadripper, once they are dominant they change their attitudes quickly. The only thing we as customers can do is to vote with our wallets and strategically support whoever does more sensible stuff, playing companies against each other.

<comment><USER15> I'm genuinely uncertain as to what you're complaining about re. AMD

<comment><USER23> They had my sympathies as they were on the verge of bankruptcy multiple times in the past decade; it's logical CUDA and MKL passed by and they couldn't react due to a lack of funds. However all the fanboys have to realize that building CUDA or MKL takes many talented/expensive developers/other personnel and obviously companies want to keep markets they created with them. And now with TR3 basically aiming for total high end, completely bypassing existing x399 and offering no "lower core number" versions of TR3, it's just a CPU for elite, not for regular folks, which never happened with AMD before and I view it as a worrying shift.

<comment><USER24> You can kind of see how they ended up where they did with TR3. With Zen 1 they had Ryzen going up to 8 cores, Threadripper between 8 and 16 and Epyc picking up from there.

For Zen 2 they've got the AM4 Ryzen with up to 16 cores and PCIe 4.0, and Epyc now goes up to 64 but it still starts at 8. Where does that leave for TR3? They got questions about whether they were even going to produce it. If a 16-core Ryzen 9 3950X is $749 and a 16-core Epyc 7302P is $825 then where would you put a 16-core TR3?

It's also possible the new I/O die pushed them to change the socket. Chopping the socket in half to create Threadripper from Epyc is one thing when the socket is already a 4-die package and you can just take two out, but what happens with the new Epyc I/O die which is designed for the whole socket, and could use it even for Threadripper? AM4 and Epyc have all the volume, reworking things like that specifically for compatibility with existing Threadripper boards may not have been worth it. Meanwhile a new socket lets them actually take advantage of the new I/O die to add more PCIe lanes etc.

It stinks that it's not compatible but it smells more like hard trade offs than malice.

<comment><USER23> Frankly, TR3 is a bit schizophrenic. So they introduced a new socket/platform for supporting up to 64 cores, yet still restricted memory to 256GB UDIMMs, instead of switching over to LRDIMM and allowing up to 4TB of RAM. And the whole platform is dead the moment DDR5 shows up (2021?). Restrict TR3 to top- end only, but limit RAM to low-to-mid-end workstation. It simply doesn't make any sense why to go through all those steps to bypass x399 while not bringing much to the table. Also, no single TRX40 board has more than 4 PCIe slots...

<comment><USER24> > So they introduced a new socket/platform for supporting up to 64 cores, yet > still restricted memory to 256GB UDIMMs, instead of switching over to LRDIMM > and allowing up to 4TB of RAM.

The only reason all desktops don't support RDIMMs in addition to UDIMMs is market segmentation. The Athlon XP supported both because they were really the same die with the same memory controller as the Athlon MP. They're playing the same stupid game as Intel now but it's nothing new with TR3. They've been doing it since the Athlon 64 or so.

Though it's especially annoying because the "workstation" models (FX/TR) have higher clocks than Opteron/Epyc, so there is no way to combine the higher clocked processors with the higher memory capacity boards.

> And the whole platform is dead the moment DDR5 shows up (2021?).

That happens either way.

You're regarding this as a whole new platform they had to design just for this, but it's more like a little tweak on the existing design with the unfortunate side effect of breaking compatibility.

> Also, no single TRX40 board has more than 4 PCIe slots...

There are boards with four real x16 slots, which is 64 lanes. The other 8 are needed for the on-board devices.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER25> Just ordered a 16" MacBook Pro with a AMD Radeon Pro 5000M. Will that now work with TF 2.0? How does it compare with the Radeons mentioned here?

<comment><USER26> I'm pretty sure it won't, at least not tensorflow-gpu. Tensorflow-cpu will though. There's no ROCm support for MacOS and obviously no CUDA support.

</comment></comment><comment><USER27> I am so excited for this. I have some AMD GPUs that I used for ethereum mining that I would LOVE to turn into an AI machine.

<comment><USER28> You can run AMD's port of Tensorflow 1.3 _today_ if you don't want to wait for main-lined support in TF 2.0.

</comment></comment><comment><USER29> I'd like to point out that bfloat16! = traditional float16

and forms of bfloat16 are not available for hardware acceleration on all GPUs yet.

</comment><comment><USER30> Surprise. Can this support Mbp 15 2019 vega?

<comment><USER31> Yes, the vega architecture is supported by rocm (basically AMD's cuda). You're best bet would be to use the pre-built docker images though.

<comment><USER18> On macOS? It is possible to use a GPU inside of Docker/macOS with Hyperkit? There's PCIe passthrough and/or SR-IOV support? ROCm requires Linux kernel extensions.</comment></comment></comment>