Ask HN: What are the least competitive consumer and enterprise markets? - <AUTHOR> My experiment in working backwards from market to problems to solutions:
1. Start with listing markets that have a low degree of competition, but don't have a mega-monopoly owning them. These will mostly be small markets.
2. Examine the problem space within each and see if new technology (SW / HW) can deliver 10x improvements.
3. Determine whether these markets are a short enough hop away from deeper ones.
It's surprisingly hard to get a "map" of existing markets, but am curious about those the community can readily identify. <comment><USER2> My experiment that I am thinking of recently is almost the opposite.

1\. Start with markets with high competition. E.g. paid alternatives to Google forms.

2\. Get a list of 50+ products competing in the space. If < 50 go back to step 1.

3\. Google to find what people, who are on the paid tiers of those products, complain about. This is easy as I am now Googling brand names so should get laser targeted results (v.s. googling "problem I had creating a form" -> Stack Overflow user who'll never pay for a form!)

4\. Interview them to dig in further. If you can't get any of these people to even spare 5 minutes to talk, then it might be an indicator that you wont get them to buy.

5\. Based on this, derive a hypothesis for a MVP that would solve the problem, along with the market it serves and where to find these people.

6\. Presell to people in #4. If they say no - dig in further as to why. If they say yes, aim for maybe $1000 monthly revenue presold, then build.

The reason for this approach is it filters for the "are people motivated enough to spend money" which I think is the biggest risk for the ideas I come up with. Since they are (they already use the "competitor" product"), can I carve out a niche where I do something better for a specific group of people? Can I reach them easily without spending crazy money on ads? And am I solving their problem?

Caveat is this is designed as an idea generator for an Indie Hacker style project, not a startup!

<comment><USER3> You should probably add a step:

5.5. Derive a hypothesis for why the 50 other competitors have not already solved this problem or couldn't do so quickly once they discover your solution.

If you don't have a good answer for this, then the odds are one of two things will happen:

1\. You will waste a bunch of time only to discover that you were ignorant of some deep constraint of the problem area.

2\. Your competition will re-implement your feature and crowd you out.

<comment><USER4> I wouldn't worry so much about #2 unless the market is extremely small or you're looking to raise money or your only vision of success is a giant company.

Most markets with existing companies serving them are big enough to support a small indie company. And there are things you can offer as a small company that's difficult for big company's to offer.

<comment><USER3> That's true, but personally, I would be really hesitant to get into a market with several large dominant players without a _significant_ edge. The reality of business in the US is constant consolidation towards a small number of huge, anti-competitive players. I can't find the source right now, but I've seen statistics that the total number of businesses in the US is _much_ smaller today than it was, say, 50 years ago.

So I think your expectation, assuming you are in the US, is that your business has one of a few trajectories:

* You deliberately aim for a market with many small businesses and forces that go against consolidation. There are fewer of these over time, but anything explicitly local, has few economies of scale, or where consumers specifically target variety is a good place to look. Think restaurants, luthiers, and other "craft" businesses. Then you can have a long moderate term success while staying small. You probably won't get rich but you might like your life.

* You try to grow very rapidly to get to be a big fish before the other big fish eat you. You take a ride on the VC train and if you're lucky escape with your soul intact.

* You aim to get bought. You lose the experience of it being _your_ business but you might get rich in return.

* You get crushed or bought against your will. Short of simply fading away from lack of success, this is probably the most likely outcome.

</comment></comment></comment></comment><comment><USER5> Beware: if there are markets that appear uncompetitive, there is a damn good reason for it. And it's never becaues no-one thought to look there. If you do find uncompetitive markets, it'll most likely be because someone has regulatory capture, or there is something preventing new entrants, high barriers to entry, or the cost of user acquisition is simply too high for new entrants.

<comment><USER6> Or, some businesses are just not worth it.

You can throw as much business intelligence and automation to, let's say, an Ice Cream truck. It will still be an Ice Cream truck. Nobody is becoming a millionaire with it.

<comment><USER7> > Or, some businesses are just not worth it. > You can throw as much business > intelligence and automation to, let's say, an Ice Cream truck. It will still > be an Ice Cream truck. Nobody is becoming a millionaire with it.

That's probably what people said about coffee and cafes prior to Starbucks.

<comment><USER8> Starbucks is an essentially American phenomenon. e.g., nobody in Europe ever thought coffee shops needed to become branded chains that set a single product quality expectation, because almost nobody who'd ever visited an Italian/mediterranean-style cafe ever thought they lacked a quality or sociability option to begin with. Sure, some chains exist, but the quality delta between them and the average cafe isn't great enough to scale fast.

Only in a place where coffee is generally awful, and VC money easy, could the concept that became Starbucks ever come into existence.

<comment><USER7> There's a whole lot factually wrong about your comment (Starbucks started as a single cafe, it wasn't funded by VC, and it started in the 70s/80s when money was far from cheap, and 1/3 of its total outlets are in Asia).

But that's all beside the point. I'm not saying an Italian would be able to replicate Starbucks in Italy. What I'm saying is that there are countless opportunities staring us right in the face, every day, and 99% of people see a cafe and think "it can't scale, it's local demand only, there's no point, why bother", the remaining 1% see Starbucks.

</comment></comment></comment></comment></comment><comment><USER9> _It 's surprisingly hard to get a "map" of existing markets_

No, it's not. It might take more than a Google search.

You can extract what you want from US business census data. See "data.census.gov". Look for NAICS codes with a small number, but greater than 3, companies, and high dollar amounts for the category.

<comment><AUTHOR> It seems there are 19 codes: [https://www.naics.com/search-naics-codes-by- industry/](https://www.naics.com/search-naics-codes-by-industry/). I'm interested in more granularity e.g. where "mobile video editing software" appears on the map, show me the market size, top companies, estimated sales, etc.

<comment><USER9> That's just the top level of NAICS codes. For example, 441320 is tire dealers. 512191 is "Teleproduction and Other Postproduction Services"[1]

"Mobile video editing software" is a product type, not a business type. If you want product categories, you need a different data source. Such as Amazon's, or Google's or Alibaba's product tree. Here's Google's product hierarchy.[2] Google category 4953 is "Software > Computer Software > Multimedia & Design Software > Video Editing Software".

If you want such results handed to you without much work on your part, you may have to pay a company which collects such data. NAICS or D&B or some of the mailing list companies could make you a list.

[1] [https://www.naics.com/code- search/?naicstrms=video](https://www.naics.com/code-search/?naicstrms=video)

[2] [https://www.google.com/basepages/producttype/taxonomy- with-i...](https://www.google.com/basepages/producttype/taxonomy-with-ids.en- US.txt)

<comment><AUTHOR> Thanks for that. I think the NAICS codes could be fleshed out a little to be more relevant and useful.

Not to beat this example to death, but there is a market for mobile video editing applications and I have no idea of its size. Wikipedia only lists one iOS product. I feel there would be a lot of value in providing this kind of information easily to people. I personally don't want to engage with a market research firm during the exploratory phase.

<comment><USER10> I've been doing research on this exact category myself (after having previously had some modest success on the image editing side.) You're severely underestimating the number of competitors. Try searching the relevant app stores instead of Wikipedia (why would you ever even check here?).

<comment><USER11> I think parent was complaining that there's very little public information (particularly hard numbers) about the incumbents. GP was cruising Wikipedia for information about the apps' parent companies rather than just trying to identify the apps.

It's easy to see how many apps are in the store. It's a lot harder to know how much money they are grossing.

</comment></comment></comment></comment></comment></comment><comment><USER12> Could be labor organizing. I've had a job for years and never even been solicited about it.

The incumbent players are ignoring massive swaths of the addressable market at a time when, judging by my social media feed, there is renewed interest in labor issues.

A big reason for this is their reliance on a high-touch, manual onboarding process and slow, high overhead contract negotiation techniques. Prime for marketing automation, SaaS tools, chatbots, etc.

Additionally, for purely historical reasons, they have segmented the market by trade. There's no particular reason for you to follow this path, may as well help everyone get a better deal from their employer (and a cut for yourself!).

<comment><USER13> I don't know much about contract negotiation, but for organizing "high touch" doesn't begin to describe it! The labor organizer's job is to convince the worker, who likely and often rightly fears for her job if she actively and publicly supports the union, that it's still the right thing to do. Not only convince intellectually, but psychologically prepare the worker to stick with the union effort in the face of a concerted effort from the employer to defeat it. To do that the organizer needs to ask the right questions, listen carefully, build trust, and finally push, and potentially do this repeatedly with the same person. It is psychologically demanding and not everyone can do it; I tried and I couldn't. A chatbot cannot convince a factory worker with kids to feed that he needs to stick up for his coworkers who in turn will stick up for him. All that said I think there is room in the process for profitable automation, but it's auxiliary to the main work.

If you're interested Jane McAlevey's book Raising Expectations and Raising Hell is a good introduction to union organizing.

</comment></comment><comment><USER14> Check out the translation industry. It's a fairly niche market, with its own quirks. There is a good book that gives an overview of how the industry works, but I can't remember the title atm. It has a yellow cover and some cartoon drawings inside... :) If you're interested I can dig a bit more and find it.

There is software for translation management, which is essentially project management, but because of the intricacies of the business there is specialized software for it. There are maybe 2 or 3 main competitors, all of which do the job but are fairly awful to use. Then there are a handful of products which come from computer-assisted translation (CAT, which is different from machine translation), and try to capture the project management part as well. Because their focus is on CAT, they're also not excellent in the management aspect.

If you want to capture the whole spectrum with all the edge cases, it's going to be a very complex product which will require a lot of user research and take a fairly long time to build. But there might be an opportunity to go to market earlier, with a subset of the functionality, and build from there.

<comment><USER15> If you can remember the title of the book I'd love to read it.

<comment><USER14> It's called 'The General Theory of the Translation Company' by R. Beninatto and T. Johnson.

<comment><USER16> Thanks a lot for diggin' it out! Looks really interesting, yay!

</comment></comment></comment></comment><comment><USER17> When it comes to market selection, I often think back to how Ryan Petersen came up with the idea of Flexport. He put up landing pages and some ads, and only when he saw interest from the biggest players did he go all in. It's a pretty darn good way to find out, even if it sounds too simple, or cheap, but if you listen to the rest of the interview, Ryan had semi-failed enough times beforehand to realize that simplicity was key. Maybe throw a bunch of landing pages up and see what works?

(original audio segment: [https://smashnotes.com/p/y-combinator/e/92-ryan- petersen/s/h...](https://smashnotes.com/p/y-combinator/e/92-ryan- petersen/s/how-did-flexport-get-its-first-customers))

<comment><USER18> That's pretty cool! Didn't know that Flexport started out like this.

Here's another example of using a landing page to validate demand, and one where the product ended up being a very successful company: [https://sumo.com/stories/80-20-business-idea- validation](https://sumo.com/stories/80-20-business-idea-validation)

I recently played around with the landing page method for an idea I had and ended up not getting much traction. That negative feedback was really useful in helping me decide whether to actually code up the app. Wrote about here, if anyone's interested: [https://abiraja.com/the-landing-page- method/](https://abiraja.com/the-landing-page-method/)

<comment><USER19> Good read. I think for anyone creating so many ideas and want to test, it is safer to try landing page method. 100 ideas, 3 really gets traffic, then those 3 really work.

<comment><USER18> Yeah, I think the key point is that the landing page method is great for identifying smashing successes (high true positive rate, low false positive rate). But ideas that fail the landing page method might still be good, just need more investigation (high false negative rate).

</comment></comment></comment></comment><comment><USER20> Language and culture is usually a barrier. For example, one of my apps really blew up because we targeted Malaysian low carb dieters. Western low carb diets are well done and cover all kinds of things like steak and pizza alternatives. We covered local dishes like laksa (spicy noodle soup) and rendang (a kind of meat dish). We got 3,000 monthly active users with very little paid marketing and 3% of them were paying customers.

This fit in with your other requirements - it would be easy to hop from low carb recipes to say, Indian vegetarian recipes.

The 10x trick is done by looking for crappy apps with a substantial user base. In our case, we were competing with a FB group with 200 thousand users and a constant stream of posts that made it difficult for people to look for recipes. Our app competitor was a HTML5 app which someone did as a technical demo, with 50 thousand users.

<comment><USER21> 90 paying users is cool but I wouldn't say it "really blew up"

<comment><USER20> It was a health food grocery app, so basket size per user was high. It was RM32-RM128, which by purchasing power translates to roughly $40-$168 per paying user per month. This is also averaging it - as a diet app, we peaked around Dec-March.

And it was more on the flow. We got 1200 users in the first 24 hours, after a two week hack. There was definitely a market there. The goods were overpriced because we didn't have economy of scale, but if we did get the prices down as intended, we'd have far more regular customers.

<comment><USER22> How did you perform market research for a country like Malaysia. Was your idea based on some personal observations?

<comment><USER20> Pivoting. It was originally a grocery price comparison app. That was illegal (long story). So we pivoted to a recipe app and try to sell ingredients. We looked at other recipe apps like Cookpad and felt that what they were missing was quality control.

We wanted a quick and dirty prototype that was a subset of the grocery market, so that if it didn't go well, it would just be controlled damage.

I was following a low carb diet so I did a low carb diet app, solving my own problem. There were some blogs and a crappy app that did this. The Facebook group had too much traffic to keep track of recipes and no quality control, so half the recipes were terrible. We released the app, expecting about 30 downloads but it was 1200 and crashed the server temporarily. We also got some feedback that calorie count wasn't helpful, so we switched to carb count, and were the only ones on the market to compile all the local recipes and have carb count for them.

The business model canvas helped a lot in the process. We drafted a business model, then identified the riskiest thing and tackled that. At first it was the business proposition (which went from find food > eat healthy > lose weight). Then it was revenue model (sell directly, instead of ads). Then operations, delivery. The target market was unintended and we could well have missed it, because low carb diets aren't part of the culture.

<comment><USER22> Damn HN has no notification system!

Interesting story. A Pakistani who is willing to get settled in Malaysia in coming years, I was wondering what kind of things I could make for the local market which made me to ask the question.

</comment></comment></comment></comment></comment></comment><comment><USER23> I know a niche in hardware, encountered while doing R&D for an F500.

High sample rate (20 Hz min.) sensor telemetry (9 DOF IMU + GPS) over cellular network, as a single encapsulated unit with built in antenna and a battery to handle power interruptions.

Data needs to go a remote server and devices should be remotely manageable if deployed as a fleet.

I searched quite a bit and eventually ended up using a bunch of Samsung Galaxy phones in a hard case, with a datalogging app and an automation that syncs log files to Dropbox. Devices were managed over TeamViewer.

There are some small players in this field each satisfying 70% of these requirements and they're printing money.

All the pieces are out there. Just need someone to make a product so enterprises can throw money at them.

<comment><AUTHOR> I like your minimalist solution. Fleet tracking / management seems like a commodity business with a lot of players in it. How do you differentiate and win here?

<comment><USER23> Fleet management solutions are built for EDL requirements. They have event based logging and won't log data at high sample rates and don't send it to a server of your choice because they want you to use their subscription based dashboard.

<comment><AUTHOR> Curious which applications require logging at high sample rates vs. doing event-based?

<comment><USER23> From my narrow experience as an R&D engineer in the auto sector:

You need to understand the duty cycle of the platform you're developing for (ex. Tires for a vehicle). A garbage truck is going to have a different loading cycle from an EV or a CAT morning truck. If you know the right conditions your product will fail on durability.

Fleets are ok with collecting this data for R&D as long as you don't bother them with extra work and give them useful analytics. Data collection devices need to be easy to install, collect for a few weeks and remove.

You can also use this as an R&D platform to develop IoT data-enabled products like predictive maintenance, route analytics, insurance and warranty claims etc. ML has a lot of potential here but you need to collect data.

Eventually when you have an IoT product you can get a vendor to make an optimized ASIC to collect and process exactly what signals you want.

There are many more applications in the auto industry for these kind of niche products.

</comment></comment></comment></comment></comment><comment><USER24> Services that sell to HOAs

This is something I've been thinking about since my HOA sent me an email a month ago warning me that the sidewalk outside my property may be damaged and in need of repair because if someone tripped I'd be liable for this. Of course the email had a bunch of red and gold text.

Luckily for me they already had a concrete cutting company come by and survey the area and provide estimates for each plot.

My bill would be about $200 to fix the sidewalk. I looked outside and it was totally level and no edges exposed.

In really small print at the bottom of the email was information telling me it was optional to do, but again I might be liable if anything happens.

Today I saw this person outside doing the concrete repairs. It was a guy with a small trailer and grinding down the sidewalk with an angle grinder and a vacuum to suck up the dust.

This basic concrete cutting job is pulling money in if he can go around and tell all the HOAs about the urgent need to fix sidewalks and HOAs pass that along to all the residents.

How many people paid the $200 "to be on the safe side"?

Now you repeat for all the other services you can think about for home maintenance and repair and contact HOAs with estimates they can pass along.

<comment><USER25> >because if someone tripped I'd be liable for this

isn't the sidewalk the city's responsibility? Also, wouldn't your homeowner's insurance cover it?

>In really small print at the bottom of the email was information telling me it was optional to do, but again I might be liable if anything happens.

Honestly the arrangement seems rent-seeking(ish) to me. At the very least it leaves a bad taste in my mouth. They're not providing any _real_ value, just scaring people into paying with an (arguably) misleading letter.

<comment><USER26> > They're not providing any real value, just scaring people into paying

Unfortunately there are plenty of business built exactly like that.

One anecdote: there was a group of lawyers that would hire people in wheelchairs to go into stores (e.g. local Verizon dealer) and ask to use the restroom, then once inside the restroom measure everything and take notes. Apparently businesses are not required to make their restrooms available to the public, but if they do, they need to be up to code. So these guys would target chains of businesses that normally didn't allow public into their restrooms, hence the restrooms were usually not up to code. After gathering enough data they would then sue the companies and offer to settle for an amount very close to what it would cost the company to deal with the case in court.

<comment><USER27> Or you could see it as them using the Ada for it's exact intended purpose, to force businesses not to exclude disabled customers by imposing an actual cost on them that is potentially greater than the cost of complying with the act. Sounds good to me!

<comment><USER28> I don't see how this is good. Some businesses allow their customers to use their private bathroom on a case by case basis. Now they will just stop doing that so everyone suffers equally.

<comment><USER27> Their bathrooms should be accessible for their employees as well, even if they're not usable by customers, because disabled people should be able to work to.

</comment></comment></comment></comment></comment></comment><comment><USER29> The least competitive markets are most likely local small-businesses with regional restrictions. By this I mean places that where you need to physically go (e.g. dry cleaner) or where someone needs to physically come to you (e.g. electrician). Because you're only competing with people in a specific region, these are going to be much less competitive than industries where you compete with the whole world (e.g. software).

This doesn't necessarily help you come up with new software, but it is something I've spent a lot of time thinking about since I'm also trying to start a business.

<comment><USER30> I would pay handsomely for a service that would find and dispatch a qualified tradesperson (electrician, residential HVAC, plumbing, &c.) that would show up when promised and get the job done. And would guarantee the work, pull permits, and get me a lien waiver afterwards. I really hate the time it takes to wade through online reviews, play phone tag, and deal with flakes and no- shows when there's something that needs to be done around the house.

<comment><USER31> For those tradespeople, running a small business can be seriously time consuming. I try to support lone tradies or very small business people, but it's almost always easier finding slightly larger ones that have enough staff that there is someone handling quotes and delegating tradespeople.

I'd suggest building a backoffice for tradies that automates as much as possible. Payments, chasing payments, reminding them of people waiting for quotes, etc. A CRM app that all their incoming enquiries go directly to, etc. Otherwise they have quote requests coming into their phone messages or emails and getting lost along with everything else. I run a small business and have 200-500+ unread emails at almost any time. If something slips off the first screen of G-Suite, it is much harder to remember it.

<comment><USER29> ServiceTitan[1] is a big company that does exactly this, but there's probably space for more than one player.

[1] [https://www.servicetitan.com/](https://www.servicetitan.com/)

</comment></comment></comment></comment><comment><USER32> So usually what happens is that an individual is an expert in their under- served area, sees the opportunity, and then puts in the effort to build a business to serve that opportunity. The point being that you won't get people who know of lots of opportunities, but rather many individuals who each know of their own opportunity. And you most likely won't find such domain experts on HN.

It might be worth looking for evidence of attempts at building such businesses, since often such domain expertise doesn't pair with business- building expertise. So if you're trying to build such a map, then maybe look at all the businesses (including all the failed ones) that served each market.

</comment><comment><USER33> Related: I highly recommend The Blue Ocean Strategy[1]. I suspect what you are looking for is the Blue Ocean market space, not necessarily the "least competitive". As others have pointed out, if a sector lacks multiple entrants, it is almost always with very good reason.

[1] [https://learn.blueoceanstrategy.com](https://learn.blueoceanstrategy.com)

</comment><comment><USER34> You can _safely_ bet than any below the highest tech-level industries (so anything not called Apple, MS, ....) are under-served.

I work in the small-bussines and before in government. have done some stuff for some of the biggest companies in my country (Colombia).

To say ALL of them are like 20-10 years behind is not say enough.

Today, I'm integrating with cobol and other stuff that only have text-based files as interface.

A problem is that not many investment are in this long tail, so when talking about solutions for this market is possible to NOT get excitement for it.

I'm building on the side a relational language that eventually could be an Access+Excel tool, but everyone is interesting in the markets that reach billons :).

</comment><comment><USER35> I like to look at industries that customers hate. Tech support, cable, wireless, health insurance,etc.

I picked tech recruiting for my most recent startup. :)

We've bootstrapped to over $2M in revenue in 18 months. People seem to like the service much better than traditional recruiting.

(Shameless plug: [https://www.facet.net](https://www.facet.net))

<comment><AUTHOR> That's awesome, congrats. I can't tell what your secret sauce is from your website. I'd love to ask you how you marketed this service if you want to shoot me an email (see profile).

</comment></comment><comment><USER36> I'm not sure whether the competitiveness of the market is very relevant for your success in finding a profitable innovation. Profitable innovation is hard, in any market.

There may be an expectation that there are higher profit margins in non- competitive markets, so you may have an advantage there that any innovation will allow you to capture more excess profits as it will take longer for those excess products to be eaten by increased competition you triggered.

On the other hand, there's an expectation that the incumbent(s) in a non- competitive market has a lot of funds at its disposal (due to years of capturing producers excess profits in a non-competitive market) to hamper innovation.

To me, this illustrates that whether the market is competitive or not is not very relevant. The question is whether you are able to outcompete the market by reducing costs and/or improving your offering compared to your market, whether that's a single monopolist or 1,000 highly competitive companies.

</comment><comment><USER37> Are you familiar with Porters five forces? and the sixth, which is regulatory environment?

If you're not, it's worth it's weight in gold when evaluating markets and looking at how do build your business model.

</comment><comment><USER38> A lot of good comments here.

A different way to approach this problem, is to consider culture: 'people like us, are attracted to certain things'.

Those fields are overwhelmed.

So, where are the 'completely unsexy' areas?

1) Human Waste. Port-o-potties are a huge business, and it's a nasty problem, pun intended.

2) Agriculture. Farms are not hip.

3) A lot of goods and services targeting low-income people who are extremely price sensitive and don't care about how cool things look.

4) Anything to do with a business that is mundane: cleaning, forensic accounting, physical security, garbage, etc..

Literally walk down the street and look at what people do, it's funny how you can get an instinct for how very 'specific' YC pattern types are, and how we're a little bit out of touch.

<comment><AUTHOR> I agree, we may have reached peak level of Seth Godin-approved consumer startups.

Re: 1, it does bother me producing all that plastic when a cardboard structure with sterile, swappable lining might do just as well in non-windy areas.

</comment></comment><comment><USER39> you need to work backwards. ask yourself: are you trying to be vc funded or bootstrap. that alone will determine what kind of clients or markets you can serve. #1 ask yourself is the market growing or dying - I made a mistake once of making software for laundromats i.e dying market n run by old people. depending on which path you chose bootstrap or vc 1% customers of your total market can be enough to sustain yourself. on the bootstrapped level. if vc, better chop up vc money n live large

</comment><comment><USER40> Anything relating to regulated banking. The challenger banks all have a chartered bank partner under the hood.

Licensed Banking: a there hasn't been a new chartered banks in a decade

Also Clearing Brokerage. Credit Bureaus. US Core banking software, KYC providers(Eg:lexis nexus), bill pay providers

<comment><USER41> Also, The Fed

</comment></comment><comment><USER42> To solve for this one would need to create metrics around measuring markets and opportunities. I suggest the following three axes:

X. Market size (actual values, divisible into competitors) Y. Market valence (positive/negative consumer/user/customer/investor perceptions of each of the competitors; e.g., some sort of objectified, normalized product/service/company rating) Z. Market momentum (vectors of growth/shrinking/stagnation of the market overall, and each of the competitors within it, broken into actual past vs future projected)

Many times a market may not grow not because the market "isn't there," but because all of the existing offerings frankly stink. Imagine movies. If you have 12 terrible science fiction movies in a row, does that mean there's no market for science fiction movies, or does that mean someone finally needs to make a _good_ science fiction movie?

You might also have to define more granular domains. Maybe the market might be for a geographic region, or a vertical or horizontal market sector.

</comment><comment><USER43> 1\. For a low competition market I am looking at browser automation of user interaction. I recently created an original solution for a niche set of problems that incidentally solves many other more common problems.

2\. The cool thing about this space is that it isn't new technology that delivers 10x improvements. The opposite actually. 10x improvements are found by doing the least possible effort in a tiny library, scaling to support a wider problem space, and doing it substantial better than large products present in the marketplace.

The few competitors are easy to search but most of the commercial products are so expensive that it's hard to really gauge the precise performance capabilities are the smaller players without a large financial investment.

3\. I am just now learning about second order effects from this thing as I extend it to support execution across networks/VMs. There are all manner of second order surprises that emerge from better software automation.

</comment><comment><USER44> anecdotal, but my roommate is a first year lawyer. A lot of his job is taking old legal docs and updating them for new deals. This is a lot of tedious search and replace and formatting in hard-to-edit PDFs.

A sample scenario is one of his senior colleagues takes a PDF and writes in some comments -- almost like a teacher grading an essay with a red pen, change this x to y, etc. This goes to my friend, who reviews, edits, and returns the draft by email. And there will be a lot of back and forth, plus other people making edits in real time.

watching this i can't help but notice how remarkably similar it is to programming, minus all the convenience of programming tools.

i don't know what the product or barriers are here, but these people could easily 10x if they knew about version control and markup languages.

<comment><USER45> I work in this niche. I actually work a niche within this niche, and have implemented solutions. We do have a couple competitors. But most new players in this arena come in with exactly that statement - "oh, this is like programming, lets bring in markdown and git and solve all the things!" Then they realize that the level of detail and the expectations of legal document authors are far higher than they ever expected. Markdown doesn't cut it. HTML doesn't cut it. CKEditor and tinyMCE don't cut it. We spend a huge amount of effort re-writing plugins for those editors to meet the actual expectations, and defining new data formats for all the citations and interlinking of legal documents.

Not that there isn't room for competition - there is. But the barriers are not what people expect. And the easy answers that coders tend to see aren't the real problems.

<comment><USER46> How I love this comment. I do socials for CKEditor and read almost everything there is to read for WYSIWYG rich text editing, collaborating.(a little late this time)Feedback, frustrations, feature requests. I'm sad to read CKEditor doesn't cut it because I'm able to see in first hand how much effort goes into it. But I'm not surprised because I still have "I would have never guessed" moments about all the ways people use the editors myself. There is ALWAYS room for improvement when it comes to writing on web and feedback from users like you are crucial :)

</comment></comment></comment><comment><USER47> Here's a list you might be interested in:

\- home services: lawn mowing, pest control, firewood delivery

\- vehicle services: mobile oil change, mobile tire sales, locksmith

\- event/seasonal services: catering, event management, tent rentals

\- entertainment: bike tours, party rentals, hunting guides

\- personal care: mobile haircuts, mobile makeup, mobile massages

\- training/coaching/consulting: pet training, college application consulting, credit repair consulting

\- trades/construction: electrical, plumbing, carpenting

\- business services: videography, bookkeeping, junk removal

\- real estate transactions: moving services, realtor, vacation rental management

Source: [https://sweatystartup.com/businesses-i- love/](https://sweatystartup.com/businesses-i-love/)

<comment><USER48> *source: jobs where you are paid the minimum wage, but maybe you can cut that down a little by building a rent-seeking company.

the only things which might remotely be well paid (trades/construction) generally have some relatively high hurdles in most countries.

<comment><USER47> >*source: jobs where you are paid the minimum wage, but maybe you can cut that down a little by building a rent-seeking company.

You're assuming that people in these jobs get paid minimum wage and that the only way to be profitable is to undercut their wages. By offering better service, you can raise your rates and pay your employees better. But something tells me you knew that already and you're just saying toxic things to discourage people from trying.

<comment><USER48> well, except for the mentioned trades, they are all lifestyle businesses and luxury things. With 80% of the population not being able to afford this regularly (and being super price concious) and a large portion of the remaining 20% being chronic sociopaths you have to be very lucky to find someone who pays you well and treats you like a human.

</comment></comment></comment></comment><comment><USER49> Commercial and military aviation lol. Trust me it's by design. And you don't want to get into this market.

</comment><comment><USER50> You should read this book: [https://en.wikipedia.org/wiki/The_Innovator%27s_Dilemma](https://en.wikipedia.org/wiki/The_Innovator%27s_Dilemma)

You should look for markets where a reconfiguration of leading technology could serve a new market that existing leaders in the space are unable to service due to structural necessities in their own established business models.

Trying to compete against incumbents by improving what they are already making progress on (sustaining innovations) is nearly impossible because they've set themselves up to be leaders in incremental innovation already.

Disruptive markets are so because they emerge at profit margins and market sizes much too small for large existing organizations to justify servicing, and also the new products do not meet the requirements of the established customer-base. Eventually, there is a chance that the disruptive technology will intersect with mainstream customer requirements and this is where customers begin to see the 10x benefits you're looking into, much further down the road.

</comment><comment><USER51> The least competitive markets don't exist yet, the next least are tiny. These attract the smallest and least well funded competition.

</comment><comment><USER52> Honestly, even if I knew of a bunch of potential markets I'm not sure I would won't you to know. Your motivation isn't that you care about a particular market or problem. It seems your just looking for the most efficient way to 10x your profit. Dispite knowing much about the people/ product/ market you may serve.

I've lost count of the things that I have used in the past that I have stopped using because the product sucks now and is of lower quality or they have blessed me with a subscription model that only cost $5 or $10 a month. Instead of sucking it up and paying for something once I have to die by a thousand cuts and pay for something the rest of my life if I want to use it. Sure it's great for the profits of that company but not for me.

So maybe that's the underserved market. A product that respects privacy and leaves you the heck alone.

</comment><comment><USER53> > It's surprisingly hard to get a "map" of existing markets, but am curious > about those the community can readily identify.

Startup idea. Automate this analysis and sell the aggregate. No one wants to do this work, but if you had a trustworthy source to exhaust all info for a niche or corner of the market, that would be cool.

<comment><USER54> Isn't this what all those "market intelligence" firms sell? Point is only established companies want to pay for it.

<comment><AUTHOR> I've met startup founders who pay for this data to help in writing their go- to-market strategy for series A.

I really wonder how much friction there is in using a market research firm and if that's a big deterrent to a lot of potential customers.

<comment><USER53> These firms have to guard their data. It's all they have. So yeah they put up big walls and add lots of hurdles (calls, agreements, phases, etc) to increase the perceived value of what they are selling you.

If this data was more accessible, it would definitely be a game changer.

Finding it and presenting it and making it explorable is the really hard part.

</comment></comment></comment></comment><comment><USER55> It has been my (unfortunately frequent) experience that funeral homes are almost universally owned by individual families.

<comment><USER56> It's actually not true, but the real owner, Service Corp International, wants you to think so.

> SCI then retains the funeral home's original name, often along with former > owners who are kept on as management. A typical funeral home that is owned > by SCI will not contain advertisements or logos for SCI

[https://en.wikipedia.org/wiki/Service_Corporation_Internatio...](https://en.wikipedia.org/wiki/Service_Corporation_International)

<comment><USER55> Interesting to know. Thanks for the link.

Edit: actually that doesn't seem to be correct.

 _Eighty-nine percent of funeral homes in the U.S. are owned by individuals, families, or closely held private corporations. The remaining 11 percent are owned by corporations whose stock is publicly traded._

From Wikipedia:

 _In the 1960s, a push for large companies acquiring smaller funeral homes and cemeteries occurred.[18] Although there has been a consistent push for consolidation, the majority of the industry still consists of small, family- owned businesses.[18] Experts and analysts of the industry have estimated that the top six funeral operators control 25 to 30% of all funeral services in North America, with the top four owning between 15 and 20% of all funeral homes._

<comment><USER56> My bad. I thought they owned more than half of the whole market.

</comment></comment></comment></comment><comment><USER57> Encrypted Messaging / File transmission / IOT control.

Current messaging solutions have different issues.

1\. Not encrypted.

2\. Transmission is encrypted but they are centralized, the message is open in the server.

3\. End to end encryption but susceptible to block because some signaling/presence is towards a shard of central servers.

4\. The infrastructure relays on well know properties of IP/TCP (proxies, ports...).

5\. the packet has some well know properties (checksums, structure...) that allows to be identified and blocked.

5\. Authentication and peer identity discovery.

There was a time (early 2000s) where Skype was a headache for system administrators, because it was able to pass any firewall and the conversations were able to pass. That is missed in the current world.

None of the current solutions, telegram, line, whatsapp, skype, email... can pass the previous checklist.

</comment><comment><USER58> Utilities have a very low level of competition, but that's because they are granted an exclusive charter by the government. Private industry still competes in various roundabout ways, such as solar panel installation, and batteries that can game peak/of-peak times.

</comment><comment><USER59> A market with no competition is a monopoly. They are characterised by extremely high barriers to entry.

You are better off finding something you are passionate about in a competitive market, then working out how you will differentiate and market it with the budgets you have to work with.

</comment><comment><USER60> Great strategy - you are saving yourself a lot of time with this approach.

What's the value of a good "map" of an existing market? What are you willing to pay for one?

<comment><USER61> I feel that maps of existing least competitive markets are low in competition.

</comment></comment><comment><USER62> I don't have any experience or insights of it, but the very first thing that came up to my mind is: frame glasses.

I've heard multiple times already that price margins are huge at that one, and nobody are determined enough to offer lower prices for them.

But that's only what I've heard from here and there, I'm not into that topic "that much" to confirm that - OP asked for some suggestions, so I gave one ;)

</comment><comment><USER63> Film cameras. Dominated by ARRI, a 100 year-old company whose worth is ~250m. There were disruptions (red, sony, etc.), but they regained dominance rather quickly (one semi-metric is oscar nominees which cameras are used, as well as top 500 grossing films over a year). Similar stuff in the same industry with services and lenses with Panavision (rental only) and dollies (Chapman Leonard, also rental only).

<comment><USER64> Or even digital imaging, which is a new technology forced onto an otherwise very conservative industry.

As an example, think of TSMC's upcoming 3nm process. It has a density of 300 million transistors per square millimetre. A Canon 5DS R has a 50 MPix resolution, which works out to about 60K photosites per square millimetre. That's a "budget" of 5,000 transistors per photosite!

That's more than enough to do "digital sensing". That is: accumulating photons in a digital counter instead of using an analog capacitor, allowing unlimited dynamic range. There would be no such thing as sensor saturation, allowing unlimited "electronic ND filter" effects without the colors being distorted even in a long time-lapse. It would allow "steering" of the data to nearby accumulators through an on-chip network at megahertz rates, providing near- perfect digital shake reduction without having to physically move the chip. Shake reduction could track moving objects and avoid blur even for very long hand-held exposures with cars or people in the scene, the same as what the Google Pixel does, but at a far higher quality. The motion vectors could be fed into a video compression stage, improving picture quality to _above_ what the compressors can normally achieve by "guessing" the motion from 24fps still frames.

Etc, etc...

<comment><USER65> What does this mean...you still need some sort of analog capacitor to collect photons and convert them to electrons.

<comment><USER64> So instead of an ADC converting an analogue charge level to a 14-bit or 16-bit signal from a "big" capacitor, you instead have a tiny capacitor (or none at all) and collect individual 1-bit signals and accumulate them digitally. Photon counting, in essence.

<comment><USER48> so what kind of collecting device do you propose then?

</comment></comment></comment></comment></comment><comment><USER66> My uninformed guess is legal "engineering", such as tariff engineering. Discovering the cheapest way to manufacture goods / run your business given the most up to date legal landscape. To some respects, privacy companies that help you navigate GDPR/CPA are in this category, but there's plenty of space here (immigration, copyright, zoning, etc).

<comment><USER67> This appears to be the IKEA practice, they less of a retailer and more of an global tax avoidance scheme

</comment></comment><comment><USER68> Dog-walking. My roommate used to make six figures by setting up standing dog walking appointments for affluent clients. She only worked six hours a day. Note that this was before the days of Rover which probably dominates the market in large cities, but I suspect suburbs and rural areas still need help.

<comment><USER69> I think there is definitely a place to compete against incumbent gig work companies with local alternatives. In the dog walking example, some people are willing to pay a premium to get the same person every time. Or, at my local dog park, a person brings a bunch of dogs and let's them play in the park for an hour instead of walking them.

I definitely believe this could be the path to carving out small lifestyle businesses.

<comment><USER31> Might be an opportunity there. Find repeat customers. Find dogs that play well together and aren't hard work. Specialise in very particular areas around parks that suit the operation. Decline dogs that make the process too difficult.

Letterbox drop around existing clients and parks to find more close by. Offer volume discounts for referring neighbouring dogs.

Take said dogs to park and let them run around while you operate your other businesses from your phone.

<comment><USER70> > Take said dogs to park and let them run around while you operate your other > businesses from your phone.

This doesn't sound like responsible dog handling.

<comment><USER31> No, that last bit was a joke. Working from your phone while your charges are playing is more what most parents do!

</comment></comment></comment></comment></comment><comment><USER71> Look for companies that have niche solutions for legacy platforms that aren't going anywhere like insurance systems, DMVs, social services etc.

Sometimes you can buy some little company that does something minor like format print output for some ancient mainframe and the extract more money or find complimentary sales.

</comment><comment><USER72> Medical imaging like MRI. Put it in a truck trailer and tour the underserved communities.

Unfortunately, States have medical device approval hurdles which often explicitly consider competition, and prohibit the economic competition.

<comment><USER54> MRIs draw a boatload of power to keep liquid helium liquid, even when "off".. just account for a pretty big generator in that truck, and make very sure you don't get stranded without heavy power overnight.

[1] [https://www.cocir.org/fileadmin/6_Initiatives_SRI/Measuremen...](https://www.cocir.org/fileadmin/6_Initiatives_SRI/Measurement/COCIR_SRI_- _MRI_measurement_of_energy_consumption_2012.pdf)

</comment></comment><comment><USER73> Take a look at Costar (Commercial Real Estate Data / GIS Demographics)

</comment><comment><USER54> For many regional and regulatory burdened fields (e.g. ISPs), be the "Stripe Atlas" making it easier for others to compete.

</comment><comment><USER74> I would argue cannabis is not competitive, other than getting a license. All you have to do is produce and test a product and it is usually sold. The players not making it in production usually cannot produce or have problems during production (low bag appeal, bugs, genetics, employees, quality of cure, etc). Sure, it costs a million in business licensing and fees, but if you can competently do it, you will succeed and your competition can't do much [legally] to stop you.

</comment><comment><USER75> I think there is a market for privacy.

</comment><comment><USER76> I think GIS falls into this.

<comment><USER77> Second GIS. ESRI is ripe for disruption.

<comment><AUTHOR> ArcGIS has a formidable feature-set to clone and users will want them all. I believe that's their moat?

<comment><USER78> While I don't know about GIS specifically, I work in a similar field where the market leaders have an enormous feature set accumulated over many decades - some from the 1960's and never stopped growing. But I squeezed in with the main basic features first then as I got customers, added what they wanted and continually grew it. My advantages are price and ease of use. Users didn't need to go on a training course without all those features to learn. Many of the features of the big boys are so niche that most customers never need them, or are obsolete but retained for legacy, or are just not really essential.

<comment><AUTHOR> Can I take a look at your product? I would be curious what a minimal ArcGIS would look like and how long it would take to build. GIS is super edge-casey, I think.

<comment><USER78> No sorry. I've created too embarrassing a trail of HN comments to want to link this identity to my livelihood. But it's not actually GIS, just a similar sort of product. It might be that an equally minimal GIS really too feature-poor to be any use at all.

</comment></comment></comment></comment></comment></comment><comment><USER79> Got create your own, sometimes there is a category of problem that can be solved you just need to find it, that one will be least competitive

</comment><comment><USER80> ISPs.

<comment><USER81> Google already tried and sort of failed/

</comment></comment><comment><USER82> Medical care.

You want medicine, you need to pay someone who's part of the Physician cartel, a pharmacy, a pharmacist, and whoever has the patent.

With computers and science you often need 0 of these except the manufacturer of the drug.

<comment><USER11> At least one person tried this and found it to be a very hard sell: [https://tjcx.me/posts/i-wasted-40k-on-a-fantastic-startup- id...](https://tjcx.me/posts/i-wasted-40k-on-a-fantastic-startup-idea/)</comment></comment>


Linux 3.4 kernel released - <AUTHOR> http://kernelnewbies.org/Linux_3.4 <comment><USER2> What I thought was more interesting (Linus noted "Nothing really exciting happened since -rc7...") was the exchange between Linus and Peter Zijlstra, in particular this Linus eruption:

<https://lkml.org/lkml/2012/5/19/97>

<comment><USER3>
      > And I *do* know that the real world simply isn't simple enough that we could
      > ever do a perfect job, so don't even try - instead aim for "understandable,
      > maintainable, and gets the main issues roughly right".

It's striking that his manner is blunt and over-general... and his technical content is also blunt and over-general. He's known for his pragmatic and effective engineering decisions.

<comment><USER2> Found this article citing Peter on NUMA scheduling in March:

Toward better NUMA scheduling <http://lwn.net/Articles/486858/>

Peter's cited email: <http://lwn.net/Articles/486850/>

Actually, in reviewing the data, it seems this issue has been around for a long time, and could be the root of Linus's irritation in the first place. Perhaps he feels no headway is being made for such a long time, even the smallest perceived misstep sets him off.

If nothing else, I would love to actually see some technically detailed papers come out of this that get to the root of the problem, propose solutions to it, and identify what of those solutions are being tested and when they are expected to be implemented. Nothing like a good argument between engineers to get a root cause rooted out and a solution in place.

</comment></comment></comment><comment><USER4> The X32 ABI is interesting. Who'd have thought this many years after 64 bit becoming mainstream, (and decades after it becoming mainstream on servers) something like this would be needed.

<comment><USER5> Anyone has a good argument why NOT to use this for every non memory intensive application? I always thought that using 64bit pointers was a waste of memory when not needed and this x32 abi to me seems fantastic since it keeps the several advantages of newer 64bit cpus (more registers for example as mentioned in the article).

<comment><USER6> If an application is not memory intensive, why would you care if there is 'waste'? Personally I wish everyone would just move to 64 bit all the time.

<comment><USER7> An application that uses 64-bit pointers but doesn't need 64-bits of _virtual_ address space is wasting _physical_ memory because your pointers are twice as large. The application would waste physical memory pages and memory bus traffic just to store 64-bit pointers with lots of unused zero bits like 0x00000000ffffffff.

</comment></comment></comment></comment><comment><USER8> "...making slow start suboptimal" caught my eye, but I didn't understand the rest. Is this for a specific circumstance or general? Did they find something better than slow start or did they break it, causing it to be suboptimal?

</comment><comment><USER9> Is default support for dynamic graphics switching (~bumblebee) meant to be implemented as part of the kernel or the graphics driver?

</comment><comment><USER10> Has the slow USB copy bug been fixed yet?

<comment><USER11> there is progress on this. the bulk of this bug (these bugs) should have been fixed one or three releases ago. there was a lwn article about it.

</comment></comment><comment><USER12> Wasn't this one supposed to have some Android kernel integration, too? Or maybe the next one.

</comment><comment><USER13> Btrfs is getting the love.

</comment><comment><USER14> Is it me or Linux kernel releases becoming more frequent?

<comment><USER15> It is you. Since 2005 (early days of 2.6), there has been four to five major releases per year. 3.4 is the second release for this year, so it seems to continue the trend almost perfectly.

</comment></comment><comment><USER16> I'm confused. Is 3.x meant for a different class of system, or are distros like Ubuntu intentionally lagging behind in 2.6.x land?

<comment><USER17> Ubuntu 12.04 runs kernel 3.2.

<comment><USER16> Ugh, thanks. I forgot I'm still running 11.10. I should've checked around first.</comment></comment></comment>


Ask HN: Learning to tune a database - <AUTHOR>

I've been wanting to expand my knowledge around databases and sql/tuning. I've worked mostly with Oracle and would like to learn Postgres.
My problem is: With a programming language, I can do the 'Hello world' app and progress from there. With a database I need a different problem to solve or a different dataset to work with. I'm not going to learn much about tuning if I release a little used webapp :)
I bet there are plently of datasets around that I could load and mash up but I'm not too such on where to start.
Any suggestions? <comment><USER2> Sure. Make an app that stores a very large amount of data. A good example would be spidering web data (unstructured), parsing some structure from it, and storing it in a database. Then, do some analysis on it, expose it to the web, etc.

The important thing is either to:

a) Have a lot of data.

or

b) Have a lot of traffic.

The first is a lot easier to get than the second.

<comment><USER3> And both are simple to simulate :)

</comment></comment><comment><USER4> I think you want more than a lot of data.

That's a good start, but you also want it to be structured in a way that requires some thought (or just in 3+ normal form). Have someone ask for a report out of it (or come up with as many reports as you can).

I wish I had a suggestion for such a data set. Microsoft's famous case is Northwind, but I don't know where you might find it (without having searched just now).</comment>


4chan is running out of money - <AUTHOR> http://www.theverge.com/2016/10/3/13155072/4chan-struggling-with-hosting-costs <comment><USER2> 4chan is a phenomenon. It's the complete antithesis to centralized, regulated public forums like Facebook that demand all information about your identity and keep permanent records on everything you've ever said and done. It's the last vestige of pre-"Web 2.0". While there have been countless horrors that have arisen from 4chan, part of me would be sad to see it go. Most of the Internet culture of the last decade has been defined by 4chan.

Would a switch to a distributed model of hosting have any chance of working? Chan@Home? I'd imagine the latency would be a lot higher, and it'd probably be ever harder to moderate than it is now.

<comment><USER3> 4chan depend on a central server, which is why lack of funding can cause a failure.

 _Usenet /netnews_ is the complete antithesis to centralized forums.

</comment></comment><comment><USER4> Nishimura is the founder of 2channel, the Japanese website than 4chan began as a direct clone of. He is known for doing sketchy stuff in order to make a buck, and when he bought 4chan, many users speculated that he would mess with the site in some way to make a profit. I don't think anyone expected him to flat-out suggest malicious ads to his users, though.

<comment><USER5> >Nishimura is the founder of 2channel, the Japanese website than 4chan began as a direct clone of.

No.

Amezou(person) started the Amezou. Nisimura Hiroyuki cloned Amezou and started 2ch.

Then, nameless person cloned 2ch and started Futaba channel.

4chan cloned from Futaba channel.

</comment></comment><comment><USER6> The king is gone, but not forgotten this is the story of Johnny Rotten it's better to burn out than to fade away...

</comment><comment><AUTHOR> Another discussion: [https://news.ycombinator.com/item?id=12635999](https://news.ycombinator.com/item?id=12635999)

</comment><comment><USER7> I thought there were a bunch of "chans". Won't the others just replace 4chan in user share?

<comment><USER8> >Won't the others just replace 4chan in user share?

Due to its position as the nexus of "internet culture" for at least a few years in the late 2000s, 4chan is very special. It is one of two major general purpose chans (the other being 8chan). All other chans, including 8chan, define themselves along the lines of "4chan but...", "4chan without...", "4chan with...", or similar. And if they don't now, they started that way.

8chan is the only one that has gained and kept a large enough dedicated userbase for significant content and memes with broad appeal to originate from there. This is a unique consequence of its origin.

Most living alt chans are specialized supplements to 4chan (the "4chan with..." category), and make no apologies for it. Many 4chan regulars frequent one or two: I enjoy cyberpunk literature and content, there's a chan for that. Some people hate the lack of a board dedicated to drugs, there's a chan for that. But most still go to 4chan, at least nostalgically, often in search of a long-gone "board culture" that we loved years ago. And 4chan remains the single point where chan culture blends, posting style is standardized, and memes catch fire.

8chan was not born as a supplement to 4chan. Like many failed chans, it was created when 4chan moderators did, or were perceived to be doing or planning, something to anger users. But 8chan and the controversy that popularized it were different. Usually moot's missteps become jokes. GamerGate convinced a significant number of people (more precisely, a subset of white male teenagers, henceforth "gators") that there really was a vast conspiracy to indoctrinate the youth using video games. 4chan's moderators were judged to be part of the media element of that conspiracy. From that point, 4chan permanently lost the trust and loyalty of gators. They went to 8chan, closely followed by far-right goons who take /pol/ very seriously and would become inconsolable when moot did routine pranks like background music or wacky CSS. Somehow this was a diverse enough group (in interests, not background) to put 8chan in direct competition with 4chan.

If 4chan folds, I predict 8chan alone will get the bulk of the users unless a direct spiritual successor emerges. Board closures probably wouldn't matter to users, neither would ads or upload limits. It's more a question of whether those measures would actually solve the money problem.</comment></comment>


Artificial Intelligence: The Pros, Cons, and What to Really Fear - <AUTHOR> https://www.defenseone.com/feature/artificial-intelligence-pro-con/ <comment><USER2> The formatting of that site is really strange. If I make it a full desktop size (1920x1080) it becomes more readable.</comment>


TensorFlow Benchmarks - <AUTHOR> https://github.com/soumith/convnet-benchmarks/issues/66 <comment><USER3> Is this surprising? I think it's expected, when most published numbers from Google suggest that this is only 2x over DistBelief, which Adam Coates improved upon 6x ([http://www.cs.stanford.edu/~acoates/papers/CoatesHuvalWangWu...](http://www.cs.stanford.edu/~acoates/papers/CoatesHuvalWangWuNgCatanzaro_icml2013.pdf)) in 2013. Coming from a HPC background, DistBelief (and sometimes MR in general) has always felt like a square being forced into a round hole. Sure it "scales" but it's not how you would build a system from scratch to meet the same computational needs. Moving data around is much slower than doing extra math.

DNNs provide some interesting opportunities for domain specific optimization with lossy math because empirically it seems that the solutions dont require significant precision. Something like a general Tensor library holds you back from using some of the tricks like synchronization-free hogwild, where it wouldn't be appropriate in other scenarios where more exact computation is necessary.

One interesting step towards this is their implementation of lossy compression with a custom 16-bit half float representation. However a lot of people are dialing down the precision even further, including one of the authors of TensorFlow ([http://petewarden.com/2015/05/23/why-are-eight-bits- enough-f...](http://petewarden.com/2015/05/23/why-are-eight-bits-enough-for- deep-neural-networks/)).

Scott Gray, one of the principle engineers at Nervana Systems, has a very wise reply in the github thread - they aren't doing much to optimize for memory locality, and wont have amazing speedups until that happens.

</comment><comment><USER4> From Google's perspective it is probably more about how TensorFlow scales out horizontally. If a researcher fires off a Borg run (or whatever they use now) and the job takes a few thousand CPUs, no problem, at least for research.

They must have better optimization a for running in production, such as in place operations.

<comment><USER5> I'm going to say something unpopular, but horizontally-scaled deep learning is overkill for most applications.

Can anyone here present a use case where they have personally needed horizontal scaling because a Titan X couldn't fit what they were trying to do? It's the thing I hear talked about the most and used the least.

The biggest misunderstanding I've heard is "I have petabytes of data so I need multi-GPU", but NNs are trained on batches of data, so it doesn't matter. It's the model that must fit, but 12gb can fit a big model.

The whole "who can build the biggest net" is like building the fastest car: it's for show, the slower ones are way more cost effective and get you there just as well.

Keras is my present deep learning library recommendation--it's like torch 7 but in python and theano. Multi-GPU is roadmap because there's a lot more going on in deep learning that acts as a bigger differentiator than horizontal scaling.

For the record, when I say scaling, I'm not talking about having multiple machines serve multiple requests (that's a good reason to purchase more machines), I'm talking about spreading your gradient computations and/or model weights across machines or GPUs.

<comment><USER6> Would image training for video recognition be a good example of this?

<comment><USER7> Our experiments might support this. With 4 or 8 titan X, training time is shortened from more than one week to one or two days. code available at [https://github.com/yjxiong/caffe](https://github.com/yjxiong/caffe)

<comment><USER5> Note: data parallel vs model parallel.

Data parallel is easy to implement and leads to linear training time speed ups, as per 1 week to 2 days with 4x hardware. But not bigger models.

Model parallelism leads to bigger models and is what I've been referring to. It is overkill and doesn't work well anyway. Frameworks should not be expected to support it because there are more interesting topics in deep learning they could support instead.

Better research/methods will come out at some point, at which point this calculus will change, but not yet! Today it is the very definition of premature optimization in nearly all cases.

</comment></comment></comment></comment></comment><comment><USER8> Until now, I've seen two responses to Google's TensorFlow from Facebook employees. Yann Le Cunn seemed to really challenge Jeff Dean about TensorFlow's scalability [1] and this benchmark puts TensorFlow down there in all the measures it tested for. I can't ignore the possibility that this criticism of TensorFlow from Facebook employees (while factually correct and constructive) might be driven by some competition and jealousy.

[1] [https://www.youtube.com/watch?v=90-S1M7Ny_o&t=39m](https://www.youtube.com/watch?v=90-S1M7Ny_o&t=39m)

<comment><USER9> This benchmark basically shows that releasing TensorFlow with cudnn v2 backend support hurts - v2 is quite a bit slower than v3 (current) and v4 (upcoming). TF has announced that they will update to v4 support, which should help quite a bit - but when many hobbyists and researchers are developing on one or two GPUs performance on that scale is more important (for them) than infinite scalability.

It is not surprising that a tool developed and focused on `Google scale` work has some imperfections in a wildly different setting. The question is - <USER2> they (or some dedicated contributor) speed up a use case the business itself may not have a use for? My gut feeling is that they will, but these things usually don`t happen overnight. Torch, Theano, and Caffe have years of work put into them and have largely been focused on the one to two GPU case.

<comment><USER10> I've done an apples to apples, TensorFlow + CuDNN R2 vs Torch + CuDNN R2

<comment><USER9> My point is more that _even_ if they were on the same footing from benchmark timings, v2 is still far behind what is supported in Torch, Caffe, and Theano right now (v3 in all IIRC). Your comparison is very fair, and it is good insight!

</comment></comment></comment></comment><comment><USER11> I don't know how much this matters, but an issue similar to this had been raised here: [https://github.com/tensorflow/tensorflow/issues/120](https://github.com/tensorflow/tensorflow/issues/120)

The responses seem to show that the way you implement things can make a big difference in runtime. Perhaps the scripts used for benchmarking can be further optimized?

That said, the lack of in-place operations might be surprising (although it has been said that they are coming)

</comment><comment><USER12> Interesting benchmarks. One hopefully constructive critique, if you say things go out of memory, it'd be really useful to know what your setup is. Maybe you've got a big array of massive GPUs or you're running it on a more normal consumer GPU+box.

<comment><USER13> According to: [https://github.com/soumith/convnet- benchmarks](https://github.com/soumith/convnet-benchmarks), it's an NVIDIA TitanX (12GB GPU memory), which is pretty much the top of the line GPU for training neural nets. From my limited experience, most deep learning in research - including much of the state of the art - is done on single GPUs. (My guess is that if your model doesn't fit in 12GB, your model has way too many parameters to practically train anyway).

<comment><USER12> Ah thank you for that, I hadn't clocked on that this was opened by the person who ran the repo. I should have checked the main readme.

</comment></comment></comment><comment><USER14> It takes 4x as long as CuDNN on AlexNet due to lack of in-place ops. What is up with that?

</comment><comment><USER15> In the announcement they called this the "reference implementation".

</comment><comment><USER16> It's almost like Google wanted everyone to use slow obsolete software and keep the _really good stuff_ for itself, while still making it look like they're doing a great thing for the community.

<comment><USER17> I think it makes more sense that:

a) It's Google, they can throw hardware at a problem such as out of memory

b) TensorFlow makes methods development so much easier that it's worth the loss of performance

c) It's early days and the compute graph scheduler has lots of opportunities, and is designed, for optimization, and in a more flexible fashion than other frameworks.

When I work on developing methods for scientific code, I worry more about whether the code is bug free/easy to understand and that it's giving the right answer. I don't usually worry about performance unless I'm actually not able to run things. Especially since when developing stuff you waste way more time on runs with bugs - I dread to think what my (published / unpublished) CPU hour ratio is. If the new approach allows less buggy implementations then that's a resource win.

<comment><USER12> > b) TensorFlow makes methods development so much easier that it's worth the > loss of performance

Indeed, if TensorFlow means I can try out an idea with 1 day of coding and 2 days of training rather than 3 days of coding and 1 day of training then I can spend a day drinking cocktails and reading books and still be finished sooner.

Based on the tutorials it seems like I'd be able to pretty quickly build a translation pipeline, and in fact there's an implementation of that I think I'll try. If it takes a week or two to train, that's fine by me, I've got other things to be getting on with.</comment></comment></comment>