Google embraces, extends, and extinguishes - <AUTHOR> https://drewdevault.com/2018/05/03/Google-embraces-extends-extinguishes.html <comment><USER2> Hey, I was just talking about this in a comment the other day! I totally agree with the author here. I was surprised not to see any mention of cloud facing open source products. Kubernetes seems like a prime example. Google lost its containerization first mover advantage to Docker (remember google was famous early for its internal use of containers). So they targeted the next level up the stack of container orchestration. Kubernetes is a great project, but its clear priority is compatibility with the cloud computing business model. It didn't take long for google to offer GKE on top of kubernetes. At least for now, there is feature parity between open source and hosted versions. But I don't expect that to last long. I would be wary of the kubernetes project introducing features that only work well in google's cloud.

The main externality of this open source EEE strategy is developer opportunity cost. When google swept into the chat market, they used their size and first mover advantage to push an open standard, XMPP. So developers who might have worked on other projects/standards naturally gravitated toward XMPP and contributed their time there. If those developers had known the future of XMPP, they likely would not have worked on it. They could have invested time in a competing, more open project instead.

This to me is the real danger. It's bad enough that google gets to benefit from thousands of free contributors to its "open source" code. It's even worse that they are taking those developers away from other projects, so that when google moves to the third phase of EEE, there are no viable alternatives.

<comment><USER3> I have always felt that, in a "typical" open-source project, I could approach the community, join it, build some rapport, and eventually help with the decision-making process (assuming I was any good and people in the community liked me).

But with Google-run (or other company-run) open-source projects, it's a completely different story. Want to be a TensorFlow maintainer? Great, sure, you can contribute, but you'll never get to make any decisions about the direction of the project itself unless you're inside Google.

To me it's a disappointing semi-abuse of the term "open source"... it doesn't mean what I think it should...

<comment> FLUX-YOU If you are good enough to contribute to Tensorflow and be able to come up with new directions, why not just get a job at Google so you can do that?

This isn't unlike any other open source project ran by people who don't want to consider all input from others. Not all suggestions are worth serious consideration, after all. Open source doesn't mean I'm beholden to your every whim just because you've contributed. That's what forks are for.

<comment><USER3> > why not just get a job at Google so you can do that?

Don't want to move to California? Can't pass the interview? Salary too low in the offer? There could be lots of reasons.

I agree with your other points, but what I'm saying is not exactly that. What I'm saying is that TensorFlow is not a _community-led_ project in the way that other open-source projects are. It's _Google-led_ , and if you don't like it, you can go away or fork it (...and if you fork it, it's unlikely you'll get many users given the Google marketing machine).

(The same complaints have been and are being regularly made against Red Hat, and also Fedora, given that Fedora is basically "experimental" Red Hat.)

<comment> FLUX-YOU Convince Google it's better as a community-led project. Maybe they will have some convincing counter-points about why it's better the way that it is that we're not considering. They've likely already been through this debate.

Consider if Tensorflow were half as useful as it is today if it was community- led and it was led by a shitty community.

</comment></comment></comment></comment></comment><comment><USER4> In 1998, Brin and Page admitted that the future Google would biased and poor quality:

"The goals of the advertising business model do not always correspond to providing quality search to users."

"We expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers."

"Advertising income often provides an incentive to provide poor quality search results."

"Since it is very difficult even for experts to evaluate search engines, search engine bias is particularly insidious. A good example was OpenText, which was reported to be selling companies the right to be listed at the top of the search results for particular queries. This type of bias is much more insidious than advertising, because it is not clear who "deserves" to be there, and who is willing to pay money to be listed."

[http://infolab.stanford.edu/~backrub/google.html](http://infolab.stanford.edu/~backrub/google.html)

</comment><comment> yoz-y One thing I would add is constantly pushing the web standards forward with little regard to other players. They are adding features so fast that only the biggest structures can keep up. Many of these seem to exist just so ChromeOS can actually be considered a real OS.

Writing a secure and up to date HTML rendering engine from ground up is now almost impossible (kudos to Mozilla to actually doing that).

<comment><USER5> The problem is that they push "standards" but the implementation of the standards often does not function as written.

Typically standard bodies avoid this by requiring two independent implementations, but the market situation really means that the standard is what Chrome implements, and nothing else.

<comment><USER6> They also ignore many things they push.

</comment></comment></comment><comment><USER7> Google's approach to open source does feel very much like virtue signaling rather than constructive adherence to the ideology. The build system for any Google project is notoriously terrible. I recently attempted to build Skia; even after rereading the scant instructions, I could not get it to do anything better than a compilation error.

<comment><USER8> I think this stems from their internal monorepo culture, e.g. they care about building everything in one go, not in isolation. This can also explain the issues with golang dependencies, it's hard for them to see problems of most regular developers as they use a unique toolchain (custom SCM, custom build tools etc.)

<comment><USER9> Not Invented Here (tm) seems like a large part of the cultural problem with being better open source contributors.

It's great they open source a lot of their stuff, but just because you're big enough as a company to rewrite {component of build chain} for yourselves... doesn't mean you should.

Caveat: I know parts of Google are awesome with contributions and using standards.

<comment><USER10> Biggest problem I think is googlers can't use code if it's licensed under AGPL.

edit: GPL -> AGPL

<comment><USER11> The license we (I am an engineer at Google) cannot use is the AGPL.

</comment></comment></comment></comment></comment><comment><USER12> > it's infamous for having an extremely strict spam filter for incoming emails > from people who run personal or niche mail servers.

I run my own mail server and I have zero problems with reaching Google Mail users. Google – like other big mail providers – just expects you to set up DMARC and DKIM authentication for your server, which is not hard to do. Linode already provides a convenient guide to setting this up for any Ubuntu or Debian installation.

</comment><comment><USER13> There are also a view counterexamples, for example that Google abandoned its native client in favour of webassembly when the standard took form. The same with progressive web apps which will replace the "web apps" in Chrome.

I was also sad when Google Talk was shutdown for Hangouts but to be honest, XMPP was going nowhere. The quality of available clients was very different. You had awful clients like Apple iChat (which is still awful), complete client like Psi which stopped at some point, clients which wanted to be everything like Adium/Pidgin, etc. pp.

Google Reader was indeed sad and really weird, they could have build on top of their social networks they already had but they wanted something new and shiny. I still look for a good replacement.

<comment><USER14> The Google Reader one is so baffling and love to hear the real story at some point.

<comment><USER15> Boring story. Vic Gundotra, the clueless jackass from Microsoft that Google made SVP for Social did not think RSS fit into his idiotic vision so he shut it off.

</comment></comment></comment><comment><USER16> _> These days, Microsoft seems to have turned the other leaf, contributing to a huge amount of open source and supporting open standards, and is becoming a good citizen of the technology community._

In some cases, but not in others. MS never joined the Vulkan working group, instead insisting on its DirectX lock-in. MS's Outlook / ActiveSync is still not an open standard which makes it a major pain to use. exFAT isn't an open standard despite MS pushing it on SD cards storage. And so on and so forth.

And XMPP story isn't any better either. Google should be blamed of course for deserting, but so is MS who never joined the effort to begin with.

</comment><comment><USER17> I think Google are just terrible at product, plain and simple.

I just looked at the new YouTube Creator Studio interface, and it's literally the biggest UI and UX abomination I've seen from a major tech company in recent memory. It's worse than the iOS 7 redesign.

<comment><USER14> Depends on the product. We just got YouTube TV and just love it and same with Google Photos. But then there are other Google products that are poor and been for years.

They are now a pretty big company with a lot of different teams so not surprising some good and some bad.

<comment><USER17> It's weird, because Google clearly have great designers and do some great work, but YouTube (as an uploader) is just the worst.

It's great that YouTube for PS4 sorta works now, though.

I do wonder why there's this crazy variance in quality.

<comment><USER14> Must use difference teams for YouTube versus YouTube TV.

YouTube TV is excellent.

</comment></comment></comment></comment><comment><USER18> This article reminded me of an old quote from J Allard, a former exec from Microsoft. Replace Microsoft/Windows with Google, and the result is scarily accurate.

"In order to build the necessary respect and win the mindshare of the Internet community, I recommend a recipe not unlike the one we've used with our TCP/IP efforts: embrace, extend, then innovate. Phase 1 (Embrace): all participants need to establish a solid understanding of the infostructure and the community — determine the needs and the trends of the user base. Only then can we effectively enable Microsoft system products to be great Internet systems. Phase 2 (Extend): establish relationships with the appropriate organizations and corporations with goals similar to ours. Offer well-integrated tools and services compatible with established and popular standards that have been developed in the Internet community. Phase 3 (Innovate): move into a leadership role with new Internet standards as appropriate, enable standard off-the-shelf titles with Internet awareness. Change the rules: Windows become the next-generation Internet tool of the future."

</comment><comment><USER19> Any company that grows this big will behave the same. The only solution for consumers is to not create companies this big. Always also buy from competitors.

<comment><USER20> Any company this big needs to have some outside control. They are clearly monopolizing. I thought the US had a law against monopolies, or does it only apply to the railroads?

<comment><USER21> > I thought the US had a law against monopolies,

It's more complicated than that. US law tries to only target "bad monopolies" that engage in anti-competitive practices or hurt consumers: [https://www.investopedia.com/insights/history-of-us- monopoli...](https://www.investopedia.com/insights/history-of-us-monopolies/)

<comment><USER22> In reality, monopolies like Google have repeatedly demonstrated anti- competitive practices and consumer harm. The unspoken reality is: We simply don't do antitrust anymore.

</comment></comment></comment></comment><comment><USER23> Google's approach to open source may be about as transparent as North Korea, but I do appreciate their contributions. VP8 and VP9 never really had any competition in their category, and their collaboration with the AOM to ship AV1 appears to have been fruitful.

<comment><USER24> Could you expand on this a bit? We've tried to be very transparent with how we approach and think about open source by publishing all of our internal documentation at [https://opensource.google.com/docs/](https://opensource.google.com/docs/). Is there something that's missing from that? Or are you thinking about certain specific projects being opaque in terms of how their managed?

<comment><USER25> The default google approach to open source is unidirectional source dumps after all the work is done. On some products you do a bit better, but still keep your cards incredibly close to the vest, and maintain very tight control. Even when you're trying not to, you can't break from that mold.

I've got this comment saved on my reddit account because it was so funny:

[https://www.reddit.com/r/programming/comments/53gb56/why_lea...](https://www.reddit.com/r/programming/comments/53gb56/why_learning_angular_2_was_excruciating/d7szso5/)

<comment><USER24> > The default google approach to open source is unidirectional source dumps > after all the work is done.

I can definitely say that is not our default approach to open source; in fact it's a very small minority of projects that actually operate in that fashion. However, I can understand that it could feel that way to some people.

We've long said and continue to believe that there is no one way to do open source. That's true of nearly every aspect of a project including licensing, governance, community management, etc. Project are released for different reasons, with different motivations and goals, and so the way they are managed often differs.

One area I know we could do better is to set better expectations for projects around many of these topics. How is the project managed, how are decisions made, how committed to this are we (ie. are we using it in production), etc? If those aspects of a given project were clearer, would that address some of your concern (with the understanding that some projects may be be held closer to the vest than others) ? Or are you objecting to the tighter control in general?

<comment><USER25> I jumped into this thread just to answer your question, I don't have a personal objection to how Google approaches open source. You can be as private as you want or have whatever project governance you want.

I posted because I find it alternatingly funny and frustrating that Googlers don't understand (or can't publicly admit) what Google is to the outside. I'm willing to believe that your comment is your own words. The problem is that you might as well have copy/pasted it from a social media playbook.

It's Google's prerogative to run the business however they want. But you pretend not to be a black hole. All companies are black holes, and that's fine. It's the dishonesty in external posturing that gets to me. Sometimes it's acting like a given opensource development process is more externally accessible or transparent than it is. Other times it's pretending like the youtube appeals process isn't a fake website to con the dumb people. Everyone knows the real appeals process is to work your professional contacts until you find someone who can reach out to a real human in google to check if they should over-ride the automated system (and/or the hourly drones that DGAF about doing their jobs well). -- That was weird tangent, but it's the thing least tied to software development process I could think of.

I know you can't admit it outloud on Hacker News, but I hope you would have had a different response if we had this conversation at a bar rather than in front of everyone like this. As long as you can admit that to yourself, privately, you're alright with me.

</comment></comment></comment></comment></comment><comment><USER26> To my memory Gmail has always had a very strict spam filter. At what point did it turn into "extinguish"?

<comment><USER27> The complaint is that they're extending the email standard by sending single- URL bodies instead of the messages inline. I can see the point, but sending URLs is still fine in my book, and this is a useful feature that's trying to get parity with Outlook.

<comment><USER28> Including not being able to read them when using a 3rd party client?

<comment><USER26> If it's just a URL, why wouldn't that work in other email clients?

(Though personally I'm not a fan of the idea itself)

<comment><USER28> Outlook has a way to send such messages that you only see a Webmail.dat attachment when using other clients, hence why I was jokingly asking if that was also part of achieving parity.

<comment><USER26> Ah, missed that sorry. Thankfully haven't received such a file yet.

</comment></comment></comment></comment></comment></comment><comment><USER29> This does feel like a familiar turning point for google- how long until they are the slow moving incumbent in the market and ripe for disruption?

If you're thinking about taking on google in a specific product area, now is probably a good time to start.

<comment><USER30> An ad blocker + reader view could very well take care of most slow loading web pages. What are the chances that Google will build an infrastructure that allows third party ad blockers to work with Chrome on Android? Yes I know that you can download other browsers,but embedded web views still use Chrome.

<comment><USER6> Google is experimenting with putting ad content seamless together with regular search results so an ad-blocker will not help you at all.

<comment><USER30> It's not about as blocking on Google. It's about the ads on third party sites (including Google ads) that make the web so obnoxious.

</comment></comment></comment></comment><comment><USER31> This is true, not just for technical side, but also on business side.

Google just did it again on the Ads business:

[https://adexchanger.com/online-advertising/googles-gdpr- cons...](https://adexchanger.com/online-advertising/googles-gdpr-consent-tool- will-limit-publishers-to-12-ad-tech-vendors/)

</comment><comment><USER32> Does Gmail still cause trouble with vanilla IMAP clients? I remember something about how "tags" in Gmail showed up as folders in the client, but since emails could have multiple tags it caused lots of problems.

(I use Fastmail, an email service that's much better and standards compliant.)

<comment><USER22> It does. Deleting things in an IMAP client usually means "removing the tag corresponding with that folder" in Gmail. There's a hilarious tendency for things you delete in IMAP to just be in "All Mail" with no labels in Gmail, for instance. I used to use Gmail on a Windows phone for a while, and the issues were painful.

Note that FastMail is developing JMAP, a replacement standard for IMAP. And even though FastMail itself doesn't support labels, they've made sure their new standard can support Gmail's currently proprietary way of handling mail correctly. Hopefully JMAP will take off and we'll see better email clients in the future.

<comment><USER16> You need to configure your client to do expunge in addition to delete.

</comment></comment></comment><comment><USER33> I wonder if their attempt to force HTTPS everywhere (by shaming HTTP-only sites as insecure in Chrome) fits into this strategy.

It would put net neutrality violators like US ISPs out of the mass surveilla- ..eeeh advertising business, so I guess that is a good thing.

<comment><USER14> Think driven to keep people safer and the same with Project Zero. A safer digital world and people will use more and Google then makes more money.

My gripe with the broader industry is that Google is finding a lot more of the big vulnerabilities than their competitors.

Should be a lot more balanced. I can not even think of one major one found by Apple. Yet Google has found so many of the really big ones. Today see an article they found additional.

<comment><USER34> What exactly should Google do differently here, when they find bugs in other companies products?

<comment><USER14> The point is spending the money to find them.

I personally believe MS should have done this years before Google.

</comment></comment></comment></comment><comment><USER35> On open source and standards Google has been a leader. Contributions:

1\. VP8, VP9 and 90% of av1

2\. WebRTC

3\. Nacl/Pnacl that was killed by W3C for no technical reason, in favor of an inferior wasm

4\. SpyDy inspiring http2

5\. V8 js runtime

6\. Angular js

7\. ProtoBuf and grpc

8\. Android

9\. Huge contributions to the looking kernel, and many many open source libraries, like boring ssl

10\. Brotli compression and other related tech

11\. Go language

12\. Tensorflow

13\. ...

It's a bit sensationalist to label Google an "extinguisher". If anything, they're a huge enabler in software.

Now, all the above doesn't make itself. A company pays competent software engineers to work on this full time. I'm in complete support of Google's attempts to monetise Gmail, one of their largest services. After all someone needs to pay for those engineers, and most of all, Google is a for profit enterprise

You're free to transition to any of the other million standards compliant providers, there's no data lock-in. You can download your email via imap, another standard that was pushed by Google when the competition was only offering web-email

<comment><USER36> > 8\. Android

Have you tried to build pure AOSP lately? Hint: you pretty much can't, it's broken. Google plays super dirty with manufacturers that want to use Android.

</comment></comment><comment><USER37> >and contributing to Chromium as a non-Googler is notoriously difficult.

so fork?

<comment><USER38> Forking a big, fast moving, security critical piece of software like a browser is totally impractical unless you have a sizable company behind you to maintain it.

<comment><USER39> So what? Lots of things are impractical for individuals.

<comment><USER40> It's not just individuals. If people can't contribute to the project it is hard to build or maintain knowledge about that technology so another party can use it at all. Sharing code that few people de facto can use shouldn't give you much credit, or at least not as much as the alternative. Unless it is, as someone suggested, about signaling.

</comment></comment></comment></comment><comment><USER5> _Microsoft...is becoming a good citizen of the technology community_

By displaying deceptive ads about competitors in their flagship product?

Get out.

<comment><USER41> To be fair, Google did the same thing. When chrome was new, they had banners on google.com suggesting you "upgrade" to Chrome for a better experience. And the banners weren't only shown on outdated/old browsers (say, IE); they were shown on Firefox too.

<comment><USER22> They also show for Edge, of course. And don't go away no matter how many times you dismiss them. Firefox is currently my main browser, and there are only two things left on the Internet which truly annoy me.

\- "Upgrade to Chrome?"

\- "Subscribe to YouTube TV?"

No matter how many times you say "No", they'll both come back. And you know that's not bad code, because YouTube literally said it was part of business plan to annoy people until they pay up.

[https://www.theverge.com/2018/3/21/17147800/youtube- streamin...](https://www.theverge.com/2018/3/21/17147800/youtube-streaming- service-lyor-cohen-ads-music-industry-spotify-free)

</comment></comment></comment><comment><USER42> Write more AGPL licensed code ;)

</comment><comment><USER14> This is a very goofy article. Google has open source so many things that it is a bit crazy they give away. But who else open sources their OSs? Both Android and ChromeOS.

But the craziest one is giving away Borg. Probably the most incredible technology Google has created and just opens sources it.

For me personally the biggest contribution from Google is

[https://research.google.com/pubs/papers.html](https://research.google.com/pubs/papers.html)

Kind of the opposite of what this article suggests.

I get some of the alt right has a problem with Google firing Damore but this article is ridiculous, IMO.

<comment><USER43> First, How does this have anything to do with the alt-right?

That's just a distraction from your argument that we shouldn't criticize the actions of Google because they give away some research. Google, and others, have turned open source into a free marketing campaign. Most of it is only open source in the "source available" sense.

You could fork it, but the sheer size and magnitude of insider contributions makes it as useful as packaged software that came with source code.

<comment><USER14> It appears a lot of the negative Google stuff I read is driven by emotions upset with the alt right thinking they are against them. Seems to have started with firing Damore.

I have ZERO problem criticizing anyone. But if we look at all the tech companies there has never been any tech company that give away more IP than Google. Nobody even close.

So my post is a lot more about removing the social politics emotions and look at actual data.

It is crazy giving away Borg and the K8S trademark. But Google has open sourced so much stuff. We have five OSs today. iOS, OS X, Windows, Android and ChromeOS.

Two are open source. Yes I realize there was a OSS of Apple OSs but that is NOT what is used today.

But then Chrome and well so many other things. But look at giving away Map/Reduce and GFS and so many other things in papers.

I tire a bit of the bitching on HN and really worry Google will end giving away stuff. It is hard to see why they continue. There is little in it for them.

<comment><USER30> _have ZERO problem criticizing anyone. But if we look at all the tech companies there has never been any tech company that give away more IP than Google. Nobody even close._

Google only "gives away" IP that isn't part of their core business - just like Apple, Amazon, and Microsoft. Where is the code for Google crawler and search algorithm? Gmail? Google Play Services? The back end for Google Docs?

<comment><USER14> Would argue nothing is more core than Borg. So would disagree.

I do think they are holding back some stuff and a big one is SDC stuff.

Plus we do not get the info until next release. So we got gen 1 TPUs after Gen 2. We will probably get gen 2 details with gen 3 release.

But Google shares so much more than any other tech company. I think possibly ever?

<comment><USER30> What has Google open sourced that would put it at a competitive disadvantage if a competitor uses it? Android? Earlier on they really didn't have a choice if they wanted to compete with Microsoft with the OEMs. When they became basically the only game in town for device manufacturers, they closef off much of what made Android Android to most people. Even Apple open sources stuff when they want to gain traction in a market as does Microsoft.

How is Google's strategy any different from Microsoft's and Netflix's? It would have been unthinkable a few years ago that MS would open source .Net and make it cross platform or gone through the trouble of creating Visual Studio Code that supports almost every popular language.

<comment><USER14> Android and Amazon using for the Echo and much of their hardware?

<comment><USER30> Google didn't have a choice but to "open source" Android to compete for OEMs against Microsoft. But as soon as they gained dominance, they started closing off more and more of what makes Android what it is to most people. Look at how many of Google Services are not only not available on Amazon devices, Google actively blocked YouTube from being on Amazon devices.

Besides, Android itself doesn't make Google money - advertising and to a lesser extent Google Play does. Why else would Google pay Apple $2 billion a year to be the preferred search provider on iOS?

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER44> >The "extinguish" with GMail is also well underway - it's infamous for having an extremely strict spam filter for incoming emails from people who run personal or niche mail servers

And thank God for that. That's why their antispam is sooo good.

<comment><USER45> Google didn't invent the reputation-based antispam, or the DNS blacklisting, or all the annoying antispam features that make running your own email server difficult. The only thing Google added to the mix was applying its famous quality of customer service (or rather, lack thereof) to the problem.

<comment><USER34> At the time I signed up for Gmail they had by far the best antispam filter I'd seen (online or local). I would regularly have ~1000 spam emails per month in my spam box and false positives or negatives were extremely rare (~1/year?)

<comment><USER46> How would you know about the emails to you that were blocked and false positives?

<comment><USER34> They went into the spam box.

</comment></comment></comment></comment></comment><comment><USER47> Google gets 97% of its income from selling ads and analytics, everything else they do or produce exists only to serve this goal and to strengthen their monopoly and entrench their position even further. A browser to dictate internet standards so they can push towards more surveillance, a mobile operating system so they can control most of the world's mobile devices with an iron fist, email service and cloud storage platform so they can get better profiling data and so on.

Any code that is shared for free under the guise of "open sourcing" is most of the time either a byproduct of their internal research that will not lose any value by sharing it (or will actually make non-employees extend it and work on it for free for Google's benefit), or something so specific to the company that nobody stands to gain anything from adopting it.

Google is first and foremost an advertising and user profiling company, it is useful to think about them from this perspective as it explains most of their actions with perfect clarity.

<comment><USER14> Actually 88% but close.

<comment><USER47> At their size it's hard to estimate but it's definitely in the 90% range. I've seen estimates of 85-99% of revenue coming from advertising.

<comment><USER48> You don't have to estimate it. They put the actual number in their earnings report.

<comment><USER14> They do on revenues but not for earnings.

</comment></comment></comment></comment></comment><comment><USER49> Am I missing something here? Just seems like moaning. And the irony of the link on the side of the page to a Minecraft clone that's been embraced, extended and then abandoned because the author just can't be bothered any more (but you can send him money and maybe he'll change his mind) isn't lost on me.

<comment><USER50> > link on the side of the page to a Minecraft clone that's been embraced, > extended and then abandoned

I'm not sure how a person doing a clone of something owned by a billion dollar corporation (Microsoft)...and then giving up on it...is the same thing. At all.

</comment></comment><comment><USER51> They're extinguishing all the parts of the web that I didn't like: email spam, slow web page loads on mobile devices, etc.

Hope they keep up the good work.

<comment><USER5> _slow web page loads on mobile devices_

Mostly slow because they need to display a lot of...Google Ads?

</comment></comment><comment><USER52> AMP's point isn't to "reward" faster pages. The mobile web is garbage and 80% of pages are hot dumpster UX fires that take several second to load.

AMP's point is to help standardize the excellence of the few pages that actually are performant. Any rankings boost was a carrot for early adopters. In july, page speed will have greater importantance ranking for pages generally.</comment>


Show HN: A JavaScript budgeting app that implements a KNN to make predictions - <AUTHOR> https://github.com/victorqribeiro/budget <comment><USER2> It's an interesting assumption to correlate prices with categories, I think this'll suffer greatly from omitted variable biases. If I end up having to check each time whether or not the number entered has changed the category, then I'm not sure that it's better than just keeping in manual.

<comment><USER3> Well, it could at least sort for categories with the highest possibility.

<comment><AUTHOR> That would be great.

</comment></comment></comment><comment><USER4> That's neat. I am thinking about building something similar just with scanned invoices. I hope that using KNN on some image features can auto-classify the invoice into existing categories well enough...

<comment><USER2> Not very common for personal finances but there are already services (AutoEntry, Trifact365 etc.) that handle this kind of categorisation for small businesses with OCR, no need for NN.</comment></comment>


Marissa Mayer's Plan to Sell Yahoo but Stay in Charge - <AUTHOR> http://www.vanityfair.com/news/2016/03/marissa-mayers-plan-to-sell-yahoo-but-stay-in-charge <comment><AUTHOR> I'm sure it will be a great morale booster for the Yahoo team to know Marissa will continue being their CEO.</comment>


Ask HN: DIY Patent - <AUTHOR> Has anyone completed their own patent application - without any legal or professional counsel?<p>Was it reasonably easy - Or a complete nightmare and not worthwhile on your own? <comment><USER3> Yes it is easy. The way to do it on your own is just copy the format and claims from other published patents. Don't worry about plagiarism - <USER2> attorneys certainly don't in practice.

The bigger issue is finding all the relevant references. The easiest way is find a half dozen patents similar to yours and just copy the references. The examiner will find the rest for you.

Where you will run into problems is at the examination stage where the patent examiner will try to throw your patent out and/or exclude all your claims. You just spend a lot of time arguing with them and eventually you reach some sort of compromise.

One approach I do think is a good idea with self-patenting is to create a thicket of patents. Put in lots of simple, short patents with very limited claims (say 3 to 4). The basic idea here is to minimise the attack surface for the examiner and just try to get some through. If you are doing this yourself it is a much more cost effective option than if you are paying a patent attorney.

I should end by saying that unless the patent(s) are for marketing purposes it is probably a waste of time. No small business can afford to enforce a patent so they are rather worthless. They can have value to a large company in an acquisition situation so I guess that is one reason to maybe do this - in this case the thicket approach is more desirable.

</comment><comment><USER4> I would start by determining what you goal of the patent is. Do you want it:

1\. for marketing purposes (to advertise a product based on patented technology)

2\. As part of a larger portfolio for future licensing

3\. To protect yourself from another company from competing with you.

Your answer to the above should guide you on how much to spend on your patent.

Case 1, it doesn't really matter how "good" your patent is a long as it gets issued. Your customers likely won't bother looking. So maybe try to save so cash and file yourself.

Case 2: you want the patent to be as broad as possible. Plus include as many alternate but less optimal solutions. Get a good attorney who is familiar with your field. They may charge $3-600/hour but it ma be worth it. You can often get a jr attorney to do a lot of the tedious parts under the guidance of the Sr partner.

Case 3: patents don't prevent others from infringing. They only give you the right to litigate. Would you be able to afford to litigate? That will cost way more than the patent itself. If the infringer is a large company, they can just litigate you out of your bank account. On this case? Why bother with even filing?

The process that works well for me is I write a detailed description of the invention and draft claims. Then hand it over to my attorneys to make better. They tell me my method works as a good data dump and that it saves them a lot of time.

Good luck!

</comment><comment><USER5> Not suggesting you do or don't do it yourself, but you might read the book "Patent it yourself: your step-by-step guide to filing at the U.S. patent office" by David Pressman from Nolo Press. That has a lot of experience in those pages. He has done four other books on the same topic that you might also read.

</comment><comment><USER6> Don't do it. The language used in patent applications is its very own special form of legalese. And hiring a patent attorney isn't all that expensive compared to your regular lawsuit legal eagles.

<comment><USER3> I disagree that patent attorneys are not that expensive.

</comment></comment><comment><USER7> You might be able to get away with writing a provisional on your own (the key is to be as broad as possible) but I wouldn't try to do a full utility application without a lawyer.</comment>


I'm not as smart as I thought I was - <AUTHOR> http://www.reddit.com/r/confession/comments/nxdzz/im_not_as_smart_as_i_thought_i_was/

<comment><AUTHOR> The third comment is particularly insightful.

</comment><comment><USER2> wasn't this submitted yesterday or the day before yesterday, yet in a different form? <http://news.ycombinator.com/item?id=3427762></comment>


Inventor of Sass Marries long time same sex partner - <AUTHOR> https://twitter.com/sjezewski/status/355003027100405761/photo/1 <comment><USER2> Congratulations Hampton! You do an awesome job for the open source community with not only sass but all your other projects(like haml). I think this moment is important because it shows that we aren't a community of prejudice but respect.

</comment><comment><USER3> Congrats Hampton!</comment>


Does invoking AWS CLI through serverless (OpenFaaS on Kubernetes) makes sense? - <AUTHOR> https://banzaicloud.com/blog/openfaas/ <comment><AUTHOR> We deploy to all major cloud providers and we need to frequently interact with them - thus we have containers for all the AWS, Azure and Google CLI. The clusters we deploy they have OpenFaaS deployed - and also the CLI container pod (we run everything in Kubernetes) for the particular cloud. When we need to fix/investigate an issue we need to get into the pod and use the CLI, etc - or SSH to a DMZ and install the CLI, add the credentials or IAM role, etc - <USER2> one idea was to wrap this into a function (as the example at end of the blog) and just be able to invoke any CLI command through the REST API (secured with OAuth2 tokens). I am less and less convinced that this is a good idea - <USER3> feedback?</comment>


What the Hell Is Kanye's Insane Tech Start Up? - <AUTHOR> http://gizmodo.com/5897185/what-the-hell-is-kanyes-insane-tech-start-up <comment><USER2> So it's a nice front end for a WHOIS lookup?</comment>


Cleaning My MacBook After 16800 Hours of Use - <AUTHOR> https://quanticdev.com/articles/cleaning-macbook-after-16800-hours-of-use <comment><USER4> This is something people often forget about with laptops. Between dust buildup and thermal paste degradation, performance will always slow over time due to rampant thermals. Even tech-savvy people often seem to think their laptop is simply "slowing down" and "too old" when the solution is often as simple as a can of compressed air.

I'd really recommend doing the dust cleanup every year or so. It's usually easy enough to remove the bottom cover of any laptop without particularly special screwdrivers.

Desktops (especially the PCs with the case Windows) are a bit easier to understand because you'll see the clogged intakes and dust buildup much more readily.

One thing I like about mechanical keyboards is that the keys can be removed making cleaning much easier. Six months is a better timeline for keyboard cleaning, these things get _nasty_ especially if you have pets.

<comment><USER5> > It's usually easy enough to remove the bottom cover of any laptop without > particularly special screwdrivers.

"particularly special"? The screwdrivers needed to open up a Macbook are special enough that I've never seen them used for any other object. So are you just glad that Apple didn't decide to create a new screwdriver standard for each Macbook model? :)

<comment><USER6> BTW, one of the most useful tools I've found for working on arbitrary modern electronics (after a basic set of ordinary screwdrivers and jewelers' drivers, a needlenose pliers, and some kind of spudger) is a $10-$20 set of as many different security screw bits as you can get. IME, it pays for itself the first time you need one of the bits, and I wish I'd bought a bigger set the first time.

<comment><USER7> I also find these very useful, but I have some that wore down and start stripping the screws.

I'd like to find a set that isn't prone to wear, unless that is simply a fact of life. I'm wondering if I was just being too cheap when I bought my first set.

<comment><USER8> I've had good luck with Wiha tools.

<comment><USER9> I second this. I had an ifixit set and then got this wiha set: [https://www.amazon.com/dp/B00JQ753W8/](https://www.amazon.com/dp/B00JQ753W8/). I ended up giving away the ifixit ones because the wiha quality was notably superior.

</comment></comment></comment></comment></comment></comment><comment><USER10> So note #1: swollen batteries are not a normal state of the battery. This is a sign of a defect and battery that is swollen is a safety hazard and should definitely be immediately replaced. Do not throw batteries in a garbage can. I don't know how it is done in US, here in Poland most large shops take used batteries for proper disposal.

Note #2: every breathing computer keeps accumulating dust and requires cleaning every couple of months of years. It is worth it if only to keep the noise down.

Note #3: it is not safe to vacuum the laptop as it can cause static discharge. Technically, even just waving a charged object over a piece of conductor can induce a current and damage susceptible component. The damage might not be immediately apparent and may cause various types of malfunctions. If you have no other way of doing it at the very least ensure it is done in humid environment (over 70% humidity but the more humid the better).

<comment><USER11> FYI there are special vacs for electronics which are made to prevent the static issue; many shops have one of these: [https://www.newegg.com/black- metropolitan-data-vac-portable-...](https://www.newegg.com/black-metropolitan- data-vac-portable-vacuum-cleaner/p/N82E16896367013)

<comment><USER12> I wonder if some aluminium foil around the vacuum cleaner's tube, connected to a wire through to ground would be a cheap alternative.

That said, I have heard that ESD isn't nearly as big an issue as it's made out to be, particularly the idea of slow damage/damage build-up. I don't know if the ESD thing is akin to an old wives tale with a kernel of truth surrounded in increasing layers of falsehoods; or perhaps something that was once true and is no longer, but persists in the collective memory. On the other hand maybe there's some solid evidence for it. Anyone have any evidence beyond anecdotes?

<comment><USER10> I work with electronics as a hobby. Some time ago I progressed to more sensitive components and so I am pretty interested in the topic.

Most ESD damage is quite insidious in nature. Imagine your laptop start loosing performance or one of your application developing a nasty error when you do some specific operation.

Even elements designed to protect from ESD have their capacity to accept shocks. For example, your phone's USB connector might be protected from ESD but that protection will run out at some point. Repated shocks to the USB connector may reduce life of your phone.

</comment></comment></comment></comment><comment><USER13> I've read a lot of the comments here.

I don't see many people mentioning the amazing reliability of this laptop.

I've had a similar experience with MacBook Pro (7 years, only problem was some keys started going bad).

There was a time you could not keep a laptop 7 years without significant component replacement or just having to replace it because it was no longer able to run modern software due to slower cpu, not enough ram (and not upgradable enough)... basically hardware obsolescence.

Now the components last AND the hardware is still fast enough to be useful after 7 years.

What a wonderful world we live in!

<comment><USER14> I have a similar 2013 vintage MBP. It's on it's 5th or 6th charger, 3rd charger board, 2nd battery, and 2nd speaker

[https://twitter.com/moreati/status/1087046181434417152](https://twitter.com/moreati/status/1087046181434417152)

Except for MagSafe it's a solid design.

<comment><USER15> I'm still on my original magsafe from 2014

<comment><USER16> Not the parent, but I wish I could say the same about my MagSafe charger.

The insulation on the MagSafe side started flaking after some time, leaving blue stuff _everywhere_. Apple's Geniuses say that it's because I mishandled it, and refuse to cover it.

</comment></comment></comment></comment><comment><USER17> A year ago, I opened up the kids' 2012 MBP because it was overheating with the fan on all the time. (This machine has had a long, hard life, including requiring a new keyboard when a glass of water was poured on it, several hard drive cables, a couple of batteries, several power adaptors, and I forget what else)

I extracted a chunk of lint about 1/2 cm thick and the consistency of felt from the fan intake.

After that, the fan didn't turn on nearly as often, and thermal management during Minecraft was much better.

<comment><USER18> Every 6 months or so I clean out the fans on my macbook. I used to take them out entirely, but realized with an old toothbrush and a soft-handed can of compressed air[0] you can do 90% of the job without the headache.

Always runs like a new machine after that.

[0] be careful not to damage the fan's teeth

</comment></comment><comment><USER19> For anyone doing this, please do not use a vacuum hose (unless its anti static) as it can cause ESD discharge, especially on exposed PCBs

I wrecked a DVR doing this with a Dyson vacuum.

Compressed air or small piece of cotton cloth with rubbing alcohol is the way to clean exposed electronics.

<comment><USER20> If you don't use a vacuum though the dust just flies out and settles in your house. I usually hold the vacuum near where I think the dust will fly out without touching any components. Never had an ESD problem yet.

<comment><USER21> > If you don't use a vacuum though the dust just flies out and settles in your > house.

Do it outside.

</comment></comment></comment><comment><USER22> I just replaced my MacBook Late 2013 after similar usage. No repairs, no hiccups or any component failure whatsoever except for a very badly degraded battery - it lasted about 1.5 hours on a full charge at the end. There's a lot to be said about Apple but honestly I don't see any other manufacturers that produce laptops that last so insanely long. Hopefully my new one will last as long as well :)

<comment><USER23> Thinkpads last insanely long, too.

I have a Thinkpad that I purchased in early 2014 that is still going strong except for, as in your case, a degraded battery.

My grandparents used to play Freecell on a very early IBM Thinkpad (manufactured in 1993, I think) that I picked up for free in a garage sale in 2001. This machine lasted well into 2005, and probably still works today.

Highly recommend trying them out. These days you can get equivalent performance to Macbook Pros for maybe 60% of the cost.

<comment><USER24> This. My main development machine is a Thinkpad T60, manufactured in 2006. Still running strong. For me, it is still a supercomputer. 2x1.8GHz. 4G ram. 1400x1050. Everything I develop on it screams on anything modern. When it breaks I'm planning on moving backward in time to see if there is an older model that I can adapt to, probably without spending a single dollar, just acquiring a thrown away computer.

Limitatations can enable innovation.

<comment><USER25> The T60 with that wonderful 1400x1050 screen is fab, I agree. (I wrote a sibling comment about using a 2014 Thinkpad as my main box, but I also have some older ones.) The main problem with it from a modern perspective is that it's strictly 32-bit, so can't address more than 4G RAM or run many contemporary Linux distros.

I also have a T40p, from 2003-ish, with the same 1400x1050 display. This is an even nicer computer to use in many ways - the keyboard in particular is even better than that in my T60 - but the difference in performance between 2003 and 2006 is striking. It's almost as much as the difference between 2006 and now. The T6x used Intel Core CPUs, but the T4x was still Pentium M days - single core and quite substantially slower. The T4x still has a PATA drive, so it's harder to put in a modern SSD. It's limited to 2G RAM for reasons I can't remember. It's also much less robust - the T60 series has a completely rigid chassis (and my T60 still looks like new) but the earlier T40 would bend if picked up from the front, which would cause both the case and the tracks on the motherboard to crack (the USB sockets stop working first, then everything else).

The T40p is a lovely, lovely computer - in some ways the best one I own - <USER2> it's for occasional document-writing only, not really for development any more.

<comment><USER24> This T60 is 64 bit. Core 2 T5600. My cutoff for referbishing a computer for someone is that it must be 64 bit and it must have at least two cores. Seems like anything Core 2 duo and later is good enough for almost anything most people do (except games).

<comment><USER26> My 2013 MBPr just gave up.

I have a T500 Core2Duo T5600 w 4GB RAM sitting in the cupboard. I pulled it out and put an SSD in it and installed Lubuntu, works fine but I have some software licenses I've paid for that are only available on MacOS & Windows.

Do you think it would run Windows 10 ok? I'd need to buy a license, but at this stage that might be cheaper than a new / secondhand laptop.

<comment><USER27> I run Windows 10 on a Dell Latitude E6400 from 2008 with a 1st gen Core 2 Duo & 4GB RAM and it runs fine.

Memory hungry apps like Chrome slow down pretty quick, but the OS itself is snappy.

</comment></comment></comment></comment></comment></comment></comment><comment><USER28> Side note on cleaning Apple products, may be of interest to iPhone users.

My 2017 6S wasn't charging properly. Charge would start, then a few minutes later, sometimes less, it would stop. I found that many times, if I inserted the plug with a bit more force, it would start charging, but later, it would stop. As is common the cable was frayed near the plug, so I replaced the cable. The situation didn't change.

I was at the point of deciding to replace the iPhone, since it was getting dated anyway, when my daughter suggested I clean the plug receptor.... the light bulb went on.

I grabbed a toothpick and started pulling out lint. Like your bellybutton lint. There was a lot of lint. It had become packed into the base of the receptor sufficiently to prevent the plug from making the contacts.

I had carried it in my pocket for 3 years, it likely gathered a few specks of lint each time I put it in there, and then I compacted the lint when I plugged it in next.

<comment><USER29> Same thing happened to me. An alarmingly massive ball of lint came out of my phone.

</comment></comment><comment><USER30> I'm on a maxed out 2013 Macbook Pro and I will fight tooth and nail to keep this laptop functioning for as long as possible. It has a decent keyboard, still powerful processor (2.3 GHz Intel Core i7), 16GB RAM, dual GPU. and MagSafe!!! Beloved MagSafe why on earth did they kill you I will never understand..

I had the Apple store replace the screen and battery about a year ago. The screen was suffering from the anti-reflective coating (1) recall, but I missed the repair program window. In every customer support conversation I had leading up to the repair, they would hint at "consumer protection law". This was a clue that if I then said "CONSUMER PROTECTION LAW" they would be forced to do the recall for free even though it was out of date. They ended up giving me the screen and battery for free.

Apple Store failed to identify and fix one issue that plagued me though, which was spontaneous black screen and poweroffs, especially when running on battery below 80%. It seemed like a power issue, or heat issue. I lived with it for two years, lugging a power brick around whenever I moved spots. With the Apple Stores closed due to covid, I contacted a local third party mac repair person. They said they've seen this before, it's a CPU power issue, and if you run a small script to keep the CPU out of really-low-power state it will stop happening (2). And they were right! So if you're experiencing this annoying problem as well, try this python script:

      from time import sleep
      import os
      while True:
        sleep(0.0002)

You may have to tweak the sleep value - higher values will let your CPU relax more but may trigger the power issue. Lower values will increase the CPU load but decrease the chance that the CPU hits this condition.

I've been running this script for 2 weeks now and no freezes. It generates about 6-8% CPU load, so obviously my battery time is suffering, but the alternative (random freezes all day) was so annoying that I'm okay living with this.

1: [https://www.macrumors.com/2017/11/17/apple-extends-free- stai...](https://www.macrumors.com/2017/11/17/apple-extends-free-staingate- repairs/)

2: [https://www.reddit.com/r/mac/comments/9pyort/apple_macbook_p...](https://www.reddit.com/r/mac/comments/9pyort/apple_macbook_pro_sudden_crash_fix_for_models/)

<comment><USER31> I'm curious if this Thunderbolt driver issue is what you've experienced (when the CPU load gets too low, something weird happens with the electrical system).

Here's [1] what's worked for me.

1: [https://www.reddit.com/r/MacOSBeta/comments/c6f6e7/catalina_...](https://www.reddit.com/r/MacOSBeta/comments/c6f6e7/catalina_random_shutdown_applethunderboltnhikext/)

<comment><USER30> Thank you!! I'm trying this now without the CPU busy script, fingers crossed

</comment></comment></comment><comment><USER32> This is why I loved the no-suffix "MacBook". No fans - no vents - no dust - <USER3> cleaning! For me, the most valuable feature and very under-appreciated.

Eventually I had to replace it and now I'm stuck with a noisy dusty MBP. I suppose the iPad now fulfills the role of "maintenance free computer".

<comment><USER21> > I suppose the iPad now fulfills the role of "maintenance free computer".

...but he didn't need to maintain his MacBook Pro in this video either. It worked fine until the battery degraded and he had to recycle it for that reason. The MacBook also had a battery that would expire after time.

<comment><USER33> Could be his battery was overheating.

</comment></comment></comment><comment><USER34> Never use a vacuum on electronics! Use cans of compressed air instead. The author partially addressed this in the article but I feel that the message doesn't really come out as clear as it should. The static generated by a standard vacuum cleaner can damage electronic components, the author is wrong in their belief that this only applies to desktops and not laptops (and why would that be the case?).

PS air compressors are not a good alternative: they might suffer from internal condensation and emit air that is too moist, that's why it's best to use cans instead.

PPS there are vacuum cleaners specifically designed for electronics. They tend to be pretty expensive. It's OK to use those.

<comment><USER35> Moist air is fine, _wet_ is not. It even helps to cut down on the static risk. And you should always let it sit for a bit after cleaning before turning it on in case there was moisture build up.

The biggest risk is to reach into the guts and zap a component, that's a much higher chance of incidence than any effect from the air itself, after all, the fan also sucks air through all day long and usually is not grounded.

Make sure you always reach for the frame first before touching anything else in the guts of a computer.

</comment></comment><comment><USER36> Wait, recycling a 2013 laptop? What? I hope he means selling it on, not actually recycling. Computers just don't age the same way anymore, a 2013 laptop is still perfectly usable nowadays, it would be an incredible waste to recycle it.

</comment><comment><USER37> Looks like the anti-glare coating is coming off too. That happened to my MacBook and Apple replaced the whole screen for free, that's pretty good considering it was 4 years old.

<comment><USER38> I ended up rubbing it off. Used a mouthwash for it (alcohol didn't work, perhaps "microcrystals" did the job). If you do that there will be lots of micro scratches on screen, but it's still better than looking at stains, and quite pleasant in dark.

I couldn't get the replacement (didn't know "stain-gate" was a thing).

</comment></comment><comment><USER39> Laptop makers who make it nigh impossible to take the back cover off without special tools: "Performance degradation due to the inescapable characteristics of typical user environments is God's sign of favor for planned obsolescence."

</comment><comment><USER40> A reminder that Apple refuses to fix 2017 Macbooks that were poorly constructed (Flexgate), even after having acknowledged that they were poorly built (by offering a free fix for the same issue on a previous model and changing the part on later models).

[https://support.apple.com/13-inch-macbook-pro-display- backli...](https://support.apple.com/13-inch-macbook-pro-display-backlight- service)

</comment><comment><USER41> I recently did the same on my MacBook Pro (Retina, 13-inch, Late 2012) A1425 while performing a battery replacement. Definitely helped to have a thorough cleaning. Documented here: [https://www.youtube.com/watch?v=CAYFDDxUxHA](https://www.youtube.com/watch?v=CAYFDDxUxHA)

<comment><AUTHOR> Good stuff right there!

</comment></comment><comment><USER42> I know very little about electricity. What is it about vacuuming particles that produces static electricity where shooting the particles with compressed air does not?

<comment><USER43> The dust traveling through the plastic vacuum tube is what gives rise to the static electricity. When you blow clean air into the fan the dust goes into a cloud in the air.

</comment></comment><comment><USER44> Mini rant only tangentially related. I have a 6 year old MacBook Pro (Mid 2014 NVidia). It __mostly __works fine.

Except, the GPU drivers are crap and Apple has no intention of fixing them. This shows up as bugs in Firefox and Chrome, the apps that hit the most GPU edge cases, and when I've filed bugs I've basically been told "too old, get a new machine"

Which isn't an unreasonable POV. I'm sure there just are not that many 2014 NVidia MBPs in the world. Apple doesn't care to fix the bugs. Why should the browser teams waste time working around GPU driver bugs for so few users?

And yet, it's really hard to justify spending $3400 for a new MBP when nothing will actually change in my day to day usage.

I also have a 3yr old 1060 Razer gaming laptop. That actually enabled something, VR, to justify spending $$$$ on it. But, getting a current dual GPU MBP gets me nothing that my 2014 MBP doesn't already get me except fixing a few annoying bug livable browser bugs.

Note I have a similar issue with the Razer. Windows won't update to the most reason version and says their are issues with the hardware and maybe some day in the future once the issues are worked around Windows will update. Unfortunately that means I can't use WSL 2 so again, I can spent $3000+ for a 2020 Razer but when I'm done all that will happen is some software bugs will be fixed, my actual day to day usage won't change.

Tech can be really annoying and it's frustrating to be told the solution is spend $7000

Just Kvetching

</comment><comment><USER45> I got the 2-for-one variant of this with my 2016 MBP where I purchased a replacement battery assembly from iFixit to replace the swolen ones. In the kit, they include: the tools, a link to their iFixit video on the steps as well as some protective gear to do the job. Highly recommended! Replacing batteries is a way more involved task on MBPs, but while you're there in the early stages, you can vacuum or take compressed air to anything you see.

</comment><comment><USER46> I'm a bit sad that his MacBook is not getting more love after 16800 hours of service.

I have 2013 MacBook Pro 15, never opened, the battery lasts for 1.5 hours, tends to overheat a bit. Same story. Morally it felt old but it was still good and fast machine for me. I was contemplating of buying 16" but just could not justify throwing away this solidly engineered machine.

During quarantine bought iFixit battery replacement. Disassembled every part, cleaned religiously every area with blower, replaced swollen battery and assembled again. Took 2 hours, works fine, no fan noise, battery lasts for a long time, finally upgraded to Catalina.

Now I have a great laptop and after 7 years of using same machine I felt that I still needed to upgrade something. So I went to buy a desktop PC. It was interesting to assemble a modern PC, try Alyx and Windows 10 with WSL. I no longer need to use Paperspace for SketchUp rendering. After COVID I am planning to leave PC at home and have my MacBook at Office / Co-working.

Not saying this is the best decision, just sharing a different path taken when it came to computer upgrading.

</comment><comment><USER47> I used my 17 inch Macbook Pro everyday for six years. It would still be going if not for a failed GPU. I did take it apart hoping it was dust and heat causing the problems but no it was a fatal hardware fault. It's now just a shiny aluminum lump.

There's nothing worse for electronics than heat. The damage is cumulative but nearly impossible to notice until it's too late.

</comment><comment><USER48> I replaced the thermal compound my trust old dell Latitude. Barely an hour of effort undoing Phillips screws and scraping off the seven year old gak. Roughly a 15 C decrease under load. Fans are less of a blowdryer and more of a hand warmer.

</comment><comment><USER49> I just got my 4 year old macbook pro's battery replaced. The guy who did it also cleaned up the internals. I can easily use my laptop for another 2-3 years. I spent 100 USD to get it cleaned and battery replaced. Best 100 USD spent.

</comment><comment><USER50> Highly recommended for keeping things clean: Metro Vacuum ED500P DataVac 500-Watt, 0.75-HP Electric Duster.

I've had one since ~2012 and it helps me maintain a small fleet of laptops and workstations for my large family (with many pets).

<comment><USER51> I also have one and am so happy with it, no more buying canned air. Also useful for blowing out vacuum filters and roomba's.

</comment></comment><comment><USER52> Here is a Schlieren visualisation of the airflow from my 16" MBP.

[https://youtu.be/88mXfMm0M58](https://youtu.be/88mXfMm0M58)

</comment><comment><USER53> My 2009 13" MacBook Pro still runs great after all these years of heavy use. I've cleaned it and reapplied thermal paste on the CPU/bridge chips twice.

<comment><AUTHOR> > reapplied thermal paste on the CPU/bridge chips twice

does it help noticeably?

<comment><USER54> I recently reapplied the thermal paste on my 2015 13" MBP. It lowered CPU temperatures by about 5°C so fan noise is only slightly diminished, but it's no longer crashing under load on a weekly basis. And this was using an old tube of cheap paste I had lying around for years.

</comment></comment></comment><comment><USER55> Thanks for sharing. My battery is swollen on my 2014 MBP, and trackpad becoming unusable. New battery replacement coming soon - I'll do this along with it.

</comment><comment><USER56> Typing this on my 2011 MBA which has been used a similar amount. Screen replaced during warranty and keyboard replaced subsequently but working fine!

</comment><comment><USER57> I use the Dyson on my Mac's exhaust every so often for the same reason, it's satisfying to see the amount of crud that comes out of it.

</comment><comment><USER58> It reminds me 6 months ago when also opened up my 6 years old macbook pro and saw the dirt inside. The battery was totally destroyed so I ordered a kit at ifixit and replaced it myself.

Now the macbook feels like new and the battery last forever. I hope we still have the option to repair our devices in 5 years.

</comment><comment><USER59> Thank you for the video + article combo. I enjoyed listening, but also that I was able to look back at the text to see specific things instead of having to rewind the video/audio and listen again.

</comment><comment><USER60> Has Apple commented on the "swollen battery syndrome"? Are these batteries just expected to behave like this when they get old? This seems to be very common problem with old MacBooks.

<comment><USER35> _some_ swelling is ok, the batteries are not rigidly enclosed so they tend to do that over their design life. Excess swelling is not ok and usually an indication of overheating for instance due to bad airflow. The laptop on display here looks like a textbook case of that.

</comment></comment><comment><USER61> I would not recommend to clean the fan with a vacuum cleaner because the rotation will generate a low voltage

</comment><comment><USER62> How about compressed air on a MacBook Air? Seems trickier than on a normal MacBook Pro.

<comment><USER63> The Air fan is basically useless, you might as well leave it be.

[https://www.reddit.com/r/apple/comments/9vbks0/retina_macboo...](https://www.reddit.com/r/apple/comments/9vbks0/retina_macbook_air_teardown_cooling_fan_does_not/)

</comment></comment><comment><USER64> my 2018 macbook pro just got cleaned by apple when they replaced the keyboard. I assume it will be done by them every 18 months or as long as the recall is in effect. (sucks that your data is all deleted though)

</comment><comment><USER65> Next time. Making peanut butter and jelly sandwich.

</comment><comment><USER66> I didn't know they needed "cleaning".

<comment><USER67> If airflow travels through something and you don't work in a dust free environment, it needs to be cleaned periodically.

It's too bad that OS' weren't more wise to this. Right now there are certainly millions of desktops and laptops that are running far below their potential because it is forced to thermally throttle, yet there is no indication for the user beyond things being slower than it once was.

<comment><USER68> Windows doesn't have a temperature display accessible to users. That strikes me as being in order to sell more hardware.

In-laws were going to throw away their computer, random freezes in the summer. Needed a thorough clean, "good as new".

PC World/Microsoft lost a sale though.

<comment><USER67> A temperature display itself probably wouldn't be of much use because the CPU will reach an operating temperature and modulate the frequencies/boost to avoid exceeding it. One user could be enjoying full performance at that temperature, while another user is running at 1/10th the speed.

The OS does see the perpetually fluctuating frequency state of the processor, though, and can monitor it over time in a correlation with the temperature and display a simple aggregate performance metric, flagging as a system alert when performance falls below a set threshold. I've had relatives whose system saw a dramatic, very noticeable usability improvement after a couple of quick bursts of compressed air.</comment></comment></comment></comment>


Show HN a simple way to write a daily diary - <AUTHOR> http://www.maildiary.net

<comment><AUTHOR> I created several diaries for myself. Each for my children to send some pictures and thoughts from my mobile or picasa and another for daily memories.</comment>


Tutorial on refining 2d gameart houses quick and easy way with vectors - <AUTHOR> http://www.2dgameartguru.com/2018/03/refining-2d-gameart-houses-quick-and.html <comment><AUTHOR> A new post on 2dgameartguru.com refining the buildings from the previous step- by-step tutorial.

Taking most of the existing elements and adding simple shapes like rectangles, circles and a few lines to add interest to the buildings in a quick and reusable way. Building your library of elements will make this a very modular approach to creating repeating, yet different 2D game assets.</comment>


When did we forget our dreams? - <AUTHOR> http://xkcd.com/137/ <comment><USER3> I don't think people ever forget their dreams. They always remember them, but just move in the direction that is expected of them, and never actually implement their dreams. But they don't forget.

Something I recently realised is this - this is it! Where I live, what I am doing, who my friends are - this is my life! This is not a temporary stop while I figure out how to make millions, after which I will actually start living, no, it already started and I'm a part of it.

Waiting may take longer than I expect, so I better enjoy the wait as much as I well can. That's why I'm not giving up my life to create a startup. Rather, I'll enjoy things as they come every day, and dedicate an appropriate amount of time towards realising my dreams. If the dreams never work out, it really is not that bad, because I enjoyed the time right up until then.

<comment><USER4> You've learned the message of Buddha and Pink Floyd well.

<comment><USER3> This is why self help books keep selling. Because we keep hearing the same wisdom over and over again, but we just do not _understand_ it. All these simple fortune cookie wisdoms, they mean nothing when you read them casually. But when something big happens to you and you really _understand_ what is meant, then it's like opening a new door and discovering magic behind it.

I finally get it. It may be stupid to everyone else, it may be obvious, but only now do I really really understand what is meant, and only now have I been able to apply what I learnt to make a fair amount of money and be happy while doing it.

<comment><USER5> Kierkegaard nailed this long ago, when he said that everything is subjective, and that no wisdom can be transferred from one person to another, except through directly living and experiencing it.

(Kierkegaard wanted his gravestone to say 'A person', because after all, who would understand what Kierkegaard was anyway.)

</comment></comment></comment></comment><comment><USER6> Yes, we can all go right to xkcd, or techcrunch or whatever and read them ourselves. Thats not the point of posting them on Hacker News. The point is that the discussion that follows about the ideas in the articles/comics is still radically different on HN than most other places.

Randall has some thoughts that apply very well in a group like this so his comics appear here often and start interesting discussions. The only thing not interesting to me is that every time one is getting started I have to wade through a boundary layer of "lets blacklist xkcd" comments.

Doc my karma if you will, but I think the silent majority has voted with the upmod arrow on the post. I am one of them.

<comment><USER7> Completely agree, but I'm refraining from upmodding you because it would surely be ironic if this becomes the top comment, instead of some real discussion on pursuing our dreams!

<comment><USER6> Ha. Definitely. Sometimes karma catches you off guard. I didn't expect to get any upmods for that. It was a bit of a rant really. When I got here, it was just about entirely a discussion about blacklisting xkcd and I thought the comic had brought up something that was really worth talking about.

Of course, as soon as I post this the _karma discussion effect_ will begin and it'll be downvoted back to something reasonable I think. Just don't - <USER2> bro... :)

</comment></comment></comment><comment><USER8> It's not that the routines cause thoughtlessness but that, ideally, the routines free us to think about and do important, creative things by making the daily doldrums automatic.

Sure, sometimes it's good to dive back into reinventing the routines (or taking a road trip and questioning the purpose of the routine), but very few people can get work done that way. And getting work done is a valuable goal.

</comment><comment><USER9> This is an old comic and normally I'd be annoyed, but this one is an exception: Reflecting on my dreams, even if I've chosen not to pursue some of them, has greatly improved my mood on this particular day.

Do it again tomorrow? :)

<comment><USER10> the essence of your comment if beautiful.. tomorrow would be great.

</comment></comment><comment><USER11> Great. My employer saw the last three frames and now I'm fired for what someone else wrote.

<comment><USER12> Can't think of a better reason to go build your startup :)

<comment><USER13> Absolutely! Just uploading the latest changes to our "stealth mode" startup site right now :) We'll be in invite-only mode by the end of the weekend, and completely public sometime next month. Stoked!

Thinking about these kinds of things, I feel sad for other people who haven't taken a risk on a dream. Caring about something as passionately as a startup you've build from the ground up is something I imagine many people never get to feel, although maybe having kids is similar (don't have any, so I couldn't say...). But as far as the 9-5 goes, I'd rather care about what I built in that time rather than feel it was wasted on something not worth doing.

Ooh, upload just finished, here I go! :)

</comment></comment></comment><comment><USER14> A classic.

And, when hiring, I certainly wouldn't hold a rant like that posted somewhere on the internet against an applicant...

</comment><comment><USER15> All this talk of following one's dreams is really making me want to finish The Alchemist, which I started last night.

<comment><USER16> Sorry, but why not go right to the good stuff?

You'll find a great list in Harold Bloom's "Genius: A Mosaic of One Hundred Exemplary Creative Minds".

<http://www.amazon.com/dp/0446691291/>

</comment></comment><comment><USER17> this is existentialism in a shiny wrapper.

<comment><USER17> read the first sentence in the wikipedia article on existentialism before you downmod please.

<comment><USER18> Wow, you said that word. It's unfashionable.

Existentialism isn't popular as self-help because it's not optimistic. There's no natural law that every truth must be shiny happy. "Existence precedes essence" accurately describes our world, but doesn't motivate. A good antidote against religion and propaganda, though.

</comment></comment></comment><comment><USER19> When the first bill came due.

</comment><comment><USER20> I've had this on my wall for months.

</comment><comment><USER21> Oh, the text in the middle is meant to be read? I read the cartoon only as "you should be more careful sdojdsa fosadf aösdpoifasdö fojahdsfösdaof sddfjjdsjdsjfsd föso fsdöfo sdfsd fuck that shit".

</comment><comment><USER22> That is bloody brilliant! Love it...

</comment><comment><USER23> I didn't forget my dreams.

</comment><comment><USER24> PG: can you please add xkcd hostname to the blacklist? Not because it's bad -- actually it's brilliant, but adds very little to this community in terms of news.

After getting techcrunched, is xkcd the second sign of HN going the reddit way?

<comment><USER25> It'd be pretty sweet to allow us to put a list of domains we don't want to see links for in our preferences.

<comment><USER24> I had asked for something similar a while back.

<comment><USER26> Why not just do it yourself via Greasemonkey?

The bitching about xkcd is getting annoying (as is the constant speculation about "Is this the moment when HN is going to jump the shark?)</comment></comment></comment></comment>


Ask HN: What you hate the most from starting a startup? - <AUTHOR>

Hi HN, I'm making a investigation, of what are the things that founders, aspiring entrepreneurs, or people who work in a startup; hate the most of a startup. I want to get some insights, to start working in solutions to stop having a bad experience from starting or working in a startup. <comment><USER2> Oh lordy, where to begin...

(1) The startup world is full of creeps and psychos. There are a lot of sociopaths, vapid fast-talking wheeler-dealers, and people who are just flat out bat nuts crazy. There's also been a huge influx of douchebags since tech became (supposedly) a way to get rich quick. I've been burned by this a couple times and have gotten better at judging character as a result, but be warned-- here be dragons.

(2) Credentialism is a lot stronger in startups than most people believe. Stanford degrees seem to count more than traction. In the end I just broke down and added "did not attend Stanford or work for Google" to my AngelList profile. If you care, you're not going to like me anyway.

(3) Everything follows an extreme power law distribution, and it always implies a catch-22. If you don't live in the Bay Area, it's exponentially harder to get funding. If you do live in the Bay Area, you need a substantial exit event to afford a down payment on a modest post-WWII starter house. This also applies to incubators, startups you might join, employees you might recruit, etc.

(4) Trendy not-invented-last-week syndrome abounds on the technology side: [http://www.jwz.org/blog/2003/02/the-cadt- model/](http://www.jwz.org/blog/2003/02/the-cadt-model/)

(5) Given the quantities of money involved, I'd expect investors to be more rational than they really are. "Raise on no numbers or good numbers" \-- that doesn't exactly strike me as rational. No numbers is better than some growth? That's not by any means the worst example either. I once had an investor type tell me with a straight face that he looks for founders with a "prominent jaw line, an alpha male look."

(6) Founding or building something is lonely, thankless, isolating, and a lot longer of a death march than you probably realize... unless of course you are "juiced" in which case you can walk into a VC firm and walk out with millions for a product that doesn't exist. Then you can hire people to build it while you wheel and deal and pimp your ego. The existence of this kind of "juice" can be enormously discouraging to mere mortals. It's enough to make you wonder if this game isn't as rigged as Wall Street. (I don't think it's _quite_ as rigged as Wall Street, but there's definitely a privilege network.)

(7) With all the talk of innovation you hear from people like Peter Thiel, the reality is that the most profitable stuff is usually some kind of equity play on the rapid growth of some trendy thing that isn't technically that interesting. Examples include Facebook, Snapchat, etc. If you want to extend the human life span, develop AI, or otherwise advance the human condition, sign up for a vow of poverty. The only hack around this seems to be to do something a bit more "boring" to make enough money to then do something more visionary. The poster boy for this is Elon Musk. There are also exceptions -- things that can be both profitable and interesting -- but those also tend to be the hardest death marches of all. While you're struggling to create real value, someone else is making millions flipping some kind of social media app of the week.

(8) Mobile is the future this week. Last week it was the web. Next week it'll be the Internet of Things and nobody will ever fund another mobile startup. Desktop is dead even though everyone builds absolutely everything on a desktop. See #4.

(9) Don't let anyone fool you -- tech is sexist as hell and geek culture is misogynistic with a faint whiff of creepy pedophilia and other chthonic horrors. If you don't have a penis, be prepared for subtile hazing and grunt work and possibly drool on your seat cushion. Racism exists too, but it's less stark and omnipresent than the misogyny.

(10) Age-ism is big too, and it contributes a lot to #8 and #4. A big reason for this is that all the incubator and accelerator programs are geared toward unmarried childless recent college grads who can move on a whim, room with six other people, and special enzymes that can derive a full spectrum of nutrients from ramen and energy drinks.

I could keep going.

Yet I keep coming back for more. Thank you master may I have another?

<comment><USER3> Don't forget to blog this on the trendy "Medium". A visionary way to "blog" and start trending with some hashtag.

Apart from my above sentence, which could be a number 11 in your post, this was a solid answer.

So let's swap it around now...

The real Valley founder checklist:

\- White \- Male \- Alpha \- 23-25 \- Talk the talk \- Know more (important) people than "how to code" \- Master human-SEO (throwing keywords to investors) \- Go to Stanford \- Use your connections to "work at Google" \- SoLoMo (Silicon Valley show) \- Misogynist

I think I got them all.

<comment><USER2> "Human SEO." I like that. Hadn't thought of it that way before. That's mad disruptive bro.

Other industries have it worse though, or have different issues all their own. Finance is a real shitshow.

<comment><USER3> You think we'll get funding if we follow this model of "disruption", brogramming and "change the world" by making a nekked-pic-sharing app that "self-destructs" by changing the image format and hiding it in cold storage?

I love finance! The only industry where you are allowed to compete with your clients in the same markets directly (and screw them over).

</comment></comment></comment></comment><comment><USER4> I think one of the most challenging things is working on the sales side of a start-up. As an engineer, coding is relatively straight forward, but sales is a different animal.

When working on sales, I think engineers underestimate how truly difficult it is for someone to sign-up, or take their wallet out to pay for a service. The assumption of creating a great product isn't enough for success.

<comment><USER5> Enterprise sales also takes ages. Even if a manager likes your product, and their boss, and their boss it can take another couple of steps (months) until somebody fills out a purchase order. And if you're unlucky you chase them to pay your bill. Not a small company, but major (publicly traded) companies with huge profits. So frustrating.

</comment></comment><comment><USER6> the anxiety!</comment>


The Top 5 Mistakes of Massive CSS - <AUTHOR> http://www.youtube.com/watch?v=j6sAm7CLoCQ#t=05m46

<comment><AUTHOR> Nicole Sullivan talks about how to manage massive CSS code bases and what to watch out for. She also was involved in facebook's effort to reduce response times.</comment>


Q: Run SQL Directly on CSV Files - <AUTHOR> https://harelba.github.io/q/ <comment><USER2> "Any file is a database if you awk hard enough." —Bryan Horstmann-Allen

[https://twitter.com/neilkod/status/914217352564137984](https://twitter.com/neilkod/status/914217352564137984)

Furthermore, no thread on CSV files can be complete without mentioning this infamous bit of fact-trolling: the ASCII standard has had unit- and record- delimiters baked into it from the beginning.

[https://ronaldduncan.wordpress.com/2009/10/31/text-file- form...](https://ronaldduncan.wordpress.com/2009/10/31/text-file-formats- ascii-delimited-text-not-csv-or-tab-delimited-text/)

<comment><USER3> Usability suffers, if you can't type the delimiters easily.

<comment><USER4> I can't stand it when I'm banging out a CSV by hand and have to manually escape the field and record separators. Happens all the time.

<comment><USER3> I use TSV for that reason. Most of the time you don't need to escape anything. And from code you need to escape/unescape only 4 chars. \t \n \NULL and \\\\.

</comment></comment></comment></comment><comment><USER5> If you're on Windows, you've had the ability to do this for Quite Some Time®: [https://support.microsoft.com/en-us/help/850320/creating- an-...](https://support.microsoft.com/en-us/help/850320/creating-an-odbc-data- source-with-the-microsoft-text-driver)

<comment><USER6> Ah, I thought you were going to link to the fascinating skunkworks Log Parser tool [https://www.microsoft.com/en- us/download/details.aspx?id=246...](https://www.microsoft.com/en- us/download/details.aspx?id=24659) Wonder if it still runs on Windows 10?

<comment><USER7> If you need a UI, don't forget the tool: Log Parser Lizard ^^

[http://www.lizard-labs.com/log_parser_lizard.aspx](http://www.lizard- labs.com/log_parser_lizard.aspx)

<comment><USER8> Good of you to also accomodate the disabled.

<comment><USER9> Is this a joke or is the tool actually accessible?

</comment></comment></comment></comment></comment><comment><USER10> Sqlite virtual tables. You can literally query any data source. All you need is a module implementation of interface exposed by sqlite for the data source of interest. I once wrote a module that could query protocol buffers encoded data

</comment><comment><USER11> Fun fact: MySQL has actually supposed CSV files as a storage backend for quite some time now[1]

1: [https://dev.mysql.com/doc/refman/8.0/en/csv-storage- engine.h...](https://dev.mysql.com/doc/refman/8.0/en/csv-storage-engine.html)

<comment><USER12> Came here to post the same. This can be used in very interesting ways, either for quick ad-hoc queries or for import/export functionality (ETL), where the core of your data is in InnoDB but you keep a CSV version for one of the phases of your processing, to interface with other tools.

This may looks like a dirty hack compared to a clean API, but if you have performance considerations or need to interface with some legacy thing that happens to understand CSV, this can be a good choice.

</comment></comment><comment><USER13> I've never heard of this before, but it reminded me of this other command line tool for wrangling csv files in a cli. This tool makes use of SQL syntax much more than xsv, so there isn't 100% overlap here.

[https://github.com/BurntSushi/xsv](https://github.com/BurntSushi/xsv)

<comment><AUTHOR> Thanks! xsv is new to me. I love Andrew Gallant's ripgrep(rg) grepping cmd tool.

<comment><USER14> His name is Burnt Sushi I think.

</comment></comment></comment><comment><USER15> Another trick along these lines is to cut out the middleman (middleprogram? middleware?) and use SQLite to do it:

        $ sqlite3
        sqlite3> .mode csv
        sqlite3> .import foo.csv foo
        sqlite3> SELECT * FROM foo WHERE bar = 'baz'; -- you get the gist...
        (a bunch of rows)

Q seems to be much easier to use (and certainly easier to remember), but I've always found it handy to have the full power of SQL at my fingertips when needing to do a bunch of CSV manipulations.

<comment><USER16> I think the csvkit CLI tool, csvsql, provides this functionality, too.

</comment></comment><comment><USER17> The perl CPAN module DBD::CSV also lets you do this. It was started in 1998 and still gets updates.

<comment><USER18> Text::CSV_XS is my goto module for csv just use that and whack it into MySQL postgress would be my advice.

<comment><USER19> nah sqlite is what you want when you want a step up from CSV.

<comment><USER18> Well all I want to do is ingest some Google analytics and combine it with the data from SEmRush

</comment></comment></comment></comment><comment><USER20> Real-world CSV files generally contain some or all of the following horrors:

\- some strings enclosed in speechmarks, but some not

\- empty fields

\- speechmarks within strings

\- commas within strings

\- carriage returns within strings

How does Q do up against a CSV file with those traits?

<comment><USER21> All of your "horrors" seem...correct? Its comma delimited, so anything that is between two commas should be parsed without issue; if it's a string with a comma in it, and unquoted, you simply have a broken csv file. If its quoted, than anything until the next (unescaped) quote is fine, including commas

Unless you're trying to parse csv files with regexes, none of those should be difficult, or even unexpected, to handls with a PEG parser, or any equivalent device

Ofc if you're accepting ambiguity then its just arbitrary how you handle it, but none of your examples afaict present any ambiguity (I'm assuming strings are either quoted or unquoted, with the former primarily allowing commas/newlines in strings; escaping exists as well; comma delimited columns, newline delimited rows)

<comment><USER20> Yes, it would be valid CSV. I suppose my point is that naive attempts to roll- your-own CSV parsers tend to fail on the points I listed. Hopefully Q does not do that.

<comment><USER22> They do? Commas and quotes are the two basic features of CSV, so it seems very strange to forget to implement half.

<comment><USER23> There is a lot of inconsistency out there. I have seen csv files saved in Excel not be import-able by Access because the latter doesn't handle breaks in fields correctly. I've seen csvs saved from various systems such as sql mngmnt studio grid view and wufoo exports not generate csv correctly. There are many lazy attempts at csv generators out there that just throw breaks between records and commas between fields and call it a day.

<comment><USER23> And even if they do wrap all fields in double quotes it is very common to forget to escape double quotes in fields, then it depends on the parser as to whether it can determine the proper structure of the record.

</comment></comment></comment></comment></comment></comment><comment><USER24> Excel has a lot of this stuff builtin as well in the PowerQuery editor [0] which supports CSV, JSON and XML data sources. Albeit not SQL, it allows you to do almost everything SQL can do, but in a GUI.

After you're done preparing your data in PowerQuery you can run PowerPivot on it for aggregations.

[0]: [https://cdn-5a6cb102f911c811e474f1cd.closte.com/wp- content/u...](https://cdn-5a6cb102f911c811e474f1cd.closte.com/wp- content/uploads/2018/02/Main-Areas-of-the-Power-Query-Editor.png)

</comment><comment><USER25> Looks like PostgreSQL has a foreign data wrapper for CSV.

[https://wiki.postgresql.org/wiki/Foreign_data_wrappers](https://wiki.postgresql.org/wiki/Foreign_data_wrappers)

In fact, seems it has several.

<comment><USER26> Yes Also trivially easy and lightning fast to import csv to table using copy command in Postgres.

</comment></comment><comment><USER27> Perl in 1998. [https://metacpan.org/pod/DBD::CSV](https://metacpan.org/pod/DBD::CSV) Okay, 2002--it took a while before anyone bothered to write a shell. [https://metacpan.org/pod/release/TLOWERY/DBI- Shell-11.9/lib/...](https://metacpan.org/pod/release/TLOWERY/DBI- Shell-11.9/lib/DBI/Shell.pm)

And there are drivers for everything from PostgreSQL and Oracle to text files and the Azure WMI.

</comment><comment><USER28> Related, but with a free web interface [https://github.com/simonw/datasette](https://github.com/simonw/datasette) , [https://github.com/simonw/csvs-to-sqlite](https://github.com/simonw/csvs-to- sqlite)

</comment><comment><USER29> csvkit includes csvsql, which does this. I've used it and liked it. [https://csvkit.readthedocs.io/en/1.0.3/tutorial/3_power_tool...](https://csvkit.readthedocs.io/en/1.0.3/tutorial/3_power_tools.html#csvsql- and-sql2csv-ultimate-power)

<comment><USER30> Came here to tell about csvkit and the csvsql that comes with it. Highly recommended. It is one of the first things I install on a new system.

</comment></comment><comment><USER31> So is this a wrapper around [https://www3.sqlite.org/csv.html](https://www3.sqlite.org/csv.html) ? It mentions sqlite.

<comment><USER32> Looks like it is implemented in python(2) with use of the stdlib sqlite3 module. [https://github.com/harelba/q/blob/master/bin/q](https://github.com/harelba/q/blob/master/bin/q)

</comment></comment><comment><USER33> We've had this for ages now: the F# type provider gives strongly types access to CSV files, and the resulting object heiarchies are LINQ compatible. There's even support for applying units of measure.

Veeeery nice for data munging :)

[http://fsharp.github.io/FSharp.Data/library/CsvProvider.html](http://fsharp.github.io/FSharp.Data/library/CsvProvider.html)

</comment><comment><USER34> The letter Q is already taken...

[https://en.m.wikipedia.org/wiki/Q_%28programming_language_fr...](https://en.m.wikipedia.org/wiki/Q_%28programming_language_from_Kx_Systems%29)

<comment><USER35> >It is proprietary software, commercialized by Kx Systems

q is available i'm thinking

<comment><USER34> What? kdb, K and Q are part of a renowned array processing toolkit/environment widely used by data scientists _for decades_.

Calling this command-line tool Q, which happens to be used for the same purpose as the Q programming language (i.e. querying data) begs for confusion and misunderstanding...

</comment></comment></comment><comment><USER36> I think tools like this don't exist much because it typically makes more sense to just put it into a database or at least SQLite ... then do whatever you want from there.

Meanwhile for all your streaming, filtering and aggregating need there is awk.

I've never been board enough to write my own little sql library for awk, but I'd be surprised if it doesn't exist.

<comment><USER37> That's exactly what q (the linked tool) and almost all other tools like it do - it loads into an sqlite database and provides some handy wrappers, that's it.

</comment></comment><comment><USER38> Many of the applications at my workplace use flat text files in CSV format for logs and configuration. For new development I've been using Sqlite to replace some of these usages. Sometimes when I need to analyze legacy log files I import the CSV data into Sqlite tables. After doing this a few times I hit upon the idea of why not skip the import (which balloons my otherwise-small db files) and write an Sqlite plugin that allows treating the flat CSV file as a virtual table? It's neat to see there's some existing work here!

</comment><comment><USER39> If you need to work with CSV or Excel or tab files from the command line I suggest having a look at alasql. Can be installed via npm and gives you a good list of export options.

</comment><comment><USER40> Eep, no Python 3 support? :(

<comment><USER41> I was about to just recommend you pandas, but just realized it doesn't natively support running sql on dataframes, but pyspark does!

</comment></comment><comment><USER42> I've done a similar thing, but using jq syntax on any object-like file: [https://github.com/jzelinskie/faq](https://github.com/jzelinskie/faq)

Ya know how some people want everything to be vim after they use it? I'm surprised more people haven't done that with other tools.

<comment><USER43> > faq is pronounced "fah queue".

So... "fuck you"?

</comment></comment><comment><USER44> > Have you ever stared at a text file on the screen, hoping it would have been > a database so you could ask anything you want about it?

Not really. I could see this being helpful for debugging, but at that point you can get by with some simple bash string operations. Doing that same operation w/SQL seems like overkill.

<comment><USER45> Maybe my bash just isn't good enough anymore, but I do this all the time. In fact, I was just doing some work this morning where I needed to load a CSV into SQLite for analysis.

Since I still had the file, I ran some queries on the same file via Q to test it out. And it works great! (With some handholding to get quoting and delimiters right). The major downside was it's pretty slow, presumably because it reloads the file into a SQLite database each time you run a command. So, I'll probably stick with loading CSVs into SQLite myself, but I could see this being a useful tool for running one-off analysis on data from stdin.

</comment></comment><comment><USER46> Not sure how they compare, but this reminded me of a thread about textql from earlier this year:

[https://news.ycombinator.com/item?id=16781294](https://news.ycombinator.com/item?id=16781294)

</comment><comment><USER47> Piling on with pre-existing tools: The demo for calcite is a CSV to SQL wrapper.

[https://calcite.apache.org](https://calcite.apache.org)

</comment><comment><USER48> This would make a nice complement to xsv and csvkit but really needs to support python3. That is the main downside to using it.

<comment><USER49> csvkit actually supports SQL queries on csv files. But performance is pretty poor.

</comment></comment><comment><USER50> Is there any use for this not handled easily by lnav?

[https://lnav.org](https://lnav.org)

</comment><comment><USER51> Great! I just wonder why did it take so long for somebody to finally invent this...

<comment><USER52> It didn't. See the other comments for about a dozen of already existing similar tools.

</comment></comment><comment><USER53> You can do something very similar to this using R's sqldf package.

</comment><comment><USER54> We are doing this with Apache Drill on MapR and it works wonderfully.

</comment><comment><USER55> What is the performance like compared to say Oracle External Tables?

</comment><comment><USER56> Is there something similar for JSON? Thanks

<comment><USER57> There's jq, which doesn't support SQL but does allow you to apply filters and reductions to data in JSON files: [https://stedolan.github.io/jq/](https://stedolan.github.io/jq/)</comment></comment>


Show HN: Stringified March Madness Brackets - <AUTHOR> http://tweetyourbracket.com/S195463721432121E185463721432424W195463721432121MW18541131021432434FFSMWMW

<comment><AUTHOR> The link is to Five Thirty Eight's predicted bracket, displayed on [http://tweetyourbracket.com](http://tweetyourbracket.com) which is my bracket challenge site.

This came out of an idea to see how/if brackets could be stored as strings and then manipulated, displayed, scored, validated, etc.

All the code is on GitHub. [https://github.com/tweetyourbracket](https://github.com/tweetyourbracket)</comment>


Google Translate's camera now works with more than 100 languages - <AUTHOR> https://www.engadget.com/2019/07/10/google-translates-camera-now-works-with-more-than-100-languages/ <comment><USER2> The sheer scale and technical complexity is mind boggling for my little brain. How does one even start designing so huge?</comment>