Kubernetes 1.10 released - <AUTHOR> http://blog.kubernetes.io/2018/03/kubernetes-1.10-stabilizing-storage-security-networking.html <comment><USER3> Is there an easy way to get a single-node production-ready Kubernetes instance? I'd like to start using the Auto DevOps features that GitLab is adding, but all the tutorials I can find either have you installing minikube on your laptop or setting up a high-availability cluster with at least 3 hosts. Right now I'm using CoreOS and managing Docker containers with systemd and shell scripts, which works all right but is tedious and kind of hard to keep track of. I don't have anything that needs to autoscale or fail over or do high availability, I just want something that integrates nicely and makes it easy to deploy containers.

EDIT: I should have clarified, I want to self-host this on our internal VMWare cluster, rather than run it on GKE.

<comment><USER4> Sure. Install kubeadm on the node, "kubeadm init", install a pod network, then remove the master taint

<comment><USER5> Reminds me of the plumbis "How is it made".

</comment></comment></comment><comment><USER6> I'm very curious to hear field reports from people who switched to using Kubernetes in production in the last year or so. Why'd you do it? What got better and what got worse? And are you happy with the change?

<comment><USER7> One data point: I've wanted to but so far have not made much progress. I'd say my biggest impediment has been documentation: I can get it installed, but making it work seems to be beyond the scope of the documentation. I got closest once I found out about "kubespray" to install the cluster rather than using the official Kubernetes installation docs process.

I spent a couple weeks not quite full time going through tutorials, reading the documentation, reading blog posts and searching for solutions to the problems I was having. My biggest problem was with exposing the services "to the outside world". I got a cluster up quickly and could deploy example services to it, but unless I SSH port forwarded to the cluster members I couldn't access the services. I spent a lot of time trying to get various ingress configurations working but really couldn't find anything beyond an introductory level documentation to the various options.

Kubespray and one blog post I stumbled across got me most of the way there, but at that point I had well run out of time for the proof of concept and had to get back to other work.

My impression was that Kubernetes is targeted to the large enterprise where you're going to go all in with containers and can dedicate a month or two to coming up to speed. Many of the discussions I saw talked about or gave the impression of dozens of nodes and months of setup.

Other options I'll probably look at when I have time to look at it again: Deis [https://deis.com/](https://deis.com/) , Dokku [http://dokku.viewdocs.io/dokku/](http://dokku.viewdocs.io/dokku/) , Flynn [https://flynn.io/](https://flynn.io/) , LXC [https://linuxcontainers.org/lxc/introduction/](https://linuxcontainers.org/lxc/introduction/) and (though I'd been trying to avoid it) Docker Swarm [https://docs.docker.com/engine/swarm/](https://docs.docker.com/engine/swarm/)

<comment><USER8> Are you trying to play around, or set up a working cluster? If you just want to play around, I'd suggest just using minikube to get things going.

Anecdotally, I got an HA cluster running across 3 boxes in the space of about a month, with maybe 2-3 hours a day spent on it. The key for me was iterating, and probably that I have good experience with infrastructure in general. I started out with a single, insecure machine, added workers, then upgraded the workers to masters in an HA configuration.

I don't think it is really that hard to get a cluster going if you have some infrastructure and networking experience, especially if you start with low expectations and just tackle one thing at a time incrementally.

<comment><USER9> Full Disclosure: I work for Red Hat in the Container and PaaS Practice in Consulting.

At Red Hat, we define an HA OpenShift/Kubernetes cluster as 3x3xN (3 masters, 3 infra nodes, 3 or more app nodes) [0] which means the API, etcd, the hosted local Container Registry, the Routers, and the App Nodes all provide (N-1)/2 fault tolerance.

Not to brag, since we're well practiced at this, but I can get a 3x3x3 cluster in a few hours, I've lead customer to a basic 3x3x3 install (no hands on keyboard) in less than 2 days, and our consultants are able to install a cluster in 3-5 working days about 90% of the time, even with impediments like corporate proxies, wonky DNS or AD/LDAP, not so Enterprise Load Balancers, and disconnected installs. Making a cluster read for production is about right- sizing and doing good testing.

[0] [http://v1.uncontained.io/playbooks/installation/#cluster- des...](http://v1.uncontained.io/playbooks/installation/#cluster-design- architecture)

<comment><USER10> One last challenge. Can you do all the setup without being root?

<comment><USER9> As long the user can install packages (say, via /etc/sudoers file), make config changes, Yes. That's supported by our installer [0].

[0] [https://github.com/openshift/openshift- ansible/blob/master/i...](https://github.com/openshift/openshift- ansible/blob/master/inventory/hosts.example#L32-L43)

</comment></comment></comment></comment></comment></comment><comment><USER11> Hi..can someone ELI5 to me what Kubernetes is? Also what's the best way to get started/tutorials you can recommend for a new user? Thank you!

<comment><USER12> To follow this up, can anyone explain the benefits over Docker? I've used Docker before but am unfamiliar with Kubernetes terminology. I do understand it's an open source project by Google.

<comment><USER13> Kubernetes USES docker, so it's not a competitor.

The main benefits over the competing docker project, Docker Swarm, is that it does WAY MORE, is 100% free and open source, and has much better adoption.

I would argue that with Docker Swarm you have to bring the glue yourself, and it doesn't really solve any of the hard problems. Kubernets on the other hand is an all-in-one package that solves a LOT of hard problems for you.

<comment><USER12> +1 for open source then. Thanks for the explanation.

</comment></comment></comment></comment><comment><USER14> They need some sort of LTS versioning. Keeping up with their breakneck development pace is a job all its own.

<comment><USER15> Kops has not released a 1.9 version yet. Even k8s projects can't keep up.

<comment><USER16> Kops generally stays one release behind. 1.9 is being end to end tested in the last week. I wouldn't use that as an example of not keeping up, it's the established cycle for the project.

<comment><USER17> kops 1.9 is very close to ready now, but this is a longer lag than normal. We've historically released kops 1.x when we consider that k8s 1.x is stable, including all the networking providers and ecosystem components. That's typically about a month after release.

User feedback has been that that we want to keep that, but that we should also offer an alpha/beta of 1.10 much sooner, so that users that want to try out 1.10 today can do so (and so we get feedback earlier). So watch for kops 1.10 alpha very soon, and 1.11 alpha much earlier in the 1.11 cycle.

<comment><USER18> For those that aren't aware, justinsb is the author of kops. Looking forward to the 1.9 release.

<comment><USER17> Ah - sorry, probably should have disclaimered that! I did write the original kops code, but now there's a pretty active set of contributors working on kops (and contributions are always welcome and appreciated!)

</comment></comment></comment></comment></comment></comment><comment><USER19> Can anybody share their experiences with running applications that use persistent volumes on bare metal kubernetes?

I mean without cloud services like Google cloud persistent disks.

<comment><USER20> +1 for "regular" Ceph. Don't bother with that rook stuff. Just setup a regular Ceph cluster and go. Kubernetes handles its stuff and a (much more reliable and stable) Ceph cluster handles blocks and files.

<comment><USER21> Can you explain what's wrong with Rook? I thought it was supposed to make life easier when running Ceph.

</comment></comment></comment><comment><USER22> Openshift from redhat is built on kubernetes. Openshift offers a free tier to try out their cloud services. I'd recommend it to anyone who wants to try it out.

I'm not affiliated with redhat in any way but I have enjoyed using the openshift platform.

Here's the link for anyone interested: [https://www.openshift.com/pricing/index.html](https://www.openshift.com/pricing/index.html)

<comment><USER23> [http://openshift.io](http://openshift.io) is also built on Kubernetes (by virtue of using openshift.com for its deployment pieces)

</comment></comment><comment><USER24> Good to see CSI is gaining traction. Only if we could build containers inside K8S pods safely, I would be very happy. Maintaining a stupid Docker cluster for just building containers is really a burden.

<comment><USER25> Seems someone else was able to get it that working: [https://blog.jessfraz.com/post/building-container-images- sec...](https://blog.jessfraz.com/post/building-container-images-securely-on- kubernetes/)

<comment><USER26> If the gitlab ci provider for kubernetes would get this feature, it'd be amazing.

I could finally run gitlab's CI safely on kubernetes and generate containers.

<comment><USER27> i have a problem of running gitlab runner within k8s. the docker layer is not cached. is there anyway to fix this?

<comment><USER28> In my experience the GitLab Runner on k8s should utilized the cache. Are you using Docker-in-Docker by chance? By default, I don't think it can cache data between jobs.

<comment><USER27> can you share your config.toml for k8s runner?

</comment></comment></comment></comment></comment></comment><comment><USER29> I switched to k8s about a year and a half ago before deploying a modest frontend/backend pair. although there is some friction, I generally like the approach (I used to use borg, so it's a pretty low barrier).

The biggest problem I have is debugging all the moving parts when there are ~10+ minute async responses to config changes.

<comment><USER30> 10 minutes? That sounds a lot. We usually have 1-2 minutes wait time for a changed ConfigMap to reach a running pod, or for a Deployment to roll over to a new ReplicaSet. (That's on k8s 1.4.)

</comment></comment><comment><USER18> Has anyone been using CoreDNS as Alpha in 1.9, or tried the Beta now in 1.10? What was your use case and reason for switching? How is it better than KubeDNS?

<comment><USER20> We've been using CoreDNS for a bit now, and it's much better than KubeDNS. We found that KubeDNS would time out and drop requests from time to time. No such issues with CoreDNS. Would recommend (at least from a reliability standpoint).

<comment><USER18> Thanks for sharing. How are you monitoring timeouts and dropped requests on KubeDNS?

<comment><USER20> We were running with shorter TTL's on service records, and upstream apps threw rashes of errors when queries timed out.

</comment></comment></comment></comment><comment><USER31> I see some people who use Elixir/Erlang ecosystems then shove them into a kubernetes system. Isn't this going against what the Elixir/Erlang system already provides ? What are the usecases for this?

<comment><USER32> > Isn't this going against what the Elixir/Erlang system already provides?

No?

So, I guess, let's dig into that a little bit. Erlang's always kind of left the health and safety of your deployed system as a whole up to you, being preoccupied with giving you tools for understanding and maintaining its internal operation. OTP provides two semi-unique things: supervision of computation with control of failure bubble-up and hot code reloading. Hot code reloading isn't used all that much in practice, outside of domains where it's _very important_ that the whole system can never be offline and load balancing techniques are not applicable. That's a specific niche and, sure, probably one that kubernetes can't service. With regard to supervision, there's no incompatibility between OTP's approach and a deployment consisting of of ephemeral nodes that live and then die by some external mechanism. Seems to me that kubernetes is no different a deployment target in this regard than is terraform/packer, hot-swapped servers in a rack or any of the other deploy methods I've seen in my career.

</comment></comment><comment><USER33> For the folks who have implemented K8 in Production - curious if you use it to do resource management as well, or, if you also use something like Mesosphere in conjunction? Or would you stick with a like-stack (DCOS+Marathon)?

There is surprisingly little online discussion/documentation on the intersect of Resource Management and Container orchestration. Not sure if it is too early in the curve, a dark art, not actually done, or what...

<comment><USER34> Kubernetes does resource management; all cluster scheduler systems do. The difference between Kubernetes and Mesos for resource management is that resource requests and limits are optional in Kubernetes and mandatory in Mesos. It is best practice to specify resources in Kubernetes.

</comment></comment><comment><USER35> I'm still wondering about going with plain kubernetes, or investing time in OpenShift. Any insights from people that have tried both ways ?

<comment><USER36> I've run vanilla k8s for about 3 years now in prod but am also fairly familiar (and really like) Openshift Origin. I usually tell people asking this question the following;

OO comes with a bunch of really nice quality of life improvements that are missing (but in a lot of cases can be added via 3rd party TPR/CRDs/etc) in k8s but you aren't deviating so far from k8s that you won't be able to go work on vanilla k8s in the future. Not at all. Most of the additional stuff are annotations that simply wouldn't do anything and you'd remove them if you moved from OO to k8s.

I think that if you're brand new to the environment OO can really help you get running quickly. You just have to make sure that you do in fact dive in to the actual k8s yaml and deal with ingresses, prometheus, grafana, RBAC, etc at some point. I haven't used OO in awhile but I believe you could successfully do most of what I do day to day via yaml/json through the OO UI.

On the flip side a lot of people will probably tell you to start from k8s, whether that's GKE or AWS or minikube or wherever and go through the k8s the hard way. Personally, and I help people quite frequently on the k8s slack, I feel like that leads a lot of people down a path of frustration. It may be perfect for your style of learning or it may just scare you off.

Now when it comes to OO you are at the behest of their releases. Their most recent release was Nov 2017 and k8s 1.10 was just cut. I'm not sure what version OO is on now, evidently they changed their versioning numbers to not correspond to the k8s version.

Join #kubernetes-users and #kubernetes-novice on slack.k8s.io if you need any help. It's a vibrant community. You can message me directly @ mikej if you'd like.

edit: Ok OO 3.7 is k8s 1.9, that's perfect. I wait a few months before jumping into new major.

<comment><USER37> I'm about to cut OO 3.9 (based on 1.9) - we've been waiting for the subpath CVE fixes and regressions to get sorted out before we cut a release.

<comment><USER36> Awesome! I mentioned this in #openshift-users. The OO website still states;

> An OpenShift Origin release corresponds to the Kubernetes distribution - <USER2> > example, OpenShift 1.7 includes Kubernetes 1.7.

I had to dig around to figure out the version, might want to update that. :)

And great work, btw, OO is fantastic.

</comment></comment></comment></comment><comment><USER38> Something I've been wondering about: how stable is Kubernetes service discovery? I.e. can it entirely replace something like eureka? Is there any reason not to use Kubernetes provided service discovery?

<comment><USER39> Depends if _all_ service discovery sources and targets are all within k8s. If so, k8s works well, if not, not so much.

</comment></comment><comment><USER40> Funny that this and Soloman leaving Docker showed up on the same page.

</comment><comment><USER41> I am hearing that everybody is interested in Kubernetes, yet relatively few people are actually using Kubernetes in production.

Are you using Kubernetes? And if not, what is your reason not to?

<comment><USER42> I'm not so sure about that last part. We (Kinvey, www.kinvey.com) have been using it for customer-facing services in production for over a year. Results have exceeded our expectations.

</comment></comment><comment><USER7> Funny so many people are talking here about how fast Kubernetes is moving... That's one of the complaints about Docker Swarm that led me to rule it out...

</comment><comment><USER43> I am looking for,

Kubectl auth login, is it available yet ?

</comment><comment><USER44> so cool, would be even cooler at 1.20</comment>


Snapchat Valued at $10B, Kleiner Perkins Invests - <AUTHOR> http://online.wsj.com/articles/kleiner-perkins-caufield-byers-agrees-to-invest-in-snapchat-1409088794

<comment><USER2> (╯ಠ_ಠ)╯︵ s||ɐʍʎɐd

Anyone have the text of the article?

<comment><USER2> Got it:

Snapchat Fetches $10 Billion Valuation

Kleiner Perkins Investment Gives the Messaging Services a Shot in the Arm

Kleiner Perkins Caufield & Byers has agreed to invest in Snapchat Inc., an ephemeral messaging service, at close to a $10 billion valuation, people with knowledge of the matter said.

Snapchat, which has talked to several potential investors in recent months, is in the process of raising a large investment round that would make it one of the most valuable private, tech startups despite virtually no revenue.

The venture-capital firm committed to invest up to $20 million in May, one person added.

At least one strategic investor has also committed to invest in the round, which isn't yet closed, two of the people said.

A spokeswoman for Kleiner Perkins wasn't immediately available for comment. A Snapchat spokeswoman said: "The valuation of our business and our capital requirements are the least exciting aspects of supporting the Snapchat community. We have no further comment at this time."

Snapchat's valuation, at about $2 billion last year, has soared even as the startup lacks a clear business model. Investors see potential to make money from the millions of teenagers and college students on the three-year-old app, a group that is highly coveted by advertisers and increasingly difficult to reach through traditional media like television and magazines.

New capital could help Snapchat gear up for its first foray into advertising, expected later this year. The company has held talks with media companies and advertisers in recent weeks about a new content service called Snapchat Discovery, a person familiar with the matter told The Wall Street Journal earlier this month. The service that would let users read daily editions of publications, video clips and ads by holding down a finger on the screen, like they do with photos and other messages on the app before disappearing.</comment></comment>


POLYGRAPH.info - <AUTHOR> https://www.polygraph.info <comment><USER2> Counter-propaganda, iow.

> "Polygraph.info is a fact-checking website produced by Voice of America > (VOA) and Radio Free Europe/Radio Liberty."</comment>


"Pump-and-Dump" schemes in Telegram groups drive penny crypto coin prices up - <AUTHOR> https://www.achtungtechnik.de/english/2018-crypto-pump-n-dump-telegram.html <comment><USER2> The real issue with all the pump and dump, fear of missing out, and all the other crazy "mooning" is that it is ruining this whole space. It is creating a reality that all the haters of cryptocurrency have been talking about for a while.

Instead of focusing on building real applications with blockchain, programmable money, and changing the world everyone is focused on trading and get rich quick schemes. This stuff could change the world it is money ripped out of government hands and it could be baked into everything we do on the internet but I am not so sure that will happen anytime soon anymore.

I am very jaded with the whole space because of all this fraud.

<comment><USER3> There's no "application" of blockchain. You'd be rather thanking thoses schemes because they are feeding the hype of what would be seems as a useless techno otherwise.

I mean come on, anything can be done without it and using blockchain just bring unessential minor improvements like decentralization that no one cares about. Prove me wrong ...

<comment><USER4> It allows you to opt out of the monetary system that's been imposed on you by your govt. This might be less of a big deal if you live in a developed country but it's a big deal if you live in an unstable country.

<comment><USER5> Why not just hold most of your savings in dollars, euros, swiss franks, or whatever? Doesn't that just as effectively opt you opt?

<comment><USER4> Let's say you have to leave the country with all your saving. I'd much more comfortable carrying it in btc than cash.

<comment><USER5> You don't need to hold physical cash to hold USD. You could have a bank account in London, Geneva, Paris, or New York with your pounds, franks, euros, or dollars. If you need to leave the country, your savings are already outside of it. Just like your bitcoins.

The answer seems to be that repressive countries haven't yet put in place effective mechanisms to be repressive with respect to bitcoins. But there's zero reason intrinsic to bitcoin to expect that to continue. On the contrary, since everything is digital the choke points are easier to control, not harder.

<comment><USER6> Good luck opening a bank account in Geneva as a US citizen. Unless you bring really a lot of cash they won't touch you with a 5 ft pole.

<comment><USER5> If you consider the US an unstable country, I don't know what to tell you.

<comment><USER4> Not what he's saying. You can't open an account even if you are from the us, much less so Iran.

<comment><USER5> "Much less from Iran" misunderstands the problem with opening an account from the US. Swiss banks don't want to subject themselves to the long reach of the United States government's aggressive criminal justice system. That's the reason they won't open a bank account for a US system anymore. See, e.g. [https://www.swissinfo.ch/eng/tax-evasion_us-programme- nets--...](https://www.swissinfo.ch/eng/tax-evasion_us-programme-nets-- 1-billion-from-swiss-bank-fines/41858248)

The reason it was impossible, until recently, for an Iranian to open a bank account in Switzerland was not because of the Iranian governments, it was because the Swiss bank was afraid of running afoul of the US sanctions regime. Because those sanction have been lifted (at least for now) an Iranian _can_ open a bank account in Europe, albeit they'll still have to go through a fairly extensive KYC process.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER7> Just a week ago I saw a guy pump and dump a random altcoin live on Twitter just to prove his "haters" that he doesn't leave his followers holding a bag when he does his scheduled PnDs. The level of arrogance is really astounding. Hell, all of the biggest bitconnect scammers with a following on YouTube have already moved on to shilling other lending platforms instead of just disappearing. Interesting to see this unravel and how long it can continue.

</comment><comment><USER8> It's only legal if you wear a suit and do it on CNBC. This is my expert opinion as a super lawyer.

</comment><comment><USER4> Can anyone recommend any good books on things like this? Idk what "this" really is lol. Black hat economics?

I've setup a slack channel for crypto discussion because most groups and subreddits are too shilly and memey. [https://hncrypto.slack.com/join/shared_invite/enQtMjk5MDg2Mz...](https://hncrypto.slack.com/join/shared_invite/enQtMjk5MDg2MzAzMTIwLTEzNmU2NzVlNWFkMjNmZDI1ZGU1NzQ3MTZmNzM0NDU2OTc0Y2Y2NWMxZjBkZmI1MzMxMDllOGIzOTgwYWQ4MmU)

<comment><USER9> Here is a draft of a book: [https://cryptofrenzy.wordpress.com/2014/02/19/wolong- writes-...](https://cryptofrenzy.wordpress.com/2014/02/19/wolong-writes-a- book/)

The author is allegedly the guy behind the Dogecoin pump 2013.

<comment><USER10> Dogecoin actually managed to pull a pump-dump-pump? Its valuation looks quite good as we speak, even though I remember the dump (2015) [1]

[https://motherboard.vice.com/en_us/article/78xqxb/the-guy- wh...](https://motherboard.vice.com/en_us/article/78xqxb/the-guy-who-ruined- dogecoin)

</comment></comment></comment><comment><USER11> I don't think that this actually has any mid/long (even short depending on how you define that) lasting impact on the market. If you watch any pump and dump charts no matter how big the group is, you see that the price is exactly the same as before mere minutes after their little schemes... The only people who make money off of this are the ones that organize it and buy way before everyone else, or maybe a lucky few very fast buyers, but that's pretty much it. I'm pretty sure even people who try this don't do it more than once.

</comment><comment><USER12> Didn't the Winklevoss twins of Facebook fame buy like 20% of all bitcoin some time ago?

What is their worth now?

[https://www.google.com/amp/s/amp.businessinsider.com/bitcoin...](https://www.google.com/amp/s/amp.businessinsider.com/bitcoin- billionaires-winklevoss-twins-2017-12)

</comment><comment><USER13> Crypto seems to be a perfect example what I think big banks are doing in a "regulated space", eg. Downgrading all currencies except dollar, earning money on top of big clients ( pump scheme,... ) And etc..

It's just less obvious

</comment><comment><USER14> What other kinds of fraud that has occurred in the past (stock market, real estate, etc) are people going to try with crypto? So far we've seen some big ones (pump-and-dump, and pyramid scheme).

<comment><USER15> Wash trading: [https://medium.com/@bitfinexed/wash-trading-bitcoin-how- bitf...](https://medium.com/@bitfinexed/wash-trading-bitcoin-how-bitfinex- benefits-from-fraudulent-trading-8bd66be73215)

Allegedly unbacked reserve currencies (tether): [https://www.coindesk.com/tether-confirms-relationship- audito...](https://www.coindesk.com/tether-confirms-relationship-auditor- dissolved/)

I'm sure there are more, any scheme/scam that gives someone an edge _will_ be tried in an unregulated market.

</comment></comment><comment><USER6> Would this go away with more regulation? Is it fraud?

<comment><USER16> Pump and dump is absolutely fraud (as it requires deception to work--note the "promote on social media" step), and you can go to prison for it. Jordan Belfort, the "Wolf of Wall Street" mentioned in the title, did.

<comment><USER17> I think if these were regulated securities being traded on regulated exchanges it certainly would be fraud and a jailable offense. But is it fraud for something like collectible beanie babies? (as far as I can tell that is basically how cryptocurrencies are regulated right now in the US).

<comment><USER16> If you make money by getting groups together to sell Beanie Babies falsely claiming that you have insider info that the fad is going to make a comeback, that is fraud, and you can go to prison for it.

<comment><USER18> How does this translate over to things like fashion/social trends in general?

Something like "Corsets are coming back for spring 2018! Get on top of the trend at XYZ Fash Inc.!" hardly seems like fraud, but by that definition it would be.

<comment><USER16> This is where a lawyer would have to answer. I'm certain that a detailed explanation would involve a mountain of cases and precedents. (I would guess that a lot of the precedent hinges on whether you are intentionally lying, or whether you are merely making a good-faith prediction. Pump-and-dump schemes are obviously the former.)

The point is that, in general, deception for financial gain is fraud, more or less by definition. It doesn't have to involve regulated securities.

</comment></comment></comment></comment></comment></comment><comment><USER19> Just like stocks [http://investorshub.advfn.com](http://investorshub.advfn.com)

</comment><comment><USER20> "You can't con an honest man."

<comment><USER21> You're not saying very much here, but...

If you're someone looking to get rich investing in cryptocurrency, it's worth considering where the money (not the 'value') actually comes from. People have to spend it, and you have to take it. If you don't care where it comes from, you may as well be stealing it.

<comment><USER20> > If you're someone looking to get rich investing in cryptocurrency

I'm not. I'm really really not.

<comment><USER21> I wasn't referring to you specifically.</comment></comment></comment></comment>


Remastering Star Trek: Deep Space Nine with Machine Learning - <AUTHOR> https://captrobau.blogspot.com/2019/03/remastering-star-trek-deep-space-nine.html <comment><USER2> ST:DS9 was originally shot on film like ST:TNG, but the market performance of the TNG remaster makes it very unlikely that DS9 will receive the same treatment.

 _" Essentially, for the HD release of _Star Trek _, all people had to do was scan each episode. For_ The Next Generation _, they would have to scan all those original pieces of film and then edit together each episode again, themselves. It's more difficult, more expensive, and much more time- consuming._

 _" Unfortunately, it wasn't actually worth it. Sales of the extravagant_ TNG _remaster—original retail price $118 for just season one—failed to reach CBS and Paramount's expectations. A similar process would have to be done for both_ DS9 _and_ Voyager _—and would actually be even harder. "_

[https://io9.gizmodo.com/the-detailed-depressing-reason- deep-...](https://io9.gizmodo.com/the-detailed-depressing-reason-deep-space- nine-and-voy-1791962332)

<comment><USER3> > original retail price $118 for just season one

Well, there's your problem right there. Expecting three figures for a single season of _anything_ is just pure madness.

<comment><USER4> I think my shock price is roughly in the ballpark of 15 to 20 USD per season, and a maximum of roughly 100 USD total for an entire show.

Yes, this means if a show lasted more than 5 to 7 seasons I expect a discount on the remastered version!

<comment><USER5> I want an online version that will exist for at least 10 more years preferably 20 for $100 with the DVD copy, sold by one of the giants with the budget to sustain that.

</comment></comment></comment></comment><comment><USER6> Wow, so this is _fascinating_. Some observations:

\- Many of the frames look like someone just applied a "sharpen" filter -- there's (as expected) no real new information, it's just sharper... so it doesn't seem like a big deal, like I could do it in Photoshop trivially

\- But then there are a few spots where new details are truly seamlessly added... the fireball in the spaceship explosion, and forehead wrinkles. Stunning, they're absolutely seamless and believable, with detail that is simply not there in the original... that's literally magic

\- But at the same time, characters in the background that are slightly out- of-focus get oversharpened when they're supposed to be blurry, like it can't tell when moderate blurriness is due to resolution or focus

Overall, I'm pretty shocked that the effect is so seamless across frames -- I totally would have expected this to produce weird discontinuities in time, but I didn't see any at all. I mean, this actually seems like it's already production-ready to throw into TV's or VLC. Which is _crazy_.

I wonder how much of this is "general-purpose", or to what extent this works on this episode because it's trained on this episode? E.g. the neural network is learning from a close-up of the Ferengi face or spaceship, to apply specifically to a later smaller version. Or to what extent this will work well across TV shows, across actors, across genres, without prior training, or with sparse training?

<comment><USER7> He's using an off-the-shelf commercial app called "AI Gigapixel", which in turn uses a process called image super-resolution. It is general purpose (works for all kinds of images) and fairly thoroughly explored by now. hope this helps

</comment></comment><comment><USER8> I don't see a huge difference between the two. Mostly the Ferengi forehead creases are sharper I guess, but unclear that the overall effect is much of an improvement. Cool proof of concept though!

<comment><USER9> Yeah I made sure the video was in 1080p mode and it was mostly subtle at best. Not only that there were several places where it wasn't subtle, but the up- rezzed faces had a super fake photo-retouched/cgi look to them. They had crossed the uncanny-valley and didn't look like real humans.

<comment><USER10> I thought that faces were mostly fine. The spots that didn't look right, to me, were when there was a face out-of-focus in the front of the frame (the bit where Sisko's talking to that dude). I wonder if the network recognized it was "supposed" to be out-of-focus, which it is, but then wasn't able to do any edge cleanup to make it more consistent.

</comment></comment></comment><comment><USER11> Interestingly, there's a project for a documentary about DS9 that's close to completion.

[https://www.indiegogo.com/projects/what-we-left-behind- star-...](https://www.indiegogo.com/projects/what-we-left-behind-star-trek- deep-space-nine-doc)

It has been delayed due to master the old footage (it's one of the last updates)

I'll leave this work it as a comment on the project.

<comment><USER12> I had no idea about this!

How did they manage to get the entire cast, EXCEPT Avery?

<comment><USER13> Avery Brooks has distanced himself from DS9 intentionally; he considers that part of his career done.

</comment></comment></comment><comment><USER14> This is interesting, but it's still not the same as a proper remaster. You can't get information out of SD video that wasn't there before with upscaling, so the algorithm is just interpolating. It's too bad that Paramount is so fan- hostile; they could just get the original film footage (assuming it was shot that way; ST:TOS and ST:TNG were, which is why we have excellent remastered versions today), scan it in as raw video, let the fans have at it, and they'd make the high-def FX for free.

<comment><USER15> No, it is not just interpolating. The underlying algorithm uses machine learning by applying a trained deep neural network. So there is value added besides a mere upscale.

You're ultimately right, though, and that a true HD is only going to come from the raw film content. What the neural network gives us are essentially plausible higher-res hallucinations.

Edit: as per the other comment, if the original exists only on video and not film, perhaps this is the best we're going to get.

<comment><USER16> That's still interpolating, by the definitions I know.

The main difference here is that the interpolation algorithm on your TV is online. It's handling 30 frames per second, over 9 million pixels per second. Doing the interpolation offline (ahead of time), you can take as long as you want, look at multiple frames to try to make better guesses, try multiple things and use some fitness measure to pick a winner, even a frame or a pixel at a time.

It's still interpolation.

<comment><USER17> No, if I interpolate a sequence 200, 400, 600, ... I might get 200, 300, 400, 500, 600, 700. I've not added info. If I look at real world situations and find that whilst the figures fall at the even hundreds it's more realistic that they fall in a range from 20 to 30 points below the hundred on odd- hundreds. Then I have added information, albeit statistically, and the resulting sequence like 200, 287, 400, 475, 600, 672 is no longer raw interpolation.

In this case they're using machine learning to add additional information about textures that isn't in the footage broadcast. They can add frames by interpolation, but the ML texturising and detailing is not interpolation.

Starting with a blob, if you interpolate you get a smoother blob, with this process you get a more structured figure.

<comment><USER18> It's more like hallucination than anything. You're just forward-projecting your assumptions on what things _ought_ to look like and hallucinating detail that just isn't there.

It can still look nicer than naive upscaling though.

</comment></comment></comment></comment></comment><comment><USER19> Pro-tip: You have to change the YouTube setting so that you are watching in 1080 HD, not the 360p that it can default you too.

You know, I think there is some real promise here!

With some HD remasters, you can start to really see the makeup, the little pores in everyone's skin, the smudges and uncleanliness of real film-making. Film-makers and directory choose the lighting and the focus with the end- product in mind. They _know_ that the screen won't capture certain things and so they know where they can skimp and save [0]. When you re-master it, you're going against the 'vision' of the directors. Not in a _big_ way at all, it's very subtle. But it's still there.

With ML techniques, you get the 'idea' that the director was going for, without seeing all the screw-ups that they knew they could get away with. It's crisper but the idea and vision are the same [0].

Peter Jackson's recent 'They Shall No Grow Old' is another great example of using ML too. In that case it was to preserve the old WW1 footage, bring it's frame-rates up to modern 24 fps, 3-D it, and colorize it. The results are literally breath-taking. Personally, I gasped when it finally hits; it's that good. Not to geek out too much here, but Jackson is _literally changing history_ with that film. He changed the way we all view old footage, as something all herky-jerky and grainy, to something that is modern and real. Those 16 year old child-soldiers become _real people_ again.

Though Jackson's work is a lot different than this effort, I think we all know that ML and the movies are here to stay. It's relatively cheap to update, takes little time, and be profitable (Remember the Disney Vault gimmick?). How long will it be before Chaplin's 'City Lights' is ML'd and remastered into 24 fps, 3-D, color, and with sound? Maybe 5 years?

Hell, I'd pay to see the best of old cinema brought back to modern standards like that.

[0] I know no film-makers, this is supposition.

<comment><USER20> There's a Criterion Collection blu ray of City Lights that I would imagine was done very well given their reputation and the reviews [https://www.criterion.com/films/27558-city- lights](https://www.criterion.com/films/27558-city-lights)

</comment></comment><comment><USER21> What is the licensing situation for remastering like? It would be a pretty interesting competitive edge if a company like Netflix could improve their entire library with some inhouse remastering magic. Or even a bargaining tool for buying new licenses...sure, we'll add your stuff to our collection and remaster it "for free".

<comment><USER22> Would definitely get me more interested in handing over my cash... Though, in this case, I think it would more likely be Amazon Video.

</comment></comment><comment><USER23> People are using this same technique for old games - [https://venturebeat.com/2019/01/18/machine-learning-is- rescu...](https://venturebeat.com/2019/01/18/machine-learning-is-rescuing-old- game-textures-in-zelda-and-final-fantasy/)

</comment><comment><USER24> I kinda hope that perhaps, if CBS doesn't find value in doing a TNG-level remaster of DS9 and Voyager, that something like this might give them a "good enough" version for a Blu-ray release. At the very least, one could imagine using this technique where applicable to at least, massively cut down the conversion cost in scenarios where re-cutting the scene would be particularly cost prohibitive.

<comment><USER25> I do wonder if having DS9 and Voyager available on so many streaming platforms right now cuts down on the incentive to remaster them. I'm not sure if a remaster would be enough of a draw to CBS All Access if you can already watch all the episodes on the other major streaming services, but perhaps Star Trek Discovery is successful enough they might consider it?

<comment><USER26> I don't think it's fair to call ST: Discovery a part of the franchise at this point.

It's this bizarre smodge of minority group agendas, dark horror-like scenes and modern political ideas disguised as the family friendly Star Trek that you could watch with your kids that we all actually loved.

And are the plot lines intelligent? Do they make you think about interesting (non overdone ie non-left) philosophical issues? Not really, and they're strung along in an awkward and long story arc that's pretty dull and predictable and takes away from the per-episode storytelling :/

<comment><USER24> I distinctly recall every flavor of "it's not real Star Trek" when Enterprise came out, and people still whine about the story choices in Voyager today. Heck, some people who were fans of the original series would be loathe to call TNG "real Star Trek". In fact, having people who complain it isn't real Star Trek might be the best proof that Discovery is, in fact, real Star Trek. ;)

Star Trek has always been incredibly progressive/left-leaning. The Federation is a communist utopia where everyone has everything they could ever want for free. If you talk to people on the original series, plenty of them will talk about things that they _wanted_ to do, in terms of gay characters and the like, but that they couldn't get permission to actually _air_. But they have always pushed the envelope of being about as far left as they can get away with from a business side perspective.

<comment><USER27> Yeah... this is a series that had the first interracial kiss, a black woman professional during a time when black women could only be nannies and servants, and a Russian serving openly amongst Americans during the Cold War.

I've seen a lot of complaints about "liberal" and "feminist" agendas being pushed in Discovery. It baffles me. Do such people not know what the Federation is about? They're literally social justice warriors in space.

<comment><USER28> The funny thing is that the concept of inclusivity has always been at the core of _Star Trek_ , so if anything, _Discovery_ is simply keeping with the status quo.

</comment></comment></comment></comment></comment></comment><comment><USER29> "it took me about two days to get everything extracted, upscaled and put it back together in a way that was pleasing. This resulted only in the first five minutes of the episode being done."

Maybe someone at Google can use spare cloud time to remaster the whole series as their 20% time project? This remastering process would be easily parallelizable.

<comment><USER29> I forgot to add a :) because remastering Star Trek is clearly not Google's business, but it could be _someone 's_ business: video remastering/upscaling services for Hollywood. There's a lot of popular old film and TV content just waiting to be sold again to fans on Blu-ray or steaming.

</comment></comment><comment><USER30> I see a lot of comments from people getting excited about superficial quality improvements, and I'm mystified. It's nice I guess, but the story and acting are unchanged by this sort of thing, and isn't that what matters to Star Trek fans?

<comment><USER31> the low visual quality is becoming more and more visible as TV screens get bigger and better. It would be great to have those old shows in a quality that fits our new screens, that's all.

</comment></comment><comment><USER32> I recently tried to rewatch Babylon 5 but the resolution was terrible and it looked bad on my TV so I gave up pretty quickly. I hope someone does the same treatment to Babylon 5 episodes!

<comment><USER33> Unfortunately, that's probably the "last best chance" for a not-awful watching experience. At some point after the show wrapped, they lost all of their 3d models, so they will never be able to re-render a single ship.

EDIT: I didn't mention fan models as a possible approach because this _is_ WB/PETN we're talking about here. They aren't Paramount-dumb, but they seemed take that as a challenge, at least _vis à vis_ B5.

<comment><USER34> There weren't enough polygons on them to look good at higher resolutions; they'd really have to be redone anyway.

<comment><USER35> you are pretty wrong . Watch the original meshes at HD resolution : [https://youtu.be/uHAuK_lDkk0](https://youtu.be/uHAuK_lDkk0)

<comment><USER34> Did we watch the same video? It looks like a 2002-era video game when rendered at 1080p. It does not realistic at all; it is nothing like improving a film video of a model (film is basically ~5k equivalent and a model is.. infinite?).

<comment><USER36> I agree that it doesn't look particularly good. The question is whether it's due to a lack of polygons.

</comment></comment></comment></comment></comment></comment><comment><USER37> As I see more of these machine learning applications, where a neural network is trained on a dataset, then applied to a similar dataset, I wonder how well it would help if the network was trained on part of the new dataset as well, as some kind of "specialization".

One could take a generic "AI Gigapixel" net, and retrain it on some of the newer Star Trek content, for which an upscaled/remastered "ground truth" exists. My guess is that it would help a lot with features that are specific to the Star Trek universe.

Taking this further, one could take the resulting samples, and "rate" them, before feeding it back in the training engine. This would make some kind of "adversarial" feedback loop, but instead of the GAN, humans are involved in the loop (which could be used to train an adversarial network as well, that said). My hope is that it would converge to much better results in a shorter time, and with less input data.

My apologies if those are common concepts in ML. If so, I'd love to look at some references that could further my understanding on these topics, and get me to use the right terminology.

</comment><comment><USER38> This seems like it would really benefit from a model trained specifically to upscale video. There's a lot of information you can get from the structure between frames that you can't exploit with a still image.

</comment><comment><USER39> im not discounting ML being able to do this better. kudos to the op. but for fun i tried it using conventional methods (NR+sharpen); similar results but an order of magnitude faster. you be the judge:

[https://pasteboard.co/I6tvYmP.png](https://pasteboard.co/I6tvYmP.png)

</comment><comment><USER40> On the one hand I'd love to see them remastered and touched up, but on the other hand, I originally watched them on a ~27 inch, old mid 90's TV....so the visuals aren't really the part which hits my nostalgia nerve.

</comment><comment><USER41> Alright, now I want to do this with Babylon 5 since it seems like the HD remaster from WB/Amazon petition has flopped. Heck, even just upscaling the SFX shots from 360p to full DVD quality would be a worthwhile project.

</comment><comment><USER42> I'm curious as to why the author chose "AI Gigapixel". What made it a better choice than say NNEDI3 or SuperRes or any other NN upscaler (there's a bunch of them out there)? Were others even considered?

</comment><comment><USER43> Anyone compared this with MadVR?

</comment><comment><USER44> Anyone knows how AI Gigapixel differs from DLSS? (outside Topaz Labs ofc). Thanks!

</comment><comment><USER17> Are films/programmes now preparing for future re-rendering in any way?

</comment><comment><USER45> Star Trek: Deep Space Nine aka, Cheers in Space!

</comment><comment><USER46> Any Friends of DeSoto here?

</comment><comment><USER47> Holy shit, you did exactly what i wanted to do after i read about the final fantasy results!11 :D

</comment><comment><USER48> You may find the original blog post from the creator here:

[https://captrobau.blogspot.com/2019/03/remastering-star- trek...](https://captrobau.blogspot.com/2019/03/remastering-star-trek-deep- space-nine.html)

<comment><USER49> The URL for this post should really be changed to that. It's nothing more than a two sentence blogspam submission with a link to the original creator.

<comment><USER50> Done! Updated from [https://www.turingtribe.com/story/star-trek-fan-remasters- st...](https://www.turingtribe.com/story/star-trek-fan-remasters-star-trek- deep-space-nine-in-hd-using-machine-learning-yEHAXy7S7PxGZEQE9).

</comment></comment></comment><comment><USER51> [https://news.ycombinator.com/item?id=19406060](https://news.ycombinator.com/item?id=19406060)</comment>


Problems with the Ring Pro - <AUTHOR> https://www.reddit.com/r/ringdoorbell/comments/8supnz/what_you_need_to_know_about_the_ring_doorbell_pro/ <comment><USER2> The comments on that Reddit post seem to indicate a little of this isn't true.

Anecdotally, I've had a Ring Pro for over a year with out the batter issue mentioned. I did buy a transformer but the Ring installation manual was quite clear that this was recommended and it was cheap.

</comment><comment><USER3> The Juicero 2.0 fails as expected.</comment>


Can America's Premier Lightning Lab Revive Its Renegade Spirit? - <AUTHOR> https://www.atlasobscura.com/articles/lightning-lab <comment><USER2> My immediate question was "what's their annual budget anyway?".

It seems like the lab is funded primarily from its initial endowment, and that for 2017 they only needed an extra $225k [1]. What's unclear is whether the additional support from NMT has stopped, NSF / grant funding for the equipment and studies has dried up, or something else.

I couldn't find any other details on the Langmuir Trust/Endowment.

[1] [https://www.nmt.edu/leadership/minutes/regents_minutes_septe...](https://www.nmt.edu/leadership/minutes/regents_minutes_september_2016.pdf)

</comment><comment><USER3> If what they need are more lightning bolts then they may need to branch out - For instance I've heard Los Alamos has higher than average incidence of lightning.

It sounds more like they're short on funds and students. It might be hard to draw students with the large vacuum of the tech industry sucking up all the best talent. So maybe they need a private industry "pivot"? Ie look for applications which could lead to patents and companies looking to develop them into products? Just articles like this are good though for drawing public interest.</comment>


The Art of SATergy: Excerpt from game theory book concerning multiple choice - <AUTHOR> http://freakonomics.blogs.nytimes.com/2008/12/18/the-art-of-satergy/

<comment><USER2> An interesting application of game theory to standardized testing. For the record, the single best approach to learning to score better on standardized tests is to read avidly in the language of the test. Here's how I put it in a FAQ message for high school students:

READ, READ

To learn how to score well on a standardized test reading section, the number one piece of advice is READ, READ, READ, and READ. Read about what you like to know more about. Read things that are fun for you. Find books and magazines about interesting topics and read them. Turn off the TV and read. Put away the video game controller and read. Read hard things, and read easy things. Read a lot.

For years, I wondered why it came so readily to mind to write "READ, READ, READ" in all capital letters like that when I give advice on this subject, as I have frequent occasion to do. Recently, I reread the section "Suggestions for Study" in the front matter of John DeFrancis's book Beginning Chinese Reader, Part I, which I first used to learn Chinese back in 1975. In that section of that book, I found this passage, "Fluency in reading can only be achieved by extensive practice on all the interrelated aspects of the reading process. To accomplish this we must READ, READ, READ" (capitalization as in original).

Also for the record, the SSAT test used by some secondary schools is not developed by as elaborate a process as the SAT test used for college admission. There is a rather higher likelihood that the SSAT will have poorly constructed questions. But for any entrance test, the first form of effective preparation is to read widely and avidly.</comment>


Ask HN: Received insults from founder, why can't we all just get along? - <AUTHOR>

Yesterday I received an abusive email from a well known founder (most people on HN would know them) after they discovered a new product of mine and decided they didn't like it.  Not only is this person WAY ahead of me in terms of their product and customers, they also have a very large and vocal following around the internet.
My question is, why bother?  Why can't we all just help each other out and offer support and encouragement?  Yes, I get business is competitive but I feel like we would all succeed more with cooperation rather than opposition.  Maybe it's because I've been around this industry a while now but I feel like it is getting worse too.
Does anyone else feel this way? What can we do about it? <comment><USER2> I was reminded of the quote by Marcus Aurelius-- "Choose not to be harmed and you won't feel harmed. Don't feel harmed and you haven't been."

The founders abusive email offers great insight into their mind. Some people insist to be heard. And of course, he is entitled to his unsolicited and wrong-headed opinion.

How you react to him is entirely under your control.

<comment><USER3> This may be true without the Internet, but with social media, a social justice warrior can start a nasty campaign against you for pretty much any reason and it could result in your career, business, and personal life being ruined.

<comment><AUTHOR> I agree, which is why I'm avoiding posting any details it's just not worth it.

</comment></comment></comment><comment><USER4> When people talk, they lose out on the advantage of psychology. compete with the future, not with their ego - (founders have an abundance of this).

Keep youre head up, watch the $$ fall down. Keep your head down, watch the $$$ go up.

<comment><USER4>
      obviously something on your end is being done right.</comment></comment>


Post Job - <AUTHOR> Hello I would like to post a new role for my company called Vela Trading Technologies. I need some assistance with this. Thanks very much! <comment><USER2> Please see the FAQ section about job posts: [https://news.ycombinator.com/newsfaq.html](https://news.ycombinator.com/newsfaq.html).</comment>


Ask YC: Do I still need web hosting if I am using Google Engine? - <AUTHOR>

If I already have my own domain, using Google Applications, Google Sites, and Google Engine. Do I still need web hosting to deploy my web based application? Any insights? Thanks. <comment><USER2> That depends on the specifics of the application and the traffic it receives. Perhaps you can tell us more about the application?

There are a subset of applications that App Engine is perfectly suited for at this specific point in time, but it is tough to give you an answer without more information.

<comment><AUTHOR> Basically it is similar to a simple accounting system. Thanks.

</comment></comment><comment><USER3> No, you don't.

<comment><AUTHOR> That will save some money. I only need to pay when my website successful enough so the usage is higher than the free quota. Hmm... cool...</comment></comment>


Flappy Colours in 1k - <AUTHOR> http://js1k.com/2014-dragons/demo/1726

<comment><USER2> Rage _and_ epilepsy-inducing! This is wasted on js1k. Hurry up and get it on the App Store. Millions of dollars await. ;-)</comment>


I am not running for president; I just want to meet every cow and Iowan - <AUTHOR> https://www.washingtonpost.com/amphtml/blogs/compost/wp/2017/08/03/i-am-not-running-for-president-i-just-want-to-meet-every-cow-and-iowan/ <comment><USER2> So that clears thing up</comment>


Ask HN: what questions to ask before joining a startup? - <AUTHOR>

For those that have had experience working for tech startups or hiring for their own tech startup, what are the questions a potential hire should be asking of that company?<p>What do you wish you would have asked about before joining? <comment><USER2> There was a prior discussion on this a while back: [https://news.ycombinator.com/item?id=7116357](https://news.ycombinator.com/item?id=7116357)

Also see: [http://mashable.com/2014/03/20/want-to-work-for-a- startup-10...](http://mashable.com/2014/03/20/want-to-work-for-a- startup-10-questions-to-ask-your-interviewer/)

[http://www.barcinno.com/5-questions-you-must-ask-yourself- be...](http://www.barcinno.com/5-questions-you-must-ask-yourself-before- joining-a-startup/)

[http://blog.expensify.com/2014/01/24/ceo- friday-5-important-...](http://blog.expensify.com/2014/01/24/ceo- friday-5-important-questions-to-ask-before-joining-a-startup/)

[http://onstartups.com/tabid/3339/bid/122/Joining-A- Startup-T...](http://onstartups.com/tabid/3339/bid/122/Joining-A-Startup- Top-6-Questions-You-Should-Ask.aspx)</comment>


I deleted Bash in Bash - <AUTHOR> https://medium.com/@jacobevelyn/i-accidentally-overwrote-bash-in-bash-e612da33da4b <comment><USER2> I once deleted libc on a Chromebook when trying to bootstrap a compiler toolchain for fun. I then discovered that ls is linked to libc. Luckily, wildcard matching and bash built-ins like echo still worked, and I had a copy saved somewhere, so with the help of echo * and cp I managed to come back from the brink of oblivion</comment>


San Jose mayor wants to require liability insurance for gun owners - <AUTHOR> https://www.axios.com/san-jose-mayor-gun-owners-liability-insurance-0af1883a-a801-41c5-a4e9-0433c48f835f.html <comment><USER2> because San Jose is pkagued with gun violence by registered gun owners. /sarc

<comment><USER3> San Jose is pretty violent.

<comment> dfeojm-zlib San Jose native. It hasn't been for many years.

Have you ever been to the south-side of Chicago, Detroit or Baltimore? Those are violent cities. San Jose isn't.

<comment><USER3> All of the US is remarkably violent. I'm comparing San Jose to cities in other countries. San Jose is more violent than London.</comment></comment></comment></comment>


Show HN: Twitterleaderboards.com - <AUTHOR> http://www.twitterleaderboards.com

<comment><USER2> very cool, just made a few of these will try it out. if it is good I'll share it. if its great I'll blog about it :)

 _only_ part I felt was a tad weird/confusing was the "nah make it so" towards the end, that phrase is a tad double negativ-ish or something in that realm.</comment>


Ask HN: review my startup, a planning/analysis app for cyclists - <AUTHOR> http://ridewithgps.com

<comment><USER2> Nice.

Questions I have: 1) Is your home location or post code displayed to anyone else? If so could you by default "approximate" the location to others unless a user specifies that they want their precise home location shown. Reason: Bike theft. Don't tell someone precisely where your bikes are, especially when combined with data showing when you're likely not going to be there.

2) Upload photos of equipment/accessories. Add a Flickr picker... I immediately balked at having to go to Flickr, download a photo just to re- upload... actually a URL in addition to a browse would sort this out pretty simply.

3) I'm pretty sure I said to hide my email address, yet options implies it's visible.

4) Time-zone... set it according to home location.

5) Health page... weight inc' bicycle and gear? Nope, you should put the weight of equipment on the equipment page... this data varies depending on what bicycle I take out.

6) Health page... age instead of date of birth?

7) Route drawing... remember prior preferences. Avid cyclists will start to begrudge always telling the tool that they aren't "Driving".

8) Full screen mode is nice, as is the elevation cross-section. A few of the route segments seemed to be wrong but I suspect this is Google's doing and not yours (it would send me needlessly down side-streets and correcting it is a pain in the ass).

9) Route planning should allow for the addition of markers. Think of things like: Lunch stop, viewpoint, toilet break, etc.

10) Consider a way of displaying several routes on the same page... think of scenarios like the Dragon Ride and the short and long rides sharing a considerable chunk of the route (show specific routes on the same page). Also consider scenarios like "I'm going to ride Ventoux, allow me to visually compare the routes in that area" (show routes that meet a search criteria on the same page).

11) Haven't yet used the GPS functionality, but you appear to have the basics covered.

12) I liked the embedding widget... but it's ugly. Does it have to be in black? Could you not have a prettier logo?

13) Before I enter in too much data I'd like to see when it's no longer free. Even if everything is free now, you should let people know at what point it ceases to be free. The issue is... "Why should I invest my time and data with you when I don't know how long the functionality offered will be offered to me at the price you're giving it to me (currently free)?". Even if you just say, we're not sure, but in this ballpark when we feel that we have a finished product... you'd increase the amount of trust and faith I have in supplying you with so much data.

14) Route sharing, route widgets... go beyond the simple. Look at this scenario: <http://www.londonfgss.com/thread31699.html> A single ride, with over 100 participants, that recurs and changes direction... yet it's obvious it should only be stored as one route, but you should allow multiple people to say that they're riding a route in the future (scheduled event rides as well as individual rides in addition to route planning). There's a big space for being the people who can route plan sponsored rides let alone community rides and things like that.

15) I'd consider adding ride photos or something to pull in photos from Google and show those. If someone is remotely route-planning (prior to a foreign vacation) they will want to know whether the place looks good... i.e. This: <http://ridewithgps.com/routes/23644> does not communicate the beauty of this: <http://images.google.co.uk/images?q=nesque%20gorges>

Overall it works really well with even the poorly mapped areas: <http://ridewithgps.com/routes/23644>

I run a large bike forum and will pimp it there and ask for more feedback for you: <http://www.londonfgss.com/thread35392.html>

<comment><AUTHOR> First off, wow! Thanks for taking the time to provide us with so many good points; it's hard to see many of the things you mention from a fresh perspective, as I have been staring at this project for a long while.

There is no option to hide your location, however it doesn't get more specific than the zip/city even if you used an actual home address to set the zip/city/etc. However, I agree it should be hidden and consider it done. Easy to add, I'll do so today. I have talked to a few people about adding a 'fuzz' factor to routes, so that the start/end doesn't touch your house. I am not sure how I want to implement this yet, so it hasn't been done. but, it is being thought about!

Soon we will have photo geotagging on routes. Should be an easy enough task, it's just been on the todo while we make what we have more solid. I like the Flickr integration and will investigate it.

I will clarify the email address hiding stuff, thanks for pointing out the confusion.

I haven't investigated how to assign timezone based on zip/city, but I would be surprised if Google doesn't provide that with geolocation information.

Thanks for reminding me about the weight problem. I will flag it as needing change, so your base weight is on your profile and gear weight on the bike. Additionally, I know people that want to track individual weight per ride, so I'll have to probably default to a riders last ride weight, profile weight or inputted weight when they log an activity.

Route planning currently does allow markers, they are called 'course points' (according to garmin), but that name is not too friendly. I'll massage that, but, you can currently add markers around the route it's just a kludgy UI. I am reworking that really soon, after I finish my week long list of bug fixes/improvements.

We have been working on multiple route display/comparison, including segment matching (same leg of even different trips), but it's very rough and not forward to the public. It will be cleaned up, released and expanded soon.

I agree the embed is ugly, it's just been so easy to forget it exists! It's on the list, highlighted, including adding more options for the person doing the embedding.

The product will always be free, but to a limited ability. I agree however, we have not expressed that very well. I'll sit down with Zack and Cam and come up with some explicit language that will help people with this.

We plan on adding events, most likely as a premium feature. I would love to have cyclists register and appear on the list of an event, with some way of showing it in their profile. We have been talking about this for a while, but haven't nailed anything down. It needs to (and will) be done before spring, when the riding season starts really taking off.

Thanks again for sharing our site around and generating the exposure. Traffic is already way up from both this post and your forum, so it is MUCH appreciated. I'll jump on that forum as well and get involved with the discussion. Additionally, I'll bug you later today when I have implemented some of your suggestions.

</comment></comment><comment><USER3> You list an awful lot of features on the home page. ("Plan, share, upload, analyze." are all features. Or take a look at all of your bullet points.)

Benefits sell software, not features.

Think of it this way -- you're a cyclist, right? Supposing you were trying to convert a family member into cycling, would you say "Involves moving a pair of pedals in circles quite a bit! Spins wheels! Forward motion while cycling!" No, you say "Do this and you will _be healthier_ , _have more energy_ , _save money on gas_ , and _not kill the planet_." (Feel free to strike the last one if you are not an environmentalist. Gaia knows I'm not. But you get the general idea.)

Figure out what your software does _for your user_ rather than what your software _does_ , and then sell them on buying that tangible benefit to their lives.

<comment><USER4> Excellent point, thanks for the feedback!

</comment></comment><comment><USER5> As a developer and cyclist, I really like your idea and implementation. It's modern and intuitive, something a lot of cycling sites lack, unfortunately.

One feature you can think about that I've often thought would be useful, would be to track road quality via a device that detects vibrations. Poorly maintained roads = big vibrations. I don't know the technical feasibility of this off hand, but hopefully it gives you a new avenue to explore.

<comment><USER4> This is a great idea for an app for a phone with an accelerometer. Thanks for the input!

</comment></comment><comment><USER6> Nice idea!

You should let people drop in their RunKeeper user URL and scan their activities GPX data from there (it's all there).

Quite a few people I know use RunKeeper for cycling now because it's one less device to buy/maintain/ride with (running on the iPhone and all). In addition, the RK guys are likely to help you promote your service -- they're quite active with respect to their community.

<comment><AUTHOR> We had an import feature that allowed users to pull in their tracks from other services, but a migration to new code combined with a cease and desist letter from the big competitor led us to disable importation for the time being.

We are investigating our options and will proceed when we understand the implications of those options. With that being said, this feature was VERY popular, and I loved watching someone bring in 100 routes from their other accounts. People love the concept of data independence and are pretty shocked when they hear one of their favourite sites doesn't allow them to backup their information. So, we are hoping to bring this back in some way that doesn't tick off the other services, or at least does so in a way that doesn't have negative implications on us or our users :P

</comment></comment><comment><USER7> No specific criticisms of the app, but I did notice this on the about us page:

 _We have put together some money and quit our jobs, with the intention of making Ride with GPS the best site of its kind on the internet._

Having 3 full-timers working on a bootstrapped project (especially one with no current revenue model, as far as I can tell) will be very tough. Ramen profitability is that much further away, and you burn through your seed money very quickly.

Since I don't want to start the whole "quit-your-dayjob vs dont-quit" debate, all I will say is this: Consider not waiting until your seed money is completely exhausted before exploring avenues of funding, and by this I mean getting some kind of job. Any sort of income that still allows you to work on your project can help you stay alive until you start making a decent income from your startup.

<comment><USER4> That text is outdated =)

</comment></comment><comment><USER8> I just started biking again for the first time since childhood, and I bought the $4.99 Cyclometer app for the iPhone.

So I came to your site as an excited novice and I'm sort of clueless. Hard to know where to go on the home screen. I assume I want to sign up, but I don't know your feature-benefits well enough to make a decision.

To that end, why are the "Start drawing" and "Sign up" buttons given equal weight in the UI? Do you expect 50% of your visitors to do each? Seems unlikely--and is that even what you want?

The right-left arrows (<|>) to bring up that other blurb is totally unexpected behaviour given a right-arrow click. The content should be coming in from the side, and be progressing me along some kind of 1, 2, 3 step process. Otherwise, why hide the first blurb in order to show me the second?

<comment><AUTHOR> Good points. I just spend a couple hours implementing some a/b testing for an emphasized version of the signup button. Been meaning to do some a/b testing, but got sidelined.

I agree on the left/right slide action comment, will flag it for fixing :)

</comment></comment><comment><USER9> This is just the kind of site that I am looking for. I've had lots of GPX/KML data from my Android phone (with the My Tracks app) and no where to put it. I'm incredibly excited to start using this daily after every ride, to both share what I've done with my buddies and keep track of how far I go.

I really wish I was doing this all day every day whenever I ride... I think I rode 10 miles a day for the past 2 years rain or shine to and from my job and it would have been super rad to see the hundreds of miles ridden all added up!

So, sorry I don't have any immediate feedback yet but I am using your app and hope to have some good feedback soon:)

Here I am - <http://ridewithgps.com/users/4151>

</comment><comment><USER10> It is good if the default map loads based on my location..currently i have no idea where is it pointing to it..and there is no easy way to change that

<comment><AUTHOR> A while ago we pulled support for IP geolocation, due to the service being slow. I have beenlooking into reliable alternatives and have failed, but am totally open to suggestions! Latest idea is to try one of the ip address databases, which claim to be relatively accurate. However, since a user can create an account and save a location to auto-center on, I haven't tried to hard. I'll have to update the wording around the site to steer people towards account creation so that location is saved to a profile.

</comment></comment><comment><USER11> Can you explain how you're getting elevation data? Last time I asked around this wasn't possible (at least according to Yahoo! maps team at FOWA)

<comment><USER4> The USGS will give you their entire data set (mostly 1/3 arcsecond) if you send them a 500 gb external drive. We expose a web service to get at this data set quickly for our planning flash app and some of our background tasks.

<comment><USER11> But nothing on an international level?

<comment><USER4> We are getting international data through their web service currently, but will be integrating an international data set soon.

</comment></comment></comment></comment><comment><USER12> Look into importing My Tracks data: <http://mytracks.appspot.com/>

<comment><USER9> All the data I've imported thus far has been My Tracks data exported as GMX - <http://ridewithgps.com/users/4151>

</comment></comment><comment><USER13> It would be great if you had more info about compatible GPS units. Are there specific brands/models that work? Are there keywords to look for (any XYZ protocol-compatible GPS)? I saw mention of the Garmin 605 and 705 in the FAQ, but are those the only models?

</comment><comment><USER14> Using Chromium nightly (build 35301) I can't update my profile settings. Clicking on "Save changes" gives no feedback and does not appear to save the changes. Other buttons in the app appear to work.

Nice work!

<comment><AUTHOR> Interesting, does this problem happen for you when using other browsers or just chrome? I'd love to help you out (or fix the issue for others!), however can't promise anything if the issue is only on chrome nightly. Send us an email to email@example.com or use the 'Feedback' button at the top of the site.

</comment></comment><comment><USER15> one thing I'd recommend, try using google maps local search api for your search box. It has much more accurate results, and seems to use a completely different database for search results, especially in the UK. Here's a link to it, try using this instead of the search call in the google maps api. <http://www.google.com/uds/solutions/mapsearch/index.html>

It's a bit more work, but the results are much better, UK postcodes will actually work instead of being random.

<comment><AUTHOR> Cool, I'll give it a looksy. Thanks for the tip!

</comment></comment><comment><USER16> Looks pretty cool and well done. I think that it's definitely something where, if you get enough people, it will also start to be really interesting seeing routes other people put in.

</comment><comment><USER17> nice app!

Reminds me somewhat of <http://www.mapmyrun.com/>

<comment><USER18> and <http://runthere.com/>

<comment><AUTHOR> and many of the other alternatives. We just aim to do several things differently: respect our users (not our advertisers, which don't exist), continually innovate and foster community involvement.

<comment><USER17> hope it will work out for you!

</comment></comment></comment></comment><comment><USER19> It seems like you have a great UI and lots of features already covered, but in case you haven't looked at these guys, they're worth knowing about: <http://www.walkjogrun.net/></comment>


Show HN: Can I Get A Receipt With That? See how your tax $ are spent - <AUTHOR> http://canigetareceiptwiththat.com/ <comment><USER2> This is great - a simple and effective way of showing how our money is spent. I didn't know what percentage of my income (nor the total number of dollars) was being given to what, and now I do. I think you accomplished your goal.

<comment><AUTHOR> Thanks for the nice feedback - I really appreciate it.

</comment></comment><comment><AUTHOR> This is my and a friend's entry into the Google-sponsored Data-Viz Challenge (<http://datavizchallenge.org/>). Check it out and tell me what you think

Thanks!</comment>


Big list of http static server one-liners - <AUTHOR> https://gist.github.com/willurd/5720255 <comment><USER2> Just notice the difference between python2 and python3 simple http server

      $ python -m SimpleHTTPServer 8000
      $ python3 -m http.server 8000

Even the python3 is nicer to read, I really like that they package the server in a module. It's almost twice as slower in my machine.

Python2 Benchmark

      Requests per second:    606.03 [#/sec] (mean)
      Time per request:       8.250 [ms] (mean)
      Time per request:       1.650 [ms] (mean, across all concurrent requests)
      Transfer rate:          4412.05 [Kbytes/sec] received

And Python3 Benchmark

      Requests per second:    412.51 [#/sec] (mean)
      Time per request:       12.121 [ms] (mean)
      Time per request:       2.424 [ms] (mean, across all concurrent requests)
      Transfer rate:          3003.17 [Kbytes/sec] received

Both 1000 requests, concurrency level of 5. Listing a large directory with thousands of files.

I usually use python simple http server to share files and provide synchronization. One hopes that with a new python release it will get faster or at least smarter, both are the same implementation with no noticeable difference, just that python3 is slower.

</comment><comment><USER3> Looks like Zawinski's Law needs to be revised: "Every program attempts to expand until it can serve HTML."

<comment><USER4> More like every standard library includes a web server. Which doesn't seem unreasonable these days.

<comment><USER5> depends on your sysadmin, LISO, SSO or network team.</comment></comment></comment>


Information Visualization for Knowledge Discovery - <AUTHOR> http://blog.feralabs.com/2009/03/information-visualization-for-knowledge-discovery/ Ben Schneiderman from the University of Maryland, gave a fascinating talk in Cambridge on 5th March 2009 about the topic of "Information Visualization for Knowledge Discovery." <comment><USER2> Report of a talk by Ben Schneiderman. Great examples of information visualization for getting insight from Data. Treemap, shapesearcher, and more...</comment>


Alternative Christmas Message - 4OD [UK] - <AUTHOR> http://www.channel4.com/programmes/alternative-christmas-message/4od <comment><USER2> Transcript:

Hi, and Merry Christmas. I'm honored to have the chance to speak with you and your family this year.

Recently, we learned that our governments, working in concert, have created a system of worldwide mass surveillance, watching everything we do.

Great Britain's George Orwell warned us of the danger of this kind of information. The types of collection in the book -- microphones and video cameras, TVs that watch us -- are nothing compared to what we have available today. We have sensors in our pockets that track us everywhere we go.

Think about what this means for the privacy of the average person. A child born today will grow up with no conception of privacy at all. They'll never know what it means to have a private moment to themselves -- an unrecorded, unanalyzed thought. And that's a problem, because privacy matters. Privacy is what allows us to determine who we are and who we want to be.

The conversation occurring today will determine the amount of trust we can place both in the technology that surrounds us and the government that regulates it. Together, we can find a better balance. End mass surveillance. And remind the government that if it really wants to know how we feel, asking is always cheaper than spying.

For everyone out there listening, thank you, and Merry Christmas.

</comment><comment><USER2> > A child born today will grow up with no conception of privacy at all. > They'll never know what it means to have a private moment to themselves -- > an unrecorded, unanalyzed thought.

This is hyperbole and also it's just not true in any real sense. We can still have private moments and we can certainly still have "unrecorded, unanalyzed thoughts".

I really wish Snowden wouldn't overstate the current state of affairs, the facts alone are enough.

<comment><USER3> If you consider that most teenagers carry around their smartphone with them all the time, those moments are no longer guaranteed to be private.

<comment><USER4> What Snowden said still isn't true though is it?

My kids have privacy.

<comment><USER5> You believe that you and your kids have privacy.

<comment><USER6> True. You can never say for sure if your kids have privacy or not. Most kids these days don't have any notion of what privacy is, given the kind of content they are sharing on Facebook and other social networking websites.

</comment></comment></comment></comment></comment><comment><USER7> I press play and get "You are using 'incognito mode' in your browser. Please open a normal browser window to view 4oD". Amazing.

<comment><USER8> Whoa. Why are they requiring that? Tell me this isn't the next trend.

<comment><USER9> 4OD want to install some stuff for content management.

Chrome needs he setting: Privacy [Content Settings] Protected Content

        Some content services use machine identifiers to uniquely identify you for the purposes of authorizing access to protected content.

        [x] Allow identifiers for protected content (computer restart may be required)

turned on, if it isn't already.

</comment></comment></comment><comment><USER10> It's not playing for me. Most news sites seem heavily dependant on loads of 3rd party javascript that gets blocked by Ghostery, so I loaded up an incognito window, same thing, just spins forever at the loading screen.

EDIT: Found a mirror on a not-shit site [http://vimeo.com/82666985](http://vimeo.com/82666985)

<comment><USER11> Thank you. I had the same problem.

You can also get the video file using the youtube_dl script:

        youtube-dl 'http://vimeo.com/82666985'

</comment></comment><comment><USER12> Perhaps it's worth updating the title to mention that this was delivered by Edward Snowden.

<comment><USER13> Sorry that is against the rules

</comment></comment><comment><USER14> I know he has very few places to run to, but putting out a message about the dangerous overreach of state surveillance as he sits in Russia takes away some of the gusto for me.

</comment><comment><USER15> It's worth noting that this was broadcast at the same time as the traditional Christmas day speech pre-recorded and delivered by the Queen. Channel 4 has a history of doing things a little differently, and this certainly piqued my interests today.

<comment><USER16> No, the queen's speech was broadcast at 3pm (BBC1, ITV). Channel 4's alternative message was at 4.15pm. They have broadcast alternate messages at the same time before, like when the Simpsons did it - just not this year.

</comment></comment><comment><USER17> Snowden is so smart: He knows how to avoid media oversaturation.

I bet every year now he'll put out nothing more than a couple 2 minute videos, and give one major nytimes interview.

In this way, every word that he says will get a helluva lot of attention... even a decade from now.

</comment><comment><USER18> Yeah I found the video shocking too ...

Saying "Happy Christmas" is just wrong man!

<comment><USER19> Maybe it is where you are, but here in the UK, saying Happy Holidays sounds really awful, and Happy (or Merry) Christmas is standard.</comment></comment>


Inspector – A drop-anywhere C++ REPL - <AUTHOR> https://github.com/inspector-repl/inspector <comment><USER2> Injecting a REPL into a given section of code is my favorite means of debugging and interactively developing. I've gotten into the habit of building up a class interactively within Emacs and re-evaluating files after they have been changed.

It feels a bit like test driven development, but much more concrete. When I don't really know what I want to build, or what is possible, it's a fun way to spike out some functionality.

Then after you see what sort of path could exist from your interactive session you can use the code as a scaffold and re-implement the functionality by writing a test and implementing the code to satisfy it.

<comment><USER3> How well does it work with class redefinition? Do your instances created with the old definition automatically update to become instances of the new definition, or do you need to recreate them?

Common Lisp goes to a lot of trouble to ensure old instances automatically become new instances - given the repl is a first class feature, the whole language is built around this concept. But with C++ I can't quite picture how it would work.

<comment><USER2> The variables and references need to be re-initialized. It would be absolutely amazing if instances updated themselves but I think only Lisps and Smalltalk are capable of what you are saying. Correct me if I am wrong.

</comment></comment></comment><comment><USER4> This comes at a good time as it's something I've been looking for. We have a physics engine and we want to allow users to write custom c++ functions that can be linked in at run-time. We'd like to allow our users to swap in their own custom integration or timestep methods for example.

As an aside, do you know if it is possible to use this to JIT C++ for CUDA kernels?

<comment><AUTHOR> You probably want to use libcling directly instead.

<comment><USER5> From the source code, it looks like that's what this program is using as well.

<comment><USER6> Hence "directly", I read it as.

</comment></comment></comment></comment><comment><USER7> Do people create stuff like this because they don't have access to good interactive debuggers? Visual Studio user here.

<comment><USER8> This is an interesting question, but the phrasing sounds a little funny to me. Don't REPLs predate symbolic debuggers? As someone coming from the Lisp world, I've occasionally wondered about the converse: did people create interactive debuggers because their language didn't have a good REPL?

I've used Visual Studio before, and it's an impressive technical achievement, but for personal productivity, I'll take a good REPL any day -- as long as the language supports it.

That said, good REPLs contain a debugger, and good symbolic debuggers contain a REPL, so they're not entirely exclusive.

</comment></comment><comment><USER9> Hmm -- I wonder why they chose to not use Jupyter as an REPL "backend" (Jupyter calls them "kernels").

<comment><USER10> this exists! recently released: [https://github.com/QuantStack/xeus- cling](https://github.com/QuantStack/xeus-cling)

<comment><USER9> Oh wow! Even a backend for Jupyter's interactive widgets!

<comment><USER11> Check out our blog post on the C++ backend to Jupyter widgets: [https://blog.jupyter.org/interactive-workflows-for-c-with- ju...](https://blog.jupyter.org/interactive-workflows-for-c-with-jupyter- fe9b54227d92)

</comment></comment></comment></comment><comment><USER12> How is this better than just attaching GDB?

<comment><AUTHOR> 1\. You can enter arbitrarily C++ code (gdb can only call some functions, it cannot even instantiate c++ classes) 2\. Variables in Inspector's scope are never optimized away by the compiler 3\. You can use it with default optimizations flags (In gdb you often want `-O0 -g`)

<comment><USER12> 1\. GDB can instantiate classes, but it's a bit wonky, I agree. You have to evaluate malloc manually and then cast the pointer to the right type and call the constructor. You can write python scripts to automate this.

2\. That's good.

3\. You can generally compile with -Og -g for faster execution while preserving enough of the program's structure to make sense of what's happening.

I can see how it can be useful though. Do you have to prebuild every cpp file, or just the one you put "#include INSPECTOR" in? Can you include inspector in more than one file?

<comment><AUTHOR> You only need to prebuild files, where the include statement is included. Ideally this task would be also automatically done in future by sneaking into the build system.

</comment></comment></comment></comment><comment><USER13> Very interesting. Does it require a C++17 compiler? Also is there any way to simply use a different way of interacting with the REPL? For instance over sockets rather than stdio.

<comment><AUTHOR> The REPL is a C++17 just-in-time compiler as a library. The project can use a different compiler. Even C could be supported with some small modifications. It uses TCP sockets + simple json messages for IPC. The REPL prompt runs in a different process independent from the program running the REPL. That means, there could be different frontends added. It also enables multi-process/multi- threading support.</comment></comment>