Web's Anonymity Makes Cyberattack Hard to Trace - <AUTHOR> http://www.nytimes.com/2009/07/17/technology/17cyber.html?_r=1 <comment><USER2> I personally loved this part of the article

 _"the consensus of most computer security specialists is that the attackers could be located anywhere in the world."_

Well... duh.

Although it raises an interesting question, if most of the computer security specialists agree that the location of the attackers is somewhere on the planet, where do the minority think the attacks are coming from?

Tinfoil hat time I think.

<comment><USER3> You're interpreting it wrong. The majority doesn't know where the attacks are coming from, but the minority has a specific hypothesis about where they are coming from (e.g. North Korea).

</comment></comment><comment><USER4> Sounds like a political PR for internet cenzorship.

Remember in the last couple of months US, UK, France, UK, Netherlands and Australia started legal procedures on internet cenzorship and against web anonimity.

As politics started to loose influence on masses due to the mainstream media crisis it would be very actual to start an internet war like this.

</comment><comment><USER5> Europe increasingly requires their ISPs to track "IP detail records" (IPDRs) summarizing network activities into/out-of/within their networks using products such as SenSage (<http://www.sensage.com/solutions/cdr.php>). A number of telcos/ISPs acknowledge using the product (see <http://www.sensage.com/customers/index.php>), and many more likely do.

No matter what you think of censorship, it will be interesting to see how law enforcement starts using gathered data to track down origins of attacks. Given a number of identified attacking zombie computers serviced by a several (regulation-complying) ISPs, network forensic detectives will increasingly manage to trace back activities through the event-funnel/layers to the computer of origin. You need not have recorded all intervening links for all zombies ... I'm sure heuristics will be developed to identify with high confidence the computer(s) of origin.

A natural rebuttal: "no set of DBs can possibly have stored so much data that they have the full network capture to trace back attacks." SenSage & competing products don't store the full network capture, but just enough info to figure out the temporal/topological aspects of an attack. That still requires many terabytes per telco for several years of data, but the products can crunch that data real fast.

Here comes the knock on the door ... you ready?

</comment><comment><USER6> Very misleading I think. The hackers were not anonymous because of Web's anonymity, but because they went through a chain of hacked computers. Hardly a method that is available to the average person.</comment>


Ask HN: 40 iPhone app promo codes for candid feedback - <AUTHOR>

I'm relatively new to iPhone app development and am having trouble with one of my apps.  The app is called PowerPlay+ and is a power hour playlist generator for the iPhone.  I realize the idea isn't that original, but I think my execution is better than similar apps.  The problem is, I'm not getting feedback from users (<1k downloaded between free and paid versions)--which isn't too surprising given the nature of the app store.
Could anyone provide some feedback on the app?  I spent a good amount of time designing the app to be 'attractive' and easy to use.
http://itunes.apple.com/us/app/powerplay-plus/id413083997?mt=8
Thank you!
Codes
99RN6EXHYY3P JKHJA4JW6TJL R73MAT3JY9EX WTK4NMA9M7T7 XLX4A9E79HLH ARH3LTT7K949 4NJ7ETY9KJX9 TWMR9LE766Y3 MYJTA67E73WY RMEWEPM3Y6LF JALAFMHPR639 A79K7XJMFYPW HMXEPRYHPPEK PELN7HPAYW3W P6MENXJMKM9F J3TAFNPXW49W 9KPPRKRPRPXA 6LMRLERALPPF W73J4KFL6HWX NJNNPYMMPHMA MN74NLJN797E 433KE6P7JPPY TWXWMMLEM6LK 7W4HJ9HL7KLF PF67KEELT77K PYFL7MYXA7XL 966XWKT4A397 AY6X6X7WK4E3 A3MMLNT6XXPM 9HRNTTAFPL9Y EN73WXKLL6XH 379KWK3KHE9E AWMR4NJPJELM YK6L7XMNAJT6 JH6PJXYH79P7 WTAHK6K4LP4J TR79PM4KH9RJ XP3MKHRYT7JM 43JRYJWL7PXE YF9R9TWLJ9TW <comment><USER2> I used code: NJNNPYMMPHMA

\- First thing I noticed was that upon launching the app it asked me to rate it. I haven't even done anything with it yet, so asking me to rate it seems out of place. You might want to wait a few launches before you request feedback. Also, it is really annoying that every time I run the app it keeps asking me rate it even though I said "No Thanks". I even agreed to rate it, and it continues to pester me.

\- In the settings menu, I think I would like it if it played the transition sound when I changed it in the menu. It was annoying to have to close out that menu and tinker with the app to hear the sounds.

\- I think I managed to flummox the app a bit. Not sure if it is as designed, or a bug, but it feels like a bug...

Here's what I did:

1) Start the app

2) Add one song to the playlist

3) Click Done

4) Click Play to start the list

5) Skip ahead to the next song

Counter says "0 down/0 to go", but the 60 second timer continues to count down. The play button doesn't do anything, the back button doesn't do anything, but the skip button will reset the 60 second timer and play a sound, despite there being no song present.

</comment><comment><USER3> First impressions: Well, initially, I didn't really know what to do... some sort of explanation would definitely be helpful (maybe brought up by hitting the "power play" button at the top). Also, I would recommend against using the UITabBar for selection or control, as it is generally associated with completely separate views. It should at least be styled differently, so that people don't think of it like, for example, the clock app. This relates to the settings selection... you can select it, and then remove the view with 'x', but settings is still highlighted. Finally, I'd recommend adding background audio support and better resume from close functionality.

Just my 2c, overall its a well executed app.

</comment><comment><AUTHOR> Thank you guys so much for taking the time to give me some feedback! I really appreciate it. I'll make sure to address those issues--didn't realize the app itself was confusing to use, great point.

</comment><comment><USER4> Can you explain how would be a "power hour playlist generator for the iPhone" useful to me (or entertain me)?

<comment><AUTHOR> "Power hour" is a popular drinking game where a group of people gather and put on a power hour playlist. The 'game' is played by setting up a playlist to play one minute of each song for 60 songs (= 1 hour). Once a song changes, everyone takes a sip of their drink/beer.

The app is supposed to make it simple to setup these playlists. Often times, people make power hour playlists ahead of time and it can take a while. With this app, power hour playlists can be made instantly and on-the-go with an iPhone. Does that give you a better idea of the intent?</comment></comment>


What to do with billions of useless humans? - <AUTHOR> http://www.koreaherald.com/view.php?ud=20160428000669 <comment><USER3> I've honestly been curious about this for years. Currently, at least in America, we have a culture where if you are not meeting your financial expectations it is your fault. So if you find yourself in an industry that has been outsourced or replaced by technology, you simply need to retool. Ignoring specifics, this is typically sufficient to satisfy concerns of people that are not directly affected.

So, what happens when retooling isn't even an option? Utopia for some, utopia for all? I just can't seem to make any sense of it, but given the lack of empathy in the current climate for people labeled as other, I would suspect the former.

<comment><USER4> Looking at how the American working class got increasing marginalized to the point where it is a popular trope on television to laugh at 'white trash' or 'hillbilly hicks' has always seemed ugly to me. That contempt has always been suspect. That there isn't even a broad recognition of class warfare as an extant reality in American society just makes it worse and further increases feelings of alienation and marginalization. To be frank it is bullying of the sort that is supposed to occur on playgrounds but on a group scale with adults.

I once asked a question on Reddit indirectly asking about working class men and what the Tiny House community (majority women) thought of them since many working class men would have the skills to help construct houses.

After that I felt pretty sure large swathes of Americans utterly despise each other. It wasn't a good feeling.

<comment><USER5> TV has lots of formerly-white-trash consumers to sell to ( I mean _LOTS_ ) , and nothing excites us* like an appeal to our superiority.

*not really.... right next to it, though. Of course I grew up "middle class" like everyone else, right?

<comment><USER4> Yes that's right.

I think rich people are well aware of the problem (appeals to superiority) in a way that most people aren't. They are constantly in contact with people trying to butter them up. Gradually they develop a sense of detachment, not really believing they're that awesome as they're being told but searching out independent metrics since the human feedback has become unreliable. Or they go batshit into fantasy land.

I may be biased myself but I honestly believe the middle class is responsible for many of the problems we see. They have been constantly buttered up by sycophant politicians and media sources.

I have even seen statements in the media to the effect that the middle class can do no wrong. Recent complaints about the reduction in size of this class often have this thrown in. It is never considered they might NOT actually be God's gift to planet earth but part of the problem.

Is it so surprising that the government has elected officials in western countries that are almost universally doctors and lawyers? Two groups of people who famously have no time for understanding anything other than their field of study because they spent the better part of their lives absorbed in that intense study. Maybe we can't be saved by lawyers and doctors, no matter that they have high IQ, because they are attracted to work on the wrong problems for society.

Consider how different a Congress of computer geeks and scientists would be. I also believe there would be substantial problems since our group also has failure modes like any other. But to say the least it would be different to what we have right now. To be frank I suspect the denizens of HN are more aware of their failure modes than present day Congress. It is not about IQ, it is about the problems you process.

There exist whole swathes of problem areas I see exactly nobody addressing! But by all means let's cross our t's and dot our i's in trade agreements and constitutional amendments.

I vote for the Brain In A Vat things I saw in Psycho-Pass. They had at least some handle on the distributed nature of knowledge and understanding.

<comment><USER5> The American middle class has been quite the overall force for positive change - in the dimensions where it can change things. IMO, the "attack on the middle class" narrative is the _other_ side of the advertising seduction dance - <USER2> people off balance so you can butter 'em up again.

I think the attraction of medicine and law is status. And you can't do anything about that. That's how hu-mans are wired.

<comment><USER4> > The American middle class has been quite the overall force for positive > change - in the dimensions where it can change things.

Oh no! They got to you too! :-)

If you have an opportunity take a gander at "Ben's Mill" It's a short twenty minute documentary on youtube. I think it is very revealing about the kind of self confidence and respect the working class used to have in western countries but which is now lost.

If nothing else it is a beautiful illustration of how things were, it's well worth the watch.

[http://www.youtube.com/watch?v=l2KJbRHO76s](http://www.youtube.com/watch?v=l2KJbRHO76s)

To render all those people unto 'trailer trash' 'white trash', is despicable. You can tell a lot about people by how they treat others they consider their inferiors.

Doctors are responsible for killing their patients by virtue of not being efficient and by preventing technological adaptions. Even properly done digital medical records are impossible in the present world. I'm sure you read recently on HN that medical staff error is responsible for an incredible number of deaths. When I think: heart disease and cancer, it seems incredible that you should add 'Doctor' into the mix of things most likely to kill us, yet it is so.

Lawyers are harming the economic progress of in the main. Don't even think that is very controversial here. Niall Ferguson has outright accused them of this, that's the establishment!

On balance saying doctors and lawyers are a positive force for good is a crapshoot. Maybe so. Maybe there's objective evidence against the intuitive position. Just maybe, they're parasites in a parasitical system that is getting worse with their intransigeance.

</comment></comment></comment></comment></comment></comment><comment><USER6> They aren't "useless" they may simply not have job skills of value in the future economy.

I look forward to a flourishing of terrible poetry, bad music, and sure, lots of TV watching etc.

<comment><USER7> > _They aren 't "useless" they may simply not have job skills of value in the > future economy._

I.e. useless. Sure, not morally, but _economically_. This is what we're talking about when discussing automation of jobs. It's important to be precise, otherwise we risk miscommunication.

<comment><USER8> Do we live to serve the economy, or does the economy work to serve human beings?

<comment><USER7> It should be the latter, but what mostly happens is the former.

Whenever you go to work to earn money for your bread; whenever you adjust your price to equalize supply and demand; whenever you change a product to make it more profitable - you're serving the economy. Single humans and single companies (hell, even most countries) are too weak to dictate how the market works.

<comment><USER8> That's why we have taxes so people who do not receive enough compensation for working have enough to live too.

I think it's important to keep in mind money isn't the end goal when we go to work. We work to live. And if we can live without working, it is a choice we can take.

</comment></comment></comment></comment></comment><comment><USER9> What do the supposedly useful humans usefully accomplish today? If we measure 'useful' by money earned then the most useful are supposed to be hedge fund managers that don't actually solve any problems.

The truth is that what really makes people 'useful' is there interactions with other humans. No matter how sophisticated the technology or what level of super-human super-intelligent or even as he speculates super-empathetic type of AI, human relationships will still be the true measure of value for people.

This concept of 'useless human beings' only makes sense if you have a primitive elitist Malthusian view in the first place. Which unfortunately many people do have that.

We should expect unenhanced human 1.0 to become the next chimpanzee analog in the hierarchy. We should hope he is right that the AIs are more empathetic towards their inferiors than we have been.

<comment><USER7> For better or worse, economic "usefulness" is directly tied to you having food in your belly. We can talk all we want about how "primitive elitist Malthusian" it is, but it doesn't change the fact that economically useless people go hungry, and we're all about to become economically useless. Fully automated economy can run fine without us.

<comment><USER10> Please explain how that works.

So you've got a completely independent "fully automated" economy of robots creating "goods" (as defined by humans) and exchanging them with one another while humans starve to death watching that?

<comment><USER7> Exactly. When robots do the manufacturing and delivery, algorithms do the orders and stock trading, you can imagine the moment when increasing automation will eventually remove humans from the loop.

The market can and does get stuck in weird stupid-ass loops. Consider, for instance, the advertising industry. Most of its output is aimed at cancelling itself out (e.g. when you and I compete for the same group of customers, my advertising expenditure goes mostly to cancel out yours, and vice versa). Also in on-line advertising, a lot of money is being moved over nothing - consider, for instance, analytics. Many people in Internet marketing are clueless about statistics, but so are their clients - so as long as they can bullshit the customers that their service is helping (graph is going up!), they get paid, and customers are happy, because they're clueless too and can't verify the effectiveness of the service they paid for.

And now for a funny twist - people _making_ those analytics tools know this too, and guess what - they're starting to offer statistically invalid measures that are likely to look like the changes marketing people introduced are working. So those marketing people are being bulshitted by their tools now.

It's not a circle per se, but it shows another example of money moving around over no value created.

<comment><USER10> Actually, no, I can't imagine that moment, and that's why I'm asking for someone to come up with a sensical narrative that takes us from here to there.

Humans dictate what the "loop" is. They define what gets manufactured, and who it gets delivered to. How do they get out of the loop?

I believe the whole idea of humans becoming "useless" or "obsolete" is the biggest piece of nonsense I've come across in the last few years.

</comment></comment></comment></comment></comment><comment><USER11> AI is going to displace workers, just like the agricultural revolution and industrial revolution! Oh wait, they figured out other things to do then too...

<comment><USER7> The article addresses your point. Industrial revolution forced people to move from physical work to mental work. AI revolution will take over mental work, and there's _no other kind of work_ left for us.

Also, I find it weird to put agricultural and industrial revolution together in this context.

Agricultural revolution was when people shifted from doing less physical work (hunting) to doing more physical work (building a civilization). It probably wasn't even by choice, but simply because agricultural societies were more efficient and would conquer those which didn't settle down. About the only thing I see here in common with industrial revolution is the forced change of lifestyle.

<comment><USER12> My thought when I read that in the article was "Physical Work -> Cognitive Work -> Creative work". So I think the third kind of work humans will move to is creative(Since AI are so far incapable of being actually creative). After that though, I don't know of another.

<comment><USER13> I think AI are already creative. See this for neural networks creating psychedelic images: [http://gizmodo.com/these-are-the-incredible-day-dreams- of-ar...](http://gizmodo.com/these-are-the-incredible-day-dreams-of- artificial-neura-1712226908).

Perhaps you mean the initial spark of an invention, the idea which strikes like a lightning, and so on? But many inventions happen by accident (like the post-it note invention, [http://www.todayifoundout.com/index.php/2011/11/post- it-note...](http://www.todayifoundout.com/index.php/2011/11/post-it-notes- were-invented-by-accident/)). And then you could argument that only human are crazy enough to find novel usage for useless things.

I don't know.

</comment></comment></comment></comment><comment><USER4> I read Yuval Harari's book.

It's a good book but he chickens out of saying difficult to hear things about us humans. It was extremely obvious to me that he skates figure eights around controversial topics because he made contradictory statements several times. In the first half of the book he'll say one thing, and then in the second half he'll kowtow to prevailing orthodoxies about humans.

During my reading I began to think he might be writing it in a Straussian style i.e. one version of reality for intellectuals, another for the plebs. My copy of his books is sprinkled with exclamation marks.

Given how popular witch hunts are these days perhaps that what it takes to get a message out there.

We are biological animals! We are flawed beings! Let's accept this and move on!

</comment><comment><USER14> Why don't we offer the masses an opportunity to create a new world? Humans are becoming useless at roughly the same time, the cost of space travel is reaching a cost some 10 - 20% of the traditional cost.

The problem is that we value human life and would never subject people to the dangers of space even if they willingly accept the risk. Life is too valuable!

But what happens when a human life is worth nothing? What happens when a human's contribution to humanity is net negative and every person is, by default, a waste of resources? What happens when a person is cheaper than a machine for some tasks?

It wasn't too long ago that people threw themselves to the ocean in the hope of establishing a better life somewhere unknown. People died, but no one found the cost too high to bear. Life wasn't too valuable then.

We have an opportunity here, but we have to devalue life to take advantage of it. And btw, I don't delude myself. I am sure that my children or grandchildren, no matter how successful I may come to be, will be part of that generation, Generation Zero, the one without value. But I hope they have a purpose, at least.

</comment><comment><USER15> >"AI ... based on its immense emotional statistical database, will know that the best thing to do is x and y and z and act accordingly."

of course the AI will be designed not to help the caller get to their ends, it will be designed to maximize shareholder value. ie, say just the right things to get you quit, and to upgrade, ie manipulate you.

<comment><USER16> In the limit you get one shareholder who owns the entire planet, a compliant AI consort, self-replicating production machines that are the AI's hands and eyes - and everyone else is disposable.

And probably dead. Except for some tiny proportion of slaves who are entertaining and/or useful.

The one shareholder will be the most aggressive, narcissistic and psychopathic son of a bitch, because those are the qualities competition selects for.

I thought this election cycle was bad. It's almost reassuring that much worse horrors are possible.

<comment><USER17> In the limit, the sole shareholder may simply be the AI.

</comment></comment></comment><comment><USER18> So humans will be useless, AI will be better. Let's say this happend. Would you want your child's first girlfriend/boyfriend to be a robot ? Would you want some robot to comfort you at the hospice while you die of cancer, or would you prefer a good caring nurse ? Or a side question - would you want a psychopath, who don't care even a little bit, with great acting abilities to comfort you at that hospice ?

Maybe for some things we will prefer caring humans, because they're not just faking it, and this has value. And maybe we could create an economy built around deep human relationships.

</comment><comment><USER19> If AI takeoff is fast? We're fucked.

If AI takeoff is slow? Basic income -> intelligence amplification -> mind merging. With the option to stop at any point and have a comfortable life.

</comment><comment><USER20> This article makes the (obvious) assumption that a humans purpose is to work so AI taking our jobs will be a bad thing. Realistically it most likely will be a really bad thing, but optimistically it will free man up to reach his full potential. If we don't need to sit in a cubicle for 8 hours a day we'll be free to contemplate existence, philosophize etc or play video games and get drunk! AI should bring about utopia.

</comment><comment><USER21> I don't like the assumption that the only faculties humans have are cognitive or physical. There's another: the ability to desire. It's what's driven all of our achievement, and it's not something even a highly sophisticated AI could manage: it has to have a sense of its own mortality or need for perpetuation, which are not things I can imagine us finding a valuable purpose for in giving them.

</comment><comment><USER22> It hijacked my back button in Firefox for Android and took me to their home page. I had to press back two times in rapid succession to come back here.

</comment><comment><USER23> As long as the AIs don't find themselves asking the same question about _all_ of humanity!

</comment><comment><USER24> I was really hoping the article would provide at least some semblance of an answer to the posed question.

Instead it postulated the already known givens.

</comment><comment><USER25> They could board ships to escape from the space goat that is about to eat earth.</comment>


Is this what it's supposed to feel like? One Year Later - <AUTHOR> http://blog.42floors.com/is-this-what-its-supposed-to-feel-like-part-2/

<comment><USER3> Funny, I've always enjoyed the mountain climbing analogy.

You look at the peak from afar and you get excited. As you get closer to the base, you start to get a little scared. Half an hour after you start, your legs are already burning and you're thinking "wtf am I doing here, I could be having a pint on the patio I passed by earlier!" But you keep on eyes on the path and keep grinding, step by step, one foot in front of the other. Your mind goes blank and you get used to the discomfort. Occasionally you stop to rest and then you see "wow, I'm actually making progress, this is pretty neat. And the view! This is way better than some random patio.". You keep going and going for years.

Few climbers make it. Mostly you give up and walk back down (ie fail). Sometimes, someone likes your climb and sends a mule to help you on the way back (acquihire) and sometimes they even send a helicoper (product acquisition). And if you make it (IPO), you find out it's a fractal game - <USER2> always keep climbing, otherwise someone will come from behind and push you over the cliff. :)

<comment><USER4> Don't forget all the false summits along the way. Right when you think success and the summit are within reach, you crest it only to realize, you have a long way to climb to reach the finish line (liquidity!).

Great analogy - thanks for sharing.

<comment><USER5> There is great danger in believing that there is a finish line.

</comment></comment></comment><comment><USER6> Best advice in the post:

"Easy. I stopped reading Techcrunch."

42Floors is a magnitude bigger than my company ([http://periscope.io](http://periscope.io)) and has always seemed like one of those runaway startup successes that we aspire to and occasionally despair of. Reading something this is incredibly motivating. Thank you for writing.

<comment><AUTHOR> Man, I wish it was a runaway success. Absolutely nothing has come easy. But we're getting there.

One of the reasons I wrote that post a year ago was that I had to own up to the fact that we looked better than we actually were.

<comment><USER7> This is why I am always so impressed by your writing Jason. Thanks for 1) taking to the time to write these articles and the more advice-style ones, they have always been useful to me as a founder and 2) being honest and open (to the extent appropriate) in these articles. Being a founder is an exercise in self-doubt above everything else, and its important to talk about.

</comment></comment></comment><comment><USER8> Great article, thanks for sharing.

Couldn't help but notice one thing on your home page: the price is given in $/sqft/year. Did you A/B test with other representations? It doesn't feel intuitive at all to me. I would expect people to have an idea of the total budget they have (/month or /year).

Also I'm sure it's in your backlog already, but a lot of your listing prices are way off ([http://42floors.com/?page=1&order%5Blistings_rate_per_sqft_p...](http://42floors.com/?page=1&order%5Blistings_rate_per_sqft_per_year%5D=desc))

<comment><USER9> The reason why most brokers, landlords, and veteran office searchers prefer $/sqft rather than $total is that $/sqft allows for easier comparison of price efficiency between spaces of different sizes. Using $/sqft is also preferable when the total size is fungible e.g., when a space can be subdivided or combined.

Several of our A/B tests have used total size but it doesn't have a big enough effect on conversion to be measurable.

<comment><USER10> $/sqft is used by most experienced people on the residential side as well.

</comment></comment></comment><comment><USER11> Someone told me that entrepreneurship is the art of blurring what is true today and what will be true shortly. 42floors has done a great job projecting success through your web presence, which certainly has at least a small part to do with why you're actually experiencing success today. I'm looking forward to reading your post about employee 100 next year. Keep up the good work!

</comment><comment><USER12> "Most people walk on paved roads in the valley below"

I know it's not the author who said this, but oh piss off with that self- congratulatory crap. How on earth do you know what _most_ people do? Let's see, according to [http://www.statisticbrain.com/world-poverty- statistics/](http://www.statisticbrain.com/world-poverty-statistics/) 50% of the world lives on less than $1.50/day. If you're lucky enough to be in the U.S. with the means and opportunity to become an entrepreneur, maybe put in a second's thought before you go off and say how uberman you are.

<comment><USER13> I don't think he's referring to 3rd world poverty. He's talking to the hundreds of thousands of people who work at Google or Facebook or PayPal vs the handful of people who try and build the next great thing.

<comment><USER12> Regardless, it's still an incredibly arrogant thing to say. The implication is that there this class of entrepreneurs who are better than the 9-5ers. Honestly, get over yourself. (And I say this as an entrepreneur who can't stand to work a 9-5.)

And next great thing? Seriously? Next great thing is curing cancer not a real estate site.

</comment></comment></comment><comment><USER14> Been influenced by your earlier blog about A/B testing and am using your experiences to guide me to a large extent.

your designs assumes ( correctly I believe) that whoever comes on to 42floors already know what they are looking for. So no need for a video or marketing spiel.

Having thought about it even a new user should get it but have you checked it out, like asked a random to go to your site and track their experience. Or would you term such randoms to most likely not be your customers.</comment>


CockroachDB 19.2 - <AUTHOR> https://www.cockroachlabs.com/blog/cockroachdb-19dot2-release/# <comment><USER2> Some comments I have regarding the documentation. It seems like there is a lot describing how to set up a cluster and how to do various SQL operations that most people probably know how to do.

I think the information that should be presented much more clearly are:

1.) How do we have to partition our data/what restrictions are in place? Basically, what considerations are necessary when designing the data model due to the constraints of the technology?

2.) What functionality that we expect from RDBMS do we give up when working across partitions? Can foreign keys exist across partitions? Can joins work? Inner/Right joins?

<comment><USER3> Head of Docs and Training at Cockroach Labs here. Thanks for this feedback, ralusek. It's spot on, and we're planning to create much more direct and prescriptive guidance and best practices for working with CockroachDB across multiple regions. That's when placement of data (via geo-partitioning or other approaches) is crucial for reducing network latency.

In case you haven't seen them, for now, we have docs on some data placement patterns for multi-region deployments: [https://www.cockroachlabs.com/docs/stable/topology- patterns....](https://www.cockroachlabs.com/docs/stable/topology- patterns.html#multi-region-patterns). We believe the first and third are best for most cases. This tutorial also walks through the impact of using those patterns in a cluster spread across 3 regions of the US: [https://www.cockroachlabs.com/docs/stable/demo-low- latency-m...](https://www.cockroachlabs.com/docs/stable/demo-low-latency- multi-region-deployment.html)

But we'll definitely continue working on better guidance here.

<comment><USER4> Last month I tried to deploy cockroachdb in Kubernetes and I felt the documentation wanted to treat me like a 5-year-old.

I don't think the your product documentation has to explain what Kubernetes is or its terminologies[1].

The worst part is that it does not tell the cluster admins what need to be setup in the Kubernetes cluster at all, instead, it wraps a bunch of `kubectl` commands in a Python script and says "just download the script and run it"[2]. I know a simple `python setup.py` command is easier for novice to just try out. But it could really gives seasoned Kubernetes admins some headache...Our clusters enforce GitOps and nothing can bypass pull requests. Running some random `kubectl` commands is simply impossible. I ended up spending my day reading and translating the script into Kubernetes object definitions by hand and I didn't enjoy it...

1\. [https://www.cockroachlabs.com/docs/stable/orchestrate- cockro...](https://www.cockroachlabs.com/docs/stable/orchestrate-cockroachdb- with-kubernetes-multi-cluster.html#kubernetes-terminology) 2\. [https://github.com/cockroachdb/cockroach/blob/master/cloud/k...](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/setup.py)

<comment><USER5> I'm curious why you would want to deploy CockroachDB in a K8 cluster. Not being critical...genuinely curious. Since it has its own idea of a cluster, it sounds complex to me. Especially since the typical geographically wide CockroachDB cluster would likely span outside a region centric k8s cluster.

<comment><USER6> (Cockroach Labs developer here) In CockroachDB parlance, a "cluster" is just some number of cockroach binaries that have local storage and can connect to one another via TCP/IP. It's entirely feasible to run a multi-node cluster on a single laptop by just starting several instances of cockroach bound to different port numbers.

The Kubernetes concept of a "cluster" is a much broader term, encompassing the compute nodes and a lot of control-plane software to make all of the magic happen.

Fundamentally, running CockroachDB on a Kubernetes cluster abstracts away the process of getting the cockroach binary running and offers a lot of convenience to the human operator vis-a-vis reliability and service discovery.

We like to say that CockroachDB is "Kubernetes native" in that you can easily build a CRDB cluster using only the basic k8s building blocks, without requiring a separate operator program to manage the deployment.

You can `kubectl apply` this config and get a fully-functioning cluster. [https://github.com/cockroachdb/cockroach/blob/master/cloud/k...](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb- statefulset-secure.yaml)

Utilities like Helm et al. are certainly easier than managing a bunch of YAML configs, but they are entirely optional.

Some other CockroachDB+Kubernetes synergies to consider:

1) When using a StatefulSet and PersistentVolumes, a CockroachDB node will easily survive being rescheduled off of its underlying host (e.g. due to maintenance or hardware failure) with no human effort needed.

2) All cockroach instances are, from the perspective of a client, homogenous. That is, a client can send a SQL query to any member of a CockroachDB cluster and get a meaningful response. This maps exactly onto the k8s Service abstraction.

3) Federated k8s clusters and multi-region network fabrics do exist, although they're not exactly common yet. CockroachDB can maintain its clustering across "non-uniform network architectures" that exist within- and cross-region.

</comment></comment></comment></comment></comment><comment><USER7> Is the primary advantage of CockroachDB over FoundationDB primarily in supporting SQL out of the box? The multi-region support seems pretty banger at first glance.

P.S.

... and I had a tirade here about the licensing information being largely absent on their FAQ or product pages, but I did find this after a google before posting:

[https://www.cockroachlabs.com/blog/oss-relicensing- cockroach...](https://www.cockroachlabs.com/blog/oss-relicensing-cockroachdb/)

which... from what I can tell at my cursory inspection, seems okay to me. I'd be pretty pissed too assuming a giant SaaS company came by, forked their product, and then resold it with improvements they didn't upstream. Mostly, because it's just fucking rude. One could have a good business relationship with another company with both prospering, but instead you take the short-n- shitty road for no reason...

With that said, it'd be nice if you put that in yer FAQ.

<comment><USER8> > Is the primary advantage of CockroachDB over FoundationDB primarily in > supporting SQL out of the box.

That + foundation db pulled the carpet out from all their users at the time apple purchased them. Cannot trust that team / product again, it would be incredibly and blasphemously foolish. At least CDB likely won't pull any such stunt.

Disclaimer: I was personally affected by the FDB team making this choice back in 2015 when events unfolded. Still leaves a bitter taste.

<comment><USER9> The situation is completely different these days. Back then they were a small startup. The DB space is incredibly challenging especially because engineers tend to want things for free and are pretty demanding. An acquisition seemed like a likely outcome. To partially offset that risk they offered source code escrow to prospective customers.

Now it's an opensource project backed by Apple. And from what I can tell the team is massively different these days. I'm not sure why the obvious conclusion is "we can't trust them".

I've heard this narrative from people that got burned by fdb's purchase so I'll never use it again multiple times. It's largely confusing to me, as it doesn't feel connected to the current realities.

</comment></comment></comment><comment><USER10> Can we actually backup the database with the community edition yet?

Last I checked this was impossible.

<comment><USER11> Backups are still an enterprise feature but doing a dump is still supported in the non paid version.

<comment><USER10> Yeah, that is not a real solution for a database of any real size, which is the only reason to use CockroachDB in the first place.

<comment><USER11> I know. It's a bummer.

I've not tried to do block level backups for our setup in production.

<comment><USER10> I have - it doesn't work sadly. :(

</comment></comment></comment></comment></comment><comment> ainar-g Excuse me if that has been discussed numerous times, but I distinctly remember CockroachDB 2.0 coming out, and it being a big deal. Where are releases 3.0 to 19.0? Was the versioning scheme changed? How? Why?

Other than that, I _really_ wish I could use CockroachDB at "$COMPANY". Our architects, unfortunately, deem it "too new" and "unproven". Bah.

<comment> a-robinson They switched to calendar-based versioning earlier this year with the 19.1 release, for reasons explained in this blog post: [https://www.cockroachlabs.com/blog/calendar- versioning/](https://www.cockroachlabs.com/blog/calendar-versioning/)

<comment> ainar-g Thanks for the link!

""" We wanted to find a solution that would minimize frustration (and time- consuming meetings) internally, while setting the correct expectations with users around quality and stability. """

Honestly, this doesn't explain much. If anything, hearing "FooDB 19.1" makes me think of stuff like React 16 or Chrome 69. That is, of "hip dudes" who "live fast and bump major versions". On the other hands, hearing "FooDB 2.16" would make me think "Yep, this thing seems stable as Perl or Linux". The meetings point also doesn't explain anything. Go simply does a minor version bump every six months. Why couldn't Cockroach Labs just do that?

Oh well, who cares, really, as long as the product is great.

</comment></comment></comment><comment><USER11> Pro-tip: if at all possible do not run this on top of Ceph. Bare metal SSDs or VMs with SSDs.

<comment><USER12> From experience I take it? Was Ceph running on any of those?

<comment><USER11> Yes it is how we run it in production and sadly it's not great.

On my own laptop I've found it to be maybe 85% or so as quick as normal single mode Postgres. That said things aren't great when the storage layer is doing consensus and replication and then the database is as well.

</comment></comment></comment><comment><USER2> Glad to see Cockroach Cloud, I signed up for the beta.

I'm only really interested in an elastic, managed program with transparent pricing. As a contractor with a fair amount of exposure to various startups around the bay, I find these to be relatively ubiquitous criteria.

<comment><USER13> CockroachCloud PM here -- we hear you on the transparent pricing. Cloud pricing is transparent, see here [https://www.cockroachlabs.com/docs/cockroachcloud/stable/coc...](https://www.cockroachlabs.com/docs/cockroachcloud/stable/cockroachcloud- create-your-cluster.html#step-2-select-the-cloud-provider)

Glad to hear you signed up for the beta. We've been letting people off in batches and hope you can use CockroachCloud for your apps! If you have any feedback on the beta product, do let us know!

</comment></comment><comment><USER14> Is this the new version that has the new license terms that prevent amazon doing an "elastic" on them?

</comment><comment><USER15> Waiting for the comments about the name.

On a more serious note, looking forward to trying this new release out. I found previous versions really straightforward to get up and running with on Kubernetes but performance was always lacklustre.

<comment><AUTHOR> You can find our more recent performance numbers on [https://www.cockroachlabs.com/docs/v19.2/performance.html](https://www.cockroachlabs.com/docs/v19.2/performance.html)

<comment><USER16> Is that saying that you are comparing Cockroach running on 81 c5d.9xls vs Aurora running on 2 r3.8xls? I get that part of what you are trying to show is that Cockroach will scale far past what any single-master system can, but it feels pretty lame to run a test comparing transaction throughput on 2900 cores vs 64 cores and a data set size comparison on 81 hosts vs 2.

The sysbench metrics seems like a much fairer comparison, and CockroachDB looks great in those metrics as well, so I don't really get why you are leading with a comparison that looks really sketchy at first glance.

<comment><USER17> Hi- product manager from CRL here. You are right--we hope to demonstrate that CockroachDB is built to scale horizontally. Deployments of CockroachDB can grow by easily adding more nodes to the cluster which in turn linearly scales throughput. We have posted the most recent published Aurora numbers as a comparison to demonstrate how architecture can influence scale.

We also hear your point about efficiency as tpmC (throughput) alone isn't sufficient to compare systems without taking hardware into account. TPC-C asks users to provide a tpmC per dollar amount. We conducted this price comparison previously in this blog post [https://www.cockroachlabs.com/blog/cockroachdb-2dot1-perform...](https://www.cockroachlabs.com/blog/cockroachdb-2dot1-performance/). These results are even lower in 19.2 because we can achieve greater tpmC with fewer nodes.

This page wasn't met to be competitive-we simply showed Aurora as a reference point. Since we want to focus on CockroachDB we will remove the Aurora comparison.

</comment></comment></comment></comment><comment><USER18> How does CockroachDB compare to YugabyteDB

<comment><USER19> CockroachDB is built from scratch in Go, uses Raft + RocksDB as a distributed key/value store, and then implements the Postgres data structures and wire protocol on top. Focuses on strong consistency and durability (which was the origin of its name). Can run active/active in multiple regions. Lacks full compatibility with Postgres.

Yugabyte is a multi-model database built in C++, uses Raft + RocksDB as a distributed key/value store, and then implements a proprietary document-store model on top called DocsDB. This is exposed as a Redis API, Cassandra API, and now PostgreSQL RDBMS interface by using the actual Postgres code for the SQL layer. Much higher compatibility with Postgres with all data types, most advanced SQL, and even some extensions that work without problems. Not as advanced on the distributed side and uses more of a primary/replica setup.

Both are great options for a distributed scalable RDBMS with Postgres.

<comment><USER20> Thanks for this overview. Is Cockroach an open-source version of Google's Spanner as well?

<comment><AUTHOR> Not technically open-source (BSL-licensed), but yes, source available on [https://github.com/cockroachdb/cockroach/](https://github.com/cockroachdb/cockroach/).

</comment></comment></comment></comment><comment> ainar-g Heh, they didn't seem to test their University website[1] before publising it. There is a gaping:

      <!-- End Google Tag Manager -->
      Additionally, paste this code immediately after the opening <body> tag:
      <!-- Google Tag Manager (noscript) -->

In the source code. That "<body>" isn't escaped either, which makes the whole thing borked-up.

This is why we need XHTML.

[1] [https://university.cockroachlabs.com/catalog](https://university.cockroachlabs.com/catalog)

<comment><USER21> Thank you, we didn't get that resolved beforehand. Should be fixed now.

<comment> ainar-g Wow, that was quick. Thank _you_ :-)

</comment></comment></comment><comment><USER22> Awesome to see a new release! We've followed Cockroach for a while, would be a good fit four our workload.

However, I think it's sad that their pricing is so opaque.

Even after a few emails back and forth with their sales people it still feels fairly arbitrary, and since it's not transparent we don't know if they're just making numbers up based on what they think we're willing to pay.

I find this sales-tactic off-putting. In theory it allow them to "not leave any money on the table", as they would probably frame it, but I'm curious how many deals they will lose simply because people don't want to be held hostage, or don't want to put up with the sales-hassle.

With AWS and Google Cloud (their main competitors), we know that prices won't be jacked-up.

———

An interesting example of this mistake is Mapbox.

They had a limited free tier, where most use-cases would be "talk to sales for pricing". We had a frustrating 2 months talking with their sales people, finally getting a reasonable deal. But we were very close to just ditching them for Google Maps (even though I liked Mapbox product better and was willing to pay more for it), just because the price negotiation and all the cheap sales-tactics was so time-consuming.

Then, 6 months later, Mapbox CEO posted this blog post:

[https://blog.mapbox.com/new- pricing-46b7c26166e7](https://blog.mapbox.com/new-pricing-46b7c26166e7)

        [excerpt, from Mapbox CEO announcing pricing/sales rethink]

        How we price our tools has a significant impact on how those tools get used
        and what gets built with Mapbox. We were pricing some of our APIs wrong —
        making things confusing and restrictive. Rolling out new pricing took
        close to five months; informed by the stories of our builders
        whose curiosity and insight gave us a lot of honest feedback.

        Here's what we were doing wrong:
        - Unpredictable pricing at scale
        - Development slowed by price modeling and negotiating volumes
        - Unintuitive billing units
        - Hard to compare measurement to our friends at Google
        - Confusion around when commercial plans are needed

        Our goal with this change is to reduce the friction to build with
        our tools and to allow our team to help developers use maps and locations
        more creatively. As we designed this new pricing, we kept our key
        pricing principles in mind:

        - Predictable and aligned with metrics our customers already measure
        - Clearly defined discount tiers as businesses scale with no surprises
        - Product usage measured in a way that's clear to all involved
        - Generous free tier to encourage building and make it easier to get started

I experienced all those downsides, which almost pushed us to leave Mapbox, so very happy to see them reconsider (for their own sake).

Perhaps CockroachDB could unlock similar benefits by making their pricing/sales more open.

<comment><USER23> Agree. My current firm has a policy of not working with companies that don't post public prices, which sadly writes off CockroachDB for us. It's a shame because we have a major use case for it.

<comment><USER13> CockroachCloud PM here. CockroachCloud pricing is transparent. You have the option of three different node sizes in GCP and AWS: [https://www.cockroachlabs.com/docs/cockroachcloud/stable/coc...](https://www.cockroachlabs.com/docs/cockroachcloud/stable/cockroachcloud- create-your-cluster.html#step-2-select-the-cloud-provider)

</comment></comment></comment><comment><USER24> Somewhat related but why not have an object store like ceph but expose the guarantees of the data (cap or whatever) and just interact with objects? Bonus points for making the API pluggable.

</comment><comment><USER25> Anyone use the cross data center clustering in the real world? Got a bunch of people considering this - a cluster spanning two data centers as the way to do transactional updates.

</comment><comment><USER26> CockroachDB seems to be getting a lot of attention and marketing cycles lately... What has made it the focus of late?

<comment><USER27> Marketing

<comment><USER28> Probably has to do with their decision to name it after the cockroach. I mean, if they had called it MaggotDB or BotFlyDB or TapewormDB, people might find it unappealing.

<comment><USER29> Those aren't nearly as resilient though. Now, if you made a version of sqlite that replicated itself inside your program then you could use botfly ...

<comment><USER30> The successor will have to be named TardigradeDB.

<comment><USER12> You can't even kill it with Boring Company's flamethrower! Tardigrades are virtually indestructible as going into space and coming back won't always kill them.

</comment></comment></comment></comment></comment></comment><comment><USER31> Love the product name!</comment>


SAP joins PowerPoint and Twitter - <AUTHOR> http://www.readwriteweb.com/enterprise/2009/11/sap-joins-power-point-and-twitter---does-this-work.php?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+readwriteweb+%28ReadWriteWeb%29 <comment><USER2> What doesn't make sense to me here: 1\. Why has nobody done this before? It seems trivial technically. The only good reason I can think of is that the Twitter crowd and the PowerPoint crowd and the Visual Studio crowd do not intersect enough for it to happen. 2\. Why is SAP doing it? That's a pretty certain way to ensure the general public will not use it.</comment>


Show HN: Build Machine Learning Web-Service with Python and Django - <AUTHOR> https://github.com/pplonski/my_ml_service <comment><USER3> Calling the ML model should be done via Django Channels or a similar background queue and not directly within the view (request handler) as if the inputs/inference is too big, it will block and cause issues for load handling.

<comment><USER4> I agree. The ML Algorithms and models can run into huge sizes. I have one with half a gig of serialised model, when running takes up close to 3.5 Gigs. This approach of directly using it in the views would pose scaling issues.

</comment></comment><comment><USER5> For anyone looking to deploy ML models a little bit easier, check out [https://inferrd.com](https://inferrd.com)

It's a simple drag and drop to deploy tensorflow, scikit, spacy, keras or pytorch.

<comment><AUTHOR> Nice!

1\. Is it open-source?

2\. Can I add models with REST API?

3\. How do you handle large models? Are they working in the background or you just use larger severs?

<comment><USER5> 1\. No it's not open source, but I'm not excluding it from the roadmap :) The selling point is really the easiness of it.

2\. Yes! We will be releasing tokens soon so that anyone can interact with the API.

3\. Models can be up to 1GB, each model gets their own server , we only do real-time predictions for now. Meaning the models are constantly running waiting for a request

</comment></comment></comment><comment><USER6> A Django app for managing training data and active labeling would be a nice complement to this. Something similar to AWS GroundTruth.

We should start thinking about an ML life cycle were data is ingested, data labeled labeled, models trained, model tested, model deployed and monitored. Rinse, lather, repeat.

<comment><AUTHOR> Agree, data labeling app in Django will be interesting. It will require a front-end with a lot of features to make it usable.

</comment></comment><comment><USER7> For anyone who is looking to make deploying ML models easier, I created [https://deepserve.ai](https://deepserve.ai)

You can run `deepserve deploy` on the command line and it will stand up a REST endpoint for your model. I'm also building client side SDKs to make it easier to call on these models in your application code.

It's still in beta but feel free to reach out to me jeff @ deepserve.ai

<comment><AUTHOR> The website is down. Is it open-source? Do you know any open-source alternatives?

</comment></comment><comment><USER8> [Gradio](gradio.app) can save a lot of time to serve the same purpose - <USER2> it super easy to serve any python based ML. It provides dozens of interface components, including text, image, audio, files, and dataframes to support many different types of predictions.

</comment><comment><USER4> I noticed that the models use CharField with a max_length of 10K. Why not use TextField?

<comment><USER9> Depending on the underlying DB, these might be handled the same way at the DB level. For example, Postgres uses the same field type for both CharField and TextField.

<comment><AUTHOR> In this tutorial I've used SQLite. I use PostgreSQL for production services. I don't remember now why I used such type.

</comment></comment></comment><comment><USER10> Why use Django when Flask is so much simpler and has less overhead?

For just a REST ML model endpoint, Flask does everything you need, and no more. It's a perfect fit.

<comment><USER11> May be Op wanted to take the advantage of batteries included features like, admin panel, templates etc. Or is just very comfortable with django.

If we are talking speed and simplicity then I would father go with Fast Api rather than flask.

<comment><AUTHOR> You are right! I like Django admin panel, without coding you get UI and I'm comfortable with Django + DRF.

In Flask you need to add many things to make it usable.

If you check the code of the tutorial you will see that it differs from the most of the ML-to-REST-API tutorials. It's not only setting the endpoint. You can have many algorithms with many versions and stages (testing, production). There is A/B testing example. Of course you can do all in Flask but I prefer Django because many things are there already.</comment></comment></comment>


Ask HN: Best to-do list tool with Android widget support? - <AUTHOR> I'm currently using Trello in combination with a thrid party Android widget. However, it is not optimal.
Most important feature for me is ease of access, which is why good widget support is important.
I want to be able to quickly see the top todo's per category and with a quick way to add a task.<p>I was just wondering what you guys are using. <comment><USER2> It feels like widgets are being out-phased in general. If you want a flexible task manager (you can tailor filters to pretty much anything, including what the widget is displaying) try [https://tasks.org/](https://tasks.org/). It is open source and syncs with Google Tasks, meaning you have also free web interface for your tasks. If you are also using Gmail, it might fit your overall workflow nicely.

It is free but has a cheap premium that adds mostly cosmetics. I use it myself for years and I don't really miss anything.

<comment><AUTHOR> Thank you! Been using it today and it's exactly what I was looking for! Fast and responsive widget, superb filtering!</comment></comment>


Pyjion – A JIT for Python based upon CoreCLR - <AUTHOR> https://github.com/Microsoft/Pyjion <comment><USER2> One of the most promissing projects :

      * it keeps CPython as is.
      * hence retains compat with extensions.
      * but still has the potential to improve performance like pypy.
      * and yet can be shipped just as an additional compiled lib inside the Python dist.

If it ever works, it will be wonderful.

<comment><USER3> It would also be nice if it worked outside of Windows.

<comment><USER4> if the speedups pan out, that is the plan! the work on performance has not been started yet however.

<comment><AUTHOR> Awesome!

A while back there was some talk of native excel-python integration. Any chance that is in the pipes, or is MS sticking to R in that sense?

<comment><USER5> You can vote for that feature at [https://excel.uservoice.com/forums/304921-excel-for- windows-...](https://excel.uservoice.com/forums/304921-excel-for-windows- desktop-application/suggestions/10549005-python-as-an-excel-scripting- language)

</comment></comment></comment></comment></comment><comment><USER6> Here's their list of passing and failing CPython tests:

[https://github.com/Microsoft/Pyjion/blob/master/Tests/python...](https://github.com/Microsoft/Pyjion/blob/master/Tests/python_tests.txt)

<comment><USER5> One thing to note about the list of failing tests is the vast majority of them are due to a single bug (or two): [https://github.com/Microsoft/Pyjion/issues/103](https://github.com/Microsoft/Pyjion/issues/103) . And we currently only have 11 open bugs so the test failures make it look worse than it is.

</comment></comment><comment><USER5> Thanks to all of the great questions we got, we have now written an FAQ for the project: [https://github.com/Microsoft/Pyjion#faq](https://github.com/Microsoft/Pyjion#faq)

</comment><comment><USER7> How does Pygion compare with IronPython?

<comment><USER8> From reading the description, and from what little I know about IronPython...

IronPython is a re-implementation of Python in C# which allows it to play nice with .NET. This is similar to Jython which is a re-implementation of Python in Java that plays nice with the JVM.

This project on the other had is not a re-implementation at all, but aims to augment the existing CPython implementation with JIT APIs and is using CoreCLR for the JIT.

<comment><USER5> What ericfrederich said is all accurate. We are also considering playing with ChakraCore as a JIT back-end as well once we are sure that our abstract interpreter code that drives the JIT emission code is compatible with Python semantics.

<comment><USER9> Woah!!! Cool

</comment></comment></comment></comment><comment><USER8> Is there a PEP for this, or is it too early, more of a proof of concept?

<comment><USER5> It's too early. We want to prove to ourselves that we can get the performance we want before proposing the changes we think we want to CPython's C API.</comment></comment>


A new approach to text rendering - <AUTHOR> http://blog.atom.io/2017/06/22/a-new-approach-to-text-rendering.html <comment><USER2> > But if we're going so far as to bypass Chrome's CSS engine, maybe we should > try to bypass the DOM entirely.

Seems like Atom will eventually just be a native app.

<comment><USER3> Yeah, technology loves cycles.

"Hey, let's write an editor for a browser. Oh wait, we need a browser for that editor".

Something good will probably come out of this eventually. I mean if they manage to turn electron into a true native cross platform app framework it's nice.

<comment><USER4> > I mean if they manage to turn electron into a true native cross platform app > framework

…then you'll end up back where you started with the JVM.

<comment><USER3> Right. We might as well just compile our JS to Java.

Instead of using web tech to build native apps I think we should focus on improving browsers so that web apps behave more like native apps and have the same capabilities. _Browsers_ should be the cross-platform VM and webassembly is a good step in that direction.

</comment></comment></comment></comment><comment><USER5> Yet another post about "technology improvements" that basically is some company (Github) with an axe to grind that wants to reinvent the wheel but has nothing new to offer.

To be more clear, no one is writhing in their sleep, or bemoaning to their team mates, this: "We have no good way to edit text in the 21st century". No one. At best, it's a nice idea to see "what if" we could build an emacs-like text editor using modern day web technologies, just to see what browsers and HTML5, etc, can do these days, and hey, maybe we will learn something and push the browser technology a bit, too. At worst, it's bunch of people with no other interesting ideas and who need to bring attention to their company.

What's next for this Electron crowd? Let's see if we can build a whole operating system into a browser? Oh wait. That's also been done with a lot of noise and no real-world impact. Well then, what about a new kind of spreadsheet? Or maybe we could do some cool photo-processing using only a TI-84 calculator!

Ugh. I'm so tired of this, and this massive post by the dev team at Atom proves the insanity of all this.

They have spent thousands of man-hours "improving" their text editing rendering for their slow text editor (but hey, we saved time by building it on top of Chrome!), and yet they are nowhere near text-rendering speeds of 40 years ago.

But never mind that. They will continue to plug ahead and write more posts about it. Meanwhile, emacs (and Vim) plug along as always.

How bad is it (to me)? I'll tell you. Even though I alter between Sublime (written in real code and not on 'borrowed' code) and BBEdit sometimes, along with Vim (still learning), and even MacVim (the horrors!)... THIS is how bad it is. It's so bad... that I would use TextEdit to edit code before I used Atom. I promise you. I actually like TextEdit and often use it anyway. I sometimes wish Apple would just add a few tiny improvements (but I'm daydreaming) and I would use it for 90% of files and projects.

But Atom? Like a lot of modern day projects (React, Bootstrap, WordPress, Facebook, etc), it's a total step backwards to reinvent what we already had. So that basically "new" people can come along and play - with less understanding, IMHO.

<comment><USER6> The fact that the only important factor you can think for a text editor to improve on is text rendering speed shows that you're not really on Atom's target demographic.

You'd think that hundreds of thousands of Atom users would convince you that maybe there's something to the editor that works for others if not for you. But nope, all those guys are happy with switching their main IDE to "a total step backwards", for some reason.

<comment><USER7> >> main IDE to "a total step backwards", for some reason.

Cargo cult is the reason.

<comment><USER6> Yeah but isn't it amazing how cargo cult only applies to the things I don't like?

</comment></comment></comment></comment><comment><USER8> I've maintained restrained excitement about Atom for years. A super hackable editor (what more could a tool-obsessed coder want)... but frustratingly laggy. This release finally feels snappy enough that I may switch.

Large files, slowish startup, and the like are all special case problems that I can use Sublime for, but until now, the almost subliminal typing sluggishness always drove me away.

Bravo Atom team for making it fast enough. I look forward to continued improvements in this space.

<comment><USER9> You may also want to give VSCode a go. I haven't used a Microsoft product (willingly) for decades, but it is a really, really nice and much snappier than Atom in my experience with things like key echos.

Their integration for TypeScript and GoLang are first class as well. Overall though I held the same excitement for Atom as you did (and I used to use Emacs for everything because I wanted an open editor in my tool belt) I've really come to appreciate the approach and polish of VSCode.

<comment><USER10> Also as a result of developing VS Code they've made HUGE improvements to Visual Studio 2017. If anyone had been programming .NET Core with VS 2015 and switched to VS 2017 you know what I'm talking about. I went from waiting 12 seconds for my project to compile to blinking and Chrome is opening up. Of course you have to turn off their browser debugging integration, that thing just sucks up so much dev time if you're not using it at all.

<comment><USER11> My experience with 2017 has been the polar opposite. It runs glacially slow compared to the 2013 installation on the same machine. The startup time makes eclipse look lean and mean, it takes over a minute to start.

<comment><USER12> Have you tried disabling package restore on open?

<comment><USER11> I'm talking about just to start the IDE with no project, opening a project is time on top of that.

I don't see how it would make a difference though, running "nuget restore" only takes a second or two.

</comment></comment></comment></comment></comment></comment><comment><USER13> > One frustrating component of the above breakdown is the time spent > recalculating styles. At this point, it looks like the only way to solve > this within the current paradigm is to use CSS less and reduce the number of > selectors in the application, but that's going to be a hard fought battle > given the huge number of themes in the ecosystem.

This is a perfect example of why we need parallel restyling. :)

<comment><USER14> This is a perfect example of why Atom should have been a native application to start with.

<comment><USER15> But Github is doubling down on Electron with their next github client on Mac/Windows etc. Also how would they write blogs about performance if it is plain old native text editor.

<comment><USER14> Also known as CV driven development.

I use TortoiseGit and IDEs to access Github.

</comment></comment></comment></comment><comment> torrent-of-ions What a waste of time. A text editor where "more responsive typing" is a feature is a joke. These problems were solved decades ago on far more limited hardware.

</comment><comment><USER16> This post mentions how the text storage has already been moved to C++, and the possibility of moving parts of the rendering to C++ as well. Is this part of a trend for Atom to move at least key components to C++ and away from the web stack? Or am I off base with this thought and it's been lower-level to begin with, but these are just optimizations on that?

<comment><USER17> Atom has always had some C++ components, but lately we have moved more core data structures to C++. This PR, which re-implements the text-buffer in C++, also landed in the latest beta.

[https://github.com/atom/atom/pull/14435](https://github.com/atom/atom/pull/14435)

<comment><USER14> With enough such improvements, Atom will become a native app, as it should always have been.

</comment></comment></comment><comment><USER18> Though I use Atom on a daily basis, I maintain the opinion that with software available such as Visual Studio Code, it's painfully obvious that Atom never had performance as a high priority to begin with.

</comment><comment><USER19> Here I was, hoping for an innovative approach to render glyphs in OpenGL or Vulkan, but they're just dealing with accidental complexity of their platform, as Alan Kay would put it.

<comment><USER20> Did you see this? [http://pcwalton.github.io/blog/2017/02/14/pathfinder/](http://pcwalton.github.io/blog/2017/02/14/pathfinder/)

</comment></comment><comment><USER21> Recently I had to edit test data for a grading platform (People upload their programs, the programs get called with the test data and they get feedback if their program works correctly). One test file had 900 000 characters in one line. Almost 1 MB, not a single linebreak. The grading platform is a webpage and you upload things through an online editor, so I pasted the file in Firefox and Firefox crashed.

</comment><comment><USER22> I thought this was going to be a post about fixing the horrible font rendering. Although I guess that's really a Chromium issue. Still can't switch to Atom to this day because of the font rendering.

</comment><comment><USER23> So, we're sort of simulating offscreen drawing by using a virtual dom? I mean don't get me wrong it's sort of nice that they did this as it's the only thing they could have done to deal with the issues I wrote in my 2014 blog post, but that also means that with a lot of effort the web is now able to do what quickdraw could almost 20 years ago...

<comment><USER13> You've always been able to query layout information about DOM nodes that are off screen. The virtual DOM is used for other reasons.

<comment><USER23> With all due respect, no it's not.

This is exactly what people used to do to optimize drawing in intensive application. You only render the visible area. The entire post explains the supporting actions needed to accomplish this for the browser. Thats why at the end of the text they talk about exploring the use of the canvas.

Here's an application from ~2000 [https://www.macintoshrepository.org/2214-vncthing](https://www.macintoshrepository.org/2214-vncthing)

The actual VNC canvas was offscreen and it would only frequently display the entire thing. Which made it for a long time more efficient than modern vnc clients. I think 2010 it was still better than any other VNC client on macosx. It's by far not an uncommon technique, it's just a pain to do when your target is markup, which is what this post is all about.

Granted, it's more complex because a tile here is a chunk of markup rather than a piece of a pixmap, but still.

<comment><USER13> Your confusion (which is understandable) points to the reason why I dislike the word "render": it's extremely overloaded. Rendering as defined by QuickDraw (which you alluded to in your above comment) or the VNC client you linked to is really _painting_ , which browsers only do for areas that are on screen or close to being on screen. Browsers have performed this optimization for decades.

What Atom is referring to by "rendering" here seems to be applying styles to parts of lines. Styling is a higher level concept than vector graphics. Partial styling is something that browsers have wanted to be able to do for a while, but it's quite complex. I've never seen an implementation of partial styling in _any_ UI library, and certainly not in QuickDraw or VNC, which have no CSS-like concept at all.

<comment><USER23> There is no confusion from my side, I'm not sure why you're trying to use that strawman, but given your standing in the Rust community it will probably work anyway.

As I said I wrote about this issue in my blog post. You're trying all sorts of hacks here, because the fundamental issue is that your data structure is the dom and css styling, so everything that other text editors would attach as few byte metadata you used to put in the dom. Which means a simple threeliner that would be a couple of bytes in a proper designed editor, becomes the following beast in atom. Yes it's better now, but the fundamental issue is that you have a weird love relationship to the browser(and no, i'm not judging, whatever floats your boat).

This is even separate from the other problem that is the actual way of dealing with strings in javascript.

            GNU GENERAL PUBLIC LICENSE

            hello

            what's up

          <input class="hidden-input" style="top: 0px; left: 0px;" />

<comment><USER24> Your comments are crossing into personal incivility. We ban accounts that do that, so please edit such bits out of your posts here, regardless of how wrong anyone else may be.

</comment></comment></comment></comment></comment></comment><comment><USER25> I still really wonder about the approach of using web tech for this editor. But I haven't tried it in a while so I will do that shortly. I have been wanting more Github integration in my editor recently, ever since I saw the Xcode demo.

<comment><USER26> I'd think that the only text editor that web browser is competent to have is <textarea>

</comment></comment><comment><USER27> ViM!

</comment><comment><USER28> Seems like such a wasted effort due to technology choice</comment>


Show HN: The vision of a single universal API for all services - <AUTHOR> http://cloudrail.com

<comment><AUTHOR> Currently in Beta and only a few cloud storage providers connected. But what do you think about the idea in general and which services should be connected in future? Thanks for any feedback :)

</comment><comment><USER2> Could it be possible to see real code examples, etc without signing up?

<comment><AUTHOR> I made our quick start manual accessible without signing up. You can find it here: [http://cloudrail.com/developers/quick-start- manual/](http://cloudrail.com/developers/quick-start-manual/) You can find the complete documentation in our developer portal after a free sign up.

<comment><USER2> Cool. Thanks.

<comment><AUTHOR> ;)</comment></comment></comment></comment>


First release candidate for TimescaleDB 1.0 - <AUTHOR> https://blog.timescale.com/announcing-1-0-enterprise-ready-time-series-database-sql-open-source-400b4342dfa0 <comment><USER2> Funny timing... just installed 1.0 rc1 this week. Dual-writing and experimenting reading from TimescaleDB in prod currently.

So far it's been perfect... besides a small config issue[1] causing a crash that should have been detected when Postgres starts up, TimescaleDB gives me reliability and durability of Postgres without having to manually create partition tables.

What will happen when Postgres supports auto creating partition tables for declarative partitioning[2]? Will TimescaleDB still be necessary?

Feature request 1: when all rows of a chunk/partition table are deleted can that chunk be automatically dropped instead of having to manually run drop_chunks[3]?

Feature request 2: for the getting started docs, add an option to install with [https://nixos.org](https://nixos.org)

[1]: [https://github.com/timescale/timescaledb/issues/733](https://github.com/timescale/timescaledb/issues/733)

[2]: [https://www.postgresql.org/docs/11/static/ddl- partitioning.h...](https://www.postgresql.org/docs/11/static/ddl- partitioning.html#DDL-PARTITIONING-DECLARATIVE)

[3]: [https://docs.timescale.com/v1.0/api#drop_chunks](https://docs.timescale.com/v1.0/api#drop_chunks)

<comment><USER3> Thanks for trying out TimescaleDB. Glad you're having a good experience. :)

We think that declarative partitioning is a great step forward for Postgres partitioning. That said the process is still quite manual (including in PG11) -- unless I am mistaken, auto-creating partitioned tables is not in PG11, but perhaps may get released with PG12 at the end of 2019.

But even with auto-creating partitioned tables, TimescaleDB is still necessary for a few reasons:

\- Support for multiple time-based query optimization in the planner

\- More efficient handling of both queries and inserts when there are many partitions,

\- Time-based analytical function like time_bucket,

\- Support for changing partition sizes, etc.

We are also working on many new features, optimizations, and capabilities slated for the next year which will improve time-series workloads in PostgreSQL even more.

</comment></comment><comment><USER4> Is TimescaleDB a good fit for events?

For example, we have data where each record is a tuple of (time, event, object), where the event is things like "viewed", "performedSearch", etc., and the object is event data as JSON. Let's say the object is a movie, in which case the payload might be something like:

       {"id": 123,
       "name": "The Godfather",
       "director_id": 456,
       "genre": "crime"}

Our reporting UI lets users do aggregations based on arbitrary dimensions, so we might do the equivalent of:

        select object->>'genre' as genre,
          object->>'director_id' as director_id,
          extract(month from time) as month,
          count(*) as count
        from events
        group by month, genre, director_id

Then we do things like pivot/nest values to display the groupings.

It's unclear to me whether TimescaleDB fits this use case, or whether we'd have to change how things are indexed. Right now we just index the whole object itself. Another scheme I've considered is that we could index it with the values as keys; so for example, the above event would be {"name:The Godfather": 1, "director_id:456": 1, "genre:crime": 1} and essentially represent the counts. A counting aggregation would then be rewritten as as a sum(). But it's unclear to me how you do intersections here without also creating all the permutations (i.e. something like {"director_id:456/genre:crime": 1}) beforehand.

We're currently using Elasticsearch for this. Performance is okay, but we're not entirely happy with it.

<comment><USER5> I, too, would be very interested to find a Postgres-based replacement for Elasticsearch.

Specifically replacing ELK by Postgres-Kibana.

Requirements for that:

       * Events are (timestamp, {arbitrarily nested JSON object})
       * Filtering by timestamp must be fast
       * Full-text search on the object is required
       * Exact constraints on all object keys must be fast
       * It should be possible to define indices on the object's fields so that WHERE clauses are fast
       * Counting the number of results should be fast, or at least have fast reasonably accurate estimates
       * Support typical Kibana searches and filters

I have tried so far to implement Kibana's access patterns on Postgres, and got quite far, but never got past the problem of [https://wiki.postgresql.org/wiki/Slow_Counting](https://wiki.postgresql.org/wiki/Slow_Counting), which essentially means postgres must scan the whole table if you write a WHERE clause, even when using indices, because it has to double-check whether the returned rows weren't actually deleted.

<comment><USER3> This seems like it could fit well with TimescaleDB but obviously would take testing. My only concern would be with Full-text search on JSON which I think is possible but I have never done. I would start with a timescaleDB hypertable on the Even table (time TimestampTz, object JSONB) with the following indexes (or some of them depending on testing):

\- BTREE(time DESC)

\- BTREE(time DESC, object)

\- GIN(object)

\- some kind of full text index

I don't know why the slow counting problem would be a problem with WHERE clauses since indexes are highly optimized to work with MVCC (e.g. hint bits etc). The wiki article itself says this isn't much of a problem when using indexes. But maybe you can elaborate?

<comment><USER5> I found that the full-text search on JSON worked remarkably well with Postgres 10 -- surprisingly this was the least of all problems.

The issue with slow counting is this from the wiki page:

> PostgreSQL will take advantage of available indexes against the restricted > field(s) to limit how many records must be counted, which can greatly > accelerate such queries. PostgreSQL will still need to read the resulting > rows to verify that they exist; other database systems may only need to > reference the index in this situation.

Typical scenario:

If you make a trivial query that matches a lot of rows (even when using an index). You want to count the number of results in order to tell the user how much they probably have to scroll through (quite important when digging through logs, to know whether you'll have to scroll through doable 3 pages, or impossible 3000 pages).

        SELECT COUNT(*) FROM logs WHERE object->>environment == 'production'

Then the WHERE will match 100 million rows, and postgres will scan them all for existence (due to reason quoted above), no matter if `object->>environment` has an index on it or not.

This will take many minutes, even on SSDs, just for showing a COUNT.

<comment><USER4> Yep, the reason ElasticSearch is fast here is that the underlying Lucene indexes essentially form a column-oriented database. This is superb for low- cardinality fields like "object->>environment"; if it has just a handful of values, then only those values are stored, as a sorted list of postings. Intersection with other field-based constraint are vector operations and can be super fast.

I suspect that to make a fast-counting time series mechanism for Postgres, you'd need to create a new index type that used a columnar approach (or even used Lucene unerneath). I don't know much about what optimization options are available to Postgres extensions, but it doesn't sound like it would be impossible.

</comment></comment></comment></comment></comment><comment><USER6> Just to give a little (estimated) timeline:

* Currently our first release candidate is available via Github and Docker

* We are aiming to release a 2nd release candidate next week

* Sometime shortly after that (1-2 weeks) we'll go final 1.0.0

Happy to answer any questions!

<comment><USER7> Congrats on the upcoming release!

... any news on RDS? :)

I did my part and registered my company's interest but I don't think there's anything else I can do.

<comment><USER6> Nothing to share at the moment, but we appreciate you pinging them!

</comment></comment></comment><comment><USER8> Hi, it's very interesting!

Do you test if TimescaleDB affect the performances of the relational part of Postgres ? In other words, can I use TimescaleDB as my primary DB to store both my users and time series data ?

<comment><USER9> We have not observed any effects on relational table performance. Basically, TimescaleDB code in on-path for each parsed query, but if the query does not specify a hypertable, we just let the query "fall through" to Postgres' standard query planning/execution.

But you do point out a very common use case and strength of TimescaleDB: you can easily JOIN your time-series data (hypertables) with existing relational tables.

</comment></comment><comment><USER10> Nice product for small/mid scale workloads. It's no Vertica or ClickHouse but if you do not need the scale should work well.

<comment><USER3> To put a number on this claim. TimescaleDB at this point can handle up to 100 TB of data.

<comment><USER11> Where does the limitation come from? Or is it just the amount of data that it's been tested on?

<comment><USER9> TimescaleDB clustering is currently limited to a single primary with multiple read-only replicas (which provide both HA failover and scaling concurrent reads).

So the limitation here is the storage capacity available for the nodes, and so the above reflects the amount of data we've seen in use in various RAIDed or network-attached storage environments (like the cloud).

One interesting thing here is that you can "elastically" add a new disk to a existing hypertable, and new writes will be automatically load balanced across the new disk. (In Postgres speak, we support multiple tablespaces in a single hypertable, and we allow you to dynamically add a new tablespace to an existing one: [https://docs.timescale.com/api#attach_tablespace](https://docs.timescale.com/api#attach_tablespace) )

</comment></comment></comment></comment><comment><USER12> Congrats on the RC! TimescaleDB is a very neat idea, there's a lot to gain by building on Postgres.

There's also a very serious limitation due to that: the requirement to predefine schemas. My primary use case for a timeseries-focused db is storing system and application metrics. Using a commercial (e.g. datadog, signalfx) or open source (e.g. influxdb, prometheus) timeseries product I can submit arbitrary data. If I had to perform a schema migration every time a developer wanted to record a new metric, it would be extremely painful.

If this has changed since I last looked at TimescaleDB, please correct me!

<comment><USER9> This is _not_ the case.

TimescaleDB has full support for storing JSON data (inherited from postgres), including with indexes, so you do not need to fully pre-define your schema for these types of applications.

In fact, we added support for TimescaleDB to be a read/write backend for long- term Prometheus metrics. You pull from Prometheus via its remote storage backend, and the data appears automatically in TimescaleDB. But then unlike Influx and native Prometheus, you get to JOIN it against additional metadata for richer questions. For more information:

[https://blog.timescale.com/sql-nosql-data-storage-for- promet...](https://blog.timescale.com/sql-nosql-data-storage-for-prometheus- devops-monitoring-postgresql-timescaledb-time-series-3cde27fd1e07)

[https://github.com/timescale/pg_prometheus](https://github.com/timescale/pg_prometheus)

<comment><USER12> Cool! It looks like the metrics view you build on top of the values and labels table makes it _reasonably_ easy to query. I still worry about how you get good autocomplete in Grafana, though.

Supporting receiving data from Prometheus is nice, but for people who aren't already invested in that it would be helpful if you either

a) picked an agent (e.g. Telegraf or Collectd) and taught it how to submit to TimescaleDB directly

b) picked a protocol already commonly used by agents (e.g. graphite plaintext protocol) and taught TimescaleDB to receive it

</comment></comment></comment><comment><USER13> I feel like I learn of a new DBMS a couple times a month. Can anyone comment on how well these upstart DBMSes tend to be tested / _proven_?

<comment><USER14> For new databases that I'm interested in, I generally check if there's a Jepsen analysis for it ([https://jepsen.io/analyses](https://jepsen.io/analyses)) or whether it's being open-sourced by a company that's been using it in production for a while like Foundation DB ([https://news.ycombinator.com/item?id=16877395](https://news.ycombinator.com/item?id=16877395)). In either of these two cases I would say it's well-tested. Otherwise, it's often hit or miss. Thankfully the number of new db products I've seen is fewer than the halcyon days of the MongoDB wave (2011-2014) and tend to be a bit more focused on specific problems that the db solves.

<comment><USER15> Jepsen only tests distributed operation. For non-distributed databases (like PostgreSQL, which is — for the purposes of this discussion — what TimescaleDB is), Jepsen does not apply.

</comment></comment></comment><comment><USER16> Any updates on full, scale-out clustering?

And are there any plans on supporting columnar storage, for very wide tables?</comment>