Ask HN: Does having no appraisal policy make a company bad for career? - <AUTHOR> 	My current employer is a medium sized, 4 year old startup of around 150 people. I have been with the company for almost an year now and to my surprise I recently discovered that it does not have an appraisal policy yet. This means no 360 degree reviews, no feedbacks and transparency in promotions. Having spent 4 years in a larger company before this, I feel that a lack of a structured performance review and appraisal is detrimental to my career. It would help if I know whether this is the trend across all companies of my employer's size. In case not, I'm sure, I would be better off in a place where I would have better and a more transparent career growth. <comment><USER2> Even if there's no formal annual performance review, you should still be getting some kind of feedback from your manager as to how well you're doing your job. If not, you can ask your manager how you're doing and whether they have any suggestions for how you can improve.

Does your company have an HR department? If it does, I'd be surprised at the lack of a formal review process.</comment>


Playing Atari with Six Neurons - <AUTHOR> https://arxiv.org/abs/1806.01363 <comment><USER2> As someone working on a reinforcement learning/neuroevolution problem right now, I find this to be extremely exciting. Fewer parameters, _ceteris paribus_ , is always better—the fact that the experiments in this paper were run on one workstation, rather than on a massive farm of TPUs à la AlphaGo, implies quicker development iteration time and more accessibility to the average researcher.

The staging of components in this paper (compressor/controller), where neuroevolution is only applied to a low-dimensional controller, reminds me of Ha and Schmidhuber's recent paper on world models (which is briefly cited) [1]. They employ a variational autoencoder with ~4.4M parameters, an RNN with ~1.7M parameters, and a final controller with just 1,088 parameters! Though it's recently been shown that neuroevolution can scale to millions of parameters [2], the technique of applying evolution to as few parameters as possible and supplementing with either autoencoders or vector quantization seems to be gaining traction. I hope to apply some of the ideas in this paper to multiple co-evolving agents...

[1]. [https://worldmodels.github.io](https://worldmodels.github.io)

[2]. [https://arxiv.org/abs/1712.06567](https://arxiv.org/abs/1712.06567)

<comment><USER3> You may be interested in an even older paper: [http://www.idsia.ch/~juergen/icdl2011cuccu.pdf](http://www.idsia.ch/~juergen/icdl2011cuccu.pdf)

<comment><USER2> Thanks so much! I read this (and a few related papers) today. Besides the novel algorithm discussed in the new Atari paper, do you have a reference implementation of online vector quantization you might be able to recommend? I think I could probably figure it out from the paper alone, but sometimes it's nice to see code other people have already optimized. :)

<comment><USER3> Uhm unfortunately I do not, I could search for some on Google but I doubt I would fare better than you at it. I went to code my own version, it is quite straightforward. You can find it here: [https://github.com/giuse/machine_learning_workbench/blob/mas...](https://github.com/giuse/machine_learning_workbench/blob/master/lib/machine_learning_workbench/compressor/vector_quantization.rb) although polluted by research's trial and error, you can easily check the minimal code necessary to run. Here's an example of how to use it: [https://github.com/giuse/machine_learning_workbench/blob/mas...](https://github.com/giuse/machine_learning_workbench/blob/master/examples/image_compression.rb) Let me know if that works for you or if you have further questions!

<comment><USER2> That's excellent! Thanks!

</comment></comment></comment></comment></comment><comment><USER4> Cool article, lots to digest, one thing caught my eye:

"To the best of our knowledge, the only prior work using unsupervised learning as a pre-processor for neuroevolution is (cite)."

Just amazing how much low-hanging fruit there still is in the space.

<comment><USER3> Author here. The idea is low-hanging indeed, several friends (including @togelius!) commented "I always wanted to do that -- eventually". Realization is another matter. Have a look at the mess necessary to make it work: we had to discard UL initialization for online learning, accept that the encoding would grow in size, adapt the network sensibly to these changes, and tweak the ES to account for the extra weights.

</comment></comment><comment><USER5> I have been wolfing down RL articles, videos and publications after a intro to deep learning via Manning's Deep Learning for some time now and while the overall concept of RL is easy to grasp (agents, actions and state etc) some of the finer details and processes are quite confusing.

I am tempted to blame inconsistency across terminology and implementations for this lack of understanding but I suspect it has more to do with approaching this field through the lens of a developer and not a researcher or academic. Trying to understand the code without grasping the "science" of the mechanisms completely.

Either way if you feel to be in a similar spot check out this resource: [https://reinforce.io](https://reinforce.io) and their respective Github repo: [https://github.com/reinforceio/tensorforce](https://github.com/reinforceio/tensorforce).

Just reading through their code, and documentation has made a lot of the concepts clearer.

And a few more resources I found really helpful: [http://karpathy.github.io/2016/05/31/rl/](http://karpathy.github.io/2016/05/31/rl/) [https://www.analyticsvidhya.com/blog/2017/01/introduction- to...](https://www.analyticsvidhya.com/blog/2017/01/introduction-to- reinforcement-learning-implementation/) [https://www.oreilly.com/ideas/reinforcement-learning-with- te...](https://www.oreilly.com/ideas/reinforcement-learning-with-tensorflow)

Edit: My point that I forgot to mention was that I always feel like I am playing catch-up to understand what is going on half the time as the amount of new content being released exceeds what I can absorb.

</comment><comment><USER4> And the Github library:

[https://github.com/giuse/DNE/tree/nips2018](https://github.com/giuse/DNE/tree/nips2018)

<comment><USER3> Ruby should be perfectly legible with a Python background, but for any question just ping me on twitter. I would be happy to build a dialog :)

<comment><USER6> Just curious, why did you pick ruby over python? Personal familiarity?

<comment><USER7> It's particularly interesting that they've chosen to wrap Python using Pycall. I'd love to hear about the tradeoffs of that.

<comment><USER3> Sure! It's quite simple: works like a charm. Completely transparent. You `import` with Python-like syntax, and you get a Ruby object that transparently forwards any message (i.e. method calls) to the corresponding Python object on the underlying Python interpreter.

This means that Ruby does not need to know _anything_ about the Python object: whatever you call on the Ruby object is just forwarded to the Python one, and whatever result is passed back to Ruby.

About the overhead, I sincerely do not know; I expected to have some so my code does part of the image pre-processing directly in Python (`narray`) in order to pass a smaller object to Ruby, but besides that I could perceive none -- grain of salt advised, as that was possibly hidden from me because my computation in Ruby was orders of magnitude more complex/time consuming than what was going on on the Python interpreter.

Definitely ping Murata-san either on GitHub [https://github.com/mrkn/pycall.rb/](https://github.com/mrkn/pycall.rb/) or Twitter `@mrkn`, I will send him a link to this thread so he can contribute if he feels like it. Personally, I am a fan of his work and elegant approach, I owe him for enabling me to keep working in Ruby while everybody publishes code in Python :)

<comment><USER7> Sounds great - I also prefer Ruby very strongly, and have tended to avoid Python code because I didn't expect to have an easy way of wrapping it, but will definitively have to play with Pycall.

</comment></comment></comment></comment></comment></comment><comment><USER8> ... and that's three more than the average Atari marketing exec had back then. No wonder they had trouble understanding the game industry :-)

<comment><USER9> OK, you're so grayed out but your bio says you've been programming since '79 and you've written games for Atari. So perhaps all we need is some elaboration? They seem like a successful company, don't they?

<comment><USER10> Story time maybe? Atari were successful for a while but they crashed pretty hard.

<comment><USER11> A story from the GP: [http://www.dadhacker.com/blog/?p=987](http://www.dadhacker.com/blog/?p=987)

<comment><USER10> Ooh, thanks - I read a bit of the blog but didn't find this one!

</comment></comment></comment></comment></comment><comment><USER12> I can post on hacker news with only 4.

<comment><USER13> Yeah, that's pretty apparent. :)</comment></comment>


Ask HN: What are some of the best podcasts around Product Management? - <AUTHOR> <comment><USER2> I particularly like:

\- [https://www.thisisproductmanagement.com/](https://www.thisisproductmanagement.com/)

\- [https://www.100productmanagers.com/interviews](https://www.100productmanagers.com/interviews)

I'd also consider [https://www.indiehackers.com/podcast](https://www.indiehackers.com/podcast) somewhat related to product management. In general, it depends on what you are looking for. If you want to go deeper into any of the areas that PMs deal with, I would look for podcasts in that specific area (e.g. UX, design, marketing, development, ...) rather than a PM podcast.</comment>


Archeological dig reshaping human history - <AUTHOR> http://www.newsweek.com/id/233844/page/1

<comment><USER2> The title of this post is a bit of an exaggeration, to say the least. There is a great discussion over at Reddit about this same article, I strongly recommend reading the highest rated comment.

[http://www.reddit.com/r/science/comments/b53f5/a_temple_comp...](http://www.reddit.com/r/science/comments/b53f5/a_temple_complex_in_turkey_that_predates_even_the/)

<comment><USER3> That is indeed a great commentary. Now I wish HN was _more_ like Reddit!

<comment><USER4> Thanks for doing your part

<http://news.ycombinator.com/user?id=yread>

</comment></comment></comment><comment><USER5> Interesting.

Synopsis is that there's a temple complex in Turkey that dates back 11,500 years, way before anything like it (6,000 years before stone henge, for example).

The earlier theories were that farming and animal domestication led to stuff like religion, since it made food more abundant and stable. This guy is saying, based on this finding, that in fact religion prompted them to build the temple complex, and the task of maintaining it prompted people to get clever about staying in one spot and supplying a population with food.

I'm not qualified to evaluate the merit of the claim, but hot damn those are old temples!

<comment><USER6> The "predates agriculture" bit is a little spun. Dates for the dawn of agriculture in the fertile crescent typically run about 12kybp. So this thing (if the dating holds -- often times new sites jump around a bit) would be contemporary. So the causality could still go either way, or you could posit a feedback situation I guess. Still a really important find.

</comment></comment><comment><USER7> This is a good opportunity for me to ask a question that has been bugging me for ages (eh, bad pun. :-)

How do sites like this get "buried"? I can understand how small objects might get lost and later buried in the soil, or perhaps get intentionally buried in a gravesite. I can also see how dinosaur remains might have been buried by geological processes over millions of years. But how do huge monuments or even cities get lost this way?

On the geologic scale, these things are very very recent, and I don't see how the Earth could have swallowed them up in such a short amount of time. There was another story in the news recently about an ancient road being re- discovered in Jerusalem. I never understood how such things get buried, and would appreciate it if someone could explain, thanks!

<comment><USER7> OK, the end of the article mentions "the entire site was buried, deliberately and all at once". But no hint as to why.

<comment><USER8> Conquered by some other tribe?</comment></comment></comment>


Show HN: Application operations platform for serverless - <AUTHOR> https://iopipe.com <comment><AUTHOR> A little over a year and a half ago, we started diving deep into the serverless space. We spoke to many users of AWS Lambda and found that the biggest issue with building complex serverless application was a lack of visibility and instrumentation.

It's difficult to know when issues start bubbling up, and relying on so many services as part of your application can be scary. We've built a set of tools to help dig deep into functions, and view outwards by tracing triggers and external calls performance and metadata surrounding it.

I'd love to hear more from Lambda users about their experiences building and operating serverless apps.</comment>


2016 GraphQL Conference takeaways - <AUTHOR> http://joelisawesome.com/2016-graphqlsummit-key-points/ <comment><USER2> Regarding predefined queries, is the idea to be like a View in SQL [1]? Notwithstanding the abstraction benefit that brings, doesn't that negate one of the claimed advantages of GraphQL vs. something like a REST API: that the client is in full control of assembling the query, and can request specific fields, receiving only as much or little information as it needs? Contrast that with a resource-based API, where the server has decided the schema and form that the responses will take ahead of time.

In either case, I can see the benefits of pre-defined queries. I have raised this [2] (along with others [3][4]) as a possible trade-off to improve cacheability; prior to us the prospect of 'prepared statements' (in SQL parlance) was raised as a performance and security benefit [5][6][7].

[1] [https://en.wikipedia.org/wiki/View_(SQL)](https://en.wikipedia.org/wiki/View_\(SQL\)) [2] [https://news.ycombinator.com/item?id=12500412](https://news.ycombinator.com/item?id=12500412) [3] [https://news.ycombinator.com/item?id=12502226](https://news.ycombinator.com/item?id=12502226) [4] [https://news.ycombinator.com/item?id=12504587](https://news.ycombinator.com/item?id=12504587) [5] [https://forums.meteor.com/t/kadiras-learn-graphql- project/11...](https://forums.meteor.com/t/kadiras-learn-graphql- project/11592/6) [6] [https://medium.com/the-graphqlhub/graphql-tour- variables-58c...](https://medium.com/the-graphqlhub/graphql-tour- variables-58c6abd10f56) [7] [http://www.rubydoc.info/github/rmosolgo/graphql- ruby/file/gu...](http://www.rubydoc.info/github/rmosolgo/graphql- ruby/file/guides/security.md#Only_execute_predefined_queries)

<comment><AUTHOR> The idea was rather intriguing, however I haven't ran into perf issues in clients yet where this would be beneficial.It does sort of negate one of the benefits of the spec, which is human-readable static queries.

It did very much feel like prepared statements. There was even talk of some implementations of predefined queries as a mechanism to whitelist queries, which was interesting.

I hope we'll see more about this as projects grow and eventually need it.</comment></comment>


Complete, the World's first community-based todo app - <AUTHOR> http://www.completeapp.com/

<comment><USER2> Thanks for posting about us atto, My name is Effi and I'm product / co-founder @ Complete. We founded Complete with the idea that a social network can provide so much more than entertainment value. With the right platform, people can make daily life easier for each other through their support, advice and connections. So Complete is where you can declare you tasks and goals, publicize them and get great value back. I'm here to answer any question.</comment>


Combating multidrug-resistant bacteria with nanoengineered peptide polymers - <AUTHOR> http://www.nature.com/articles/nmicrobiol2016162 <comment><USER2> The good thing is it works on gram negatives which we really need new antibiotics for more than gram positives. I would be concerned how non-toxic it in humans as peptide antibiotics have historically been pretty nasty with lots of bad side effects.

Here is the link to the abstract [1].

1\. [http://www.nature.com/articles/nmicrobiol2016162](http://www.nature.com/articles/nmicrobiol2016162)

</comment><comment><AUTHOR> Full title: Combating multidrug-resistant Gram-negative bacteria with structurally nanoengineered antimicrobial peptide polymers

Media stories [http://www.sciencealert.com/the-science-world-s-freaking- out...](http://www.sciencealert.com/the-science-world-s-freaking-out-over- this-25-year-old-s-solution-to-antibiotic-resistance) [http://www.telegraph.co.uk/women/health/does-this-25-year- ol...](http://www.telegraph.co.uk/women/health/does-this-25-year-old-hold-the- key-to-winning-the-war-against-th/)

</comment><comment><USER3> This is definitely cool, but I don't see why bacteria wouldn't develop a resistance to this as well. I have a hard time getting excited about such things while the basic problem of reckless antibiotic use still exists.

<comment><USER4> They will develop resistance, but that is not a reason to not do it. I fully agree with you that reckless antibiotic use is the core problem, but public health solutions are slow and not sexy. It reminds me that Everyone wants a pill to treat for obesity--hundreds of millions of dollars have been spent chasing that dragon. in reality, lifestyle choices would be more effective, but habits like Big-Gulps and Couch-sitting die hard.

<comment><USER3> >They will develop resistance, but that is not a reason to not do it.

Clearly. I take aspirin when I have a cold, so I have no qualms bout treating symptoms.

I just have trouble seeing this as a meaningful step in solving the problem at hand.

</comment></comment></comment><comment><USER5> I initially read "Combining ..." and got a little bit scared

</comment><comment><USER6> I hate to do this since someone on another thread recently admonished HN commenters as being like crabs that pull other crabs back down into the pot, but I _think_ that this paper is largely PR fluff and worthy of criticism.

That being said, I am only going off of the abstract (If anyone has access to the paper, I'd love to read it) and I'd be happy to be proven wrong about any and all of what I say below.

ABSTRACT [w/ comments & opinions] _Here, we show that a class of antimicrobial agents, termed 'structurally nanoengineered antimicrobial peptide polymers' (SNAPPs) exhibit sub-μM activity_ [if they had good MIC90 values ([http://medical-dictionary.thefreedictionary.com/MIC90](http://medical- dictionary.thefreedictionary.com/MIC90)), they would have boasted about them. The fact that they don't discuss MIC90 values implies that they really haven't tested their compound against a large representative sample of Gram- pathogens. Either that, or they have done the work and the results aren't compelling.]

 _against all Gram-negative bacteria tested_ [My suspicion is that "all" is a small number (see above). When I did this type of work, we routinely tested our compounds against about 200 random hospital isolates for a _single species._ ]

 _including ESKAPE and colistin-resistant and MDR (CMDR) pathogens, while demonstrating low toxicity._ ["low toxicity" is a weasel phrase]

 _SNAPPs are highly effective in combating CMDR Acinetobacter baumannii infections in vivo, the first example of a synthetic antimicrobial polymer with CMDR Gram-negative pathogen efficacy._ [This is noteworthy only from the standpoint that no one had previously been able to make _synthetic polymers_ with this activity.]

 _Furthermore, we did not observe any resistance acquisition by A. baumannii (including the CMDR strain) to SNAPPs._ [This is a negative result with no measured boundary for resistance frequency. They don't say anything like: "We estimate that resistance to our SNAPPs is < 1e-11 per..." My suspicion is that they ran a quick experiment or two and didn't see any resistance, but they did not undertake a thorough analysis.]

 _Comprehensive analyses using a range of microscopy and (bio)assay techniques revealed that the antimicrobial activity of SNAPPs proceeds via a multimodal mechanism of bacterial cell death by outer membrane destabilization, unregulated ion movement across the cytoplasmic membrane and induction of the apoptotic-like death pathway,_ [The authorts want to argue that it is more difficult for microbes to acquire resistance to SNAPPs because the SNAPPs have multiple purported mechanism(s) of action (OM destabilization, ion gradient destabilization, etc.). If you kill the microbe 3 different ways, then it will be difficult for the microbe to evolve resistance against you.

There are several problems with this conclusion. First, their assays do not pinpoint the MOA of their compound. It is possible that their compound "hits" a _single_ target that has multiple effects on membrane stability in Gram- negatives. If this is correct, then resistance might be trivial.

Alternatively, if they are correct, and the SNAPPs have multiple, independent destabilizing effects on the microbial membranes, I would be concerned that the SNAPPs would be indiscriminate enough to have an increased likelihood of being toxic to all membranes (human included). This is speculation on my part, however.

Finally, there is no mention of the efficacy of the SNAPPs against Gram- that contain multi-drug resistant efflux transporters ([https://en.wikipedia.org/wiki/Efflux_(microbiology)](https://en.wikipedia.org/wiki/Efflux_\(microbiology\))). I would be interested to know how effective SNAPPs are against these pathogens.]

 _possibly accounting for why we did not observe resistance to SNAPPs in CMDR bacteria._ [Again, I think they're speculating wildly based on a few negative results.]

To me, this is a standard story that you occasionally see from academic chemistry labs. They have an interesting compound and/or technology. They see some antimicrobial activity with their compound and they collaborate with a microbiology lab to demonstrate some level of efficacy.

Here, the microbiology lab is one that has been historically devoted to _oral_ microbiology ([http://findanexpert.unimelb.edu.au/display/person12757#tab-p...](http://findanexpert.unimelb.edu.au/display/person12757#tab- publications)). My point is that neither the chemistry lab nor the oral micro lab are really equipped to know the ins and outs of antibiotic drug development. They are simply making a splash with the terms multi-drug- resistance and nanoengineered.

</comment><comment><USER7> This will have a lot of unintended and nasty consequences. I'm going to bet on evolution in the long run.

<comment><USER4> I agree resistance will develop, but i can't imagine unintended and nasty consequences. Anti-microbial peptides already exist in nature.

</comment></comment><comment><USER8> A peptide is already a polymer. This is basic definitional stuff:

>"peptides fall under the broad chemical classes of biological oligomers and polymers" [https://en.wikipedia.org/wiki/Peptide](https://en.wikipedia.org/wiki/Peptide)

Really the people doing the research have responsibility to speak out against this kind of jargon loading hype... Oh I see it is the title they chose for their own paper. I didn't read the paper, but the presence of hyping activities meant to appeal to the lowest common denominator has proven itself a decent heuristic so I won't bother.

Also, pretty much every bacteria is "multi-drug resistant".

<comment><USER9> These are peptides bound to a polymer particle that is not made out of peptides, but out of another polymer (PAMAM).

The vast majority of easily cultured bacteria have demonstrated the ability to acquire multi-drug resistance over time, but that doesn't mean that most bacteria that currently exist are multi-drug resistant.

<comment><USER8> I don't see how this is a correction of what I wrote. It is completely usual for a peptide to not only be a polymer itself, but also be attached to other polymers: [https://en.wikipedia.org/wiki/Glycosylation](https://en.wikipedia.org/wiki/Glycosylation)

That term is totally meaningless in the title. Similarly, the term "multi-drug resistance" has always been about hyping up your paper. It was used until 2010 or so without any definition at all, look at all those papers:

[https://www.ncbi.nlm.nih.gov/pubmed/?term=multi+drug+resista...](https://www.ncbi.nlm.nih.gov/pubmed/?term=multi+drug+resistant)</comment></comment></comment>


Show HN: bookspp.com - <AUTHOR> http://www.bookspp.com

<comment><AUTHOR> I was looking for a job because I am graduating this summer, and after I had some interviews, I realized that I need to learn some programming like c/c++, data structures and algorithms, socket programming and multi-threading programming. I like hackerbooks.com but found that the database is quite limited there. So I spent around two weeks to set up the website.

The idea is to use the amazon tagging to connect books. it seems to be a very light-weight app, :D</comment>


DIY.org - Learn all kinds of different skills - <AUTHOR> https://diy.org/skills <comment><USER2> thanks now I have 17 tabs open...</comment>


Ex-Goldman Programmer Described Code Downloads to FBI (Update1) - <AUTHOR> http://www.bloomberg.com/apps/news?pid=20601127&sid=aDH30o9l0jn0 <comment><USER2> > "The reason I uploaded to svn.xp-dev.com was because it was not blocked by > Goldman Sachs security policy," Aleynikov wrote. The phrase, "not blocked by > Goldman Sachs security policy," was crossed out and he added: "I wanted to > inspect the work later in a more usable environment."

I'm not necessarily defending Aleynikov, but to me this just sounds like a case of paranoid security measures causing exactly what they were intended to prevent.

<comment><USER3> Wait, are you saying you don't normally erase the bash history after downloading some "open source" software off your employer's server?

</comment></comment><comment><USER4> Is the original document avaialable somewhere? I'd rather read that than Bloomberg explaining what tarballs and bash history is.

I can only imagine the thoughts going through the average person's head when they read "subversion hosting".

</comment><comment><USER5> Seems like this dude messed with the wrong people. If there's one group that has the clout and the connections to make your life a living hell, it's the bankers. They're in direct contact with your credit rating, they go deep in government, and they have tons and tons of money. It's a bad idea to piss them off.

<comment><USER6> I don't see how his life is ruined. Charges were filed against him, he was questioned by investigators, and he is free on bail. He is getting a fair criminal trial, one that the finance industry's money can't really affect.

This is what happens to everyone that is accused of committing a crime more serious than jaywalking.

(As for his credit rating, what does it matter? If you're making 1.2MM a year, you can probably live without borrowing money.)

</comment></comment><comment><USER7> He's 39 so if he didn't do anything lets hope things do work out that way.

</comment><comment><USER8> So...its just a crazy coincidence that the guy from London, the xp-dev.com guy, is/was a trading systems developer as well? Right...that guy knew about it being on his servers, hell, he was part of the scam! These guys were up to something. I don't know if the article was incorrect or what, but it says the Russian left a $400,000/yr job for one that paid $100,000 less? The government doesn't see anything fishy about that? AND, on the other link posted on HN the other day about the Goldman code torrent honeypot, the Citadel group connected to the torrent...and they are involved with the company that the alleged theif was leaving Goldman for! I mean come on, these avenues need to be explored.

<comment><USER9> Reports were the new job paid 3 times as much (about $1,200,000 annually). Which is consistent with the idea of going from GS to a hedge fund. This article says he mentioned that to co-workers. Now, the financial affidavit quotes $300,000 but this may be just the base pay (before bonuses).

<comment><USER8> Possibly, but a million dollar bonus, when his base isn't even at a million? Isn't that a little out of ordinary?

<comment><USER10> not really. typically people at these firms make most of their money from bonuses rather than salaries.

Driss Ben-Brahim received a $100M bonus in 2007 from GS as a prop trader. I would guarantee his base salary is much smaller.</comment></comment></comment></comment>


Ask HN: How to find an idea for a product / genuine need? - <AUTHOR>

Dear HN,
I have quit my job recently with an idea of spending some time on need finding and further customer and product development.
My initial plan is to catch up with as many various small to medium business owners and try to figure out what sort of problems they may be having. Admittedly, this is just one way of finding a need / idea for a product.
How would you approach this task, or may be have already approached it in the past?
Thank you! <comment><USER2> I left a corporate job a couple of years ago to pursue technology development in areas that I think will be making a huge impact across all industries in the near future, notably computer vision and artificial intelligence.

One of my biggest problems is that though I have many ideas about how to implement advanced technologies I haven't had much luck finding intersections between the following requirements :

1) Problems of a scale that don't require a large team to achieve meaningful results (ie. no competing with IBM at natural language query or building humanoid robots).

2) Problems for which people with money are willing to pay for solutions.

Generally speaking I have few social connections and most of the ones I do have haven't proved very helpful in meeting my goals.

I too believed (and to an extent still do) that there can be good opportunities in meeting the needs of smaller businesses. However to date my only attempt at this sort of thing yielded no results. You can read about my idea here : <http://cogniception.com/products/rmds.html>. I sent out a dozen or so emails to restaurants in my vicinity but didn't receive a single response.

I think that one of the problems is that there is a tremendous gulf in knowledge between someone like me who has worked with these types of technologies and understands what they can accomplish and small business people who's concept of what computers can do doesn't extend much beyond Word and Excel.

If you're interested maybe we could work together. At the very least you seem to have a greater interest than I in the social aspects of business. I'm definitely open to business ventures with anyone who would be able and willing to help with business development, both at the stage of identifying good matches between business needs and my technology (ie, pre-product development) and with actual sales once products have been developed.

<comment><USER3> "I sent out a dozen or so emails to restaurants in my vicinity but didn't receive a single response."

You need to call them.

</comment></comment><comment><USER4> Well, first off, you're already on the right track. So many "entrepreneurs" start off with some idea and try to make problems "fit" this idea. You've started by trying to first identify problems.

Disclaimer: I'm pretty new to this too.

I obtain ideas by talking to friends/family about areas they're expert in. Sometimes I'll bounce ideas off of people in my network. Sometimes I'll check out stuff from <http://ideaswatch.com> too.

Then I'll try to minimally validate the idea and run through my "filter". It's a list of questions and validation methods I've obtained from the web.

Example: A couple weeks ago I talked to a friend about improving comic book store inventory systems. So I went to the local comic shop, and spent some time talking to the owner about his story - how he opened the store.

I mentioned I am a web developer and like solving problems. I asked if we could meetup the next day and discuss some of his day-to-day problems in the shop.

Well we did meetup and it turns out he doesn't really have problems in regard to inventory management (but does in other areas).

This is just my approach but I'd love to hear what does and doesn't work for you in the future.

<comment><AUTHOR> Wow, I must say this is brilliant - the sort of answer I am looking for!

Would you be able to share what your "filter" is? I.e. what sort of questions do you ask when evaluating new product idea?

Thanks!

<comment><USER4> I'll be posting a video about this tomorrow/Tuesday so hopefully you can wait till then. Definitely start by googling "lean startup machine validation board". Watch those videos.

<comment><AUTHOR> Dude, totally awesome video, thanks! Loved the idea of using Graigslist for assumption validation too.

<comment><USER4> Hey so I posted a video that talks about validating ideas. There is a link to a big list of questions I've compiled.

<comment><AUTHOR> Only found that link in your profile :) Thanks!

PS I wish HN notified users of replies...</comment></comment></comment></comment></comment></comment>


What's Next for R? - <AUTHOR> https://qz.com/1661487/hadley-wickham-on-the-future-of-r-python-and-the-tidyverse/ <comment><USER2> I would highly recommend the use of the package data.table over tibble or the basic data.frame if you are doing any type of modeling in R with larger datasets. Yes R has many data structures but knowing how to use data.table will blow your mind in term of efficiency. Matt and other contributors have built something extremely fast and flexible.

I get that R is not for everyone but used correctly it is a beast.

Now this is anecdotal, but we have in the insurance industry what we call on level premium calculators. It is basically a program that will rerate all policies with the current set of rates.

Our current R program can rate 41000 policies a second fully vectorized on a user laptop that has a an i5 from 2015.

In contrast, the previous SAS program could do 231 policies a minute on xeon 64 core processor from 2017.

For our workload and type of work, R has been a godsend.

Bonus, we can put what our data scientist develop in R directly in production. (after peer review, testing, etc, not different than any other production code)

Back when I started in 2005, we modeled in some proprietary software like Emblem, used Excel to build a first draft premium calculator, rebuilt the computation in SAS for the onlevel program and sent specs to IT to rebuilt the program again for production. All three had to produce the same results.

I've tried Python, Go, Rust, Julia. I'd say Python could be a good alternative but speed of data.table, RStudio IDE and ease of package management in R makes R an obvious choice for us. I believe Julia to be the future but so far the adoption rate in house has been low.

<comment><USER3> As someone "fully fluent" in both, for many workflows that can be properly implemented in SAS, you would expect on a technical level the SAS program could be faster. It's a fully compiled language, it's a "simple" compilation model (compared to R), and the interaction between incremental compilation and the macro system allows you to do some really good blurring between run-time and compilation when performance matters. Plus, by abusing the fact you can define both sql and data step views to further minimise disk read/write, database pass through on certain procedures, and allowimg for in-memory operations (like R) with the sasfile command, from a purely technical point of view, an experienced user of both should be able to beat R in SAS.

But... and here's the big but...I almost never actually meet anyone capable of putting all these steps together in SAS these days that actually understands the SAS computation model end to end.

And SAS's strength, a computation model not being limited by memory by default, becomes a performance weakness when everyone reads/writes every step out to disk and programs without understanding all those little intricacies. SAS hasn't helped any of this by trying to move its eco system away from "programmer" to "application users", so now "programmers" can pick up an interpreted language like R with in-memory default vectorised operations and beat SAS.

Course, I'd still recommend places move to python/R these days because of the broader ecosystems, university talent pool, and avoiding the extensive lock in of proprietary software, but I still feel I have to reflexively respond to "R faster than SAS" claims :p

<comment><USER2> Believe me, I know. The code just becomes unreadable when you put all execution inside the same data step and use hash table to do fast small to big merging. And not to mention debugging that mess when you have a macro layer on top of it. Not having access to function source code, installation process being what it was. I do not miss it.

And yes technically SAS is faster than R but part of the equation is how many people can make SAS code faster than R/python. I had maybe, 1-2 people that could write efficient SAS code.

One version we had was a bunch of macro producing hash merge plus the whole how can I do something without having to get out of the data step. Just horrible. Number of characters in a line of code? You forgot your quote somewhere and now you have to run the magic line.

I hope I'm not too emotional when I say I hope SAS disappears from my industry and we embrace less adversarial licensing.

<comment><USER3> I don't think that's being emotional at all.

I'm being emotional when I say I have a soft spot for it because of some nostalgia and occasionally dropping in to do some "rock star" programming moments with it. But that's the opposite of what I'd want if/when I was running my own ship.

I too almost always try to steer myself and others away from it now because of the licensing/customer hostility. It's absolutely ridiculous...

</comment></comment></comment></comment><comment><USER4> I'm so thankful for R, it's community and their great libraries! I've built a eight year (so far) career in data science using R to model data and perform experiments. I love R's functional programming style / dplyr which makes manipulating data a delight. ggplot2 is such a great plotting library, well worth the investment to learn. Then there's all the stats tools like glm, MASS, through brms for advanced Bayesian analysis ([https://github.com/paul- buerkner/brms#brms](https://github.com/paul-buerkner/brms#brms)). With R and Python, it's a great time to be a statistician-programmer!

I recommend folks looking to start with R check out: [https://r4ds.had.co.nz/](https://r4ds.had.co.nz/)

<comment><USER5> There is also "Advanced R" by Wickham, that goes into more technical details on how the language itself works (and datastructures, etc).

It is also available for free.

</comment></comment><comment><USER6> Cannot comment from my personal impressions, as I have almost zero knowledge of R, compared to several years of using Python for writing apps and working with data. I like R's focus on functional programming, though.

However, a couple of years ago, my wife tried to transition from business consulting to a data analytics / data science role. She started with taking an R course. She was put off by R's complexity and the course's early focus on the details of R syntax, function definitions, closures etc. and abandoned it.

The year after, she decided to try again and enrolled in a course that used Python (with numpy+pandas+scipy as data science stack) and she reported it to be much simpler, more intuitive and easier to learn compared to her previous experience with R. Now she has successfully completed the program and is employed as a data analyst.

<comment><USER7> I guess that's more an issue with the courses than the language per se. Sometimes it is a good idea to begin the course with direct application, instead of focusing on the language.

<comment><USER8> I have encountered a lot of really terrible R learning materials. One data viz course I took (a very, very reputable and widely-used course on a major MOOC platform) taught how to make several simple chart types in each of base R, a library called lattice that I've never encountered since, and ggplot2. I think a lot of it comes from R instructors who started out back before the tidyverse trying to teach the path they _took_ to learning the language, rather than the quickest path to being proficient in the language as it exists today.

The tidyverse is incredibly controversial in parts of the R community; it's essentially an opinionated set of packages that basically comes with its own "standard" library. But I think that wholeheartedly embracing it, and hiding the way to do things in R that you would do them without the affordances that the tidyverse offers, is absolutely the right way to teach R these days. Unfortunately, a lot of courses and books haven't caught up to that yet.

</comment></comment></comment><comment><USER9> What's Next for R?

Doing the exact same thing we did before!

We have a new library called "dtplyr" (no seriously!) it is designed to save users from the arcane and obtuse sides of R by combining the power of "dplyr" and "data.table", the two libraries that were designed to save users from the arcane and obtuse sides of packages such as "data.frame" and ....

I wish I were kidding. There is the absurd contention in the R world that by introducing yet another weirdly named package people can avoid having to learn and suffer through the "real" R.

</comment><comment><USER10> I started at a company using Shiny for their applications and R as part of their data pipelines.

A huge pain point for us is the packaging system. It is absolutely awful. Packages constantly get overridden so we have to install packages in a specific order. Whenever I have reached out to the community (including prominent members, which have written R books) I have always been told to just use the latest version of all packages and just get on with it, which as anybody knows, isn't always possible, especially as there are constantly breaking API changes.

I understand R's history and that in general, it is a lot better than it use to be, but I would only recommend R is used for notebook style work and to keep it well away from production.

We have migrated to Python, which isn't perfect, but the difference in logging and packaging has been night and day.

<comment><USER11> I have also found R in production to be a nightmare. On packaging, the renv package seems to be the new way to try to manage things. It's not perfect but seems to be a step up on what was around before. Have you tried it out at all?

<comment><USER10> I haven't, thank you for the suggestion. I will give it a go.

</comment></comment></comment><comment><USER12> Disappointed in the lack of discussion of R-Shiny or Plumber.

R-Shiny is a full stack platform for web apps, and it's how I leveraged my data science background to get into web development. It's incredibly powerful in my opinion, with the only obvious limitation being the speed of R itself.

And Plumber. It's become the defacto method for deploying R code in a REST api. It too is still maturing, but I see it eventually becoming the Flask of R.

Truth be told, however, after developing quite a few projects on the Shiny/Plumber stack, I wouldn't recommend anyone do it.

If for some reason you can only have an R interpreter, go for it. But learning multiple languages really is the best solution if you want to manage efficient applications. I say this, however, realizing that all of my colleagues writing R don't have engineering backgrounds.

I can't help but feel like R is like JavaScript in many ways. Ease of use and the ease of publishing packages very quickly clutters the repository.

R will always have a special place in my heart, after all it's the language that made me discover programming. However, I can't help but feel that my thirst for efficiency is making me outgrow it as a language quickly.

<comment><USER13> on the shiny note - check out streamlit. declarative python equivalent. it's pretty incredible how easy it is to use

</comment></comment><comment><USER14> When I used R in University (majored in Applied Mathematics and Statistics) I was always awestruck at how every sort of novel modeling technique from GLM, to Beta Regressions, to GARCH, is all easily accessible for free, with proper academic paper and documentation, and with a cohesive standard support.

It was really useful to be able to apply most theory I was learning to actual research datasets. This is what I miss the most since moving to Python.

What I don't miss is R's terrible packaging system and how it made collaborating with colleagues near impossible. I can't count the amount of times I had to debug dependencies on others' script just to be able to move forward with some team project.

<comment><USER15> What didn't you like about the packaging system? Even if you hate R the language, R has among the most user-friendly, cross-plaform packaging systems I'm aware of.

<comment><USER16> [https://stackoverflow.com/questions/10947159/writing- robust-...](https://stackoverflow.com/questions/10947159/writing-robust-r- code-namespaces-masking-and-using-the-operator)

Historically, the conventional way to write R code was one that tended to result in shadowed names (and hence brittle code).

</comment></comment></comment><comment><USER17> I used R when I took an online course on Data Analysis. I didn't like it at all. Its syntax is weird and painful to read. The only nice things about R are Tidyverse and ggplot. I found Python to be a better alternative. You can use Pandas for data analysis y EDA. Matplotlib and Seaborn for plotting. Scikit- learn for training your models. An additional benefit is that Python is a general purpose language that you can use to build a complete application.

<comment><USER18> In almost all of the use cases you mentionned, R blows Python out of the water.

Working with dataframes in R is much much more convenient than Pandas (loc, iloc, etc??)

Plotting is an obvious win for R, matplotlib is horrible, it's powerful yes but it is an absolute pain when compared to ggplot.

Scikit is definitely unmatched but caret is not so far behind. Also, R has a plethora of implemented models that Python lacks (from something as basic as decent quantile regression to time series analysis tools).

As for building a complete application, Python is indeed the go-to.

Syntax wise, using magrittr's pipes is an absolute pleasure. Good luck doing that with Python.

<comment><USER19> Just as an FYI - the statsmodels python package just released numerous new time series tools in version 0.11 rc1 [1] and also has functions for quantile regression [2]

[1] [https://github.com/statsmodels/statsmodels/releases](https://github.com/statsmodels/statsmodels/releases) [2] [https://www.statsmodels.org/dev/examples/notebooks/generated...](https://www.statsmodels.org/dev/examples/notebooks/generated/quantile_regression.html)

</comment></comment></comment><comment><USER20> This currently missing are better LSP (Language Server Protocol)[1] (it supports only some of the LSP features), better linter[2] and static analysis, better integration with GitHub[3], and so on. More on the tooling side, I believe.

[1] [https://cran.r-project.org/web/packages/languageserver/readm...](https://cran.r-project.org/web/packages/languageserver/readme/README.html)

[2] [https://github.com/jimhester/lintr](https://github.com/jimhester/lintr)

[3] [https://github.com/github/semantic/issues/382](https://github.com/github/semantic/issues/382)

</comment><comment><USER21> I also got excited when I found out about R Markdown, and how well is integrated with RStudio. I believe that it is a decent alternative to Jypyter Notebook.

</comment><comment><USER22> I hope a hospice. Ugh that language has damaged me worse than Perl.

</comment><comment><USER23> I know this is a dead horse, but I think R seriously shot itself in the foot with its data structures[1]. I don't really see a solution for this, as fixing it would never be backward compatible. I'll always pick Python over R because the data structures actually make sense to me as a programmer (objects that look like lists, dicts, matrices, etc. or any combination of the above, and they all behave in very predictable ways). I think this puts off a lot of other people like me.

[1]: [https://jamesmccaffrey.wordpress.com/2016/05/02/r-language-v...](https://jamesmccaffrey.wordpress.com/2016/05/02/r-language- vectors-vs-arrays-vs-lists-vs-matrices-vs-data-frames/)

<comment><USER24> True, the default semantics of R's data structures are somewhat arcane (of course as they're based on S [1] from the 70's). And the current support for e.g. 64bit integers leaves something to be desired.

But behind the scenes, R is just a lisp with some data structures that are adapted to statistics and data science.

All base data structures are by default immutable. And e.g. the vector type is extremely performant as it's just a thinly wrapped C Array. In Python you need to reach for Numpy for anything similar, and you do feel some pain when converting between native python types and Numpy types for various functions which support one or the other.

The data frame is immensely powerful. And has excellent performance characteristics as it's built upon vectors. A list of objects, like you'd make in python is just a lot slower and more unwieldy to deal with. And much harder to make generalizable functions upon.

Hadley Wickham's Tidyverse[2] is exactly an attempt to hide away the arcane details and create a modern, coherent and consistent language on top of R, keeping the power of all the great statistics R libraries. The fact that R behind the scenes is a Lisp, with support for macros, makes this possible. For doing data-transformations and statistics, I can't think of anything currently as powerful as CRAN + Tidyverse.

[1] [https://en.wikipedia.org/wiki/S_(programming_language)](https://en.wikipedia.org/wiki/S_\(programming_language\))

[2] [https://www.tidyverse.org/](https://www.tidyverse.org/)

<comment><USER25> This 5 minute video by Wickham was eye opening for me regarding the lispiness or R.

[https://youtu.be/nERXS3ssntw](https://youtu.be/nERXS3ssntw)

<comment><USER26> modern Lisps don't use unquote/quote like that.

This looks more like 'FEXPRS' from decades ago.

1962 the ideas of macros were introduced and macros are source code transformers, which take source code and generate new source code. This can also used in a compiled implementation, where macros translate the code before compiling.

FEXPRs are then functions which get arguments unevaluated and can decide at runtime which to evaluate and how.</comment></comment></comment></comment>


How this hacker can virtually 'kill' you, and what to do about it - <AUTHOR> http://www.csmonitor.com/World/Passcode/2015/0810/How-this-hacker-can-virtually-kill-you-and-what-to-do-about-it?cmpid=TLS

<comment><AUTHOR> The part that got my attention was around the Birth certificate angle: USA & other countries have made the document issued more secure, but method to create birth record still not secure. Ugh.</comment>


Chipworks Disassembles Apple's A8 SoC: GX6450, 4MB L3 Cache and More - <AUTHOR> http://www.anandtech.com/show/8562/chipworks-a8 <comment><USER2> There is also a core for security in there somewhere running an L4 microkernel, as described in the security guide[1] page 7

[1] [http://images.apple.com/privacy/docs/iOS_Security_Guide_Sept...](http://images.apple.com/privacy/docs/iOS_Security_Guide_Sept_2014.pdf)

<comment><USER3> Wow, thats news to me. Its been there since the A7 apparently and is used to process the fingerprint sensor data:

"The Secure Enclave uses encrypted memory and includes a hardware random number generator. Its microkernel is based on the L4 family, with modifications by Apple. Communication between the Secure Enclave and the application processor is isolated to an interrupt-driven mailbox and shared memory data buffers.

Each Secure Enclave is provisioned during fabrication with its own UID (Unique ID) that is not accessible to other parts of the system and is not known to Apple. When the device starts up, an ephemeral key is created, entangled with its UID, and used to encrypt the Secure Enclave's portion of the device's memory space."

</comment></comment><comment><USER4> They found CPU, GPU and cache. What is in the remaining 40% of the chip? There are a lot of individual blocks in there.

<comment><USER5> This is where Apple really adds value and why they have their own silicon design teams.

Some of the blocks:

\-- M7 motion chip

\-- Image processing chip to support camera

\-- (I suspect) audio processing chip to do compression and feature extraction for SIRI

\-- (suspect) video codec (though this may be done by GPU)

\-- Secure Enclave -- CPU with secure data, random number generator, Apple Pay support, Fingerprint Sensor support

\-- (I suspect) memory management units and io co-processors and the like that are too mundane to get much public mention

<comment><USER6> The motion chip is a separate package.

<comment><USER7> Chipworks decapped the M7 on the iphone5 and found it to be a custom model of the LPC18xx from NXP

[http://www.chipworks.com/en/technical-competitive- analysis/r...](http://www.chipworks.com/en/technical-competitive- analysis/resources/blog/inside-the-iphone-5s/)

[http://www.nxp.com/products/microcontrollers/cortex_m3/lpc18...](http://www.nxp.com/products/microcontrollers/cortex_m3/lpc1800/)

</comment></comment></comment></comment><comment><USER8> If these kind of raw IC photos interest you, here's a couple links describing what you're looking at:

"The 6502 CPU's overflow flag explained at the silicon level"

[http://www.righto.com/2013/01/a-small-part-of-6502-chip- expl...](http://www.righto.com/2013/01/a-small-part-of-6502-chip- explained.html)

And a handful of links by other HN members about how to interpret silicon- level images:

[https://news.ycombinator.com/item?id=7481973](https://news.ycombinator.com/item?id=7481973)</comment>


Why no one cares about your brand - <AUTHOR> https://www.wired.co.uk/article/highsnobiety-david-fischer <comment><USER2> Why did that article go into the number of engagements of a random fashion blog...</comment>


About 1400 Words of Skepticism about Markdown, and an Imagined Alternative - <AUTHOR> http://cforster.com/2015/06/markdown-skepticism/

<comment><USER3> I'm a huge fan of _Markdown_ because it frees up extra taps/pecks on the keyboard with the knowledge that my final document is portable and understandable in its raw unrendered plain text to non-techies.

I expect all my documents to be web friendly also. Markdown ensures that extra tweaks (more work) will not be necessary when rendering my MD file for the web.

Blockquotes inside paragraphs are harder on the eyes - best kept as block elements for web reading.

Inline quotes are sufficiant for snippets of quotes within paragraphs - <USER2> most use-cases.

Markdown keeps things simple while enforcing good web writing practises for non-HTML folks or those of us who just want to focus on content and not tags in my opinion. I guess the semantic web is less a concern for me with Markdown because I'm focused on the final deliverable, a willing trade-off of a little non-semantic for the portability and readability benefits.

The bottom line for me is producing content with the least barriers. MD always seems to fit the bill.

<comment><USER4> I've found markdown failing for me for very simple things.

Someone suggested ASCIIDOC as an alternative with similar feeling of being easily editable and readable as is. This was posted on HN a while ago

[https://medium.com/@chacon/living-the-future-of-technical- wr...](https://medium.com/@chacon/living-the-future-of-technical- writing-2f368bd0a272)

</comment></comment><comment><USER5> I get the feeling this is one of the reasons John Gruber was so annoyed by the CommonMark effort - he felt his invention, the thing he called 'Markdown', was the idea of a human-friendly pidgin syntax for a more complex markup format. His _personal_ Markdown implementation happened to target HTML, because that's what he personally needed, but I guess he expected all kinds of markdowns for all kinds of document syntaxes, with varying similarity to the one he wrote himself.

The idea of constraining Markdown to one particular syntax targeting one particular document type must have seemed like epically missing the point... which is a common outcome when a new idea meets popular approval.

From that point of view, Markdown's nearest relative would be AsciiDoc, which is a toolkit for making simple pidgins for the DocBook XML syntax. That tool happens to come with some reasonable defaults you can use as a base, but sure enough there's now the AsciiDoctor tool, which hard-codes AsciiDoc's defaults instead of re-implementing the generic framework.

<comment><USER6> I'm pretty sure he was irritated they tried to appropriate the name.

Gruber's logic is something like: I named my thing Markdown, please don't use that name without my permission (read the third term of the license: [http://daringfireball.net/projects/markdown/license](http://daringfireball.net/projects/markdown/license) ).

The logic of the (at the time) Standard Markdown people was something like: screw that, you aren't even making any of the changes we need to your project.

</comment></comment><comment><USER7> Did you guys try Emacs' org-mode? I feel its syntax to be similar to markdown but way more powerful.

<comment><USER8> That's on my list to try, and I've watched a number of demos (the YouTube video of the Google Talk is quite impressive).

But... I use vim ;-)

</comment></comment><comment><USER9> So, something like jade then? If your annoyances are semantics and the verbosity of other markups. Roll your own semantics with +mixins, and otherwise, just write HTML, without the fluff?

Human readable, concise, fluent, and HTML is a lingua franca of its own. Even in jade, I still make frequent use of the :md filter for simple text, but otherwise, structure is utterly up to you. Markdown alone doesn't quite cut it even for CMS style 'content-block-goes-here' editing.

<comment><USER10> Jade is a templating engine, not a markup language as such.

Personally, I don't understand the persistent popularity of Markdown in the face of a much better alternative like Restructured Text, where the extension story is _not_ an alternative between "hope this flavour of Markdown does what you want" and "just write HTML". Even something simple like tables is not covered by the Markdown "spec".

<comment><USER9> yes, but the author's issues with markdown are limitations of a limited markup. markdown is appealing thanks to the lack of verbosity, hence jade - terse, simple, yet extensible. markdown is nigh one-dimensional, your content should fit in a box, and offers little ability to provide semantics. HTML / XML are overly verbose, and painful for novice / non-technical users. a subset of jade solves these eloquently for me.

</comment></comment></comment><comment><USER8> First off, Chris expand your acronyms. LaTeX is sufficiently well known there's no need for that. The Text Encoding Initiative, (which doesn't seem to be an actual encoding standard?), despite being active since the 1980s, and my own use of various markup systems since about the same time, was unknown to me:

[https://en.wikipedia.org/wiki/Text_Encoding_Initiative](https://en.wikipedia.org/wiki/Text_Encoding_Initiative)

Even knowing that, it's not clear what problem exactly TEI solves.

Second: what's your goal? _How are you looking to use Markdown?_

For offline use in your _own_ document preparation activities, I think Chris answered his own question. Markdown is extensible via HTML. If you want to use _semantic_ elements such as emphasized text or Some Famous Book, then simply include those elements within your own source document.

A lot of us are using Markdown on third-party sites -- for myself it turns up on Reddit, Ello, Diaspora, and StackExchange ... in a mazy of twisty passages, all different. And I really _don 't_ have control over what's parsed or supported (Ello's lack of blockquote and Reddit's lack of images both drive me equally nuts).

To a large extent, for works of a few dozen to 10,000 words, possibly even longer, _heading structured documents work great_. Markdown is just that: a quick and only slightly dirty way to create content for which presentation specification (e.g., bold and italic specs) is more than sufficient. I mean, yeah, it's fucking wonderful that HTML has, _in theory_ , all this wonderful semantic markup. Dig into the source of pretty much any site out there, with _very_ few exceptions, and you'll find they're simply butchered. Table layouts and  directives for titles and headings and absolute positioning all over the page.... Or worse, custom styles which make even  and  look stunningly attractive.

If anything, documents tend to go to the _other_ extreme: too many entities, too much markup, too much crud, too much design. To the point that I'm far too frequently scraping raw text from documents and rebuilding them (usually with Markdown) to a simple basic PDF or HTML page _that I can actually fucking read._

OCD, it's a hell of a drug.

My suggestion: get over it.

 _You cannot get both a SIMPLE and a COMPLETE markup language._ One is the enemy of the other.

Settle for good enough.

The problems I've got with Markdown (or any of the other lightweight markup languages: [https://en.wikipedia.org/wiki/Lightweight_markup_language](https://en.wikipedia.org/wiki/Lightweight_markup_language)) is that standards aren't, parsers and interpreters (particularly on third- party sites) vary and/or suck, and stuff I'd really like to see (superscript, subscript, footnot, ToC generators, equation support) aren't well supported. Hell, we can't even rely on blockquotes, images, character escaping, and superscripts being uniformly treated.

Use Mardown. Or switch to another LWML. And extend it with HTML. Use post- processing scripts to add those div wrappers and metadata you want (I'm basically doing the same for my own work).

The one other element that might make sense, and which you don't address, is a document standard -- I'd _really_ appreciate if documents had standard metadata: title, author, publication date. Something vaguely like an RFC #822 of text document metadata (and fortunately we've got *so many standards to choose from: [https://en.wikipedia.org/wiki/Metadata_standards](https://en.wikipedia.org/wiki/Metadata_standards) \-- Ob xkcd 927).

It's also not clear how or why LaTeX itself isn't suitable here. It's plaintext, more structured than Markdown, yet generally easier to code than HTML.

I do appreciate the tip on ScholarlyMarkdown, which looks like it might address a few of my own interests.

<comment><USER11> I never really understood the point of these simplified markup languages. I see Markdown and ReST being pushed on non-technical writers who just don't want to use a text-based markup language. They want Word, or Google Docs, or a rich edit box, or something like that.

As for me, yeah I like writing in plain text but given the choice I'd rather just write in HTML which I already know, rather than use something like Markdown. I don't find Markdown to be particularly time-saving or easier to use. Or if I'm writing something where I particularly care about beautiful typesetting or I need a table of contents and an index I'll use LaTeX which has been developed for decades specifically for scholarly and technical writing.

<comment><USER12> These users want familiarity, and get false simplicity / silo-lock-in. I'm not fond of giving emacs to everyone anymore, but even the simple fact of showing that simple text can be transformed into formatted beauty can open their mind.

Otherwise, IMO, they don't understand they have a computer wanting to compute for them, they see a weird typewriter simulator that they should appease as much as possible otherwise that god might crash taking their hard (because the software is obscure) work down the drain.

Now I also find markdown good enough. Headers, paragraphs, lists, quotes, links... Gruber's page list 10 kinds, easy to remember for anyone; balanced, minimal and generic document feature set.</comment></comment></comment>


What Happened To Uploading A File To The FTP Site And Hitting Refresh? - <AUTHOR> http://www.floopsy.com/post/32661616732/uploading-a-file-to-the-ftp-site-and-hitting-refresh

<comment><USER2> I still use FTP to live update production sites. I do this for my personal blog, and on my projects that don't get much traffic.

It really only becomes a sin when you have a lot of visitors who are relying on your site. For my larger sites, I have a build process. It's all a mater of scale and how much risk is acceptable for your website.

<comment><AUTHOR> You have a very good point - Thanks. Any suggestions on a "build process" best practice, etc.?

</comment></comment><comment><USER3> I think the audience of devs uploading a single PHP file have moved over to CMS's. Wordpress, Joomla & Drupal have very much taken over that, which in a way can be a good thing. They can be lightweight, easy enough to use and can grow as big as you want. The other big reason is plugins/component/libraries too.

</comment><comment><USER4> If you want to deploy a site using FTP, why not do it? Why does it matter what people think of your deploy process?

<comment><USER5> There's absolutely nothing wrong with it in principle.

It's only that as the web grew up, and websites became less a bunch of static files and increasingly an application, or more commonly a pile of application- like messes run by many different people at the same time, involving a bunch of dependencies between files, whether organized code or haphazard things piled on things piled on things, doing staged releases comes to be the more sensible option.

At a certain level of complexity, FTP can become a liability and cost you business.</comment></comment>


Ask HN: How come we don't have 1024 cores by now? - <AUTHOR> In 2007, I wrote an article about concurrency (https://www.gamasutra.com/view/feature/129998/the_transition_to_concurrency.php) and about how Moore's Law was coming to an end. That has happened but to my surprise, my brand new laptop still barely has 4 cpu's. I was certain we would have started advancing "horizontally" instead but that never truly happened. How come? <comment><USER2> We do in GPUs, which is where thread-heavy computation is at these days.

</comment><comment><USER3> A few things.

Advances in computing are limited by memory bandwidth. Even if you could accelerate a computation 10,000x by implementing it as an ASIC, you would need to increase memory bandwidth 10,000x as well.

People are getting better at taking advantage of multiple cpus, but we have limited ability to get parallelism for many applications that matter, such as web browsing, so we aren't ready to break up those tasks to run on 1024 low- performance but low-power CPUs.

You can get server machines with 64 cores on a "chip", even put two cores together for 128. Those sell like hotcakes for cloud computing.

<comment><AUTHOR> According to activity monitor, I have around 2200 threads running at the moment. Obviously most of those are idle but it still seems to me that we should be able to utilize multiple cores even without switching to whole new paradigms?

<comment><USER3> It depends on the workload. If you can cut the work onto 1 millisecond chunks that barely interact you can scale fine in this paradigm.

Threads are used both to schedule work across cores but also to manage waits on i/o, a server will have threads running when it is waiting for a client and a Windows or Linux or Mac has many servers running for local use.

Threads are not efficient for I/O scheduling (async/await or select or similar apps are better, but harder to code for.). They are efficient enough for your laptop to have 2000 of them.</comment></comment></comment>


Introducing Rubeque.com, an interactive learning site for Ruby - <AUTHOR> http://rubeque.com

<comment><USER2> Very clean site with some challenging problems.

</comment><comment><USER3> More people should try this.

</comment><comment><USER4> very cool</comment>


Show HN: Rebout.me, build nameplate sites with CTA buttons and export feature - <AUTHOR> https://rebout.me/?just=true <comment><AUTHOR> Feel free to take a look at my page [http://marcograf.me](http://marcograf.me)</comment>


FCC tries to bury finding that Verizon and T-Mobile exaggerated 4G coverage - <AUTHOR> https://arstechnica.com/tech-policy/2019/12/fcc-tries-to-bury-finding-that-verizon-and-t-mobile-exaggerated-4g-coverage/ <comment><AUTHOR> I found the article interesting enough to post if only because I used to work for a company (RootMetrics; since bought by $SOMEONE) that had people driving around and going to airports, lugging a big case with a half dozen phones recording signal strength. There are other options as well (OpenSignal, or summat?). I'm not saying the FCC needed to go give RootMetrics, et. al., ongoing money. But I would think that instead of driving around in FCC cars they would have, you know, first asked "anybody else already do this? Think they'd sell us some of those data?"

OTOH, that comes with a very large assumption of no cronyism nor glad-handing the industry.</comment>


Bad kitty "Rookie mistake" in Cryptocat chat app makes cracking a snap - <AUTHOR> http://arstechnica.com/security/2013/07/bad-kitty-rooky-mistake-in-cryptocat-chat-app-makes-cracking-a-snap/ <comment><USER2> Was this link submitted by a bot or something? Why is the exclamation mark missing from the title?</comment>


Website Listing Entry Level Positions Only? - <AUTHOR>

Entry level, New Graduate, Junior... it's hard to search for these kind of positions, especially in tech centric areas. Just wondering if anybody is trying to intelligently aggregate them. <comment><USER2> probably not exactly what you're looking for, but better than a stick in the eye

[http://www.jobdensity.com/QueryGrid.aspx?q=886&t=junior%20so...](http://www.jobdensity.com/QueryGrid.aspx?q=886&t=junior%20software%20engineer&qt=6/15/2015%206:48:21%20PM)</comment>


Ask HN: What's NH's opinion of the next big thing in tech? - <AUTHOR> What does everyone think the next big innovation that will be widely adopted be? Be sure to put your time horizon into the discussion so we can tell when you think it will occur. <comment><USER2> It's still the early days of the internet when it comes to small business. Fully 40% lack a Facebook page or website and 60% lack having both. The vast majority of brick and mortar businesses to not allow online sales nor have even a mechanism for people to view actual inventory on a digital medium of some kind.

If you're looking for the next big tech opportunity it's in helping businesses with fewer than 50 employees expand their reach outside of the 10 mile radius around their physical locations.

<comment><USER3> Are those real stats or guess?

<comment><USER2> Real. I ran enterprise sales for a very very large web host and these are figures we commissioned. Lots of other data in there too, like how 6,000,000 people on earth received money in 2015 for providing a digital service or some kind. A number expected to grow to 20 million by 2020.

</comment></comment></comment><comment><USER4> Tech detox, I increasingly see people lamenting the invasiveness of tech in their lives and wanting an escape from some of it

</comment><comment><USER5> The IOTA project [http://www.iota.org](http://www.iota.org) is a dystopian mess at the moment but ternary computing, the internet of everything, the tangle paradigm instead of blockchain and fee-less transactions are what many are dreaming of and possibly at hand even before quantum computing (therefore sooner than later).

</comment><comment><USER6> I don't know about the next big innovations that will be widely adopted, but I think I know that if the following innovations happen, they will probably be widely adopted...

\- Applications that aren't slow: "In a world where an app opens right when you click on it".

\- Fast websites: type in the url, hit enter, and you're in. You click on a link, and voilà..

\- Split second boots.

\- Chrome not killing my computer when opening too many tabs (I used to open 120+ I think with Firefox).

\- Make sbt not download the whole internet (yes, cache and all).

\- Printers that actually print when I want to. Also make printing in black not require other colors.

\- Automatic transmission cars that don't do a jerk motion when they stop.

\- Universal cartridge format.

\- Time tracking app that magically monitors what you do during the day without your intervention and can show you a treemap. Over the years, you can learn something.

\- A huge dataset on every company and human that was ever formed and its life through success or failure. Maybe we can learn from that.

\- A way to encode and distill the whole human knowledge. A graph of "knowledge" of sorts.

\- Peer review that works.

\- Time management, cognition, meta-cognition, deliberate practice, empathy, cognitive biases, and how memory works lessons in primary school.

\- Voice calls on (Skype, Viber, WhatsApp, Messenger) that sound like a phone call.

\- A way to simulate your life with varying parameters and find a path to a desirable outcome.

\- Natural computation: A billion objects falling don't require nature to spin up more VMs to handle all their trajectories and write a Medium post about how it did it. Physics seems to handle great loads, concurrent events, several processes mutating the same object simultaneously, out of the box. A computer that does _that_.

<comment><USER7> On the topic of SBT: [https://www.lihaoyi.com/mill/](https://www.lihaoyi.com/mill/), a response to [http://www.lihaoyi.com/post/SowhatswrongwithSBT.html](http://www.lihaoyi.com/post/SowhatswrongwithSBT.html)

</comment></comment><comment><USER8> Honestly I feel our political and economic structure need to change to reflect new tech. Democratic republic and capitalism as we know it.

The age of information seems to have become a breeding ground for misinformation. I don't really mean "fake news", but deceptive, plausible, half true news. This had made democracy a little... weird lately, especially in an era you can deliver different news to someone based on their location.

Our economic system has been based on supply and demand. But what if we basically have unlimited supply, as is the case with software? AI is going to hit hard here, with it being smart enough to replace all minimum wage foreign labor. A lot of governments are experimenting with basic income in their own way.

We've also had this weird effect where a small company can raise millions in investment and seriously threaten well-established companies within a decade.

These changes will take a while to hit, at least a decade. But I think the moves will have an impact on what kind of tech emerges.

<comment><USER9> Ditto this one, though perhaps from a slightly different angle.

Consider when most of our economic and political systems were first founded. Information was sparse. Game theory wasn't even a thing yet. None of the founding fathers (of our nation or any other) could have possibly fathomed the sort of interactions happening today.

At some point, we're going to have to face the fact that, over the years, the underlying rules of how things works has shifted but that the laws on the books haven't matched.

</comment></comment><comment><USER10> Please tech gods, make personal jetpacks a thing. I would pay $50,000.

They're currently about $275,000 and I only know of one on this Earth. Also jet fuel is expensive.

</comment><comment><USER11> Boring answer, but for me I'd bet on self-driving cars that you can hail via Uber type platforms.

<comment><AUTHOR> Not a boring answer in the slightest. I can definitely see self-driving cars going mainstream at some point. I think self-driving trucks doing long-haul transport will go mainstream but the driver will still be required due to driving laws most likely.

</comment></comment><comment><USER12> 5G and next generation GPS.

</comment><comment><USER13> *HN's (sorry for the nitpick)</comment>


Cheap renewables won't stop global warming (2018) - <AUTHOR> https://energy.stanford.edu/news/cheap-renewables-won-t-stop-global-warming-says-bill-gates <comment><USER2> Project Drawdown [1] deserves a mention in any discussion in this field. They are a substantive research enterprise trying to map what can (technically, not politically) be done to reach the point where CO2 ppm start to fall in the next 30 yrs. Good thing is: it can be done. Bad thing is: the incentives aren't there right now. Bill is right, energy isn't the largest component. It's food and (delta) people that need to change the most. Step away and we need all measures to work and succeed in concord, including energy.

There's a podcast for you to have a taste for the quality of the modellers and what they are up to in the next fase. [2].

My 2 cents is a CO2-tax would align incentives quite painfully. All we get in return is the drama of creating an artificial pool for a unlimited 'resource'. The cost of all measures is quite affordable, on a global, long-term scale.

[1] [https://www.drawdown.org](https://www.drawdown.org) [2] [http://www.thedrawdownagenda.com/about/](http://www.thedrawdownagenda.com/about/)

<comment><USER3> From a CO2 standpoint wouldn't it be good to stop subsidizing immigration to the first world? It's not just delta-population it's people moving from a low consumption environment to a high consumption one.

<comment><USER4> Better option regarding both CO2 and immigration, IMHO:

* Global programme to drastically reduce birthrate.

* Global development programme based on good governance and actual results, not just "sending money".

<comment><USER5> Global programmes to drastically reduce birthrate already exist. They accomplish it by educating women.

<comment><USER4> Considering the growth of the world's population they don't seem to be accomplishing much...

</comment></comment></comment></comment></comment><comment><AUTHOR> This contains a link to the actual discussion [1] which I highly recommend watching. Bill demonstrates a sophisticated understanding of energy systems, guided by one of his favorite authors (Vaclav Smil) who recently wrote the incredible and humbling book, Energy and Civilization [2].

[1] [https://www.youtube.com/watch?v=d1EB1zsxW0k](https://www.youtube.com/watch?v=d1EB1zsxW0k)

[2] [https://www.gatesnotes.com/Books/Energy-and- Civilization](https://www.gatesnotes.com/Books/Energy-and-Civilization)

<comment><USER6> It didn't sound particularly sophisticated to me. For instance, he asks the rhetorical question, what kind of battery is going to power Tokyo at 22 GW through a cyclone?

One way to answer that question is to ponder the following question - if we can reduce the price 90% of the time by say half, how much are you willing to pay for power in the remaining 10% of the time?

There are a multitude of technical solutions.

But you can only see them if you actually look for them. If you're down in a nuclear 24/7-constant-output-except-for-one-refueling-month-per-year pit, then you can't.

Cheap renewables are going to be favoured by the marked forces. Yes, we need to solve the gaps, but will just have to do so. Nuclear can't do it, so there's really little room for nuclear in the future.

<comment><AUTHOR> That's not a rhetorical question at all. To do that you need 22 GW of storage that's only used a few percent of the time. Why are we going to price gouge everyone to pay for reliability when we could just build reliable energy sources in the first place? Show me the battery that can do 22 GW for three days.

As for cheap renewables being favored: check out this other thread about Californians banning new solar installations in the desert [1]. As we ramp wind and solar from 3% of total energy (not just electricity) towards 100%, you can bet you'll see significant NIMBY stuff at work. Plus, what are you going to do with the waste from all the gargantuan retiring solar and win farms and chemical battery systems?

You mention that nuclear can't do it without substantiating it. Care to expand on why that'd be impossible? France and Ontario are doing pretty well in the decarbonized world at the moment thanks to nuclear. [2]

[1] [https://news.ycombinator.com/item?id=19277653](https://news.ycombinator.com/item?id=19277653) [2] [https://www.electricitymap.org/](https://www.electricitymap.org/)

<comment><USER7> Just repeating "building nuclear is much cheaper" doesn't make it so. Nuclear is safe. It has some very admirable technical qualities. But it's difficult to find examples of reactors built quickly and at low cost any more[1]. South Korean reactors built _in_ South Korea seem to be the last good example.

South Korean reactors in UAE are behind schedule. China takes 6 years to build reactors in China and their costs aren't very transparent due to opaque government and censored press. Russia takes even longer to build reactors in Russia and abroad. India takes a decade+ to finish a reactor. France can't build their new reactors swiftly _or_ affordably, either in Finland or in France. We all know how Westinghouse's AP1000 has fared in South Carolina, Georgia, and China.

Ontario experienced drastic cost overruns building the Darlington reactors in the first place and rejected new Darlington reactors a decade ago because the projected costs were too high.

 _The Ontario government put its nuclear power plans on hold last month because the bid from Atomic Energy of Canada Ltd., the only "compliant" one received, was more than three times higher than what the province expected to pay, the Star has learned._

 _Sources close to the bidding, one involved directly in one of the bids, said that adding two next-generation Candu reactors at Darlington generating station would have cost around $26 billion._

[https://www.thestar.com/business/2009/07/14/26b_cost_killed_...](https://www.thestar.com/business/2009/07/14/26b_cost_killed_nuclear_bid.html)

I have some hopes that small modular reactors will eventually break out of the rut of disappointment that recent Western reactor projects have furrowed. I'm not trusting anybody's cost/schedule estimates until some SMR projects are actually in commercial operation.

Finally, if NIMBY is a legitimate obstacle to renewable projects, it's also a legitimate obstacle to nuclear projects. (I think that NIMBY is _not_ a good reason to block either renewable facilities or nuclear plants, but I'm not King of Earth.)

[1] You might think that requiring swift completion on top of affordability is excessive, but my view is that Western projects that aren't fast will become expensive as well. Even projects that never get within earshot of the NRC, like building transmission lines or refurbishing bridges, are very predictably expensive if they stretch beyond 5 years.

</comment></comment></comment></comment><comment><USER8> Posting this from Australia, being +50 and having lived through one of the hottest summers I can remember.

Strangely enough today we officially went into Autumn yet that too feels like Summer across most of Australia.

This leads me to believe that in fact the world has seriously under estimated climate change.

For what little it is worth, my last 10+ years of lived experience here in Australia has seen things getting hotter and hotter.

My take on the situation is the IPC have actually got it wrong and there are way more, unidentified positive feedback loops in play, meaning the global climate system is actually showing signs of accelerated warning.

Here in Australia we are now in Autumn and the temperature is +10 above average, making it feel like the middle of Summer.

Time will tell, if in the next few year's we once again manage to break this current Autumn heatwave record. I suspect it will.

<comment><USER9> > Here in Australia we are now in Autumn and the temperature is +10 above > average, making it feel like the middle of Summer.

Thing is, when they say global average of +1 or +2 degrees C, people have a hard time mapping that to how that works out.

If you measure 4, 5, 5, 6 the average is 5. If you measure 1, 2, 9, 9 the average just increased to 5.25. So very hot summers can be "masked" by colder winters.

</comment></comment><comment><USER10> No one mentioning carbon extraction technologies here?

Just adding less CO2 to the atmosphere is not enough to stop the trend, we actually need to undo what we've been doing all these decades and need to _massively_ invest in carbon extraction technologies. Trillions of dollars, probably - about the same amount of what we spent on the Iraq war, so I'd say it's not undo-able.

<comment><USER11> I've read about this neat process for extracting CO2 from the atmosphere. You place a ultra compact box in the ground containing it's own building instructions and tooling for separating CO2 in to oxygen and building giant structures of the carbon. It's apparently called photosynthesis.

<comment><USER12> You're joking, but producing biochar is actually a decent way of doing carbon capture that doesn't require any fancy technology. You even get some energy out of it. Unfortunately you'd need to plant absolutely massive amounts of trees to put a dent into the trillion tonnes of CO2 we've released in the last 150 years.

<comment><USER13> Also algae could be neat allies. But there we are playing with fire, maybe GMO algae can be engineered to be supper efficient, but at the risk of compromising marine ecosystems.

<comment><USER14> The fern Azolla already did this once! linky --> [https://en.wikipedia.org/wiki/Azolla_event](https://en.wikipedia.org/wiki/Azolla_event)

</comment></comment></comment></comment></comment><comment><USER15> > _"Electricity is just 25 percent of greenhouse gas emissions," said Gates. > "There is no substitute for how the industrial economy runs today."_

That's why...

<comment><USER16> Electricity may be only around 25% (it's actually a little more than that for the US and almost half globally), but cheap renewables don't just affect the share currently from electricity, they also (by way of emerging use of electric vehicles) attack the share that comes from transportation, which together with electricity is over half in the US and close to 70% globally.

US: [https://www.epa.gov/ghgemissions/sources-greenhouse-gas- emis...](https://www.epa.gov/ghgemissions/sources-greenhouse-gas-emissions)

Global: [https://ourworldindata.org/grapher/carbon-dioxide- co2-emissi...](https://ourworldindata.org/grapher/carbon-dioxide- co2-emissions-by-sector-or-source)

<comment><USER17> Don't forget space heating also.

There's a lot of things that can be electrified, that currently aren't.

<comment><USER18> Funny thing, space heating is probably the last thing you want electrified, instead investing in heat (re)capture and building design.

Air conditioning including plain old ventilation is the hard one to tackle and is already electrified.

<comment><USER17> By heat recapture do you mean heat pumps and insulation?

I (personally) include heat pumps in the heating category, and yes most houses are probably under insulated. Insulation can lower the air con load also.

</comment></comment></comment></comment></comment><comment><USER19> The weaknesses of our technocratic mindset on HN are so apparent whenever this subject comes up.

This is not a technical problem. We know the threat, we know what we should do, but we can't actually get it done.

This is a political problem.

And to my eye, it's a particularly intractable one. In democracies, in order to make the huge sweeping changes required to avoid the worst of climate change, you need the population on board. And they are _not_ on board for those kinds of changes. You can scream and wave your hands all you want about CO2 taxes and smaller houses and not eating meat, but as soon as you start hitting people in their pockets _hard_ to encourage other actions, they'll just vote you out of office instead. There will _always_ be another politician who is happy to tell the population that they don't need to put up with this crazy sacrifice, there are better ways, etc. So either you get voted out of office or you just cave to the pressure.

Humans are going to suffer from climate change, a lot. And even then, I suspect technology is what will save us, not billions of people deciding (for the first time ever) to make great individual sacrifices for the greater long- term good. It's the ultimate collective action problem, and we are not up for it.

<comment><USER20> This isn't a political problem, all problems are technical, because we are only ever going to solve problems politically.

</comment></comment><comment><USER21> It's a little off IMHO to argue that energy storage affects fuel usage only in electricity. The most visible company in the business right now is Tesla which primarily goes after transportation. Together, electricity and transportation account for 56% of US emissions:

[https://www.epa.gov/ghgemissions/sources-greenhouse-gas- emis...](https://www.epa.gov/ghgemissions/sources-greenhouse-gas-emissions)

Another large chunk of emissions come from burning fuel for heating, at least 11% (commercial/residential heating slice) but probably about half of the industry slice as well comes from local energy generation. This tends to be harder to replace with batteries because of the form of energy desired, e.g. you can't replace an acetylene torch with electricity.

If we _very generously_ assume that most modes of transportation excepting airplanes and snowmobiles can be run on batteries, and gas heating can be replaced with electric heat pumps, the proportion of emissions affected by these two technologies could be around 60-70%. That's not a complete fix, but it is a big piece of the puzzle. However, the technology turnover is extremely difficult and expensive, which I think is the bigger problem hiding behind "renewables".

<comment><USER18> You can replace an acetylene torch with gas electric arc welding and cutting. It is done, but is currently more expensive (not a whole lot mind you).

The problem is not manufacturing but mostly smelting and refining. (Including recycling)

You need huge sustained _and_ peak energy input for that, renewables do not deliver it.

</comment></comment><comment><USER22> There are many pieces to this puzzle. Energy Storage is a hugely important piece to solve.

[https://blog.syllablehq.com/can-we-create-the-worlds- largest...](https://blog.syllablehq.com/can-we-create-the-worlds-largest- battery-deep-below-sea-level/)

^ This is a proposal that we consider the ocean as a source for relatively affordable pumped storage.

</comment><comment><USER23> These conversations inevitably turn towards population control. Here's the thing: we started this process of global warming when we had a lot less people on the planet, and it would continue even if we cut the population in half. If the science is correct – and I personally think it is – we have something like a decade to get onto a different trajectory. While reaching sustainable population levels in the long term is in vital, population control is not going to address the crisis in the next ten years. Even if we cut the population in half. And, how on earth would you even do that without throwing the world into turmoil?

There's no escaping the fact that the developed world emits more carbon per capita than the rest of the world. To me, it's obvious: the developed world needs to become drastically more efficient with carbon, and it needs to do it _yesterday_. We need to lead the way, and we need to pay for it. It makes no sense to put this on the backs of the developing world – who are already suffering the consequences of our emissions.

If I had to put it on a bumper sticker, it would be: "It's Agriculture, stupid." Conventional agriculture damages topsoil, and destroys forests, which are two of the most effective carbon sinks we have. When you combine that with the amount of petrochemicals involved in conventional farming, along with fuel burned freighting food halfway across the world on a daily basis (how are those out-of-season cherries?), we're just beginning to come to terms with just how big of a contributor our food system is to global warming.

Here's my list of what we should be doing:

1) Carbon tax

2) Eating less meat, and less out-of-season produce

3) Repairing soil and forests through conservation and more sustainable farming

4) Letting people work remotely to cut down on car and air travel

5) Massive public and private investment in infrastructure

6) Universal access to family planning

<comment><USER24> No, sadly, no. None of the above.

Here's an uncomfortable fact few know about and even fewer are talking about:

If humanity, and all we have ever created, evaporated from the planet tomorrow, it would take 50,000 to 75,000 years for atmospheric CO2 concentration to come down by 100 ppm.

Source: Ice core data going back 800,000 years. Look it up and see for yourself (sorry can't link right now).

<comment><USER23> I initially misread your comment and thought you were arguing something entirely different.

If what you're saying is that simply removing humans from the equation wouldn't change atmospheric CO2 for a long time, I think we're actually in agreement there. It supports my point that population control isn't how we're going to get on a different trajectory in the decade or so we have.

But I don't think it addresses the ideas of effective carbon capture. Soil carbon capture, for one, is incredibly promising. Reforestation is another.

</comment></comment></comment><comment><USER25> Lesser consumption will.

<comment> barry-cotter Poverty for all! Smaller houses and apartments, heated less, wearing more clothes in winter and less AC in summer, all combined with a collapse in international travel. That's a real vote winner.

Technological solutions, megadeaths to reduce consumption or dealing with climate change. These are the only realistic options. China and India aren't staying poor.

<comment><USER17> Compare the US and Western Europe. Europe has lower Co2 emissions, smaller houses, and still maintains quality of life.

No doubt you could go much further along that road.

And what's wrong with wearing more clothes in the winter?!

<comment> barry-cotter All but 13 US states are richer than the Netherlands.All but six are richer than Germany. All but five are richer than Belgium. Only Mississippi is poorer than France or the UK. Every single US state is richer than Italy.

[https://en.wikipedia.org/wiki/Comparison_between_U.S._states...](https://en.wikipedia.org/wiki/Comparison_between_U.S._states_and_sovereign_states_by_GDP_\(nominal\)_per_capita)

There is no way you are going to get Americans to voluntarily reduce their standard of living outside of a war. That applies to Europeans, Australians, Canadians, Chinese and everyone else too. Americans aren't special.

I don't think there's anything wrong with wearing more clothes in winter but it is a consistent vote loser, like all the other realistic proposals for individual action on climate change. This past winter I wore plenty of extra clothes but people like being able to walk around inside in a T-shirt and pants. Everywhere they can afford to they do it more or less as soon as it's affirdable, maybe with a thirty year generation gap.

<comment><USER17> "There is no way you are going to get Americans to voluntarily reduce their standard of living"

My point is you don't have to reduce your standard of living (quality of life). Does the average American have a better standard of living than the Average European, and is it so much better that it accounts for the much higher carbon footprint?

I've never voted for anyone on the basis of wearing more or less clothes. Id hazard a guess that most voters wouldn't identify it as a top 10 issue. Plus didn't Jimmy Carter tell people to wear a sweater?

<comment> barry-cotter > Does the average American have a better standard of living than the Average > European, and is it so much better that it accounts for the much higher > carbon footprint?

Yes. The only countries with higher average individual consumption than the US are Norway and Switzerland.

[https://en.wikipedia.org/wiki/List_of_countries_by_household...](https://en.wikipedia.org/wiki/List_of_countries_by_household_final_consumption_expenditure_per_capita)

The only country with bigger houses than the US is Australia.

[https://www.theglobeandmail.com/report-on- business/average-h...](https://www.theglobeandmail.com/report-on- business/average-home-size-by-country/article27181802/)

Jimmy Carter wasn't re-elected.

<comment><USER17> You've mentioned gdp, consumption and house size. None of those directly translate to quality of life, although I agree they are strongly correlated.

The average European living in a city may not have a car, that is not the same as the average American who lives in a city and doesn't own a car. European cities are have a lot better public transportation, so you don't need a car.

In my experience houses in the city tend to be smaller than houses in the country. But the city is where the wealth is created, so it doesn't seem to me that larger houses should be an indicator of wealth in and of itself. I'm undecided whether living in the city with a smaller house but higher income would lead to a higher quality of life versus living in a bigger house in the country, but with a lower income. I suspect its probably down to personal preference.

<comment><USER19> I love Europe for the reasons you mentioned, but it's a different culture. Most Americans have no interest in living a European-style life and would see it as a huge decrease in their quality of life, _especially_ if it were forced on them.

Which means it's effectively impossible to do, because as soon as you try, they'll vote you out of office. Short of suspending elections, I don't see how democracies make the level of changes needed until we're actually suffering badly from climate change. But historically it doesn't seem like the types of people who become tyrants do so because it's the only path available to save the society from itself.</comment></comment></comment></comment></comment></comment></comment></comment>