Ask HN: What are the most interesting emerging fields in computer science? - <AUTHOR> Hey HN,<p>What do you think is the most interesting emerging field in Computer Science? I'm interested in PHD areas, industry work, and movements in free software. <comment><USER2> Secure Multi-party Computation.

The basic idea is developing methods for two (or more) parties with sensitive data to be able to compute some function of their data without having to reveal the data to one another.

The classic example is developing an algorithm that allows two people to figure out who is paid more without either revealing what their salary is.

Such algorithms get significantly more complicated if the threat model starts changing from "we're all acting in good faith, but we just don't want to share this private info" to "I'm not sure some of the people involved in this are acting in good faith."

Based on my (admittedly limited) look into this field, it seems like there has been some theoretical progress made here, but there's nothing like a generalized framework or library for general development with it. Instead, practical applications seem to be one-offs. For example, a contractor a while back developed a system that lets parties (nation-states or private space firms) figure out if their satellites are going to run into each other without revealing anything about the location or orbit of their satellites. That way they don't share sensitive data, but they can move their satellites if they're on a collision course with somebody else.

Personally, I got interested in this when working for the government. I was working on an extremely cool data integration project (State Longitudinal Data System grant form US Department of Education) that basically went nowhere because we couldn't get over the legal hurdles to data sharing... If we didn't have to share data, but could still compute interesting statistics about the data, that would have been really cool.

<comment><USER3> This is a big deal.

If we can find ways to perform secure, multi-party computation, we could develop fully distributed computational, networking, and power delivery systems.

Your solar roof tiles could be, basically, CPUs or GPUs with embedded wireless networking.

<comment><USER4> Can you elaborate why solar roof tiles need such intelligence, especially the secure part? I don't see what data they would need to hide.

<comment><USER5> Distributed power electronics are inherently dangerous. If you have a surplus of power and your neighbor has a deficit, it is desirable for the two systems to communicate to send power from you to the neighbor. If a bad actor can leverage this system to tell everyone to send all of their power to the grid, they could easily damage the infrastructure in a very costly way.

<comment><USER6> Ok so the grid (that knows everything already) commands every house what to do, using a pinned certificate to authenticate. What part of this needs to be zero knowledge?

<comment><USER7> It's also important to point out the grid is archaic in many places and the knowledge isn't omnicompetent across the system. In the US, we have three major grids which are also connected to Canada's (which I can't speak too). [0] From my understanding, with an amateur interest, we know a lot about where those major grids connect but very little about the distributed nodes that make up the network; The Northeast Power Blackout of 2003 is a really great example of this. [1] Essentially, there was a grid failure that occurred in Ohio which overlapped into the other sections of the grid. In short, a wire contacted with a tree in Ohio which caused a cascading failure to NYC.

So, let's now bring IoT into the mix. You and I have smart houses, with smart solar tiles. John attacks our tiles plus all our neighbors and directs a major electrical spike towards our local substation. Now it's a physics question, where are all those joules of energy going? It's a heat problem and right now there is no way to dissipate that heat from the system which will melt our substation. Let's say our neighborhood is between several other neighborhoods and the main power station, we just killed a node and guess who else doesn't have access to power because they don't have tiles like we do.

That's the basic concept on why security and authenticity are important in relation to the energy grid. It would be nice if there was an effective way to dissipate an "electrical DDoS"? I'm not sure if it's called something else. If you're interested in the energy dissipation within the energy grid, this is a great question on SE. [2]

That all said, is also a major reason why the government is consistently freaking out about our power grid being hacked.

[0] [https://en.wikipedia.org/wiki/Continental_U.S._power_transmi...](https://en.wikipedia.org/wiki/Continental_U.S._power_transmission_grid) [1] [https://en.wikipedia.org/wiki/Northeast_blackout_of_2003](https://en.wikipedia.org/wiki/Northeast_blackout_of_2003) [2] [https://electronics.stackexchange.com/questions/117437/what-...](https://electronics.stackexchange.com/questions/117437/what- happens-to-excess-energy-fed-into-the-power-grid)

</comment></comment></comment></comment></comment></comment><comment><USER8> I think most interesting computer science fields are actually application of CS in other domains.

Science changed a lot in the last decades, moving from a genius in a room looking at the data and coming up with grand theory to have vast amounts of data that no single human can make sense of. The work of the computer scientist is to quickly understand problems from various fields then solve it using tailor-made algorithm that leverage the prior knowledge, the data structure.

One of such interesting fields (which I'm working on), is computational biology. We're working on leveraging sparse experimental data for protein structure prediction. To do that, we end up using algorithms and ideas from different various CS fields, from machine learning, to robotics, to distributed systems. Other people are working on exciting fields like computation protein design, studying drug protein interaction in silico..

<comment><USER9> Yes! I studied law before CS and now I learn all these algorithms which deal with questions about how to do something efficiently – and these algorithms are unkown by all these people thinking about important questions in this field.

And I think this also applies to other fields. I gave the book "Algorithms To Live By" (which is basically an overview of CS algorithms) to a medicine student and he was immediately inspiried and came up with ideas on how to apply these ideas on his research. CS algorithms are just so basically true that I think they should be more universally known.

<comment><USER10> Slightly off-topic but I wanted to ask why and when did you start studying CS after law.

I recently graduated from law school and now am an intern at a law firm. I have a strong interest for CS, and it bothered me for a long time that I went to law school instead of CS.

I'v overcome those feelings over the years and dedicated myself to become a lawyer. But your post caught my interest.

I'd be glad if you could share some of the story behind you studying cs after getting your law degree.

<comment><USER9> After my law degree I worked for a year in a big international law firm (I didn't yet had my license, so I was a kind of trainee – similar to you position right now). I realised that there is a huge interest in technical solutions to make the work more efficient (often labeled as "Legal Tech"). But there was very little actual understanding of technologie, which I think is one of the reasons why there aren't yet many real world application which are really making a difference. That was when I decided to go back to university for a CS degree.

Some learnings so far: 1) I get great feedback for my decision from other lawyers, who are genereally very interested but not well versed in tech. 2) Legal Tech feels a bit overhyped right now, but eventually it will change the field drastically. Law firms need lawyers who have technical skills. And that doesn't necessarily mean a whole CS degree, some programming skills etc. will already do it.

I personally love tech that much that I don't want to go back to a law firm to practise law, but rather actually develop technology. But for you, if you want to become a lawyer, I can promise you that you will find a fertil ground for your interest. It soon will be one of the most sought after skills for law firms. So if you learn some programming (maybe you already know some), take some online courses (there are great resources for CS online), then the next time your law firm gets offered a (as magic advertised) ML tool or needs to implement a new tech solution which really influcences the workflow, you will be the star of the firm for being a critical but competent colleague. Or if you're starting your own law firm, I think there is great potential for a more automated workflow. In your position, I would be very glad for you CS interest – you in the right field and it is the right time for it!:)

<comment><USER11> If you don't mind me asking - whereabouts (geographically) are you based? I'm hoping to get into law after a couple of years working in technology but I haven't been able to find that much information online about the meeting point of law and technology and my searches haven't found me any communities for law similar to HN for technology.

<comment><USER9> Germany. If you look up Legal Tech meetups (often hosted by law firms) you will find some law students/lawyers interested in tech. But it's still a small community. And for real techies in law, that's an even smaller pool of people.

<comment><USER11> Cool, what you said about going to do another degree gave me the impression you were probably based in Europe! I'm in the UK myself. Yeah I think that's probably the way to go, thanks for the advice.

</comment></comment></comment></comment></comment></comment></comment><comment><USER12> Almost all of the answers on this list are not fields that are "emerging" but fields that "have already emerged".

The ideal emerging field is one that's so obscure we haven't heard of it yet, but so important that we will. If there are widely disseminated books on Amazon about your field, it's not emerging. If there are hundreds of professionals cranking out papers about your field, it's also not emerging.

Emerging fields are underrated and under-recognized. What are they?

<comment><USER13> On the philosophical side, I recently published a paper which could potentially lead to a whole new genre: making actual scientific (=falsifiable) progress on the previously-ineffable question, "Do we live in a simulation?"

"A type of simulation which some experimental evidence suggests we don't live in" [https://philpapers.org/archive/ALEATO-6.pdf](https://philpapers.org/archive/ALEATO-6.pdf)

<comment><USER14> The x - ˆx property is very easy to avoid when building a simulator. Most server-grade computers already use error correcting codes for their memory. Or, the simulator could just abort and restart at a recent checkpoint if an error is detected. It's possible to detect errors with arbitrarily low false- negative rate for a small additional cost of computing and storing checksums.

Nevertheless, it's an interesting observation that we can now easily do experiments that demonstrate correct behavior of logic to the 10^-15 level. If Descartes were looking for evidence of the fallibility of a daemon creating his sense data, it would have been hard to demonstrate better than 10^-3 or 10^-4.

<comment><USER13> You're right of course. Nevertheless there's a difference between saying "the simulating computer probably uses error-correcting codes or something" (speculation) vs. saying "an experiment suggests (same thing)" (science).

To borrow from Nick Bostrom: suppose we run two types of simulations. Important simulations and un-important simulations. For the important sims, we use error-correcting codes, we save checkpoint images, etc. For the unimportant sims, we don't do those things, in order to save money. This allows us to run far more unimportant sims than important sims. Thus, if someone is incarnated randomly in one of the sims, it's probably one of the cheap ones (just because there are more cheap sims than important sims, by basic economics). The point is just to show that it is _possible_ for a philosopher to argue against error-correcting codes etc. Indeed, if we leave it to philosophers, we'll probably never make progress.

We need to appeal to the muse of science, that harsh mistress who serves us cold hard facts, every single one of which throws 50% of philosophers out into the darkness where there is wailing and gnashing of teeth :)

</comment></comment></comment></comment><comment><USER15> Process Mining [1]. When I programmed a rather complex logistics simulator at work I told my coworkers 'whoever comes up with a way of instantiating a simulator from data will be praised forever'. Turned out process discovery is a thing (well, one of _the_ things in PM). And there's so much cool stuff to do and being done. I'm now on the last stretch of my doctorate researching the mining of typical plans in non-competitive environments.

[1] [https://en.wikipedia.org/wiki/Process_mining](https://en.wikipedia.org/wiki/Process_mining)

<comment><USER16> Have you ever had a look into analyzing processes with a graph database like e.g. ArangoDB. Wonder if that would make sense for your needs. You can traverse along the processes, find patterns or use distributed graph processing with Pregel analyze from different angels. edit:typo

<comment><USER17> It is not about the big scale of processes that make process mining interesting. But also the tooling that comes with the field, look for example at the tool Disco: [https://www.youtube.com/watch?v=pmXZQhFSv10](https://www.youtube.com/watch?v=pmXZQhFSv10)

It provides automatic visualisation of graphs, analysing of bottlenecks, and lots of analytics. While you only need system logs linked to an id.

</comment></comment></comment><comment><USER18> AI, machine learning, and neural networks are, of courwe, booming, but I consider them to be hyped.

I consider type theory and formal verification to be more promising (but more academic). Distributed systems and everything having to do with parallel and/or high-performance systems is a good midway between what the industry likes and what's interesting from an academic point of view.

<comment><USER19> Haha.

Formal verification has been around for 40/50 years and we can't say it is a wide success from a industrial point of view. It has some achievements in terms of results/methods and projects checked, but on a daily basis, pretty much no one uses it. We are ages away of having every programmer understanding formal verification and having all programs verified/proved.

Type theory is in a similar situation. Many issues in code could be solved with basic typing algorithms but people and companies favor languages with poor/no typing (python, Javascript).

<comment><USER20> The biggest barrier to adoption of formal verification that I have seen as someone just starting in the field (working through Software Foundations and have a number of projects planned with SPARK, Frama-C, and LiquidHaskell) is the lack of groundwork. Verifying just your own code is complex enough as it is but working with libraries without any clear specification of their interfaces and behaviours makes this so much harder.

I think there is real value in having verified libraries or at least libraries with well defined specs so that interfacing with other code wasn't so tedious. I think this issue is starting to be overcome with regard to the usage of strong type systems. Truly strongly typed languages are finally getting the libraries and communities built up so that they don't seem quite as daunting.

<comment><USER21> To me, personally, the biggest barrier was lack of a proper introduction with a _lot_ of examples.

I try to break this barrier a bit with my upcoming book: Gentle Introduction to Dependent Types with Idris.

I am very interested in this area but it is impossible for newcomers to get a grasp of it without too much digging. Logical Foundations was OK but I was still missing the theoretical explanation ("why does this tactic work? it is magic!").

So with accumulated knowledge from IRC, forums I hope to address this.

<comment><USER20> Ooh I'll check this book out once I get through the pile of stuff I have right now.

And as you noted there definitely is a lot of "magic" when it comes to the inner workings of theorem proving tactics. I'm slowly figuring all of that out but like you said it definitely takes time and digging at the moment.

</comment></comment></comment></comment></comment><comment><USER22> Various field of Deep Learning. Right now - Reinforcement Learning.

See: [https://www.forbes.com/sites/louiscolumbus/2018/01/12/10-cha...](https://www.forbes.com/sites/louiscolumbus/2018/01/12/10-charts- that-will-change-your-perspective-on-artificial-intelligences- growth/#38090e534758) or in general any other marker like NIPS submissions or arXiv preprints on DL.

Of course focus changes, and maybe in the next 2 years it will be on something different than RL. But still, even in Computer Vision it is still a very vibrant field, since its breakthrough in late 2012 ([https://www.eff.org/ai/metrics](https://www.eff.org/ai/metrics)). The majority of more traditional disciplines of CS had their breakthroughs a few decades ago.

</comment><comment><USER23> Homomorphic encryption is a mind-blower. But I fear that we may never see it in it's fullest glory. It's going to be computationally too expensive or too impractical for reason or another. One can still hope.

<comment><USER24> Aren't ring confidential transactions (RingCT), used in some cryptocurrencies a form of homomorphic encryption which is being applied now?

<comment><USER20> Monero uses ring confidential transactions as of now and zCash's zkSNARKs take advantage of some form of homomorphic encryption.

There are probably more but those are the ones off the top of my head.

</comment></comment></comment><comment><USER25> DNA computing might be an interesting new domain [1]. The idea is to use DNA as a memory, while using proteins/RNA as logic operators. This can provide massive speed and efficiency gains, especially for optimisation problems that need parallelization. Just consider that 4bits of information on DNA take only about 1nm^3 of volume, where solid state memory has about 3Tb/in^2 which is roughly equivalent to 10^7 nm^3.

To me it is still not clear how scalable the DNA computing is, but there are nice proofs of concept already [2].

[1] [https://en.wikipedia.org/wiki/DNA_computing](https://en.wikipedia.org/wiki/DNA_computing) [2] [https://www.nature.com/articles/s41586-018-0289-6](https://www.nature.com/articles/s41586-018-0289-6)

<comment><USER26> DNA computing is going to be huge. Nobody is talking about it but a handful of people are slowly pushing it forward.

<comment><USER27> I am willing to bet that both DNA computing and DNA manufacturing (organically 3D print things, but like how organisms grow) will be yuuuuuuge.

Not sure when it will have its internet moment, but the universe has been doing this for a long time and once we unlock its secrets, we become a wee-bit closer to Gods.

</comment></comment></comment><comment><USER28> Algorithmic Game Theory [1]

Going into CS as an undergrad, I didn't anticipate the depth that the field had in other domains -- and for some time, I wanted to double major in {math, biology, economics} to supplement my education.

However, while in the algorithms course, I stumbled upon a connection between linear programming and 2-player zero-sum games (the minimax theorem [2]). Up to that point, I had never considered the idea of using a computational lens to view problems outside of CS, such as "what is the complexity of Nash equilibrium?"

It turns out Algorithmic Game Theory can be applied to study theory of auctions (Why does Ebay use 2nd-priced auctions?) [3], tournament design (Why would a team purposely lose?) [4], or something as basic as routing (Why does building more roads lead to more congestion?).

[1] [https://en.wikipedia.org/wiki/Algorithmic_game_theory](https://en.wikipedia.org/wiki/Algorithmic_game_theory)

[2] [https://en.wikipedia.org/wiki/Minimax_theorem](https://en.wikipedia.org/wiki/Minimax_theorem)

[3] [https://en.wikipedia.org/wiki/Auction_theory](https://en.wikipedia.org/wiki/Auction_theory)

[4] [https://theory.stanford.edu/~tim/f13/l/l1.pdf](https://theory.stanford.edu/~tim/f13/l/l1.pdf)

</comment><comment><USER29> I don't know for sure, but I certainly _hope_ we'll see some fresh thinking about user interface design and construction. The past couple of decades seem to have been substantially about recapitualating what came before in the web browser, and while webification has it's good sides (easier deployment), the actual interfaces for data-entry type tasks still seem as clunky as ever.

AR is potentially an interesting sub-field, but doesn't seem to be the answer for everything (e.g. those form-like data entry tools...)

<comment><USER30> I think UI progress is unlikely without good AI, and good AI has to be much better than human to be passable.

(If you're not convinced, try watching how often you have to ask your fellow humans what they meant by a communication and/or a request for information. It's probably more often than you expect - but you give fellow humans a pass because you're used to it, and so are they.)

Either that, or personal data has to stored in a central server so it can be accessed on demand by web apps - which would eliminate a lot of web forms, but would have uncomfortable political and social implications.

There's still room to improve form-based pages, because there's still far too little research into best practice. But forms are an efficient way to collect information, so it's hard to imagine a secure and private UI paradigm that would eliminate them altogether.

<comment><USER29> _Either that, or personal data has to stored in a central server so it can be accessed on demand by web apps - which would eliminate a lot of web forms, but would have uncomfortable political and social implications._

That doesn't necessarily require centralisation. Web browsers have some form- filling capabilities now, and that data can stay under end-user control. Perhaps there's scope for building on something like this (although the growth of, _e.g._ "social login" doesn't leave me too optimistic. That perhaps _does_ count as an example of UI innovation, although one which hasn't particularly registered with me since I tend to avoid it).

 _There 's still room to improve form-based pages, because there's still far too little research into best practice. But forms are an efficient way to collect information, so it's hard to imagine a secure and private UI paradigm that would eliminate them altogether._

Agreed. I don't see easy wins, but trying to make forms as good as they can be seems a very worthwhile area of endeavour. I suspect part of this might be trying _not_ to go too far in terms of baking "business rule" type stuff into forms, which has a tendency to leave people in impossible states (thinking, for instance, of academic grant systems which can end up with some very strong assumptions about career paths built in)

</comment></comment></comment><comment><USER31> Deep Video Portraits - [https://web.stanford.edu/~zollhoef/papers/SG2018_DeepVideo/p...](https://web.stanford.edu/~zollhoef/papers/SG2018_DeepVideo/page.html)

DeepFake Creation Tools - [https://voat.co/v/DeepFake/2405562](https://voat.co/v/DeepFake/2405562)

Adobe Voco = [https://en.wikipedia.org/wiki/Adobe_Voco](https://en.wikipedia.org/wiki/Adobe_Voco)

</comment><comment><USER32> CRDT's looks interesting and are new enough, that there will be more to learn about them. Seems like an important component in distributed systems (almost all new systems)

[https://en.wikipedia.org/wiki/Conflict- free_replicated_data_...](https://en.wikipedia.org/wiki/Conflict- free_replicated_data_type)

</comment><comment><USER33> Computational law. See: [http://logic.stanford.edu/complaw/complaw.html](http://logic.stanford.edu/complaw/complaw.html)

<comment><USER34> About formalisation of laws in code (read: "write down laws as code"), I wonder how far this can go (eg. in a 10 or 20-years timespan). Any idea what are the most suitable parts of law (notably public) where to apply this principle? And how far can companies/startups go without the help of governments in this venture?

Another note on "smart contracts" in the first sense, or as I would put it, trusted computation as way of executing multi-party agreements. This approach already in application in electronic markets for example, and public blockchains seem to be a way to bring this to the masses. But I think it's still hard to say how this can interplay with "wet" decisions (involving a judge or an arbitrator). That's probably one of the interesting questions in this domain.

</comment></comment><comment><USER35> One interesting approach might be to work backwards from desired practical applications: [https://www.gartner.com/smarterwithgartner/gartner- top-10-st...](https://www.gartner.com/smarterwithgartner/gartner- top-10-strategic-technology-trends-for-2018/)

</comment><comment><USER36> Personally, I think higher order cognition in AI will be hot. Deep learning has monetized the introduction of AI into numerous mainstream domains (e.g. smart NLP search, vision, speech, game RL), which will motivate and underwrite efforts to push AI beyond the shallow hacks of the past, finally breaking through AI's brittleness problem.

Some problem domains are killer apps, like self driving cars, and on smartphones, personal digital assistants and verbal interfaces. There's no stopping these initiatives. They only question is how far each can go without moon shot levels of investment. But between the economic interests of especially Google and Apple to advance their mobile devices, and the military to make weapons and intel as smart as possible, I'm convinced there's enough critical mass for AI's pile to stay hot for a couple of decades or more.

The trick is to avoid the roadblocks that today's academic agenda inflicts on researchers by demanding they publish frequent shallow incremental novelties. What's needed is 5-10 years to develop the infrastructure that's needed to enable the fielding of robust general reasoning w/ causation and rich knowledgebases.

</comment><comment><USER37> Within a year or two, we will see grad-level courses at top CS programs in Chaos Engineering and SRE. Just as we've seen the addition of Distributed Systems classes in the past few years with introductions to Zookeeper, Paxos, BigTable, Raft and Spanner. There will be an explosion in academic work on the science of "failure-injection" methods ;)

<comment><USER38> Supporting evidence: "Chaff Bugs: Deterring Attackers by Making Software Buggier" [https://arxiv.org/abs/1808.00659](https://arxiv.org/abs/1808.00659)

</comment></comment><comment><USER39> There are currently huge opportunities in applied computing for people who can break out of the status quo. There has never been such a big gap between what technology can do and what technology culture can't. Of course it isn't easy. As there also never been easier to waste time in technology.

</comment><comment><USER40> I think anything "emerging" will come from the forms of unconventional computing [1] ... ML/DL/AI are being rehashed on faster silicon hardware, I wouldn't call it hype but it will be better applied to another form of hardware - once it's realized. I personally think reversible computing [2] (once understood) to make the most sense in terms of energy efficiency in CS (much needed) ...

[1] [https://en.wikipedia.org/wiki/Unconventional_computing](https://en.wikipedia.org/wiki/Unconventional_computing)

[2] [https://en.wikipedia.org/wiki/Reversible_computing](https://en.wikipedia.org/wiki/Reversible_computing)

</comment><comment><USER24> Out-of-order execution and caches are once again emerging fields, unfortunately.

<comment><USER41> Emerging as in not leaking through side channels emerging?

<comment><USER24> Yup, that's what I was referring to.

</comment></comment></comment><comment><USER42> Not so much science in this list, more personal and practical view:

As a web developer: WebAssembly

As a DevOp: Kubernetes

As a backend engineer: headless (CMS) API systems like Strapi or Wagtail

<comment><USER43> "Drive mobile and Javascript front ends from wagtail's API"

What is the meaning of headless anyway ?

<comment><USER44> It's essentially a back-end only content management system that makes content accessible via a RESTful API.

According to wikipedia, the term "headless" comes from the concept of chopping the "head" (the front-end, i.e. the website) off the "body" (the back-end, i.e. the content repository).

</comment></comment></comment><comment><USER45> Help me create a field of human programming that's informed by computer science? Below is a simplified description of some ideas informing what I do. These days I'm more focused on language and behaviors in myself and primary relationship in preparation for our first child, so it'd be nice if someone else started working on the theoretical stuff. I'm also down for informally experimenting with things anyone comes up with from this.

Here's the basis:

Start with a category theoretical model connecting neuroanatomy and thought (MENS, category theory and the hippocampus). Combine with the concepts of universal embedding and fully abstract languages. Replace computers in the previous sentence with computational model of human; I'm playing with modified versions of differentiable neural computers and perceptual sets comprised of beliefs, emotions, intentions, and behavior/thought patterns. Choose a human language to mathematically hack into a strict subset of itself so it meets the requirements for a target language in universal embedding. I suspect some form of type theory might be needed for that; coeffects seem like they could be useful, as well as quantitative type theory. Use yourself as the primary experimental subject (ie. the test machine) to help guide things and don't worry about reproducible results...trust that you're an ordinary human with essentially the same cognitive functions as everyone else, for now. Explore how this can impact human relationships. Discover ways to organize the self in such a way as to more effectively organize at scale.

Teach the world how to program itself at an individual level.

<comment><USER46> > Start with a category theoretical model connecting neuroanatomy and thought > (MENS, category theory and the hippocampus).

Yeah... um... let us know when you've got that in a form that is really true to neuroanatomy, really true to human thought, and really solid category theory. I'm pretty sure you're not going to get there in this lifetime.

</comment></comment><comment><USER47> Only because I don't yet see it mentioned, the one emerging field to rule them all: program synthesis. :P

<comment><USER48> Here's sort of a relevant critique to that idea: [http://www.commitstrip.com/en/2016/08/25/a-very- comprehensiv...](http://www.commitstrip.com/en/2016/08/25/a-very- comprehensive-and-precise-spec/)

Not saying that there isn't merit to the idea. Just saying that program synthesis is more or less synonymous with programming language design when you take into account the challenges involved.

</comment></comment><comment><USER49> Zero knowledge proofs.

Secure execution environments.

</comment><comment><USER50> Graphical models[0] & probabilistic programming[1], with the latter making it easier for developers to dive into this growing AI trend. Research in the field for the past decade has been steadily booming with more companies like Microsoft leading the way. I recommend checking out some MOOCs[2] in coursera.

[0][http://www.computervisionblog.com/2015/04/deep-learning- vs-p...](http://www.computervisionblog.com/2015/04/deep-learning-vs- probabilistic.html)

[1][http://probabilistic-programming.org/wiki/Home](http://probabilistic- programming.org/wiki/Home)

[2][https://www.coursera.org/specializations/probabilistic- graph...](https://www.coursera.org/specializations/probabilistic-graphical- models)

</comment><comment><USER51> An informal HN survey on what some feel is the future vs. over-hyped: [https://news.ycombinator.com/item?id=17129481](https://news.ycombinator.com/item?id=17129481)

</comment><comment><USER52> Functional correctness, formal verification and automated bug fixing.

</comment><comment><USER53> Since you mentioned movements in free software, "open core" has been emerging in the last 4 years as a viable way to do business while allowing other individuals and companies to build onto your core platform while still being able to monetize your work. For example, if you launch a startup specializing in building foo, you can maintain a library called libfoo and sell a larger foo application or foo plugins or foo services using the open-source library you created.

</comment><comment><USER54> Open source computing hardware. RISC V[0]

[0] [https://riscv.org/risc-v-foundation/](https://riscv.org/risc-v- foundation/)

</comment><comment><USER55> Low power sensor and associated networks.

Its hard to do and has lots of real world applications.

</comment><comment><USER41> I am _certain_ there will be an emerging field in AI for engineering. Suspension that 'learns' how to keep the car flat; buildings that start shuffling warm air from a to b before it's needed ... things like that. Programming is going to change from "explain how to do it" to "show it what you want", and this has got to be a big deal.

<comment><USER56> [https://en.wikipedia.org/wiki/Self- levelling_suspension](https://en.wikipedia.org/wiki/Self-levelling_suspension)

[https://www.youtube.com/watch?v=eSi6J-QK1lw](https://www.youtube.com/watch?v=eSi6J-QK1lw)

You don't need AI to keep your car's body level. The technology has been around in various forms for over 60 years.

So much of what people believe we need AI for is amenable to classical engineering techniques.

<comment><USER41> Indeed, couldn't agree more. But quite possibly AI will prove simpler than classical techniques, and more adaptable to i.e. changes in tyre pressure.

</comment></comment></comment><comment><USER57> Fully Homomorphic Encryption

</comment><comment><USER58> Amorphous computing has always seemed interesting to me. Computing with emergent phenomenon amongst scatterings of large numbers of unreliable simple processors - like the sort of thing you could mix into paint. It's very young, with lots of fundamentals to be worked out, but that's what makes it interesting!

</comment><comment><USER59> Wide-scale adoption and promotion of open source software in the industry (e.g. Microsoft, Facebook, Google)

</comment><comment><USER60> Adversarial Machine Learning. Fake news detection using ML. Integrating 'good old fashioned AI' ideas with modern ML techniques - to some extent Alpha Go went in this direction. While I am glad AI has moved far more towards the machine learning direction, i suspect the decades of AI research that preceded it may come back in a form that is combined with more modern techniques in some way. I see Alpha Go (and Alpha Zero) as steps in that direction. Also, applying deep learning to search engines and making that scale efficiently. I suspect google has partly solved this already, but haven't gone public with any of the tech, but that's pure speculation.

<comment><USER60> I meant those as 4 separate areas. I don't think my post makes that clear.

</comment></comment><comment><USER61> Quantum computing?

<comment><USER62> Yes. Once we build a scale-able quantum computer it will revolutionize so many fields. Simulating chemistry suddenly would become practical. We could broaden our understanding of biochemistry and materials science without designing fickle experiments, just by simulating things. This would be a real game changer and will probably lead to a bunch of breakthroughs on the way to protein-based nanotechnology.

</comment></comment><comment><USER63> Swarm Computing. The hardware and networking to make it practical and useful exist now, but the field is still in its infancy. There's some discussion about its use in autonomous driving, construction, warfare.

<comment><USER64> What makes swarm computing different from distributed computing?

<comment><USER63> Swarm computing is about moving, cooperating devices with sensors, possibly AI features. Distributed computing is just a minor aspect of it.

</comment></comment></comment><comment><USER65> Just as an angle to answering the question (I don't have answers of my own)....

What was the most interesting CS field(s) in 2008, 98, 88, etc?

<comment><USER66> 88 (taking a stab here as I was pretty young):

DTP

OO

RISC CPUs

'graphics' (as in render farms)

98:

Linux, Apache, Mozilla, OSS in general

Perceptual audio compression : MP3 (layer3.org, MP3 vs TwinVQ, codecs created during the period before Fraunhofer announced the source code it uploaded to ISO without a license and that people had been working on for free, in fact had a license and everyone owed them 10 grand).

2008:

Cloud

mobile (location in particular). Think Foursquare vs Gowalla vs Burbn, Grindr, other early mobile location-aware apps. App stores for popularised by Apple that same year.

AJAX, Rails

blogging.

<comment><USER67> 88

OSI network stacks. They were going to replace the 'old' Internet protocols.

Relational Databases, SQL and two phase commit.

Formal methods and verification.

<comment><USER68> > Formal methods and verification.

Still emerging.

</comment></comment></comment></comment><comment><USER69> Computer Vision.

<comment><USER35> Emerging for 50 years, and still going strong

<comment><USER41> Yes, but it works now.

</comment></comment></comment><comment><USER70> Functional programming!

Functional programming languages have several classic features that are now gradually adopted by none FP languages.

Lambda expressions [1] is one such feature originating from FP languages such as Standard ML (1984) or Haskell (1990) that is now implemented in C#3.0 (2007), C++11 (2011), Java8 (2014) and even JavaScript (ECMAScript 6, 2015).

Pattern matching [2] is another feature that is now^2015 implemented in C#7.0. My bet is that Java and other will follow in the next versions.

Here is a list of FP features. Some of which are already adopted by none FP languages: Lambda expressions, Higher order functions, Pattern matching, Currying, List comprehension, Lazy evaluation, Type classes, Monads, No side effects, Tail recursion, Generalized algebraic datatypes, Type polymorphism, Higher kinded types, First class citicens, Immutable variables.

[1] [https://en.wikipedia.org/wiki/Lambda_calculus](https://en.wikipedia.org/wiki/Lambda_calculus)

[2] [https://en.wikipedia.org/wiki/Pattern_matching](https://en.wikipedia.org/wiki/Pattern_matching)

</comment><comment><USER71> The use of Lattice Boltzmann Equations to parelleize computation. It's alreadg being used in doing dynamic fluid simulations but it's applications are pretty endless. I wouldn't be surprised to see LBM translated for use in Machine Learning, Cognitive AI, NLP, Computational Biology, etc, etc.

</comment><comment><USER72> Programatic identification and comprehension of morality in software applications is a fascinating area!

</comment><comment><USER73> I think the core disciplines are the same, sometimes just some "updates" are happening. Like AI or CV in the last years.

The big changes are happening in engineering (software and hardware).

Many things that were known for decades are now accessible for a broader audience.

</comment><comment><USER74> Quantum computing and cryptography

</comment><comment><USER75> Human-computer interaction. The field is not new but the way people interact with computers has drastically changed in the last ten years and will probably continue to do so.

<comment><USER76> And within that, Distributed/shared UI.

</comment></comment><comment><USER77> The hype is where the money is and if you look at the established emerging tech, AI and IoT are projected to get most funding and create most disruption in the years to come [https://uk.pcmag.com/feature/94662/blockchain-and- robots-buz...](https://uk.pcmag.com/feature/94662/blockchain-and-robots-buzzy- but-not-yet-vc-blockbusters)

The cutting-edge emerging tech I feel will be in the way we engage with data and tech and Augmented Intelligence (assistive) will see huge advancements.

</comment><comment><USER78> Internet of Things. It is still in development and there are lots of stuff to work on.

<comment><USER79> Just another hype. Sensors and internet are here for decades. Internet is ok, but "things" are way too expensive and not reliable yet.

<comment><USER80> IMO the best IoT is the DIY IoT, the kind that isn't really IoT but rather "I put Wifi on a raspi and connected it to a PCB".

Thankfully the online resources around electronics are plentiful and PCBs can be had for under 10$ incl. S&H.

That way I can make all my lighting IoT without having to deal with the garbage of the IoT industry.

<comment><USER79> Yes, IoT is great when you can DIY. But when you need to send a technician 2-4 times a year for each node... it's a problem, not a business.

<comment><USER81> It's a business for repair technicians :)

</comment></comment></comment></comment></comment><comment><USER82> direct acyclic graphs for blockchain use. homomorphic cryptography.

</comment><comment><USER83> Targeted Advertising

</comment><comment><USER84> Quantum Computing

</comment><comment><USER85> Computer Networks

<comment><USER86> This Internet thing is going to be huge-- but it's going to tear us apart. Mark my words.

</comment></comment><comment><USER87> Blockchain, though no one on hn really understands it. :D

<comment><USER88> It's an unfortunate characteristic of many in the cryptocurrency community, that they think anyone who doesn't support crypto simply doesn't understand it. And by extension, as soon as they do understand it they will become supporters.

No. There are those who do understand blockchain and still don't support it. A great example is professor Jorge Stolfi. He is one of the more prominent detractors, and yet he routinely displays a very thorough understanding of the technology.

<comment><USER89> I tried looking up the arguments of Jorge Stolfi, however my Spanish is insufficient.

I don't claim to contradict that he is against the concept of cryptocurrencies or blockchain in general, but I fail to find evidence that he is against the technology in general.

I do find evidence he is opposed to Bitcoin in specific, or at least warns against it.

Could you point me to English writings where he argues against blockchain/cryptocurrency in general?

<comment><USER90> Perhaps the reason why your Spanish was insufficient for his writings is that his writings are in Portuguese ;)

Here's his primary English writing on Bitcoin and cryptocurrency in general (not necessarily blockchain), sent to the SEC: [https://www.sec.gov/comments/sr- batsbzx-2016-30/batsbzx20163...](https://www.sec.gov/comments/sr- batsbzx-2016-30/batsbzx201630-2.htm)

<comment><USER89> Thanks for pointing out it's Portugese!

This critique of Bitcoin is quite short, and seems directed at Bitcoin in particular, in no way do I conclude that he is against the concepts of blockchain (say non-currency), or perhaps even cryptocurrencies that do not take on some of the Ponzi aspects.

After reading this I can perfectly imagine (but do not claim so) that he might support certain other forms of blockchains and/or cryptocurrency...

<comment><USER91> This is his reddit account:

[https://www.reddit.com/user/jstolfi](https://www.reddit.com/user/jstolfi)

He primarily posts in buttcoin, a sub that exists to mock bitcoiners. It's pretty fair to say he thinks all coins are crap, not just bitcoin.

Also, please include me in the "Understands cryptocurrencies and yet doesn't support it" bucket please :)</comment></comment></comment></comment></comment></comment>


You Can't Trust Network Time - <AUTHOR> https://diode.io/burning-platform-pki/2019/04/26/why-we-cant-trust-network-time.html <comment><USER2> DHCP already has support for providing an NTP server IP; there has been at least one proposal[1] to also provide a DHCP option specifying the "rough current time", so that devices have at least _something_ to start with. As for trust, you're already going to be trusting DHCP to give you an IP address, you could also trust it for this (with some safeguard in place to catch known- wrong values).

This kindof reinforces how having an 'IoT Gateway' would be a good idea: The gateway can provide DHCP, and rough time, and (S)NTP, and DNS, and can also act as a proxy to the outside.

[1]: [http://www.potaroo.net/ietf/idref/draft-ogud-dhc-udp-time- op...](http://www.potaroo.net/ietf/idref/draft-ogud-dhc-udp-time-option/)

<comment><USER3> > This kindof reinforces how having an 'IoT Gateway' would be a good idea: The > gateway can provide DHCP, and rough time, and (S)NTP, and DNS

Aka a router. Am I missing something?

<comment><USER4> I doubt most routers provide DHCP. Maybe consumer ones.

<comment><USER5> What's the point of this nitpick?

Networks that configure everything statically are fairly rare, and do so intentionally. If they don't, they have a DHCP server somewhere already, and if they aren't consumer-level networks they sure as hell don't want some random "IoT gateway" box providing DHCP. So it's clear what "router" means in this context.

<comment><USER4> I have a separate dhcp server so if the router has issues machines can still attach to the network and talk to local services.

</comment></comment></comment></comment></comment><comment><USER6> Articles that go around in circles while avoiding reflection or synthesis are infuriating. Of course relying on the PKI isn't going to magically solve every problem. But the article wasn't titled "Why you can't trust the PKI", because we're all bored of reading about that.

At a certain point, you _must_ choose some _immutable_ trust root. If your kink is to use public CAs (fuck me harder, (go)daddy), it implies attacks. If you chose to setup your own private signing key, it implies attacks. If you choose longest sha256 chain ("blockchain"), it implies attacks. If you simply stop trying to retain absolute control over devices after they're owned by someone else, it implies attacks.

You can hardcode further constraints as mitigations, but the fundamental drawbacks remain. If you want direct synthesis from the immediate complaint, keep a bit of a state to ensure that the date only ever moves monotonically forward. However you'll still be open to an attack if the device is off while a certificate is compromised.

</comment><comment><USER7> This is a genuine problem across a variety of platforms, not just IoT. There really is no great way to "solve" for trust when one party might have extremely stale information. Awareness of this problem is surprisingly poor.

> In the next blog post of this series we want to introduce a new secure > network time protocol based on blockchain technology and show how even the > smallest devices can establish completely secure connections.

This is a pretty disappointing way to end the article. Most "solutions" end up being some form of obfuscation, I'd guess this is too.

<comment><USER8> A verifiable way to put bounds on time is a very useful thing. If you know the time within an hour or even within a day can be a big help. There are cases where I've used firmware build time as a lower bound for current time.

</comment></comment><comment><USER9> There is only a catch-22 if we need _that time source_ to verify its time.

That's not how things work, though: we have hundreds available, so there is no catch-22 whatsoever: we check multiple sources and go with the majority vote, then flag an error for any outlier(s).

<comment><USER10> If your wifi connection is MITMed, your adversary could MITM all the connections, even if you make hundreds.

If you have trouble imagining a user taking their wifi lightbulb to a coffee shop, imagine instead you're a device vendor and a customer is trying to MITM their own device to embarrass you in a Defcon presentation...

<comment><USER9> Sure, but now you've turned it from "You can't trust network time" into "you can't trust your wifi" and while 100% correct: now you're commenting on a different article altogether.

</comment></comment></comment><comment><USER11> If we can't get precise time securely and do not have real-time clock, it does not mean, there are no options left.

For instance, device may be booted for the first time in secure controlled environment and synchronize time fro trusted source.

Then, every N seconds (like 60 to 3600), device will write current time to local file. Most devices have some kind of writable local storage.

Any next synchronization will not be able to return time before already experienced moment. Certificate expiration will actually work, with a bit higher tolerance of a few hours.

</comment><comment><USER12> Adam Langley's Roughtime protocol[1] largely solves some (but not all types) of time malfeasance, and a variant of it is already an IETF draft specification[2]. There is also NTS, which is secure NTP and provides some protections, but not nearly as many.

[1]: [https://roughtime.googlesource.com/roughtime/](https://roughtime.googlesource.com/roughtime/)

[2]: [https://tools.ietf.org/html/draft-roughtime- aanchal-01](https://tools.ietf.org/html/draft-roughtime-aanchal-01)

</comment><comment><USER13> An (interesting, but impractical) solution to this problem would be to use astronomy to deduce the current date first, which should be good enough to validate the certificate.

See this QA on Astronomy StackExchange: [https://astronomy.stackexchange.com/questions/12886/get- date...](https://astronomy.stackexchange.com/questions/12886/get-date-and- time-by-position-of-the-sun-and-the-observer-position)

</comment><comment><USER14> One could pin a cert, hope the device is powered on in the validity period to update it? Obviously a bad idea for other reasons but maybe it's the best worst choice.

</comment><comment> yjftsjthsd-h Maybe dumb, but... Why not just make ntpd actually check certificate revocation? Easier than fixing everything that uses HTTPS, and seems to fix it?

<comment><USER15> 1\. The device doesn't have a battery, so when it boots it thinks the time is Jan 1, 1970. X.509 wasn't even invented at that point so there are no valid certificates for that date.

2\. The device is being MITMed so it receives a time of, say 2010 with a certificate that is also valid for 2010 even though that certificate is long expired in reality. The device has no way to know otherwise.

I don't know the point of this attack though.

<comment><USER16> >1\. The device doesn't have a battery, so when it boots it thinks the time is Jan 1, 1970. X.509 wasn't even invented at that point so there are no valid certificates for that date.

Defaulting to date of manufacture instead of unix timestamp 0 would get you closer, although it's still not a solution.

You could also periodically store time in a non volatile memory, to be loaded on boot.

<comment><USER17> OpenWrt can use the last modified timestamp of a file, eg the most recently updated log file. It's a decent middle ground.

</comment></comment></comment></comment><comment><USER18> Assuming something in the trusted side of the network chain has a persistent clock or verified time then this issue could be avoided. For things that connect directly to untrusted networks e.g. public cellular internet the only solution I can see is using devices with a persistent clock themselves.

</comment><comment><USER19> IETF working group - [https://datatracker.ietf.org/doc/draft-ietf-ntp-network- time...](https://datatracker.ietf.org/doc/draft-ietf-ntp-network-time- security/)

</comment><comment><USER20> So ignore the time on the cert? Isn't that better than having no cert?

<comment><USER21> The reason I think they're effectively equivalent is say I have an IOT device that it's command and control is on example.com.

When using NTP and having stolen a certificate in the IOT devices trust store for example.com. I spoof the time and then I can spoof example.com because the devices clock is now in the validity time for the stolen certificate.

When using server time over TLS and having stolen a certificate, the device on cold boot (Or all the time depending on how well written it's implemented) will connect to the server ignoring the validity time, and again get it's clock set to a time that matches the certificate.

Either of these methods require compromising the private key for the cert. Basically the weakness is using time to limit the window for a compromised cert is a lot harder on IOT devices which routinely don't maintain their clocks.

Now, if the TLS based time service was running under a different cert, it might change the surface area a bit. Since the cert for the TLS based time service would also need to be breached, along with the example.com certificate (2 certs in theory should be harder to breach, but it depends on the org and how they leaked the secret in the first place).

This also depends on the IOT device manufacturer not signing a certificate for 25 years that will basically never expire regardless of the time (which I've seen).

</comment></comment><comment><USER22> I'm probably the only one who thinks it's really stupid to connect things like refrigerators and washing machines to the internet, but I'm going to say it anyway.

<comment><USER23> And it's even stupider to give the user zero local tools to interact with them short of letting them loose on the big bad internet.

I wouldn't mind if my washing machine could ding my phone when there's a load of damp clothes in there that I need to handle before they grow mold. That's cool, I'm forgetful, and I waste a lot of water being that forgetful. But it shouldn't need to talk to the internet to do that.

The problem is not smart appliances, per se, it's the centralized and vulnerable way those smarts are being implemented. Because it's in the manufacturer's interest to compromise my own security and autonomy.

Which is why I don't own any, and I'm sitting here working on my own washing machine door transmitter instead.

<comment><USER8> Mine has an LED that indicates "clean" and slaves from the internal door switch. Handy source.

<comment><USER24> This could be eg. an SNMP metric/trap, along with other sensors in the machine, then parent could add the "smartness" via their own software.</comment></comment></comment></comment>


Ask HN: Have you given up your dream for your dream job? - <AUTHOR>

I had an interview a few hours ago. They basically offered me my dream job and a possible partnership after a year. And I was planning to quit my current job at the end of next year to start my own business.<p>Have you been in a similar situation? Did you choose your dream or your dream job? Do you have any regrets? <comment><USER2> Actually, I just came from the same situation.

I applied for a job at company X, at Facebook, at Google, and I also applied to YC. After interviewing at company X, I thought this is the best job ever, I would love to do this.

I then got YC, so cancelled the Google and Facebook interviews. I told company X, who took it well. So well they offered me the job anyway---to take in the likely event of my startup failing. They also started my visa paperwork---an $8-15K expense---and said they'd just absorb the loss if my company worked out.

There isn't really a point to this, except that maybe you can have your cake and eat it. Also, if you find an amazing company, maybe they'll prove to be even more amazing than you thought.

<comment><USER3> Obviously you don't want to reveal what company X is, but just out of curiosity, how big a company is it? Taking on the visa process for a "possible" hire seems like something only a large-ish company could afford to do.

<comment><USER2> They're a technology company with 50-100 technical employees. As far as I know, they're rolling in money.

</comment></comment></comment><comment><USER4> "A good plan, violently executed now, is better than a perfect plan next week."

I haven't been in this situation before but I think what George Patton said applies.

</comment><comment><USER5> If it is your dream job why don't you accept the offer? If you don't like it, you can still start your own business after a year or later (I don't know how old you are).

If you like it, you can also start you own business, maybe a bit later, but with enough experience and an established partnership behind your back, if the two is not competing with each other.

</comment><comment><USER6> Take it, and throw yourself into it for a year. Your business was a year away anyway, so it's worth discovering if it's your dream job. If it is your dream job, and the partnership materializes, go for it.

On the other hand, you may discover that your dream job isn't what you really thought it was. Then you can move on starting your own business with greater commitment, either full time or on the side.

You should also evaluate your reasons for wanting to start your own business. Being a partner in a business that you love could actually fulfill all or most of those goals, without some of the risks and downsides of entrepreneurship.

</comment><comment><USER7> I'm relativity young to give up on my dream job just yet, but I'm having some serious doubts I'll ever get it.

That being said, if it's your "dream job", and they can provide you with _some_ safety, I say go for it. Use your free time to work on your own business.

If you ever get to a point where keeping the two is too much work, that will be a good sign, and you can choose between the two.

</comment><comment><USER8> maybe i'm mis-reading, but isn't your dream job, by definition, one of your dreams? so shouldn't you always take your dream job no matter what?

could you give an example of a dream that you cannot pursue because you were taking your dream job? it seems like any reasonable non-work-related dream can be pursued _while_ you are working at your dream job. or maybe not, i dunno, i'm confused.

</comment><comment><USER9> I'm not even sure I know what my dream job is yet. Right now I'm drifting between eventually (or soon) doing a startup and something totally unrelated.

I'm curious, at what age did other HNers feel that they had figured out what they wanted to do job-wise (even if they weren't doing it yet)?

</comment><comment><USER10> I chose following my dream. To me, it was 100% worth it.

"Happy those who dream dreams and are whiling to pay the price to make them come true." (From an old skydiving movie)

</comment><comment><USER11> Can you pursue your dream job on the side, on weekends?

Also, what kind of a place interviews on Christmas day?

<comment><AUTHOR> I wasn't asking for advice, just wondering if any HNers had been in a similar situation.

We don't have the Christmas holiday (I'm Turkish) although we have other holidays to more than make up for it.

</comment></comment><comment><USER12> Temptation certainly seems to creep up at the most inconvenient moments.

</comment><comment><USER13> I think the answer to a conundrum like this is very personal and tells you a lot about what you value more.</comment>


Rust 1.39 - <AUTHOR> https://blog.rust-lang.org/2019/11/07/Rust-1.39.0.html <comment><USER2> An important bugfix in 1.39.0 that wasn't mentioned in the release announcement is probably: [https://github.com/rust- lang/rust/pull/63402](https://github.com/rust-lang/rust/pull/63402)

It's important because unless rustfmt is available, bindgen would put everything onto one line: [https://github.com/rust-lang/rust- bindgen/issues/1600](https://github.com/rust-lang/rust-bindgen/issues/1600)

Rustc would generate (unrelated) warnings for the generated code but before the bugfix it'd print the entire line, hitting the maximum log size limit on CI and causing the build to fail because of that.

<comment><AUTHOR> Another fix (one mentioned in detailed release notes) is where asinh(-0.0) used to return 0.0

These unexpected behaviours are tough to nail down when one is trying to get the code/setup working. Really appreciate the time these fixes potentially save for future users. The complex nature probably also hints at maturity of Rust codebase.

</comment></comment><comment><USER3> >

        #[cfg(windows)] slice: &[u16]

Am I the only one who finds this syntax ugly - to the point of being inappropriate for a general-purpose programming language?

<comment><USER4> As someone who uses rust a fair amount, I can say that the syntax looks pretty normal to me. It's saying "only define this field if we're compiling on windows." I parsed that out immediately.

Maybe it just takes a bit of familiarity with the language, but everything does fit together really well after the initial learning phases.

</comment></comment><comment><USER5> Discussion here: [https://news.ycombinator.com/item?id=21473259](https://news.ycombinator.com/item?id=21473259)

<comment><USER6> They're two different things; this post is the general release announcement, that post is a deep dive on one specific feature.

That said, only one or the other should probably be on HN; it's a tough call as to which. This post is more general, but people are going to care about async/await more than anything else...

<comment><AUTHOR> Completely agree. That ways discussion would be at one place and front page won't be hogged with possible duplicates.

Right now the other post is enjoying more attention and has meaningful engaging discussion. Requesting @dang to remove this (21473257) from front page and possibly tag the other one (21473259) with 1.39.0 release.

</comment></comment></comment><comment><USER7> Time to give this lang a try. Been wanting to build a web assembly large data table mechanism for a while, and rust appears to be an idea candidate from what i read (speed and reliability). Although it could be done using SQLite, i think could be a nice little project for learning Rust.

<comment><USER8> It's a pleasant suprise how easy targeting wasm is from Rust. Just pass on the right `--target` flag to `cargo build` if you have the toolchain installed.

<comment><USER9> And installing the right target is typically as simple as:

        rustup target add wasm32-unknown-unknown

Targeting browser APIs is a little more involved, generally requiring either:

        cargo install cargo-web
        cargo web build [...]

for stdweb, or for web-sys:

        cargo install wasm-pack
        wasm-pack build [...]</comment></comment></comment>


Intro to Distributed Hash Tables - <AUTHOR> http://www.freedomlayer.org/articles/dht_intro.html <comment><USER2> Might as well learn an important concept from the original authors (the article does link to this paper): Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications [1]

[1] [http://pdos.csail.mit.edu/papers/chord:sigcomm01/chord_sigco...](http://pdos.csail.mit.edu/papers/chord:sigcomm01/chord_sigcomm.pdf)

<comment><USER3> Robert Morris? THE Robert Morris?

<comment><USER4> He's cofounder of YC [https://news.ycombinator.com/user?id=rtm](https://news.ycombinator.com/user?id=rtm)

Read a book that had a section on him (and pg) written before they founded Viaweb. [http://www.amazon.com/CYBERPUNK-Outlaws-Hackers-Computer- Fro...](http://www.amazon.com/CYBERPUNK-Outlaws-Hackers-Computer- Frontier/dp/0684818620)

Also, there are two Robert Morris, rtm, and his dad who also has a wikipedia article [http://en.wikipedia.org/wiki/Robert_Morris_%28cryptographer%...](http://en.wikipedia.org/wiki/Robert_Morris_%28cryptographer%29)

</comment></comment></comment><comment><USER5> A DHT is a fairly easy to understand technology that can give you a lot of insight into distributed systems. In school we had to build a crawl/index cluster of 8 machines and we used Chord to split the load. I had no experience with multi-machine computing but it made sense immediately at the time to split the crawling load by creating a DHT over the URL keyspace. Of course this doesn't balance actual load but it's a nice approximation to get you off the ground.

For someone who has never tried to do any Distributed Computing, a DHT can solve many simple scaling problems in a way that is fairly easy to reason about on paper.

<comment><USER6> > this doesn't balance actual load

As someone who doesn't have much experience with distributed computing, this part confused me a little. What do you mean by that?

<comment><USER7> I believe poster was trying to say that load balancing is probabilistic rather than deterministic: Load is levelled across the DHT only if the hashing function maps keys evenly over the nodes (assuming each key has an equal amount of work associated with it).

<comment><USER8> I think the main concern is that in many systems each key doesn't have an equal amount of work associated with it (this sort of thing is usually referred to as a "hotspot").

An example: suppose you have some distributed system storing article metadata and all of the sudden one of your articles becomes very widely shared. The machine that the popular key hashes to gets slammed. Perhaps we'd want to adjust it so that that particular machine is just dedicated to that one article, or some other way to distribute that one article across multiple machines. But we're just using a hash function, so without doing something fancier, we can run into problems when the load suddenly becomes wildly uneven.

<comment><USER7> Solving for read hot-spots is not difficult if you're willing to accept a small read penalty:

Your typical Kademlia DHT has k-replicas of each piece of data, so you read _near_ the target node (node closest to the target key) rather than directly from it. This way, nodes at different points in the network read from many different replicas.

Of course, this depends on your consistency requirements.

</comment></comment></comment></comment></comment><comment><AUTHOR> Hi. OP here. The article is part of the Freedom Layer project, where we describe the research of creating a scalable and secure mesh network. (It is currently still an open problem, at least by what we know).

The first things we have decided to do is to document known stuff. Then we move on to documenting our research. We will try to make it as down to earth as possible, though later articles might get more complicated. Some new results are also going to be discussed.

<comment><USER9> Scalable secure mesh networks might be the cold fusion of computing: possibly impossible, but with great consequences if it can ever work. I'd be interested to follow your research, especially if it's as easy to follow as this paper. Is there somewhere where you aggregate your research?

<comment><AUTHOR> I agree with you :) The stuff that I know (Hopefully everything that is relevant) will show up at the Freedom Layer website project. It will probably take me a month or two to write everything.

I might add some kind of RSS or a mailing list to make it easier to follow.

<comment><USER10> Looking forward to joining that mailing list :)

</comment></comment></comment></comment><comment><USER11> If someone wanted to deploy a DHT today what protocol and/or library would they use?

<comment><USER12> Kademlia. Certainly the most used (it's the default for Bittorrent) and, from a quick reading of other protocols, also the simplest. Moreover it has the advantage of naturally not depending on other peers behaving correctly (some other rely on peers forwarding search and coming back eventually when they have a result)

There's a working implementation over here [0] in Go, and there will most certainly be something for other language. Search for "DHT Kademlia Mainline" (mainline is the name of the network used by bittorrent peers as described here: [1])

[0] [https://github.com/nictuku/dht](https://github.com/nictuku/dht)

[1] [http://www.bittorrent.org/beps/bep_0005.html](http://www.bittorrent.org/beps/bep_0005.html)

<comment><USER13> While I'm inclined to agree that Kademlia is the frontrunner for DHTs, I'd like to comment on:

> it has the advantage of naturally not depending on other peers behaving > correctly

This is, depending on the variety of pedantic hat you put on, not true. While Kademlia is resistant to some simple malfunctions, it can be attacked by an adversary. Or, to put it formally, Kademlia is not Byzantine Fault Tolerant [2].

It can be strengthened, but at the cost of greater complexity. See [3] for an example. Further, it's arguable that zero-trust systems where anyone is allowed to join can _never_ be BFT because of Sybil attacks [4]. The paper in [3] elaborates on how to mitigate this variety of attack, but it's unlikely that Sybil attacks can ever be "solved".

1\. "Attacking the kad network" [http://www.p2p.tu- darmstadt.de/fileadmin/user_upload/Group_P...](http://www.p2p.tu- darmstadt.de/fileadmin/user_upload/Group_P2P/lecWS12/a23-wang.pdf)

2\. [http://en.wikipedia.org/wiki/Byzantine_fault_tolerance](http://en.wikipedia.org/wiki/Byzantine_fault_tolerance)

3\. "S/Kademlia: A Practicable Approach Towards Secure Key-Based Routing" [http://doc.tm.uka.de/SKademlia_2007.pdf](http://doc.tm.uka.de/SKademlia_2007.pdf)

4\. [http://en.wikipedia.org/wiki/Sybil_attack](http://en.wikipedia.org/wiki/Sybil_attack)

<comment><USER12> Of course, a global attacker can always muck with the system. I was talking about single independent peers behaving badly on their little scale, such as returning wrong or no information, which I believe is more likely to happen.

Also, the guys at Bittorrent are pushing for a limitation on how you can form your own ID based on your external IP [0], which should make it much more costly to perform a large-scale attack.

Thank you for being pedantic :)

[0] [http://blog.libtorrent.org/2012/12/dht- security/](http://blog.libtorrent.org/2012/12/dht-security/)

</comment></comment></comment></comment><comment><USER14> A related question I have for some time: Can you hijack/join an existing DHT? For example, use the Bittorrent DHT (which is large and bootstrapped) to implement this "phone list"? Would that be parasitic or symbiotic?

<comment><USER15> Joining is easy. Not sure exactly what you mean by hijack.

The mainline bittorrent DHT only holds values IP:Port, so that's just a few bytes. But it would be quite handy for instance for anyone that needs peers to discover each other. Not so handy for using it as an arbitrary key store or distributed filesystem.

I'd say if you are a full member of the DHT (you correctly handle other nodes queries ) that you would have a symbiotic relationship with it. If you don't maintain your own routing table and ignore other peers requests you'd have a parasitic relationship with the mainline DHT.

So you could be a malicious peer, but it wouldn't have a noticeable effect on the DHT unless you ran quite a few of them. Even then most clients just use the DHT as one of several peer discovery methods.

In fact for any given hash you post to the DHT you'll find peers claiming to be members of that torrent, even if said torrent doesn't exist.

<comment><USER16> Actually the bittorrent DHT has been extended to support storing arbitrary data[1]. This extension is supported by uTorrent and libtorrent-rasterbar which constitute a majority of the bittorrent DHT nodes.

[1] [http://www.libtorrent.org/dht_store.html](http://www.libtorrent.org/dht_store.html)

<comment><USER17> Up to 1000 bytes of signed data, actually. Pretty neat IMHO

</comment></comment></comment></comment><comment><USER18> What is the accepted identifier for nodes? I did a rough implementation of Chord once using IP address as the identifier and ran into the issue where the node would see itself at a different IP than others. The obvious solution is just "don't use it through NAT" I'm guessing.

<comment><AUTHOR> It depends. If you own all the computers, you could just pick random identifiers (Of some large space). You could also choose the identifiers to be a public key, or a hash on a public key. Then when talking to a node claiming that he has a certain ID, you could ask him to prove it.

Even when using the public key as an ID idea, it is still possible to get IDs as close as one wants to a particular number. (It is a bit harder computationally, but still possible).

There are more secure things you can do. Shortly, you would prefer the network itself to choose the ID for a new node, and not let the new node choose it itself. It appears that even this will not give you a secure enough solution. I will write about it in future articles.

<comment><USER10> hey!

First of all, awesome explanation, thanks!

What I didn't understand from your explanation (I don't know if it's explained or if I missed it) is how you identify which node has the value for the key K when the keys are not in the Bs domain. How do you know where to stop the search? What if you have gone to a further node?

I'd be very grateful if you can clarify that :)

</comment></comment></comment><comment><USER19> I get the hunch that skip lists are intimately related to graph search algorithms. This is like A* search where the heuristic is to pick the highest ID that doesn't exceed the target.

</comment><comment><USER20> Can we get a RSS for that blog? I really want to follow it.

</comment><comment><USER21> This doesn't explain what is to me the hard part: namely coping with additions and removals of peers, especially unannounced removals of peers.

<comment><AUTHOR> I was going to write more articles explaining DHTs in more detail, but I can at least write here a short answer.

My general belief about dealing with nodes leaving the network is as follows: You just notice at some point that a node doesn't respond, and then you look for a better one. That is because a node leaving the network could be caused by a failure or a user that kills the program. A node leaving the network is an interesting time, because the node doesn't have any incentive to behave well.

If a node x wants to join the network, he will first contact a representitive y on the network. y will then search for x inside the DHT, and find the best location. y will then send x that location on the DHT, and x will just join there. You might be wondering how will the links be set up at this point.

At the Chord DHT there is an operation called stabilize that is run from time to time. This operation is where every node updates his finger table. It is mentioned in a more detailed fashion in the original article.

I didn't deal with those stuff in the article I wrote because I will deal with them in the future, talking about DHT security and stability.

EDIT: One thing that people forget from time to time is that DHTs have some hidden assumption that allows them to work. A DHT assumes that things don't change too fast. At least not too fast that individual nodes can not follow.

DHTs can deal with a few nodes joining and leaving, but if lots of nodes become offline in one moment, you might face a problem.

Thanks for reading.

<comment><USER15> The mainline DHT (bittorrent's Kademlia) is quite resistant to "lots of nodes becoming offline in a moment". Often 5-10M peers are connected, so each peer tracks 180 peers or so. So unless a very large fraction of the DHT is unreachable then you are likely going to be fine.

Also even if by some freak chance all 180 of those peers (distributed across the globe) are gone, you'll get incoming queries from the DHT (assuming you were connected long enough to be in their routing table). Once that happens you can start adding peers again. This is the reason that the DHT is pretty resistant to churn. As long as a reasonable fraction of your 180 node routing table is still around in 10-15 minutes (it's often refreshed that often) you can add as many peers as you want.

So in practice the DHT is pretty robust compared to other protocols/services.

</comment></comment></comment><comment><USER22> I'm not done reading yet, this is very well explained. Thanks a lot.

I found a little typo in the line : "if A and B are very __var __ "

<comment><AUTHOR> I'll fix it. Thanks :)

</comment></comment><comment><USER23> Good read. Its explained in lay man terms.</comment>


A new Go API for Protocol Buffers - <AUTHOR> https://blog.golang.org/a-new-go-api-for-protocol-buffers <comment><USER2> > The github.com/golang/protobuf module is APIv1.

> The google.golang.org/protobuf module is APIv2. We have taken advantage of > the need to change the import path to switch to one that is not tied to a > specific hosting provider.

That's such a weird choice, and will be quite confusing. Now every time you see protobuf being imported, you have to be sure to pay extra attention to the domain used, and correctly remember which one is v1 or v2. If only go modules established a clear way to differentiate API versions using the import path!

<comment><USER3> > If only go modules established a clear way to differentiate API versions > using the import path!

It did, they just chose not to follow it :/ I wonder what rsc thinks about this decisions

<comment><USER4> Rsc?

<comment><USER5> Russ Cox [https://news.ycombinator.com/user?id=rsc](https://news.ycombinator.com/user?id=rsc)

<comment><USER4> Thanks!

</comment></comment></comment></comment></comment><comment><USER6> Not to be too Rust evangelist here, but why the emphasis on reflection instead of further support for generics?

On the Rust side, macros can[1] transform .proto files into data structure definitions at compile time, and instances of the structs passed around have the benefit of strong typing, code completion in editors, lints and tests can easily validate properties of protobuf objects and so on. And all without adding more codegen steps which is nice - the macros can replace a build system having to do codegen as a separate step.

How often do you need to mutate _any_ protocol buffer and remove (redact as they say) fields generating a completely invalid protocol buffer object? Will anything downstream match the ".proto" of an object that's missing a bunch of required fields? Wouldn't it be easier to just all of the sensitive data into an optional "sensitive_data" field that's defined by another .proto, and strip that?

[1] - [https://github.com/danburkert/prost#generated-code- example](https://github.com/danburkert/prost#generated-code-example)

<comment><USER7> A Go API for protobufs can't support generics because Go doesn't support generics, yet. I guess the protobufs team didn't want to wait until Go v2, probably a good call given Go v1.x is going to be around for years.

In Go, a code generator converts the .proto files into Go code, which give the same benefits you see with the Rust macros. Its arguable if a code generation step is better or worse than macros.

The reflection is needed if you have received a protobuf the code don't have a definition for. You can now implement something like a gRPC proxy or gRPC load balancer in Go, without needing to compile it with code from the specific .proto files. You also appear to be able to access annotations on the message definitions, which are not embedded in the generated Go structs. Rust may well have similar features in its API. Java certainly does. A gRPC proxy is a use case redact sensitive data, for when you use it to create an audit log of the messages in the requests and responses.

<comment><USER8> If you don't have the proto definition, there's nothing you can do except pass the object through unmodified. And you should not need a reflection API for that (unless the v1 API was totally messed up in other ways).

Without the definition (and knowing the type!) there won't be anything around to tell the reflection API what the names, types and annotations are. All you would have is field numbers mapped to opaque blobs of data.

<comment><USER9> You can pass around a type descriptor (itself a proto message) along with the opaque message.

</comment></comment></comment></comment><comment><USER10> > _The github.com /golang/protobuf module is APIv1._

> _The google.golang.org /protobuf module is APIv2._

> _We have taken advantage of the need to change the import path to switch to > one that is not tied to a specific hosting provider._

> _(Why start at version v1.20.0? To provide clarity. We do not anticipate > APIv1 to ever reach v1.20.0, so the version number alone should be enough to > unambiguously differentiate between APIv1 and APIv2.)_

Stuff like this is what makes Go confusing to newcomers.

For some time I couldn't understand why people were complaining so much about _—the now deprecated—_ GOPATH environment variable, and later the confusion of how to use all the different and incompatible dependency managers. Then, suddenly last year, it hits me… Go is confusing to people who haven't followed the language from the beginning.

I started programming in Go in 2013 so for me it was very easy to adapt to every change in the language and the community.

I can only imagine all the ambiguous stuff people have to understand when learning the language in 2019-2020.

<comment><USER11> I loved Go's simplicity when it came out.

However, it has become clear to me that it was a little _too_ simple to handle all the new requirements and additions that came after v1.

That's how you get weird comments-as-code (// +build linux,386 darwin,!cgo), strings-as-attributes (tags), tacked-on modules versioning / vendoring, non- typed generics( interface{} everywhere), somewhat-typed generics (go generate), etc.

Go was not _quite_ ready to handle all these directions people wanted the language to stretch to and as a result it's now already full of annoying warts.

I'll still pick Go if I need a simple service with amazing performance. Funnily enough though, I have no need for any of the (non-performance) additions that came after 1.6.

</comment></comment><comment><AUTHOR> Interesting that even Google decided to do complex versioning gymnastics and introduce a new namespace in order to avoid how `go mod` adds the version as a path component.

<comment><USER12> Nah, we just really wanted to stop using an import path tied to a specific hosting provider. Once we decided to change to an entirely different path, we waffled a lot over whether to tag it v1 or v2. There were arguments for and against either, so we eventually picked one and went with it.

<comment><USER13> Sure, but why version the new package as v1.20 instead of v2.x?

<comment><USER14> Because they would had to add a `/v2` suffix to the import path and seems like they didn't wanted to add that. The question is, why is this not released as v1.0.0 then? Still trying to understand the reasoning for this.

<comment><USER12> Reasoning went something like this:

We could tag the new API v2:

\+ Makes the "v1" and "v2" distinction very clear in the import path.

\- Confusing: google.golang.org/protobuf@v1 doesn't exist, but v2 does.

\- In ten years, hopefully nobody cares about the old github.com/golang/protobuf and the confusion is gone.

We could tag the new API v1:

\- Less visually distinct in the import path.

\+ Seems to make sense for the first version of google.golang.org/protobuf to be a v1.

\+ If we decide it was a terrible idea, it's easier to go from v1 to v2 than to roll back from v2 to v1.

We waffled back and forth for quite a while on this, and eventually decided that the first version of google.golang.org/protobuf would be a v1. Then as we got closer to release (but with a certain amount of usage of v0 in the wild), we decided not to second-guess that decision but to start with a version that wouldn't overlap with any version of github.com/golang/protobuf to avoid confusion when someone reports a bug in "v1.0.1".

Maybe it was the wrong choice. If it was the worst choice we've made in the new API, I'll be happy!

<comment><USER15> > The google.golang.org/protobuf module is APIv2. We have taken advantage of > the need to change the import path to switch to one that is not tied to a > specific hosting provider.

This seems like a good example why tying the name of the package in the code with the place where it can be downloaded was a very bad idea. Simply changing hosting providers is now a backwards compatibility break for your clients.

If Go imports didn't work this way, you could have simply offered the old v1 from both github.com and google.golang.org, and used a clear v2 for the v2. Sure , you would have similar problems if you wanted to change the logical package structure for some reason (say, when an open-source package moves to a different organization), but that is a much rarer case than switching code repos.

However, given that that ship has probably long sailed, you probably picked the right choice.

I would also note that v2 in the import path feels a bit strange, since it makes users of the newest code need to know about the history of the package, but there are also clear advantages which probably out-weigh this (I believe this is absolutely not the case for the decision to include the hosting provider in the import path).

<comment><USER12> Indeed, tying the name of a package to a hosting provider is a bad idea! That's why we changed to an import path which isn't tied to one. google.golang.org/protobuf is not tied to any particular provider, and we can redirect it wherever we want.

See "go help importpath" for details on how this works.

<comment><USER16> > _\- Confusing: google.golang.org /protobuf@v1 doesn't exist, but v2 does._

> _google.golang.org /protobuf is not tied to any particular provider, and we > can redirect it wherever we want._

I agree, this _is_ confusing. Why doesn't google.golang.org/protobuf@v1 return what's at github.com/golang/protobuf, then, if only to provide a tidy answer for those who wonder what v1 looked like?

<comment><USER15> Because the Go package name _is_ github.com/golang/protobuf. Serving it from a different address is already a breaking change.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER17> Congrats on the big release!

One thing which saddens me a lot though, is returning types you have to type assert because they had to avoid an import cycle. I don't know if it could be avoided (probably not) but it's still unergonomic.

> (Why the type assertion? Since the generated descriptorpb package depends on > protoreflect, the protoreflect package can't return the concrete options > type without causing an import cycle.)

</comment><comment><USER18> Does someone know if this new version uses fast generated code for (de)serialization? The previous version was pretty slow (because based on reflection I think) so many people used [https://github.com/gogo/protobuf](https://github.com/gogo/protobuf) for good performance.

</comment><comment><USER19> It is odd that the Go team is all in on semantic import paths, but basically everyone else is like "I will blow up the moon if it means I don't have to use v2 in my import path.

<comment><USER20> I wonder if we will start seeing people add v1.example.com, v2.example.com etc... to their git servers to avoid having to change the path's in all of their private projects.

<comment><USER21> Seems like you need something like a DNS server for repo's.

</comment></comment></comment><comment><USER22> Hopefully this is good news. The old Protobuf package had ossified, with lots of ergonomic issues that the maintainers seemed uninterested in addressing.

One perennial annoyance is how awkwardly some Protobuf stuff ends up being represented in Go. For example, "oneof" types:

      message Event {
        oneof payload {
          Create create = 1;
          Delete delete = 2;
        }
      }

      message Create {
        string id = 1;
      }

      message Delete {
        string version = 1;
      }

You get these messy types (I've elided most of the yucky marshaling-related stuff):

      type Event struct {
        Payload isEvent_Payload `protobuf_oneof:"item"`
      }

      type isEvent_Payload interface {
        isEvent_Payload()
        MarshalTo([]byte) (int, error)
        Size() int
      }

      type Event_Create struct {
        Create *Create `protobuf:"bytes,1,opt,name=create,proto3,oneof"`
      }

      type Event_Delete struct {
        Delete *Delete `protobuf:"bytes,1,opt,name=delete,proto3,oneof"`
      }

There are multiple problems here. One: Why the extra struct? For example, to create a single event, you have to do:

      Event{
        Payload: &Event_Create{
          Create: &Create{ID: "123"},
        },
      },

...instead of just:

      Event{
        Payload: &Create{ID: "123"},
      },

Secondly, isEvent_Payload is private! So given a _Create or an_ Delete, you can't store it in a single variable:

      func nextEvent() isEvent_Payload { // <-- not possible
        if something {
          return &Create{...}
        } else {
          return &Delete{...}
        }
      }

The reason for this appears to be so that the generated serialization can be very dumb and not rely on type switches. But I don't buy that this is how it has to be.

There are other issues. Overall, the whole package seems designed from the bottom up for machines ( _mumbles_ possibly also _by_ machines), not humans.The upshot is that using Protobuf types as "first-class" types — meaning the types you actually use internally in the meat of your app, as opposed to in the controller glue that lives in your API and mediates between the API and the internals — feels super messy.

As an aside, anyone know what issues this paragraph refers to?

      The google.golang.org/protobuf/encoding/protojson package
      converts protocol buffer messages to and from JSON using the
      canonical JSON mapping, and fixes a number of issues with the
      old jsonpb package that were difficult to change without
      causing problems for existing users.

Did they finally fix the zero value problem with jsonpb.go [1]?

[1] [https://github.com/gogo/protobuf/issues/218](https://github.com/gogo/protobuf/issues/218)

<comment><USER23> To quote one of the maintainers of the golang/protobuf implementation:

> The jsonpb package is intended to the be faithful implementation of the > proto<->JSON specification, for which the C++ implementation is considered > the canonical "reference" implementation.

And that C++ implementation does some kinda dumb things. For example, it marshals the protobuf type of int64 into a string when marshaling from a protobuf struct into JSON.

[https://github.com/golang/protobuf/pull/916](https://github.com/golang/protobuf/pull/916)

I believe that the package they mention there is meant to be a more "canonical" protobuf <-> json marshaller than the existing jsonpb package.

<comment><USER24> IMO marshaling int64 to string is required for interoperability; RFC7159 recommends not assuming that more than IEEE754's 53 bits of integer precision are available:

[https://tools.ietf.org/html/rfc7159#section-6](https://tools.ietf.org/html/rfc7159#section-6)

</comment></comment></comment><comment><USER25> Hmm. I think it would be quite simple to build an authentication proxy/firewall based on this without having to know the exact underlying message.

Maybe tagging fields with auth_userid and/or auth_token.

</comment><comment><USER26> Hackernews, where the highlight of mentioning common libraries is the fact that they're done so in Go and Rust.

<comment><USER27> I find that to be a feature. If I open the article and/or comments and find out this was about a library for Ruby or Java then I wasted my time because I don't use those. Likewise, a non-Go or non-Rust dev can ignore Go and/or Rust tagged library articles.

</comment></comment><comment><USER28> Umm ... When will pb allow me to decode a field in buffer without having to decode the entire object in the buffer? I'm talking a field in a pb message NOT a byte stream I hand encoded without reference to a message eg more like flat buffers.

</comment><comment><USER29> This article doesn't do a great job explaining exactly what new features are a part of this release. Seems to me like it's finally enabling some of the useful type iteration stuff that is possible in other versions of protobufs.

</comment><comment><USER30> I thought the general consensus on protocol buffers was "Meh"?

<comment><USER31> I think for large organizations they probably make sense. They're potentially a lot less data per request than JSON.

For the average app? Overkill, sure.

<comment><USER32> Overkill? It doesn't take a lot of work to add proto buffers to an existing project (if you're already using a reasonable build system, anyway).

<comment><USER33> If you have weeks to spend optimizing build systems, linters/editors, CI/CD, etc...</comment></comment></comment></comment>


Robert Scoble - Apology to Twitter. Chris Sacca, investor in Twitter, has… - <AUTHOR> https://plus.google.com/111091089527727420853/posts/birhHh9jWfK <comment><USER2> Love him or hate him, I think it's great when someone makes a mistake, owns up to it publicly. Even allow others to contact them directly and list all levels of accountability even to the top.

I have heard of Scoble but never follow him.I have made the same mistake he has made, where a friend I highly trust, who I think is rarely wrong forwards me info which I use for an argument and it turns out to be completely false. Yes it was laziness, no doubt about it and I learn that the hard way, and so has Scoble.

</comment><comment><USER3> "I'd be happy to sit down with anyone from the company and any investor too to give their side of the story." and then he refers to himself as a "journalist".

He's nothing but a shit stirrer, why do people still pay attention to him? He just creates needless _drama_ with everything that he does...

</comment><comment><USER4> Jesus, Silicon Valley / tech is one big soap opera. Is there any other industry outside of show business that has this much gossip and fake drama?

</comment><comment><USER5> Can't find the original article he wrote that stirred up all this commotion...

<comment><USER6> Here's his original post, now with an update at the beginning: [https://plus.google.com/111091089527727420853/posts/X58hnS9P...](https://plus.google.com/111091089527727420853/posts/X58hnS9PMct)

</comment></comment><comment><USER7> People still care what Scoble says?</comment>


All your bad PDF are belong to me - <AUTHOR> HackerNews crowd, I am calling all security engineers and system administrators to please send me your malicious PDF files! Why you ask? Well, I am doing research to combat against malicious PDF files and need known bad or suspicious samples to collect data from.
Consult your SPAM filters, quarantined files or repositories of malicious data and head over to here to submit: http://allyourpdf.9bplus.com/ . If you can point me to any other sources besides the malware domain list, then that is good too!
Thanks and Happy Holidays! <comment><USER2> I fail to understand why pdf is not a purely representational document format (i.e. without scripting and executable content).

<comment><AUTHOR> It would certainly solve a lot of the problems associated with the files. Requiring JavaScript should only be allowed if a sufficient justification is provided. Simple things could be done to solve a lot of these problems, but alas, we love our Internet calling functions.

<comment><USER3> However there are "sufficient justifications"; the best I've seen is a Missouri individual state income tax form, which besides the expected sorts of things generates a 2D barcode with all the important data encoded into it.

Print it out, they scan it in, I can't imagine how much time, money and hassle it saves. Allows for _fast_ refunds as well.

So, given that and other useful form type pages, how could we properly manage allowing Javascript for the masses?

<comment><AUTHOR> Offhand I can't really think of a way to manage the problem. Instead I am looking at ways to identify the bad parts of it and improve detection. Email me if you would like more details.

At this point the specification is not going to just remove or change the problem elements simply because outside the context of the Internet, their not bad. The specification merely offers flexibility to those using or creating PDF files. It is the abuse of that flexibility that make it extremely difficult to identify "known bad" files or protect against them.

What I was getting at was a centralized approval system similar to an issuing certificate. In other words, to use certain functionality, you would need to request it from a trusted entity. This of course would be outrageous and likely to be circumvented, but it never hurts to dream.

</comment></comment></comment></comment><comment><USER4> Can the title be edited as Crowdsource HN: ....

More descriptive of whats going on, so people actually click and listen :P

<comment><AUTHOR> If you know a way to change it, please let me know and I will do so. Shoot me an email and I can discuss some of my research with you. If you have any interest or appreciation on the subject then I think you will like what I have going on.</comment></comment>