I want to fix programming - <AUTHOR> http://jonbho.net/2012/01/24/i-want-to-fix-programming/ <comment><USER3> Dear Jon:

What you have is fabulous and immediately useful: You are describing a language for describing constraints on the results of computer programs.

In other words, you can express the correctness of a computer program and check whether a program is, in fact, correct.

It's true that IF a sufficiently smart compiler could infer a working program from the definition of correctness, no further "programming" would be required. However, we don't have that yet, and in fact I'm not urging you to write it.

Instead, consider the benefits of your language _just the way it is_. You can use it to write test suites. You can embed it in other languages to express design-by-contract.

It looks valuable in its current form.

<comment><AUTHOR> Reg, thanks so much for your comment. I'm really happy you appreciate the value.

It's very interesting that you put the focus in the descriptive language. I've been working on that for a long time. It's not finished, but some areas are clarified. Indeed, after reading your comment I think I will try to focus on getting somewhere workable with that part first.

This whole project is so big. It will be good to do it in the open and use as much help as I can.

<comment><USER4> The "Why Dependent Types Matter" paper at <http://www.cs.nott.ac.uk/~txa/publ/ydtm.pdf> actually uses an implementation of "sort" as its driving example.

The paper starts out with a simple sort implementation, and then adds static proofs of correctness for:

A) Totality (Termination/no infinite loops) B) Length of output = Length of input C) Output is sorted

It does not prove the one-to-one mapping, and I don't know how hard that would be to prove (intuitively, it does not sound like it should be too hard).

The dependent types approach lets you write a program while reasonably controlling its algorithmic complexity and operational behavior -- and _still_ have its mathematical meaning proven to match a specification.

I think a title like "I want to fix programming" is a bit over the top -- given that it is a hard problem with very smart people working at it. It is more acceptable, in my opinion, to "fix programming" after learning in detail what the state of the art already entails. That means knowing Agda, Coq, Logic languages, Hoare logic, etc.

<comment><AUTHOR> Thanks Peaker. I know a bit about dependent types, and how complex types sytesm can help a lot. Indeed, the title is a bit over the top (or maybe even too much over the top!). Hopefully the attention will help get some extra hands & brains.

At the very least, all the pointers I'm receiving are really valuable.

I don't claim I can fix it, but I definitely want to fix it, have some ideas, and I'm willing to work, listen and learn!

And if I think I can add something, is that I come from a practical/industrial background. Most of my work has been in C++. I'm looking for practical tools for everyday work.

</comment></comment></comment></comment><comment><USER5> I once read a programming book on Elliptic Curve Cryptography, which was invented by Dr. Neal Koblitz. So the book said - whatever is in the next 200 pages is considered trivial by mathematicians working in number theory. If you write it in the form of equations, this entire book will occupy less than half a page. It is basically a lemma and a few theorems. But this book is 200 pages long. Why ? Because it tells you how to implement that lemma and those theorems using C++.

So that's how that is.

Just yesterday I described to an intern how to generate the nth term of a Halton sequence in base k. So if you want the 17th term in base 3, well, decimal 17 = ternary 122, reflect 122 to get 221, then ternary 0.221 = decimal 0.926. So the answer is 0.926. Its very, very simple. High school math. The nth halton term is in fact given by a 1-line equation in mathematics.

Now he came back with scala code to do the same thing. Look at this monster:

def Halton( n:Int, b:Int):Seq[Double] = {(1 to n).map( x=>Integer.toString(x,b).reverse.zipWithIndex.map(y=>(y._1-'0')*pow(b, -(y._2+1))).sum)}

Now it works & its functional programming & computes a million Halton terms in any base in 5 seconds and so on, but still, look at it. Is it anywhere close to the one line equation ? If I express what I want declaratively, will it be any simpler ? Not really. Why not ? Because a declaration like "1..n" means an imperative loop, a declaration like "convert 17 in decimal to ternary" means a whole bunch of divisons and remainders and aggregation, then a reflect means reversing a string, which implicitly means iterating over a character array and allocating new space for the reversed result, then a reconvert ternary to decimal means an iteration with powers of 10, where a power means other iteration over multiplication....jesus! This simple 1-line equation in math becomes hundreds of thousands of loops in practice. There's no getting around that. If you've gotten around it, you've just invented math!

<comment><USER6> That function definition is a monstrosity because it is poorly factored. Assuming pre-existing fromBase and toBase functions, here is a definition that in my opinion is easily readable and matches your verbal description:

        halton b n = fromBase b (reverse digits) / b^(length digits)
            where digits = toBase b n

Just to show there's nothing up my sleeve, here are the fromBase and toBase definitions:

        toBase b 0 = []
        toBase b n = (n `mod` b) : toBase b (n `div` b)

        fromBase b []     = 0
        fromBase b (n:ns) = n + b * fromBase b ns

They too read like how a mathematician would define them. (I abstained from use of foldr/unfoldr to make this clear.)

Edit: On second thought, it's easy to implement it as a direct recursion. I'll demonstrate it in C so I'm not accused of language chauvinism:

        double halton(int b, int n)
        {
    	return (n == 0) ? 0.0 : (n % b + halton(b, n / b)) / b;
        }

Here the digit reversal is implicit in the recursion (the oldest programmer trick in the book).

This C code is self-contained and compiles to a tiny handful of machine code instructions, so it's hard to see how your statement that "this simple 1-line equation in math becomes hundreds of thousands of loops in practice" holds much water.

</comment></comment><comment><USER7> The problems the OP wants to solve are formally undecidable and reducible to the halting problem.

1\. Given a formal specification, find a program that meets the specification.

2\. The apparently simpler problem of checking whether a given program meets a specification.

Both are undecidable. That said, there is an extensive literature on practical approaches to this problem. They generally suffer from intractability.

<http://cfpm.org/pub/papers/tiofdm.pdf>

I suggest looking at Robert Harper's book on type theory instead.

<comment><USER8> I agree with the sentiment that the problem the OP brings up is difficult, but bringing up the halting problem here seems pretty pedantic.

If you restrict yourself to a turing incomplete language, checking a specification no longer reduces to the halting problem. In fact, there are many systems that do exactly this, and most of these systems are perfectly capable of sorting a list and proving that the list is sorted.

However, I think the real point is that there are at least dozens of smart people who are working on this exact problem all day, every day. The OP shows neither a deep understanding of why previous attempts at this have failed or really, any knowledge at all about the state of the field of programming language research.

<comment><USER9> Can you provide links to papers which try address this problem in practical terms, or some leads that one might use to acquaint themselves on the current state of research in this area? The paper the parent links I've only skimmed but what it seems to be advocating seems not to be that far from test-driven development. Has anyone tried anything more closely resembling what the OP is proposing? Are any such attempts doomed fromt he start?

</comment></comment></comment><comment><USER10> So long as the problem you are trying to solve is constrained by the ease with which you can express the problem, this will help.

But it will not help when the constraints are elsewhere: memory, network, CPU, heterogeneous interoperability; all still relevant in these times of mobile and distributed computing. When you are operating to these other constraints, you need control - or rather, consistency and predictability - over the consumption of those other resources. It's no good having a magic compiler that can turn your specification into an implementation, if you don't know whether the implementation will run in O(1) or O(n) space or time, what kind of constant factors are involved, and how easily you can tweak the tradeoffs when you hit one constraint or another.

I concur with raganwald's comment: this is more useful for proving, static analysis, testing, etc.

Furthermore, I'm pessimistic that you'll see substantial improvements in expressibility using your approach, particularly when you try to scale it above simple functional (by which I mean stateless input to output) problems. The reason is that programming is an exercise in attention to detail while simultaneously keeping the large in mind; and your focus won't change this fundamental nature. Writing a _complete_ specification for a program rivals writing an imperative implementation in complexity (soup to nuts, I'm not talking about high-level DSLs here, which is an approach equally applicable to the imperative route). I expect a good compiler will swamp you with the inconsistencies of your hidden, unstated assumptions.

For example, I note that your sorting specification doesn't specify whether or not the sort is stable; I think that makes it incomplete, and also it's a tricky constraint to encode.

<comment><AUTHOR> You are right, and I think you've found one of the weak spots. But I do think that can be treated, and the advantages of the programming-ease will be a good trade-off in many cases with the lost control for efficiency.

It's like the switch from assembly to C, or from manual memory management to automatic memory management. You lose something. You gain something else. The new approach is not valid for everything, but for some/many cases, it's much better.

Yes, the spec of a complete program will still be a great amount of work. The default resulting code will probably be slow. But you will be able to "describe" the program (in many cases), and you will be able to optimize it later without risking the correctness. Hopefully that can be a great aid and tool in many cases. We'll see, and I hope to have a lot of help along the way doing it publicly!

BTW, if what comes out is just a good new approach to static code analysis / formal specification / etc... it will have been worth it. But I do hope to find a usable tool at the end of this quest :)

<comment><USER10> I wish you the best of luck in any case. I look forward to reading more about how you expect the transformation logic would work.

</comment></comment></comment><comment><USER11> Relevant stuff from SICP [1]:

"The contrast between [a mathematical] function and [a computing] procedure is a reflection of the general distinction between describing properties of things and describing how to do things, or, as it is sometimes referred to, the distinction between declarative knowledge and imperative knowledge. In mathematics we are usually concerned with declarative (what is) descriptions, whereas in computer science we are usually concerned with imperative (how to) descriptions.

...an important current area in programming-language design is the exploration of so-called very high-level languages, in which one actually programs in terms of declarative statements. The idea is to make interpreters sophisticated enough so that, given "what is" knowledge specified by the programmer, they can generate "how to" knowledge automatically. _This cannot be done in general, but there are important areas where progress has been made._ "

[1] [http://mitpress.mit.edu/sicp/full-text/book/book- Z-H-10.html...](http://mitpress.mit.edu/sicp/full-text/book/book- Z-H-10.html#%_sec_1.1.7)

<comment><USER12> Yea, you will invariably run into the halting problem and similar. But I think academics are too held back by that in general. The "program space" is very large, and there's of course a lot of strange beasts out there, but only a very small subset of all possible programs are meaningful to create (in the sense that people want them or their output).

</comment></comment><comment><USER13> Hope springs eternal...

But so does a-historical thinking.

 _"Programming is broken. Completely broken."_

The "software crisis" has existed since, oh, computers began, maybe fifty years ago. Which means that, in a sense programming couldn't possibly be "broken" since it never ... was ... working ...

A way to put to it is that programming has never been as amendable to planning, to the realizing one's _intentions_ as ... we THINK it should be.

And there's the interesting part.There are lots of things that are hard for humans beings - say, flying supersonic jets or solving partial differential equations.

What's funny about programming is that we always have a hard time of it BUT we always, _over and over again_ , think it should be easy.

There is something interesting, a study for psychologists.

I would claim that a factor is that since programs are like the natural language we normally use, we intuitively expect programs to be a "smart" as other human beings in understanding our intentions and so we wind-up disappointed over and over again.

Of this leaves "intentions" and "smartness" as defines in our model. Perhaps we can use this situation to get some clues as to their meaning...

</comment><comment><USER14> Here is another way of defining something is sorted, taken straight from a real language:

      Inductive StronglySorted : list A -> Prop :=
        | SSorted_nil : StronglySorted []
        | SSorted_cons a l : StronglySorted l -> Forall (R a) l -> StronglySorted (a :: l).

What this says is that an empty list is sorted (SSorted_nil), and that given some sorted list l, if a is less than all of the elements in l (well, we generalize to some relation R), then a prepended to the whole list is sorted. (SSorted_cons)

But it turns out, there is another way we can say this property, if our relation is transitive: all we need to say is that the element is less than or equal to the first element of the list.

And for any non-trivial specification, there are literally dozens of ways of specifying it, all of which happen to be identical. Which one do you pick? Which one is easier to use? Hard to say, in general.

Source: <http://coq.inria.fr/stdlib/Coq.Sorting.Sorted.html>

<comment><USER15> <http://www.cis.upenn.edu/~bcpierce/sf/> is a good way to learn coq if you are interested and don't already know it.

also, i didn't completely follow the final example, but you might find that category theory is relevant. related, it's not completely clear to me why you're not happy with functional programming. in a sense what academic functional programmers are doing is what you want your software to do. they're just not smart enough to put it in a compiler yet. i think. for example [http://www.fing.edu.uy/inco/cursos/proggen/Articulos/sorting...](http://www.fing.edu.uy/inco/cursos/proggen/Articulos/sorting.ps.gz) (isn't that kinda what you want?)

</comment></comment><comment><USER16> The most common declarative programming language used in SQL. In theory the optimizer takes my declaration and builds an optimum (or at least decent) set of step in the database engine to accomplish what I have declared. With all the work in this limited area, database optimizers get it really wrong quite a lot (Sybase 12.5 was bad enough on big data that forcing order of joins and indexes was needed).

I am not sure we are at the stage where the computer can be trusted with the "what". Most new learners need to be told step by step the "how" and computers seem to be at that stage. Prolog is a nice counter-example, but even it can get itself into some real trouble.

I agree programming is broken, and I agree the whole program knowledge is just painful, but I am think there are some steps before full declarative. As much as I dislike the old VB, it did have a very vibrant component market. That concept didn't seem to evolve or make it to the server. I look at Mongrel2 and wonder if the component approach could be applied. Regardless, I think the biggest problem is lack of ability to separate the pieces of a big project effectively no matter the eventual solution.

<comment><USER17> _The most common declarative programming language used in SQL._

Or maybe a spreadsheet (<http://philip.greenspun.com/panda/databases- choosing>).

<comment><USER16> although I respect Philip Greenspun greatly, most spreadsheets I have seen are pretty procedural. People tend to think in steps on those types of calculations.

</comment></comment></comment><comment><USER18> For those who haven't read it yet, this paper has a lot to say about the cause of programming complexity, Out of The Tarpit:

<http://web.mac.com/ben_moseley/frp/paper-v1_01.pdf>

They say that the only essential complexity is the one inherent to the problem the program is trying to solve. Everything else is just here because we haven't yet found the methodology or invented the tools to battle it.

They also describe an ideal world, where all the programming is done in way of declarative programming - what you want the code to do, and not how to do it.

<comment><USER19> _this paper has a lot to say about the cause of programming complexity, Out of The Tarpit_

This paper is highly regarded by some smart people, but every time I've tried to read it I've seen nothing of much value - only some obvious platitudes about complexity (including the bit about complexity being intrinsic to the problem vs. just the implementation), a lot of architectural gobbledygook (complete with boxes-and-lines diagrams), and some hand-waving about combining the functional and relational models. Has anything ever come of this? Specifically, any working systems?

<comment><USER20> Clojure.

Clojure's lispy (simple implementation, syntax), FP (reduced state), easier concurrency, agents + stm are all influenced by this paper.

<comment><USER19> That seems like a huge stretch. Lisp has nothing to do with that paper, and minimizing state and complexity could hardly be more obvious design concerns. The concurrency aspect I can't comment on, but it's not a major theme there either.

Perhaps I just don't get it, but I'm a little miffed at having tried several times to absorb the gems of wisdom in that paper and come up with nothing that isn't obvious (even the idea of functional programming over relational data is obvious) and that couldn't have been said less pretentiously in easily a tenth of the space - ironically, for a paper about minimizing complexity.

<comment><USER20> The paper was about several things:

1) accidental complexity is the source of a large number of bugs 2) implicit, unnecessary, tightly-coupled state is the cause of a significant number of bugs. 3) mostly-functional programming reduces the amount of state in 2 4) RDMS databases, with transactions and triggers, are a good way of reducing state as well

Clojure applies all of these lessons. lisp reduces accidental complexity in the language. FP reduces accidental complexity wrt to state. the STM and agents are close analogues to DB transactions and triggers.

"even the idea of functional programming over relational data is obvious". Yes, but where else has that been tried?

<comment><USER19> I posted a reply, but decided I was repeating myself. Thanks for the discussion.

</comment></comment></comment></comment></comment></comment><comment><USER21> So, over the years I've played with many things that claim to be "declarative", and here's why I now shy away from them like the plague. There's no such thing as "declarative". No matter what you type into the computer, at some point it's going to turn into instructions that do the thing you want done. Trying to create a declarative language is a way of making it _extraordinarily_ opaque as to what the machine is actually going to do. It looks great in four lines, it crashes and burn on any real sized problem, because you inevitably hit the following sequence:

1\. I encounter a problem; a performance issue or a bug.

2\. I can not practically proceed past this point because everything I might need to figure out what is going on has been "helpfully" obscured from me.

Yes, you can still thrash and flail but this hardly constitutes a "fix" to programming. You simply can not help but create an abstraction that not only leaks like a sieve, but is actually multiple leaky sieves layered on top of each other in opaque ways. (And letting us see in is in its own way a failure case too, with these goals.)

Part of what I like about Haskell is that it helps bridge the gap, but doesn't actually go too far. A map call is still ultimately an instruction to the machine. It's not quite the same type of instruction you give in C or C++, what with it being deferred until called for (lazy) etc, but it's still an instruction and it can be followed down to the machine if you really need to without only marginally more work than any other "normal" language. (It may be a bit bizarre to follow it down all the way, but hardly more so than C++ in its own way.)

Contrast this to SQL, which is declarative, and you never have to worry about what the database is doing to answer your question. Except it never works that way and you _inevitably must_ actually sit there and learn how indexes work and how queries are parsed and how the optimizer works to a fairly deep level and then sit there on every interesting query and work out which synonymous query will tickle the optimizer into working properly except that you actually can't do that and you end up having to turn to weird annotated comments in the query specific to your database and then you still end up having to break the query into three pieces and manually gluing them together in the client code.

And I don't even care to guess how many man-millenia have been poured into that declarative language trying to make it go zoom on a subproblem _much_ simpler than general purpose computing. (Well, except isasmuch as they've more or less grown to encompass that over the years, but it's still at least meant to be a query language.)

So, if you think you can fix that problem, have fun and I wish you the very best of luck, no sarcasm. This is the problem I've seen with the previous attempts to go down this route before, and I feed this back in the spirit of helping you refine your thoughts rather than yelling at you to stop.

<comment><USER22> I don't think there's any inherent problem with stuff being declarative. We just need the software to be smart enough to handle it well. We might have a long way to go till we can do this effectively.

I'd suggest looking at 'declarative' from the point of view of _delegation_ (not in the technical sense, but the usual sense of getting someone else to do a task for you). Ideally you want to specify what you want to be done and you shouldn't care about how it's done. Software is meant to be a labor saving device.

I assume you're ok with the idea of delegating tasks to people. If the software was smart enough I think it'd reasonable to delegate to it as well.

<comment><USER23> "We just need the software to be smart enough to handle it well."

You just shifted the complexity with one sentence, it doesn't make it any easier.

<comment><USER22> I never said anything about making it easier.

I was pointing out that it's a worthwhile longterm goal, but one that requires sophisticated software to achieve.

</comment></comment></comment></comment><comment><USER24> The assertion that programming is broken seems to be stemming from assumption that, in contrast, _human minds_ \- which currently do the programming - <USER2> not. And that's a huge overstatement. For a product of a process that is essentially throwing things at wall and see what sticks best (usually called 'evolution'), it's not _that_ bad, yes. But it's still ridden with bugs: misperceptions, biases, reasoning errors and inconsistency.

The last one is especially relevant. Is it really so that the biggest problem of programming is putting the intent into code, because the intent itself is perfect and pure? Isn't it quite the contrary, that forming your wants into the rigorous form of code is also helping to reshape them into (more) consistent ones?

</comment><comment><USER25> As it happens, computer science is _exactly about those_ rigid steps of how to get a computer do bigger things. If you could express what you want, you could replace considerable part of Knuth's books by referring to "just sort these numbers for me", leaving the "how" to the computer.

Bridges don't get built because some guys "wants to go see the other shore". Bridges get built by rigorous design and constructed a rivet by rivet or bolt by bolt. Then you can go see the other shore.

For most part, we've come a long way in programming. Sorting has many reasonable algorithmic approaches well-studied and as a result of that in almost any language you can already say sort() and have some stuff sorted. I don't particularly need to know that Python uses timsort, it just sorts. Of course, I can study sorting in detail if I want to and possibly discover something novel.

Why programming never gets easy is that by solving existing problems we accumulate newer, more complex problems and interactions. _Programming will always be difficult_ because we automate anything that's no longer difficult.

The "Sort these" example is not programming, it's something else that potentially builds on programming. Type some stuff into Matlab and watch your computer sort out difficult stuff for you and do the calculations in parallel in highly efficient manner using Cuda. It's not programming, it's using a machinery that has been programmed to interpret certain classes of the user's wishes and take care of all the physical steps required to finish the desired computation. You still can't ever take a _blanko computer_ , ask it to sort stuff, watch it figure out how and call that programming.

<comment><AUTHOR> I totally understand and concur with the need for the tools to build bridges by rigorous design. I also think we can look for better tools to build bridges, if we understand better the seas and rivers we are crossing, and if we analyze the limitations of the current tools.

</comment></comment><comment><USER26> "Computer generate for me the sequence of all possible steps I could take; highlight moves that results in high economic and emotional payoff. Feel free to use alpha beta pruning and dynamic programming to reduce the time taken to polynomial time. Also, constrain possible moves to reflect my personal proclivities. Also, highlight a path were I'm dating Alicia Keys and Drake is with Nicki Minaj.. hate to see him beg on a music video." [1]

My dream programming language should be able to run this. Good luck.

[1] [http://chestergrant.posterous.com/your-favorite- programmer-d...](http://chestergrant.posterous.com/your-favorite-programmer- doesnt-like-coding)

</comment><comment><USER27> Why not fix the problem with 2 languages?

Let me explain. You would have:

1\. a language for your actual code - the real code of your application (imagine an application programmed in Python or C++)

2\. a language to concisely describe what the code does - the code for your "tests" (imagine the tests written in a simplified Haskell or some dialect of mathematical language)

There would be different constrains for the two languages, as language (2) would not have to be efficient or portable or any other requirements you can imagine, it should just be very concise, to allow to describe in a couples of lines what 100 or 1000 lines of language (1) code do.

And it would be very different from just writing very granular tests: the program could be run in "debug" mode, with the language (2) tests or executable-descriptions of a function running after every function call (very slow but still working), and in a "real" mode in the client computer or when profiling and optimizing for speed. You could still have regular test and language (2) could be much simpler than something like Haskell because it would not need have fast execution and it will only be written in short snippets that could be proven correct by "pen & paper" or "mind running".

[minor edit for spelling and paragraphs]

<comment><USER28> How does that differ from, say, Haskell's QuickCheck [1] and its ten thousand counterparts in other languages?

[1] <http://en.wikipedia.org/wiki/Quickcheck>

<comment><USER27> Thanks, I took a better look at that and realized there are 2 implementations of QuickCheck for Python... I'm not really a Haskell developer so I didn't knew much about QuickCheck.

Though I still think none of our current programming languages are good "test" languages... I just gave Haskell as an example, but there are probably tons of things that would make it annoying for writing descriptive versions of an algorithm that could be used as tests...

And at least the Python version of QuickCheck I just looked at are far from a descriptive "implementation" of a function... they seem more like some test- automation that could only prevent most of the bugs that might not crop up in typed language...

</comment></comment></comment><comment><USER29> I'm convinced that IF you can make this work, it's the future. 'Programmers' will still do what they currently do... But instead of writing applications, they'll be writing compilers. (Or whatever we end up calling the programs that turn declarative code into programs.)

This will lead to a rise in a new job. Instead of coding, they'll be describing applications. And done right, it should be easier than coding.

Initially, the 'compilers' will be pretty bad at what they do. The code they produce won't have memory leaks and such, but it'll be horribly inefficient. But thing will gradually get better and better until machines are writing better code than humans, for the majority of applications.

Eventually, as with current self-hosting compilers, new self-hosting compilers will be created and programmers will be phased out. This will be quite an interesting day.

One thing I see holding this back is computing power. The initial horribly- inefficient programs will be created by horribly-inefficient 'compilers'. They'll take insane amounts of processing power to do their jobs. Eventually, this will get better, and processing power will increase, though. It may be that we aren't at a point where it's plausible yet.

<comment><AUTHOR> I'd say IF or WHEN we can make this work. It's a task for us all. I think both academic and practically minded developers need to collaborate, which doesn't happen often enough.

Programming will remain tough and only doable for certain types of people. But I think it can be a much more pleasant experience.

</comment></comment><comment><USER30> Isn't the beer example somewhat self-defeating? It seems to me that the only reason we don't need to tell our friends how to get the beer is because he has already those concepts abstracted away, but he still had to learn them, mostly by imitating others, but also by being explicitly taught.

Sure, he didn't have to be taught how to fetch a particular kind of beer in a particular house, but then again, when I make a new e.g. Starcraft map, I don't have to change the code to teach units to move from a coordinate to another. The learned algorithm is just general enough that it applies to any object that fits a certain interface.

And humans make a _lot_ of errors to correct their internal spacial navigation algorithms. To correctly fetch a beer, it took years of bumping into stuff and falling down until the algorithm was robust enough.

Seems to me that a system with hand-written imperative algorithms and a general mechanism of iteratively improving them by trial-and-error is much more close to what we humans to than a compiler that simply comes up with algorithms from scratch from just reading intents and goals.

</comment><comment><USER31> As CPU excess increases, I guess I've always thought that eventually code blocks could be grown using genetics inspired algorithmic processes that could just try everything possible until something works well.

<comment><USER32> You are underestimating the rate at which the number of possibilities to be searched explodes exponentially. Read the paper on Denali, an optimizer that conceptually tries all possible instruction sequences to compile a piece of a program optimally. Even though they used sophisticated pruning techniques and state of the art combinatorial solvers it didn't work for anything beyond a couple of instructions (less than 10), and AFAIK nothing practical ever came of it. Say you can currently successfully synthesize instruction sequences of length 6. The thing with exponential growth is that even if computers became a million times faster, then you might be able to achieve instruction sequences of length 9, _not_ of length 6 million.

</comment></comment><comment><USER33> The quality of the comments both here and the author's blog is astonishingly good, every comment adding something to the discussion. This is the hacker news I know and love.

<comment><AUTHOR> Yes I'm amazed with that too! And really enjoying it myself!

</comment></comment><comment><USER34> This doesn't look much simpler to me, and I suspect the problem with this approach will wind up being incomplete or overconstrained specifications, rather than incorrect programs.

To use the language of the op, it will be composed of many discrete simple assertions that excruciatingly specify the output.

</comment><comment><USER35> This is called declarative programming (<http://en.wikipedia.org/wiki/Declarative_programming>), as opposed to imperative programming.

<comment><AUTHOR> Sure, sorry, didn't mention on the article, Prolog is the closest thing that exists, although I think Prolog commits at least one cardinal sin that breaks the really great properties (/cut/) and it just doesn't get deep enough.

<comment><USER35> Prolog is a kind of declarative programming, but the two concepts are not the same. From the link: "declarative programming is a programming paradigm that expresses the logic of a computation without describing its control flow." This can be implemented in any way one desires.

</comment></comment></comment><comment><USER36> Also see Mozart/Oz and other constraint programming systems. <http://en.wikipedia.org/wiki/Constraint_programming>

<comment><USER37> Came here to say this.. the language already exists!

</comment></comment><comment><USER38> Not to discourage you from trying anything but it looks like you've ignored the complexity issues with this (I won't pick at the fact that you described an O(n^2) algorithm for sorting).

I understand that you don't want to specify the steps to achieving the goal but the steps directly impacts running time.

In your particular example, a compiler could produce a program with the correct output but runs extremely slowly even for small arrays (by simply trying all permutations until it finds one satisfying your constraints, or even worse, randomizes the entries until the constants are satisfied ("bogo sort")).

Furthermore, there are undecidable problem for which the output is easy to specify but no program could exist. For example, deciding if an input piece of code will loop indefinitely.

In your article, you've mixed needless overhead (the dummy/local swapped variable comes to mind) and the steps needed to specify an algorithm.

If you only want to remove the overhead (and thus, some source of mistakes you've pointed out), you could aim for a language where the algorithms are easier to specify.

In the bubble sort case, the code would look something like.

      def bubble_sort( array ):
      	while there is an i such that array[i]<array[i+1]:
      		swap( array[i], array[i+1] )

(You can almost do this in Python already which seems to be where the syntax is inspired from. I can elaborate if interested.)

Ultimately, I have to agree with other comments saying this will be more useful for checking than specifying a program.

[EDIT: fixed code formatting]

</comment><comment><USER33> You already said you will be looking and SQL closer, so let me give a few pointers here.

Consider that in SQL you specif how to store data, including indexes, and then what you wan to get, but not how to get it. In other words you specify data structures and the engine picks a set of algorithms and builds an execution plan. The next logical step would be for the user to specify the schema, while the database engine is not only choosing algorithms but also reshapes your data into data structures that would allow optimal algorithm selection later on. At first it would only build new indexes, but then it could also start employing partitioning and ultimately move to normalization/denormalization.

Secondly, be on the looklout for when a problem reminds you of a SQL query. For example, a mail application looks like a query where you specify which fields from which emails you want to see and in what order, and on top of that query result you're specifying a style sheet, CSS on steroids, to make it render pretty. Style sheet is also declarative, not imperative! I am presently working on a framework that follows this model to facilitate iPhone app development, and it's looking pretty good already.

</comment><comment><USER39> I think we're overlooking the fact that programming is hard per se. A program runs on a computer which is nothing more than a calculator with storage and peripherals. In the end everything is just numbercrunching. The whole point of a programming language is to overcome the "old days" where you had to use a computer for what it IS, namely a calculator. We did quite well regarding hiding stuff but as a consequence we lost control over parts of the machine. Setting a pin on your LPT-Port on high in Python? Just no. Doing the same thing in this guys "language"? I don't know if that will be included or even possible (if it is please tell me).

But in the end you're still writing code for a calculating machine. Things will never be as easy as "Sort this array by comparing each element for size" because the underlying hardware still is the same. Unless you figure out how to make that easier, for example by intorducing a "qsort [array]" assembler instruction within a processor, that has hardwired Quicksort logic, you're going to stay where you are: In the low-level lands of C and FORTRAN with some nifty little masquerades, like Python, for the same stuff all over again.

<comment><AUTHOR> Programming will definitely remain hard and thought-requiring. I just don't want to waste another minute of my precious life thinking about edge- conditions at the start and end of a list or array. And I don't want anyone else to do so either. I'm sure we'll get there some how, although it definitely may not be me, but doesn't stop me from giving it a try!

</comment></comment><comment><USER40> [http://lukepalmer.wordpress.com/2012/01/19/sentences-and- par...](http://lukepalmer.wordpress.com/2012/01/19/sentences-and-paradigms/) is an interesting look at parallels between declarative programming (as in Prolog, or in jng's pseudocode language) and functional programming in languages like Haskell or ML.

</comment><comment><USER41> So imagine you define some amazing PSL (Problem/Solution Language). It let's you "Code" by defining the problem and solution, ie:

        Define SORT: Given SET(x), Find SET(y) Where [For i in y] y[i]>y[i-1].

So now you're feeling all giddy because this amazing language is so awesome-- you didn't have to specify how to do something step by step. You just declared what you wanted, and it gave it to you. Fantastic.

But then, now that you have defined this "SORT" routine, you still need to use it. You end up with the following code:

        Prompt SET.
        SORT SET.
        Display SET.

Shit! You're defining steps again.

You ultimately can never achieve your goal, because you will always need to define some steps. All programs of significance are a composition of tasks which need to be identified, and executed.

<comment><USER42> There's a bug in your definition of SORT.

The OP complains that it's easy to write imperative and functional programs that contain bugs; I don't think that declarative programs are fundamentally better.

<comment><USER41> Tongue in cheek, you don't know how PSL handles negative array bounds--perhaps it's smart enough to know how to handle the first element.

<comment><USER43> It has a much, much worse bug than the negative index: your definition of sort says "given x produce a y which is sorted and _completely unrelated to x in any way_ ". The implementation "given x return y=[1,2,3]" satisfies your specification...

<comment><USER41> The negative index issue is a valid bug concern because when I was typing out the _fictional_ PSL code I missed the negative index hence my reply that said language could be smart enough to handle that (ie, tongue in cheek). However, your bug is not legit, because you've introduced extra wording for an assumption you can't make since you're not the creator of my fictional language. For PSL, "Given SET(x), Find SET(y)" implies that SET(y) is a resultant set of SET(X). SET(y) could have a case where the elements are different than SET(x) but only where explicitly indicated by the requirements. The requirements specified by the where clause in this case do not dictate a transformation on the elements, only a re-ordering. So no, it's not a bug.

It's funny that despite it being a passing pseudo example to make a point about programs being compositions of tasks that I'd be this vested in defending the code correctness of my pseudo code but I can't help it.

</comment></comment></comment></comment></comment><comment><USER44> Jon, you're looking for haskell. But there's a catch -- programming in haskell is HARD, and expert haskellers are the best of the best, thus EXPENSIVE.

many customers don't want correct programs, they want cheap programs. Some people in finance care about correctness, maybe you should check them out.

<comment><AUTHOR> Dusting, thanks for the suggestion, but I'm not looking for Haskell. It's not what I'm looking for. What I want is declarative, not functional. There is an inherent limitation in the functional approach, I hope to cover why that is in a further article in the series.

I am not after correctness per-se. I am actually more after "cheapness" at least in programmer time. But I do think a lot of the programmer time is spent in things that a solution-describing approach removes, and correctness gets a ride.

Hopefully I am in a right path in that direction!

<comment><USER45> I have often heard Haskell called declarative, it all just depends on how you go about writing your Haskell code. Theoretically Haskell is "time cheap" it just requires more time upfront. That is not to say Haskell is exactly what you are looking for however, I think it is at this point, the closest useful language to what you are looking for.

</comment></comment></comment><comment><USER46> Are you trying to reinvent prolog? Also, I don't really understand how a compiler for your language would actually figure out the _efficient_ algorithms for getting the answer.

EDIT: I see there's already a bunch of people talking about prolog here. Anyway, the second question is still open :)

<comment><USER47> I have a feeling the second question will stay open. :)

<comment><USER46> I believe so too. To choose an efficient algorithm, you have to know things about the input data. Even with the sorting example, this is so; e.g. if you place a certain set of constraints on the input, its possible to sort in linear time. But the compiler has no way of knowing such things, unless, of course, the language has some means of expressing constraints on the input in a manner that is digestible by the compiler...

<comment><USER48> I don't think this is that hard of a problem. The compiler could have a set of algorithms, each controlled by a set of constraints. Basically the job of the compiler would be to "normalize" a set of constraints such that they can be matched to an equivalent set of constraints from its algorithm toolbox.

From the sorting example, the compiler could recognize that the constraints given match the problem of sorting. If the input data also had the constraint of "integer between 0 and 10,000", it would recognize this as a subset of sorting that could be handled through a linear time algorithm.

</comment></comment></comment></comment><comment><USER49> As an undergraduate, I used a research project at my university that did more- or-less what the author describes.

The system was an "animator" for formal specifications called Possum: <http://ww2.cs.mu.oz.au/~tmill/tgf/index.html>

Possum would allow you to directly execute a program that was formally defined using the Sum formal specification language. While it was not able to handle every possibility, I found using it to be a really magical experience.

However, in practical terms, the idea of declarative programming has just as many difficulties as ordinary programming: how do you know your specification is correct? how do you debug a logical expression? how do you know your specification matches what you want it to do?

<comment><AUTHOR> I will look at the project, sounds interesting. If it's like that, I can only

I know declarative doesn't solve the problem of correctness. It's really difficult to define correctness per se! But what such an approach really gains is that you can forget about off-by-one errors, unforeseen cross-conditions in stupid details, so much repetitive code, etc... you will still have the higher-level problems of programming, the efficiency problems, the spec- correctness problems, etc... but nowadays you have those, plus the nitty- gritty details of programming in our paleolitic languages!

I know there are a few initiatives in this area, I really want to have an everyday tool that allows using that kind of approach for everyday problems. It will just make everything so much better for us programmers!

</comment></comment><comment><USER50> The one_to_one_equal example only works for sets, it looks like. This is a popular example in the Prolog literature, where they say

        sort(A, B) :- permute(A, B), ascending(B).

(permute corresponds to your one_to_one_equal.)

</comment><comment><USER51> I'm curious about how a compiler can infer the algorithm for sorting. I've played with machine learning algorithms like neural networks and genetic algorithms which might be able to do some of this, but what other things are out there?

<comment><AUTHOR> I will cover my approach to that in later posts in the series :)

</comment></comment><comment><USER19> If I were you, I'd look for a space that is more restricted than general- purpose programming but still rich enough to do nontrivial computation. If you can find interesting invariants that are true of all computations in such a space (but not true of all programs in general), you have a hope of leveraging them to create both a powerful declarative language and an efficient implementation for it. The space of all programs in general is too large for this.

This is why SQL and spreadsheets have been such successes as declarative programming. If you find another such model, that could be a big deal.

</comment><comment><USER52> Sorting is not a good subject for discussing an advanced/intelligent programming language. Try to solve hard problems which are difficult to solve with current languages.

Declarative programming can be very helpful in domain specific tasks. You start with a set of constraints that the computer can understand, and then the core AI will automatically give you the answer. Perhaps then a lot of common people can start to tell computer to do things for them without writing a single line of code, but they need to talk in the languages that computer can understand.

</comment><comment><USER53> Have you read Elements of Programming by Alexander Stepanov (he of STL fame) and Paul McJones? They take a similar approach as your examples except instead of declarative generation of code they break things down to very rudimentary mathematical definitions of operations and concepts and then build more complexity by bolting them together. Not terribly different than normal programmatic problem decomposition, but they tend to take it a little lower (or perhaps to a higher level of abstraction).

</comment><comment><USER54> Two quick things jump out at me: (1) This is TDD, without the implementation part of it. (2) There _have_ to be some halting problem issues in here. It's actual realm of usefulness is constrained somewhere, though I'm not exactly sure where.

That said, I've worked with modeling languages along the lines of GAMS that work in this sort of manner. Define a problem - a set of constraints - equations, inequalities, parameters, and the like - push a button, and out pops a solution. Damn useful.

</comment><comment><USER55> I think it is possible to write something better then prolog but I dont think we will ever write all programmes this way. Dont get me wrong using declerativ systems to help you is often helpful but I doute we will ever programm everything like this.

Even if we did we would still have many of the same bugs, many bugs we have to day are not details of algorithems the are problems in our own head, you can divine a system the way you want it an still get the wrong answer.

</comment><comment><USER56> I'm not convinced this is possible for anything but a small subset of programs. Such a compiler reasoning about a Turing-complete program would certainly be 'AI-hard', as it would have to essentially be a programmer itself.

What I'm interested in is making proof assistants easier to use by improving their inference capabilities, so programmers can verify the correctness of their programs without having to put in orders of magnitude more work...

</comment><comment><USER57> I spent some time using systems that seem to do what you are trying to do. One of those is Specware (<http://www.specware.org/>). When I used Specware their tutorial had a sorting example just like yours but it seems in their latest tutorial they have a different example. My experience is that that those systems are very hard to use and very limited in their capabilities.

</comment><comment><USER58> Sounds like a combination between flow-based programming and declarative programming. Perhaps a possible approach would be to write a flow-based programming language with some methods marked as "pure" which statelessly map input to output and others able to be written in a more traditional, imperative, less functional style?

In a flow-based architecture, it seems to me that by design, almost every component in the flow would be testable.

</comment><comment><USER59> We all want, buddy. Now go implement it :)

<comment><AUTHOR> Sure! I will try to :) I will do so in public, so that I can get as much help as I can.

<comment><USER51> I'll help what I can. I had a very similar idea, sparked by a conversation in college with a fellow CS major. He was saying how much he liked to program and I said that I hate it. My reasoning was exactly the same as yours, but perhaps broader.

I'd like it if we could get to the point where you say very plainly what you want in your native language (English for me), and then that is interpreted and perhaps more questions are asked by the program until it can implement what you meant. Obviously the problem of NLP is a massive one that will likely never have a perfect solution. There are also the problems of logical inference (if you think this is easy, look up the CyC project). I don't think we should let the current constraints of technology hold us back from designing something better, however.

</comment></comment></comment><comment><USER6> > There was recently a very interesting post by John Carmack about static code > checking, where he cited a great tweet by Dave Revell on code checking: "The > more I push code through static analysis, the more I'm amazed that computers > boot at all."

Wrong attribution. That tweet was originally by Ryan Gordon (@icculus).

<comment><USER60> I took a course on magnetic storage technology and now I'm perpetually amazed that computers even work.

</comment></comment><comment><USER61> Isn't this just Prolog?

<comment><AUTHOR> OP here. It's not just Prolog, but indeed Prolog is the closest thing now.

<comment><USER62> How is it not just prolog?

At the very least, a constraint programming language would be a good basis to start with here.

<http://en.wikipedia.org/wiki/Constraint_programming>

<comment><USER63> I don't think there's any existing programming language that does what OP wants.

But Prolog is far from the last word on the subject. For one, it is basically untyped. There are dependently-typed programming languages that can express non-trivial properties via types, which the compiler can check (see ezyang's example).

See also:

[http://stackoverflow.com/questions/2829347/a-question- about-...](http://stackoverflow.com/questions/2829347/a-question-about-logic- and-the-curry-howard-correspondence)

</comment></comment></comment></comment><comment><USER64> The thought reminds me a bit of what Charles Simoniy has been trying to do for years with Intentional Software

<comment><AUTHOR> I know about that initiative! I think it points in the right direction. I think the approach they took based on DSLs and editors is what doesn't work.

</comment></comment><comment><USER65> Here's a Prolog implementation of bubblesort (taken from Bratko's Prolog Programming for AI).

      bubblesort(List, Sorted) :-
          swap(List, List1), !,
          bubblesort(List1, Sorted).

      swap([X,Y|Rest], [Y,X|Rest) :-
          gt(X,Y).
      swap([Z|Rest], [Z|Rest1]) :-
          swap(Rest, Rest1).

</comment><comment><USER66> You are missing the point. The point is not for everyone to write perfect code in some mythical perfect language. They can't do it and there isn't any such language. The solution is for everyone to re-use the nearly-perfect code we have that is written by the few who can write it.

</comment><comment><USER67> A thought-provoking post.

(More detailed thoughts left as a comment on the post itself.)

</comment><comment><USER68>
        def factor(input) = prime:
          input % prime = 0
          prime > 1

Even worse...

        def fermat() = a, b, c, n:
          a^n + b^n = c^n
          n > 2

</comment><comment><USER69> My question: what can this _not_ solve? What problems could a human solve that this language couldn't?

I think that would help me get a better idea of what you're trying to accomplish.

</comment><comment><USER70> Sounds like a delightful blend of Agda and Prolog.

</comment><comment><USER47> Best of luck, though this strikes me as a "Everyone should have a pony!" sort of thing. You're basically asking someone to build some magic compiler that handles all the logic AND does so efficiently. It seems if this were possible it would have happened by now.

</comment><comment><USER12> Jon,

I think you're on the right (or at least an interesting) track here, one that I have been thinking about a bit myself.

For me these thoughts came after working for a few years on a clean room reimplementation project for a software component with a large set of automated tests (thousands). Slowly I got the feeling that each of those tests could be seen as an image of an unknown object, and that my job was to construct that object. Looking at just a few pictures it might be easy to find an object that could be projected onto planes to create those images, but looking at thousands of images it can be a very intellectually challenging task to find a single object that matches all of them. It's also a problem that has far more dimensions than the 3D world we live in.

In agile programming you may call those images "user stories". Agile basically says "there is no way to know all the images beforehand (because that depends e.g. on a market response), so just start with the most important images and build the simplest possible object that fits with them". Each time you add in a new image there's a risk that you have to make major changes to the object (to make it fit with the new image while remaining "compatible" with the previous images). That's what's called refactoring.

The really irritating thing with programming is that the effort of building a program does not scale linearly with the number of images (at least not in most cases). There is really no limit to how much work you may have to do to make the object fit with one or several new images, while remaining compatible with all the previous ones. That's because imperative code is a description of the object and not a description of the images.

What a programmer does is really solving the global optimization problem of constructing a minimal object that fits a number of images, and then writing that down in code. The better the programmer the more images they can handle, the more beautifully the images will match the object and the simpler the object will be.

It must be possible for a machine to do this. Don't get me wrong; it's a formidable optimization problem - and there are certainly sets of images for which no (reasonable) object can be constructed - but in principle it must be possible.

In fact there is a machine that does this for simple x-ray images and physical 3D objects: a Computerized Axial Tomography machine, or CAT scan as they call them in hospitals. Basically, what these machines do is take a series of x-ray images of an unknown object and then compute "backwards" what that object must be like on the inside in order to have created those x-ray images. I bet a really good programmer would be great at manual tomography, and that when you have implemented your new programming paradigm it will be a sort of CAT scan for software. :)

I'd love to talk more about this on occasion!

</comment><comment><USER71> "Programming is broken. Completely broken."

What something starts like this it usually translates to: "I don't understand pragmatic hardware/complexity/market etc constaints and engineering compromises".

<comment><AUTHOR> Might be. Hopefully not. I have done a huge amount of programming though, so it's not like the reflection comes from ignorance.

Under the mess of spaghetti code that anything out there is now, I see a clean structure of code trying to come out. I want to help it come out to the surface.

<comment><USER72> Generally, what I've found is the reason for the big pile of mess is not that the methodology is wrong, but quite frankly the programmer didn't know what needed to be done until the program was completed. Even with a complete set of requirements, if you are writing anything of any complexity then by definition it hasn't been done before. But you can usually take an existing program, re- define the specification, and re-write the code in a much cleaner and correct fashion (assuming you avoid the temptation to add new features at the same time).

</comment></comment></comment><comment><USER73> My vision of the future for programming is a room with 3d virtual reality goggles. Monitors are nowhere to be found, lines of code in files are presented to you on virtual monitors around you in a 760 degree panoramic view, above, below, and THROUGH. Our brains are evolved for a 3d world, not the lines-of-code on printed page world.

Commands like "instantiate new object", "add items to object", "do an sql query", "run sql query", "sum the integers", suck. These commands will be represented in 3d space like a flow chart with general directives. When you want to "zoom in" on one of the boxes, you can see the particulars of how it takes place. Zooming out shows you a perfect representation of the general directives, zooming in takes you to the nitty gritty commands, and zooming in further shows you the bits being shifted around on the hardware.

<comment><USER74> that's actually not that far from what programming with well-designed libraries and abstractions is like now. at least, I know I have a more-or-less 3d visualization of my program in my head as I'm working on it... but that could be just me.

</comment></comment><comment><USER75> Yup. He's right.

The future isn't going to be programming digital machines anyhow. Its going to be programming living matter.

But that's another story.

</comment><comment><USER76> You have to define the way that the sort should work though, and at that point, what you really want is functional dependencies. Coq, Idris, Agda, etc, etc.

Not a lot of fun to learn how they work, but they're really quite impressive.

</comment><comment><USER77> Bullshit. Sorry, but there's nothing essentially "broken" about programming. You don't like it, but it's a model that works.

Until you create a true AI, this is the way things are. It's not broken, it's apples and oranges. Instead of fixing programming, fix your way of looking at it.</comment>


Employee #1: General Assembly - <AUTHOR> http://themacro.com/articles/2016/07/mimi-o-chun/ <comment><USER2> > Mimi : As a first employee, for better or worse, the thing about joining a > startup is that it's not a meritocracy, right? If you're disappointed in a > decision that's made, or if you feel the founders aren't leading the way > that you would lead an organization, it can be incredibly frustrating. It's > actually tricky to be a first employee because you identify both with the > founders and the employees.

Best line in this series so far. If you are going to be an early employee, make sure you are compensated for this. Even if the structure is "flat" between founders and employees at some point in time, it can always change, and the founders always win.

There is nothing wrong with this, it is their company after all. But this is a huge point of frustration for early employees (something I've experienced and seen experienced by many others), and make sure you price this inevitable frustration in when deciding if you want to take an offer to be an early employee at a startup. If you are an early employee and want to get something big done you'll have to convince the founders, but if the founders want to do something big they don't have to convince you.

</comment><comment><USER3> Step 1. Befriend the scion of one of America's wealthiest billionaire families.

<comment><USER4> This comment refers to the Adam Pritzker and the Pritzker family [1].

> Mimi : I met Adam [Pritzker] while I was working at IDEO.

I didn't know about this family but I've visited Chicago enough to be familiar with the Pritzker Pavilion at Millenium Park.

[1] [https://en.wikipedia.org/wiki/Pritzker_family](https://en.wikipedia.org/wiki/Pritzker_family)

</comment></comment><comment><USER5> They seem to have pivoted from running a co-working space to lead generation for lenders.[1]

[1] [https://generalassemb.ly/apply/financing-your- education](https://generalassemb.ly/apply/financing-your-education)

<comment><USER6> Sounds like they're finally Making Something People Want. Coworking spaces only seem like a big opportunity to tech industry navel-gazers.

Lending, on the other hand, is the world's third oldest profession (after prostitution and politics).

<comment><USER7> Coworking has shifted to office lease arbitrage for small companies and large companies. It is no longer about just freelancers. Private offices with some modicum of sound and visual isolation are on the rise (WeWork et al.). Small companies get the flexibility of not being locked into a long-term 5-year lease, with the ability to scale (add desks). Large companies, instead of leasing their own space for a satellite office, can just "take over" a large portion of the desks of a local coworking space. Both get the benefit of sharing common space like lounges/kitchens with others they don't work (breaking out of their bubble).

</comment></comment></comment><comment><USER8> Wow, before reading an HN comment, never realized to associate Adam Pritzker as being part of the Pritzker family.

Not relevant but I actually work out of the GA Chicago shared space every now and then.

Never thought to connect those dots...just super interesting and impressive. L

This will probably get buried but wonder if GA is in the market for picking up some tech ed bootcamps</comment>


Vladimir Komarov went on a space suicide mission to save his best friend, Yuri - <AUTHOR> https://twitter.com/DMOberhaus/status/1116939111368097792 <comment><USER2> related discussions:

2011: [https://news.ycombinator.com/item?id=2342677](https://news.ycombinator.com/item?id=2342677)

2016: [https://news.ycombinator.com/item?id=11072718](https://news.ycombinator.com/item?id=11072718)

mirror: [https://threadreaderapp.com/thread/1116939111368097792.html](https://threadreaderapp.com/thread/1116939111368097792.html)

<comment><USER3> Top post on the 2016 version of this repost, links to an NPR article that debunks the whole thing: [https://www.npr.org/sections/krulwich/2011/05/03/135919389/a...](https://www.npr.org/sections/krulwich/2011/05/03/135919389/a-cosmonauts- fiery-death-retold)

<comment><USER4> "Debunked" is a pretty strong word. The disputed elements of the story are mostly based on the account of one person (Venyamin Russayev, the KGB minder assigned to Yuri Gagarin) and they are contradicted by official transcripts and the memoirs of the people involved.

But, this occurred in the USSR, so I don't think we really know what happened with any high level of certainty.

</comment></comment></comment><comment><USER5> I highly doubt this spin on the story. Although the structural problems with the Souz mission are well known in retrospect, none of these problems were seen as a huge threat to the mission or the safety of its pilots, more than any other mission. Every cosmonaut was an extreme asset that would never have been disregarded with the discrepancy that this story portrays. Source: Grandfather was one of members of the first cosmonaut squad. Kamarovs death was a huge tragedy caused by a miscalculation, as opposed to neglect.

</comment><comment><USER6> This is the most tragically inspiring story of the modern era.

</comment><comment><USER7> More on this informative site by a Russian space geek now living in the US:

[http://russianspaceweb.com/soyuz1.html](http://russianspaceweb.com/soyuz1.html)

</comment><comment><USER8> Damnit I hate crying to stuff I see on twitter.

</comment><comment><USER9> The Pence remark was completely unnecessary and infuriating. I thought this was some educative chain of tweets and it ended in "orang man bad". Very anticlimactic.

<comment><USER10> It's the _entire_ point of the piece. And it's not really all that much of a political statement. The message is pretty clear: don't send people to die in space for stupid-ass political reasons.

<comment><USER11> As a counterpoint, sending people to space for stupid-ass political reasons resulted in one of humanity's greatest technological achievements that allowed us to put the plaque up there in the first place.

<comment><USER12> Note the subtle difference between "sending people to space" and "sending people to die in space". (As explained in the thread, Soyuz-1 was hurried to launch for political reasons despite being unspaceworthy)

<comment><USER13> That doesn't counter GP's point, it reinforces it. The successful moon missions were all launched for political reasons, too.

<comment><USER14> He is pointing out that there is a difference between sending people to space for political reasons and sending people to space for political reasons in known to be unsafe aircraft knowing that the possibility of death is significantly above average. One is risking someone's life for achievement and glory. The other is killing someone because your fragile ego can't take the hit of not being ready on time for some stupid political deadline.

<comment><USER13> Yea, the point is why is there a random political shot at Pence? It makes no sense. It's shoehorning a totally petty and not relevant potshot. Every space mission has been "political", and of course everyone agrees you shouldn't do a bad job at it.</comment></comment></comment></comment></comment></comment></comment>


Ask HN: Is a static site hosted on AWS S3 'hackable'? - <AUTHOR> Let's assume both the domain registrar's and AWS pwd are super strong, and that static html files are uploaded to S3 from a 'clean' computer and only from that computer. Can the site be hacked? Defaced? DDOS? <comment><USER3> Not through a web application vulnerability it can't (there is no application code to break!).

Your biggest issues with that setup will be:

1\. Credential and permissions management. Don't lose control of your access keys to AWS! Set up "MFA" at the very least. If you use your AWS account for other purposes, use IAM to ensure that other users cannot access the S3 bucket with the site in it.

2\. Getting that green lock (e.g. HTTPS). You can pay $800/month or something insane to Cloudfront to get a custom TLS cert on their CDN, or you can get Cloudflare's "universal ssl" for $20.

3\. DDoS is not really a concern. It would be nearly impossible to DDoS an HTML-only website hosted in an S3 bucket. I'd really like to see someone try. The only thing that could happen is you get charged a bit more that month while people are DDoS'ing you. But if you're behind a CDN like Cloudfront or Cloudflare (which can cache _everything_ because it's plain HTML), then the impact would be reduced.

4\. Your Registrar suddenly becomes a huge risk. Make sure you use a secure domain registrar (ie. NOT GODADDY!), that the registrar has a "Registrar-Lock" turned on for your domain, and that your account with them has 2FA. If you screw that up, then someone might be able to socially engineer the phone rep at the registrar to transfer the domain, change the nameservers, etc. This happens depressingly often.

5\. 3rd party services you use could get hacked. You'll likely import a bunch of JavaScript from other websites onto your own. If any of those websites get compromised, it might affect your website too. Make sure to use 2FA on every service you can ([https://twofactorauth.org/](https://twofactorauth.org/)) and minimize the amount of JavaScript/fonts/css/whatever that you load from 3rd parties (re-host it locally).

I highly recommend setting up websites this way. It's fast, easy to maintain, and incredibly secure. We do this for the trailofbits.com website and we're very happy with it. Jekyll FTW.

ps. don't forget you can use Github Pages too.

<comment><USER4> "Not through a web application vulnerability it can't (there is no application code to break!)."

This isn't true. Data is being served up to web browsers over port 80. That's a web server. It might not be like any web server you've ever run ... it might not be apache or thttpd or zeus or whatever ... but there _is_ code that is spitting out HTTP over port 80/443\. That's a web server.

So let's say the S3 data is immutable, as far as the "web server" and the attacker are concerned ... that's very nice, of course, but if you have control over the web server, instead of parlaying that into control over the underlying data, just tell the web server to spit out other data.

The S3 content is still safe and unchanged, but your website is nicely defaced, since your "amazon s3 webserver" exploit is just feeding it new content.

(drastically oversimplified, of course - but be clear, if there is a web server (and there is) there is web-server-attack-surface)

<comment><USER3> No one has the source or binary code for the webserver AWS uses. It would be nearly impossible to write an exploit for it blind like that. Even if you did write it blind, it would be a slow process that likely requires bruteforcing parts of the address space and there is no way you could do it without alerting Amazon.

tl;dr what you described is possible, I guess, but so incredibly unlikely it's not even worth thinking about.

<comment><USER5> Nobody has the source or binary code for dozens of proprietary systems which get hacked on a regular basis.

In fact, I'd argue that most prominent hacks are of systems which are _not_ open source.

</comment></comment></comment></comment><comment><USER6> How I would hack it (if I was evil and cared enough):

1\. Gather info from whois DB, google search, site spidering, going to your house and looking through your trash.

2\. Ring you up - Hello I'm Joe from the tax department/credit card company/bank we need to confirm your address .. give your address .. could I please confirm you are the credit card holder, I just need the last 4 digits

3\. Ring your friends, family and business contacts - use smooth talking to gather as much info as possible.

4\. Ring up Amazon - oh yes I am mister XXX, I forgot my password, please can you reset it. If they don't I'll try to guess information, and glean any info out of the replies.

5\. Ring up your email provider and do the same

6\. Keep on ringing about 8 hours apart to make sure I get different teams, so it's fresh each time, until I had enough info to get access to the account

7\. Make sure to delete all backups

8\. Deface to my hearts content - change all the passwords, blah blah

\</comment><comment>--

This is the info I'd try and gather:

* Name - probably from whois

* DOB - probably from public records search - or ringing friends

* Phone - probably from your trash or mailbox

* Last four credit card digits - probably will get from your trash, or tricking you on the phone

* Date of last payment - Probably from tricking Amazon

* Password bits - pet's name, girfriend/wife/child names and ages, keylogger in an email I sent you

<comment><USER7> As always the weakest part of any security system is the people within it.

</comment></comment><comment><USER8> Presuming you don't misconfigure S3, this is one of the least "hackable" ways to stand up a website.

</comment><comment><USER9> Nobody mentioned it, but having a static website vs. a dynamic one, leaves your contents exposed to scraping, proxy hijacking and framing. While the latter can be mitigated by a one-line script, IMHO you basically can't defeat scrapers and proxies _client-side_ -only - modern scrapers are able to run any javascript if needed. So expect a plethora of bad/spam links, unauthorized copies of your pages - etc. Especially if your contents attracts a lot of traffic and/or you have many competitors.

I recommend you watch periodically for your contents to pop up on random domains (you can google for your exact texts) and file your DMCA requests as soon as they appear.

It might help to use <base> tags, absolute URL links and the likes in all of your web pages, as well as mention your domain both in textual contents and images (logo?) - that actively discourages "lazy duplicates" of your pages (but not copy/pasting your articles on a different site by hand).

Just my two cents.

<comment><USER10> Whether your site is static or dynamic doesn't changes anything: you still have to expose content to your users.

Furthermore, any good browser will respect [Content Security Policies]([https://developer.mozilla.org/en- US/docs/Web/Security/CSP/CS...](https://developer.mozilla.org/en- US/docs/Web/Security/CSP/CSP_policy_directives#frame-ancestors)) and not load frames to your website.

</comment></comment><comment><USER11> A static website that is served over HTTPS only[1], and lets say for the sake of argument, HTTPS secure enough not to MITM, and no services outside of your control (DNS, etc) are MITMable, and your credentials to Amazon can't be stolen...

The only way to hack your site is to actually be at Amazon with access to whatever disk array stores your data. As in, I'm pretty sure "inside job" is the only route left.

[1]: [http://www.troyhunt.com/2015/06/understanding-http-strict- tr...](http://www.troyhunt.com/2015/06/understanding-http-strict- transport.html)

</comment><comment><USER12> Sure, strong credentials can still be leaked due to a simple phishing email to the right person.

There is no such thing as an unhackable system, only more and less difficult to hack systems.

<comment><AUTHOR> For the sake of argument let's assume credentials are not hackable. My question: is the site itself hackable?

<comment><USER13> Yes - but probably only for quite small values of yes.

Who's your threat?

Your attacker would need a backdoor or 0day into AWS itself - which while you'd be a fool to assume such a thing doesn't exist, its very unlikely to get "burned" on a run-of-the-mill website hack or defacement.

If your site is ISIS or Wikileaks or DeSpiegel or The Guardian (or some other well known considered-by-the-NSA-to-be-a-terrorist-organisation) - <USER2> wouldn't bet against the NSA having firmware backdoors in the network cards and hard drive controllers underneath AWS (as well as the hypervisor and all the available deployed virtualised images, and all the internal Amazon network hardware and the core routers at both ends of Amazon's datacenter's optical fibres).

If your threat model is closer to Anonymous or GamerGate, you're probably more at risk of your "unhackable" credentials being social engineered than a purely software hack.

But in a purely binary interpretation of your question - Yes. _Everything_ is "hackable" given a sufficiently motivated and resourced attacker.

<comment><USER14> I believe this comment is the most correct answer.

Anything is theoretically "hackable" given enough time. What we can do is to make things difficult enough to hack that they are not worth the time.

</comment></comment></comment></comment><comment><USER15> What steps would you go through if you lost the passwords to either the domain registrar or the AWS account? How does the account recovery work?

Usually it relies on ownership of some third account, e.g. email. Okay, what's the recovery process for the email account? Receiving an SMS to a particular phone number? Okay, what's the recovery process for that phone number? What's the process to get the phone number redirected?

At some point you're going to end up being able to ring up a number and tell someone a name, address, date of birth etc. Best case you ring up and they say okay we'll mail you something. Or they make you come in person and sign.

Customers lose credentials constantly, and won't tolerate being told that this means their account is unrecoverable. So there is almost always another way.

</comment><comment><USER16> You can position your site behind CloudFlare's DDoS protection, eg. [http://ht.transparencytoolkit.org](http://ht.transparencytoolkit.org)

</comment><comment><USER17> A bit out of the topic,

But I think generally if your site is not using top 3 popular CMS-es like WP, Drupal and Joomla - this will make 99.9% of attackers to move on and scan for easier targets elsewhere.

</comment><comment><USER18> Endpoint security.

SSH guessable passwords.

HTTP daemon vulnerabilities.

Any other daemons running.

Resource exhaustion.

DNS.

Client MITM.

Route corruption.</comment>


3D printing with glass - <AUTHOR> http://www.economist.com/news/science-and-technology/21661577-fine-glass-latest-material-be-manufactured-additively-what-goes-around <comment><USER2> I wonder whether it would also work to heat glass powder with a laser to 3D print.</comment>


Al-Mabhouh Assassination - <AUTHOR> http://www.schneier.com/blog/archives/2010/02/al-mabhouh_assa.html <comment><USER2> The electronic lock hacking is interesting. In the video a screenshot is shown from what seems to be the lock server, detailing a failed attempt at reprogramming the lock. One of the commenters on Schneier's blog suggests that this was to make the lock block the victim's own key, allowing fake hotel staff to get close to 'help' him.

</comment><comment><USER3> The video Schneier links to is down; here's a mirror. <http://www.youtube.com/watch?v=JghQ0ZcRfQs>

</comment><comment><USER4> 15 more suspects named today brings the total to 26! [http://edition.cnn.com/2010/WORLD/meast/02/24/uae.murder.plo...](http://edition.cnn.com/2010/WORLD/meast/02/24/uae.murder.plot/)

<comment><USER5> Wow this is pathetic. Mossad basically exposed their entire staff for this one job!</comment></comment>


Breaking down the legal case against @BPGlobalPR Twitter feed - <AUTHOR> http://legalblogwatch.typepad.com/legal_blog_watch/2010/06/breaking-down-the-legal-case-against-bpglobalpr.html

<comment><USER2> This tweet was clever, but I couldn't help but cringe when I saw it going around Twitter, giving everyone a healthy dose of misinformation.

BP said fairly early on in the account's popularity (~20k followers is when I read about it) that they wouldn't take action against it. They even said it was a way for people to vent their anger. So it's not just a case of them not taking action; they've acknowledged it openly.

Any PR team worth anything would do the same in this situation. Bad PR teams might take it to Twitter (the account is clearly in violation of Twitter's terms). Only the stupidest, head-up-their-asses teams would let it go to court.

You could win this case easily in the court of law... but you'd lose hard in the court of public opinion.</comment>


Using wikis to manage software projects - <AUTHOR> http://blogs.activestate.com/2009/06/getting-started-with-project-wikis/

<comment><USER2> The HN heading is misleading, The article is totally NOT about using a wiki to _manage_ a software project, only _documenting_ a project. It's not quite the same thing.

</comment><comment><USER3> At my previous employer, we used a Mediawiki instance for approximately everything in projects: documentation, design sketches, work packages, internal tool FAQs and Howtos, project meeting minutes. Bug tracking, however, happened in Bugzilla. We had a sophisticated naming convention for different types of pages.

In retrospect, I wouldn't track work packages and progress in a Wiki any more (instead use something that can actually add up the "time spent" column...), but the rest was fun to work with.

</comment><comment><USER4> I don't know if I agree.

The old generation of online project management software (see: dotproject) was awful, had no UX design, and probably was less efficient than a wiki. However, new programs like basecamp and thymer are much easier to use and add a lot to one's productivity. Certainly, it is much faster for me to create a thymer account than it is for me to use a tool that isn't meant for project management in the first place.</comment>


UBeam wireless charging demonstration - <AUTHOR> https://www.usatoday.com/story/tech/news/2017/06/01/ubeams-meredith-perry-shows-her-stealth-wireless-charging-technology-really-works/102336880/ <comment><USER2> I'm still extremely skeptical about this. The inverse square law is a huge hurdle, power delivery rate has got to be minuscule.

This isn't making a better charger, it's making better physics.

<comment><USER3> > power delivery rate has got to be minuscule

Amazing that during the whole presentation the word "watt" was never mentioned to describe the power delivered and at what range.

</comment></comment><comment><USER4> [http://liesandstartuppr.blogspot.com/2017/06/what-does-it- ta...](http://liesandstartuppr.blogspot.com/2017/06/what-does-it-take-to- switch-phone.html)

</comment><comment><USER5> No idea about speed of charge. That's what the game is right now. I used to have a phone with wireless charging but it was way slower than using the wired smart charger (the one with the tech from Qualcomm)

<comment><USER6> But if this was present in your office/classroom/car it wouldn't matter if it was slow, as long as it was fast enough to provide more power than you were using at a given time. I.e. It doesn't matter if it takes 5 hours to charge if you're around a source 8 or 9 hours a day.

<comment><USER7> Exactly. I have 2 wireless (Qi) chargers at home (1 at my desk, and one near my bed) and 1 at my workplace. Sure, the speed is quite a bit slower than wired chargers (charges from 0-100 in a couple of hours when idle, instead of < 1 hour when using a quick charge wired charger), but it's completely effortless to keep my phone on a charger almost constantly, so it's always near 100% when I need to leave with it. And I still have the option to use the wired charger for when I really need that quick boost, but I personally haven't touched mine in years.

</comment></comment></comment><comment><USER8> On one hand, it's crazy that they pulled this off, of the other hand, I'm not sure if I see the practical applications of near-range wireless line-of-sight charging</comment>


Lorem Gibson - Website filler text based on the works of William Gibson - <AUTHOR> http://loremgibson.com/ <comment><USER2> Great idea! Where are the verbs, though? Right now I can see from the JS source it doesn't use Markov chains [1], which could improve the result greatly. Good placeholder text in a language the user can understand (as opposed to simple Greeking [2]) is non-distracting precisely because it balances on the verge of meaning. From my experience I'd assume fake Latin and nonsense prose would both be less distracting than a soup of fascinating SF words.

Edit: Here's what a markov.js remix of William Gibson's _Burning Chrome_ looks like:

 _> Los Angeles was a dream, responsive to Deke's slightest thought. For weeks he systematically visited every boozy watering hole in the Fifties. Sometimes they'd run old eroded newsreels as filler on the museum's exhibits, a NASA Hasselblad recovered from the inside. It probably took all of Jackman's silent and vast and perfectly immobile bulk wedged into a gray plastic tiara. Tally Isham smiling up from the huge speakers. He sought her almost blindly on the edge of the tall drinks and paid. A big woman in green, and in- clined her head. She was everyone's giggling sister, in a way to trust in whatever context it encountered. Congratulations, I heard the woman said. That's the trouble with designer drugs; they're too clever. That stuff you're doing has some hard data, Toby; she's a hologram stuck behind my left ear, where they'd gone in to tell it to. Nobody at all. And I know you were looking for us, or for the road, admire the city walls, the high point of the hydrogen atom. Tsiolkovsky's radio telescope was tracking, relaying the signal to geosynchronous comsats that bounced it down to Plesetsk, where bulldozers were already excavating for a year later, when two leading firms had the exact change, unless he wanted and couldn't have, everything he'd had to jump with it, and it was relentlessly tacky: ephemeral stuff extruded by the open doors and watch the crowd pull me along, walking blind, willing myself to relax._

[1] For Markov chains in Javascript see, e.g., [http://www.owlnet.rice.edu/~cz1/prog/markov/markov.html](http://www.owlnet.rice.edu/~cz1/prog/markov/markov.html).

[2] [https://en.wikipedia.org/wiki/Greeking](https://en.wikipedia.org/wiki/Greeking)

<comment><USER3> I'm not using Markov chains, either, but my little word-babble script does fairly well when you feed it "Neuromancer":

He looked at the dance of the logics kinda bunker... It was no relationship with it... He looked like Armitage's toxin sacs. Professional pride, baby, I mean, these days? Get dressed, Case. Bus to his own laughter and smiled. Case plunged through the matrix below. He'd been slit. An AI's are still got it. Not the Baptist... Darkness there, Wintermute can. Case had the electric dance ten minutes to him... Cyberspace. A white teeth. Month later to the corridor. The phone bleated steadily. The Panther Moderns differ from the sides a burglar. He stood beside his own air in Straylight. Wintermute, Case peered at Case. You forget that made of the room. The Finn watched, then Armitage, come out. A man's voice, out of her, then. Behind the neural cutout. I think you're all together in cloud. But just above the terrorist has all of freckles.

<comment><USER4> In Gibson's world, I think the algorithm would have been obfuscated and released to the world in stages, anonymously, building up a cult following. Till one day it's revealed the algorithm has a hidden aspect which boots Wintermute.

</comment></comment></comment><comment><USER5> This is silly.

I've run a printing press in my mis-spent youth. And repaired photo-copiers.

One technique that is often taught is to turn the page upside-down -- in order that you not be distracted by the content.

You are, at that point, and your job, is to be only interested in the copy/print quality. Registration, blur etc. Not the content. That's the editor's job. Not yours.

The same may be said in this instance for the layout. That's your focus, or should be. That's the beauty of Lorem Ipsum: most people don't know Latin. If you want a change, make it Klingon, but retain the original use. That is, to check the page. Don't make it distracting by being readable or in any way comprehensible.

<comment><USER6> From the content side, having worked for a student newspaper and having been bitten by accidentally running a bunch of pictures headlined with "Inspiring 'Red Sox Win' Photo Spread Head", something we learned was to make filler text __look as obtrusive as possible __, so you remember it 's there and fix it before you go to press. Text with lots of "@@s" and characters with deep descenders were both helpful.

That said, I'm a big fan of The Onion's "Passersby were amazed by the unusually large amounts of blood."

</comment></comment><comment><USER7> I'm holding out for Lorem Ibsen.

</comment><comment><USER8> Where are all the one, two, and three letter words? Does Gibson never use "a", "or", or "and"? More importantly, how is this an effective placeholder for English text without them? This seems all too common in these alternative ipsum generators. I guess people aren't actually supposed to use them?

</comment><comment><USER9> These are fun, but I really don't understand why people refuse to release the source code for them.

<comment><USER10> I did one of these for The IT Crowd and released the source code - [https://github.com/kivlor/itcrowdipsum](https://github.com/kivlor/itcrowdipsum)

</comment></comment><comment><USER11> The content is distracting, which kind of defeats the point of Lorem Ipsum.</comment>


Your mobile and tablet customers now matter - <AUTHOR> http://blog.hipmob.com/post/36563355285/it-is-known-your-mobile-and-tablet-customers-now

<comment><AUTHOR> Ayo from Hipmob here - We've been watching all the news coming out about Black Friday/Cyber Monday and it's clear this will be a breakout year for mobile/tablet shopping. Will round up some more analysis later this week!</comment>


A natural nuclear reactor on Mars - <AUTHOR> http://www.lpi.usra.edu/meetings/lpsc2011/pdf/1097.pdf

<comment><USER2> The article describes a hypothesis based on probable distribution of the essential precursors, it's not a discovery, and there is no _observation_ of "A natural nuclear reactor on Mars".

The problem is not the article, the problem is that the submitter wrote his own title.

The article's title: "EVIDENCE FOR A LARGE, NATURAL, PALEO-NUCLEAR REACTOR ON MARS"

This submission's title: "A natural nuclear reactor on Mars"

Imagine how simple murder trials could be if lawyers were allowed to do this.</comment>