Loop invariants can give you coding superpowers - <AUTHOR> https://yourbasic.org/algorithms/loop-invariants-explained/ <comment><USER2> I think there's an opportunity to name variables after their invariant.

For example, in the typical sum example, the mutated variable is usually called "sum" or "total". But you could also call it sum_from_0_to_i, and then a reader can immediately see that the result is the full sum because i equals the array length. It's like a proof-by-variable-name.

The same trick tends to work as well with less trivial invariants. You get long variable names, but not horribly so.

You can do the same for the accumulator in a reduce/fold operation. Way too often variables in reduce calls (and other recursive functions) have _horrible_ names, making the entire reduce needlessly hard to understand. Naming the accumulator properly solves this:

Eg

        const total = array.reduce(
            (totalUptoPrevious, current) => totalUptoPrevious + current,
            0
        );

So much easier to follow! I've often had to look up, when _reading_ a reduce, which argument is the accumulator in this particular language or library. When you use the invariant for the accumulator name, you fix this entirely.

<comment><USER3> I mean generally I tend to prefer operations that are commutative and associative, so it really doesn't matter which one is the accumulator.

But I still really like your naming, it seems to bridge the gap in making reduces (which are conceptually cleaner) understandable to people who are used to iterative code.

<comment><USER4> Floating point sums are one example of a place where the only usable operator doesn't commute.

<comment><USER5> Which floating point operator is not commutative? Addition is.

<comment><USER6>
      (small number)+(small number)+ ... +(small number)+(much bigger number)

may not give the same result as

      (much bigger number)+(small number)+(small number)+ ... +(small number)

<comment><USER7> That is due to associativity. You're implicitly relying on a consistent left- to-right evaluation order, but the fundamental problem can be phrased as (using s for small and b for big):

       (((b + s) + s) + s) + s

Can be different to

        b + (s + (s + (s + s)))

This re-parenthesising is related to associativity, not commutivity.

<comment><USER6> _You 're implicitly relying on a consistent left-to-right evaluation order,_

That's standard, I think.

<comment><USER7> Indeed, it is standard, but eliding the parentheses doesn't mean they aren't implicitly there, and doesn't turn non-associativity into non-commutativity. Commutivity means that these are equal:

       (((b + s) + s) + s) + s
       ==
       s + (s + (s + (s + b)))

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> To sum (haha) up, I am no n00b.

However, I've looked over the page a couple times and still can't figure out the use of this, let alone what "superpowers" it may grant. It's just comments? The best parts I can see are the mostly-prose ones farther down that just say what the loop's supposed to do, but even those are just repeating the "input" and "output" info, really, which are sort of also comments I think. Some of this might be useful if you could express it in types or some other machine-checkable fashion, but in comments they're just kind of redundant and, like any comments, must be treated with suspicion anyway.

What am I missing? I just have no idea what I'm looking at here, or rather I _think_ I do but I'm entirely missing why it should be in any sense exciting which leads me to think maybe I don't.

<comment><USER9> You're not missing anything - it's 100/200 level computer science. (Freshman/Sophomore)

<comment><USER8> Yeah I'm just getting "if the loop can be expressed as induction you can... write that in a comment, for some reason". Alright, that's nice. Could we express that as tests instead plzkthx?

So I thought I must be missing something.

<comment><USER10> For the rare languages that support invariants (or has a library that implements invariants) you would write it as code rather than a comment and the compiler will compile time error if the compiled code is capable of violating the invariant.

</comment></comment></comment></comment><comment><USER11> The following anecdote is from [https://courses.csail.mit.edu/6.042/spring18/mcs.pdf](https://courses.csail.mit.edu/6.042/spring18/mcs.pdf):

The Invariant Principle was formulated by Robert W. Floyd at Carnegie Tech in 1967. (Carnegie Tech was renamed Carnegie-Mellon University the following year.) Floyd was already famous for work on the formal grammars that transformed the field of programming language parsing; that was how he got to be a professor even though he never got a Ph.D. (He had been admitted to a PhD program as a teenage prodigy, but flunked out and never went back.)

In that same year, Albert R. Meyer was appointed Assistant Professor in the Carnegie Tech Computer Science Department, where he first met Floyd. Floyd and Meyer were the only theoreticians in the department, and they were both delighted to talk about their shared interests. After just a few conversations, Floyd's new junior colleague decided that Floyd was the smartest person he had ever met.

Naturally, one of the first things Floyd wanted to tell Meyer about was his new, as yet unpublished, Invariant Principle. Floyd explained the result to Meyer, and Meyer wondered (privately) how someone as brilliant as Floyd could be excited by such a trivial observation. Floyd had to show Meyer a bunch of examples before Meyer understood Floyd's excitement — not at the truth of the utterly obvious Invariant Principle, but rather at the insight that such a simple method could be so widely and easily applied in verifying programs.

</comment><comment><USER12> If you're interested in playing around with loop invariants for non-trivial programs, I recommend the Dafny programming language which can automatically verify the invariants using SMT solvers. (Dafny is much more convenient that messing around with operational semantics in Coq.)

There's a tutorial + web interface at: [https://rise4fun.com/Dafny/tutorial/Guide](https://rise4fun.com/Dafny/tutorial/Guide). The official repository is here: [https://github.com/Microsoft/dafny](https://github.com/Microsoft/dafny) \-- you might want to switch to a local installation once the online tutorial whets your appetite. A good initial challenge, once you've gotten past the baby stuff in the tutorial, is implementing insertion and deletion on a binary search tree with appropriate pre- and post-conditions.

<comment><USER13> Yes!

Dafny puts this stuff front and center, it's all well and good to think about invariants, but if they're just expressed in a comment and not actually verified, they might as well be filler text!

If there's anything I hope the mainstream adopts at some point, it is some variation of what Dafny offers here.

</comment></comment><comment><USER14> I think what many commenters are missing is why this is important.

Invariants like this allow one to _prove_ the correctness of their code. That may not seem like a big deal most of the time, but imagine you're working on a satellite that's going to Jupiter, where a bug in the code could mean 10 years from now a billion dollar project gets scuttled. Imagine you're building subroutines that will go into 10,000 pacemakers next year. Imagine you're writing the navigation logic for a cruise missile.

There are many times where proving your code correct is taken very, _very_ seriously. Invariants are a tool to do so more easily.

<comment><USER15> To prove the correctness of your code you would write tests in similar way as SQLite does, no? They have, like, 100x more lines of code for tests than actual code they test?

<comment><USER16> Test cases do not establish proof.

There could always be another rare edge case.

</comment></comment></comment><comment><USER17> Lot's of material around on invariants for anybody interested

      Cornell:
      Invariants Playlist: https://www.youtube.com/playlist?list=PLTD_NtzzD4VC6l2uLdzbsm9wW21mMaWwj
      http://www.cs.cornell.edu/courses/cs2110/2017sp/online/loops/01aloop1.html
      https://www.cs.cornell.edu/courses/cs1110/2018sp/materials/loop_invariants.pdf

      CMU:
      https://www.cs.cmu.edu/~15122/handouts/01-contracts.pdf
      http://www.cs.cmu.edu/%7Efp/courses/15122-s11/recitations/recitation02.html
      (former CMU prof) https://www.youtube.com/watch?v=lNITrPhl2_A

Of course if you really wanted to understand the challenge of finding loop invariants, there's the software foundations series of books which even if I don't understand all of it, has been still worth my time to go through [https://news.ycombinator.com/item?id=19565365](https://news.ycombinator.com/item?id=19565365) and Cornell has some good introductory Coq material [https://www.cs.cornell.edu/courses/cs3110/2018sp/a5/coq- tact...](https://www.cs.cornell.edu/courses/cs3110/2018sp/a5/coq-tactics- cheatsheet.html)

<comment><USER6> Cornell:

[http://www.cs.cornell.edu/courses/cs2110/2017sp/online/loops...](http://www.cs.cornell.edu/courses/cs2110/2017sp/online/loops/01aloop1.html)

[https://www.cs.cornell.edu/courses/cs1110/2018sp/materials/l...](https://www.cs.cornell.edu/courses/cs1110/2018sp/materials/loop_invariants.pdf)

CMU:

[https://www.cs.cmu.edu/~15122/handouts/01-contracts.pdf](https://www.cs.cmu.edu/~15122/handouts/01-contracts.pdf)

[http://www.cs.cmu.edu/%7Efp/courses/15122-s11/recitations/re...](http://www.cs.cmu.edu/%7Efp/courses/15122-s11/recitations/recitation02.html)

(former CMU prof)

[https://www.youtube.com/watch?v=lNITrPhl2_A](https://www.youtube.com/watch?v=lNITrPhl2_A)

</comment></comment><comment><USER18> Loop invariants are a lot more intuitive if you convert your loops into while() form and then further into tail recursion - they turn into a proof by induction.

For the first example, consider:

      go (n, i, sum): given sum = 1 + 2 + ... + i-1 returns (1 + 2 + ... + n) :
        per cases of (i <= n)
          assume (i <= n) {
            note: (sum = 1 + 2 + ... i-1 -> (sum + i) = 1 + 2 + ... + i)
            return go (n, i + 1, sum + i);
          } else assume (i > n) {
            note: thus i = n + 1
            then: sum = 1 + 2 + ... + i - 1 = 1 + 2 + ... + n
            return sum;
          }

      sum n: returns 1 + 2 + ... + n = {
        note: 0 = 1 + 2 + ... + (1 - 1)
        // ^^^ Need to prove our precondition for go!
        return go (n, 1, 0);
      }

The other examples are a bit more complex for sure, but structurally similar. It's nonetheless easy to see how even very limited changes in complexity (such as the choice of loop indexing) can create quite a few pitfalls for truly rigorous proof!

</comment><comment><USER19> This is a great technique; I learned it in David Gries' Intro to CS at Cornell, but a lot of my classmates didn't think so, which elevated this to meme status at Cornell: [https://quotes.cs.cornell.edu/quote/1369/](https://quotes.cs.cornell.edu/quote/1369/)

</comment><comment><USER20> I think both for sum and max a better invariant should include the uncalculated part.

Sum(0..X)=Sum(0..i)+sum(i+1..X) with the moving i. So it is true at first because the unsummed part is empty and the missing part is the goal. Then we take away an i from the solution to-go pile and put it on the done pile and when the loop has done X the to-do pile is enpty and per invariant the done pile equals the goal.

Including the goal in the invariant sometimes gives you nicer properties. The equivalence relation often also implies an algorithm.

For max I would say the equivalence is: Max(0..X)=Max(Max(0..I),max(I+1..X)) This of course only easier if the max of empty is negative infinity. Then you can move the i from 0 to X.

<comment><USER6> You are merging the _invariant_ with the _variant_. The loop invariant is supposed to tell about the state of the variant at any loop start. The variant in this case is the tupple (i,sum). Your version does not say anything about the stepwise evaluation of the sum.

<comment><USER20> Of course this tells me about the state of the variant. And it's the same invariant the author uses but with the equation balanced differently.

The author's invariant is Sum = 1+2+..I Mine is Sum+(I+1)+...+X = 1+2+..I+(I+1)+...+X

That's both fine and tells us about I. I think the version that contains the value we want to obtain is easier to prove correctness for.

</comment></comment></comment><comment><USER21> Interesting article, but if one considers the possibility of overflow, the invariants in some of the examples aren't true.

Which doesn't mean the article is _wrong_ so much as it demonstrates one needs to prove the invariants are actually true.

<comment><USER22> Several static analyzers (like Frama-C [1]) can extract loop invariants and try to prove them, giving you the best of both worlds.

[1] [https://frama-c.com/](https://frama-c.com/)

</comment></comment><comment><USER23> This isn't necessarily a new idea. Tony Hoare discussed the idea for a while with his creation of Hoare triples; we are learning about the application right now in my Software Foundations course. It has a lot of interesting application within Coq for the analysis of the validity of programs, though. A good place to learn more about the application is here: [https://softwarefoundations.cis.upenn.edu/current/plf- curren...](https://softwarefoundations.cis.upenn.edu/current/plf- current/Hoare.html)

</comment><comment><USER24> Learned that and the "weakest precondition" stuff from Pierre-Arnoul de Marneffe in 1994 at the University of Liège (ULg). Very useful but we didn't grasp the full power of it at that time.

<comment><USER25> We had a course at Georgia Tech called "Introduction to Proofs" or something for the CS students that introduced these as well. But it was so early in our education and so separated from the programming courses that it wasn't obvious to most of us how it'd be used. It wasn't until much later that I understood these ideas and could apply them. It needed to be better integrated into the other courses (to provide motivating examples of these methods and techniques).

</comment></comment><comment><USER26> The article has invariants in comments. I like to put invariants in assertions so they get checked at run time.

I know this is common practice but I haven't seen it mentioned here and it might help someone.

edit: Jonsen's reply question, for which I don't have an answer, is instructive: these loop invariants are often impossible or impractical to achieve in assertions. I modified the above to be less wrong.

<comment><USER6> How would write an assert for the sum loop invariant?

</comment></comment><comment><USER27> I don't think the utility of invariants is restricted to proving program correctness, as is suggested by a number of commenters in this thread. Instead, they identify some underlying _structure_ in your problem, and that structure can be exploited to simplify your method of calculation.

Another great example of exploiting a (not-really-loop) invariant to design an algorithm is Sean Parent's implementation of a generic 'gather' algorithm, which takes advantage of the fact that the set of objects below/above the gathering point is unchanged, allowing you to split the problem into two easier sub-problems. Here's his explanation and implementation (video should be linked to 16:50):

[https://youtu.be/IzNtM038JuI?t=1009](https://youtu.be/IzNtM038JuI?t=1009)

</comment><comment><USER28> The article describes the concept of a loop invariant and gives examples for a few algorithms, but unless I am missing something, it is not giving you any way to computationally verify those invariants?

(I guess I am a bit surprised by the short length of this article because when I write code, I tend to think of it in those terms already, so I expected it to go further; but I guess not everyone has been exposed to that idea or taught to program like that in the first place so I realize that that's my own bias).

What should I check out if I want to be able to formally check invariants for e.g. my Python or Swift code?

<comment><USER29> When we covered this in my CS 101 class back in the day, our professor actually had a system he had put together built on Python's unit testing infrastructure. Really, any construct in a programming language that lets you say "assert that this thing is true and if it isn't, do something" can be used for loop invariant checking.

<comment><USER6> Loop invariant checking should assert that _if the invariant is true before the loop starts, it will also be true at the end of the loop_. That's not a general capability of assert constructs.

</comment></comment></comment><comment><USER30> Udi Manber's book on algorithms and induction is also a great resource to learn about invariants. Invariants and induction go hand in hand.

<comment><USER31> You mean "Introduction to Algorithms: A Creative Approach"

</comment></comment><comment><USER32> Eiffel has loop invariants, for example see:

[http://se.ethz.ch/~meyer/publications/methodology/invariants...](http://se.ethz.ch/~meyer/publications/methodology/invariants.pdf)

Also see Meyer's _Design By Contract_ book, which you should read even if you will never use Eiffel - it's a gem.

</comment><comment><USER33> I discovered the power of loop invariants during a course at university. A professor even developed a compiler like this, starting by defining invariants and writing the code accordingly. Really inspiring!

I wonder, are there such techniques tailored to functional programming? (immutability, recursion,...)

</comment><comment><USER31> Is there a list of loop invariants for algorithms that are frequently used in interviews?

<comment><USER34> Not sure what you mean by your comment. You can use loop invariants when solving problems that call for an iterative solution.

<comment><USER6> "Can you write code to reverse a linked list in place?"

"Sure, I'll just implemment a loop according to this invariant ... "

Would be nice to have the invariant ready then.

<comment><USER35> Why not just keep the implementation of the solution to any question you might be asked in your pocket?

<comment><USER6> A given pocket can contain many more invariants than implementations.</comment></comment></comment></comment></comment>