how does AES GCM in SSH work? I was looking at RFC5647: AES Galois Counter Mode for the Secure Shell Transport Layer Protocol and had some questions about section 7.1 "IV and Counter Management". Here's that section:

7.  Processing Binary Packets in AES-GCM Secure Shell

7.1.  IV and Counter Management

   With AES-GCM, the 12-octet IV is broken into two fields: a 4-octet
   fixed field and an 8-octet invocation counter field.  The invocation
   field is treated as a 64-bit integer and is incremented after each
   invocation of AES-GCM to process a binary packet.

         uint32  fixed;                  // 4 octets
         uint64  invocation_counter;     // 8 octets

           Figure 3: Structure of an SSH AES-GCM Nonce

   AES-GCM produces a keystream in blocks of 16-octets that is used to
   encrypt the plaintext.  This keystream is produced by encrypting the
   following 16-octet data structure:

         uint32  fixed;                  // 4 octets
         uint64  invocation_counter;     // 8 octets
         uint32  block_counter;          // 4 octets

           Figure 4: Structure of an AES Input for SSH AES-GCM

   The block_counter is initially set to one (1) and incremented as each
   block of key is produced.

   The reader is reminded that SSH requires that the data to be
   encrypted MUST be padded out to a multiple of the block size
   (16-octets for AES-GCM).


So let's say you invoke AES GCM twice and that the invocation counter is 0 (for simplicity sake). The first invocation encrypts two blocks and the second invocation encrypts one. What would be the source material for the encrypted keystream the second invocation?

For the first invocation, first block, it'd be this:

     uint32  fixed;
     uint64  0;
     uint32  1;


For the first invocation, second block, it'd be this:

     uint32  fixed;
     uint64  0;
     uint32  2;


For the second invocation, first (only) block, does the block_counter advance, as well, or does it reset? Here's what it'd be in the case of the former:

     uint32  fixed;
     uint64  1;
     uint32  3;


Here's what it'd be in the case of the latter:

     uint32  fixed;
     uint64  1;
     uint32  0;


eg. is the block_counter reset or incremented when the invocation_counter incremented?

Also, unless I'm mistaken, this approach breaks with tradition for SSH encryption in that encrypt(S1) || encrypt(S2) == encrypt(S1 || S2) (where || is the concatenation operation). That's how pretty much how it works with every other symmetric cipher for SSH but not this one.
The CTR-IV either must be random or regularly incremented in a way that the IV never repeats again. If there is a repetition then the encryption will be insecure.

In rfc5647, it is the second way; regularly incremented.  


invocation_counter is incremented per package. Section 7.1;


  The invocation field is treated as a 64-bit integer and is incremented after each invocation of AES-GCM to process a binary packet.

block_counter is incremented per AES encryption within the package. It must be initialized to 1 for per package, otherwise, an overflow will occur and therefore the IV's cannot be unique. Section 7.1;


  The block_counter is initially set to one (1) and incremented as each    block of key is produced.



The 4-octet block counter matches with the size of the package that is defined in section 5.2 as;

uint32    packet_length;  // 0 &lt;= packet_length &lt; 2^32


Thus, this counter can be incremented for all encrypted blocks within one package. For a new package the set to 1 (that is your latter case).

For the second part, this can happen if ECB mode is used and the data sizes are multiple of the block cipher. Even in this case, a padding scheme must be applied. that will prevent your equation.