Encryption using matrix transformations I have been thinking lately about a block cipher which takes a block of bits and arranges them in a square matrix.  Then defining transforms on submatrices of the square matrix to scramble the bits.  The key would be a sequence of bits, which identify specific transformations to apply to the submatrices.  To unscramble the block simply reverse the order in which you execute the inverse of the transformations.  Has anyone else done any research along these lines?

For example:
Given a matrix of 100 x 100 bits. Divide the matrix into overlapping submatrices.

Define the following operation codes:


00 - rotate each sub-matrix clockwise by 90 degrees.
01 - flip each sub-matrix about it's left to right downward sloping diagonal.
10 - flip each sub-matrix about it's let to right upward sloping diagonal.
11 - rotate each sub-matrix counterclockwise by 90 degrees.


Thus a 1024 bit random number would be a mini program for executing these transformations.  Each transformation is invertible and the sequence could be processed in reverse order.

In many ways, it is like scrambling a Rubic's cube.  And I know that there are algorithmic solutions for solving Rubic's cubes.  But if you don't know the colors of each square I wonder if such algorithms would be much help.

I know that such an algorithm might be more memory intensive and possibly much slower than existing encryption algorithms, but if high throughput were not a primary consideration could such a cipher be deemed secure?
Unknowingly, you've actually answered your own question in the title.


  transformations


Imagine if you put and hold your finger on one of the bits in a test message. Then encrypt.  After a long time and possibly some cramp, your finger will have passed through the transformation matrix and out into the cipher text. One bit in and the same bit out, albeit possibly in a different position.   You're only transforming the position of the individual bits. This is entirely linear and is called a transposition cipher.  Security to cryptanalysis is very low indeed, and breaking one by frequency analysis is demonstrated in https://crypto.stackexchange.com/a/1578/23115.

Contemporary encryptions require other properties such as the following introductory list:-


Unbiased output - As only a single bit is transformed, there will be no uniformity of output frequencies.  Your message pops out the other side, only bit shifted, with exactly the same biases it had originally.
Confusion and diffusion -  Each bit should affect the other bits to a degree.  Your finger should morph into many, or even non.
Constant execution time - Cryptographers make huge effort to ensure that an algorithm flows/executes in constant time. Having major parts of it under control of the key/attacker gives too much away.
Some would argue that a 1024 bit key into a 10,000 bit block state may be unnecessarily high. There's some wonderful discussion of the futility of brute forcing large keys at How much would it cost in U.S. dollars to brute force a 256 bit key in a year?


There's quite a lot more to cipher design than my examples, but they give some context to the degree of weakness of design. Sorry.  There's a much more comprehensive list that answers What are the qualities of a good block cipher? Begginner, Intermediate, Advanced, Expert?