How do the Serpent S-boxes work?
Serpent uses 8 different 4-bit S-boxes (i.e. each S-box contains 16 elements) a total of 32 times. There are no &quot;rows&quot; or &quot;columns&quot; as it is a one-dimensional array containing a derangement of 16 elements. For each S-box, a 4-bit integer from 0 to 15 is mapped to another 4-bit integer from 0 to 15. When implementing the S-box as an array, the input is used as the index and the contents of the array are what make up the S-box itself. The output is simply the element at the index given by the input.
The 8 different Serpent S-boxes (ignoring the inverse S-boxes) are described by the following table:
\begin{equation}
\begin{array}{|c|c|c|c|c|c|c|c|c|}
\hline
\text{Input}&amp;\operatorname{S_0}&amp;\operatorname{S_1}&amp;\operatorname{S_2}&amp;\operatorname{S_3}&amp;\operatorname{S_4}&amp;\operatorname{S_5}&amp;\operatorname{S_6}&amp;\operatorname{S_7}\\
\hline
0&amp;3&amp;15&amp;8&amp;0&amp;1&amp;15&amp;7&amp;1\\
1&amp;8&amp;12&amp;6&amp;15&amp;15&amp;5&amp;2&amp;13\\
2&amp;15&amp;2&amp;7&amp;11&amp;8&amp;2&amp;12&amp;15\\
3&amp;1&amp;7&amp;9&amp;8&amp;3&amp;11&amp;5&amp;0\\
4&amp;10&amp;9&amp;3&amp;12&amp;12&amp;4&amp;8&amp;14\\
5&amp;6&amp;0&amp;12&amp;9&amp;0&amp;10&amp;4&amp;8\\
6&amp;5&amp;5&amp;10&amp;6&amp;11&amp;9&amp;6&amp;2\\
7&amp;11&amp;10&amp;15&amp;3&amp;6&amp;12&amp;11&amp;11\\
8&amp;14&amp;1&amp;13&amp;13&amp;2&amp;0&amp;14&amp;7\\
9&amp;13&amp;11&amp;1&amp;1&amp;5&amp;3&amp;9&amp;4\\
10&amp;4&amp;14&amp;14&amp;2&amp;4&amp;14&amp;1&amp;12\\
11&amp;2&amp;8&amp;4&amp;4&amp;10&amp;8&amp;15&amp;10\\
12&amp;7&amp;6&amp;0&amp;10&amp;9&amp;13&amp;13&amp;9\\
13&amp;0&amp;13&amp;11&amp;7&amp;14&amp;6&amp;3&amp;3\\
14&amp;9&amp;3&amp;5&amp;5&amp;7&amp;7&amp;10&amp;5\\
15&amp;12&amp;4&amp;2&amp;14&amp;13&amp;1&amp;0&amp;6\\
\hline
\end{array}
\end{equation}
This means that the integer $9$ (binary $1001$) fed through the 3rd S-box ($\operatorname{S_2}$) would be the element at index 9, which happens to be the integer $1$ (binary $0001$). In other words, $\operatorname{S}_2[9] = 1$. The S-box that is used in any given round is determined by the round number. Round $r$ uses $\operatorname{S}_{r\bmod 8}$. The Serpent authors decided to use 8 different S-boxes instead of 32 because the security benefit did not appear to be worth the added ROM footprint, and fewer than 8 would be insecure for such a small S-box.
Each of the 4-bit S-boxes operate on 4-bit chunks of input, returning 4-bit chunks of output. Each round's substitution step involves 32 parallel applications of the same S-box. This substitution is done 32 times (one time for each round). This is realized by splitting the 128-bit block into four 32-bit words. The first bit of each word is used as input for the first S-box. The second bit of each word is used as input for the second S-box. This continues 32 times. Since there are always four words, each S-box only takes 4-bit inputs. The following is taken from a blog post on an implementation of Serpent which explains how a 4-bit S-box is capable of operating on four 32-bit words (i.e. the 128-bit round state):

The diagram below shows how the s-boxes are implemented in bitslice mode. The 128 bit input block is divided up into 4 x 32 bit words: |w0|w1|w2|w3|. These are then arranged as shown in the diagram. The substitution is then applied by taking 32 slices of 4 bits each and putting then through 32 copies of the s-box in parallel. Bit 0 is the lsb in the 4 bit value.