How did early computers handle variable word length?
I've recently programmed an IBM 1401, so I can explain how its variable length words work.

For the motivation, remember that the IBM 1401 was a low-cost ($2500/month) 1960's computer for business applications on punched cards, for example payroll processing or inventory processing. The 1401 typically reads a file of punched cards, where each card holds fixed-length fields for one data record. The computer does some processing on each card, and prints out a report on the line printer, with fixed-length fields. Thus, it is convenient if the computer can easily manipulate fields of characters or numbers where each field has an arbitrary but fixed length.

The 1401's variable-width words are how it supports fixed-length fields. The programmer initializes the field lengths at the beginning of the program. Then a single "move" instruction will move a string field, regardless of how long it is. You don't need to loop over each character; the processor does that automatically. And you can add two numbers with a single "add" instruction, whether a 2-digit number or a 40-digit number. This is efficient, saves very expensive program memory, and is a good match for card-based applications.

Now, for the 1401's implementation of variable-length words (or fields). The IBM 1401 uses 6-bit characters (coded in BCDIC, precursor to EBCDIC), holding a text character or a decimal digit. (Note that the 1401 did decimal arithmetic, which worked well for business applications.) Each location in core memory has an additional "word mark" bit, which you can think of as a "metadata" bit. The programmer sets the word mark bit on the leftmost (high-order) character in a field. An instruction will typically process an entire field from right to left, until it hits a word mark, and then stops.

The important thing is that word marks aren't part of the 6-bit character. When you read a card, move a field, or so forth, the word marks are unchanged. Thus, the programmer can initialize the word marks once (with special word mark instructions), and then they stay put while you process the data.

Word marks are also used in the 1401 for variable-length instructions (1 to 7 characters), with a word mark after each instruction. One puzzle is that word marks aren't stored on the punched card, so how can you read in a program from cards? The solution is that when you load a program, you're actually loading and running a loader program that inserts the word marks as necessary for the program you actually want to run.

There is no limit on the size of a 1401 word, other than memory size. As a test, I created two 1000-digit numbers and multiplied them with a single "multiply" instruction. The computer churned away for literally a minute on this one instruction before generating the result.

The 1401 Reference Manual is online and provides more information, in case you didn't get enough here.