In general, how does a DFA know how to successfully process a string the intended way? Suppose we have:
$$A\text{ }\colon=\{x, y, z\}$$
$$M\text{ }\colon=\text{some DFA using A}$$
$$S\text{ }\colon=xyzxyzxyz$$
Intuitively, one might say $S$ is fed to $M$ on a per-character basis.
This means that somehow we have an undisclosed mechanism that can tell where a symbol starts and ends.
One might say, simply use the maximum valid substring similar to how Lexers tokenise plaintext. To that I say, suppose instead that we defined $A$ as:
$$A\text{}\colon= \{x, xx, xxx\}$$
Now we have 3 unique symbols, that, as it so happens, using the maximum valid substring will yield in a restriction to what our our $M$ can actually process, because any string longer than 2 characters will always be assumed to start with $xxx$ rather than perhaps, $x$ and $xx$.
One way I see around this is to actually have a character synonymous to a symbol. That is, $x$ and $xxx$ (from $A$) are both a single character each.
Thoughts?
EDIT
In the case my question is unclear:
The extended transition function is defined so as to extend the processable input of a DFA from simply one alphabetic character at a time, to an arbitrary amount. Usually we go about defining it as follows for a string wa:
$$\hat{δ}(q,w) = δ(\hat{δ}(q, x), a)$$
My issue is: What is converting the input from a string to a partition? Whatever this object may be, it has to follow some lexing rules, does it not?
Are there any conventions that go about this?
The transition function of a DFA has the type $\delta : (Q \times \Sigma) \rightarrow Q$. That is, it's a function that takes a state (from the set of states) and a symbol (from the set of symbols) and returns another state.
This function doesn't have to know anything about tokenization. It's given a single symbol, and that's all it cares about.
Similarly, the input to a DFA is generally defined to be an ordered series of symbols: this is what &quot;a string over the alphabet $\Sigma$&quot; means mathematically. If it helps, think of it as a &quot;list&quot; rather than a &quot;string&quot; in programming terms. So if the alphabet were $\{x, xx, xxx\}$, then the input might look something like $[x, xx, xxx, x, xx, xxx, x, xx, xxx]$. No ambiguity there.
In practice, most real-world DFA implementations (like foma) do have to care about tokenizing. But the details of how to do that are up to the implementation; the definition of a DFA doesn't care one way or the other.