Finite-State Automata over a real-valued alphabet
Automata over infinite alphabets have been studied in many contexts (see e.g., this paper as a starting point).
Usually, the model is somewhat different than what you suggest, since just having an infinite alphabet is not very interesting, for the following reason:
Given an automaton with $n$ states, there are only $2^{n^2}$ transition matrices. Thus, if your alphabet has more than $2^{n^2}$ letters, then some letters will induce the same transitions. That is, you can partition them to classes of equivalent symbols.
This essentially reduces the model back to finitely many letters.
Note that the final weights do make a difference, but this has more to do with the standard model of weighted automata.
Typically, we're interested in infinite alphabets in the context of data words: these are words over a finite domain $D$, that represents variables whose range is an infinite domain. Then, a common model is that of variable automata, which have been studied a lot (e.g., this paper and references therein).