What methods allow us to determine the language used in a monoalphabetic substitution cipher? Working on a cipher (which I assume to be a mono-alphabetic substitution cipher due to the letter frequency) I struggle with the fact that I don't know which language the plain text is written in. Looking at the letter frequency only, gives some hints, at least to exclude some languages, but far from pointing out one. I have also studied the index of coincidence, but several languages are quite close to each other and I am uncertain how much I can trust this.

Both these ways indicates that my particular cipher isn't written in English. It's also a patristocrat (no separations between words), which makes word-matching harder.

My question is: what methods are there, and which one tends to produce optimal results?
I can't tell you which method is the best, but I can point out some places to look. The longer the message the easier it is easier to identify. How long is the message?

n-gram frequency: Look at the likelihood that groups of n-letters appear next to each other (often called n-grams). For example is does the n-gram AAA appear frequently (not many languages have words that contain the same letter repeated three times). N-grams are often used for language recognition. Bi-gram and Tri-gram (2-gram and 3-gram) frequency tables can be found here. Any long subsections of the ciphertext repeat?

Writing analysis Is this a hand written note? Often different languages that share the same character sets have different ways of writing characters (Germans/Mathematicians tend to put a line in the middle of their Z's, members of the US military tend to cross-slash their zero's).

Size of Alphabet: Are there 26 different letters in the ciphertext? Are the characters the standard latin alphabet. Any whitespace? Any punctuation? Numbers (many WW1 ciphers were broken by attacking ciphered numbers)? Capitals/Lowercase?

Plaintext: Do you have any clues about what the plaintext might be? Even if you don't know the language if you know that the ciphertext might be about battleships you can try translating water/gas/ships/boats/knots/depth into the candidate languages and see if you find any matches. You can try a similar approach with common words (equivalents of 'to be', 'to', 'it', 'hello' etc).