Log file analysis: extracting information part from value part I'm trying to build a data set on several log files of one of our products.

The different log files have their own layout and own content; I successfully grouped them together, only one step remaining...

Indeed, the log "messages" are the best information. I don't have the comprehensive list of all those messages, and it's a bad idea to hard code based on those because that list can change every day.

What I would like to do is to separate the indentification text from the value text (for example: "Loaded file XXX" becomes (identification: "Loaded file", value: "XXX")). Unfortunately, this example is simple, and in real world there are different layouts and sometimes multiple values.

I was thinking about using string kernels, but it is intended for clustering ... and cluseting is not applicable here (I don't know the number of different types of messages and eventhough, it would be too much).

Do you have any idea?

Thanks for your help.

P.S: For those who programs, this can be easier to understand. Let's say that the code contains as logs printf("blabla %s", "xxx") -> I would like to have "blabla" and "xxx" seperatated
How about considering each string as a process trace and applying alpha-algorithm?
That would give you a graph and nodes with a big number out-edges will most likely point to values.

You can mark these nodes and for every new string parse/traverse the graph until you reach those areas.