Simple decision tree in Haskell
Just a few random comments:


Both elemAt and maximumBy give hints that you're expecting to operate on non-empty structures. Maybe give Data.List.NonEmpty a try.
A few places could be more clear with more pattern matching. E.g. max (k1, v1) k2 v2 instead of max acc k v. Or (maxDelta, splitter) = maximumBy …
map snd is more conventional than snd . unzip. I suspect it would be more efficient too but I might be wrong.
In several places you're traversing the same list multiple times. In general, it's better to avoid this as it's likely to force the spine of the (potentially large) list in memory. You might be able to merge these multiple traversals into one (e.g. using the foldl package). More likely, you should simply use vector.
In giniDelta you could use Data.List.partition to construct left and right.
Apply top-down ordering in your where-clauses. E.g. in train, majority should come first as it is the declaration that is referenced from the main function body.


EDIT: All in all I think readability is actually pretty good!