Why do many talented scientists write horrible software?
I'll have a go at this. This is mainly my personal view based on my use and implementation of academic software. Like many of the comments already mentioned, I don't think bad software is specific to or even more frequent in  academia. That said, I think there are a few reasons why it occurs that are specific to the field.

Software is not a priority

In academics, the key performance indicators are all about paper publications. Software, while highly useful in my opinion, has very little value. More often than not, software implementations are side tracks or proof-of-concepts at best used to bump up the citation count (ofcourse exceptions exist).

As a software engineer, I am sure you are aware of the knowledge, time and effort required to produce quality software. Given the publish or perish mantra in academia, this time investment is often spent writing a new publication. It's a risk-reward consideration.

My personal point of view is that quality software is very important. For example, often, the first step in comparing a new algorithm with previous state-of-the-art is reimplementing the state-of-the-art due to a lack of implementation. Obviously this introduces a time delay and can cause a series of bugs. That said, I think in general little will change until software gets valued by performance indicators somehow.

Researchers are not programmers

Most researchers have little or no programming experience, though YMMV depending on the field. I think it's fair to say the majority of researchers learned to program by themselves when confronted with problems where they need it. Superficially learning a language is typically not the problem, but you need more than superficial knowledge to produce quality software (choosing data structures, design patterns, deep knowledge of the language, ...). This is a hurdle for self-taught programmers without a computer science background.

Another problem that less experienced programmers face is not realizing when refactoring is necessary. You can find countless pieces of poorly structured software. In research, this is a natural consequence of iteratively implementing while designing an algorithm which evolves into a monstrous piece of software full of hacks. That monster might still do what it was meant to, though, if you know exactly how to ask it nicely. For many researchers that's where the story ends: get the results and put the beast away forever. It often takes a serious time investment to wash the monster prior to talking it out for a walk in public. This is not always worth it.

Trends in machine learning

My field is machine learning, in which software is being valued increasingly. Examples of this trend include a growing software repository and a position paper by some big names in the field. I am very happy with this evolution, because quality software allows the field as a whole to progress faster and increases reproducibility.

A current patch to the problem is being able to publish papers about peer-reviewed software implementations. I know this is possible in machine learning and statistics. Such software is usually of higher quality.