IMAP account stopped working after upgrade to iOS 9 - problem with CAcert certificate?
It turns out that in my case the problem is not the CAcert certificate, nor the TLS version - it's the enryption ciphers offered by OpenSSL on the server, or rather, that some of them use parameters that are too weak.

When Apple shipped their iOS 8.4 update a while ago, they made improvements to their TLS library coreTLS in order to prevent the so-called Logjam attack. It is safe to assume that these improvements are also part of iOS 9. As Apple mentions in this technote, coreTLS no longer accepts export-strength ephemeral DH cipher suites. In my case, this is not the problem, though, because my server does not offer such ciphers to its clients.

What Apple "forgot" to say in their technote is that they also added new requirements for the remaining DH ciphers. In this, they probably followed the Guide to Deploying Diffie-Hellman for TLS, which is an "official" document with recommendations made by the people who discovered the Logjam attack. Specifically, coreTLS now requires that DH cipher suites use a minimum DH group size.

The "DH group" is a parameter for DH ciphers whose strength is measured by its size in bits. The guide referenced above mentions that modern browsers now require a minimum DH group size of 1024. Apparently coreTLS has also adopted this requirement, because this is what I discovered on my server...

The Courier IMAP server has the following line in its config file /etc/courier/imapd-ssl:

TLS_DHPARAMS=/etc/courier/dhparams.pem


When I examine the DH params file, I get the following output:

$ openssl dhparam -in /etc/courier/dhparams.pem -noout -text
    DH Parameters: (768 bit)
        prime:
            00:e0:01:64:54:ec:1c:17:86:f3:02:81:08:44:75:
            67:e6:ab:5c:dd:61:0a:a6:49:1e:23:48:98:29:e9:
            48:36:d3:6b:9c:0f:0f:89:7d:7b:7a:17:1f:03:f3:
            53:4f:cf:d7:4d:a3:8f:00:6e:af:fb:e2:95:e6:45:
            71:c3:8b:74:d2:b4:8c:7c:7d:4b:e1:11:12:eb:7e:
            31:fb:c2:ff:f0:60:3d:07:69:d8:36:19:43:03:00:
            52:43:5b:99:21:5f:c3
        generator: 2 (0x2)


As can be seen, this DH group only has a size of 768 bits, which is definitely not up to modern standards.

To solve the problem I created a new DH group with increased size. I followed the advice in the "Guide to Deploying DH" and created a group with size 2048 bits:

openssl dhparam -out /etc/courier/dhparams-2048-bit.pem 2048


Change permissions of the new file so that they match the original file's permissions:

chmod 600 dhparams-2048-bit.pem
chown daemon:daemon dhparams-2048-bit.pem


I then updated the Courier IMAP config file:

TLS_DHPARAMS=/etc/courier/dhparams-2048-bit.pem


and restarted the server

/etc/init.d/courier-imap restart


After that the Mail app worked fine. Problem solved.



PS: Above I said that the coreTLS changes were shipped with iOS 8.4, yet in my question I claim that I have iOS 8 devices that do not have any IMAP connection problems. Both statements are true, but as I now realize I forgot to mention in my question that those iOS 8 devices are still using iOS 8.1.x. Sorry for that.



Additional Protocol tests: I played around with the Courier IMAP setting TLS_STARTTLS_PROTOCOL, in order to force the client (iOS 9 Mail app) to a specific TLS protocol version. Strangely enough I discovered that neither TLSv1.2 nor TLSv1.1 seem to work (SSL3 also does not work, but that's OK). The only option that does work is

TLS_STARTTLS_PROTOCOL=TLS1


(this remains true even after I upgraded the DH group size)

I tested the same settings with iOS 8.1.2 - there the Mail application is able to connect with all 3 protocol versions (TLSv1.2, TLSv1.1, TLS1), and even SSL3.

This is really, really weird. Although it's hard to believe, at the moment it seems as if iOS 9 Mail app can only use TLS1. Despite improvements in security on the DH cipher front, not supporting TLSv1.2 would be a bad regression, IMO. It might also be that my server is somehow misconfigured in a way that I don't recognize at the moment. It would therefore be useful if someone else could make similar tests to either confirm or discard my findings.