Apple-Mail set S/MIME algorithm
There appears to be no public default or setting to adjust the algorithm used by Mail – at least as of OS X 10.11.

Augment or Switch Your Mail Client

Consider using a third party tool like gpgtools to augment your e-mail security:


  GPG for Mail is an open source plugin for Apple Mail. Encrypt, decrypt, sign and verify mails using OpenPGP with a few simple clicks.


Alternatively, consider switching to another e-mail client that better addresses your security concerns.

Tell Apple

Be sure to provide feedback about your concerns directly to Apple via Mail feedback. Every report helps justify the engineering time.

You may also find the CSDA mailing list useful for reaching the involved Apple engineers.