Forcing specific Cipher Suites to FOPE - TechNet Articles - United States (English) - TechNet Wiki

See http://www.ietf.org/rfc/rfc2246.txt for the TLS RFC

Brief background on how this works from the RFC.

• Client Hello from sending SMTP server to FOPE server
  • RFC2246 7.4.1.2 The CipherSuite list, passed from the client to the server in the
       client hello message, contains the combinations of cryptographic
       algorithms supported by the client in order of the client's
       preference (favorite choice first). Each CipherSuite defines a key
       exchange algorithm, a bulk encryption algorithm (including secret key
       length) and a MAC algorithm. The server will select a cipher suite
       or, if no acceptable choices are presented, return a handshake
       failure alert and close the connection.
• FOPE server responds with Server Hello
  • RFC 2246 7.4.1.3 The single cipher suite selected by the server from the list in
           ClientHello.cipher_suites. For resumed sessions this field is the
           value from the state of the session being resumed.

Example scenario:

Customer needs to enforce TLS_RSA_WITH_AES_256_CBC_SHA256 cipher suite while sending to FOPE or some comparable all 256 bit level suite for both symmetric and hashing algorithms.

Sending a cipher suite list in the Client Hello message with the following:

TLS_RSA_WITH_AES_256_CBC_SHA256
TLS_RSA_WITH_AES_128_CBC_SHA
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P256

Would result in TLS_RSA_WITH_AES_128_CBC_SHA being the agreed upon cipher by the Server.  The reason behind this is that the TLS spec does not say that it has to use the highest available encryption algorithm or that the server must honor the client's preference if it does have that cipher suite available.

When a Windows server receives the Hello message it goes down the Cipher Suite list that Schannel supports on the system and picks the first one on the server that matches up to one that is available in the Client Hello message.  When looking at http://msdn.microsoft.com/en-us/library/aa374757(VS.85).aspx the first match on the cipher suite list order for 2008 is TLS_RSA_WITH_AES_128_CBC_SHA so when it compares this entry to the Client Hello cipher suite list it will find a match and present this to the client as the agreed upon cipher suite to proceed with.

This presents a quandary to a system that examines the session afterwards and requires a certain level or type of suite to be agreed on for the exchange.  Cipher list suites could be changed on the server to change the higher level suites to be higher on the list however from a performance perspective you typically want the more commonly used suites to be closer to the top of the order.

So in order to remedy and indeed "force" a specific level or specific cipher suite then the sending server must dictate which cipher suites are acceptable to it as the server side must be able to accept many different cipher suite levels in order to accommodate all the various different SMTP servers that send to it.

Limiting Ciphersuites used by Sendmail

•  In order to limit cipher suites offered by Sendmail you must make use of the CipherList feature which is located under the LOCAL_CONFIG section of the sendmail.mc file.
  • You will need to ensure that +HIGH is part of the line
  • You will also want to exclude AES 128 by adding a -AES12-SHA to the line
  • For a full list of cipher suites see the following list http://www.openssl.org/docs/apps/ciphers.html (you have to use the corresponding short names)
  • Once modified you will need to remake your sendmail.cf file and restart the sendmail daemon

Limiting Ciphersuites used by Postfix

• Modify the main.cf config file for Postfix to include the following
  • Set smtp_tls_mandatory_ciphers to "high"
  • Add "AES128-SHA" to smtp_tls_exclude_ciphers

Limiting Ciphersuites used for Schannel (i.e. Windows SMTP service, Exchange)

• For Windows 2003 use http://support.microsoft.com/kb/245030/ to limit which cipher suites the OS utilizes, if you need 256bit/AES ciphers in 2003 you will need to install http://support.microsoft.com/kb/948963 to add those.
• For Windows 2008 you have to use the SSL Cipher Suite Order and create a custom list with the order you prefer see http://technet.microsoft.com/en-us/library/cc766285(WS.10).aspx for more detail regarding this.