SSL Key Troubleshooting for PCI Compliancy Notes

Mindwatering Incorporated

Author: Tripp W Black

Created: 04/29/2010 at 12:39 PM

 

Category:
Domino Server Issues Troubleshooting
Web/HTTP

Issue:
Your Domino server has no "personal data" related to Visa cards on it. But the entire organization gets to be PCI complaint, because Notes is a good place to store such data someday.
These are notes are for making changes to the Internet Site documents, and using the SSL debug parameter to view various Internet protocols to verify SSL security.

Solution:
To be PCI compliant (as of 9/2009 and 4/2010), you have to ONLY use SSL3 for SMTP, IMAP, POP3, and HTTP. Ironically, you are generally still allowed to run completely insecure ports (e.g. 110 vs 995), but you cannot use medium strength ciphers or SSLv2 which has a vulnerability.

These notes assume you are using Internet Sites documents.

PART A - Update Your Documents

Open your web/HTTP Internet site document.
1. On the Basics tab,
a. Verify the domain name(s) and the unique IP are specified for the specific SSL key used for this site document.

2. On the Configuration tab,
a. Make sure that GET, HEAD, and POST are the only options. TRACE, OPTIONS, and PUT will cause you to fail the security test.
(Note: This will break WebDAV. No worry's PCI compliance is way more important than your WebDAV site working, right!)

3. On the Domino Web Engine tab,
a. Verify you are using SSO/session authentication. (It's better anyway if you aren't.)

4. Switch to the Security tab,
a. Set the TCP Authentication to what you are allowed by the auditing company.
You may be required to Redirect all normal non encrypted traffic to SSL. If so, change Redirect TCP to SSL to Yes.
b. For SSL Authentication, verify that both Anonymous and Name & password fields are set to Yes.
c. Make sure the SSL key is properly specified for the Site.
d. Make sure that the Protocol version is set to V3.0 only.
(This will break older Safari, Internet Explorer web browsers as they need a V2 to V3 handshake. Sorry, but so far, I've never been able to talk the PCI compliance companies into letting me leave this one. You just going to have to tick off a small community of folks who should upgrade their browsers anyway.)
e. Under SSL Security, remove any options that are below your PCI compliance company's threshold for ciphers.
The last company I just did allowed these for HTTP:
AES encryption with 128-bit key and SHA-1 MAC
AES encryption with 256-bit key and SHA-1 MAC
RC4 encryption with 128-bit key and SHA-1 MAC
Triple DES encryption with 168-bit key and SHA-1MAC
DES encryption with 56-bit key and SHA-1 MAC
(Note: the DES with 56-bit was allowed by a couple of the PCI compliancy auditors but not others. It was not allowed for the other protocols, at all.)

5. Save the site document and repeat for the other protocols.
(LDAP, IMAP, POP, SMTP)


PART B - Turn SSL Debugging On

Turn on SSL debugging by via the console:
1. Admin client --> Server --> Status --> Live button (green play button) --> Domino Command (field) --> Enter the following command and click Send.
set config debug_ssl_handshake=1
(To turn the debug off, just enter same command with a 0 instead.)

2. Stop and start each service as needed to make sure the Internet Site document changes are in effect.
tell pop3 quit (wait)
load pop3
tell http quit
load http
tell ldap quit
load ldap
tell imap quit
load imap


PART C - Testing
Use an OpenSSL client to be the client to test what is allowed.
(If you aren't normally a Linux shop, get Ubuntu and install in a VM. Then install that package if not already installed via Ubuntu's package manager.)
Since the debugger doesn't print which service the debug printout is for, we found it best to issue a sh tasks, click enter to issue the OpenSSL command, and then quickly type sh tasks. Otherwise, on our busier servers, we had no idea which printout was our test.

Do each test like so:
# OpenSSL <enter>
# OpenSSL> s_client -connect myserver.com:995 -ssl2 <enter>

- POP and IMAP Tests
The server will printout something like one of the two examples below for POP and IMAP.
In the failure example, if you see the bolded line 0x2 then you're server accepted the V2 SSL connection and you are not PCI compliant.
In the success example, if you see no 0x2 line and you see the server respond with SSL 3.0 only, then you're server responded by negotiating a V3 only connection. You are PCI compliant.

FAILURE EXAMPLE:
11/19/2009 12:50:49.14 PM [0908:000B-0554] SSL_Handshake> Enter
11/19/2009 12:50:49.14 PM [0908:000B-0554] SSL_Handshake> Current Cipher 0x0004 (RSA_WITH_RC4_128_MD5)
11/19/2009 12:50:49.14 PM [0908:000B-0554] SSL_Handshake> After handshake2 state 3
11/19/2009 12:50:49.16 PM [0908:000B-0554] SSL_Handshake> Protocol Version = 0x2
11/19/2009 12:50:49.16 PM [0908:000B-0554] SSL_Handshake> KeySize = 0x80
11/19/2009 12:50:49.16 PM [0908:000B-0554] SSL_Handshake> Current Cipher = 0x0004 (RSA_WITH_RC4_128_MD5)
11/19/2009 12:50:49.16 PM [0908:000B-0554] SSL_Handshake> SSLErr = 0
11/19/2009 12:50:49.16 PM [0908:000B-0554] SSL_Handshake> Exit Status = 0
11/19/2009 12:50:49.16 PM [0908:000B-0554] int_MapSSLError> Mapping SSL error 0 to 0

SUCCESS EXAMPLE:
04/28/2010 06:20:08.32 PM [16293:00010-2617731984] SSLInitContext> User is forcing 3079 cipher spec bitmask
04/28/2010 06:20:08.32 PM [16293:00010-2617731984] int_MapSSLError> Mapping SSL error 0 to 0
04/28/2010 06:20:08.32 PM [16293:00010-2617731984] SSL_Handshake> Enter
04/28/2010 06:20:08.32 PM [16293:00010-2617731984] SSL_Handshake> Current Cipher 0x0000 (Unknown Cipher)
04/28/2010 06:20:08.32 PM [16293:00010-2617731984] SSL_Handshake> SSL 3.0 only attempt
04/28/2010 06:20:08.32 PM [16293:00010-2617731984] SSL_Handshake> After handshake state= 3 Status= -6996
04/28/2010 06:20:08.32 PM [16293:00010-2617731984] SSL_Handshake> Exit Status = -6996
04/28/2010 06:20:08.32 PM [16293:00010-2617731984] int_MapSSLError> Mapping SSL error -6996 to 4166

- HTTP Tests
For HTTP, the printout is much longer. Below are a couple successful examples using OpenSSL with the client specifying different ciphers:
We know that they are successful because of the bolded line showing 0x3. Had the line ended with a 0x2 it would have been a V2 connection and not been PCI compliant like so:
04/28/2010 05:20:37.16 PM [12871:00011-2303626128] SSL_Handshake> Protocol Version = 0x3

Note: The Mapping SSL error can be ignored. This simply means the client is requested a SSL connection using the wrong DNS name or the IP rather than the right name. In these examples, we were using the IP.

SUCCESS EXAMPLES:
04/28/2010 05:15:49.72 PM [12871:00011-2303626128] SSLInitContext> User is forcing 15 cipher spec bitmask
04/28/2010 05:15:49.72 PM [12871:00011-2303626128] SSLDisableExportCiphers> Server key (2048 bits) too strong for EXPORT1024 ciphers. Disabling cipher: RSA_EXPORT1024_WITH_DES_CBC_SHA
04/28/2010 05:15:49.72 PM [12871:00011-2303626128] int_MapSSLError> Mapping SSL error 0 to 0
04/28/2010 05:15:49.72 PM [12871:00011-2303626128] SSL_Handshake> Enter
04/28/2010 05:15:49.72 PM [12871:00011-2303626128] SSL_Handshake> Current Cipher 0x0000 (Unknown Cipher)
04/28/2010 05:15:49.72 PM [12871:00011-2303626128] SSL_Handshake> SSL 3.0 only attempt
04/28/2010 05:15:49.72 PM [12871:00011-2303626128] SSL_Handshake> After handshake state= 11 Status= -5000
04/28/2010 05:15:49.72 PM [12871:00011-2303626128] SSL_Handshake> Exit Status = -5000
04/28/2010 05:15:49.72 PM [12871:00011-2303626128] int_MapSSLError> Mapping SSL error -5000 to 4176
04/28/2010 05:15:49.72 PM [12871:00011-2303626128] SSL_Handshake> Enter
04/28/2010 05:15:49.72 PM [12871:00011-2303626128] SSL_Handshake> Current Cipher 0x0004 (RSA_WITH_RC4_128_MD5)
04/28/2010 05:15:49.82 PM [12871:00011-2303626128] SSL_Handshake> After handshake2 state 13
04/28/2010 05:15:49.82 PM [12871:00011-2303626128] SSL_Handshake> Exit Status = -5000
04/28/2010 05:15:49.82 PM [12871:00011-2303626128] int_MapSSLError> Mapping SSL error -5000 to 4176
04/28/2010 05:15:49.82 PM [12871:00011-2303626128] SSL_Handshake> Enter
04/28/2010 05:15:49.82 PM [12871:00011-2303626128] SSL_Handshake> Current Cipher 0x0004 (RSA_WITH_RC4_128_MD5)
04/28/2010 05:15:49.82 PM [12871:00011-2303626128] SSL_Handshake> After handshake2 state 14
04/28/2010 05:15:49.82 PM [12871:00011-2303626128] SSL_Handshake> Exit Status = -5000
04/28/2010 05:15:49.82 PM [12871:00011-2303626128] int_MapSSLError> Mapping SSL error -5000 to 4176
04/28/2010 05:15:49.82 PM [12871:00011-2303626128] SSL_Handshake> Enter
04/28/2010 05:15:49.82 PM [12871:00011-2303626128] SSL_Handshake> Current Cipher 0x0004 (RSA_WITH_RC4_128_MD5)
04/28/2010 05:15:49.82 PM [12871:00011-2303626128] SSL_Handshake> After handshake2 state 3
04/28/2010 05:15:49.82 PM [12871:00011-2303626128] SSL_Handshake> Protocol Version = 0x3
04/28/2010 05:15:49.82 PM [12871:00011-2303626128] SSL_Handshake> KeySize = 0x80
04/28/2010 05:15:49.82 PM [12871:00011-2303626128] SSL_Handshake> Current Cipher = 0x0004 (RSA_WITH_RC4_128_MD5)
04/28/2010 05:15:49.82 PM [12871:00011-2303626128] SSL_Handshake> SSLErr = 0
04/28/2010 05:15:49.82 PM [12871:00011-2303626128] SSL_Handshake> Exit Status = 0
04/28/2010 05:15:49.82 PM [12871:00011-2303626128] int_MapSSLError> Mapping SSL error 0 to 0
...

...
04/28/2010 05:20:33.39 PM [12871:00011-2303626128] SSLInitContext> User is forcing 3079 cipher spec bitmask
04/28/2010 05:20:33.39 PM [12871:00011-2303626128] int_MapSSLError> Mapping SSL error 0 to 0
04/28/2010 05:20:33.39 PM [12871:00011-2303626128] SSL_Handshake> Enter
04/28/2010 05:20:33.39 PM [12871:00011-2303626128] SSL_Handshake> Current Cipher 0x0000 (Unknown Cipher)
04/28/2010 05:20:33.39 PM [12871:00011-2303626128] SSL_Handshake> SSL 3.0 with 2.0 Hello attempt
04/28/2010 05:20:33.39 PM [12871:00011-2303626128] SSL_Handshake> After handshake state= 11 Status= -5000
04/28/2010 05:20:33.39 PM [12871:00011-2303626128] SSL_Handshake> Exit Status = -5000
04/28/2010 05:20:33.39 PM [12871:00011-2303626128] int_MapSSLError> Mapping SSL error -5000 to 4176
04/28/2010 05:20:33.39 PM [12871:00011-2303626128] SSL_Handshake> Enter
04/28/2010 05:20:33.39 PM [12871:00011-2303626128] SSL_Handshake> Current Cipher 0x002F (RSA_WITH_AES_128_CBC_SHA)
04/28/2010 05:20:33.60 PM [12871:00011-2303626128] SSLSendAlert> Sending an alert of 0x0 level 0x2
04/28/2010 05:20:33.60 PM [12871:00011-2303626128] SSL_Handshake> After handshake2 state 2
04/28/2010 05:20:33.60 PM [12871:00011-2303626128] SSL_Handshake> Exit Status = -5000
04/28/2010 05:20:33.60 PM [12871:00011-2303626128] int_MapSSLError> Mapping SSL error -5000 to 4176
04/28/2010 05:20:33.60 PM [12871:00011-2303626128] SSL_Handshake> Enter
04/28/2010 05:20:33.60 PM [12871:00011-2303626128] SSL_Handshake> Current Cipher 0x002F (RSA_WITH_AES_128_CBC_SHA)
04/28/2010 05:20:33.60 PM [12871:00011-2303626128] SSL_Handshake> After handshake2 state 2
04/28/2010 05:20:33.60 PM [12871:00011-2303626128] SSL_Handshake> SSL Error: -6989
04/28/2010 05:20:33.60 PM [12871:00011-2303626128] int_MapSSLError> Mapping SSL error -6989 to 4165
04/28/2010 05:20:34.53 PM [12871:00011-2303626128] SSLInitContext> User is forcing 3079 cipher spec bitmask
04/28/2010 05:20:34.53 PM [12871:00011-2303626128] int_MapSSLError> Mapping SSL error 0 to 0
04/28/2010 05:20:34.53 PM [12871:00011-2303626128] SSL_Handshake> Enter
04/28/2010 05:20:34.53 PM [12871:00011-2303626128] SSL_Handshake> Current Cipher 0x0000 (Unknown Cipher)
04/28/2010 05:20:34.53 PM [12871:00011-2303626128] SSL_Handshake> SSL 3.0 with 2.0 Hello attempt
04/28/2010 05:20:34.54 PM [12871:00011-2303626128] SSL_Handshake> After handshake state= 11 Status= -5000
04/28/2010 05:20:34.54 PM [12871:00011-2303626128] SSL_Handshake> Exit Status = -5000
04/28/2010 05:20:34.54 PM [12871:00011-2303626128] int_MapSSLError> Mapping SSL error -5000 to 4176
04/28/2010 05:20:34.54 PM [12871:00011-2303626128] SSL_Handshake> Enter
04/28/2010 05:20:34.54 PM [12871:00011-2303626128] SSL_Handshake> Current Cipher 0x0005 (RSA_WITH_RC4_128_SHA)
04/28/2010 05:20:37.16 PM [12871:00011-2303626128] SSL_Handshake> After handshake2 state 13
04/28/2010 05:20:37.16 PM [12871:00011-2303626128] SSL_Handshake> Exit Status = -5000
04/28/2010 05:20:37.16 PM [12871:00011-2303626128] int_MapSSLError> Mapping SSL error -5000 to 4176
04/28/2010 05:20:37.16 PM [12871:00011-2303626128] SSL_Handshake> Enter
04/28/2010 05:20:37.16 PM [12871:00011-2303626128] SSL_Handshake> Current Cipher 0x0005 (RSA_WITH_RC4_128_SHA)
04/28/2010 05:20:37.16 PM [12871:00011-2303626128] SSL_Handshake> After handshake2 state 14
04/28/2010 05:20:37.16 PM [12871:00011-2303626128] SSL_Handshake> Exit Status = -5000
04/28/2010 05:20:37.16 PM [12871:00011-2303626128] int_MapSSLError> Mapping SSL error -5000 to 4176
04/28/2010 05:20:37.16 PM [12871:00011-2303626128] SSL_Handshake> Enter
04/28/2010 05:20:37.16 PM [12871:00011-2303626128] SSL_Handshake> Current Cipher 0x0005 (RSA_WITH_RC4_128_SHA)
04/28/2010 05:20:37.16 PM [12871:00011-2303626128] SSL_Handshake> After handshake2 state 3
04/28/2010 05:20:37.16 PM [12871:00011-2303626128] SSL_Handshake> Protocol Version = 0x3
04/28/2010 05:20:37.16 PM [12871:00011-2303626128] SSL_Handshake> KeySize = 0x80
04/28/2010 05:20:37.16 PM [12871:00011-2303626128] SSL_Handshake> Current Cipher = 0x0005 (RSA_WITH_RC4_128_SHA)
04/28/2010 05:20:37.16 PM [12871:00011-2303626128] SSL_Handshake> SSLErr = 0
04/28/2010 05:20:37.16 PM [12871:00011-2303626128] SSL_Handshake> Exit Status = 0
04/28/2010 05:20:37.16 PM [12871:00011-2303626128] int_MapSSLError> Mapping SSL error 0 to 0

____

Other Notes:
Lotus Domino had a bug where the V2 to V3 Handshake option actually allowed a V2 connection in 8.0.0 and 8.5.0. It was verified as fixed in 8.5.1. There was also a hotfix that is/was available which we ran for a point release or two. This issue is documented in SPR # BMKH7ESNN5.

Update for 2012:
PCI / PII compliance has now dictated that the AES ciphers in Section A, step 4e above, are no longer acceptable. Only the RC4 128-bit key and MD5 MAC, RC4 128-bit key and SHA-1 MAC, and the Triple DES (3DES) with 168-bit key and SHA-1 MAC are acceptable.

In addition, the TLS renegotiation on SMTP is no longer allowed. It must be disabled, as well. (There is a technote on how to do this in this application.)


Update for 2015:
PCI / PII compliance has now dictated that RC4 ciphers are no longer acceptable. Triple DES still remains for B+, however, to get A-, you have to implement TSL 1.2.





previous page