<$BlogRSDURL$>

A splat of all my blathering.

## Using CAPICOM and OpenSSL

I was recently challenged with a project that required sending a secure message from a Windows network to a *nix network. The client insisted that we use CAPICOM on the Windows side and the partner on the *nix side was adamant about using OpenSSL. I figured since both libraries were capable of support PKI they must be compatible but I was unsure of how. I was also skepticle because CAPICOM is notorious for its proprietary implementation. After many unsuccessful attempts, talking with folks at Microsoft, and reading gads of web pages on the subject I was becoming quite unsure that it could be done. However, MS suggested that it was possible and that the PKI implementation of CAPICOM did support the standards. The trick was to figure out how.

After quite a bit of reading and playing with OpenSSL for Win32, I was able to come up with a way to use CAPICOM 2.0 and OpenSSL together to send a secure message. The key was with the S/MIME capabilities of OpenSSL.

In my situation I needed to sign and envelope a message using CAPICOM (on Windows of course) and send the message to a *nix machine using OpenSSL to decrypt and verify it.

For this example I will use VB 6.0 (sorry) to drive the CAPICOM calls. I am following the best practices of PKI solutions such as PGP as closely as I could (I think). In particular, I am embedding the plain-text content within the signature for use in the verification process later. The signature is then encrypted within the envelope so that only a single file needs to be sent. This results in a slightly larger envelope but simplifies the process by not requiring separate messages to be sent.

One thing to note is that CAPICOM is a scriptable COM interface and therefore uses BSTR as the string format of choice. This means that all content will be represented in UNICODE and will require translation on the server so that *nix tools like OpenSSL can interpret them.

The following VB code shows 4 sub-routines that read, write, sign, and encrypt a clear-text text file as well as an event sub-routine for a simple form with a single button for kicking the process off.

I am simplifying the example by removing the error handling and making some assumptions about the storage of the certificates being used. I am assuming that the certificates are stored in the local user store called “My” and that the certificate location is known. Almost certainly, neither of these would be the case in the real world.

' I do not claim any responsibility for the correctness or security of this code ' and any or all portions should be used at your own risk. Option Explicit Const ForReading = 1, ForWriting = 2, TristateFalse = 0 Const TemporaryFolder = 2 Private Sub Command1_Click()   SignFile "c:\temp\in.txt", "c:\temp\signed.txt"   EnvelopeFile "c:\temp\signed.txt", "c:\temp\smime.txt" End Sub Sub SignFile(ByVal sInFile As String, ByVal sOutFile As String)   ' Open the MY store and retrieve the first certificate from the   ' store. The signing operation will only work if this certificate is   ' valid and has access to the signer's private key.   Dim oCertStore As New Store   oCertStore.Open CAPICOM_CURRENT_USER_STORE, "My", CAPICOM_STORE_OPEN_READ_ONLY      Dim oSigner As New signer   ' The signer's certificate is at index 1   oSigner.Certificate = oCertStore.Certificates.Item(1)   Set oCertStore = Nothing      ' Set the content to be signed.   Dim oSignedData As New SignedData   oSignedData.Content = ReadFile(sInFile)      ' Save the time the data was signed as a signer attribute   ' NOTE: the name 'Attribute' is not a unique name   ' and must be preceded by 'CAPICOM.'   Dim oSigningTimeAttr As New CAPICOM.Attribute   oSigningTimeAttr.Name = CAPICOM_AUTHENTICATED_ATTRIBUTE_SIGNING_TIME   oSigningTimeAttr.Value = Now   oSigner.AuthenticatedAttributes.Add oSigningTimeAttr      ' Sign the content using the signer's private key.   ' A value of 'True' in the second parameter indicates that the   ' content signed is not included in the signature string.   ' A value of 'False' indicates that it is   Dim s As String   s = oSignedData.Sign(oSigner, False, CAPICOM_ENCODE_BASE64)   ' Strip off the last blank line   s = Mid(s, 1, Len(s) - 2)   ' Add the PKCS #7 header and footer so that the signature conforms to what   ' OpenSSL expects.   s = "-----BEGIN PKCS7-----" + vbCrLf + s + vbCrLf + "-----END PKCS7-----"   WriteFile sOutFile, s      Set oSignedData = Nothing   Set oSigner = Nothing   Set oSigningTimeAttr = Nothing End Sub Sub EnvelopeFile(ByVal sInFile As String, ByVal sOutFile As String)   ' Open the MY store and retrieve the first certificate from the   ' store. The signing operation will only work if this certificate is   ' valid and has access to the signer's private key.   Dim oCertStore As New Store   oCertStore.Open CAPICOM_CURRENT_USER_STORE, "My", CAPICOM_STORE_OPEN_READ_ONLY      ' Declare and initialize an EnvelopedData object   Dim oEnvelopedData As New EnvelopedData   oEnvelopedData.Content = ReadFile(sInFile)      ' The signer's certificate is at index 1   ' NOTE that you can add as many recipient certs as you like.   oEnvelopedData.Recipients.Add oCertStore.Certificates.Item(1)   Set oCertStore = Nothing      ' Set the encryption algorithm and key length. Comment out   ' or remove the following lines to use the default algorithm   ' and key length. The triple DES algorithm and 128 bit key   ' length may not be supported.   oEnvelopedData.Algorithm.Name = CAPICOM_ENCRYPTION_ALGORITHM_3DES 'or ENCRYPTION_ALGORITHM_RC4   oEnvelopedData.Algorithm.KeyLength = CAPICOM_ENCRYPTION_KEY_LENGTH_128_BITS   'Declare the string that will hold the enveloped message.   Dim sEnvelopedMessage As String   'Encrypt the message into sEnvelopedMessage. The enveloped message   'string contains everything that an intended recipient will need to   'decrypt the message, including information on the encryption algorithm 'key length.   sEnvelopedMessage = oEnvelopedData.Encrypt   ' Release the EncryptedData object and the Store object.   Set oEnvelopedData = Nothing   ' Add the MIME headers to the content   Dim sMimeHeader As String   sMimeHeader = "MIME-Version: 1.0" + vbCrLf   sMimeHeader = sMimeHeader + "Content-Disposition: attachment; filename=""smime.p7m""" + vbCrLf   sMimeHeader = sMimeHeader + "Content-Type: application/x-pkcs7-mime; name=""smime.p7m""" + vbCrLf   ' these last two line feeds are important   sMimeHeader = sMimeHeader + "Content -Transfer - Encoding: base64" + vbCrLf + vbCrLf   sEnvelopedMessage = sMimeHeader + sEnvelopedMessage   WriteFile sOutFile, sEnvelopedMessage End Sub Sub WriteFile(ByVal sFileName As String, ByVal sText As String)   Dim oFS, oTF As Object   Set oFS = CreateObject("Scripting.FileSystemObject")   Set oTF = oFS.OpenTextFile(sFileName, ForWriting, True, TristateFalse)   oTF.Write (sText)   oTF.Close   Set oTF = Nothing   Set oFS = Nothing End Sub Function ReadFile(ByVal sFileName As String) As String   Dim oFS, oTF As Object   Set oFS = CreateObject("Scripting.FileSystemObject")   Set oTF = oFS.OpenTextFile(sFileName, ForReading, False, TristateFalse)   ReadFile = oTF.ReadAll   oTF.Close   Set oTF = Nothing   Set oFS = Nothing End Function 

Then to decrypt and verify the message you can use the OpenSSL command-line tools with the smime command.

This first command will take the envelope and decrypt it to find the signature (containing the plain-text content).
 openssl smime -decrypt -in smime.txt -recip ..\cert.pem -inkey ..\privkey.pem -out signature.unicode.txt 
Recall that the output from CAPICOM is always UNICODE so we need to translate the files back to ANSI before proceeding. I use a simple tool that I wrote to do this.
 Unicode2AnsiConvert.exe signature.unicode.txt signature.txt 
This second command will take the signature and verify it. If the signature is verified the tool will write “Verification successful” to the console. Please note that verification requires correct certificate resolution to be configured for the machine but this is necessary for security.
 openssl smime -verify -in signature.txt -inform PEM -out message.unicode.txt 
Finally, translate the message to ANSI.
 Unicode2AnsiConvert.exe message.unicode.txt message.txt 
The resulting message.txt file will contain the original message input into CAPICOM.

Posted at 4/13/2004 01:20:00 PM |