SMIME TOOL

SMIME TOOL
29.10.1999, 17.11.1999, Sampo Kellomaki <[email protected]>

NOTE: This is still highly experimental code and build system has not been
perfected yet. No Windows build is known to exist (contributions?).

OFFICIAL WEB SITE
http://www.bacus.pt/Net_SSLeay/smime.html

BUILDING

(build and install OpenSSL-0.9.4, from www.openssl.org)
tar xzf smime-0.7.tgz
cd smime-0.7
cons smime # get cons from http://www.dsmit.com/cons/
cons SMIMEutil.so # build the perl module (optional)
./smime -help # shows quick usage
./smime -dv ../smime-0.7.tgz
(cut my certificate and distribution signature from the web
site and paste to stdin)

TUTORIAL PART 1: SIGNING AND ENCRYPTING

First you need to have certicate and private key in pem format. To
produce them, use openssl tool or export them from your browser.
I illustrate the latter method first, because I'm going to use
Netscape browser for interoperability testing later. You can
peek at TUTORIAL PART3, Key generation if you need to do this
yourself.

- Go to security info dialog in Netscape browser.
- From Certificates-Yours export your certificate (if you
don't have a certificate installed yet, read the FAQs and mailing
list archieves at www.openssl.org), save it as me.p12. It
will ask for password to protect your private key.

openssl pkcs12 -clcerts <me.p12 >me.pem
- it will ask for the password to open your private key and
then asks you to invent a new password that will be
used to protect your private key in pem format

more me.pem

You should see something like this:

Bag Attributes
friendlyName: [email protected]
localKeyID: F3 85 A8 4B DA 39 B6 40 6B D6 20 01 39 46 6A 94 47 9D 2C 0F
Key Attributes: <No Attributes>
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: DES-EDE3-CBC,541E04862A13F6B1

8+2vo6Iz49uj/Mf31JTgaRuIq9ueHsknsHXhmXp7s1BmS8xulT22Zzpdh6g1yqAO
(snip)XeQsZrWykdWvN2qGu/cNa2HnUQAG0p25tNZ3CKmqpJBVg0RXr20JlQ==
-----END RSA PRIVATE KEY-----
Bag Attributes
friendlyName: [email protected]
localKeyID: F3 85 A8 4B DA 39 B6 40 6B D6 20 01 39 46 6A 94 47 9D 2C 0F
subject=/C=PT/L=City/O=Company/OU=Dept/CN=Your Name/[email protected]
issuer= /C=PT/L=City/O=Your CA/OU=Personal Certs/CN=End user CA/[email protected]
-----BEGIN CERTIFICATE-----
MIIDEzCCAnygAwIBAgIBAzANBgkqhkiG9w0BAQQFADCBlTELMAkGA1UEBhMCUFQx
(snip)Tj0JYGZMzSUfzOG3wajK6B39d6EyXK8=
-----END CERTIFICATE-----

Ok. Now you are all set to use smime tool. First lets create
simple mime entity (see RFC1521 for definition):

echo foo | ./smime -mime text/plain | tee foo.mime

Signing
Now, let's sign it:

./smime -s me.pem password <foo.mime | tee foo.smime

And send it:

./send.pl 'Sig test' [email protected] [email protected] <foo.smime

Now go to your email reading software (Netscape Communicator suggested
for this exercise) and read the mail you just sent. It should display
as signed. In the previous command the second argument is the
From: header which must match friendlyname/EMAIL in me.pem

You can repeat this success using following pipeline:

echo foo | ./smime -mime text/plain | ./smime -s me.pem password \
| ./send.pl 'Sig test' [email protected] [email protected]

(Note how \ is used for folding the lines. In reality you should
type all the stuff in one line.)

Clear signing
The previous method produces a base64 blob that you probaly would not
like to send to a news group. Clear signing allows mail to be read
even if the reader is not S/MIME aware:

echo foo | ./smime -mime text/plain | ./smime -cs me.pem password \
| ./send.pl 'Sig test' [email protected] [email protected]

Note how the message has "foo" visible. Now go read this with your
mail reader. It should also display as signed. If not, then its
possible that canonization was not correctly done. That might even
be an error in my part.

Encrypting
To encrypt you need to know recipient's cert. Here I use our own
because its by definition already installed in our browser.

echo foo | ./smime -mime text/plain | ./smime -e me.pem \
| ./send.pl 'Enc test' [email protected] [email protected]

Now you should be able to read your mail (it may ask for a password
to open your private key) and see the message in plain text, but
marked as encrypted.

Signing and Encrypting
Next we'll first sign a message and then wrap it in encryption. This
is the usual way and hides the signatories from eavesdroppers:

echo foo | ./smime -mime text/plain | ./smime -cs me.pem password \
| ./smime -e me.pem | ./send.pl 'test' [email protected] [email protected]

Again your mail reader should show the message marked as
Encrypted and Signed.

Encrypting and Signing
The other way is to first encrypt and then sign the encrypted
message. This might be useful in some automated context where a robot
would verify the signature but the robot should not be allowed to see
the message:

echo foo | ./smime -mime text/plain | ./smime -e me.pem \
| ./smime -cs me.pem password \
| ./send.pl 'Enc test' [email protected] [email protected]

Now Netscape shows two icons: one saying "Signed" and placed near
the headers. Other icon that says "Encrypted" appears in the
content area.

In fact S/MIME specification allows arbitrary nesting of encryptions
and signatures, what ever your application may need.

Multipart content
Now for the final part that is not really S/MIME specific, but
nice to have. You can first compose a normal mime multipart message,
possibly even containing some encrypted components and some not.
Try this:

echo bar | ./smime -m image/gif some.gif \
| ./smime -cs me.pem password | ./smime -e me.pem \
| ./send.pl 'Enc test' [email protected] [email protected]

Now you should see signed and encrypted message with an image
attached. The multipart functionality implemented by -m is very
basic and by no means anything generic. Its only a demonstration.

TUTORIAL PART 2: DECRYPTING AND VERIFYING SIGNATURES

Decrypting
To demonstrate decrypting, I'll cut the mail from the loop. Basically
I'll demonstrate introperability with myself. If you want to try
interoperability with Netscape, you can use Netscape to send mail
and grab it from mail spool and then feed it into pipeline. As this
is a bit messy and might involve editing the file manually, I
wont go into that now. Here's a simple encryption - decryption:

echo foo | ./smime -mime text/plain | ./smime -e me.pem \
| tee foo.p7m | ./smime -d me.pem passwd

foo wrapped in mime headers should come out. I also used tee(1) to
put the encrypted form in a file in case I also want to verify
it with Netscape

./send.pl 'Enc test' [email protected] [email protected] <foo.p7m

or in qmail

/var/qmail/bin/qmail-inject [email protected] < foo.p7m

This is most convenient method because it allows you to import
certificates belonging to others than yourself and still
deliver mail to yourself. You want this, because you will soon
discover that Netscape can hold in its certificate database
only one certificate per email address (or is it distinguished name?)
and if that one "slot" is occupied, say, in "People" section, then
you can't import it any more to "Yours" section. Hence the solution
is to use different email address (and friendly name) every
single time.

Verifying signature
Due to somewhat incomplete nature of OpenSSL-0.9.4 PKCS7 signature
verification code, I have implemented my own signature verfication
scheme that works in five steps

1. find out who signed the message
2. verify message signature using signer's certificate
3. find out who certified the signer's certificate
4. verify signer's cert using certifier's cert
5. repeat 3. & 4. until at root of the chain

This has the advantage that there are less obscure stuff and
assumptions hidden inside the program, i.e. user must understand
what signature and certificate verification is all about and
you get to observe every step of the way. The down side is that
its less automatic. In my particular application this does not
happen to be a problem because I have out of band information
about who is supposed to have signed what.

Here we go: lets produce a signature to play with

echo foo | ./smime -mime text/plain | tee foo.mime \
| ./smime -s me.pem password | tee foo.p7m

Now see who signed it

./smime -qs < foo.p7m

Now I assume that I have some way of finding the certificate using
the issuer DN and serial number returned in the previous step. These
two pieces of information constitute unique ID for a certificate.
Perhaps I keep my certs in LDAP or some other database. Anyway,
assume the certificate is found, then

./smime -v me.pem <foo.p7m | tee foo2.mime
diff foo.mime foo2.mime # just to check!

Note that diff may show white space differences (try diff -b) because
line endings in your mime message were canonized to CRLF for signing.
This is mandated by S/MIME specification.

Now proceed with verifying the certificate:

./smime -qc <me.pem

Now out of band means are used to find the signer's certificate. Then
to check that the signature matches:

./smime -vc ca.pem <me.pem

Verifying clear signature
(does not currently work :-(

Under construction: chain verification (does not currently work)
mkdir certs
cd certs
cp ../me.pem .
../hash-certs.pl *.pem
cd ..
echo foo | ./smime -mime text/plain | ./smime -s me.pem password \
| ./smime -v

TUTORIAL PART 3: A KEY GENERATION METHOD

Bootstrapping a simple public key infrastructure

Suppose entity M is funding projects and needs to have all project
leaders (Ps) sign a contract specifying responsibilities. M
distributes a simple application that incorporates the smime tool. Ps
use the application to sign the contracts and send them electronically
to M. If irregularities develop, M sues P to court and uses the
digitally signed contract as evidence. For this to work

I. M must convince the court that the system does not have
technical flaws that could work in his favor. This proof
is much easier with digital signatures than with systems
that depend on procedural integrity (e.g. passwords over
SSL protected connection may prove at the moment the
intention of P, but when document is recovered from backup
10 years later, all procedural proofs vanish)

II. P's signature must be as valid as paper and ink
1. P's real world identity must be connected to
digital signature
2. P must have understood what it means to sign
contract digitally
3. P must have sufficient integrity or the system
must technically guarantee that P's private
key could not have been used by any one else
4. P must have acted by free will and in full powers
of mind
5. law must not prohibit digital signature

M needs to establish a simple public key infrastructure. I propose
that the application generates key pairs for P's and sends
certification requests, further, it also prints the certification
request in paper form including,

- fingerprint of public key (as number and as bar code)
- full dump of public key and all attributes appearing
in the certification request
- legal language to guarantee 2. & 3. (on point 3 we rely
on integrity)
- details of conventional identification of P
- space for signature

With this paper P goes to some commonly agreed and trustworthy
notary. This could be notary public or it could be some trusted
administrative organ in P's organization. It could even be M, but
that would cause a bureaucracy bottle neck at M, and hence increase
costs of the solution.

P signs the paper and proves his identity in presence of the notary
who confirms the act. The paper is sent to M. This takes care of
1. and 4. As paper contract about use of digital signatures now
exists between M and P, 5. is only of concern if it explicitly
nullifies such contract (or if court practice still does not consider
digital signature valid?).

Finally paper arrives to M where a clerk processes it (the bar
code helps here). He finds the certification request from data base
and sees that it matches the paper and issues a digital cerificate
that is immediately placed on a public server for everybody to
see. The paper is securely archieved forever.

Once the certificate is publically available, signing and verifying
contracts using it is trivial and can even be done between P and
some other party than M (e.g. certificate could be used for email).

However, to bootstrap the system it should, ideally, be possible
to sign your first contract even without waiting for the
certification to happen. After all, the impulse for P to adhere
to PKI of M came from needing to sign a contract (dead line for
research proposals may be very close). The contract would be
in signed, but not verified status until the certification happens.

Here the OpenSSL (or PKCS7?) does not serve us well, because it needs
a certificate even for the signing operation, although common
sense says that the private key and attributes of certification
request (ok, you can't know what serial number will be assigned)
should be enough. I solve this problem by signing the first contract
with a self signed certificate. Although this is quite suboptimal,
it allwos me to get going without hacking OpenSSL innards too much.
Only complication arises from needing to establish that also the
certified public key is able to decrypt the hash of the signed
material.

Key generation

Smime tool contains simple key generation command that will
make certification request as well as self signed cert. This
is bit simplistic and really geared towards my particular application
as the X509v3 certificate options are hardwired. Be sure to read
the source code to check they are the way you want.

echo "commonName=Joe Smith|[email protected]" \
| ./smime "d=foo" passwd req.pem >priv_ss.pem

The stuff that is echoed to stdin is your distinguished name. You
must use long forms of attribute name and you can only use attributes
known to OpenSSL.

The req.pem should be sent to certification authority for signing.
Meanwhile you can use the self signed certificate which was output
to stdout, here priv_ss.pem. Note that the private key is also
output to the stdout so do not give that file to anyone. If you need
to give the certificate, you should edit a copy of priv_ss.pem
and remove the private key.

To be able to import your private key and certificate to Netscape
you can use

./smime -pem-p12 [email protected] passwd pw-for-p12 <priv_ss.pem >me.p12

The first argument (the email address) is the friendly name. Netscape
appears to match this against From mail header when verifying
signatures. For minimum troubles you should keep this equal to
emailAddress field of your certificate.

Being a certificate authority

Once you have made your req.pem, you can send it to some commercial
certification authority or you can just be your own. The CA
functionality of smime tool is not very complete. Basically
it allows you to do the crypto part (signing certificate request with
CA's private key) but you must manually do book keeping to ensure
uniqueness of serial numbers and to make sure you do not issue the
same certificate twice, etc.

Here's how you'd sign a request (you probably used -kg to make
the CA's certificate):

./smime -ca ca-id.pem very-secret 1 <req.pem >cert.pem

The numebr one is the serial number. As I said, you should do
bookkeeping to ensure you never reuse a serial number. Many
systems depend on being uniquely able to identify certificate
by its issuing authority and serial number. A certificate
authority that can not guarantee uniqueness of serial numbers is
trustworthy.

Please note that the -ca hard wires all X509v3 options and extensions.
Be sure to read the source to check they are the way you want them.

TUTORIAL PART 4: SIGNING AND VERIFYING SOFTWARE DISTRIBUTIONS

smime tool has detached signature feature which is meant for
signing and verifying software distributions. Here's how. First
make yourself an identity

echo 'commonName=my dist key|[email protected]' \
| ./smime -kg '' pw dist-req.pem >dist-id.pem

Now open dist-id.pem in editor and save the certificate part as
dist-cert.pem. Pubish dist-cert.pem on your web site.

Then sign your software package

./smime -ds dist-id.pem pw <your-dist-1.00.tgz >your-dist-1.00.sig

I suggest convention of naming the signature file with same name
as the tarball, but extension `.sig'. Now put the .sig file
available where ever you put your tarball. You should also publish
it on your web site so people can get it even if they forgot to
download the .sig file.

Or you could produce a combination signature and certificate file
and put that available on your website

./smime -ds dist-id.pem pw <your-dist-1.00.tgz \
| cat - dist-cert.pem >your-dist-1.00.sigcert

To verify a distribution signature one would say

./smime -dv your-dist-1.00.tgz
(cut certificate and distribution signature from the web
site and paste to stdin)

The dv looks for the -----BEGIN/END CERTIFICATE----- separator to
figure out where the certificate ends and signature starts.

If you already had the .sig or the certificate you could just say

cat dist-cert.pem your-dist-1.00.sig | ./smime -dv your-dist-1.00.tgz

(OK, PGP already exists to do this stuff, but I always found it
quite messy to deal with detached signatures in PGP. I'm sure
poor usability leads to less people verifying the signatures.)

SMIMEUTIL LIBRARY AND PERL MODULE

The smimeutil library is documented in smimeutil.h, see smime.c
for some examples of usage. The SMIMEUtil:: perl module is
not currently documented. For usage examples see test.pl.

CAVEATS
For signing to work correctly, your mime entity must be canonized
the same way as the recipient will canonize it. In general this
means that you must use CRLF as line termination and must include
all headers.

If you are clear signing, then you may want to consult RFC2311 for
some ways the message might get ruined (e.g. changes in whitespace).
In my experience most important requirement seems to be to not
use any trailing whitespace. YMMV.

For signatures to verify correctly, the From: header of the mail
must be equal to "friendlyname:" and EMAIL fields in your cert.

When encrypting, do not forget to wrap your message in mime entity.
If you don't, ./smime -d will silently return emptiness, Netscape
reports "improperly formatted DER-encoded message". This
will _not_ work:

echo "foo" | ./smime -e me.pem | ./smime -d me.pem secret # WRONG!

SECURITY CAVEATS
Passing passwords on command line is insecure. The smime tool is
intended more as a demonstration than a production tool. See
pass-password.pl for an example how to use file descriptor to
pass the passwords more securely.

smimeutil.c compiles by default to use DES-EDE3-CBC cipher which
is not known by export versions of many browsers. See around
line 400 in smime-enc.c if you need to change this. Be ware that
RC2-40-CBC can be cracked in real time by trivial resources.
Never-the-less its the only cipher that interoperates with all
versions of browsers.

Randomnumbers are not (yet) initialized as they should.

Certificate verification scheme puts the burden of verification
on user. For example, the user must notice if purported CA certificate
has X509v3 attribute that forbids it from being CA cert.

TIP
You can use Netscape to encrypt and sign messages and then look
at them in mail spool. This way you see their raw structure before
any mail reader gets to interpret it. This is the best way to
debug differences between what you produce and what is presumably
standards compatible.

TO DO
Parsing mime messages in robust and fully correct way

BUGS
Due to the way the API is defined all stuff is kept in memory. While
this is simple and easy, you might get into trouble with large files.
I'd say the memory consumption will not exceed five times the file
size, but don't bet on it.

Signature verification in perl module still needs some work.

SEE ALSO
RFC1521 (MIME)
RFC2111 (S/MIME v2)
*** RFCXXXX (S/MIME v3)
http://www.openssl.org
http://www.bacus.pt/Net_SSLeay/smime.html (this stuff)

USAGE
(reproduced from ./smime -help)

./smime -cs private password <mime-entity >smime # clear sign
./smime -cv cert <smime-entity >data # verify clear sig
./smime -ds private passwd <file >smime-sig # make detached sig
./smime -dv file <smime-sig-entity # verify detached
./smime -s private password <mime-entity >smime # sign
./smime -qs <smime-entity >signing-cert-info # find out who signed
./smime -v cert <smime-entity >signer-dn # verify signature

./smime -vc cacert <cert # verify certificate

./smime -e public <mime-entity >smime-ent # encrypt
./smime -d private password <smime-entity >mime # decrypt

./smime -qr <req.pem # Query all you can about request
./smime -qc <cert.pem # Query all you can about certificate
./smime -ca ca_cert passwd serial <req.pem >cert.pem # sign a req

./smime -p12-pem p12pw pempw <x.p12 >x.pem # convert PKCS12 to pem
./smime -pem-p12 [email protected] pempw p12pw <x.pem >x.p12

./smime -m type1 file1 type2 file2 type3 file3 <text # make multipart
./smime -m image/gif foo.gif <message | ./smime -s private pass >smime

./smime -kg attr passwd req.pem <dn >priv_ss.pem # keygen

./smime -base64 <file >file.base64
./smime -unbase64 <file.base64 >file
./smime -mime text/plain <file >mime-entity
./smime -mime_base64 image/gif <file.gif >mime-entity
./smime -split dirprefix <multipart # splits multipart
./smime -base64 <in | ./smime -unbase64 >out
./smime -cat <in >out # copy input to output using slurp and barf

./smime -kg 'description=Test' secret req.pem <me.dn >ss.pem

--Sampo