toidiu

ASN.1, PKCS #1, PKCS #8, pem, der, Pika Pika Chuuuu!

Ever feel like cryptography is just a bunch of random acronyms that someone just made up? I felt the same way and decided to change that by figuring out how these concepts are used in real world cryptography.

What is ASN.1?

In order to demystify the link between theory and application I will take a practical first approach to dissecting this question. We will start by looking at a real ASN.1 object before diving into the theory. Finally, we will tie it back to "real" world cryptography by seeing how these concepts are used in engineering.

Practical

We start by inspecting the ASN.1 object representation of the certificate of this website (i.e. the certificate for this website can be represented as a ASN.1 object). Notice how the ASN.1 object is similar to struct and contains a bunch of different fields pertaining to the RFC 5280. See if you can find:

The Subject which should be "toidiu.com".
The Issuer which should be the intermediate ("WE1") and root ("Google Trust Services") certificates.
The Subject Public Key Info associated with the certificate (search for "subjectPublicKeyInfo" and "subjectPublicKey"). This is the public key associated with the site "toidiu.com"
The signatureAlgorithm (search "signatureAlgorithm"). Notice that signatureAlgorithm is different from the "Subject Public Key Info" above. signatureAlgorithm is CA's (Certificate Authority) signature computed on the contents of this certificate. The "signature" is what helps us verify that the CAiwactually issued the certificate for "toidiu.com".

Theory

For a detailed dive into ASN.1, I highly recommend reading the post Warm Welcome to ASN.1 and DER. ASN.1 is an interface definition language (IDL). An IDL needs to be able to do 2 things:

defining data structures (e.g. define structure of a certificate)
serialization and deserialization of those data structures (e.g. convert struct to bytes and back)

The advantage of writing ASN.1 definitions instead of Go or C definitions is that they are language-independent.

While it is possible to represent a certificate in your application language (e.g. Rust), the ASN.1 representation is language agnostic and can be used across multiple languages (ASN.1 parser implementations are available for multiple languages). I like to use the analogy of Java, which allows you to "write code once, run anywhere". Similarly, ANS.1 allows your to "define/serialize an object in one application and deserialize/reconstruct in any other application".

// C representation
struct point {
  int x, y;
  char label[10];
};

// Go representation
type point struct {
  x, y int
  label string
}

// ASN.1 representation is language agnostic
Point ::= SEQUENCE {
  x INTEGER,
  y INTEGER,
  label UTF8String
}

There are some other languages that do the same things as ASN.1. For instance, Protocol Buffers offer both a language for defining types and a serialization format for encoding objects of the types you’ve defined... but ASN.1 (1984) had the significant advantage of already existing when certificates (1988) and HTTPS (1994) were invented.

ASN.1, while not perfect (a ASN.1 parser is complicated to implement), has become the standard in cryptography.

"Real" world usage

Now that we have an understanding of what ASN.1 objects are, lets see how we can make use of them in real engineering applications. TLS, PKI, and VPNs are just a few examples where these concepts are applied.

In the following code snippet we are representing a private key as ASN.1 and then encoding it as der/pem. In real cryptography we often have to do similar operations on other secret material such as public key, certificates, etc.

// 1. generate cryptrographic key (ex. ecdhe)
let private_key = ecdhe_handshake();

// 2. generate a BIG_NUM
let big_num_key: BIG_NUM = BN_set_word(private_key);

// 3.
// asn1_obj = ASN.1: ()
//   - PKCS #1: defines the RSA format
//   - PKCS #8: defines the key format (RSA, EC, ..)
let asn1_obj = Pkcs8::new(private_key);

// 4: priv.der
// The der encoded ans1 object is simply the raw bytes
let der = asn1_obj.to_bytes();

// 5: priv.pem
// The pem encoded asn1 object is the base64 encoding
// of the raw bytes.
//
// In bash: `pem = cat private_key.der | base64`
let pem = der.to_base64();

private_key: private key generate perhaps from a key exchange
big_num_key: generate a bignum from the private key
create a PKCS #8 encoded ASN.1 object: asn1_obj

ASN.1: a language for defining data structure. In cryptography, we care about a few encodings:
- PKCS #1: format for encoding and decoding RSA private and public keys
- PKCS #8: format for encoding cryptographic private keys, often containing pairs of private and public keys. Can encode multiple key types (RSA, ECDSA, etc) and should be preferred over PKCS #1.
- X.509: format for encoding digital certificates

der is just the raw bytes of the ans1 object
pem is the base64 representation of the der bytes

I hope this example illustrates how a "private key" ends up into a "priv.pem" file that you often see in cryptographic contexts. The "priv.pem" file could then then be given to your application, which could reconstuct the key and use it to securely communicate with a database.

Conclusion

In engineering applications we care about taking some secret material (key, certificate) and serializing it so that we can then share (via network, file, memory) it our peer.

ASN.1 is the common format that cryptographic implementations have agreed upon. We looked at PKCS #1 and PKCS #8 which are optional of ASN.1 formats used primarily for storing key material (there are many other PKCS formats for different contexts). We also saw how the binary representation of an ASN.1 object is called DER, and how PEM is simply the base64 representation.

Resources: