appencryption

Asherah - Go

Application level envelope encryption SDK for Golang with support for cloud-agnostic data storage and key management.

• Quick Start
• How to Use Asherah
  • Define the Metastore
  • Define the Key Management Service
  • Define the Crypto Policy
    • (Optional) Enable Session Caching
  • (Optional) Enable Metrics
  • Build a Session Factory
  • Performing Cryptographic Operations
• Documentation
• Development Notes

Quick Start

package main

import (
    "context"

    "github.com/godaddy/asherah/go/appencryption"
    "github.com/godaddy/asherah/go/appencryption/pkg/crypto/aead"
    "github.com/godaddy/asherah/go/appencryption/pkg/kms"
    "github.com/godaddy/asherah/go/appencryption/pkg/persistence"
)

func main() {
    crypto := aead.NewAES256GCM()
    config := &appencryption.Config{
        Service: "reference_app",
        Product: "productId",
        Policy:  appencryption.NewCryptoPolicy(),
    }
    metastore := persistence.NewMemoryMetastore()
    key, err := kms.NewStatic("thisIsAStaticMasterKeyForTesting", crypto)
    if err != nil {
        panic(err)
    }

    // Create a session factory. The builder steps used below are for testing only.
    factory := appencryption.NewSessionFactory(config, metastore, key, crypto)
    defer factory.Close()

    // Now create a cryptographic session for a partition.
    sess, err := factory.GetSession("shopper123")
    if err != nil {
        panic(err)
    }
    // Close frees the memory held by the intermediate keys used in this session
    defer sess.Close()

    // Now encrypt some data
    dataRow, err := sess.Encrypt(context.Background(), []byte("mysupersecretpayload"))
    if err != nil {
        panic(err)
    }

    //Decrypt the data
    data, err := sess.Decrypt(context.Background(), *dataRow)
    if err != nil {
        panic(err)
    }
}

A more extensive example is the Reference Application, which will evolve along with the SDK.

How to Use Asherah

Before you can start encrypting data, you need to define Asherah's required pluggable components. Below we show how to build the various options for each component.

Define the Metastore

Detailed information about the Metastore, including any provisioning steps, can be found here.

RDBMS Metastore

Asherah can connect to a relational database by accepting a connection string and opening a connection to the database using a database driver name. See https://golang.org/s/sqldrivers for a list of third-party drivers.

// Open a DB connection using a database driver name and connection string
connectionString := ...;

// Parse the DSN string to a Config
dsn, err := mysql.ParseDSN(connectionString)
if err != nil {
    return err
}

// Open a connection to the database using the driver name and the connection string
db, err := sql.Open("driver-name", dsn.FormatDSN())
if err != nil {
    return err
}

// Build the Metastore
metastore := persistence.NewSQLMetastore(db)

You can also use the WithSQLMetastoreDBType option to configure the metastore for use with a specific type of database/sql driver. This is required when using an Oracle or PostgreSQL database.

// Build the Metastore for use with a Postgres DB
metastore := persistence.NewSQLMetastore(
    db,
    persistence.WithSQLMetastoreDBType(persistence.Postgres),
)

DynamoDB Metastore

// import "github.com/godaddy/asherah/go/appencryption/plugins/aws-v2/dynamodb/metastore"

// Create a new DynamoDB Metastore using the default configuration
metastore, err := metastore.NewDynamoDB()
if err != nil {
    panic(err)
}

You can also either use the WithXXX functional options to configure the metastore properties.

• WithDynamoDBClient: Specifies the DynamoDB client to use, useful when you want to use custom configurations.
• WithRegionSuffix: Specifies whether regional suffixes should be enabled for DynamoDB. Enabling this suffixes the keys with the DynamoDb preferred region. This is required to enable Global Tables.
• WithTableName: Specifies the name of the DynamoDb table.

// import "github.com/aws/aws-sdk-go-v2/aws"
// import "github.com/aws/aws-sdk-go-v2/config"
// import "github.com/aws/aws-sdk-go-v2/service/dynamodb"

// Load the default AWS SDK configuration with the desired region
awsCfg, err := config.LoadDefaultConfig(context.TODO(),
    config.WithRegion("us-west-2"),
)
if err != nil {
    panic(err)
}

// Create a new DynamoDB client with the loaded configuration and a custom endpoint
client := dynamodb.NewFromConfig(awsCfg, func(o *dynamodb.Options) {
		o.BaseEndpoint = aws.String("http://localhost:8000")
})

// Build the Metastore
store := metastore.NewDynamoDB(
    metastore.WithDynamoDBClient(client),
    metastore.WithRegionSuffix(true),
    metastore.WithTableName("myTableName"),
)

In-memory Metastore (FOR TESTING ONLY)

metastore := persistence.NewMemoryMetastore()

Define the Key Management Service

Detailed information about the Key Management Service can be found here.

AWS KMS

// import "github.com/godaddy/asherah/go/appencryption/plugins/aws-v2/kms"

// Create a map of region and ARN pairs that will all be used when creating a System Key
regionArnMap := map[string]string {
    "us-west-2": "ARN FOR US-WEST-2",
    "us-east-2": "ARN FOR US-EAST-2",
    "eu-west-2": "ARN FOR EU-WEST-2",
    ...,
}
crypto := aead.NewAES256GCM()

// Build the Key Management Service using the region dictionary and your preferred (usually current) region
keyManagementService :=  kms.NewAWS(crypto, "us-west-2", regionArnMap)

Static KMS (FOR TESTING ONLY)

crypto := aead.NewAES256GCM()
keyManagementService := kms.NewStatic("thisIsAStaticMasterKeyForTesting", crypto)

Define the Crypto Policy

Detailed information on Crypto Policy can be found here. The Crypto Policy's effect on key caching is explained here.

Basic Expiring Crypto Policy

cryptoPolicy := appencryption.NewCryptoPolicy()

The default key expiration limit is 90 days and revoke check interval is 60 minutes. These can be changed using functional options.

cryptoPolicy := appencryption.NewCryptoPolicy(
    appencryption.WithExpireAfterDuration(24 * time.Hour),
    appencryption.WithRevokeCheckInterval(30 * time.Minute))

(Optional) Enable Session Caching

Session caching is disabled by default. Enabling it is primarily useful if you are working with stateless workloads and the shared session can't be used by the calling app.

To enable session caching, simply use the WithSessionCache option when creating the crypto policy.

cryptoPolicy := appencryption.NewCryptoPolicy(
    appencryption.WithExpireAfterDuration(24 * time.Hour),
    appencryption.WithRevokeCheckInterval(30 * time.Minute),
    appencryption.WithSessionCache(),
    appencryption.SessionCacheMaxSize(200),
    appencryption.WithSessionCacheDuration(5 * time.Minute),
)

(Optional) Enable Metrics

Asherah's Go implementation uses go-metrics for metrics, which are enabled by default. If metrics are to be disabled, we simply use the WithMetrics functional option while creating the SessionFactory.

factory := NewSessionFactory(config, metastore, kms, crypto, WithMetrics(false))

The following metrics are available:

• ael.drr.decrypt: Total time spent on all operations that were needed to decrypt.
• ael.drr.encrypt: Total time spent on all operations that were needed to encrypt.
• ael.kms.aws.decrypt.<region>: Time spent on decrypting the region-specific keys.
• ael.kms.aws.decryptkey: Total time spend in decrypting the key which would include the region-specific decrypt calls in case of transient failures.
• ael.kms.aws.encrypt.<region>: Time spent on data key plain text encryption for each region.
• ael.kms.aws.encryptkey: Total time spent in encrypting the key which would include the region-specific generatedDataKey and parallel encrypt calls.
• ael.kms.aws.generatedatakey.<region>: Time spent to generate the first data key which is then encrypted in remaining regions.
• ael.metastore.sql.load: Time spent to load a record from sql metastore.
• ael.metastore.sql.loadlatest: Time spent to get the latest record from sql metastore.
• ael.metastore.sql.store: Time spent to store a record into sql metastore.
• ael.metastore.dynamodb.load: Time spent to load a record from DynamoDB metastore.
• ael.metastore.dynamodb.loadlatest: Time spent to get the latest record from DynamoDB metastore.
• ael.metastore.dynamodb.store: Time spent to store a record into DynamoDB metastore.

Build a Session Factory

A session factory can now be built using the components we defined above.

sessionFactory := appencryption.NewSessionFactory(
    &appencryption.Config{
        Service: "reference_app",
        Product: "productId",
        Policy:  appencryption.NewCryptoPolicy(),
    },
    metastore,
    kms,
    crypto,
)

NOTE: We recommend that every service have its own session factory, preferably as a singleton instance within the service. This will allow you to leverage caching and minimize resource usage. Always remember to close the session factory before exiting the service to ensure that all resources held by the factory, including the cache, are disposed of properly.

Performing Cryptographic Operations

Create a session to be used for cryptographic operations.

sess, err := factory.GetSession("shopper123")
if err != nil {
    panic(err)
}
// Close frees the memory held by the intermediate keys used in this session
defer sess.Close()

NOTE: Remember to close the session after all cryptographic operations to dispose of associated resources.

Encrypt/Decrypt

This usage style is similar to common encryption utilities where payloads are simply encrypted and decrypted, and it is completely up to the calling application for storage responsibility.

originalPayloadString := "mysupersecretpayload";

// encrypt the payload
dataRowRecord, err := sess.Encrypt(context.Background(), []byte(originalPayloadString))
if err != nil {
    panic(err)
}

// decrypt the payload
decryptedPayload, err := sess.Decrypt(context.Background(), *dataRowRecord)
if err != nil {
    panic(err)
}

Custom Persistence via Store/Load methods

Asherah supports a key-value/document storage model. Use custom Storer and Loader implementations to hook into the session's Store and Load methods.

An example map-backed implementation:

type storage map[string][]byte

func (s storage) Store(_ context.Context, d appencryption.DataRowRecord) (interface{}, error) {
    b, err := json.Marshal(d)
    if err != nil {
      return nil, err
    }

    key := uuid.NewString()
    s[key] = b

    return key, nil
}

func (s storage) Load(_ context.Context, key string) (*appencryption.DataRowRecord, error) {
    var d appencryption.DataRowRecord
    err := json.Unmarshal(s[key], &d)

    return &d, err
}

An example end-to-end use of the session Store and Load calls:

mystore := make(storage)

originalPayloadData := []byte("mysupersecretpayload")

// Encrypts the payload and stores it in the map
key, err := sess.Store(context.Background(), originalPayloadData, mystore)
if err != nil {
    panic(err)
}

// Uses the persistenceKey to look-up the payload in the map, then decrypts the payload and returns it
payload, err := sess.Load(context.Background(), key, mystore)

Documentation

appencryption package: See the godocs for api documentation.

Development Notes

Unit Tests

Some unit tests will use the AWS SDK, If you don’t already have a local AWS credentials file, create a dummy file called ~/.aws/credentials with the below contents:

[default]
aws_access_key_id = foobar
aws_secret_access_key = barfoo

Alternately, you can set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables.