Request: Add ake.Server Serde methods

[everyCTO]

TL;DR

Please provide func (s Server) Serialize() []byte and func (s *Server) Deserialize(data []byte) error in package ake so that state can be persisted externally to the ake.Server instance between steps in a protocol run.

If you're open to it, I would be happy to implement these methods and submit a PR.

Rationale

The OPAQUE protocol is comprised of two flows--registration and authentication--both of which have four steps. Both the client and the server need to maintain some state between steps in each flow.

Clients generally are implemented to be run on a single endpoint (web browser, mobile device, etc.) where state is easily maintained between steps in a protocol run.

Servers in production environments, however, are typically implemented to be as stateless as possible, sharing state via some persistent store. For a fleet of servers implemented in this way, it's not currently possible to complete the authentication flow because necessary state cannot be persisted server-side between steps of the flow. Even a single HTTP server cannot trivially maintain the state between HTTP requests correlating to steps in the flow.

Focusing on the authentication flow:

• Step 1: the client begins the flow with ke1 := client.Init(password). ke1 is sent to the server, probably along with a username. The client's only input so far is password and maybe username.

• Step 2: the server does ke2, err := server.Init(ke1, serverIdentity, serverSecretKey, serverPublicKey, oprfSeed, record). All of the parameters passed here are externally known. ke1 was provided by the client, serverIdentity, serverSecretKey, serverPublicKey, and oprfSeed all come from configuration, and record can be fetched from a persistent store using a username that the client probably provided. As a side effect of server.Init, some state is set internally to server.Ake. ke2 is passed back to the client.

• Step 3: the client does ke3, exportKey, err := client.Finish(idc, ids, ke2) using the ke2 it got back from the server, and the client and server identities ostensibly already known to the client. ke3 is sent to the server.

• Step 4: the server attempts err := server.Finish(ke3). The only input available here is ke3, which comes from the client. There's an implicit dependency on state internal to server which was set in Step 2. If the instance of server being used here is not the same instance of server being used in Step 2, then that internal state is not available and this step will fail.

An implementation of an OPAQUE server can store the server identity, server public and private keys, and OPRF seed(s) in a secure configuration store, and it can store client records in a database. If there was a way to serialize the internal state created in Step 2 and deserialize that state in Step 4, then the implementation can move all of its state into persistent stores.

Focusing on TestFull

The existing TestFull in ./tests is helpful in illustrating this request.

Stateful server

The current test suite uses the same instance of server for both server-side steps of the registration flow and the authentication flow.

Segmenting func testRegistration

Using bare scopes, we can segment the flow of func testRegistration to clearly identify state that carries over from one step to another. client will be running on a user's endpoint, and it's reasonable to expect that it will track its own state. Therefore, client will live in the top-level scope of the function.

	// Client
	client := p.Client()

The first step in the registration process is client.RegistrationInit. The only output that is not internal to client here is the m1s message. So, we create m1s in the function's top-level scope, and assign it within a bare scope once its value is known.

	var m1s []byte
	{
		reqReg := client.RegistrationInit(p.password)
		m1s = reqReg.Serialize()
	}

In this round, server will need to keep some internal state, so it will exist as a single instance in the function's top-level scope.

	// Server
	server := p.Server()

server will recieve the m1s message sent from client, and it will respond with the m2s message. server will also generate a random credID which it needs to remember in a later step. Therefore, both m2s and credID are created in the function's top-level scope before continuing into a bare scope where server.RegistrationResponse is called.

	var m2s []byte
	var credID []byte
	{
		m1, err := server.DeserializeRegistrationRequest(m1s)
		if err != nil {
			t.Fatalf(dbgErr, p.Mode, err)
		}

		credID = utils.RandomBytes(32)
		respReg, err := server.RegistrationResponse(m1, p.serverPublicKey, credID, p.oprfSeed)
		if err != nil {
			t.Fatalf(dbgErr, p.Mode, err)
		}

		m2s = respReg.Serialize()
	}

Back to the client for the next step where client.RegistrationFinalize will happen. In this step, client will finish by outputting m3s for the server. Also in this step, client will learn the value of exportKeyReg, which is one of the return values of the function. Both m3s and exportKeyReg are created in the function's top-level scope, and assigned within the following bare scope.

	// Client
	var m3s []byte
	var exportKeyReg []byte
	{
		clientCreds := &opaque.Credentials{
			Client: p.username,
			Server: p.serverID,
		}

		var clientSecretKey []byte
		if p.Mode == opaque.External {
			clientSecretKey, _ = client.KeyGen()
		}

		m2, err := client.DeserializeRegistrationResponse(m2s)
		if err != nil {
			t.Fatalf(dbgErr, p.Mode, err)
		}

		upload, key, err := client.RegistrationFinalize(clientSecretKey, clientCreds, m2)
		if err != nil {
			t.Fatalf(dbgErr, p.Mode, err)
		}
		exportKeyReg = key

		m3s = upload.Serialize()
	}

Finally, server recieves m3s and can store a ClientRecord. There is nothing new learned in this step, but this is when server persists the ClientRecord to complete the registration process. Note that server here is using credID which it had to remember from earlier.

	// Server
	{
		m3, err := server.DeserializeRegistrationUpload(m3s)
		if err != nil {
			t.Fatalf(dbgErr, p.Mode, err)
		}

		return &opaque.ClientRecord{
			CredentialIdentifier: credID,
			ClientIdentity:       p.username,
			RegistrationUpload:   m3,
		}, exportKeyReg
	}

Collapsing the bare scopes, it's easy to identify that the three registration messages (m1s, m2s, and m3s) must persist between steps. credID also must persist because server must remember it between steps 2 and 4. Finally, exportKeyReg must persist because it is a return value of the function.

The complete `func testRegistration`

func testRegistration(t *testing.T, p *testParams) (*opaque.ClientRecord, []byte) {
	// Client
	client := p.Client()

	var m1s []byte
	{
		reqReg := client.RegistrationInit(p.password)
		m1s = reqReg.Serialize()
	}

	// Server
	server := p.Server()

	var m2s []byte
	var credID []byte
	{
		m1, err := server.DeserializeRegistrationRequest(m1s)
		if err != nil {
			t.Fatalf(dbgErr, p.Mode, err)
		}

		credID = utils.RandomBytes(32)
		respReg, err := server.RegistrationResponse(m1, p.serverPublicKey, credID, p.oprfSeed)
		if err != nil {
			t.Fatalf(dbgErr, p.Mode, err)
		}

		m2s = respReg.Serialize()
	}

	// Client
	var m3s []byte
	var exportKeyReg []byte
	{
		clientCreds := &opaque.Credentials{
			Client: p.username,
			Server: p.serverID,
		}

		var clientSecretKey []byte
		if p.Mode == opaque.External {
			clientSecretKey, _ = client.KeyGen()
		}

		m2, err := client.DeserializeRegistrationResponse(m2s)
		if err != nil {
			t.Fatalf(dbgErr, p.Mode, err)
		}

		upload, key, err := client.RegistrationFinalize(clientSecretKey, clientCreds, m2)
		if err != nil {
			t.Fatalf(dbgErr, p.Mode, err)
		}
		exportKeyReg = key

		m3s = upload.Serialize()
	}

	// Server
	{
		m3, err := server.DeserializeRegistrationUpload(m3s)
		if err != nil {
			t.Fatalf(dbgErr, p.Mode, err)
		}

		return &opaque.ClientRecord{
			CredentialIdentifier: credID,
			ClientIdentity:       p.username,
			RegistrationUpload:   m3,
		}, exportKeyReg
	}
}

Segmenting func testAuthentication

The same exercise is performed for func testAuthentication.

Once again, client will exist in the function's top-level scope so that it can track its state.

	// Client
	client := p.Client()

In the first step, client creates m4s which is sent to server.

	var m4s []byte
	{
		ke1 := client.Init(p.password)
		m4s = ke1.Serialize()
	}

Again for this round, server will exist in the same scope as client.

	// Server
	server := p.Server()

In step 2, server recieves m4s and does server.Init. Other inputs to server.Init include p.serverSecretKey and p.oprfSeed, which are known externally and passed in as parameters, and record which server ostensibly would retrieve from a persistent store. The output of this step is m5s, which is sent to client. There is also some internal state created in server.Ake.

	var m5s []byte
	{
		m4, err := server.DeserializeKE1(m4s)
		if err != nil {
			t.Fatalf(dbgErr, p.Mode, err)
		}

		ke2, err := server.Init(m4, p.serverID, p.serverSecretKey, p.serverPublicKey, p.oprfSeed, record)
		if err != nil {
			t.Fatalf(dbgErr, p.Mode, err)
		}

		m5s = ke2.Serialize()
	}

In step 3, client will create m6s to be sent to server. Also, client will learn exportKeyLogin and clientKey, both of which are used at the end of the function to validate that the protocol has run successfully; therefore, those values persist between steps.

	// Client
	var m6s []byte
	var exportKeyLogin []byte
	var clientKey []byte
	{
		m5, err := client.DeserializeKE2(m5s)
		if err != nil {
			t.Fatalf(dbgErr, p.Mode, err)
		}

		ke3, key, err := client.Finish(p.username, p.serverID, m5)
		if err != nil {
			t.Fatalf(dbgErr, p.Mode, err)
		}
		exportKeyLogin = key

		m6s = ke3.Serialize()
		clientKey = client.SessionKey()
	}

In step 4, server recieves m6s and does server.Finish. Here, server will learn serverKey, which also is used at the end of the function to validate that the protocol has run successfully, so it will persist beyond this step.

	// Server
	var serverKey []byte
	{
		m6, err := server.DeserializeKE3(m6s)
		if err != nil {
			t.Fatalf(dbgErr, p.Mode, err)
		}

		if err := server.Finish(m6); err != nil {
			t.Fatalf(dbgErr, p.Mode, err)
		}

		serverKey = server.SessionKey()
	}

Finally, the test asserts the equality of clientKey and serverKey, and returns exportKeyLogin for its caller to compare to exportKeyReg.

	if !bytes.Equal(clientKey, serverKey) {
		t.Fatalf("mode %v: session keys differ", p.Mode)
	}

	return exportKeyLogin

With this segmentation in place, the test suite still passes.

$ go test -v -run=TestFull ./tests
=== RUN   TestFull
--- PASS: TestFull (0.00s)
PASS
ok      github.com/bytemare/opaque/tests        0.189s

Stateless server

If different instances of opaque.Server are used for step 2 and step 4 of the registration flow, registration will still complete successfully as long as those two instances are configured in the same way (i.e., using the same OPAQUE configuration) because there is no state carried internally to server between the steps of the registration flow.

Stateless Registration

func testRegistration is modified so that a new instance of server is created within the bare scope of each server-side step in the flow.

diff --git a/tests/opaque_test.go b/tests/opaque_test.go
index 5e39f01..b5474d2 100644
--- a/tests/opaque_test.go
+++ b/tests/opaque_test.go
@@ -78,11 +78,10 @@ func testRegistration(t *testing.T, p *testParams) (*opaque.ClientRecord, []byte
        }

        // Server
-       server := p.Server()
-
        var m2s []byte
        var credID []byte
        {
+               server := p.Server()
                m1, err := server.DeserializeRegistrationRequest(m1s)
                if err != nil {
                        t.Fatalf(dbgErr, p.Mode, err)
@@ -127,6 +126,7 @@ func testRegistration(t *testing.T, p *testParams) (*opaque.ClientRecord, []byte

        // Server
        {
+               server := p.Server()
                m3, err := server.DeserializeRegistrationUpload(m3s)
                if err != nil {
                        t.Fatalf(dbgErr, p.Mode, err)

With this change, the test suite still completes successfully.

$ go test -v -run=TestFull ./tests
=== RUN   TestFull
--- PASS: TestFull (0.00s)
PASS
ok      github.com/bytemare/opaque/tests        0.570s

Stateless Authentication

func testAuthentication is modified so that a new instance of server is created within the bare scope of each server-side step in the flow.

diff --git a/tests/opaque_test.go b/tests/opaque_test.go
index b5474d2..dc67319 100644
--- a/tests/opaque_test.go
+++ b/tests/opaque_test.go
@@ -151,10 +151,9 @@ func testAuthentication(t *testing.T, p *testParams, record *opaque.ClientRecord
        }

        // Server
-       server := p.Server()
-
        var m5s []byte
        {
+               server := p.Server()
                m4, err := server.DeserializeKE1(m4s)
                if err != nil {
                        t.Fatalf(dbgErr, p.Mode, err)
@@ -191,6 +190,7 @@ func testAuthentication(t *testing.T, p *testParams, record *opaque.ClientRecord
        // Server
        var serverKey []byte
        {
+               server := p.Server()
                m6, err := server.DeserializeKE3(m6s)
                if err != nil {
                        t.Fatalf(dbgErr, p.Mode, err)

With this change, the test suite fails.

$ go test -v -run=TestFull ./tests
=== RUN   TestFull
    opaque_test.go:200: Mode 1: failed to authenticate client: invalid client mac
--- FAIL: TestFull (0.00s)
FAIL
FAIL    github.com/bytemare/opaque/tests        0.276s
FAIL

This failure is from a call to server.Finish which then calls server.Ake.Finalize, which calls p.MAC.Equal(server.Ake.clientMac, ke3.Mac) (where p is server's Parameters specifying the MAC to use). server.Ake.clientMac is set during step 2 as a side effect of server.Init, so the original value was lost when execution exited the bare scope of step 2.

Making it Work

ake.Server is composed of two unexported fields, clientMac and sessionSecret, both of which are []byte. In order to persist this state externally from the instance of ake.Server (and the opaque.Server that referencs it), it's necessary to use reflect and unsafe.

AKE Server State Serde

The first goal is to extract the state so that it can be persisted elsewhere. This is accomplished by using reflect to access the unexported fields, and encoding/gob to copy the values of those fields into a []byte returned to the caller.

func serializeAkeServerState(v interface{}) []byte {
	var buf bytes.Buffer
	enc := gob.NewEncoder(&buf)

	rv := reflect.ValueOf(v)
	// if it's a pointer, then derefence
	if rv.Kind() == reflect.Ptr {
		rv = rv.Elem()
	}

	enc.EncodeValue(rv.FieldByName("clientMac"))
	enc.EncodeValue(rv.FieldByName("sessionSecret"))

	return buf.Bytes()
}

The second goal is to rehydrate an ake.Server by deserializing the state onto a new instance. This is not easily done, since the target fields are unexported and therefore not settable by reflect. However, since they are []byte, which is a reference type, unsafe provides a solution. For each field, the state is decoded into var b []byte, and the field's address in memory is updated to point to the decoded value.

func deserializeAkeServerState(data []byte, v interface{}) error {
	dec := gob.NewDecoder(bytes.NewBuffer(data))

	rv := reflect.ValueOf(v)
	if rv.Kind() != reflect.Ptr {
		return fmt.Errorf("cannot decode onto non-pointer %s", reflect.TypeOf(v))
	}
	if rv.IsNil() {
		return fmt.Errorf("cannot decode onto nil")
	}

	rv = rv.Elem()

	for _, name := range []string{"clientMac", "sessionSecret"} {
		var b []byte
		err := dec.Decode(&b)
		if err != nil {
			return err
		}

		addr := rv.FieldByName(name).UnsafeAddr()
		val := (*[]byte)(unsafe.Pointer(addr))
		*val = b
	}

	return nil
}

Update func testAuthentication to Persist State

In step 2, a new value state will persist at the end of the step. Within the step's scope, it is assigned as state = serializeAkeServerState(server.Ake). Then, in step 4, state is applied to the new instance of server like this:

		if err := deserializeAkeServerState(state, server.Ake); err != nil {
			t.Fatalf(dbgErr, p.Mode, err)
		}

Full patch

diff --git a/tests/opaque_test.go b/tests/opaque_test.go
index dc67319..dd5e5f0 100644
--- a/tests/opaque_test.go
+++ b/tests/opaque_test.go
@@ -10,7 +10,11 @@ package opaque_test

 import (
 	"bytes"
+	"encoding/gob"
+	"fmt"
+	"reflect"
 	"testing"
+	"unsafe"

 	"github.com/bytemare/cryptotools/utils"

@@ -152,6 +156,7 @@ func testAuthentication(t *testing.T, p *testParams, record *opaque.ClientRecord

 	// Server
 	var m5s []byte
+	var state []byte
 	{
 		server := p.Server()
 		m4, err := server.DeserializeKE1(m4s)
@@ -164,6 +169,8 @@ func testAuthentication(t *testing.T, p *testParams, record *opaque.ClientRecord
 			t.Fatalf(dbgErr, p.Mode, err)
 		}

+		state = serializeAkeServerState(server.Ake)
+
 		m5s = ke2.Serialize()
 	}

@@ -196,6 +203,10 @@ func testAuthentication(t *testing.T, p *testParams, record *opaque.ClientRecord
 			t.Fatalf(dbgErr, p.Mode, err)
 		}

+		if err := deserializeAkeServerState(state, server.Ake); err != nil {
+			t.Fatalf(dbgErr, p.Mode, err)
+		}
+
 		if err := server.Finish(m6); err != nil {
 			t.Fatalf(dbgErr, p.Mode, err)
 		}
@@ -281,3 +292,47 @@ func testAuthentication(t *testing.T, p *testParams, record *opaque.ClientRecord
 //	}
 //}
 //
+
+func serializeAkeServerState(v interface{}) []byte {
+	var buf bytes.Buffer
+	enc := gob.NewEncoder(&buf)
+
+	rv := reflect.ValueOf(v)
+	// if it's a pointer, then derefence
+	if rv.Kind() == reflect.Ptr {
+		rv = rv.Elem()
+	}
+
+	enc.EncodeValue(rv.FieldByName("clientMac"))
+	enc.EncodeValue(rv.FieldByName("sessionSecret"))
+
+	return buf.Bytes()
+}
+
+func deserializeAkeServerState(data []byte, v interface{}) error {
+	dec := gob.NewDecoder(bytes.NewBuffer(data))
+
+	rv := reflect.ValueOf(v)
+	if rv.Kind() != reflect.Ptr {
+		return fmt.Errorf("cannot decode onto non-pointer %s", reflect.TypeOf(v))
+	}
+	if rv.IsNil() {
+		return fmt.Errorf("cannot decode onto nil")
+	}
+
+	rv = rv.Elem()
+
+	for _, name := range []string{"clientMac", "sessionSecret"} {
+		var b []byte
+		err := dec.Decode(&b)
+		if err != nil {
+			return err
+		}
+
+		addr := rv.FieldByName(name).UnsafeAddr()
+		val := (*[]byte)(unsafe.Pointer(addr))
+		*val = b
+	}
+
+	return nil
+}

With these changes in place, the test is now passing.

$ go test -v -run=TestFull ./tests
=== RUN   TestFull
--- PASS: TestFull (0.00s)
PASS
ok      github.com/bytemare/opaque/tests        0.193s

[everyCTO]

Alternatively, an accessor for ake.Server.clientMac like func (s Server) ClientMAC() []byte might be just as useful. ake.Server.Finish really is just comparing ake.Server.clientMac to the MAC in ke3.

This would result in a smaller change to the existing implementation of ake.Server and would not involve directly setting state within the server, which I suppose is more secure. The tradeoff is that a caller would need to be more directly aware of the comparison, making a call like server.MAC.Equal(expectedMAC, m6.Mac) instead of server.Finish(m6).

Here's the full diff of taking this approach instead of implementing serde methods on ake.Server:

diff --git a/internal/ake/server.go b/internal/ake/server.go
index 29525e9..f8d89fb 100644
--- a/internal/ake/server.go
+++ b/internal/ake/server.go
@@ -84,3 +84,8 @@ func (s *Server) Finalize(p *internal.Parameters, ke3 *message.KE3) bool {
 func (s *Server) SessionKey() []byte {
 	return s.sessionSecret
 }
+
+// ClientMAC returns the expected client MAC if a previous call to Finalize() was successful.
+func (s *Server) ClientMAC() []byte {
+	return s.clientMac
+}
diff --git a/tests/opaque_test.go b/tests/opaque_test.go
index dc67319..1a48dfa 100644
--- a/tests/opaque_test.go
+++ b/tests/opaque_test.go
@@ -10,6 +10,7 @@ package opaque_test
 
 import (
 	"bytes"
+	"errors"
 	"testing"
 
 	"github.com/bytemare/cryptotools/utils"
@@ -152,6 +153,8 @@ func testAuthentication(t *testing.T, p *testParams, record *opaque.ClientRecord
 
 	// Server
 	var m5s []byte
+	var expectedMAC []byte
+	var serverKey []byte
 	{
 		server := p.Server()
 		m4, err := server.DeserializeKE1(m4s)
@@ -165,6 +168,9 @@ func testAuthentication(t *testing.T, p *testParams, record *opaque.ClientRecord
 		}
 
 		m5s = ke2.Serialize()
+
+		expectedMAC = server.Ake.ClientMAC()
+		serverKey = server.SessionKey()
 	}
 
 	// Client
@@ -188,7 +194,6 @@ func testAuthentication(t *testing.T, p *testParams, record *opaque.ClientRecord
 	}
 
 	// Server
-	var serverKey []byte
 	{
 		server := p.Server()
 		m6, err := server.DeserializeKE3(m6s)
@@ -196,11 +201,9 @@ func testAuthentication(t *testing.T, p *testParams, record *opaque.ClientRecord
 			t.Fatalf(dbgErr, p.Mode, err)
 		}
 
-		if err := server.Finish(m6); err != nil {
-			t.Fatalf(dbgErr, p.Mode, err)
+		if !server.MAC.Equal(expectedMAC, m6.Mac) {
+			t.Fatalf(dbgErr, p.Mode, errors.New("failed to authenticate client: invalid client mac"))
 		}
-
-		serverKey = server.SessionKey()
 	}
 
 	if !bytes.Equal(clientKey, serverKey) {

[bytemare]

Hi 👋

First off: thank you for your question :) Indeed, this is a desirable feature and something that we've been discussing in the spec.

I'm just back from vacation and will tackle this in the coming days. I'm thinking about something very simple, where the server can export its state in a serialized form, and import an existing state to complete the AKE (which is just the mac and session key).

[everyCTO]

I've updated my PR to implement the behavior you've described where the server can export its state in a serialized form, and import an existing state to complete the AKE.

For my use case, however, it's useful to have an accessor for just the expected MAC. By adding this, I can make a single call to the database (in a transaction) to assert that the MAC I have from the client matches the expected MAC, and persist an authenticated session. This is instead of calling server.Finish to assert that the MAC provided by the client matches the MAC expected by the server.

This means that the database is comparing MACs, which loses the benefit of subtle.ConstantTimeCompare. However, it seems to me that network jitter would introduce enough noise that a timing attack against that comparison would be difficult to perform. I welcome additional thoughts here.

[bytemare]

Merged in #8