Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add more of a narrative to the converters page #3187

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 23 additions & 3 deletions docs/develop/dotnet/converters-and-encryption.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,12 @@ tags:
- Encryption
---

This page shows the following:
By default, Temporal payloads are stored unencrypted inside of its data store.
Consequently, this means that string payloads can be read from the Temporal Web UI and CLI in plain text.
When working with sensitive data, Temporal implementers may need to adopt encryption algorithms, manage encryption keys, or restrict a subset of their users from viewing payload output.

- [Use a custom Payload Codec](#custom-payload-codec)
- [Use a custom Payload Converter](#custom-payload-converter)
A Custom Codec allows a developer to transform the payload of a message sent or received by a Temporal Client.
This ensures that the data is encrypted as it travels across the network and when it is stored in the Event History, readable only by those with access to the key.

## Custom Payload Codec {#custom-payload-codec}

Expand Down Expand Up @@ -79,6 +81,19 @@ var myClient = await TemporalClient.ConnectAsync(new("localhost:7233")
});
```

- Data **encoding** is performed by the client using the default converter provided by Temporal or your custom Data Converter when passing input to the Temporal Cluster. For example, plain text input is usually serialized into a JSON object, and can then be compressed or encrypted.
- Data **decoding** may be performed by your application logic during your Workflows or Activities as necessary, but decoded Workflow results are never persisted back to the Temporal Cluster. Instead, they are stored encoded on the Cluster, and you need to provide an additional parameter when using the [temporal workflow show](/cli/workflow#show) command or when browsing the Web UI to view output.

For reference, see the [Encryption](https://github.com/temporalio/samples-dotnet/tree/main/src/Encryption) sample.

### Using a Codec Server

A Codec Server is an HTTP server that uses your custom Codec logic to decode your data remotely.
The Codec Server is independent of the Temporal Cluster and decodes your encrypted payloads through predefined endpoints.
You create, operate, and manage access to your Codec Server in your own environment.
The `temporal `CLI and the Web UI in turn provide built-in hooks to call the Codec Server to decode encrypted payloads on demand.
Refer to the [Codec Server](/production-deployment/data-encryption) documentation for information on how to design and deploy a Codec Server.

## Payload conversion

Temporal SDKs provide a default [Payload Converter](/dataconversion#payload-converter) that can be customized to convert a custom data type to [Payload](/dataconversion#payload) and back.
Expand All @@ -89,6 +104,11 @@ The order in which your encoding Payload Converters are applied depend on the or
You can set multiple encoding Payload Converters to run your conversions.
When the Data Converter receives a value for conversion, it passes through each Payload Converter in sequence until the converter that handles the data type does the conversion.

Payload Converters can be customized independently of a Payload Codec.
Temporal's Converter architecture looks like this:

![Billing page showing Credits tab](/img/converter-architecture.png)

### Custom Payload Converter {#custom-payload-converter}

**How to use a custom Payload Converter with the .NET SDK.**
Expand Down
32 changes: 24 additions & 8 deletions docs/develop/go/converters-and-encryption.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,12 @@ tags:
- Temporal SDKs
---

This page shows how to do the following:
By default, Temporal payloads are stored unencrypted inside of its data store.
axfelix marked this conversation as resolved.
Show resolved Hide resolved
Consequently, this means that string payloads can be read from the Temporal Web UI and CLI in plain text.
When working with sensitive data, Temporal implementers may need to adopt encryption algorithms, manage encryption keys, or restrict a subset of their users from viewing payload output.

- [Use a custom Payload Codec](#custom-payload-codec)
- [Use custom payload conversion](#custom-payload-conversion)
- [Use a custom Payload Converter](#custom-payload-converter)
A Custom Codec allows a developer to transform the payload of a message sent or received by a Temporal Client.
This ensures that the data is encrypted as it travels across the network and when it is stored in the Event History, readable only by those with access to the key.

## Use a custom Payload Codec in Go {#custom-payload-codec}

Expand Down Expand Up @@ -118,12 +119,22 @@ c, err := client.Dial(client.Options{
//...
```

For reference, see the following samples:
- Data **encoding** is performed by the client using the default converter provided by Temporal or your custom Data Converter when passing input to the Temporal Cluster. For example, plain text input is usually serialized into a JSON object, and can then be compressed or encrypted.
axfelix marked this conversation as resolved.
Show resolved Hide resolved
- Data **decoding** may be performed by your application logic during your Workflows or Activities as necessary, but decoded Workflow results are never persisted back to the Temporal Cluster. Instead, they are stored encoded on the Cluster, and you need to provide an additional parameter when using the [temporal workflow show](/cli/workflow#show) command or when browsing the Web UI to view output.

- [Codec server](https://github.com/temporalio/samples-go/tree/main/codec-server)
- [Encryption](https://github.com/temporalio/samples-go/tree/main/encryption)
For reference, see the [Encryption](https://github.com/temporalio/samples-go/tree/main/encryption) sample.

## Use custom payload conversion {#custom-payload-conversion}
### Using a Codec Server
axfelix marked this conversation as resolved.
Show resolved Hide resolved

A Codec Server is an HTTP server that uses your custom Codec logic to decode your data remotely.
The Codec Server is independent of the Temporal Cluster and decodes your encrypted payloads through predefined endpoints.
You create, operate, and manage access to your Codec Server in your own environment.
The `temporal` CLI and the Web UI in turn provide built-in hooks to call the Codec Server to decode encrypted payloads on demand.
Refer to the [Codec Server](/production-deployment/data-encryption) documentation for information on how to design and deploy a Codec Server.

For reference, see the [Codec server](https://github.com/temporalio/samples-go/tree/main/codec-server) sample.

## Use custom Payload conversion {#custom-payload-conversion}

**How to customize the conversion of a payload using the Go SDK.**

Expand All @@ -133,6 +144,11 @@ The order in which your encoding Payload Converters are applied depend on the or
You can set multiple encoding Payload Converters to run your conversions.
When the Data Converter receives a value for conversion, it passes through each Payload Converter in sequence until the converter that handles the data type does the conversion.

Payload Converters can be customized independently of a Payload Codec.
Temporal's Converter architecture looks like this:

![Billing page showing Credits tab](/img/converter-architecture.png)
axfelix marked this conversation as resolved.
Show resolved Hide resolved

## How to use a custom Payload Converter in Go {#custom-payload-converter}

**How to use a custom Payload Converter using the Go SDK.**
Expand Down
31 changes: 24 additions & 7 deletions docs/develop/java/converters-and-encryption.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,12 @@ tags:
- Temporal SDKs
---

This page shows how to do the following:
By default, Temporal payloads are stored unencrypted inside of its data store.
Consequently, this means that string payloads can be read from the Temporal Web UI and CLI in plain text.
When working with sensitive data, Temporal implementers may need to adopt encryption algorithms, manage encryption keys, or restrict a subset of their users from viewing payload output.

- [Use a custom Payload Codec in Java](#custom-payload-codec)
- [Do custom Payload conversion in Java](#custom-payload-conversion)
A Custom Codec allows a developer to transform the payload of a message sent or received by a Temporal Client.
This ensures that the data is encrypted as it travels across the network and when it is stored in the Event History, readable only by those with access to the key.

## Custom Payload Codec in Java {#custom-payload-codec}

Expand Down Expand Up @@ -151,12 +153,22 @@ WorkflowServiceStubs service = WorkflowServiceStubs.newLocalServiceStubs();
.build());
```

For example implementations, see the following samples:
- Data **encoding** is performed by the client using the default converter provided by Temporal or your custom Data Converter when passing input to the Temporal Cluster. For example, plain text input is usually serialized into a JSON object, and can then be compressed or encrypted.
- Data **decoding** may be performed by your application logic during your Workflows or Activities as necessary, but decoded Workflow results are never persisted back to the Temporal Cluster. Instead, they are stored encoded on the Cluster, and you need to provide an additional parameter when using the [temporal workflow show](/cli/workflow#show) command or when browsing the Web UI to view output.

- [Encrypted Payloads](https://github.com/temporalio/samples-java/tree/main/core/src/main/java/io/temporal/samples/encryptedpayloads)
- [Remote Data Encoder and Codec Server](https://github.com/temporalio/sdk-java/tree/master/temporal-remote-data-encoder)
For reference, see the [Encryption](https://github.com/temporalio/samples-java/tree/main/core/src/main/java/io/temporal/samples/encryptedpayloads) sample.

## Custom Payload conversion {#custom-payload-conversion}
### Using a Codec Server

A Codec Server is an HTTP server that uses your custom Codec logic to decode your data remotely.
The Codec Server is independent of the Temporal Cluster and decodes your encrypted payloads through predefined endpoints.
You create, operate, and manage access to your Codec Server in your own environment.
The `temporal `CLI and the Web UI in turn provide built-in hooks to call the Codec Server to decode encrypted payloads on demand.
Refer to the [Codec Server](/production-deployment/data-encryption) documentation for information on how to design and deploy a Codec Server.

For reference, see the [Codec server](https://github.com/temporalio/sdk-java/tree/master/temporal-remote-data-encoder) sample.

## Using custom Payload conversion {#custom-payload-conversion}

**How to do custom Payload conversion using the Java SDK.**

Expand All @@ -171,6 +183,11 @@ The order in which your encoding Payload Converters are applied depend on the or
You can set multiple encoding Payload Converters to run your conversions.
When the Data Converter receives a value for conversion, it passes through each Payload Converter in sequence until the converter that handles the data type does the conversion.

Payload Converters can be customized independently of a Payload Codec.
Temporal's Converter architecture looks like this:

![Billing page showing Credits tab](/img/converter-architecture.png)

Create a custom implementation of a [PayloadConverter](https://www.javadoc.io/doc/io.temporal/temporal-sdk/latest/io/temporal/common/converter/PayloadConverter.html) interface and use the `withPayloadConverterOverrides` method to implement the custom object conversion with `DefaultDataConverter`.

`PayloadConverter` serializes and deserializes method parameters that need to be sent over the wire.
Expand Down
27 changes: 21 additions & 6 deletions docs/develop/python/converters-and-encryption.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,12 @@ tags:
- Temporal SDKs
---

This page shows the following:
By default, Temporal payloads are stored unencrypted inside of its data store.
Consequently, this means that string payloads can be read from the Temporal Web UI and CLI in plain text.
When working with sensitive data, Temporal implementers may need to adopt encryption algorithms, manage encryption keys, or restrict a subset of their users from viewing payload output.

- [How to use a custom Payload Codec](#custom-payload-codec)
- [How to use a custom Payload Converter](#custom-payload-converter)
A Custom Codec allows a developer to transform the payload of a message sent or received by a Temporal Client.
This ensures that the data is encrypted as it travels across the network and when it is stored in the Event History, readable only by those with access to the key.

## Custom Payload Codec {#custom-payload-codec}

Expand Down Expand Up @@ -86,10 +88,18 @@ client = await Client.connect(
)
```

For reference, see the following samples:
- Data **encoding** is performed by the client using the default converter provided by Temporal or your custom Data Converter when passing input to the Temporal Cluster. For example, plain text input is usually serialized into a JSON object, and can then be compressed or encrypted.
- Data **decoding** may be performed by your application logic during your Workflows or Activities as necessary, but decoded Workflow results are never persisted back to the Temporal Cluster. Instead, they are stored encoded on the Cluster, and you need to provide an additional parameter when using the [temporal workflow show](/cli/workflow#show) command or when browsing the Web UI to view output.

- [Custom converter](https://github.com/temporalio/samples-python/tree/main/custom_converter)
- [Encryption](https://github.com/temporalio/samples-python/tree/main/encryption)
For reference, see the [Encryption](https://github.com/temporalio/samples-python/tree/main/encryption) sample.

### Using a Codec Server

A Codec Server is an HTTP server that uses your custom Codec logic to decode your data remotely.
The Codec Server is independent of the Temporal Cluster and decodes your encrypted payloads through predefined endpoints.
You create, operate, and manage access to your Codec Server in your own environment.
The `temporal `CLI and the Web UI in turn provide built-in hooks to call the Codec Server to decode encrypted payloads on demand.
Refer to the [Codec Server](/production-deployment/data-encryption) documentation for information on how to design and deploy a Codec Server.

## Payload conversion

Expand All @@ -101,6 +111,11 @@ The order in which your encoding Payload Converters are applied depends on the o
You can set multiple encoding Payload Converters to run your conversions.
When the Data Converter receives a value for conversion, it passes through each Payload Converter in sequence until the converter that handles the data type does the conversion.

Payload Converters can be customized independently of a Payload Codec.
Temporal's Converter architecture looks like this:

![Billing page showing Credits tab](/img/converter-architecture.png)

### Custom Payload Converter {#custom-payload-converter}

**How to use a custom Payload Converter using the Temporal Python SDK.**
Expand Down
Binary file added static/img/converter-architecture.png
axfelix marked this conversation as resolved.
Show resolved Hide resolved
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.