Minutes 2021 06 03

SNIA Cloud Storage Technical Working Group

Meeting Minutes, 2021-06-03

1. IP Statement:

“You are hereby informed that, under the SNIA IP Policy, your presence at this meeting may obligate you and/or your company/organization with respect to any of the work items of the SNIA Cloud Storage TWG.”

2. Attendance

Attendees: See Causeway listing

Guests: Gary M., OEData

3. Agenda review

No changes to agenda

4. Previous meeting minutes review and approval

No objections to approving last meeting minutes as documented: https://github.com/SNIA/CDMI-spec/wiki/Minutes-2021-05-27

5. Continued discussion re. approaches to container/object duality changes

Review of last week's notes.

One additional item is that we place constraints on object names, e.g. an object named ".././*/?/" would not be a valid CDMI data object or container name. We will also have to deal with this.

Use cases

• I have a CDMI namespace, which I want to be exposed via S3. - Supported today if we added S3 export
• I have an S3 bucket, which follows standard path naming conventions (e.g. "/" delimiter, etc) - Supported today if we added S3 export
• I have an S3 bucket with arbitrarily named S3 objects, some of which include "/", ".", etc. - Not supported today unless we add container/object duality
• I have a CDMI namespace, which I export via S3, and my S3 clients create objects with arbitrarily named S3 objects, some of which include "/", ".", etc. - Not supported today unless we add container/object duality

Restrictions on object names

• We escape data object/container names, so we can support all names that are expressible in URI escaping.
• URI % escaping can encode arbitrary binary data
• JSON escaping can only support escaping unicode characters.

This limits the names of objects to unicode characters.

Can an S3 object be named "%00"?

I believe that the CDMI spec is silent on what will happen if you try to create an object with a name of "%00". This will break because we need to store the object name in JSON, and this can't be represented in JSON.

Gary checked what the current S3 behaviour is for object names (key):

• sequence of Unicode characters with a maximum length of 1024 characters.
• character ranges 00–1F hex (0–31 decimal) and 7F (127 decimal) are supported.
• Non-printable ASCII characters (128–255 decimal characters) are supported.

We need to determine if these supported character ranges can be expressed as a JSON escaped string.

The following JSON is valid:

{
	"objectName": "\u0000"
}

To access this object via a URL, the spec states the following would be used:

http://www.example.com/cdmi/%00

A high-unicode character would be represented as:

{
	"objectName": "🌳"
}

http://www.example.com/cdmi/%F0%9F%8C%B3

Running tcpdump and looking at the over-the-wire request, we get:

GET /%F0%9F%8C%B3

Thus, this approach will let us represent any object name in CDMI with both JSON and URL encodings supported.

BUT

We have to look at how we handle percent decoding. E.g. if you have a request for http://www.example.com/cdmi/test%24 does this get decoded to "test/", then treated as a container, or treated as an object named "test/".

Bucket Names

There are several issues around this, but since CDMI container names are a proper superset of S3 buckets, we are in pretty good shape.

We will need to treat them similar to NFS exports, where there are strict restrictions on allowed characters. If a user tries to create an S3 export on a CDMI container that contains characters in the container name that are disallowed for S3 buckets, we can return an error.

Further discussion will continue in the next meeting

6. Prepare agenda for next meeting

7. Summary of any planned major votes at next meeting

8. Adjournment