From 230e12c2a6da377a17dbb35f639011a960576f78 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Montgomery=20Edwards=E2=81=B4=E2=81=B4=E2=81=B8?= <57072051+x448@users.noreply.github.com> Date: Sun, 30 May 2021 16:51:52 -0500 Subject: [PATCH] Update README.md (#281) Prepare for fxamacker/cbor release 2.3.0. --- README.md | 221 ++++++++++++++++++++++-------------------------------- 1 file changed, 90 insertions(+), 131 deletions(-) diff --git a/README.md b/README.md index 96b97111..7f513f8f 100644 --- a/README.md +++ b/README.md @@ -1,61 +1,97 @@ - - # fxamacker/cbor [![](https://github.com/fxamacker/cbor/workflows/ci/badge.svg)](https://github.com/fxamacker/cbor/actions?query=workflow%3Aci) [![](https://github.com/fxamacker/cbor/workflows/cover%20%E2%89%A598%25/badge.svg)](https://github.com/fxamacker/cbor/actions?query=workflow%3A%22cover+%E2%89%A598%25%22) [![](https://github.com/fxamacker/cbor/workflows/linters/badge.svg)](https://github.com/fxamacker/cbor/actions?query=workflow%3Alinters) -[![](https://github.com/fxamacker/images/raw/master/cbor/v2.2.0/release_version_badge.svg?sanitize=1)](https://github.com/fxamacker/cbor/releases) -[![](https://github.com/fxamacker/images/raw/master/cbor/v2.2.0/license_badge.svg?sanitize=1)](https://raw.githubusercontent.com/fxamacker/cbor/master/LICENSE) +[![Go Report Card](https://goreportcard.com/badge/github.com/fxamacker/cbor)](https://goreportcard.com/report/github.com/fxamacker/cbor) +[![](https://github.com/fxamacker/images/raw/master/cbor/v2.3.0/release_version_badge.svg?sanitize=1)](https://github.com/fxamacker/cbor/releases) +[![](https://github.com/fxamacker/images/raw/master/cbor/v2.3.0/license_badge.svg?sanitize=1)](https://raw.githubusercontent.com/fxamacker/cbor/master/LICENSE) + +[__fxamacker/cbor__](https://github.com/fxamacker/cbor) is a CBOR library in [Go](https://golang.org). It's designed to be safe, fast, small, and easy to use. -[__fxamacker/cbor__](https://github.com/fxamacker/cbor) is a CBOR encoder & decoder in [Go](https://golang.org). It's designed to be small, fast, safe and easy. +Features include CBOR tags, duplicate map key detection, float64→32→16, Go struct tags (`toarray`, `keyasint`, `omitempty`), and a standard API. Each release passes hundreds of tests and 250+ million execs of coverage-guided fuzzing. -Features include a standard API, CBOR tags, duplicate map key detection, float64→32→16, `toarray`, `keyasint`, and more. Each release passes 375+ tests and 250+ million execs of coverage-guided fuzzing. +[CBOR](CBOR_GOLANG.md) ([RFC 7049](https://tools.ietf.org/html/rfc7049) & [RFC 8949](https://tools.ietf.org/html/rfc8949)) is a binary data format inspired by JSON and MessagePack. CBOR is an [Internet Standard](https://en.wikipedia.org/wiki/Internet_Standard) by [IETF](https://www.ietf.org) used in W3C [WebAuthn](https://en.wikipedia.org/wiki/WebAuthn), COSE ([RFC 8152](https://tools.ietf.org/html/rfc8152)), CWT ([RFC 8392 CBOR Web Token](https://tools.ietf.org/html/rfc8392)), and CDDL [(RFC 8610)](https://datatracker.ietf.org/doc/html/rfc8610). -[CBOR](CBOR_GOLANG.md) ([RFC 7049](https://tools.ietf.org/html/rfc7049) & [RFC 8949](https://tools.ietf.org/html/rfc8949)) is a binary data format inspired by JSON and MessagePack. CBOR is an [Internet Standard](https://en.wikipedia.org/wiki/Internet_Standard) by [IETF](https://www.ietf.org) used in W3C [WebAuthn](https://en.wikipedia.org/wiki/WebAuthn), COSE ([RFC 8152](https://tools.ietf.org/html/rfc8152)), CWT ([RFC 8392 CBOR Web Token](https://tools.ietf.org/html/rfc8392)), and etc. +[CBOR Library Installation](https://github.com/x448/cbor/edit/patch-11/README.md#cbor-library-installation) shows how to install and begin using this CBOR encoder and decoder. -## CBOR Library Security +## CBOR Security -__fxamacker/cbor__ is secure. It rejects malformed CBOR data and can detect duplicate map keys. It doesn't crash when decoding bad CBOR data by having extensive tests, coverage-guided fuzzing, data validation, and avoiding Go's `unsafe` pkg. +__fxamacker/cbor__ is secure. It rejects malformed CBOR data and can detect duplicate map keys. It doesn't crash when decoding bad CBOR data by having extensive tests, coverage-guided fuzzing, data validation, and avoiding Go's `unsafe` package. -| | fxamacker/cbor 1.0-2.x | ugorji/go 1.1.0-1.1.7 | +| | fxamacker/cbor (all versions) | ugorji/go (1.1.0 - 1.1.7) | | :--- | :------------------ | :--------------- | -| **Malformed CBOR 1** | 59.8 ns/op, 32 B/op, 1 allocs/op | :boom: fatal error: out of memory | -| **Malformed CBOR 2** | 149 ns/op, 128 B/op, 3 allocs/op | :boom: runtime: out of memory: cannot allocate | -| | Correctly rejected bad data. | :warning: Only 1 decode < 10 bytes can exhaust memory. | +| **Malformed CBOR 1** | 87.5 ns/op, 24 B/op, 2 allocs/op | :boom: fatal error: out of memory | +| **Malformed CBOR 2** | 89.5 ns/op, 24 B/op, 2 allocs/op | :boom: runtime: out of memory: cannot allocate | +| | Correctly rejected bad data in all versions.
Benchmark is from latest release. | :warning: Just 1 decode of 9 bytes can exhaust memory. | -For more info, see [RFC 8949 Section 10 (Security Considerations)](https://tools.ietf.org/html/rfc8949#section-10) or [RFC 7049 Section 8](https://tools.ietf.org/html/rfc7049#section-8). +fxamacker/cbor CBOR safety settings include: MaxNestedLevels, MaxArrayElements, MaxMapPairs, and IndefLength. -## CBOR Library Performance +For more info, see: + - [RFC 8949 Section 10 (Security Considerations)](https://tools.ietf.org/html/rfc8949#section-10) or [RFC 7049 Section 8](https://tools.ietf.org/html/rfc7049#section-8). + - [Go warning](https://golang.org/pkg/unsafe/), "Packages that import unsafe may be non-portable and are not protected by the Go 1 compatibility guidelines." -__fxamacker/cbor__ is fast without sacrificing security. +## CBOR Performance -![alt text](https://github.com/fxamacker/images/raw/master/cbor/v2.2.0/cbor_speed_comparison.svg?sanitize=1 "CBOR speed comparison chart") +__fxamacker/cbor__ is fast without sacrificing security. It can be faster than libraries relying on `unsafe` package. + +![alt text](https://github.com/fxamacker/images/raw/master/cbor/v2.3.0/cbor_speed_comparison.svg?sanitize=1 "CBOR speed comparison chart") __Click to expand:__
- CBOR Program Size Comparison

+

👉 CBOR Program Size Comparison

__fxamacker/cbor__ produces smaller programs without sacrificing features. -![alt text](https://github.com/fxamacker/images/raw/master/cbor/v2.2.0/cbor_size_comparison.svg?sanitize=1 "CBOR program size comparison chart") +![alt text](https://github.com/fxamacker/images/raw/master/cbor/v2.3.0/cbor_size_comparison.svg?sanitize=1 "CBOR program size comparison chart")

-
- CBOR Memory Comparison

+

👉 fxamacker/cbor 2.3.0 (safe) vs ugorji/go 1.2.6 (unsafe)

-__fxamacker/cbor__ uses less memory for encoding and decoding data such as CBOR Web Tokens. +fxamacker/cbor 2.3.0 (not using `unsafe`) is faster than ugorji/go 1.2.6 (using `unsafe`). -| | fxamacker/cbor 2.2 | ugorji/go 1.1.7 | -| :--- | :--- | :--- | -| Encode CWT | 176 bytes/op     2 allocs/op | 1424 bytes/op     4 allocs/op | -| Decode CWT | 176 bytes/op     6 allocs/op |   568 bytes/op     6 allocs/op | - -

+``` +name old time/op new time/op delta +DecodeCWTClaims-4 2.06µs ± 1% 1.25µs ± 0% -39.57% (p=0.000 n=10+9) +DecodeCOSE/128-Bit_Symmetric_Key-4 1.47µs ± 1% 0.86µs ± 0% -41.25% (p=0.000 n=9+9) +DecodeCOSE/256-Bit_Symmetric_Key-4 1.50µs ± 2% 0.88µs ± 0% -41.63% (p=0.000 n=10+10) +DecodeCOSE/ECDSA_P256_256-Bit_Key-4 2.22µs ± 2% 1.45µs ± 0% -34.65% (p=0.000 n=10+10) +DecodeWebAuthn-4 1.55µs ± 0% 1.32µs ± 0% -14.97% (p=0.000 n=9+10) +EncodeCWTClaims-4 1.46µs ± 0% 0.78µs ± 0% -46.52% (p=0.000 n=10+10) +EncodeCOSE/128-Bit_Symmetric_Key-4 1.79µs ± 1% 0.91µs ± 0% -49.38% (p=0.000 n=9+10) +EncodeCOSE/256-Bit_Symmetric_Key-4 1.79µs ± 1% 0.91µs ± 0% -49.15% (p=0.000 n=10+10) +EncodeCOSE/ECDSA_P256_256-Bit_Key-4 2.09µs ± 1% 1.14µs ± 0% -45.41% (p=0.000 n=10+10) +EncodeWebAuthn-4 981ns ± 0% 823ns ± 1% -16.05% (p=0.000 n=10+10) + +name old alloc/op new alloc/op delta +DecodeCWTClaims-4 760B ± 0% 176B ± 0% -76.84% (p=0.000 n=10+10) +DecodeCOSE/128-Bit_Symmetric_Key-4 800B ± 0% 240B ± 0% -70.00% (p=0.000 n=10+10) +DecodeCOSE/256-Bit_Symmetric_Key-4 816B ± 0% 256B ± 0% -68.63% (p=0.000 n=10+10) +DecodeCOSE/ECDSA_P256_256-Bit_Key-4 913B ± 0% 352B ± 0% -61.45% (p=0.000 n=10+10) +DecodeWebAuthn-4 1.56kB ± 0% 0.99kB ± 0% -36.41% (p=0.000 n=10+10) +EncodeCWTClaims-4 1.36kB ± 0% 0.18kB ± 0% -87.06% (p=0.000 n=10+10) +EncodeCOSE/128-Bit_Symmetric_Key-4 1.97kB ± 0% 0.22kB ± 0% -88.62% (p=0.000 n=10+10) +EncodeCOSE/256-Bit_Symmetric_Key-4 1.97kB ± 0% 0.24kB ± 0% -87.80% (p=0.000 n=10+10) +EncodeCOSE/ECDSA_P256_256-Bit_Key-4 1.97kB ± 0% 0.32kB ± 0% -83.74% (p=0.000 n=10+10) +EncodeWebAuthn-4 1.31kB ± 0% 1.09kB ± 0% -17.07% (p=0.000 n=10+10) + +name old allocs/op new allocs/op delta +DecodeCWTClaims-4 6.00 ± 0% 6.00 ± 0% ~ (all equal) +DecodeCOSE/128-Bit_Symmetric_Key-4 4.00 ± 0% 4.00 ± 0% ~ (all equal) +DecodeCOSE/256-Bit_Symmetric_Key-4 4.00 ± 0% 4.00 ± 0% ~ (all equal) +DecodeCOSE/ECDSA_P256_256-Bit_Key-4 7.00 ± 0% 7.00 ± 0% ~ (all equal) +DecodeWebAuthn-4 5.00 ± 0% 5.00 ± 0% ~ (all equal) +EncodeCWTClaims-4 4.00 ± 0% 2.00 ± 0% -50.00% (p=0.000 n=10+10) +EncodeCOSE/128-Bit_Symmetric_Key-4 6.00 ± 0% 2.00 ± 0% -66.67% (p=0.000 n=10+10) +EncodeCOSE/256-Bit_Symmetric_Key-4 6.00 ± 0% 2.00 ± 0% -66.67% (p=0.000 n=10+10) +EncodeCOSE/ECDSA_P256_256-Bit_Key-4 6.00 ± 0% 2.00 ± 0% -66.67% (p=0.000 n=10+10) +EncodeWebAuthn-4 4.00 ± 0% 2.00 ± 0% -50.00% (p=0.000 n=10+10) +``` +
-Benchmarks used example data from [RFC 8392 Appendix A.1](https://tools.ietf.org/html/rfc8392#appendix-A.1) and default options for CBOR libraries. +Benchmarks used Go 1.15.12, linux_amd64 with data from [RFC 8392 Appendix A.1](https://tools.ietf.org/html/rfc8392#appendix-A.1). Default build options were used for all CBOR libraries. Library init code was put outside the benchmark loop for all libraries compared. ## CBOR Library API @@ -67,15 +103,24 @@ __Standard API__. Function signatures identical to [`encoding/json`](https://go __Standard Interfaces__. Custom encoding and decoding is handled by implementing: `BinaryMarshaler`, `BinaryUnmarshaler`, `Marshaler`, and `Unmarshaler`. -It's also designed to simplify concurrency and allow fast parallelism. CBOR options can be used without creating unintended runtime side-effects. +__Predefined Encoding Options__. Encoding options are easy to use and are customizable. + +```go +func CanonicalEncOptions() EncOptions {} // RFC 7049 Canonical CBOR +func CTAP2EncOptions() EncOptions {} // FIDO2 CTAP2 Canonical CBOR +func CoreDetEncOptions() EncOptions {} // RFC 8949 Core Deterministic Encoding +func PreferredUnsortedEncOptions() EncOptions {} // RFC 8949 Preferred Serialization +``` + +fxamacker/cbor designed to simplify concurrency. CBOR options can be used without creating unintended runtime side-effects. -## Time-Saving Features +## Go Struct Tags -__fxamacker/cbor__ saves time. It has time-saving features like __`toarray`__ and __`keyasint`__ struct tags. +__fxamacker/cbor__ provides Go struct tags like __`toarray`__ and __`keyasint`__ to save time and reduce encoded size of data.
-![alt text](https://github.com/fxamacker/images/raw/master/cbor/v2.2.0/cbor_struct_tags_api.svg?sanitize=1 "CBOR API and Go Struct Tags") +![alt text](https://github.com/fxamacker/images/raw/master/cbor/v2.3.0/cbor_struct_tags_api.svg?sanitize=1 "CBOR API and Go Struct Tags") ## CBOR Features @@ -92,72 +137,19 @@ __fxamacker/cbor__ is a full-featured CBOR encoder and decoder. | ☑️ | Basic validity checks | Check UTF-8 validity and optionally check duplicate map keys. | | ☑️ | Security considerations | Prevent integer overflow and resource exhaustion (RFC 8949 Section 10). | -
- - - -⚓ [Quick Start](#quick-start) • [Status](#current-status) • [Design Goals](#design-goals) • [Features](#features) • [Standards](#standards) • [API](#api) • [Options](#options) • [Usage](#usage) • [Fuzzing](#fuzzing-and-code-coverage) • [License](#license) - -## Installation - -Using Go modules is recommended. - -Use "/v2" when using `go get`. +fxamacker/cbor supports Go 1.12 and newer versions. Init the Go module, go get v2, and begin coding. ``` -$ GO111MODULE=on go get github.com/fxamacker/cbor/v2 +go mod init github.com/my_name/my_repo +go get github.com/fxamacker/cbor/v2 ``` -Also use "/v2" when importing. - ```go -import ( - "github.com/fxamacker/cbor/v2" // imports as package "cbor" -) +import "github.com/fxamacker/cbor/v2" // imports as cbor ``` -If Go modules aren't used, delete or modify example_test.go to change the import -from `"github.com/fxamacker/cbor/v2"` to `"github.com/fxamacker/cbor"` - -## System Requirements - -Using Go modules is recommended but not required. - -* Go 1.12 (or newer). -* amd64, arm64, ppc64le and s390x. Other architectures may also work but they are not tested as frequently. - -If Go modules feature isn't used, please see [Installation](#installation) about deleting or modifying example_test.go. - ## Quick Start 🛡️ Use Go's `io.LimitReader` to limit size when decoding very large or indefinite size data. @@ -173,11 +165,8 @@ If default options are acceptable, package level functions can be used for encod ```go b, err := cbor.Marshal(v) // encode v to []byte b - err := cbor.Unmarshal(b, &v) // decode []byte b to v - encoder := cbor.NewEncoder(w) // create encoder with io.Writer w - decoder := cbor.NewDecoder(r) // create decoder with io.Reader r ``` @@ -256,42 +245,12 @@ The following sections provide more info: ⚓ [Quick Start](#quick-start) • [Status](#current-status) • [Design Goals](#design-goals) • [Features](#features) • [Standards](#standards) • [API](#api) • [Options](#options) • [Usage](#usage) • [Fuzzing](#fuzzing-and-code-coverage) • [License](#license) ## Current Status -Latest version is v2.2, which has: +Latest version is v2.3 (May 30, 2021), which has: * __Stable API__ – Six codec function signatures will never change. No breaking API changes for other funcs in same major version. * __Passed all tests__ – v2.x passed all 375+ tests on amd64, arm64, ppc64le and s390x with linux. * __Passed fuzzing__ – v2.2 passed 459+ million execs in coverage-guided fuzzing on Feb 24, 2020 (release date) -and 3.2+ billion execs on March 7, 2020. - -Release 2.3 is tentatively planned for summer 2021. - - +and 3.2+ billion execs on March 7, 2020. v2.3 passed 357+ million execs on May 30, 2021 (and is continuing to fuzz).
@@ -701,7 +660,7 @@ __Click to expand:__
Example Using CBOR Web Tokens

-![alt text](https://github.com/fxamacker/images/raw/master/cbor/v2.2.0/cbor_struct_tags_api.svg?sanitize=1 "CBOR API and Go Struct Tags") +![alt text](https://github.com/fxamacker/images/raw/master/cbor/v2.3.0/cbor_struct_tags_api.svg?sanitize=1 "CBOR API and Go Struct Tags")

@@ -931,7 +890,7 @@ The `keyasint`, `toarray`, and `omitempty` struct tags make it easy to use compa
-![alt text](https://github.com/fxamacker/images/raw/master/cbor/v2.2.0/cbor_struct_tags_api.svg?sanitize=1 "CBOR API and Struct Tags") +![alt text](https://github.com/fxamacker/images/raw/master/cbor/v2.3.0/cbor_struct_tags_api.svg?sanitize=1 "CBOR API and Struct Tags")
@@ -1025,12 +984,12 @@ __This library is safer__. Small malicious CBOR messages are rejected quickly b __This library is smaller__. Programs like senmlCat can be 4 MB smaller by switching to this library. Programs using more complex CBOR data types can be 9.2 MB smaller. -![alt text](https://github.com/fxamacker/images/raw/master/cbor/v2.2.0/cbor_size_comparison.svg?sanitize=1 "CBOR speed comparison chart") +![alt text](https://github.com/fxamacker/images/raw/master/cbor/v2.3.0/cbor_size_comparison.svg?sanitize=1 "CBOR speed comparison chart") __This library is faster__ for encoding and decoding CBOR Web Token (CWT). However, speed is only one factor and it can vary depending on data types and sizes. Unlike the other library, this one doesn't use Go's ```unsafe``` package or code gen. -![alt text](https://github.com/fxamacker/images/raw/master/cbor/v2.2.0/cbor_speed_comparison.svg?sanitize=1 "CBOR speed comparison chart") +![alt text](https://github.com/fxamacker/images/raw/master/cbor/v2.3.0/cbor_speed_comparison.svg?sanitize=1 "CBOR speed comparison chart") __This library uses less memory__ for encoding and decoding CBOR Web Token (CWT) using test data from RFC 8392 A.1. @@ -1137,7 +1096,7 @@ __Words of encouragement and support__ ## License -Copyright © 2019-present [Faye Amacker](https://github.com/fxamacker). +Copyright © 2019-2021 [Faye Amacker](https://github.com/fxamacker). fxamacker/cbor is licensed under the MIT License. See [LICENSE](LICENSE) for the full license text.