notation rc.2 scope

[yizha1]

Here are the candidates for notation rc.2 from user scenarios point of view.

• Inspect signatures

• https://github.com/notaryproject/notation/issues/172

• Key Discovery #95

• Support OCI Image manifest to store signatures

• [Fallback Support]Notation uses OCI image manifest to store the signature in the repository. notation#444

• Use CLI to manage Trust Policy

• https://github.com/notaryproject/roadmap/issues/39

• Signature filtering

• During verification use signature filtering  notation-go#197

• Implementing signature filtering notation-go#70

• Offline Sign

• [User Story] - notation sign/verify Offline/Local artifacts roadmap#15

• Certificate revocation

• [Spec] Certificate revocation roadmap#11

• Signature revocation

• Signature RFC3161 timestamping

• Tag signing

• Quality: E2E Testing automation

• API Testing notation-go#52

• E2E Automation Testing for CLI commands roadmap#47

[yizha1]

Copied from @iamsamirzon 's slack message

Fall back support - 1
Notation Inspect signatures - 2
Complete Debug option -2
Signature filtering - 2
E2E test cases - 2
CLI cmds for TP Management - 2
Timeout Implementation - 3
API Documentation - 4
CLI Documentation - 4
Pull and Push command feature in notation - unknown (edited)

[yizha1]

Summary

Notation rc.2 scope consists of

• rc.1 leftovers
• New features
• rc.1 bugs reported after the release (not clear for now, but need reserve time for it)

Backlogs

Title	Github issue	Priority	Comments	cost estimation (Impl+test+PR review)
Fall back support	notation#444	P0	rc.1 leftover, but new scenario	2w
noation offline sign/verify	roadmap#15	P0	New feature	4w
E2E testing	roadmap#47, roadmap#63	P0	rc.1 leftover, Quality improvement	3w
CLI cmds for TP management	roadmap#39	P0	New feature	2w
inspect signatures	notation#172	P1	New feature	1w
debug option	notation#300	P1 -> P0	rc.1 leftover, troubleshooting, -> sign and verify debug logs should be implemented in rc.1	1w
signature filtering	notation-go#197	P1	New feature	2w
certificate revocation	roadmap#11, roadmap#60	P2	New feature	8w ~ 12w
Timestamp signature authority	roadmap#59	P2	New feature	8w ~ 12w
Timeout implementation	notation-go#201	P2	rc.1 leftover
API documentation	notation-go#189	P3	New feature	2w ~ 4w
CLI manual documentation	website#88	P3	Documentation improvement	2w ~ 4w
pull and push command	notation#245, notation#246	P4	any user scenarios we want to enable? could be offline sign/verify	2w
Tag signing	roadmap#16	TBD	new feature
code clean-up	notation#392, notation#446, notation-go#80, notation-core-go#96	TBD	rc.1 leftover, suggest P0	1w
Behavior of VerificationOutcome	notation-go#214	TBD	rc.1 leftover	2w ~ 3w
VerifyOptions.MaxSignatureAttempts	notation-go#212	TBD	rc.1 leftover
Release Automation via GH actions	roadmap#66	TBD	rc.1 leftover
Threat Modeling	notation#391, notation#409	TBD	rc.1 leftover	2w ~ 4w
notation list pagination	notation#403	TBD	rc.1 leftover
Support expiry duration in days	notation#423	TBD	rc.1 leftover	1w
Update plugin spec to support extended attributes	[notation#201] (#201)	TBD	rc.1 leftover

TODOs

• Align priorities including the format of priority using P0 or 1. (Done)
• Cleanup/create/update issues in roadmap repo based scope, so that we have a central place to manage the roadmap issues, otherwise the issues will be in different repos (notation, notation-go, notation-core-go). Align what kinds of issues are roadmap issues (User scenarios?) --> then we can create a roadmap board from roadmap repo issues to show what has been done, what is planned, what is in the backlogs, so that user can easily understand notation project
• Need input on the cost estimation, then we can know what is reasonable scope for rc.2 and post-rc.2 releases.
• code freeze check-in before release.
• Documentations or blogs for rc.1, since it is a major milestone. (Feynman is on it)
• Decision on release cadence: one month or two months? One month seems a bit tight considering PR review lead time.

/cc: @shizhMSFT @FeynmanZhou @toddysm @iamsamirzon @vaninrao10 @priteshbandi

[FeynmanZhou]

Can you add two links to API and CLI documentation to this table?

• API documentation: doc: provide API documentation and API architecture diagram for notation-go notation-go#189
• CLI documentation Notary project information architecture for the documentation section notaryproject.dev#88

[FeynmanZhou]

For notation pull & push commands, they are deprecated in the alpha release. These two scenarios can also be covered by ORAS CLI. Is it still valid to add them back to RC.2?

[yizha1]

rc.2 release

Date: Jan 6, 2023
Scope:

• Fall back support, 2 weeks + spec ready first, P0
• E2E testing, 3 weeks, P0 ( phased approach)
• debug option, <1 week for leftovers
• code clean-up, 1 week

[vaninrao10]

Date Proposed for rc.2 release - Jan 16, 2023

Couple of more items added for the scope of rc.2

• Semantic versioning, P0 (1 week)
• Inspect Command, P1, (3-4 weeks)
• Attestation [https://github.com/Update plugin spec to support extended attributes #201]. P1 (2 weeks)
• Revocation Spec - Only Spec (P2), (1 week)
• Review PRs

[yizha1]

rc.2 scope based on the discussion with @vaninrao10 :

Target date

Jan 6, 2023

Scope

Title	GitHub issue	Priority	need spec?	cost estimation (implement + test)	PR review	Total	comment
*Fall back support	notaryproject/notation#444	P0	Yes	2w	2w	4w
*E2E testing	notaryproject/roadmap#47, notaryproject/roadmap#63	P0	No	1.5w	1.5w	3w	Phased approach, may not be complete in rc.2
Apply semantic versioning for plugins	notaryproject/notation-go#102	P0	Yes	1w			need spec first
CLI cmd for TP Management (CLI spec)	https://github.com/notaryproject/roadmap/issues/39	P0	Yes	1w	1w	2w	only CLI spec in rc.2
Inspect signatures	notaryproject/roadmap#74	P1	Yes	1w
*debug option	notaryproject/roadmap#71	P0	No	3d	3d	6d	rc.1 leftovers
Revocation (spec)	notaryproject/roadmap#11	P2	Yes	4d			only spec in rc.2
Attestation	#201, notaryproject/roadmap#67	P1
Sign/verify offline artifacts (spec)	notaryproject/roadmap#15	P0	Yes				start spec work in rc.2