Implement certificate transparency monitoring workflow #536

linus-sun · 2024-11-14T20:42:15Z

Summary

Per changes outlined in this doc, this PR implements the certificate transparency monitoring workflow functionality by doing the following:

Implements RunConsistencyCheck to verify the consistency proof of a certificate transparency log's signed tree head
Implements runnable ct_monitor.go file to perform the identity monitor and consistency check functionality on a certificate transparency log

This PR depends on #527, #534, #535

Release Note

NONE

Documentation

N/A - future PRs will update documentation once the certificate transparency monitor is runnable from a reusable monitoring workflow

codecov · 2024-11-14T20:43:40Z

Codecov Report

Attention: Patch coverage is 1.68067% with 117 lines in your changes missing coverage. Please review.

Project coverage is 58.09%. Comparing base (d271ec7) to head (1d50007).
Report is 174 commits behind head on main.

Files with missing lines	Patch %	Lines
cmd/ct_monitor/main.go	0.00%	91 Missing ⚠️
pkg/ct/consistency.go	7.69%	24 Missing ⚠️
cmd/rekor_monitor/main.go	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #536      +/-   ##
==========================================
- Coverage   64.02%   58.09%   -5.94%     
==========================================
  Files           4       18      +14     
  Lines         303     1477    +1174     
==========================================
+ Hits          194      858     +664     
- Misses         78      535     +457     
- Partials       31       84      +53

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚨 Try these New Features:

Flaky Tests Detection - Detect and resolve failed and flaky tests

haydentherapper · 2024-11-18T19:11:56Z

cmd/ct_monitor/main.go

+
+	monitoredValues := identity.MonitoredValues{
+		CertificateIdentities: config.MonitoredValues.CertificateIdentities,
+		Subjects:              config.MonitoredValues.Subjects,


We don't need subjects, correct? Since subjects are just for special cases of SSH certificates or emails in keys

removing subjects and fingerprints from this construction

haydentherapper · 2024-11-18T19:15:57Z

cmd/ct_monitor/main.go

+			return
+		}
+
+		err = ct.RunConsistencyCheck(fulcioClient, config.LogInfoFile)


Can we flip the order of these, to first check consistency before the identity check?

also adding in the current STH as a return value to RunConsistencyCheck to reduce number of calls to GetSTH

haydentherapper · 2024-11-19T20:47:55Z

cmd/ct_monitor/main.go

+
+// Default values for monitoring job parameters
+const (
+	publicRekorServerURL     = "https://rekor.sigstore.dev"


Need to change this to ctfe.sigstore.dev/2022, and change the var name.

Can we also leave a TODO about figuring out a way to handle log shards? Can link it to #57

haydentherapper · 2024-11-19T20:49:11Z

cmd/ct_monitor/main.go

+// Default values for monitoring job parameters
+const (
+	publicRekorServerURL     = "https://rekor.sigstore.dev"
+	logInfoFileName          = "logInfo.txt"


Let's create different default file names so they don't collide if run locally. maybe ctLogInfo.txt and ctIdentities.txt or something similar?

haydentherapper · 2024-11-19T20:49:37Z

cmd/ct_monitor/main.go

+	interval := flag.Duration("interval", 5*time.Minute, "Length of interval between each periodical consistency check")
+	flag.Parse()
+
+	if *configFilePath == "" && *configYamlInput == "" {


Can we copy over the same changes here, that configs aren't required?

haydentherapper · 2024-11-19T20:49:55Z

cmd/ct_monitor/main.go

+			fmt.Fprintf(os.Stderr, "start index %d must be strictly less than end index %d", *config.StartIndex, *config.EndIndex)
+		}
+
+		_, err = ct.IdentitySearch(fulcioClient, *config.StartIndex, *config.EndIndex, monitoredValues)


Only run if there's monitorable values?

haydentherapper · 2024-11-19T20:50:51Z

cmd/ct_monitor/main.go

+		inputEndIndex := config.EndIndex
+
+		var currentSTH *ctgo.SignedTreeHead
+		currentSTH, err = ct.RunConsistencyCheck(fulcioClient, config.LogInfoFile)


Can we copy over the changes that RunConsistencyCheck returns the previous, and it's read in below? Otherwise we'll hit the same bug that we read in the latest as the previous.

haydentherapper

LGTM!

haydentherapper

One more rebase!

Signed-off-by: linus-sun <[email protected]>

linus-sun force-pushed the linussun/ct-consistency-check branch from 8d35132 to ced634c Compare November 14, 2024 20:45

linus-sun requested review from mihaimaruseac and haydentherapper November 14, 2024 20:55

linus-sun marked this pull request as ready for review November 14, 2024 20:55

haydentherapper reviewed Nov 18, 2024

View reviewed changes

linus-sun force-pushed the linussun/ct-consistency-check branch from ced634c to 56f9bc5 Compare November 19, 2024 00:26

haydentherapper requested changes Nov 19, 2024

View reviewed changes

linus-sun force-pushed the linussun/ct-consistency-check branch from 0723243 to 8936fdb Compare November 19, 2024 23:18

haydentherapper reviewed Nov 20, 2024

View reviewed changes

linus-sun force-pushed the linussun/ct-consistency-check branch 2 times, most recently from 35bfd81 to c512f58 Compare November 20, 2024 01:03

mihaimaruseac approved these changes Nov 20, 2024

View reviewed changes

haydentherapper approved these changes Nov 20, 2024

View reviewed changes

linus-sun added 5 commits November 20, 2024 02:12

add functionality for scanning OID matchers in a CT log entry

dd4ac1b

Signed-off-by: linus-sun <[email protected]>

refactor extension functions to support x509 and google_x509

948b2b5

Signed-off-by: linus-sun <[email protected]>

refactor extension functions to support x509 and google_x509

f736eed

Signed-off-by: linus-sun <[email protected]>

add consistency check and monitor functionality

3e6ff40

Signed-off-by: linus-sun <[email protected]>

refactoring ct workflow

1d50007

Signed-off-by: linus-sun <[email protected]>

linus-sun force-pushed the linussun/ct-consistency-check branch from c512f58 to 1d50007 Compare November 20, 2024 02:12

haydentherapper approved these changes Nov 20, 2024

View reviewed changes

haydentherapper merged commit a88973a into sigstore:main Nov 20, 2024
8 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement certificate transparency monitoring workflow #536

Implement certificate transparency monitoring workflow #536

linus-sun commented Nov 14, 2024 •

edited

Loading

codecov bot commented Nov 14, 2024 •

edited

Loading

haydentherapper Nov 18, 2024

linus-sun Nov 19, 2024

haydentherapper Nov 18, 2024

linus-sun Nov 19, 2024

haydentherapper Nov 19, 2024

haydentherapper Nov 19, 2024

haydentherapper Nov 19, 2024

haydentherapper Nov 19, 2024

haydentherapper Nov 19, 2024

haydentherapper left a comment

haydentherapper left a comment

Implement certificate transparency monitoring workflow #536

Implement certificate transparency monitoring workflow #536

Conversation

linus-sun commented Nov 14, 2024 • edited Loading

Summary

Release Note

Documentation

codecov bot commented Nov 14, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

haydentherapper left a comment

Choose a reason for hiding this comment

haydentherapper left a comment

Choose a reason for hiding this comment

linus-sun commented Nov 14, 2024 •

edited

Loading

codecov bot commented Nov 14, 2024 •

edited

Loading