Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Release v2.0.7 #5650

Merged
merged 4 commits into from
Jul 10, 2024
Merged

chore: Release v2.0.7 #5650

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
492446e
feat: [TKC-2194] improve workflow execution telemetry (#5648)
povilasv Jul 10, 2024
40e4ee9
TKC-2161: skip tls verification when saving logs if STORAGE_SKIP_VERI…
dejanzele Jul 10, 2024
b7f8118
build(deps): bump anchore/sbom-action from 0.16.0 to 0.16.1 (#5649)
dependabot[bot] Jul 10, 2024
1443737
feat(performance): improve Log Processing performance (#5647)
rangoo94 Jul 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 11 additions & 11 deletions .github/workflows/docker-build-api-executors-tag.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
uses: actions/checkout@v4

- uses: sigstore/[email protected]
- uses: anchore/sbom-action/[email protected].0
- uses: anchore/sbom-action/[email protected].1

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium

score is 0: third-party GitHubAction not pinned by hash
Remediation tip: update your workflow using https://app.stepsecurity.io
Click Remediation section below for further remediation help

- name: Set up Docker Buildx
id: buildx
Expand Down Expand Up @@ -94,7 +94,7 @@
uses: actions/checkout@v4

- uses: sigstore/[email protected]
- uses: anchore/sbom-action/[email protected].0
- uses: anchore/sbom-action/[email protected].1

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium

score is 0: third-party GitHubAction not pinned by hash
Remediation tip: update your workflow using https://app.stepsecurity.io
Click Remediation section below for further remediation help

- name: Set up Docker Buildx
id: buildx
Expand Down Expand Up @@ -178,7 +178,7 @@
uses: actions/checkout@v4

- uses: sigstore/[email protected]
- uses: anchore/sbom-action/[email protected].0
- uses: anchore/sbom-action/[email protected].1

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium

score is 0: third-party GitHubAction not pinned by hash
Remediation tip: update your workflow using https://app.stepsecurity.io
Click Remediation section below for further remediation help

- name: Set up QEMU
uses: docker/setup-qemu-action@v3
Expand Down Expand Up @@ -249,7 +249,7 @@
uses: actions/checkout@v4

- uses: sigstore/[email protected]
- uses: anchore/sbom-action/[email protected].0
- uses: anchore/sbom-action/[email protected].1

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium

score is 0: third-party GitHubAction not pinned by hash
Remediation tip: update your workflow using https://app.stepsecurity.io
Click Remediation section below for further remediation help

- name: Set up QEMU
uses: docker/setup-qemu-action@v3
Expand Down Expand Up @@ -317,7 +317,7 @@
uses: actions/checkout@v4

- uses: sigstore/[email protected]
- uses: anchore/sbom-action/[email protected].0
- uses: anchore/sbom-action/[email protected].1

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium

score is 0: third-party GitHubAction not pinned by hash
Remediation tip: update your workflow using https://app.stepsecurity.io
Click Remediation section below for further remediation help

- name: Set up QEMU
uses: docker/setup-qemu-action@v3
Expand Down Expand Up @@ -442,7 +442,7 @@
uses: actions/checkout@v4

- uses: sigstore/[email protected]
- uses: anchore/sbom-action/[email protected].0
- uses: anchore/sbom-action/[email protected].1

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium

score is 0: third-party GitHubAction not pinned by hash
Remediation tip: update your workflow using https://app.stepsecurity.io
Click Remediation section below for further remediation help

- name: Set up QEMU
uses: docker/setup-qemu-action@v3
Expand Down Expand Up @@ -510,7 +510,7 @@
uses: actions/checkout@v4

- uses: sigstore/[email protected]
- uses: anchore/sbom-action/[email protected].0
- uses: anchore/sbom-action/[email protected].1

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium

score is 0: third-party GitHubAction not pinned by hash
Remediation tip: update your workflow using https://app.stepsecurity.io
Click Remediation section below for further remediation help

- name: Set up QEMU
uses: docker/setup-qemu-action@v3
Expand Down Expand Up @@ -669,7 +669,7 @@
uses: actions/checkout@v4

- uses: sigstore/[email protected]
- uses: anchore/sbom-action/[email protected].0
- uses: anchore/sbom-action/[email protected].1

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium

score is 0: third-party GitHubAction not pinned by hash
Remediation tip: update your workflow using https://app.stepsecurity.io
Click Remediation section below for further remediation help

- name: Set up QEMU
uses: docker/setup-qemu-action@v3
Expand Down Expand Up @@ -741,7 +741,7 @@
uses: docker/setup-qemu-action@v3

- uses: sigstore/[email protected]
- uses: anchore/sbom-action/[email protected].0
- uses: anchore/sbom-action/[email protected].1

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium

score is 0: third-party GitHubAction not pinned by hash
Remediation tip: update your workflow using https://app.stepsecurity.io
Click Remediation section below for further remediation help

- name: Set up Docker Buildx
id: buildx
Expand Down Expand Up @@ -790,7 +790,7 @@
uses: actions/checkout@v4

- uses: sigstore/[email protected]
- uses: anchore/sbom-action/[email protected].0
- uses: anchore/sbom-action/[email protected].1

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium

score is 0: third-party GitHubAction not pinned by hash
Remediation tip: update your workflow using https://app.stepsecurity.io
Click Remediation section below for further remediation help

- name: Set up QEMU
uses: docker/setup-qemu-action@v3
Expand Down Expand Up @@ -863,7 +863,7 @@
fetch-depth: 0

- uses: sigstore/[email protected]
- uses: anchore/sbom-action/[email protected].0
- uses: anchore/sbom-action/[email protected].1

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium

score is 0: third-party GitHubAction not pinned by hash
Remediation tip: update your workflow using https://app.stepsecurity.io
Click Remediation section below for further remediation help

- name: Set up QEMU
uses: docker/setup-qemu-action@v3
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release-dev.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@
with:
fetch-depth: 0
- uses: sigstore/[email protected]
- uses: anchore/sbom-action/[email protected].0
- uses: anchore/sbom-action/[email protected].1

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium

score is 0: third-party GitHubAction not pinned by hash
Remediation tip: update your workflow using https://app.stepsecurity.io
Click Remediation section below for further remediation help
- name: Download Artifacts for Linux
uses: actions/download-artifact@master
with:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@
with:
fetch-depth: 0
- uses: sigstore/[email protected]
- uses: anchore/sbom-action/[email protected].0
- uses: anchore/sbom-action/[email protected].1

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium

score is 0: third-party GitHubAction not pinned by hash
Remediation tip: update your workflow using https://app.stepsecurity.io
Click Remediation section below for further remediation help
- name: Download Artifacts for Linux
uses: actions/download-artifact@master
with:
Expand Down
6 changes: 5 additions & 1 deletion cmd/api-server/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -273,7 +273,11 @@ func main() {
configRepository = cloudconfig.NewCloudResultRepository(grpcClient, grpcConn, cfg.TestkubeProAPIKey)
// Pro edition only (tcl protected code)
testWorkflowResultsRepository = cloudtestworkflow.NewCloudRepository(grpcClient, grpcConn, cfg.TestkubeProAPIKey)
testWorkflowOutputRepository = cloudtestworkflow.NewCloudOutputRepository(grpcClient, grpcConn, cfg.TestkubeProAPIKey)
var opts []cloudtestworkflow.Option
if cfg.StorageSkipVerify {
opts = append(opts, cloudtestworkflow.WithSkipVerify())
}
testWorkflowOutputRepository = cloudtestworkflow.NewCloudOutputRepository(grpcClient, grpcConn, cfg.TestkubeProAPIKey, opts...)
triggerLeaseBackend = triggers.NewAcquireAlwaysLeaseBackend()
artifactStorage = cloudartifacts.NewCloudArtifactsStorage(grpcClient, grpcConn, cfg.TestkubeProAPIKey)
} else {
Expand Down
25 changes: 20 additions & 5 deletions pkg/cloud/data/testworkflow/output.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package testworkflow
import (
"bytes"
"context"
"crypto/tls"
"io"
"net/http"

Expand All @@ -18,11 +19,25 @@ import (
var _ testworkflow.OutputRepository = (*CloudOutputRepository)(nil)

type CloudOutputRepository struct {
executor executor.Executor
executor executor.Executor
httpClient *http.Client
}

func NewCloudOutputRepository(client cloud.TestKubeCloudAPIClient, grpcConn *grpc.ClientConn, apiKey string) *CloudOutputRepository {
return &CloudOutputRepository{executor: executor.NewCloudGRPCExecutor(client, grpcConn, apiKey)}
type Option func(*CloudOutputRepository)

func WithSkipVerify() Option {
return func(r *CloudOutputRepository) {
transport := &http.Transport{TLSClientConfig: &tls.Config{InsecureSkipVerify: true}}
r.httpClient.Transport = transport
}
}

func NewCloudOutputRepository(client cloud.TestKubeCloudAPIClient, grpcConn *grpc.ClientConn, apiKey string, opts ...Option) *CloudOutputRepository {
r := &CloudOutputRepository{executor: executor.NewCloudGRPCExecutor(client, grpcConn, apiKey), httpClient: http.DefaultClient}
for _, opt := range opts {
opt(r)
}
return r
}

// PresignSaveLog builds presigned storage URL to save the output in Cloud
Expand Down Expand Up @@ -59,7 +74,7 @@ func (r *CloudOutputRepository) SaveLog(ctx context.Context, id, workflowName st
if err != nil {
return err
}
res, err := http.DefaultClient.Do(req)
res, err := r.httpClient.Do(req)
if err != nil {
return errors.Wrap(err, "failed to save file in cloud storage")
}
Expand All @@ -79,7 +94,7 @@ func (r *CloudOutputRepository) ReadLog(ctx context.Context, id, workflowName st
if err != nil {
return nil, err
}
res, err := http.DefaultClient.Do(req)
res, err := r.httpClient.Do(req)
if err != nil {
return nil, errors.Wrap(err, "failed to get file from cloud storage")
}
Expand Down
25 changes: 23 additions & 2 deletions pkg/telemetry/payload.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,16 @@ type Params struct {
ErrorType string `json:"error_type,omitempty"`
ErrorStackTrace string `json:"error_stacktrace,omitempty"`
TestWorkflowSteps int32 `json:"test_workflow_steps,omitempty"`
TestWorkflowExecuteCount int32 `json:"test_workflow_execute_count,omitempty"`
TestWorkflowParallelUsed bool `json:"test_workflow_parallel_used,omitempty"`
TestWorkflowMatrixUsed bool `json:"test_workflow_matrix_used,omitempty"`
TestWorkflowServicesUsed bool `json:"test_workflow_services_used,omitempty"`
TestWorkflowIsSample bool `json:"test_workflow_is_sample,omitempty"`
TestWorkflowTemplates []string `json:"testWorkflowTemplates"`
TestWorkflowImages []string `json:"testWorkflowImages"`
TestWorkflowTemplateUsed bool `json:"test_workflow_template_used,omitempty"`
TestWorkflowImage string `json:"test_workflow_image,omitempty"`
TestWorkflowArtifactUsed bool `json:"test_workflow_artifact_used,omitempty"`
TestWorkflowImage string `json:"test_workflow_image,omitempty"`
TestWorkflowKubeshopGitURI bool `json:"test_workflow_kubeshop_git_uri,omitempty"`
License string `json:"license,omitempty"`
Step string `json:"step,omitempty"`
Expand Down Expand Up @@ -84,9 +91,16 @@ type RunContext struct {

type WorkflowParams struct {
TestWorkflowSteps int32
TestWorkflowTemplateUsed bool
TestWorkflowExecuteCount int32
TestWorkflowImage string
TestWorkflowArtifactUsed bool
TestWorkflowParallelUsed bool
TestWorkflowMatrixUsed bool
TestWorkflowServicesUsed bool
TestWorkflowTemplateUsed bool
TestWorkflowIsSample bool
TestWorkflowTemplates []string
TestWorkflowImages []string
TestWorkflowKubeshopGitURI bool
}

Expand Down Expand Up @@ -290,7 +304,14 @@ func NewRunWorkflowPayload(name, clusterType string, params RunWorkflowParams) P
ClusterType: clusterType,
Context: getAgentContext(),
TestWorkflowSteps: params.TestWorkflowSteps,
TestWorkflowExecuteCount: params.TestWorkflowExecuteCount,
TestWorkflowParallelUsed: params.TestWorkflowParallelUsed,
TestWorkflowTemplateUsed: params.TestWorkflowTemplateUsed,
TestWorkflowMatrixUsed: params.TestWorkflowMatrixUsed,
TestWorkflowServicesUsed: params.TestWorkflowServicesUsed,
TestWorkflowIsSample: params.TestWorkflowIsSample,
TestWorkflowTemplates: params.TestWorkflowTemplates,
TestWorkflowImages: params.TestWorkflowImages,
TestWorkflowImage: params.TestWorkflowImage,
TestWorkflowArtifactUsed: params.TestWorkflowArtifactUsed,
TestWorkflowKubeshopGitURI: params.TestWorkflowKubeshopGitURI,
Expand Down
5 changes: 5 additions & 0 deletions pkg/testworkflows/testworkflowcontroller/channel.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ type Channel[T any] interface {
Channel() <-chan ChannelMessage[T]
Close()
Done() <-chan struct{}
CtxErr() error
}

type channel[T any] struct {
Expand Down Expand Up @@ -168,3 +169,7 @@ func (c *channel[T]) Close() {
func (c *channel[T]) Done() <-chan struct{} {
return c.ctx.Done()
}

func (c *channel[T]) CtxErr() error {
return c.ctx.Err()
}
8 changes: 4 additions & 4 deletions pkg/testworkflows/testworkflowcontroller/controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@ func (c *controller) StopController() {
}

func (c *controller) Watch(parentCtx context.Context) <-chan ChannelMessage[Notification] {
w, err := WatchInstrumentedPod(parentCtx, c.clientSet, c.signature, c.scheduledAt, c.pod, c.podEvents, WatchInstrumentedPodOptions{
ch, err := WatchInstrumentedPod(parentCtx, c.clientSet, c.signature, c.scheduledAt, c.pod, c.podEvents, WatchInstrumentedPodOptions{
JobEvents: c.jobEvents,
Job: c.job,
})
Expand All @@ -222,7 +222,7 @@ func (c *controller) Watch(parentCtx context.Context) <-chan ChannelMessage[Noti
v.Close()
return v.Channel()
}
return w.Channel()
return ch
}

// TODO: Make it actually light
Expand Down Expand Up @@ -281,15 +281,15 @@ func (c *controller) Logs(parentCtx context.Context, follow bool) io.Reader {
case <-c.podEvents.Peek(parentCtx):
case <-alignTimeoutCh:
}
w, err := WatchInstrumentedPod(parentCtx, c.clientSet, c.signature, c.scheduledAt, c.pod, c.podEvents, WatchInstrumentedPodOptions{
ch, err := WatchInstrumentedPod(parentCtx, c.clientSet, c.signature, c.scheduledAt, c.pod, c.podEvents, WatchInstrumentedPodOptions{
JobEvents: c.jobEvents,
Job: c.job,
Follow: common.Ptr(follow),
})
if err != nil {
return
}
for v := range w.Channel() {
for v := range ch {
if v.Error == nil && v.Value.Log != "" && !v.Value.Temporary {
if ref != v.Value.Ref && v.Value.Ref != "" {
ref = v.Value.Ref
Expand Down
Loading
Loading