Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Go1.19 sigma #49

Merged
merged 43 commits into from
Sep 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
f3e18e5
feat(core): added multi-driver support to the processor
terylt Apr 19, 2022
7ec1213
feat(go): upgrade to go 1.19 compat
araujof Dec 12, 2022
97603b5
feat(core): added multi-driver support to the processor
terylt Apr 19, 2022
9f73140
Merge branch 'multi_driver' of github.com:sysflow-telemetry/sf-proces…
araujof Dec 12, 2022
b7afe11
feat(core): initial refactoring to support multi-language policy engine
araujof Jan 24, 2023
14d0f43
refactor(core): refactor policy monitor; add default creation for int…
araujof Jan 25, 2023
84afd21
fix(core): fix remaining issues with exporter encoders after refactor…
araujof Jan 25, 2023
f63fbec
feat(core): add regexp operation
araujof Feb 7, 2023
2ff68ec
feat(core): sigma frontend with transformers
araujof Feb 8, 2023
5d1d556
feat(core): working version of the sigma compiler, supporting fieldma…
araujof Feb 9, 2023
bd48294
refactor(core): introduce generilized operators for string and intege…
araujof Feb 9, 2023
abdf6ca
feat(core): refactored operators
araujof Feb 10, 2023
168c7ce
feat(core): add transformer support for sigma and warning/handling of…
araujof Feb 13, 2023
8133742
feat(core): add regexp operation for sysflow flatrecords
araujof Feb 13, 2023
523c149
feat(core): add recursive reading of input policies; fix recursive lo…
araujof Feb 13, 2023
991dafd
feat(core): modify operations interface to return errors
araujof Feb 13, 2023
88a863d
feat(core): add String() method to Operator type
araujof Feb 14, 2023
0f48104
chore(resources): add process_creation sigma rules for Linux (for tests)
araujof Feb 14, 2023
d0ef6cc
chore(README): fix build shield URL
araujof Feb 14, 2023
0b441fe
fix(core): fix logical conjunction predicate in selections of field maps
araujof Feb 14, 2023
ffb675a
tests(core): add tests for compilers
araujof Feb 14, 2023
90cf5bb
fix(ci): bump linter and golang versions
araujof Feb 14, 2023
f58e450
chore(core): add missing copyright headers
araujof Feb 15, 2023
6e29484
Migrated plugins to new arch and added make plugins target
san-zrl Feb 16, 2023
3ead78a
Merge branch 'go1.19-sigma' of github.com:sysflow-telemetry/sf-proces…
san-zrl Feb 16, 2023
72dffb3
added make plugins target
san-zrl Feb 16, 2023
0eb10e7
update(operators): added operator functions for slices and ipv6 data …
terylpt Feb 16, 2023
6fb7495
Merge branch 'go1.19-sigma' of github.com:/sysflow-telemetry/sf-proce…
terylpt Feb 16, 2023
939cf64
feat(core, resources): add additional sysflow sigma mappings, a few c…
araujof Feb 17, 2023
a506fc9
new(resources): add sample trace for shellshock attack
araujof Feb 17, 2023
df56e6a
feat(source): updated processor to support compiling in other data so…
terylpt Feb 17, 2023
707d88b
tests(core|driver): add perf logging and event rate measurement
araujof Feb 22, 2023
135a9bf
feat(tests): add bench pipeline configs
araujof Feb 22, 2023
b16054b
new(resources): add k8s traces
araujof Feb 23, 2023
5561cdc
new(tests): add summarization script for benchmarks
araujof Feb 23, 2023
867a9de
feat(tests): new benchmark scripts
araujof Feb 23, 2023
be366c5
refactor(core): add benchmark options
araujof Feb 23, 2023
568a17d
test(scripts): update summarize script
araujof Feb 24, 2023
2798f9a
chore(core,driver): updated go mod and sum
araujof Sep 5, 2023
5b60115
fix(ci,core,driver): fix lint issues and deprecated Go features
araujof Sep 5, 2023
a223356
feat(core,plugins): track newest sf-apis; fix example plugins to work…
araujof Sep 5, 2023
20dde65
chore(merge): fix merge conflicts with dev branch
araujof Sep 5, 2023
f504d18
chore(merge): fix remaining merge conflicts
araujof Sep 5, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,13 +35,13 @@ jobs:
- name: Setup Go
uses: actions/setup-go@v3
with:
go-version: 1.17
go-version: 1.19
- name: Lint core module
uses: golangci/golangci-lint-action@v3
with:
version: v1.47.1
version: v1.51.1
working-directory: core
args: --disable=errcheck
args: --disable=errcheck --build-tags=flatrecord
lint-driver:
needs: lint-core
runs-on: ubuntu-latest
Expand All @@ -51,11 +51,11 @@ jobs:
- name: Setup Go
uses: actions/setup-go@v3
with:
go-version: 1.17
go-version: 1.19
- name: Lint driver module
uses: golangci/golangci-lint-action@v3
with:
version: v1.47.1
version: v1.51.1
working-directory: driver
args: --disable=errcheck
docker:
Expand Down
6 changes: 3 additions & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright (C) 2021 IBM Corporation.
# Copyright (C) 2022 IBM Corporation.
#
# Authors:
# Frederico Araujo <[email protected]>
Expand Down Expand Up @@ -38,8 +38,8 @@ ENV SRC_ROOT=/go/src/github.com/sysflow-telemetry/sf-processor/
RUN dnf update -y --disableplugin=subscription-manager && \
dnf install -y --disableplugin=subscription-manager wget gcc make git device-mapper-devel

RUN wget https://go.dev/dl/go1.17.7.linux-amd64.tar.gz && \
tar -C /usr/local -xzf go1.17.7.linux-amd64.tar.gz && mkdir -p $SRC_ROOT
RUN wget https://go.dev/dl/go1.19.4.linux-amd64.tar.gz && \
tar -C /usr/local -xzf go1.19.4.linux-amd64.tar.gz && mkdir -p $SRC_ROOT

# Copy sources
COPY core ${SRC_ROOT}core
Expand Down
14 changes: 11 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@ include ./makefile.manifest.inc
# Basic go commands
PATH=$(shell printenv PATH):/usr/local/go/bin
GOCMD=go
GOBUILD=$(GOCMD) build -trimpath -tags exclude_graphdriver_btrfs
GOBUILD=$(GOCMD) build -trimpath -tags "exclude_graphdriver_btrfs flatrecord"
GOCLEAN=$(GOCMD) clean
GOTEST=$(GOCMD) test -tags exclude_graphdriver_btrfs
GOGET=$(GOCMD) get -tags exclude_graphdriver_btrfs
GOTEST=$(GOCMD) test -tags "exclude_graphdriver_btrfs flatrecord"
GOGET=$(GOCMD) get -tags "exclude_graphdriver_btrfs flatrecord"
BIN=sfprocessor
OUTPUT=$(BIN)
SRC=./driver
Expand Down Expand Up @@ -80,3 +80,11 @@ pull:
up:
sudo docker-compose -f docker-compose.yml up

.PHONY: plugins
plugins:
@for dir in `find plugins -type d`; do \
if [ -f $${dir}/Makefile ]; then \
$(MAKE) -C $${dir}; \
fi; \
done

179 changes: 90 additions & 89 deletions core/exporter/encoders/ecs.go

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions core/exporter/encoders/encoder.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,13 @@ package encoders

import (
"github.com/sysflow-telemetry/sf-processor/core/exporter/commons"
"github.com/sysflow-telemetry/sf-processor/core/policyengine/engine"
"github.com/sysflow-telemetry/sf-processor/core/policyengine/source/flatrecord"
)

// Encoder converts and serializes a record into export data.
type Encoder interface {
Register(codecs map[commons.Format]EncoderFactory)
Encode(recs []*engine.Record) ([]commons.EncodedData, error)
Encode(recs []*flatrecord.Record) ([]commons.EncodedData, error)
Cleanup()
}

Expand Down
Loading
Loading