Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge its-happening <- dev #1003

Merged
merged 204 commits into from
Feb 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
204 commits
Select commit Hold shift + click to select a range
12975c2
Merge pull request #877 from SiaFoundation/dev
ChrisSchinnerl Jan 8, 2024
bb6d512
Merge pull request #883 from SiaFoundation/dev
ChrisSchinnerl Jan 10, 2024
3cabde9
worker: close pipeReader at the end of serveContent (#891)
ChrisSchinnerl Jan 12, 2024
4e55e0f
Avoid deadlock in uploads in case context gets cancelled (#892)
peterjan Jan 12, 2024
edc2889
Merge pull request #896 from SiaFoundation/chris/deadlock-fixes
ChrisSchinnerl Jan 12, 2024
13d25bc
worker: try renew contract when max revision is reached
peterjan Jan 16, 2024
969b48a
Merge pull request #908 from SiaFoundation/chris/cherry-pick-upload-d…
ChrisSchinnerl Jan 19, 2024
738d1c8
worker: add TestRefreshUploaders
peterjan Jan 22, 2024
7b45ecc
worker: update mocks
peterjan Jan 23, 2024
90bf115
worker: use host blockheight
peterjan Jan 23, 2024
71807f1
worker: update mocks
peterjan Jan 23, 2024
23b1ace
worker: improve contract and host interaction recorders
peterjan Jan 22, 2024
ac05a6e
worker: use host blockheight
peterjan Jan 23, 2024
c81e287
Merge branch 'dev' of github.com:SiaFoundation/renterd into pj/test-r…
peterjan Jan 23, 2024
7757409
cmd: add mysql settings to config prompt
n8maninger Jan 23, 2024
d0b2c3d
ui: v0.44.0
ChrisSchinnerl Jan 25, 2024
e9bb012
ci: add dispatch.yml
ChrisSchinnerl Jan 30, 2024
5a51ff0
ci: make dispatch a step in publish.yml
ChrisSchinnerl Jan 30, 2024
1f2407b
ci: include workflow run id in the dispatch
ChrisSchinnerl Jan 30, 2024
402398b
Merge pull request #916 from SiaFoundation/ui/update
ChrisSchinnerl Jan 31, 2024
4f4937c
Merge branch 'master' into dev
ChrisSchinnerl Feb 5, 2024
aa365cd
Merge pull request #914 from SiaFoundation/nate/add-mysql-config
ChrisSchinnerl Feb 5, 2024
54615f7
ci: update client-payload
ChrisSchinnerl Feb 5, 2024
4f8e584
Merge pull request #912 from SiaFoundation/pj/test-refresh-pricetables
ChrisSchinnerl Feb 5, 2024
8e9a748
Merge pull request #913 from SiaFoundation/pj/improve-recorder-shutdown
ChrisSchinnerl Feb 5, 2024
1469b66
worker: fix build
ChrisSchinnerl Feb 5, 2024
c06dea1
autopilot: adjust min score if fewer candidates than needed are avail…
ChrisSchinnerl Feb 5, 2024
ab95e07
autopilot: add TestCalculateMinScore
ChrisSchinnerl Feb 5, 2024
82df50b
Merge pull request #930 from SiaFoundation/chris/update-min-score
ChrisSchinnerl Feb 6, 2024
d83e5a0
Create dependabot.yml
ChrisSchinnerl Feb 6, 2024
1005ab7
build(deps): bump gorm.io/gorm from 1.25.5 to 1.25.7
dependabot[bot] Feb 6, 2024
c509772
build(deps): bump golang.org/x/crypto from 0.16.0 to 0.18.0
dependabot[bot] Feb 6, 2024
1c113b2
build(deps): bump github.com/klauspost/reedsolomon from 1.12.0 to 1.12.1
dependabot[bot] Feb 6, 2024
f205497
Merge pull request #933 from SiaFoundation/dependabot/go_modules/gola…
ChrisSchinnerl Feb 6, 2024
1f35c94
Merge pull request #934 from SiaFoundation/dependabot/go_modules/gith…
ChrisSchinnerl Feb 6, 2024
1205044
build(deps): bump github.com/minio/minio-go/v7 from 7.0.65 to 7.0.66
dependabot[bot] Feb 6, 2024
7534f4b
Merge pull request #936 from SiaFoundation/dependabot/go_modules/gith…
ChrisSchinnerl Feb 6, 2024
4526fb1
Merge pull request #932 from SiaFoundation/dependabot/go_modules/gorm…
ChrisSchinnerl Feb 6, 2024
cd2c6df
Merge pull request #919 from SiaFoundation/chris/repo-dispatch
ChrisSchinnerl Feb 6, 2024
006cc45
all: upgrade go
peterjan Feb 6, 2024
7b4df14
Merge pull request #938 from SiaFoundation/pj/upgrade-go
ChrisSchinnerl Feb 6, 2024
20087f6
api: use TimeRFC3339 for PruneMetric timestamp
peterjan Feb 7, 2024
9922e60
ci: change condition to ignore rc tags
ChrisSchinnerl Feb 7, 2024
5a8d60c
test.yml: run tests for go1.22
ChrisSchinnerl Feb 7, 2024
753df82
Merge pull request #939 from SiaFoundation/pj/prune-metric-timerfc3339
ChrisSchinnerl Feb 7, 2024
c0267d3
ci: add needs
ChrisSchinnerl Feb 7, 2024
8955d02
Merge pull request #940 from SiaFoundation/chris/update-go
ChrisSchinnerl Feb 7, 2024
2723561
worker: get rid of 'used' and 'curr' fields in slabDownload
ChrisSchinnerl Feb 8, 2024
6d0037f
worker: move sector index check from launch to nextRequest
ChrisSchinnerl Feb 8, 2024
36a01e7
worker: remove debug logging
ChrisSchinnerl Feb 8, 2024
c321276
worker: have mgr.fastest return *downloader
ChrisSchinnerl Feb 8, 2024
a9cf515
worker: set error when running out of hosts
ChrisSchinnerl Feb 8, 2024
78f9ddd
worker: attach interaction recorder to worker shutdownCtx
ChrisSchinnerl Feb 8, 2024
7050587
Merge pull request #943 from SiaFoundation/chris/fix-recorder-panic
ChrisSchinnerl Feb 8, 2024
b051523
Merge pull request #941 from SiaFoundation/chris/download-unused
ChrisSchinnerl Feb 9, 2024
d374294
worker: try again if scan fails
ChrisSchinnerl Feb 9, 2024
dc5c0a0
publish.yml: don't trigger respository dispatch for releases with a dash
ChrisSchinnerl Feb 9, 2024
5683d83
Merge pull request #945 from SiaFoundation/chris/release-dash
ChrisSchinnerl Feb 9, 2024
95e86aa
worker: check shutdownCtx before recording
ChrisSchinnerl Feb 9, 2024
0406a8e
stores: TestUpsertSectors
ChrisSchinnerl Feb 10, 2024
0bc2afe
stores: fetch one sector id at a time in upsert
ChrisSchinnerl Feb 10, 2024
16c85e1
Merge pull request #946 from SiaFoundation/chris/upsert-sectors-batch
ChrisSchinnerl Feb 10, 2024
7a0446c
build(deps): bump golang.org/x/crypto from 0.18.0 to 0.19.0
dependabot[bot] Feb 12, 2024
4d5a18a
worker: fix error message
ChrisSchinnerl Feb 12, 2024
3ab9e53
Merge pull request #944 from SiaFoundation/chris/double-scan
ChrisSchinnerl Feb 12, 2024
4142780
Merge pull request #949 from SiaFoundation/dependabot/go_modules/gola…
ChrisSchinnerl Feb 12, 2024
6b53986
build(deps): bump gorm.io/driver/sqlite from 1.5.4 to 1.5.5
dependabot[bot] Feb 12, 2024
296c4e9
Merge pull request #951 from SiaFoundation/dependabot/go_modules/gorm…
ChrisSchinnerl Feb 12, 2024
279f620
stores: add TestUploadObject
ChrisSchinnerl Feb 12, 2024
c9d27dd
stores: extend test to cover contract sectors
ChrisSchinnerl Feb 12, 2024
7d75df5
stores: rename test
ChrisSchinnerl Feb 12, 2024
5399488
stores: gofmt
ChrisSchinnerl Feb 12, 2024
122715a
worker: fix debug message
ChrisSchinnerl Feb 12, 2024
e5f52f1
Merge pull request #953 from SiaFoundation/chris/test-upload-object
ChrisSchinnerl Feb 12, 2024
3b851fd
autopilot: increase resolver timeout to 10s
ChrisSchinnerl Feb 12, 2024
bbb0590
worker: fail upload/download request if uploader/downloader was stopped
ChrisSchinnerl Feb 13, 2024
49475fa
worker: add tests
ChrisSchinnerl Feb 13, 2024
645f93a
worker: increase timeout in tests for CI
ChrisSchinnerl Feb 13, 2024
afafc0d
alerts: add pagination to alerts endpoint and add another endpoint to…
ChrisSchinnerl Feb 13, 2024
87de904
worker: remove context value interaction recorder
n8maninger Feb 13, 2024
fee0b6c
worker: ensure applying MigrationSurchargeMultiplier never prevents a…
peterjan Feb 14, 2024
3ca11cb
worker: get rid of interaction middleware
peterjan Feb 14, 2024
742aa13
Merge pull request #959 from SiaFoundation/nate/remove-context-based-…
ChrisSchinnerl Feb 14, 2024
d90e42f
stores: allow for filtering object stats by bucket
ChrisSchinnerl Feb 14, 2024
b22e96e
bus: address comments
ChrisSchinnerl Feb 14, 2024
dff33cd
bus: fix jape
ChrisSchinnerl Feb 14, 2024
7ac03a4
Merge pull request #960 from SiaFoundation/pj/adjusted-gouging-settings
ChrisSchinnerl Feb 14, 2024
ba41276
Merge pull request #954 from SiaFoundation/chris/uploader-downloader-…
ChrisSchinnerl Feb 14, 2024
ea173eb
stores: binary currency type
ChrisSchinnerl Feb 14, 2024
fe79e07
ui: v0.45.0
ChrisSchinnerl Feb 15, 2024
72d514b
Merge pull request #967 from SiaFoundation/ui/update
ChrisSchinnerl Feb 15, 2024
793dd2f
autopilot: dismiss lost sector alerts when there are no lost sectors
ChrisSchinnerl Feb 15, 2024
73fd775
Merge pull request #961 from SiaFoundation/chris/objects-stats-bucket
ChrisSchinnerl Feb 15, 2024
a52b3a7
stores: fix metric migrations
peterjan Feb 15, 2024
cc83fc9
stores: improve performance of objects stats
ChrisSchinnerl Feb 15, 2024
1592760
stores: use select count with subquery
ChrisSchinnerl Feb 15, 2024
47a91ac
stores: use sectors table and don't filter sectors stats by bucket
ChrisSchinnerl Feb 15, 2024
3e463cf
stores: use sum over total shards
ChrisSchinnerl Feb 15, 2024
3f6dad7
stores: use contract size for total uploaded size
ChrisSchinnerl Feb 15, 2024
2c627d9
testing: fix TestUploadDownloadExtended
ChrisSchinnerl Feb 15, 2024
c3802d8
stores: fix TestSlabBufferStats
ChrisSchinnerl Feb 16, 2024
6ed8e75
stores: use WHERE EXISTS
ChrisSchinnerl Feb 16, 2024
5f8233f
stores: remove subquery
ChrisSchinnerl Feb 16, 2024
f68e6ee
stores: remove contract query
ChrisSchinnerl Feb 16, 2024
e19ff56
Merge pull request #964 from SiaFoundation/chris/binary-currency
ChrisSchinnerl Feb 16, 2024
074cbef
Merge pull request #968 from SiaFoundation/chris/dismiss-lost-sectors…
ChrisSchinnerl Feb 16, 2024
025df16
Merge pull request #958 from SiaFoundation/chris/dismiss-all-alerts
ChrisSchinnerl Feb 16, 2024
fe2097a
bus: merge alert dismiss routes
ChrisSchinnerl Feb 16, 2024
a13313b
bus: add onlymetadata flag to objects endpoint
ChrisSchinnerl Feb 16, 2024
0100379
testing: fix TestS3Basic
ChrisSchinnerl Feb 16, 2024
781290d
worker: fix ndf in TestDownloaderStopped
ChrisSchinnerl Feb 16, 2024
f0ea509
worker: fix ndf in TestDownloaderStopped
ChrisSchinnerl Feb 16, 2024
814b8e5
bus: add generateKey option to create multipart endpoint
ChrisSchinnerl Feb 16, 2024
a2035a8
bus: remove DisablePreshardingEncryption
ChrisSchinnerl Feb 16, 2024
cd72207
Merge pull request #971 from SiaFoundation/chris/merge-dismiss-routes
ChrisSchinnerl Feb 16, 2024
07bbbb1
testing: fix TestMultipartUploads
ChrisSchinnerl Feb 16, 2024
9051e47
stores: get rid of migrate bool
peterjan Feb 16, 2024
e0b3dc0
object: fix panic in objects.String
ChrisSchinnerl Feb 16, 2024
e7f6bb4
testing: fix TestS3Basic
ChrisSchinnerl Feb 16, 2024
93bb567
build(deps): bump github.com/minio/minio-go/v7 from 7.0.66 to 7.0.67
dependabot[bot] Feb 19, 2024
9431952
build(deps): bump gorm.io/driver/mysql from 1.5.2 to 1.5.4
dependabot[bot] Feb 19, 2024
38cdc3c
Merge pull request #978 from SiaFoundation/dependabot/go_modules/gorm…
ChrisSchinnerl Feb 19, 2024
264e4a6
Extend contract set change alerts with host info (#966)
peterjan Feb 19, 2024
6dc7acc
Merge pull request #977 from SiaFoundation/dependabot/go_modules/gith…
ChrisSchinnerl Feb 19, 2024
ac15fc8
go.mod: uprade hostd dependency
ChrisSchinnerl Feb 19, 2024
0ed4313
stores: fix TestPartialSlab
ChrisSchinnerl Feb 19, 2024
d7a51d2
stores: fix TestSQLMetadataStore
ChrisSchinnerl Feb 19, 2024
3990ece
stores: fix TestObjectBasic
ChrisSchinnerl Feb 19, 2024
f075c0c
stores: TestObjectMetadata
ChrisSchinnerl Feb 19, 2024
812bf95
Merge pull request #979 from SiaFoundation/chris/upgrade-hostd
ChrisSchinnerl Feb 19, 2024
2756576
worker: use background context for unlocking accounts
peterjan Feb 19, 2024
2102b91
ci: add Test Stores - MySQL
peterjan Feb 19, 2024
d7e0cf5
testing: fix TestS3MultipartUploads
ChrisSchinnerl Feb 19, 2024
267d5b3
Merge pull request #969 from SiaFoundation/pj/metric-migrations
ChrisSchinnerl Feb 19, 2024
361c6cf
stores: add index to object size
ChrisSchinnerl Feb 19, 2024
df94b1b
Merge pull request #983 from SiaFoundation/chris/objects-size-idx
ChrisSchinnerl Feb 20, 2024
9bb5972
stores: migration code
ChrisSchinnerl Feb 20, 2024
1f00422
stores: call pruneSlabs every time an object or multipart object were…
ChrisSchinnerl Feb 20, 2024
671f8da
stores: add migration to performMigrations
ChrisSchinnerl Feb 20, 2024
2fbcc26
worker: use different context in scanHost
ChrisSchinnerl Feb 20, 2024
0a23dea
stores: fix TestSlabCleanupTrigger
ChrisSchinnerl Feb 20, 2024
50c201a
stores: adapt unit tests to MySQL
peterjan Feb 19, 2024
e8c5c92
ci: enable all tests
peterjan Feb 20, 2024
96d4bba
stores: only drop triggers if they exist
ChrisSchinnerl Feb 20, 2024
4cb5efa
worker: address review comment
ChrisSchinnerl Feb 20, 2024
c0d92cf
Merge pull request #974 from SiaFoundation/chris/multipart-create-key
ChrisSchinnerl Feb 20, 2024
9cdcd60
Merge pull request #970 from SiaFoundation/chris/optimise-stats-bucket
ChrisSchinnerl Feb 20, 2024
adafec0
stores: add missing newline
ChrisSchinnerl Feb 20, 2024
c346017
bus: change response type for paginated alerts request
ChrisSchinnerl Feb 20, 2024
ce3e692
bus: revert dismissall alerts endpoint
ChrisSchinnerl Feb 20, 2024
1198034
worker: don't register alert for cancelled download
ChrisSchinnerl Feb 20, 2024
69bde06
worker: don't register download error alert for errDownloadCancelled …
ChrisSchinnerl Feb 21, 2024
709acfe
bus: fix japecheck
ChrisSchinnerl Feb 21, 2024
aacaaa7
stores: remove frand.Shuffle
peterjan Feb 22, 2024
414090b
stores: change bCurrency type to BigEndian (#991)
ChrisSchinnerl Feb 22, 2024
7fb1349
Merge branch 'dev' of github.com:SiaFoundation/renterd into pj/mysql-…
peterjan Feb 22, 2024
9edb677
Merge pull request #982 from SiaFoundation/pj/mysql-stores-tests
ChrisSchinnerl Feb 22, 2024
c29adb0
Merge pull request #986 from SiaFoundation/chris/remove-interaction-r…
ChrisSchinnerl Feb 22, 2024
b5d2c4f
Merge pull request #972 from SiaFoundation/chris/only-metadata
ChrisSchinnerl Feb 22, 2024
302aaa4
Merge pull request #988 from SiaFoundation/chris/download-cancel-alert
ChrisSchinnerl Feb 22, 2024
5b968e7
stores: numDeleted
ChrisSchinnerl Feb 22, 2024
fb5bc35
worker: add benchmarks for upload through uploader
ChrisSchinnerl Feb 21, 2024
e1d45a4
worker: add benchmark results
ChrisSchinnerl Feb 21, 2024
fd166d8
worker: docstring update
ChrisSchinnerl Feb 21, 2024
9f27c17
worker: remove hash reader
ChrisSchinnerl Feb 21, 2024
24fb81a
worker: update benchmark result
ChrisSchinnerl Feb 21, 2024
11d0152
worker: encrypt using multiple goroutines
ChrisSchinnerl Feb 21, 2024
c918925
worker: update benchmark results
ChrisSchinnerl Feb 21, 2024
5ceccfc
worker: ComputeEtag method
ChrisSchinnerl Feb 22, 2024
cfb8cb6
worker: pass expected root to UploadSector
ChrisSchinnerl Feb 22, 2024
67a6c87
worker: update benchmark results
ChrisSchinnerl Feb 22, 2024
3f11ef6
worker: implement PR remarks
peterjan Feb 22, 2024
bcdeb0d
worker: return err
peterjan Feb 22, 2024
542709a
Merge pull request #981 from SiaFoundation/pj/use-background-context-…
ChrisSchinnerl Feb 22, 2024
9af03a3
worker: add BenchmarkDownloaderSingleObjecdt
ChrisSchinnerl Feb 22, 2024
c9dc1b6
worker: decrypt in parallel
ChrisSchinnerl Feb 22, 2024
aa08202
worker: update download benchmark results
ChrisSchinnerl Feb 22, 2024
29f4201
worker: remove irrelevant benchmarks
ChrisSchinnerl Feb 22, 2024
01805e6
Merge branch 'dev' into chris/prune-slabs-cascade
ChrisSchinnerl Feb 22, 2024
10e88fd
stores: fix mysql migration
ChrisSchinnerl Feb 22, 2024
3e92f08
ui: v0.46.0
ChrisSchinnerl Feb 23, 2024
1192264
Merge pull request #987 from SiaFoundation/chris/alerts-pagination
ChrisSchinnerl Feb 23, 2024
99014c7
stores: drop indices first
ChrisSchinnerl Feb 23, 2024
72557ad
Merge pull request #985 from SiaFoundation/chris/prune-slabs-cascade
ChrisSchinnerl Feb 23, 2024
42338c7
Merge pull request #994 from SiaFoundation/ui/update
ChrisSchinnerl Feb 23, 2024
89b4729
testing: fix TestUploadPacking
ChrisSchinnerl Feb 23, 2024
b45b80f
autopilot: accumulate churn information into single alert
ChrisSchinnerl Feb 21, 2024
c17252a
autopilot: update churn alert to contain timestamp
ChrisSchinnerl Feb 23, 2024
e9a593d
bus: only log lost sectors when there was a contract with that host
ChrisSchinnerl Feb 23, 2024
3f09191
Merge pull request #989 from SiaFoundation/chris/churn-alert-accumulator
ChrisSchinnerl Feb 23, 2024
9e02505
build(deps): bump go.sia.tech/coreutils from 0.0.1 to 0.0.3
dependabot[bot] Feb 26, 2024
b1efe2c
build(deps): bump go.uber.org/zap from 1.26.0 to 1.27.0
dependabot[bot] Feb 26, 2024
6866e11
Merge pull request #999 from SiaFoundation/dependabot/go_modules/go.u…
ChrisSchinnerl Feb 27, 2024
97b017e
Merge pull request #998 from SiaFoundation/dependabot/go_modules/go.s…
ChrisSchinnerl Feb 27, 2024
0ef1757
bus: filter by alert severity
ChrisSchinnerl Feb 27, 2024
75451b0
Merge pull request #990 from SiaFoundation/chris/upload-benchmark
ChrisSchinnerl Feb 27, 2024
b7e14e8
stores: check for exact value in TestDeleteHostSector
ChrisSchinnerl Feb 27, 2024
fc72fa9
Merge pull request #996 from SiaFoundation/chris/mark-sector-lost-log…
ChrisSchinnerl Feb 27, 2024
7c5b3c9
bus: breakdown totals
ChrisSchinnerl Feb 27, 2024
6db1e38
Update alerts/alerts.go
ChrisSchinnerl Feb 27, 2024
fc3625a
Merge pull request #1002 from SiaFoundation/chris/alerts-filter-severity
ChrisSchinnerl Feb 27, 2024
39933c4
Merge branch 'dev' of github.com:SiaFoundation/renterd into its-happe…
peterjan Feb 27, 2024
1beaf32
stores: fix TestInsertAnnouncements
peterjan Feb 27, 2024
3279ced
testing: fix TestSQLContractStore, TestRenewedContract, TestRecordCon…
peterjan Feb 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,16 @@ jobs:
uses: n8maninger/action-golang-test@v1
with:
args: "-race;-short"
- name: Test Stores - MySQL
if: matrix.os == 'ubuntu-latest'
uses: n8maninger/action-golang-test@v1
env:
RENTERD_DB_URI: 127.0.0.1:3800
RENTERD_DB_USER: root
RENTERD_DB_PASSWORD: test
with:
package: "./stores"
args: "-race;-short"
- name: Test Integration
uses: n8maninger/action-golang-test@v1
with:
Expand Down
91 changes: 58 additions & 33 deletions alerts/alerts.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,9 @@ const (

type (
Alerter interface {
Alerts(_ context.Context, opts AlertsOpts) (resp AlertsResponse, err error)
RegisterAlert(_ context.Context, a Alert) error
DismissAlerts(_ context.Context, ids ...types.Hash256) error
DismissAllAlerts(_ context.Context) error
}

// Severity indicates the severity of an alert.
Expand Down Expand Up @@ -66,11 +66,27 @@ type (
}

AlertsOpts struct {
Offset int
Limit int
Offset int
Limit int
Severity Severity
}

AlertsResponse struct {
Alerts []Alert `json:"alerts"`
HasMore bool `json:"hasMore"`
Totals struct {
Info int `json:"info"`
Warning int `json:"warning"`
Error int `json:"error"`
Critical int `json:"critical"`
} `json:"totals"`
}
)

func (ar AlertsResponse) Total() int {
return ar.Totals.Info + ar.Totals.Warning + ar.Totals.Error + ar.Totals.Critical
}

// String implements the fmt.Stringer interface.
func (s Severity) String() string {
switch s {
Expand All @@ -87,15 +103,8 @@ func (s Severity) String() string {
}
}

// MarshalJSON implements the json.Marshaler interface.
func (s Severity) MarshalJSON() ([]byte, error) {
return []byte(fmt.Sprintf(`%q`, s.String())), nil
}

// UnmarshalJSON implements the json.Unmarshaler interface.
func (s *Severity) UnmarshalJSON(b []byte) error {
status := strings.Trim(string(b), `"`)
switch status {
func (s *Severity) LoadString(str string) error {
switch str {
case severityInfoStr:
*s = SeverityInfo
case severityWarningStr:
Expand All @@ -105,11 +114,21 @@ func (s *Severity) UnmarshalJSON(b []byte) error {
case severityCriticalStr:
*s = SeverityCritical
default:
return fmt.Errorf("unrecognized severity: %v", status)
return fmt.Errorf("unrecognized severity: %v", str)
}
return nil
}

// MarshalJSON implements the json.Marshaler interface.
func (s Severity) MarshalJSON() ([]byte, error) {
return []byte(fmt.Sprintf(`%q`, s.String())), nil
}

// UnmarshalJSON implements the json.Unmarshaler interface.
func (s *Severity) UnmarshalJSON(b []byte) error {
return s.LoadString(strings.Trim(string(b), `"`))
}

// RegisterAlert implements the Alerter interface.
func (m *Manager) RegisterAlert(ctx context.Context, alert Alert) error {
if alert.ID == (types.Hash256{}) {
Expand All @@ -136,17 +155,6 @@ func (m *Manager) RegisterAlert(ctx context.Context, alert Alert) error {
})
}

// DismissAllAlerts implements the Alerter interface.
func (m *Manager) DismissAllAlerts(ctx context.Context) error {
m.mu.Lock()
toDismiss := make([]types.Hash256, 0, len(m.alerts))
for alertID := range m.alerts {
toDismiss = append(toDismiss, alertID)
}
m.mu.Unlock()
return m.DismissAlerts(ctx, toDismiss...)
}

// DismissAlerts implements the Alerter interface.
func (m *Manager) DismissAlerts(ctx context.Context, ids ...types.Hash256) error {
var dismissed []types.Hash256
Expand Down Expand Up @@ -175,19 +183,34 @@ func (m *Manager) DismissAlerts(ctx context.Context, ids ...types.Hash256) error
})
}

// Active returns the host's active alerts.
func (m *Manager) Active(offset, limit int) []Alert {
// Alerts returns the host's active alerts.
func (m *Manager) Alerts(_ context.Context, opts AlertsOpts) (AlertsResponse, error) {
m.mu.Lock()
defer m.mu.Unlock()

offset, limit := opts.Offset, opts.Limit
resp := AlertsResponse{}

if offset >= len(m.alerts) {
return nil
return resp, nil
} else if limit == -1 {
limit = len(m.alerts)
}

alerts := make([]Alert, 0, len(m.alerts))
for _, a := range m.alerts {
if a.Severity == SeverityInfo {
resp.Totals.Info++
} else if a.Severity == SeverityWarning {
resp.Totals.Warning++
} else if a.Severity == SeverityError {
resp.Totals.Error++
} else if a.Severity == SeverityCritical {
resp.Totals.Critical++
}
if opts.Severity != 0 && a.Severity != opts.Severity {
continue // filter by severity
}
alerts = append(alerts, a)
}
sort.Slice(alerts, func(i, j int) bool {
Expand All @@ -196,8 +219,10 @@ func (m *Manager) Active(offset, limit int) []Alert {
alerts = alerts[offset:]
if limit < len(alerts) {
alerts = alerts[:limit]
resp.HasMore = true
}
return alerts
resp.Alerts = alerts
return resp, nil
}

func (m *Manager) RegisterWebhookBroadcaster(b webhooks.Broadcaster) {
Expand Down Expand Up @@ -231,6 +256,11 @@ func WithOrigin(alerter Alerter, origin string) Alerter {
}
}

// Alerts implements the Alerter interface.
func (a *originAlerter) Alerts(ctx context.Context, opts AlertsOpts) (resp AlertsResponse, err error) {
return a.alerter.Alerts(ctx, opts)
}

// RegisterAlert implements the Alerter interface.
func (a *originAlerter) RegisterAlert(ctx context.Context, alert Alert) error {
if alert.Data == nil {
Expand All @@ -240,11 +270,6 @@ func (a *originAlerter) RegisterAlert(ctx context.Context, alert Alert) error {
return a.alerter.RegisterAlert(ctx, alert)
}

// DismissAllAlerts implements the Alerter interface.
func (a *originAlerter) DismissAllAlerts(ctx context.Context) error {
return a.alerter.DismissAllAlerts(ctx)
}

// DismissAlerts implements the Alerter interface.
func (a *originAlerter) DismissAlerts(ctx context.Context, ids ...types.Hash256) error {
return a.alerter.DismissAlerts(ctx, ids...)
Expand Down
20 changes: 12 additions & 8 deletions api/object.go
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ type (
Object struct {
Metadata ObjectUserMetadata `json:"metadata,omitempty"`
ObjectMetadata
object.Object
*object.Object `json:"omitempty"`
}

// ObjectMetadata contains various metadata about an object.
Expand Down Expand Up @@ -212,13 +212,14 @@ type (
}

GetObjectOptions struct {
Prefix string
Offset int
Limit int
IgnoreDelim bool
Marker string
SortBy string
SortDir string
Prefix string
Offset int
Limit int
IgnoreDelim bool
Marker string
OnlyMetadata bool
SortBy string
SortDir string
}

ListObjectOptions struct {
Expand Down Expand Up @@ -316,6 +317,9 @@ func (opts GetObjectOptions) Apply(values url.Values) {
if opts.Marker != "" {
values.Set("marker", opts.Marker)
}
if opts.OnlyMetadata {
values.Set("onlymetadata", "true")
}
if opts.SortBy != "" {
values.Set("sortBy", opts.SortBy)
}
Expand Down
58 changes: 21 additions & 37 deletions autopilot/alerts.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,13 @@ import (
)

var (
alertAccountRefillID = frand.Entropy256() // constant until restarted
alertLostSectorsID = frand.Entropy256() // constant until restarted
alertLowBalanceID = frand.Entropy256() // constant until restarted
alertMigrationID = frand.Entropy256() // constant until restarted
alertPruningID = frand.Entropy256() // constant until restarted
alertRenewalFailedID = frand.Entropy256() // constant until restarted
alertAccountRefillID = randomAlertID() // constant until restarted
alertChurnID = randomAlertID() // constant until restarted
alertLostSectorsID = randomAlertID() // constant until restarted
alertLowBalanceID = randomAlertID() // constant until restarted
alertMigrationID = randomAlertID() // constant until restarted
alertPruningID = randomAlertID() // constant until restarted
alertRenewalFailedID = randomAlertID() // constant until restarted
)

func alertIDForAccount(alertID [32]byte, id rhpv3.Account) types.Hash256 {
Expand Down Expand Up @@ -54,6 +55,20 @@ func (ap *Autopilot) DismissAlert(ctx context.Context, ids ...types.Hash256) {
}
}

func (ap *Autopilot) HasAlert(ctx context.Context, id types.Hash256) bool {
ar, err := ap.alerts.Alerts(ctx, alerts.AlertsOpts{Offset: 0, Limit: -1})
if err != nil {
ap.logger.Errorf("failed to fetch alerts: %v", err)
return false
}
for _, alert := range ar.Alerts {
if alert.ID == id {
return true
}
}
return false
}

func newAccountLowBalanceAlert(address types.Address, balance, allowance types.Currency, bh, renewWindow, endHeight uint64) alerts.Alert {
severity := alerts.SeverityInfo
if bh+renewWindow/2 >= endHeight {
Expand Down Expand Up @@ -137,37 +152,6 @@ func newContractPruningFailedAlert(hk types.PublicKey, version string, fcid type
}
}

func newContractSetChangeAlert(name string, additions map[types.FileContractID]contractSetAddition, removals map[types.FileContractID]contractSetRemoval) alerts.Alert {
var hint string
if len(removals) > 0 {
hint = "A high churn rate can lead to a lot of unnecessary migrations, it might be necessary to tweak your configuration depending on the reason hosts are being discarded from the set."
}

removedReasons := make(map[string]string, len(removals))
for k, v := range removals {
removedReasons[k.String()] = v.Reason
}

return alerts.Alert{
ID: randomAlertID(),
Severity: alerts.SeverityInfo,
Message: "Contract set changed",
Data: map[string]any{
"name": name,
"set_additions": additions,
"set_removals": removals,
"hint": hint,

// TODO: these fields can be removed on the next major release, they
// contain redundant information
"added": len(additions),
"removed": len(removals),
"removals": removedReasons,
},
Timestamp: time.Now(),
}
}

func newLostSectorsAlert(hk types.PublicKey, lostSectors uint64) alerts.Alert {
return alerts.Alert{
ID: alertIDForHost(alertLostSectorsID, hk),
Expand Down
68 changes: 68 additions & 0 deletions autopilot/churn.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
package autopilot

import (
"time"

"go.sia.tech/core/types"
"go.sia.tech/renterd/alerts"
)

type (
accumulatedChurn struct {
additions map[types.FileContractID]contractSetAdditions
removals map[types.FileContractID]contractSetRemovals
}
)

func newAccumulatedChurn() *accumulatedChurn {
return &accumulatedChurn{
additions: make(map[types.FileContractID]contractSetAdditions),
removals: make(map[types.FileContractID]contractSetRemovals),
}
}

func (c *accumulatedChurn) Alert(name string) alerts.Alert {
var hint string
if len(c.removals) > 0 {
hint = "A high churn rate can lead to a lot of unnecessary migrations, it might be necessary to tweak your configuration depending on the reason hosts are being discarded from the set."
}

return alerts.Alert{
ID: alertChurnID,
Severity: alerts.SeverityInfo,
Message: "Contract set changed",
Data: map[string]any{
"name": name,
"set_additions": c.additions,
"set_removals": c.removals,
"hint": hint,
},
Timestamp: time.Now(),
}
}

func (c *accumulatedChurn) Apply(additions map[types.FileContractID]contractSetAdditions, removals map[types.FileContractID]contractSetRemovals) {
for fcid, a := range additions {
if _, exists := c.additions[fcid]; !exists {
c.additions[fcid] = a
} else {
additions := c.additions[fcid]
additions.Additions = append(additions.Additions, a.Additions...)
c.additions[fcid] = additions
}
}
for fcid, r := range removals {
if _, exists := c.removals[fcid]; !exists {
c.removals[fcid] = r
} else {
removals := c.removals[fcid]
removals.Removals = append(removals.Removals, r.Removals...)
c.removals[fcid] = removals
}
}
}

func (c *accumulatedChurn) Reset() {
c.additions = make(map[types.FileContractID]contractSetAdditions)
c.removals = make(map[types.FileContractID]contractSetRemovals)
}
Loading
Loading