dannykopping/remove cache stats #14
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- milli -> nano conversion in the querier - add deletes to store requests from the ingester
* Add some comments Signed-off-by: Michel Hollands <[email protected]> * Use deletes in tail Signed-off-by: Michel Hollands <[email protected]> * Change error message Signed-off-by: Michel Hollands <[email protected]>
* fix(gateway): make DNS resolver configurable Signed-off-by: Ivan Rizzante <[email protected]> * fix: lint
This fixes a bug in the index gateway when querying values for a label.
Since the gRPC handler for LabelValuesForMetricName in the index gateway
allows empty matchers in the LabelValuesForMetricNameRequest, we need to
check if the matchers string is an empty matcher (`{}`) before we parse
the string.
This bug was introduced with the index gateway api refactoring in #5892
Fixes: #5965
Signed-off-by: Christian Haudum <[email protected]>
when index gateway does not implement the new API. This bug did not yield an error but only returned incorrect results because both functions `c.IndexStore.LabelNamesForMetricName` and `c.IndexStore.LabelValuesForMetricName` return `([]string, error)`. Signed-off-by: Christian Haudum <[email protected]>
Signed-off-by: irizzant <[email protected]>
* Restrict gRPC calls to ingesters when requesting series if there is no overlap within the query_ingesters_within value * Add tests for isWithinIngesterMaxLookbackPeriod * Adding tests for calculateIngesterMaxLookbackPeriod * Only send query for range overlapping with ingesters * Restrict gRPC calls to ingesters when requesting series if there is no overlap within the query_ingesters_within value * Add tests for isWithinIngesterMaxLookbackPeriod * Adding tests for calculateIngesterMaxLookbackPeriod * Only send query for range overlapping with ingesters * Add test for when to query ingester and storage * Fix deadlock in Series when just queriying the ingesters * Refactored slightly for better readability * Added CHANGELOG entry * Fix typo * Rephrase test cases names * Move block to outer loop Co-authored-by: Salva Corts <[email protected]>
* Generate cache invalidation numbers in the delete store * Get cache generation numbers from the store on request * changlog * rename tombstones to something more meaningful * User invisible module * query frontend relies on a compactor to get the cache generation number * fix serialization * source -> name * lint errors * lint errors * log non-200 responses * add jsonnet changes * lint * review feedback * review feedback * client rename
This is part 1 of 2 PRs and updates the build image to version 0.10.4 and adding the buf utility which will later be used for formatting *.proto files
* Disable calling new index-gateway client's API. * Remove infinite recursive call * Correct mock setup for test
* Remove empty match within concat of regex * add back tests
* Revendor grafana/tail. - This revendoring brings the fix to relative symlinks, as described in #3374. * Prune package from `go mod tidy`. * noop
…ranteed query sort order when receiving entries for the same stream that have duplicate timestamps. (#6042) * Add a per tenant ability to fudge duplicate timestamps in the distributor Signed-off-by: Edward Welch <[email protected]> * update docs Signed-off-by: Edward Welch <[email protected]> * lint Signed-off-by: Edward Welch <[email protected]> * update changelog Signed-off-by: Edward Welch <[email protected]>
* Add two metrics to the IndexGateway. - Add a new `query_readiness_duration_seconds` metric, that reports query readiness duration of a tablemanager/index gateway instance. We should use it later to report performance against the ring mode - Add a new `usersToBeQueryReadyForTotal` metric, that reports number of users involved in the query readiness operation. We should use it later to correlate number of users with the query readiness duration. * Remove `usersToBeQueryReadyForTotal`. - It will report all users always for now, so it isn't too helpful the way it is. * Rename metric help text to not mislead people. * Log queryReadiness duration. * Fix where log message and duration and triggered. * Join users list in a single string. - This is necessary since go-kit doesn't support array type. * Tweak queryReadiness log messages. - As suggested by Ed on #5972 (comment) and #5972 (comment) * Ensure queryReadinessDuration metric. - It is redundant with a recently added log line. * noop
* Revise the getting started section
* Docs: revise getting started guide
* Intermediate commit of getting started material.
* Intermediate revision of the getting started guide.
Add instructions to download, configure, and deploy the test environment.
* Add sample queries and instructions for breaking down the test env
* Revise prose using reviewer comments.
* update to use promtail and flog in the docker compose file
Signed-off-by: Edward Welch <[email protected]>
* Revise guide to put all parts into Docker containers.
* Add README files for examples directory. Revise getting started instructions.
* Address DylanGuedes review comments
* Getting started guide: incorporate suggestions from ssncferreira
Co-authored-by: Edward Welch <[email protected]>
This component has been around for a while but has not been added to the loki-mixin yet. Signed-off-by: Christian Haudum <[email protected]>
Signed-off-by: Arve Knudsen <[email protected]>
…tor (#5879) * Add filter parameter to rebound Signed-off-by: Michel Hollands <[email protected]> * Fix linting issues Signed-off-by: Michel Hollands <[email protected]> * Add filter function to delete request Signed-off-by: Michel Hollands <[email protected]> * Fix linting issues Signed-off-by: Michel Hollands <[email protected]> * Enable api for filter and delete mode Signed-off-by: Michel Hollands <[email protected]> * Add settings for retention Signed-off-by: Michel Hollands <[email protected]> * Use labels to check and add test Signed-off-by: Michel Hollands <[email protected]> * Simplify filter function Signed-off-by: Michel Hollands <[email protected]> * Also enable filter mode Signed-off-by: Michel Hollands <[email protected]> * Remove test settings in config file for docker Signed-off-by: Michel Hollands <[email protected]> * Add extra (unused) param for ProcessString in filter Signed-off-by: Michel Hollands <[email protected]> * Empty commit to trigger CI again Signed-off-by: Michel Hollands <[email protected]> * Update changelog Signed-off-by: Michel Hollands <[email protected]> * Fix flapping test Signed-off-by: Michel Hollands <[email protected]> * Remove commented out unit tests and add some more Signed-off-by: Michel Hollands <[email protected]> * Add extra test case for delete request without line filter Signed-off-by: Michel Hollands <[email protected]> * Use chunk bounds Signed-off-by: Michel Hollands <[email protected]> * check if the log selector has a filter if the whole chunk is selected Signed-off-by: Michel Hollands <[email protected]> * fix lint issue: use correct go-kit import Signed-off-by: Michel Hollands <[email protected]>
…by the AST (#6346) * Panic with an actual error if clone (which calls range aggregation expr's validate) errors, instead of swallowing the error and then attempting to execute a nil function, which panics anyways. Signed-off-by: Callum Styan <[email protected]> * Call the validation function on a modified expression before attempting to execute it. If the validation fails we should return the original expression. Signed-off-by: Callum Styan <[email protected]>
Signed-off-by: Christopher Norris <[email protected]>
So that you can do quantile calculation on metadata queries. Signed-off-by: Kaviraj <[email protected]>
This updates to a version that contains the fix to CVE-2022-28948.
#6310) * Add support for client-side TLS certs to loki-canary * fix lint errors, make the case of using TLS but not client certs more clear, check that -tls is set if client certs are used * move changelog to canary section instead of main * wrap httpClient errors and simplify dialer creation for websocket
* index sampling endpoint * fix bug in withTenantLabelMatcher * show zero values in stats * handles new errcase * fixes comments
Signed-off-by: Trevor Wood <[email protected]>
* cmd/loki: minor consistency patch * cmd/loki: fixed a typo
* begins building queryrange compatible types for index sampling * adds MatcherGroups() method to SampleExpr * MatcherGroups test + ShardResolver interface * index stats codec wiring * millisecond precision on index stats codec * lint * cleanMatchers in tsdb.IndexClient for consistency + adds catchall matcher if nil
* Add metrics for journal target There are no metrics for the journal target yet, this adds two basic ones: - Total number of lines processed - Total number of errors processing lines (with a reason) Because of the way the journal works, tailing one file with many possible process writing to it, I chose note to expose counts per unit. * Add changelog entry * Fix compilation of (unsupported) journalmanager target on windows * Extract possible errors to constants & add tests
* Backport memberlist dependency version pin to k102 Signed-off-by: JordanRushing <[email protected]> * Revert "Loki Prometheus mixin: templatize cluster label" (#6377) * Revert "Loki Prometheus mixin: templatize cluster label (#6353)" This reverts commit 459036f. * Replace `loki-mixin` dependency to working commit. Signed-off-by: Kaviraj <[email protected]> (cherry picked from commit 666bac2) Co-authored-by: Kaviraj Kanagaraj <[email protected]> (cherry picked from commit 372961d) Co-authored-by: JordanRushing <[email protected]>
) (#6493) (cherry picked from commit efda5b2) Co-authored-by: Travis Patterson <[email protected]> (cherry picked from commit f79ff95)
Signed-off-by: Michel Hollands <[email protected]> (cherry picked from commit 671afe6) Co-authored-by: Michel Hollands <[email protected]> (cherry picked from commit a7f8be2)
* Add more logging for delete requests Signed-off-by: Michel Hollands <[email protected]> * Fix typo Signed-off-by: Michel Hollands <[email protected]> * Rephrase logging message and add chunk ID Signed-off-by: Michel Hollands <[email protected]> * Rename userID field to user Signed-off-by: Michel Hollands <[email protected]> * rename more userID Signed-off-by: Michel Hollands <[email protected]> * Fix existing log messages Signed-off-by: Michel Hollands <[email protected]> (cherry picked from commit 20db4df) Co-authored-by: Michel Hollands <[email protected]> (cherry picked from commit c707faf)
…) (#6496) * Compactor: add per tenant compaction delete enabled flag (#6410) * Add per tenant compaction delete enabled flag Signed-off-by: Michel Hollands <[email protected]> * Remove changes in wrong place Signed-off-by: Michel Hollands <[email protected]> * Add compactor deletion enabled field Signed-off-by: Michel Hollands <[email protected]> * Use limit in compactor Signed-off-by: Michel Hollands <[email protected]> * Use http middleware and add test Signed-off-by: Michel Hollands <[email protected]> * Fix lint issue Signed-off-by: Michel Hollands <[email protected]> * Add changelog Signed-off-by: Michel Hollands <[email protected]> * Revert to default setting if no override Signed-off-by: Michel Hollands <[email protected]> * Add default value command line option Signed-off-by: Michel Hollands <[email protected]> * Update the docs Signed-off-by: Michel Hollands <[email protected]> * Enable access to deletion API for integration test Signed-off-by: Michel Hollands <[email protected]> * Rename flag to allow_deletes Signed-off-by: Michel Hollands <[email protected]> * Update per review comments Signed-off-by: Michel Hollands <[email protected]> (cherry picked from commit b4e6c59) * Fix changelog Signed-off-by: Michel Hollands <[email protected]> (cherry picked from commit df48f93) Co-authored-by: Michel Hollands <[email protected]>
) * Remove whole stream deletion mode (#6435) * Remove whole-stream-deletion mode Signed-off-by: Michel Hollands <[email protected]> * Remove whole-stream-deletion from docs Signed-off-by: Michel Hollands <[email protected]> * Update the changelog Signed-off-by: Michel Hollands <[email protected]> * Sort changelog entries Signed-off-by: Michel Hollands <[email protected]> * Remove link to wrong configuration Signed-off-by: Michel Hollands <[email protected]> * Fix integration test Signed-off-by: Michel Hollands <[email protected]> * Set default deletion mode to disabled Signed-off-by: Michel Hollands <[email protected]> * Remove extra white line in documentation Signed-off-by: Michel Hollands <[email protected]> * Fix default value in docs Signed-off-by: Michel Hollands <[email protected]> * Fix changelog Signed-off-by: Michel Hollands <[email protected]> * Add DeletionEnabled method on mode Signed-off-by: Michel Hollands <[email protected]> * Rename test Signed-off-by: Michel Hollands <[email protected]> (cherry picked from commit f80e487) * Fix changelog Signed-off-by: Michel Hollands <[email protected]> * Fix changelog link Signed-off-by: Michel Hollands <[email protected]> (cherry picked from commit da3fbd9) Co-authored-by: Michel Hollands <[email protected]>
This reverts commit 36e0979.
* disable tests which are failing from hardcoded certs which are now expired. Signed-off-by: Edward Welch <[email protected]> * lint Signed-off-by: Edward Welch <[email protected]>
Signed-off-by: Danny Kopping <[email protected]>
dannykopping
force-pushed
the
dannykopping/remove-cache-stats
branch
from
2a5bb86
to
7c90e53
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
grafana/tailto fix promtail symlink behavior (Revendorgrafana/tailto fix promtail symlink behavior grafana/loki#6034)Loki - Reads Resourcesdashboard in production/loki-mixin (Add IndexGateway Query Readiness Duration panel toLoki - Reads Resourcesdashboard in production/loki-mixin grafana/loki#6014)static_config(promtail: Add support for exclusion patterns instatic_configgrafana/loki#5943)/label/<name>/valuesendpoint to use right set of middlewares (Fix(query-frontend):/label/<name>/valuesendpoint to use right set of middlewares grafana/loki#6072)sources/fundamentals/architecture/deployment-modes.md(Docs: fix pic location in `sources/fundamentals/architecture/deployme… grafana/loki#6089)metrics.gosupport for metadata queries(labels and series) (metrics.gosupport for metadata queries(labels and series) grafana/loki#5971)sources/fundamentals/architecture/deployment-modes.md(Docs: fix pic location in `sources/fundamentals/architecture/deployme… grafana/loki#6089)" (Revert "Docs: fix pic location in `sources/fundamentals/architecture/deployme…" grafana/loki#6093)/servicesAPI endpoint (Document/servicesAPI endpoint grafana/loki#6111)defaultsprig template function to logql label/line formatter (LogQL: add thedefaultsprig template function to logql label/line formatter grafana/loki#6163)querier_forget_delay(query-scheduler: Now proper support forquerier_forget_delaygrafana/loki#6174)resultlength after response validation. (fix(querier): Recordresultlength after response validation. grafana/loki#6176)faketenant (Logcli: fix boltdb shipper local query in logcli and supportfaketenant grafana/loki#6282)multi_kv_configblock to overrides ConfigMap when multi-kv mode is enabled grafana/loki#6323)MetricsNamespaceconfigurable for MemberlistKV module in Loki (SetMetricsNamespaceconfigurable for MemberlistKV module in Loki grafana/loki#6330)series|labelsquery_type inlogql_query_duration(Supportseries|labelsquery_type inlogql_query_durationgrafana/loki#6341)