Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Normalize counter values #199

Merged
merged 2 commits into from
Aug 14, 2023
Merged

Normalize counter values #199

merged 2 commits into from
Aug 14, 2023

Conversation

didierofrivia
Copy link
Member

@didierofrivia didierofrivia commented Aug 9, 2023

This PR is part of the many changes in the data model, tracked by #182

There's no relevant change in performance, I've included a snapshot for reference:
Screenshot 2023-08-09 at 18 20 06

@codecov-commenter
Copy link

Codecov Report

Merging #199 (938a46e) into main (317037d) will increase coverage by 0.18%.
The diff coverage is 100.00%.

❗ Current head 938a46e differs from pull request most recent head 9711f63. Consider uploading reports for the commit 9711f63 to get more accurate results

@@            Coverage Diff             @@
##             main     #199      +/-   ##
==========================================
+ Coverage   73.17%   73.36%   +0.18%     
==========================================
  Files          30       30              
  Lines        5048     5065      +17     
==========================================
+ Hits         3694     3716      +22     
+ Misses       1354     1349       -5     
Files Changed Coverage Δ
limitador/src/storage/disk/rocksdb_storage.rs 87.73% <100.00%> (+0.23%) ⬆️
limitador/src/storage/in_memory.rs 80.40% <100.00%> (+0.90%) ⬆️
limitador/src/storage/mod.rs 91.20% <100.00%> (-0.15%) ⬇️
limitador/src/storage/redis/redis_sync.rs 95.86% <100.00%> (+0.08%) ⬆️
limitador/src/storage/wasm.rs 83.41% <100.00%> (+0.23%) ⬆️

... and 2 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@didierofrivia didierofrivia marked this pull request as ready for review August 10, 2023 12:46
@@ -96,7 +96,7 @@ impl CounterStorage for InMemoryStorage {
delta: i64,
load_counters: bool,
) -> Result<Authorization, StorageErr> {
let mut limits_by_namespace = self.limits_for_namespace.write().unwrap();
let limits_by_namespace = self.limits_for_namespace.read().unwrap();
Copy link
Member

@alexsnaps alexsnaps Aug 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change introduces a race condition when we eventually update the counters here. By the time we get to our call .update on our AtomicExpiringValue references, their value could have already been updated by another thread. This is why we made .update(...) -> u64, i.e. return the value at that updating moment. But we now need to recheck we didn't pass the threshold... worst, if we did on say the second counter in our vec, what do we do with the increment to the first one?

So, tl;dr, I'm thinking lets leave grabbing the write lock for now and address the concurrency problem in a subsequent PR, wdyt?

* Only used in InMemory storage for normalizing counters
* Other storage implementations return `Ok(())`
* Included in `storage.add_limit`, unwrapping for now since it can't
  fail
* Not needed anymore since counters are normalized when limits are
  created
@didierofrivia didierofrivia force-pushed the normalize-counter-values branch from 9711f63 to c96f478 Compare August 11, 2023 12:02
@didierofrivia didierofrivia merged commit 490826c into main Aug 14, 2023
@didierofrivia didierofrivia deleted the normalize-counter-values branch August 14, 2023 08:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants