Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add historic metrics #74

Merged
merged 15 commits into from
Sep 24, 2024
Merged

Add historic metrics #74

merged 15 commits into from
Sep 24, 2024

Conversation

dmytrotkk
Copy link
Contributor

@dmytrotkk dmytrotkk commented Sep 12, 2024

In this PR:

  • Added historic metrics collection
  • Added MySQL DB to store historic metrics
  • Added new fields in metrics.json: last day tx, last 7 days tx, last 30 days tx
  • Added types for metrics data
  • Added linter and tests pipeline for metrics

@dmytrotkk dmytrotkk marked this pull request as ready for review September 17, 2024 12:22
.github/workflows/test_metrics.yml Outdated Show resolved Hide resolved
.github/workflows/test_metrics.yml Outdated Show resolved Hide resolved
.github/workflows/test_metrics.yml Show resolved Hide resolved
metrics/src/collector.py Outdated Show resolved Hide resolved
metrics/src/config.py Outdated Show resolved Hide resolved
metrics/src/db.py Outdated Show resolved Hide resolved
metrics/src/main.py Outdated Show resolved Hide resolved
metrics/src/main.py Outdated Show resolved Hide resolved
scripts/backup_db.sh Show resolved Hide resolved
scripts/restore_db.sh Show resolved Hide resolved
BACKFILL_DB_DAYS = 30


async def get_current_total_transactions(session, chain_name: str, address: str) -> int:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is better to move this method into src.explorer

Copy link
Contributor Author

@dmytrotkk dmytrotkk Sep 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved


logger = logging.getLogger(__name__)


def init_w3():
logger.info(f'Connecting to {ENDPOINT}...')
logger.info(f"Connecting to {ENDPOINT}...")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why double-quotes, but not single-quotes in this module?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

def run_migrations():
logger.info('Running database migrations...')
try:
from models import Address, TransactionCount
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move models import to the other imports

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved

logger.info('Metrics collector iteration started...')
current_date = datetime.now().date()

if last_run_date is None or current_date > last_run_date:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you are running collect_metrics at the beginning of the day, then in daily_transactions field for this date in db will be in fact daily transactions count from the previous day. Is it supposed to work like this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

Comment on lines 33 to 34
TRANSACTION_COUNT_FIELD = 'transactions_count'
BACKFILL_DB_DAYS = 30
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably it's better to move db constants to src/config.py

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved

@dmytrotkk dmytrotkk merged commit 95f4e5f into develop Sep 24, 2024
6 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Sep 24, 2024
@dmytrotkk dmytrotkk deleted the add-historic-metrics branch October 21, 2024 18:50
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update metrics collector to store historic app stats
4 participants