Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simpler stats rewrite #18

Open
Antar1011 opened this issue Jun 8, 2017 · 0 comments
Open

Simpler stats rewrite #18

Antar1011 opened this issue Jun 8, 2017 · 0 comments

Comments

@Antar1011
Copy link
Owner

At this point, it's kinda obvious that Onix has stalled out. I regret nothing, in that I learned a metric ton about SQLAlchemy, CICD, packaging, documentation...

but at the end of the day, Onix is probably over-engineered.

I'd like to spec out an "80/20" stats rewrite solution--something that's robust and easily maintained, but not nearly as complex as Onix.

I'm thinking:

  • preserve the current workstream (read logs into intermediate files, process intermediate files into stats)
  • clean up and modularize code, adhering to coding best practices (four spaces instead of tabs!)
  • some amount of unit tests (probably more for the high level stuff like the tier update code, rather than the low-level log-reading)
  • move as much of the bash logic into python as possible (glob! concurrent.futures!)
  • optimize workstream by doing stuff like computing stats for all baselines at once
  • pull code from Onix where appropriate (Contexts are nice)
  • py3 only -- I just find it super painful to write py2 code now. Don't worry about pypy--if I want performance, I'll see if I can numbafy anything.

I'm thinking the rewrite will live in this repo.

It's probably still going to be months before I can get around to tackling this, but the reduction in scope should make this actually attainable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant