Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ftrace: improve binary parser speed by optimizing I/O #31

Conversation

fabrizioiannetti
Copy link
Contributor

  • mmap the whole file for faster parsing
  • re-use the same mapping to read events

on my (oldish) laptop with ubuntu 24.04 the file is parsed an order of magnitude faster

on a recent windows laptop I measured two orders of magnitude faster

@fabrizioiannetti fabrizioiannetti force-pushed the faster_ftrace_dat_parse branch 2 times, most recently from 8838680 to 352b4a1 Compare June 4, 2024 13:46
@MatthewKhouzam
Copy link
Contributor

This is still WIP. I know your approach will work, we didn't have time to get it done. (We start by coding defensively, then optimize) This will be great, please remove the printlns though ;)

@fabrizioiannetti
Copy link
Contributor Author

This is still WIP. I know your approach will work, we didn't have time to get it done. (We start by coding defensively, then optimize) This will be great, please remove the printlns though ;)

missed that one :-), removed the println.

@fabrizioiannetti fabrizioiannetti changed the title WIP: improve binary ftrace parsing speed (~10x) ftrace: improve binary parser speed by optimizing I/O Jun 5, 2024
@fabrizioiannetti
Copy link
Contributor Author

Hi, is something missing in the PR for review?

MatthewKhouzam
MatthewKhouzam previously approved these changes Jun 14, 2024
Copy link
Contributor

@MatthewKhouzam MatthewKhouzam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missed the update, sorry!

Copy link
Contributor

@arfio arfio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change looks great, there is just one more change needed and then I will approve it 😄

super.dispose();
// release (indirect) references to mem-mapped file buffers so that tracecompass
// can garbage-collect them and unlock the file (e.g. if the user wants to delete it)
fStrategy = null;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be a dispose method in the IBinaryFTraceStrategy that does this for the BinaryFTraceV6Strategy.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review, done: I moved the clearing of the reference to the strategy class.
I did not go deeper in the structure, let me know if it's good like this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems I broke something with that, need to look into it :-)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change is perfect, it just needs to pass the test now :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like it needs a null check before calling dispose().

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the hint! Took me a while to find the right logs

* mmap the whole file for faster parsing
* re-use the same mapping to read events

Signed-off-by: Fabrizio Iannetti <[email protected]>
@MatthewKhouzam MatthewKhouzam merged commit 286f29e into eclipse-tracecompass-incubator:master Jun 18, 2024
2 checks passed
@fabrizioiannetti fabrizioiannetti deleted the faster_ftrace_dat_parse branch June 18, 2024 18:50
@MatthewKhouzam
Copy link
Contributor

@fabrizioiannetti thank you so much for the contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants