Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support parsing/ignoring Java Flight Recorder events #44

Open
charlesverdad opened this issue Nov 26, 2024 · 2 comments
Open

Support parsing/ignoring Java Flight Recorder events #44

charlesverdad opened this issue Nov 26, 2024 · 2 comments

Comments

@charlesverdad
Copy link

charlesverdad commented Nov 26, 2024

Async-profiler has the --jfrsync feature which allows me to add additional Java Flight Recorder data like GC collection, network io, thread information, etc. which can be viewed via the JDK Mission Control app.

I expect jfr-parser to be able to work with async-profiler profiles even with the jfrsync=default flag. However if I try to ingest my profile to Pyroscope (v 1.10.0), jfr-parser throws this error:

❯ cat my_profile.jfr | curl --data-binary @- -X POST 'localhost:4040/ingest?name=test_rpc&format=jfr'
{"code":"unknown","message":"parsing IngestInput-pprof failed jfr parser ParseEvent error: error reading CP: error reading class{name: jdk.types.DeoptimizationReason, id: 156, fields: [{Name:reason Type:214 ConstantPool:false Array:false}]} int overflow @ 490455"}

The way our system works is we collect (via async-profiler) and store all *.jfr files to S3 and ingest a percentage of those to Pyroscope. In rare instances where devs need detailed analysis we allow them to download the raw *.jfr file. Some devs have seen the importance of also collecting flight recorder data but if they enable that via jfrsync flag, the profiles cannot be ingested into pyroscope anymore.

I believe it's because it doesn't understand what DeoptimizationReason is. There's a lot more in this list that Flight Recorder collects. I'd like the ingester to either one day support this, but for now happy if it just ignores all these events (instead of throwing an error) and just parse the current cpu, memory and lock information.

@korniltsev
Copy link
Collaborator

korniltsev commented Nov 28, 2024

There have been a bunch of fixes for parsing flight recorder files https://github.com/grafana/jfr-parser/issues?q=is%3Apr+author%3Asivachandran

I think jfr parser has not been updated in the pyroscope repo since then.

Please share an example JFR file so that we can include it in our testsuite and fix the parsing issue.

In the mean time I will update the parser in the pyroscope repo

@charlesverdad
Copy link
Author

charlesverdad commented Dec 8, 2024

Thank you!

Unfortunately I can't include the JFR I'm using. I'll try to generate a similar one that I can give out publicly.

I tested the same JFR file in the latest Pyroscope weekly build (weekly-f95-b2e5d0a36) and it seems to now fail silently. The curl command receives a status 200 but the profile does not appear in Pyroscope.

When I get time I'll try to debug the issue more and verify if it is jfr-parser related.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants