Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SkillCorner extrapolated_data.jsonl loading functionality #215

Closed
wants to merge 13 commits into from

Conversation

UnravelSports
Copy link
Contributor

I noticed there was no support for loading extrapolated_data.jsonl files, only deprecated structured_data.json files.

As you can see in the messy commit history I've added a couple things:

  • Functionality to load jsonl files if that file suffix is in place for raw_data. This functionality simply converts the json lines to a list of json objects such that it looks exactly like the structured_data.json files. We also rename the timestamp key to time to make it compatible with the rest of the code (that relies on the old time key).
  • Added improved timestring parsing because the extrapolated files have a slightly different timestamp format. The old format is "92:37.60" whereas the new format would display this as "01:32:37.60". The _timestamp_from_timestring() function can now handle both cases, converting both of these formats to the same output.
  • Added an improvement to reset the time at the start of each half to 0. As mentioned in Different start time second half SkillCorner and DataFactory compared to other Providers #214 this was not implemented yet. This might be a breaking change for some people relying on timestamps, so we might have to think about how to actually implement this last part.

Finally, I would not be surprised if the tests fail because of the change in period start time...

@DriesDeprest
Copy link
Contributor

DriesDeprest commented Dec 6, 2023

Thanks for the work @UnravelSports, I'm also interested in the .jsonl loading functionality for SkillCorner data!

@JanVanHaaren @koenvo what are your thoughts on the last point raised regarding the start timestamp of the second period?

@DriesDeprest
Copy link
Contributor

@koenvo @UnravelSports this commit should still be applied prior to merging imo. This way we use the SkillCorner ID as the player ID in the dataset. I'd love to do it myself but I can't contribute to this PR. So can any of you guys apply the changes?

@koenvo do you know why some of the checks are failing?

@JanVanHaaren
Copy link
Collaborator

@koenvo do you know why some of the checks are failing?

The checks should succeed again after merging in master. The issue was fixed in #231.

@UnravelSports
Copy link
Contributor Author

I've added a minor change as requested by @DriesDeprest which is a simple rewrite of what player ids we use in the SkillCorner parser.

See his personal fork 277a714

@JanVanHaaren
Copy link
Collaborator

@UnravelSports Could you please merge the master branch into your feature branch? Thanks!

@UnravelSports
Copy link
Contributor Author

It seems I've made a mess out of this, I'm creating a new PR to make a clean PR 😆

@UnravelSports UnravelSports mentioned this pull request Jan 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants