Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

turn comments into json #33

Open
fgregg opened this issue Mar 4, 2024 · 2 comments
Open

turn comments into json #33

fgregg opened this issue Mar 4, 2024 · 2 comments
Assignees

Comments

@fgregg
Copy link
Member

fgregg commented Mar 4, 2024

The comments field in the events table looks highly structure. Please turn into a json object.

@antidipyramid
Copy link
Collaborator

@fgregg Not all rows in the events table are in key: value form. For example, some rows have comments with values like Alias, Proof of service, or Citation To Discover Assets To POE.

For each row, we could create a JSON object and stuff any unstructured data into its own field, e.g. Proof of Service becomes {text: Proof of Service}. Would that work?

Also, should we make this transformation in the Python scrapers or as part of the SQL import process?

@antidipyramid antidipyramid self-assigned this Mar 15, 2024
@fgregg
Copy link
Member Author

fgregg commented Mar 15, 2024

let's hold off on this for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants