OpenRefine as tool to identify and repair broken data entries

[tholzheim]

• Raw data: or_events.zip (data from wiki backup files extracted with wikiextract)
• Blank project: openresearch_events.openrefine.tar.gz
• Project with some applied changes: events-json.openrefine.tar.gz

Identify invalid data

• Duplicate acronyms #164
• Duplicate Event Entries #157

Example usage for #71

1. Trim whitespace
   
2. Converte to date
   
3. Select Timeline facet
   
4. Select all entries that could not be converted
   

[WolfgangFahl]

Impressive. I'd love to see the process from the OpenRefine Results to the import. Can we do that as a unix-pipeline already?

• Input-Query -> list of events/series
• list of events/series -> PTP -> computer readable signature of events/series (JSON) with provenance ids
• signature lists -> OpenRefine (optional if data is messy or incomplete (e.g. provenance)) -> cleaned up list of signatures (JSON)
• computer readable event / event series list cleaned up with signatures & provenance
• import into wiki via wikirender

the whole above procedure would be related to Exchange/Transfer Button for trials of import