Note
This is a template for how to structure a data diary. This should always be paired with a methodology document. In here, you should log the strategies you try, mistakes you make, and how you solved the challenges you came up against.
Warning
This document is structured for a yearslong data diary. You can scale that down by simply removing the broadest timeframe header and increasing the header type for each tier you remove
Reporter: First Last
Work Completed
After talking with my editor and our community reporter about the charts I made, we narrowed down on what they want me to visualize.
Also cut the lines of code I needed in half
- Created lists for the types of crimes I needed
- Added columns and fixed date format before splitting the data into the two time periods so I only had to do the grouping once rather than twice
- This setup also actually made the charts I coded simpler and shorter to write
To Do Next
Export data to visualize in interactive tool
Work Completed
Wrote up code that separates data into the 10- and 5-year chunks then made charts
- Learned how to formatted the columns as dates properly
- Created new columns to flag the types of crimes that community members say they see more of
- Made charts at the annual, monthly, weekly, and daily level for the last 5 years and decided that I like how the monthly charts look better
Charts show that change in business did match a change in the crime levels like the residents said they experienced.
To Do Next
- clean up my data by changing the order I do the steps
- Talk to editor and reporter about findings
Work Completed
Figured out how to import data from the open data portal without having to download locally.
- Carto API access is very difficult to use, so instead ended up learning how to set up the access URL to be the
.csvdownload structure - Added instructions on how to do this to the reference guide
⭐ Professional Development This means I can go back and update up past projects with this more efficient data import process. I can collect data faster without using up storage space. This also applies moving forward.
To Do Next
I need to start cleaning and analyzing the data I have, looking at the last 10 years and last 5 years.
Spoke with the community reporter about a story that looks at crime data at a specific intersection in our city. They're curious if the changing businesses affected the crime levels, because that's what residents in the area say what happens.
We went over the story outline and I decided I will use crime data and business license data.