Skip to content
This repository has been archived by the owner on Mar 5, 2020. It is now read-only.

Curriculum Database Research Findings

Jump to bottom
Luke Pacholski edited this page Feb 9, 2016 · 11 revisions

#Research Findings: How educators search for and evaluate teaching resources online.

Research performed by Ricardo Vazquez, Sabrina Ng and Luke Pacholski.

###Table of Contents

##Key Findings

Here are the key findings for each of behaviors that we asked about...

####Finding Materials

  • Even though educators are aware of many great sources of materials and content, they tend to start searches with Google.
  • Educators identify multiple interesting materials when searching via cursory examination and dig into them later when evaluating.

####Evaluating Materials

  • Educators want to know if the materials they’re looking at is developmentally appropriate for the level they are teaching.
  • Educators don’t seem to put much stock into “ratings”, but are interested in reading about other educators’ experiences using a specific material.

####Using Materials

  • Educators like materials that provide instructions and guidance on how adapt the materials to the level they are teaching.
  • Educators want engaging, interactive activities for their students, and like materials with handouts, worksheets, templates and hands-on activities.

####Discovering Materials

  • Teachers rely on all sorts of ways to discover new content, but personal relationships and networking seem to play a key role in material discovery.
  • Teachers tend to share their own curriculum informally with each other, and rely on a wide range of services, online communities, and tools to do so.

##Introduction & Summary

####What we wanted to find out

We wanted to find out how educators who teach digital skills, including web literacy, currently find, evaluate, use and share teaching materials and resources. We wanted to find this out to inform our design ideas and thinking around the curriculum database that we are thinking of creating.

####How we did it

We identified potential candidates through our internal networks and contacted them individually to gauge their interest and schedule interviews. In the end, we interviewed 7 participants, who are listed below. All of the interviews were conducted remotely over Skype, Vidyo or Google Hangouts.

Two team members from the design team interviewed each participant. The first person led the interview while the second person remained largely silent and took notes. The interviews took roughly 45 minutes each and were divided into two sections: the interview followed by an observation session.

The interview was designed to get to know the educator, where and how they taught, and to ask directly about what we wanted to find out. During the observation session, we asked the participant to share their screen while they performed their usual steps to find teaching materials or resources about a digital skills topic we agreed on beforehand.

Notes were kept for each participant in separate documents and then compiled into the summary you’re reading now. Each participant was rewarded with a $25 amazon gift card and thanked profusely.

##What we found

In general, we learned a great deal about the varied ways in which educators find, evaluate, use and share materials online - we’ve divided the findings into the sections below, along with recommendations and ideas for how our findings could inform the curriculum database design.

####What we’ll do next

To continue this process, we’ll create two sets of wireframes for a curriculum database proposal. These wireframes will incorporate what we learned during this research process.

####Follow-up Questions

  • Some things we could have asked but didn’t.
  • Does the cost of a material or resource matter when deciding to use it?

####What worked well

  • The participants that we interviewed were very happy to talk to us, providing us with lots of detailed, candid answers.
  • Many of the participants were interested in finding out the results of our work and wanted to be kept updated.
  • Two person interviews, with one note taker and one primary interviewer, worked well.
  • Mixing up the roles of note-taker and interviewer among the three designers who performed the research worked well too, and gave each of us first-hand experience interacting with our target users.
  • The screenshare observation activity was very useful. Participants seemed to enjoy narrating their search process and it allowed us to get a glimpse into their individual processes and methods on their home turf.

####What didn’t

  • Many of the questions were qualitative in nature, so it is hard to draw comparisons across participants.
  • Asking educators to imagine a “perfect world” or solution for finding materials online didn’t yield any good insights.
  • I think this was too hard to come up with on the fly for the participants.
  • We should have them react to ideas or solutions that we’ve prepared (like the wireframes in the next phase of research)
  • It might have been good to start with a “hypothesis” for the research like: “Most educators are frustrated with finding teaching materials online”.
  • This would have made drawing a conclusion from the research a little easier.
  • We had a wide range of participants with different backgrounds and experiences but only had a line of questioning that was appropriate for one type of persona.
  • In the future we could adjust our questions to better match the participant’s circumstances.
  • We could screen better to ensure the participants fit our target audience more closely.
  • While we know more about how educators find materials, I’m not sure that we have a sense if that process is fundamentally broken and frustrating or just dandy in their eyes.

####Ideas for similar future down the road

  • Include a mix of qualitative and quantitative questions.
  • When multiple interviewers perform the same interview, it’s really important that the interview questions are good...
  • They must be granular enough that you can go through them one by one.
  • They must be robust enough to cover all of the data points you need.
  • We have to leave space for the interviewer to pursue leads during the interview, but not rely on that to get complete data.
  • We need to be more deliberate about who we ask to participate, and to make sure that they fit the personas that we are interested in. While we got a lot of good information, not all of it came from our ideal demographic.
  • We had an idea to try using a google form as an aid for the note taker. They could place their notes into the form so that they would be easier to retrieve and compare across participants.
  • We can ask participants to fill out an online form with some of their basic information ahead of time.
    • This would save time in the interview for the good questions.
    • It would let us adjust our line of questioning to better meet their circumstances.

##Our Esteemed Participants

We interviewed 7 participants with a wide range of backgrounds.

RV - College Educator Teaches web design and development topics (HTML, CSS, UI, UX, etc.) at Sheridan College in Ontario. Courses are 12 weeks long with students ranging from roughly 18 to 40 years of age.

RM - After School Educator Teaches digital skills to primarily middle school (but some high school) students as part of the Afterschool Network.

AZ - Tech Integration Specialist Works with her School District and K-12 Teachers?

RD - Executive Director of Digital Learning Started as a high school math teacher, but now trains educators to about CS curriculum in their classes.

WS - Afterschool educator with Bright Futures Started as a middle and high school science teacher, then transitioned to teaching with Bright Futures, where he teaches digital skills in after an after school program.

MY - Middle School Teacher Teaches science at a middle school and also runs digital skills section designed to introduce Tablets into the school curriculum. Works with students in the digital program to come up with ways to integrate the tablets.

GH - Higher-Ed Badge Trainer Helps higher education institutions learn and incorporate badging into their assessment

##How Educators Search for Materials

We asked participants where and how they look for teaching materials and resources online, and also observed over a screenshare as they did so.

####Insights

  • Most educators start with Google when searching for curriculum or material.
  • When searching with Google and looking at results, tend to evaluate sources based on their past history with that source or author - they care about who produces the materials.
  • Attracted to reputable sources or authors they recognize.
    • When scanning a list of results, recognized the sources by teh title listed in the search results or the URL/domain.
    • Most educators are highly biased towards publications or websites that they know well.
  • Educators tend to evaluate or explore a source by looking for links and exploring those resources.
  • They tended to look for information about a topic first, and not for “learning” or “teaching” or similar keywords.
    • But then added terms like “middle school” to narrow their results.
    • or added terms for particular medium or software
  • They open a lot of tabs and download many resources to review at a later date.
  • Educators didn’t want to have to sign in to a system and share their information just to search and review materials.

####What can the platform do to facilitate this?

  • Allow users to browse and search material by topic area.
  • Allow users to make keyword based searches.
  • Surface the source of the material prominently.
  • Allow users to mark certain authors or sources as favorites (ie. follow).
  • Allow users to mark & collect certain materials for later reading and review (ie. star, bookmark, heart, etc.).
  • Present users with links and materials that are related to their current search so that they can explore more.

##How Educators Evaluate Materials

We asked participants how they evaluate if materials are high quality and appropriate for their needs. Their primary concern is that the material is developmentally appropriate, and secondly that is engaging for their students.

####Insights

  • Educators want to know what the terms and conditions (license type?) for the materials they want to use.
  • Is it developmentally appropriate? - who is the material is aimed towards?
  • Age or grade-level
  • Difficulty level
  • Educators really want to know if kids are engaged. Their response to the material is very important, they want things that are fun.
  • They want reviews about and indications of this when reading about an activity, for example.
  • Students want materials that are hands on, have interactive components (like worksheets, handouts etc).
  • Seeing evidence that it’s been used and tested in a real world setting.
  • Educators like to know if the content comes with interactive activities, templates, files, downloadables, or other resources that they can incorporate into their lessons.
  • Educators like content that is adaptable.
  • To change for students who are more advanced.
  • To adapt the base curriculum to two different grade levels.
  • Educators want to know how long it takes to deliver the material or perform an activity.
  • The content is clear and organized with a good description.
  • How open and accessible to tools and technologies that are needed to perform the activities in a piece of curriculum, for example...
    • Are the apps free of charge
    • Are the restricted to a specific platform
  • Educators like content divided into sections, scannable or with clear numbered steps which reduces cognitive load and lengthy pieces of text

####What can the platform do to facilitate this?

  • Expose, and make it easy to search, filter by the following criteria:
  • Clear and easy to read description
  • Age/developmental appropriateness
  • And adaptability for more advanced students
  • And adaptability for younger and older students
  • Time to deliver
  • If there are resources & materials come with handouts, worksheets, and other supplementary collateral for the students.
  • Type of activity (group work, lecture, hands-on, watching a video, etc.)
  • Type of learning
  • Indicate what type of delivery format the material supports
    • Printout
    • Desktop
    • Tablet

##How Educators see Ratings and Reviews

In general, educators do not put a lot of initial emphasis on seeing a rating (like a star system) on teaching materials they are evaluating - they rely more on their own analysis of the material through a closer reading.

That said, educators appreciate in-depth reviews from other educators that describe how the material was used and what the reactions of the students were.

####Insights

  • Educators seem to not care about “ratings” but care about reading about the “experiences” that educators had using curriculum.
  • For example, stories about how they used and adapted the curriculum or like where the ‘sticking’ points were in an activity.
  • Educators prefer to make up their own minds about the activity by reading it carefully or by trying it out.

####What can the platform do to facilitate this?

  • Allow users to rate & review materials in a way that makes the reviews informative. For example...
  • How engaged were the students
  • How much did the students enjoy the activity
  • Students need help the most during this part of the activity

##How Educators Share Materials

We asked educators if and how they currently share the content they create & use with other educators.

####Insights

  • Educators share the curriculum they use informally, through Twitter and Email, for example.
  • Educators tend to rely on their personal networks to share & collaborate.
  • Most educators have ways that they already use to distribute their own curriculum, like personal blogs or websites.
  • Some educators post or contribute to a larger educational platform related to their school district.
  • Most educators are open to sharing their materials with other educators.
  • Educators like to engage and connect with each other when using each other's curriculum.

####What can the platform do to facilitate this?

  • Provide a way for users to collect and organize their favorite online resources for themselves and for others.

##How Educators Organize Materials We asked educators how they keep track of and organize their teaching materials.

####Insights

  • Educators have a variety of ways, both online and on their computers of organizing resources and materials for personal consumption.
  • Bookmarks
  • Folders on their computer for saved resources
  • Pinterest
  • Evernote
  • Google pages
  • Many educators are using Google apps, both for creating, sharing & storing content.
  • Two educators mentioned using Evernote - we should benchmark it to see what features are interesting and useful.

####What can the platform do to facilitate this?

  • Provide a way for users to collect and organize their favorite online resources for themselves and for others.
  • Users value the organization of these resources based on similar criteria they use to evaluate content (topic, length, developmental appropriateness, etc).

##Additional links & Resources

Good online resources mentioned by participants...

Table of Contents

Clone this wiki locally