Skip to content

Latest commit

 

History

History
626 lines (446 loc) · 50.3 KB

README.md

File metadata and controls

626 lines (446 loc) · 50.3 KB
Raw

COMM 4940: Governing Human-Algorithm Behavior

J. Nathan Matias <nathan.matias AT cornell.edu> , Department of Communication, Field Member, Information Science

  • Spring 2024 (see course listing)
  • Time: Tuesday & Thursday 11:40am - 12:55pm
  • Location: Malott Hall 224

Paradise Wildfires, YouTube algorithms, Wilmer Catalan-Ramirez

Images: Google Waze blocks wildfire escape route, The international affiliation network of YouTube Trends, and Immigrant released from detention after being wrongly listed as gang member.

Algorithms that monitor and influence human behavior are everywhere—directing the behavior of law enforcement, managing the world's financial systems, shaping our cultures, and flipping a coin on the success or failure of movements for change. Since human-algorithm feedback is already a basic pattern in society, we urgently need ways to assess the impact of attempts to steer that feedback toward justice.

In 2023, the challenge became even clearer when Attorney General Letitia James and 31 other state Attorneys General filed a federal lawsuit against Meta for allegedly "harming young people's mental health and contributing to the youth mental health crisis." One of the lawsuit's claims is that Meta, through Facebook and Instagram, encouraged compulsive use of the platform through its algorthms and design features—contributing to a widespread mental health crisis.

In this course for (15) upper-level undergraduates and (5) PhD students, you will learn about the design of adaptive algorithms and the feedback patterns they create with human behavior. You will learn about the challenge they represent for social policy, about ways to research their behavior, and about emerging policy ideas for governing these complex patterns. You will get first-hand experience at diagnosing and attempting to change a feedback system. Along the way, you will hear from pioneers in policy, advocacy, and scholarship.

This course is an excellent stepping stone for anyone interested in a career in policy, advocacy, academia, or industry research.

How to Register for this class

Upper Level Undergraduates: to join this class, register for COMM 4940 like any class.

PhD Students: to join this class, you have two options:

  1. Enroll in an independent study (COMM 7970) with Professor Matias and then attend the class (preferred). Before enrolling, please write a brief note to Professor Matias with "Enrolling in COMM [4940]" in the subject. The email should:
    • Introduce yourself, your program, and where you are at (1-2 sentences)
    • Explain why you want to take the course. For example, is it related to your research? An interest in policy? A project you want to develop in the course? (2-3 sentences)
    • Describe the perspectives and skills you bring to the class (a few bullet points) (for example, an understanding of certain theory, or capabilities at qualitative research, data analysis, policy writing, advocacy, media-making, or software development)
  2. Enroll in COMM 4940 as a piece of elective coursework

What You Learn

In this seminar class, you will engage with the science and policy challenges of regulating human-algorithm behavior. Along the way, you will work with scientific issues of methods, theory, and ethics, policy questions about how to govern these situations, and how to bridge between science and policy in a democracy. For a final project, student teams (3-4) will produce a novel project that makes a contribution at the intersection of science and policy.

By the end of the semester, students will be able to:

  • Identify, analyze, and evaluate claims about how human and algorithm behavior interact
  • Summarize and criticize major approaches to governing human and algorithm behavior
  • Describe and evaluate major governance approaches
  • Design and analyze research methodologies for creating policy-relevant evidence for transparency, accountability, and change
  • Understand the uses of social science and computer science research in the policy process
  • Design an intervention into scholarly and policy conversations on human-algorithm behavior

PhD students are encouraged to connect the course to an existing research question that they wish to connect with policy, or to develop a project that could become part of their wider research.

Activities

Weekly Activities: Throughout the semester, students will read a selection of articles and discuss that reading in class and on Canvas. Once teams have been formed, students will also submit regular progress reports on their final project.

Algorithm Incident Report: The midterm is an analysis of an algorithm-involved event in the news, based on publicly-available information.

Project Proposal: Your project proposal will include a description of the project, a bibliography, a list of the roles that team members will play, and a timeline.

Final Project: An intervention into academic and/or policy conversations on human-algorithm research. In 2024, the class will support projects in three areas:

  • An in-depth case study and/or forensic analysis of some failure of an adaptive algorithm (see the AI Incident Database for ideas)
  • A proposal for a "black box" or data schema of what to record to inform future analysis of incidents
  • A digital simulation or card game that can be used to engage lived experience experts in critical thinking about human-algorithm behavior.

Grading: Participation in class & online: 30%. Midterm: 30%. Final project: 40%.

About the Instructor

Dr. J. Nathan Matias (@natematias) organizes community behavioral science for a safer, fairer, more understanding internet. A Guatemalan-American, Nathan is an assistant professor in the Cornell University Department of Communication. He is also a field member in Information Science.

Nathan is the founder of the Citizens and Technology Lab, a public-interest project at Cornell that supports community-led behavioral science—conducting independent, public-interest audits and evaluations of social technologies. CAT Lab achieves this through software systems that coordinate communities to conduct their own research on social issues. Nathan has worked with communities of millions of people to test ideas to prevent online harassment, broaden gender diversity on social media, manage human/algorithmic misinformation, and audit algorithms.

Schedule

Tues 01-23 Why Should We Care About Human-Algorithm Behavior? (Intro class)

This class session will introduce the course and pose initial questions.

Thurs 01-25 Tech and Methods: How Do Adaptive Algorithms Work?

In this class, we will discuss the code and mathematics behind adaptive algorithms.

Questions to consider:

  • What are the specific claims being made about Meta and mental health?
  • What are social recommendations? How do they differ from personalized recommendations?
  • When would you want a personalized recommendation, and when would you want a recommendation that is not personalized?
  • How might a recommender system contribute to mental health harms? How might they reduce mental health harms?

Tues 01-30 Feedback Loops: Simple Models to Explain Complex Behaviors

Questions to think about from Windigo Footprints:

  • Think about an event related to algorithms and AI systems that matters to you. It might be an incident related to mental health, or discrimination, or something else
  • Imagine the forces akin to "hunger" (the drive for more) and "gratitude" (a drive for less of something) that direct individuals and institutions toward more and which ones direct them toward less
  • Reflect on how those forces might have contributed to that event.

Thurs 02-01 Incidents and Disasters

In this class, we're going to ask the question "What is an incident" and "What is an accident/disaster/catastrophe?" while also discussing how understanding them might help society: Readings:

Questions to think about — choose a specific risk, harm, or incident involving AI, and consider:

  • Based on the article by Pinch, list out at least six of the "causes" of the Challenger disaster, and label them (were they technical, political social, etc). Could you name the most important one?
  • List out things that might possibly go wrong
  • What might you want to record in order to go back and figure out what went wrong?

A cool article I sadly wasn't able to include:

  • Knowles, S. (2014). Engineering risk and disaster: Disaster-STS and the American history of technology. Engineering Studies, 6(3), 227-248.

Tues 02-06 Case Study: Algorithms and Discrimination

To build our intuitions about governing algorithms and what to do about it, we're going to consider a series of classes about discrimination online. This class introduces the problem of discrimination in online labor markets.

Questions to think about for class and discussion:

  • How does Mullainathan think about discrimination and bias?
  • In Mullainathan's view, what does it mean to "fix" bias?
  • How does McGhee think about discrimination and bias?
  • How might McGhee think about "fixing" discrimination and bias?
  • What do we gain and lose by redefining our understanding of social issues only to the things that algorithms can meaningfully address?

Thurs 02-08 Case Study on Discrimination: Laws, Individuals, Organizations, and Code

Discrimination has a history and is reinforced through feedback from multiple sources beyond software: laws, individuals, organizations, and code. In this class, we will discuss how they interact.

Questions to consider from The Color of Law:

  • How was discrimination built into:
    • Government housing allocation
    • Access to education
    • The geography of the Bay area
    • Government services
    • Policing
    • Jobs & employment
    • Access to loans
    • Public advertising

Questions to consider from Algorithms of Oppression:

  • What is "racialization" ?
  • How might the patterns described in The Color of Law influence the patterns observed in Algorithms of Oppression?
    • Who gets to create algorithms
    • Who gets to teach the people who create algorithms
    • What information the algorithms learn from
    • What questions people type into search boxes
    • What ads appear next to search terms

Overall questions:

  • how might we describe these patterns as feedback loops?
  • what makes algorithms similar to other sources of discrimination, and what makes them different?

Thurs 02-15 Could Things Be Different?

In this class, we will discuss the challenge of determining responsibility for past harms and the difficulty of identifying interventions for change.

Tues 02-20 What is an AI Incident?

In order for algorithms to be governed, we need to be able to describe what happens when one goes wrong. So what is an incident anyway, and how would we establish that harm had occurred?

To prepare for class, think about the story by Sophie Mellor that we read at the beginning of class. Imagine that you are an employee at Instagram who is asked to investigate a mental-health related incident experienced by a young person who was an Instagram user.

  • create a definition for the incident type— what is it and what isn't it? How would you tell what counts as an incident or not?
  • make a list of 6 things that you would want to record from what might be know by Instagram, friends/family, and institutions

In class, we will work to collaborate on an incident report template for several kinds of mental health related incidents.

Thurs 02-22 Discussing final project ideas

In this class, we will discuss possible final project ideas for the course.

In preparation for the class, meet with a possible project team to develop ideas for collective feedback and discussion.

As a reminder, final project ideas include one of three options:

  • An in-depth case study and/or forensic analysis of some failure of an adaptive algorithm (see the AI Incident Database for ideas)
  • A proposal for a "black box" or data schema of what to record to inform future analysis of incidents
  • A creative project that can be used to involved lived experience experts in critical thinking about human-algorithm behavior and incident reporting.

FEBRUARY BREAK

Thurs 02-29 Feedback Pattern: Reinforcement

In a series of sessions, we will discuss common patterns of human-algorithm feedback. The first is reinforcement, which happens with individuals in the case of personalization and with groups in the case of herding behavior.

Tues 03-05 Feedback Pattern: Herding

Algorithms can amplify herding behavior by encouraging people to do something that's already popular. Ranking software has been credited with inspiring large-scale acts of charity, spreading prejudice and encouraging harassment mobs. When enough people engage in collective discrimination, adaptive systems can entrench injustice further. This class will investigate herding as a phenomenon of human + algorithm behavior.

Questions to discuss:

  • How does herding happen even without algorithms? What ideas from your other classes or research might help explain herding? (COMM 2450 students - remember threshold models?)
  • How might herding create social change you want to see in the world, as well as harms to avoid?
  • How might we interrupt herding, and how might you test it?
  • Who should have the right to interrupt herding?

Thurs 03-07 Feedback Pattern: Thresholds

Patterns of human and algorithm behavior can dramatically change when their behavior reaches a certain threshold. For example, a popular conversation within a marginalized group might attract harassment if an algorithm amplifies it to a wider population where hatred prevails.

Questions for discussion:

  • How might adaptive algorithms change how threshold models work, by influencing certain thresholds or influencing who is visible to whom?
  • If people have thresholds, how might algorithms also have thresholds? How can we influence algorithm thresholds?
  • How might thresholds be useful tools for governing human-algorithm behavior?

Tues 03-12 Methods Area: Simulations

People claim that algorithms amplify and spread hatred, misogyny and extremism. How could we tell, and what can we do about it? In this class, we will use the example of the spread of misogyny on Reddit as the basis of imagining simulations.

Thurs 03-14 How Much Complexity Should Incident Reporting Embrace?

So far this semester, we have looked at simple incident reports such as the systems used by the U.S. National Transportation Safety Board to report a crash— introduced in the 1960s. Since then, scholars have developed more comprehensive ideas about how to map and describe failures. How far should we go toward offering more complex models? And how much should we understand/engage with the politics at play?

In this class, we're going to learn more about those systems, in the case of the Richmond CA Refinery fire of 2012. We're also going to consider how incident reporting fits into wider political engagements over error reporting.

Tues 03-19 Finalizing Student Projects

In your final-project teams, students will present a 3 minute presentation to summarize and analyze an issue in human-algorithm feedback that you would like to work on for your final. Fellow students will submit feedback and suggestions.

Thurs 03-21 How Can Evidence Inform Governance?

What is evidence-based governance and how might research contribute (or not) to policy? In this session, we'll discuss a model where researchers are advisors, versus a model where researchers are producing evidence that is used in courts and other contestational settings.

Questions for discussion in Canvas and in class:

  • For your team's final project area:
    • What kind of evidence might be needed for policymaking
    • Is that kind of evidence possible yet?
    • Does the evidence already exist?
    • How does evidence already influence policy, if at all?
    • What forces are preventing evidence from being available?-->

Tues 03-26 Policy Topic: What Is Policy Anyway?

We've been talking about policy and governance in the abstract, but what is it really, and how is that changing in a world where algorithms are also being given governance power?

Questions for discussion:

  • What is a tech policy, really?
  • What is a platform policy, really?
  • What kind of policy outcomes can be achieved (or not) with code?

Thurs 03-28 Project Check-in

In this class, we will:

  • Support group collaboration time student projects
  • Discuss common questions about projects and final proposals

SPRING BREAK (Syllabus Adjustment Check-In)

Thurs 04-09 Policy Topic: How Does Evidence Actually Inform Policy?

In previous classes, we discussed theories for how evidence could inform policy. How does it really happen?

  • Each Team share on Canvas an example of a piece of research, journalism, or activism that genuinely impacted policy (ideally related to your project), and comment on how it did so. A especially high quality submission will be able to cite an example where someone made a claim about the relationship between the corporate/government policy change and the project. Comment on other team's examples, come prepared to discuss your team's example, and be prepared to ask questions and discuss other team's examples.
  • Examples:
    • Edelman, B. G., & Luca, M. (2014). Digital discrimination: The case of Airbnb.com. Harvard Business School NOM Unit Working Paper, (14-054).
    • Buolamwini, J., & Gebru, T. (2018, January). Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency (pp. 77-91). PMLR.
    • Penney, J. W. (2016). Chilling effects: Online surveillance and Wikipedia use. Berkeley Tech. LJ, 31, 117.
    • Matias, J. N. (2019). Preventing harassment and increasing group participation through social norms in 2,190 online science discussions. Proceedings of the National Academy of Sciences, 116(20), 9785-9789.
    • Bossetta, M. (2020). Scandalous design: How social media platforms’ responses to scandal impacts campaigns and elections. Social Media+ Society, 6(2), 2056305120924777.
    • Orben, A. (2020). The Sisyphean cycle of technology panics. Perspectives on Psychological Science, 15(5), 1143-1157.
  • Contandriopoulos, D., Lemire, M., Denis, J. L., & Tremblay, É. (2010). Knowledge exchange processes in organizations and policy arenas: a narrative systematic review of the literature. The Milbank Quarterly, 88(4), 444-483.

Thurs 04-11 Policy Topic: Algorithmic Literacy

In order for democratic societies to govern a problem, people need to know about it and care enough to do something. How does the public understand and care about human-algorithm feedback?

Discussion questions:

  • If the design of algorithms is controlled by engineers and most people aren't engineers, how would algorithm literacy actually make a difference in people's lives if at all?
  • What kinds of knowledge about algorithms does a democratic public need?
  • How could your project area be transformed by algorithmic literacy — and how not?

Tues 04-16 Policy Topic: Public Engagement with Incident Reporting

If incident reporting is to work, the public need to understand incidents. In this class, in conversation with Dr. Jennifer King at Stanford University, we will look at the history of the Dark Patterns Tip Line.

  • Nguyen, Stephanie (2021) Key learnings from the Dark Patterns Tip Line. Rita Allen Foundation
  • King, J., & Stephan, A. (2021). Regulating Privacy Dark Patterns in Practice-Drawing Inspiration from the California Privacy Rights Act. Georgetown Law Technology Review, 5(2), 250-276.

Questions for discussion on Canvas and in class:

  • For your team's topic, how might it be possible to ask the public to report incidents?
  • Would the public be able to notice and distinguish possible incidents?
  • What kind of public awareness will be necessary?

Thurs 04-18 Policy Topic: Creating Change With Courts

What influence can courts have on the behavior of companies, populations, and algorithms?

Questions to consider:

  • What role did courts play in the policy debates about the Strategic Subjects List?
  • How can laws and agencies influence what courts are able to do?
  • How might courts be important to the project you're working on?

Tues 04-23 Final Project Check-in

In the class, teams will be supported to meet in person, workshop your ideas with peers, and get feedback on your project progress.

Thurs 04-25 Case Study: Myanmar and Meta

Guests:

  • Susan Benesch
  • Erin Kissane

Tues 04-30 Final Presentations Part I

In this session, teams will give a final presentation of their project, with an opportunity to receive feedback from peers.

In the discussion, please post in advance a question for feedback you would like your peers to consider during your session in the class.

Thurs 05-02 Final Presentations Part II

In this session, teams will give a final presentation of their project, with an opportunity to receive feedback from peers.

In the discussion, please post in advance a question for feedback you would like your peers to consider during your session in the class.

Ties 05-07 Reflections on Learning & the Future of Algorithms and Society

In this session, we will reflect on the topics in the class. Please submit a discussion post, which will inform our group conversation:

  • Name something you believed at the beginning of the course
  • Reflect on how your thinking about that question has changed
  • Name one way this topic might continue to be relevant to your future life, whether in your role as citizen or your work

Final Project Deadline: May 16

Course Practices and Policies

Weekly Workload

Before each class, I will assign two readings I expect you to read before class. As part of class participation, you will submit a reaction comment on one of the readings to the relevant discussion on Canvas and respond to at least one other student's comment by 9pm Eastern the night before class. Please come to class with:

  • The ability to summarize:
    • the goal/question of the paper
    • the field or ecosystem that the author is in
    • what constitutes an advancement in that field
    • how the paper advances the conversation
    • a question or observation that links the paper to the theme of the day or the theme of the course

Each week, students are expected to post at least one news article to Canvas add at least one comment in response to one other student's posted link.

Graduate students will have additional readings. Graduate students will also rotate responsibility for summarizing presenting to the class on the readings assigned to everyone. Summarizing the reading will involve:

  • Creating 2-3 slides for the reading
    • Slide one: introducing the core question of the paper, and the authors
    • Slide two: summarize the methods and findings of the paper
    • Slide three: one to two discussion questions that link the paper to the theme of the class
  • Alt: if the author of the paper is a guest speaker, the graduate student will interview the guest

Since this is a discussion course, attendance is expected.

Participation in Discussions

Please post discussions about readings to Canvas.

Whenever the course has assigned reading for a session, you are expected to post at least one top-level comment and one reply to someone else's comment. Participation will be 30% of your course grade.

Team Progress Reports

During the project period of the class, teams will submit on Canvas a weekly progress report no more than one page long, as group homework. This progress report will be graded 0/1 based on whether it was submitted. Reports should include the following details (a sample template is available here):

  • What your team made progress on
  • What your team is doing next
  • Who contributed what
  • Where your team is stuck
  • Any updates to your timeline

Formats

Written assignments should be uploaded to Canvas in one of the following formats

  • PDF
  • Text file
  • Word-compatible document

Slide decks should be submitted in one of the following formats:

  • PowerPoint
  • KeyNote
  • Google Slides
  • A Markdown slide presentation system (such as Marp)
  • PDF

In cases where students choose to submit code or analysis as part of a project, I can accept assignments in the following languages:

  • R (including R Markdown or Sweave)
  • Python
  • Ruby
  • PHP
  • C / C++

Group Work and Academic Integrity

I expect students to follow the Cornell University Code of Academic Integrity. You should submit your work as your own, cite sources and outside assistance, and credit people for their contributions.

This class includes group work and individual assignments. On group projects, you are encouraged to work together on the activities for the class, but you are only only able to put your name on projects to which you made an intellectual contribution. If you have any doubts about what is appropriate, ask me.

Grading

I use the following grading scale, derived from Matt Salganik's grading practices.

All grades are final. There will not be any make-up or extra credit assignments offered.

Per university procedures, I will only give a grade of A+ in exceptional circumstances.

Letter Grade Numeric Grade

  • A 93 - 100
  • A- 90 - 92.99
  • B+ 87 - 89.99
  • B 83 - 86.99
  • B- 80 - 82.99
  • C+ 77 - 79.99
  • C 73 - 76.99
  • C- 70 - 72.99
  • D 60 - 66.99
  • F 0 - 59.99

Grace Period

Late Submissions: All assignments have an automatic one-day grace period. On time and early papers are always encouraged and will be graded the same week. Students can turn in the assignment up to a day late, no questions asked, with the expectation that it could take substantially longer to receive a grade and feedback. After that, an automatic one grade (A to B, B to C, etc) is dropped on the assignment.

If you turned in an assignment on time and haven't received a grade within the expected period, please contact the instructor in case of a technical glitch.

Accommodations

I am committed to working with students with recognized SDSes to ensure you have the best possible class experience. Please follow these steps to get started.

Health Accommodation for Professor Matias

As part of a medical condition of my own, I may on rare occasion need to teach remotely. Since I may not be alerted to this need until the morning before class, I will ask the designated presenting graduate student to also set up a video chat feed for me to join.

Counseling Resources

It is common for students to experience stressful events at some point during graduate school. Students sometimes experience depression, anxiety, family stress, the loss of loved ones, financial strain, and other stressors. It is perfectly normal for students to seek the service of mental health professionals to provide them with support and skills to cope with these experiences. Below I have provided the contact information for some of the mental health services available to Cornell University students so that you will know where you can go if you or a friend would like to take advantage of these resources.

Cornell Health: 110 Ho Plaza, Ithaca NY. Phone: 607-255-5155.

If you require a short term accommodation, please let me know as soon as possible.

Acknowledgments

I am grateful to my co-author Lucas Wright for his contributions to a review article that this course is based on, and to the Human-Algorithm-Behavior Research Collective for input. Several sections of these policies have been inspired by syllabi from Matthew Salganik, Adrienne Keene, and Neil Lewis Jr.