Skip to content

Latest commit

 

History

History
857 lines (535 loc) · 62.1 KB

README.md

File metadata and controls

857 lines (535 loc) · 62.1 KB

Responsible Data Sharing (RDS) Guidebook Resources

Repository for resources included in the Responsible Data Sharing Guidebook written by Natalie Evans Harris.

Phase One: Build the Collective

1.1 Define the Collective

1.2 Establish Shared Vision

1.3 Understand Data-Sharing Capacity

1.4 Understand the Data Being Shared

1.5 Survey Security and Privacy Related to the Data

1.6 Examine the Ethical Implications of Sharing the Data

1.7 Draft your Charter

Phase Two: Define the Operations

2.1 Determine Governance Framework Structure

2.2 Formalize Responsible Data-Sharing Practices

2.3 Examine the Sustainability of the Operations

2.4 Develop an Ethical Framework for Data Sharing

2.5 Define What Success Looks Like

2.6 Draft Data-Sharing Agreement

Phase Three: Drive Impact

3.1 Reinforce, Update, and Share Governance Best Practices

3.2 Reinforce, Update, and Share Best Practices in Ethics Governance

3.3 Monitor and Assess Privacy and Security Approaches

3.4 Develop Processes to Improve Data Quality Over Time

Data Sharing Today


Phase One: Build the Collective

1.1 Define the Collective

1. Key Resources and Digital Tools to Utilize:

a. What problem are you solving?

  1. Problem Definition Worksheet - Wageningen University
    A worksheet in the Multi-Stakeholder Partnerships portal to help you define your problem in depth.

  2. Are You Solving the Right Problems? - Harvard Business Review
    An article that talks about the importance of defining a problem and how an organization might go about ensuring that it is solving the right one.

b. What impact are you pursuing and what is your theory of change?

  1. All In - Data for Community Health Online
    An online community of individuals dedicated to improving health through multi-sector data sharing and collaboration, designed to help you connect to professionals tackling common challenges, share resources and news, and learn about new ideas and best practices.

  2. IssueLab - Candid
    One of the largest open repositories for social sector knowledge; IssueLab helps organizations better manage, synthesize, and share what they are learning.

  3. Theory of Change: A Practical Tool for Action, Results and Learning - AECF
    This 49-page report will help you understand how to create a Theory of Change and align it with your community for optimal results.

c. Who are they key stakeholders?

  1. Stakeholder Analysis: Securing the Buy-In You Need - Lucidchart
    A blog article on how to identify your stakeholders and get their buy-in.

  2. The Three Goals and Five Functions of Data Stewards - GovLab
    A blog post describing what a data steward is and what roles and goals data stewards have in pursuing social impact in the information era.

  3. Using Data for Action and for Impact - Stanford Social Innovation Review
    This article reminds readers that the nonprofit and public sector have to deal with many more stakeholders and outcomes than private companies and for this reason they often struggle with how to properly acquire and use data responsibly. =

  4. Learning in Action - Evaluating Collective Impact -Stanford Social Innovation Review
    An article that will help you think through what collective impact is and why it is essential to consider in the context of partnerships for tackling social problems.

2. Additional Resources:

a. Defining and Analyzing the Problem - University of Kansas
Chapter 17 of the Community Tool Box focused on helping you think through what the problem you are working on is and whether you should pursue it.

b. The 100 Questions Initiative - GovLab
This initiative seeks to map the world's most pressing, high-impact questions that could be answered if relevant datasets were leveraged in a responsible manner.

c. Social Value Proposition - Forbes
A four step framework on building social value propositions that are compelling and relevant.

d. Matching Administrative Data to Inform Policy - Northwestern University
Newsletter article describes an effort at the university to create a new network, set goals for the network, and start to build community around matching administrative data for policy valuation.

e. Five Principles for Applying Data Science for Social Good - O'Reilly
A blog post on best practice principles for using big data for the social good, written by Jake Porway from DataKind. The second principle discusses the difficulty of finding problems. The fourth principle says that teams need a diverse set of stakeholders because they need a diverse set of viewpoints.

f. AISP Network Site Case Studies - Actionable Intelligence for Social Policy
The importance of a top-down, problem-first approach to getting an organization on board is covered in several Integrate Data System (IDS) case studies. For example:

  1. A study of how New York City set up its IDS found that programmatic needs have to come first and drive all decision-making, or else partners will not see the value in sharing.

  2. A study of how Los Angeles County developed its IDS found that focusing too much on the technology at first was a mistake, and that what really allowed success was to find one person at the top who can cheerlead a project into existence.

  3. A study of how Washington State's DHHS developed its IDS found that getting leadership to generate momentum was key, so that the value of sharing was recognized, capacity was generated, and initial reluctance to share could be overcome.

  4. Learning in Action: Evaluating Collective Impact - Stanford Social Innovation Review An article that will help you think through what collective impact is and why it is essential to consider in the context of partnerships for tackling social problems.

  5. Stakeholder Analysis Guidelines - World Health Organization A detailed report from Partnerships for Health Reform on how to precisely define your policies and stakeholders.

g. Scaling Big Data for Social Good: The Need for Sustainable Business Models - GSMA
A report on making sure that your social impact solution is sustainable and will scale up.

1.2 Establish Shared Vision

1. Key Resources and Digital Tools to Utilize:

a. What are the motivations and goals of stakeholders in wanting to share data?

  1. Data Collaboration for the Common Good - World Economic Forum
    See page 12 of section 2, part 1 on entitled "Stakeholder Alignment" that gives tips for building trust through stakeholder alignment, advice for responding to challenges and tools and resources for these issues.

b. What are the barriers to sharing data and can they be overcome?

  1. A Solutions-Based Approach to Building Data-Sharing Partnerships - Journal for Electronic Health Data and Methods
    A paper by Wieche et al. describing a taxonomy of barriers and solutions to sharing data for health research.

  2. A Decision Model for Data Sharing - 13th International Conference on Electronic Government
    A literature review of data-sharing governance practices from Eckartz, Hofman, and Veenstra that includes three brief case studies in the logistics sector and a data-sharing decision model to help actors identify and address barriers to sharing.

  3. Please see below for specifics as it relates to legal, organizational, and ethical barriers to sharing data.

2. Additional Resources:

a. Announcing the Data Collaboratives Research Network - GovLab
A blog post announcing and offering basic descriptions of the data collaboratives effort by GovLab at New York University.

b. Understanding Corporate Data Sharing Decisions - Future of Privacy Forum
A paper that explores when and why corporations donate data for academic research.

c. The Potential of Social Media Intelligence to Improve People's Lives - GovLab
This resource lists examples and a detailed discussion for each of the five kinds of motivation.

d. Designing and Implementing Cross-Sector Collaborations: Needed and Challenging - PublicAdministration Review
A well-written academic paper from Bryson, Crosy, and Stone on the barriers to cross-sector collaboration.

e. Data Driven Social Partnerships: Exploring an Emergent Trend in Search of Research: Challenges and Questions - Government Information Quarterly
A detailed literature review from Susha, Grönlund, and Van Tulder sketching out the field of data sharing for social good and detailing what barriers and solutions others have found.

f. Barriers to Using Government Data - Bipartisan Policy Center
An analysis of findings from the U.S. Commission on Evidence-Based Policymaking's Survey of Federal Agencies and Offices that details the context, barriers, and next steps to sharing government data.

g. A Systematic Review of Barriers to Data Sharing in Public Health - BMC Public Health
A literature review by van Panhuis et al. describing a taxonomy of barriers and solutions to sharing data for health research.

h. Lessons Learned in Using Hospital Discharge Data for State and National Public Health Surveillance - Journal of Public Health Management and Practice
A case study from Love, Rudolph, and Shah of the strengths and barriers to sharing health data in a collaboration between federal and state government.

i. The Law, Politics and Ethics of Cell Phone Data Analytics - Data Pop Alliance
A framework, along with a discussion of the legal, political, and ethical implications, for sharing mobile data for the social good.

j. Legal Barriers for Sharing Data:

  1. Navigating Legal Parameters for Cross-Sector Data Collaboration - National Center for Complex Health and Social Needs
    A brief on the legal issues of sharing data in the health sector that yields surprisingly generalizable lessons.

  2. Legal Issues for IDS Use: Finding a Way Forward - Actionable Intelligence for Social Policy
    A report that outlines some findings from an expert panel on setting up a cross-agency integrated data system (IDS), including common concerns, tools and tips, specific laws that could create legal issues, and potential legal instruments that could be used as solutions.

  3. Legal Guide to Administrative Data Sharing for Economic and Workforce Development - State Data Sharing Initiative)
    A report that outlines common legal issues and potential processes to solve them in the context of sharing data for economic and workforce development.

  4. Data Sharing Resources - State Data Sharing Initiative
    A listing of the states that have passed data-sharing legislation and set up data management bodies and MOUs to address legal uncertainties.

k. Organizational Barriers for Sharing Data:

  1. Becoming a Knowledge-Sharing Organization - World Bank Group
    A helpful guide to navigating organizational barriers. Begin at page 12 of handbook, at section "Why Are Leadership and a Knowledge-Sharing Culture Important?

  2. Building Trust for Cross-Sector Data Collaboration - National Center for Complex Health and Social Needs
    A report on the importance of building trust when sharing healthcare data that revolves around six core strategies.

  3. Access to New Data Sources for Statistics: Business Models and Incentives for the Corporate Sector - OECD Statistics Working Papers
    A discussion from Klein and Verhulst of the risks of and incentives for sharing corporate data; begin at page 11.

  4. What organizations need in order to share more data - Open Data Institute
    A summary of insights from interviewing different actors in various sectors about the barriers and opportunities of sharing data.

  5. Data Donation: Sharing Personal Data for Public Good - Digital Economy All Hands Meeting)
    A conference paper by Skatova, Goulding, and ng that presents two case studies of sharing health data for research and finds that organizations that share data for self-benefit act very differently than those who do so prosocially. Public perception of the receiving research organization was also found to be an important factor.

  6. Incentivizing Data Donation Nature Biotechnology
    An article on the importance of creating incentives to get people to share their health data and some of the current problems with this data-sharing landscape; highlights an organization that has emerged to try to address those problems.

  7. Team Building - World Health Organization
    A guide to building teams and preventing failure. See point 10 on page 11, "Why teams fail?"

  8. Gaining Executive-Level Buy-In for Data Governance Strategy - AIM Consulting
    A brief blog post on how to get executive sponsorship for your data-governance strategies and what to do when you get it.

  9. Data Collaboratives as "Bazaars"? Transforming Government: People, Process and Policy
    A conceptual framework from Susha, Janssen, and Verhulst for thinking about collaboration.

  10. The Right Data Governance for Your Organization's Culture - Dataversity
    A discussion of what organizational culture means and why it matters for data governance.

l. Ethical Barriers to Sharing Data:

  1. The Ethics of Data Sharing - Accenture
    A white paper that covers some of the arguments, concerns, best practices, models for governance, and a framework for ethical data sharing. Includes a sample risk assessment framework for data sharing and sample timeline.

1.3 Understand Data-Sharing Capacity

1. Key Resources and Digital Tools to Utilize:

a. Have you taken stock of your data-sharing capacity?

  1. Technology for Civic Data Integration - MetroLab Network
    A report that talks through the technical aspects of managing data sharing within the context of setting up a cross-agency integrated data system (IDS). For a discussion of the technical barriers and capacity for data sharing for social impact, see page 7.

b. Have technical barriers to data sharing been identified?

  1. 5 Models for Data Stewardship - SAS Best Practices
    A resource to help you think through the different kinds of data stewards, the problems with each, and the barriers to creating a data steward role.

c. Have data literacy programs or processes been created?

  1. We Are the Data Literacy Project - Data Literacy Project A collaborative effort to help society become more data literate by encouraging leading organizations to provide data literacy training, creating a library of resource-based learning for data literacy, and supporting global action should be employed to protect personal data?

  2. Data Literacy Program - Qlik
    An online resource for data literacy for what might the recovery process look like?

d. Who will receive the shared data? In what form will the data be shared?

2. Additional Resources:

a. Online Courses - School of Data
A collection of online courses on data science available to the public.

b. Technology for Civic Data Integration
educational institutions to place data literacy into their curricula.

c. College & University Data Science Degrees - DataScience.Community
A listing of all data science academic programs in the U.S.

1.4 Understand the Data Being Shared

1. Key Resources and Digital Tools to Utilize:

a. What is the quality of the data?

  1. Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries - Microsoft
    A remarkably complete listing of potential quality issues of social data.

b. Have appropriate data standards been identified?

  1. Five Principles for Applying Data Science for Social Good - O'Reilly
    A blog post on best practice principles for using big data for the social good, written by Jake Porway from DataKind. The first principle discusses how the way that organizations have published data might not be all that useful.

2. Additional Resources:

a. Connecting the Dots: A Data Sharing Framework for the Local Public Health System - National Association of County & City Health Officials
A guide for data sharing for local public health. See page 3, table 2: "Elements of a Data Management Plan."

b. Interoperability and Data Sharing - OPRE
A portfolio that discusses the benefits of collaboration and interoperability.

1.5 Survey Security and Privacy Related to the Data

1. Key Resources and Digital Tools to Utilize:

a. How sensitive are the data?

  1. Technology for Civic Data Integration - MetroLab Network
    This document explores many of the technical aspects of data sharing, with a discussion of security and privacy in this context starting on page 10.

  2. NNIP's Resource Guide to Data Governance and Security - National Neighborhood Indicators Partnership
    A resource guide on responsible data governance practices; see pages 18--30 for a discussion of good data security practices.

  3. The "Five Safes": a framework for planning, designing and evaluating data access solutions - Zenodo
    A paper by Felix Ritchie that dives into strategies for ensuring your data access and management approaches allow for security.

b. How is data encryption being managed through collection, transit, and access?

  1. Data Protection: Data In Transit vs. Data At Rest - Digital Guardian
    Web resource that provides a basic introduction to the different considerations and barriers to securing data while in transit or at rest.

  2. Data Collaboration for the Common Good - World Economic Forum
    See appendix on pages 26--27 for a discussion of centralized versus decentralized (federated) technology solutions to data sharing.

  3. Open Up Corporate Data While Protecting Privacy - Omidyar Network
    A blog post by Verhulst and Sangokoya on some of the types of data-sharing partnerships and some of the risks that corporate data-sharing schemes run.

c. What are the likely feelings of the relevant communities towards privacy?

  1. POTs: Productive Optimization Technologies - Cornell University
    A study that provides a solution to the privacy concerns of optimization technology through a framework that allows individuals to protect against unwanted consequences.

d. What forms of anonymization?

  1. Enabling Data Discoverability, Linkage, and Re-Use - National Academy of Sciences)
    A workshop summary that discusses how to frame relationships of paramount importance. Rephrasing "data should not be used for research or linked unless it can be done safely and securely" into "data should be available for research and linking unless it cannot be done safely and securely" can significantly increase support.

2. Additional Resources:

a. Sharing Data for Better Results - National League of Cities
A summary of major federal laws surrounding education, health, social services, and crime data: see pages 16--47.

b. The "Five Safes": A framework for planning, designing and evaluating data access solutions - Zenodo
A paper by Felix Ritchie that dives into strategies for ensuring your data access and management approaches allow for security.

c. Nothing to Hide: Tools for Talking and Listening about Data Privacy for Integrated Data Systems - Actionable Intelligence for Social Policy
A report that talks about data privacy within the context of setting up an intra-governmental IDS system.

d. Minimizing Disclosure Risk In HHS Open Data Initiatives. C.The Mosaic Effect - U.S. Department of Health & Human Services
A discussion of how seemingly unrelated and "non-sensitive" bits of data can be recombined in ways that generate sensitive data to form the mosaic effect.

e. Are Data Sharing and Privacy Protection Mutually Exclusive? - Cell
An article by Joly, Dyke, Knoppers, and Pastinen on trade-offs between privacy and data sharing in the context of health data.

f. How to Use SFTP to Securely Transfer Files with a Remote Server - Digital Ocean
A summary of SFTP along with how to best implement it.

g. The Legacy of Inbloom - Data Society
A case study on the story of educational technology in New York through Inbloom.

1.6 Examine the Ethical Implications of Sharing the Data

1. Key Resources and Digital Tools to Utilize:

a. Who is impacted and what ethical considerations does this population or its data pose?

  1. Reimagining Measurement: Enhancing Social Impact through Better Monitoring, Evaluation, and Learning - Deloitte
    A discussion of how to re-imagine your approach to focus on measures that reflect serving the community of interest and not just your funder's organizational goals.

  2. Why We Need Community-Driven Tech Ethics: Two Years of the Global Data Ethics Project - Tech at Bloomberg
    A summary of the growing movement of data ethics and the Global Data Ethics along with the importance of community involvement.

b. Has the urgency of the problem been weighed against the potential harm of the solution?

c. How can the community be engaged from the very early stages?

  1. Community Advisory Boards - West Virginia Clinical & Translational Science Institute
    A brief presentation of what a CAB is and when it makes sense to create and use one.

2. Additional Resources:

a. Data Collaboration for the Common Good - World Economic Forum
Helpful background on public private partnerships. See page 8 for information on the issue.

b. Machine Bias: Investigating Algorithmic Injustice - ProPublica
A collection of stories investigating the ways in which algorithms negatively affect society by perpetuating injustice and bias.

c. How Big Data Is Automating Inequality - New York Times Review of a book by Virginia Eubanks that argues that instead of technology increasing the efficiency of government services it actually increases the inequality of said services.

d. Weapons of Math Destruction: Cathy O'Neil - Adds Up the Damage of Algorithms - The Guardian An interview where Cathy O'Neil discusses her book about how algorithms can cause more damage to society than we think.

e. Civil Rights, Big Data, and Our Algorithmic Future: A Report on Scoial Jusice and Technology - Upturn
A report on the social justice and bias issues of modern technology.

f. Reforming the U.S. Approach to Data Protection and Privacy - Council on Foreign Relations)
A brief overview of why the current approach to regulating privacy and consumer data in the U.S. does not create meaningful protection for individuals.

g. Ethics Washing Is When Ethics is Substitute for Regulation - DataEthics
Discussion of the concept of "ethics washing".

h. The Seductive Power of "Solving" Bias in Artificial Intelligence - OneZero
A discussion of the effects of bias in AI, the best solutions to these problems, and the implications of the data associated with these technologies.

i. Oxfam Responsible Data Program Policy - Oxfam
A Phase One exemplar that lists data rights for users.

1.7 Draft your Charter

1. Key Resources and Digital Tools to Utilize:

a. Creating a Charter for Your Policy Team - National Institute of Corrections
A roadmap that helps policy teams think through and draft a charter.

b. Project Charter Template - Centers for Disease Control and Prevention
A template for a team charter.

c. Generic Team Charter Template - Acquire.gov
A template for a team charter (warning - direct download from Wayback Machine).

d. Acquisition Team Charter Template - GA Tech
A template for a team charter.

e. MITA Governance Charter - Medicaid
A template for a team governance charter.

2. Helpful Questions to Consider:

a. What is the purpose of your collective? What is the context and background of the collective and impacted populations and stakeholders? What are their values and motivations?

b. Have the most salient legal, security, privacy, organizational, and ethical considerations been acknowledged?

c. What is the scope of the project and the goals of the collective?

d. Who are the members of the collective, their roles, and points of expertise?

e. Is there an authority or a way to confer authority on specific actors to ensure effective operations?

f. What are the operational plan, decision-making mechanisms, membership change procedures, rule-making processes?

g. How will performance be assessed?

h. What timeline and milestones are in place to guide performance assessments and how long will the collective operate?

Phase Two: Define the Operations

2.1 Determine Your Governance Framework Structure

1. Key Resources and Digital Tools to Utilize:

a. What models for data sharing support your work?

  1. Mapping the Next Frontier of Open Data: Corporate Data Sharing - Medium
    An essay that describes the current trend in corporate data sharing for social impact and introduces GovLab's taxonomy of types of data-sharing agreements.

b. Which model best fits your needs?

  1. An Introduction to Data Collaboratives - GovLab
    A brief overview of the different ways to organize data collaboratives.

  2. [The Emergence of Data Collaboratives in Numbers - GovLab
    A blog post that details examples of each of the kinds of data collaboratives according to GovLab.

  3. Structuring a Partnership - William T. Grant Foundation
    A (still sparse) platform with guiding questions, examples, and resources for structuring your partnership.

c. How will the public and affected communities be informed and engaged when determining the data-sharing governance process?

d. Who will receive the shared data, and in what form?

  1. Data Maturity Framework Questionnaire - University of Chicago A checklist of qualities for data teams and assets that you can use to evaluate the maturity of your data from the Center for Data Science & Public Policy.

2. Additional Resources:

a. The Emergence of Data Collaboratives...in Numbers - GovLab
A blog post that describes the most common sectors for GovLab's taxonomy of data-sharing agreements.

b. How to Scale Data Collaboratives? Key Takeaways from the London Data Stewards Camp - Medium
A blog post that describes a meeting of the Data Stewards Network.

c. Data Collaborative Explorer - GovLab
A listing of all data collaboratives GovLab has found, paired with a brief description, a link to the source, and a classification (that is, which type of data collaborative it is in GovLab's taxonomy).

d. Data Philanthropy: Unlocking the Power of Private Data for Public Good - Urban Institute
A report describing corporate data-sharing for the social good as a new form of philanthropy, offering up different pathways to creating data-sharing arrangements depending on the sensitivity of the data, and describing two use cases.

e. AISP Network Site Case Studies - Actionable Intelligence for Social Policy
A collection of case studies of local and state government-created integrated data systems (IDS): when different agencies, sometimes in partnership with private research entities, share information to form a richer database that allows for better service delivery.

2.2 Formalize Responsible Data-Sharing Practices

1. Key Resources and Digital Tools to Utilize

a. Have the roles and responsibilities of each party been established?

  1. Developing Data-Sharing Agreement - William T. Grant Foundation
    A (still sparse) platform with guiding questions, examples, and resources for developing a data-sharing agreement.

b. Has feedback from stakeholders been solicited and adapted throughout the defining process? Have rules around modifying the stakeholder mix of the collective been drafted?

c. How are data used externally beyond the collective?

  1. Governance International Association for Impact Assessment
    A tip-sheet that briefly describes what to consider and what to do when designing good impact assessment governance.

  2. Recommendations of the Council on Principles for Public Governance of Public-Private Partnerships - Organisation for Economic Co-operation and Development
    A report that gives recommendations on how to design a value-focused institutional framework for effective public-private collaboration.

2. Additional Resources

a. A Decision Model for Data Sharing - 13th International Conference on Electronic Government
A literature review from a conference paper by Eckartz, Hofman, and Veenstra on data-sharing governance practices that includes three brief case studies in the logistics sector and a data-sharing decision model to help actors identify and address barriers to sharing.

b. Mastercard Establishes Principles for Data Responsibility - Mastercard
A press release outlining the company's proposed six data responsibilities (security & privacy, transparency & control, accountability, integrity, innovation, and social impact) for the organization and the digital economy.

2.3 Examine the Sustainability of the Operations

1. Key Resources and Digital Tools to Utilize:

a. How is the collective funded? For what period of time will this collective exist?

  1. Data Collaboration for the Common Good - World Economic Forum
    A resource on public private partnerships. See section 5, page 21: "Economic sustainability and scalability."

  2. 7 Tools to Help Build Sustainable Data Governance - Earley Information Science
    A toolkit with guides and resources on how to build a sustainable data governance model.

2. Additional Resources:

a. Guide to Data Science and Sustainability - Discover Data Science
An extensive guide from DDS to incorporating sustainability into data science.

2.4 Develop an Ethical Framework for Data Sharing

1. Key Resources and Digital Tools to Utilize:

a. How will data governance ensure that ethical concerns are considered, heard, and addressed throughout the project?

  1. Digital Decisions Tool - Center for Democracy & Technology
    An interactive tool that asks a series of ethics and fairness-centric questions along the process of building your data-sharing algorithm.

  2. Building Frameworks, Setting Standards for Ethical Data Use: Our Conversation with Natalie Evans Harris - Medium
    An interview on the importance of community-focused data sharing through ethical standards, educational suggestions on sharing data, and comment on the state of data sharing in society.

b. How are practitioners thinking about data ethics?

  1. Oaths, Pledges and Manifestos: a Master List of Ethical Tech Values - Medium
    An overview of ethical initiatives in technology.

  2. Tech's Ethical "Dark Side": Harvard, Stanford and Others Want to Address It - New York Times
    A summary of certain ethical issues within the tech industry and proposed solutions.

c. How are practitioners acting on principles of data ethics?

  1. Managing Data Ethics: A Process-Based Approach for CDOs - Deloitte Insights
    A summary of past and present practices in managing data ethics along with summaries of concepts and skills for CDOs in the field of data ethics.

2.Additional Resources:

a. Tech Ethics Curricula
An open spreadsheet listing most tech ethics courses available in the U.S.

b. Data Science Ethics - Coursera
A University of Michigan course on Data Ethics that is open to the public.

c. Ethics and Law in Data and Analytics - EdX
A course from Microsoft on Date Ethics that is open to the public.

d. Data Ethics Course - National Forum on Education Statistics
A course on data ethics that is open to the public.

e. Data Ethics: The New Competitive Advantage - TechCrunch
An article that describes how data ethics can be used by companies for competitive advantage in an era where consumers value companies which behave ethically with their data.

f. Data Practices Courseware - Linux Foundation Project
A repository of workshops that non-technical people can take to help them understand the basics of good data practices.

g. Ethics and Data Science - O'Reilly
A booklet from Patil, Mason, and Loukides that gives a solid introduction to the concept of data ethics and the urgency with which it needs to be applied to all facets of data science.

h. MS and E 234 - Stanford
A course that engages with the difficult ethical challenges in the modern practice of data science.

i. Association for Computing Machinery Code of Ethics and Professional Conduct
An annotated code of ethics for computing professionals that outlines general principles, enumerates responsibilities, and offers advice for leadership.

j. Ten Simple Rules for Responsible Big Data Sharing
An annotated 10-point framework of key data ethics principles.

k. American Statistical Association's Ethical Guidelines for StatisticalPractice
A detailed list of ethical principles aimed at statistical professionals centered around promoting integrity and highlighting responsibilities.

l. Data Science Oath of National Academies of Science, Engineering, and Medicine
An actual oath that data practitioners can take, situated alongside the Hippocratic oath.

m. Manifesto for Data Practices of data.world
A simple 12-point list of ethical principles centered around four values: inclusion, experimentation, accountability, and impact.

2.5 Define What Success Looks Like

1. Key Resources and Digital Tools to Utilize:

a. What impact metrics are appropriate?

  1. Reimagining Measurement: Enhancing Social Impact through Better Monitoring, Evaluation, and Learning - Deloitte
    A white paper discussing the importance and difficulties in creating good impact evaluations.

  2. Toward Metrics for (Re)Imagining Governance - GovLab
    A white paper discussing the use of metrics in evaluating interventions and governance innovations.

  3. Assessing Significance in Impact Assessment of Projects - International Association for Impact Assessment
    A tip sheet that defines and provides guidelines for measuring the scale/significance of social impact for specific projects.

b. How are data used externally beyond the collective?

2. Additional Resources:

a. International Principles for Social Impact Assessment - Impact Assessment and Project Appraisal
An article that defines social impact measurement and provides guidelines and values for it.

b. Situating the Next Generation of Impact Measurement and Evaluation for Impact Investing - IssueLab
A white paper promotes a convergence of methods, building from both the impact investment and evaluation fields.

c. Policy Brief on Social Impact Measurement for Social Enterprises - Organisation for Economic Co-operation and Development
A white paper that seeks to define exactly what social impact is.

2.6 Draft Data-Sharing Agreement

1. Key Resources and Digital Tools to Utilize:

a. NNIP Concept - National Neighborhood Indicators Partnership
A repository of example documents, including data-sharing agreements, codification of various security and privacy procedures, and confidentiality agreements.

b. Data-Sharing: Creating Agreements in Support of Community-Academic Partnerships - Colorado Clinical and Translational Sciences Institute
A report that outlines best practices, processes, recommendations, and a set of resources to help stakeholders navigate data sharing, data management plans, and data-sharing agreements in the context of community-academic partnerships.

c. Guidelines for Developing Data Sharing Agreements to Use State Administrative Data for Early Care and Education Research - U.S. Department of Health & Human Services
A report that walks readers through the creation of data-sharing agreements that seek to repurpose administrative data from state agencies for health-sector research.

d. Introducing Contracts for Collaboration: New Project on Legal Conditions for Data Sharing - Thematic Research Network on Data and Statistics
News article that previews an upcoming project that seeks to create an online library of data-sharing agreements accessible to those seeking to draft their own.

e. Data Trust Agreement Template - BrightHive
A sample data trust agreement.

f. Putting the Trust in Data Trusts - Register Dynamics
A summary of the benefits of data trusts along with sample case studies.

2. Helpful Questions to Consider:

a. What data will be used, how will they be shared and used?

b. Which stakeholders are involved in what part of the data collection, processing, and analysis?

c. What security and ethical safeguards have been put in place?

d. How are collective-wide decisions made and what process do these decisions go through?

e. When are confidentiality agreements required and how are they drafted?

f. How are ethical and security reviews and audits conducted?

g. How are external requests for data or insights including review, approval, and pricing handled?

h. Who contributes what resources to the project?

i. Who owns the intellectual property of the data and of the resulting publications and who decides whether to publish or share them externally?

j. Who gets credit for what part of the collective?

k. How are potentially ambiguous terms that might interfere with mutual understanding of the contract defined, especially if the partners are from different sectors, been clearly defined?

l. How are changes that members want addressed?

Phase Three: Drive Impact

3.1 Reinforce, Update, and Share Governance Best Practices

1.Key Resources and Digital Tools to Utilize:

a. How can you ensure that parties continue to adhere to previously established standards and practices?

  1. Effective Stakeholder Engagement - International Association for Impact Assessment
    A quick summary of tips and advice on stakeholder engagement from planning to things to know and things to do.

b. How can data-sharing agreements be changed based on lessons you learn from evaluating progress on each step of the theory of change?

  1. Rapid Evaluation Approaches for Complex Initiatives - ASPE
    This paper builds a framework for how to evaluate different projects based on their levels of complexities.

c. How do you decide on and take on new research projects? 1.

d. How can iterative improvements best be documented and publicized to stay transparent, help other organizations learn from mistakes and failures, and enrich the literature to promote data-sharing for social impact elsewhere?

  1. Iterative Development - Code for America
    A discussion of what iterative development means along with a list of tools and resources.

e. How do we continue to minimize and assess risk as we continue working?

2. Additional Resources:

a. Public Participation: International Best Practice Principles - International Association for Impact Assessment
A brief guide for how to integrate public participation into your impact assessment including a summary of objectives and best practices.

b. Sharing Results: A Guide for Communicating Promise Neighborhoods Outcomes to Diverse Audiences - Urban Institute
A paper that walks you through how to properly frame and convey your results to a variety of audiences.

c. Data Walks: An Innovative Way to Share Data with Communities - Urban Institute
A report that helps researchers design ways to share data and findings with community residents and program participants that ensure better interpretation and dissemination of the results and inspire action within the community of interest.

d. Nothing to Hide: Tools for Talking (and Listening) About Data - Privacy for Integrated Data Systems - Actionable Intelligence for Social Policy
A report that talks about data privacy within the context of setting up an intra-governmental IDS system.

e. IDS Case Study: Los Angeles - County Actionable Intelligence for Social Policy
A report on IDS organization in Los Angeles describes how a governing body meets annually to turn research ideas into a research agenda, where agency staff can provide the county board with data sources and information that creates context around the county's initiatives.

f. The Role of Case Studies in Effective Data Sharing, Reuse and Impact Quarterly - IASSIST
A discussion about the importance of case studies in encouraging data sharing, noting that there is currently a dearth of well-documented studies in this area that can help practitioners work through this process.

g. Cross-Sector Data in Action (National Center for Complex Health and Social Needs
A discussion of a few case studies on the topic of cross-sector data sharing.

3.2 Reinforce, Update, and Share Best Practices in Ethics Governance

1. Key Resources and Digital Tools to Utilize:

a. Who are the affected communities?

  1. Closing the Feedback Loop - SSIR
    A program that gives advice on community engagement.

  2. Community Engagement Toolkit - Collective Impact Forum
    A tool kit to effectively engage the community for data-driven solutions.

b. Are there new externalities and have previously identified ones been properly addressed? Has technology changed in a way that increases the potential for data misuse? Can fail-safes be implemented to prevent misuse of data?

c. Is conducting an equity audit appropriate?

Additional Resources:

a. Ethics and Data Science - O'Reilly
A booklet by Loukides, Mason, and Patil on data ethics and how to incorporate these standards into your everyday work life.

3.3 Monitor and Assess Privacy and Security Approaches

1. Key Resources and Digital Tools to Utilize:

a. Has the structure of the collective changed? Has the technological environment changed? Has the regulatory environment changed? Has the organizational culture of any stakeholder organization changed? Has society changed in ways that affect the impacted communities?

  1. CISO Tips: How CISOs Can Reduce Risk - CIO
    An article providing a framework for CISOs of organizations to tackle the evolving technological and privacy environment with five quick tips.

b. Are there system processes that can be put in place to minimize or resolve human errors?

  1. Considerations for Using Data Responsibly at USAID - USAID
    A Phase Three exemplar that shows how to best assess risk.

2. Additional Resources:

a. Advice for CISOs: Want More Resources? Think Beyond Pure Tech - Symantec
A guide for how CISOs can increase security budgets and further tackle privacy and security concerns.

3.4 Develop Processes to Improve Data Quality Over Time

1. Key Resources and Digital Tools to Utilize:

a. 5 Hands-On Strategies to Improve Data Quality - Ringlead
A quick tips guide on five strategies for improving data quality over time.

b. What is Metadata and Why is it as Important as the Data Itself? - opendatasoft
This article provides an overview of metadata and how to best use it to our advantage.

c. Scoping Fast Tips
A definition and discussion of what scoping means.

2. Helpful Questions to Consider:

a. What metadata can be collected to make the data more useful?

b. How can the documentation process be improved and can insight about potential bias be baked in?

c. How can capacity be built over time, whether technical infrastructure or data management expertise?

d. How has the situation or your understanding of the situation changed in a way that changes your idea of effective impact metrics? Can it be changed to improve benchmarking and comparisons with other efforts?

Data Sharing Today

Resources

  1. Understanding Corporate Data Sharing Decisions - Future of Privacy Forum
    This report covers the Future of Privacy Forum's research on how companies make data available, why companies share data, and the risks perceived with sharing.

  2. The U.S. Data Federation Wants to Make It Easier to Collect, Combine, and Exchange Data across Government - General Services Administration
    A paper from 18F discussing the potential new ease of data sharing and exchange across government through the GSA Technology Transformation Systems.

  3. AISP Network Site Case Studies - Actionable Intelligence for Social Policy
    A group of case studies on securing and maintaining legal agreements, establishing and adapt governance processes, managing data and analytic processes, and confronting economic and political realities in sustaining Integrated Data Systems (IDS) operations, documenting the group's own exemplary research uses.

  4. Sharing Data is a Form of Corporate Social Responsibility - Harvard Business Review)
    An article outlining a new thought process for public-private data sharing.

  5. Personal Data for the Public Good: New Opportunities to Enrich Understanding of Individual and Population Health - Health Data Exploration Project
    A report on enhancing how to generate knowledge and outcomes from health data concentrating on the opportunities of privacy and data ownership, informed consent, data sharing and access and data quality.

  6. Clinical Trial Participants' Views of the Risks and Benefits of Data Sharing - New England Journal of Medicine
    In this article, Mello et al. track opinions on data sharing, concluding that few trial participants had fears about sharing their information.

  7. It's Time for Data Ethics Conversations at Your Dinner Table
    An opinion piece that emphasizes the importance of having conversations on data.

  8. Data Driven Social Partnerships: Exploring an Emergent Trend in Search of Research Challenges and Questions - Government Information Quarterly
    In this article, Susha, Grönlund, and VanTulder emphasize the benefit of data collection for social impact and the common good.

  9. The New Ecosystem of Trust - Nesta
    This blog post builds on a series of Nesta think pieces on data and knowledge commons published over the last decade and current practical projects that explore how data can be mobilised to improve healthcare, policing, the jobs market, and education.

  10. Americans and Privacy: Concerned, Confused and Feeling Lack of Control Over Their Personal Information - Pew Research Center
    An article summarizing recent research that discusses Americans' opinions about the privacy of data, their lack of control over privacy, and their overall feelings about data sharing and collection.

  11. Data Governance Colorado Department of Education
    A Phase Three exemplar of how to create a governance board for a community

  12. Pathways to Grade-Level Reading - North Carolina Early Childhood Foundation
    A Phase Three exemplar that emphasizes driving impact through data collection (and subsequent action).

  13. Supporting Community Data Engagement: An NCVHS Roundtable - National Committee on Vital and Health Statistics
    Phase Three exemplar that held a roundtable on effectively engaging the community

  14. Empowering Families Project (Children's Services Council of Broward County
    A Phase Three exemplar of a family-centered project to improve educational outcomes through an integrated data repository that included important items such as a community feedback loop to continually improve processes.