diff --git a/public/Upset-Alttext-User-Survey/assets/T1.png b/public/Upset-Alttext-User-Survey/assets/T1.png index 690af8b01..9944732e9 100644 Binary files a/public/Upset-Alttext-User-Survey/assets/T1.png and b/public/Upset-Alttext-User-Survey/assets/T1.png differ diff --git a/public/Upset-Alttext-User-Survey/assets/consent.md b/public/Upset-Alttext-User-Survey/assets/consent.md index 59f034860..867f83c34 100644 --- a/public/Upset-Alttext-User-Survey/assets/consent.md +++ b/public/Upset-Alttext-User-Survey/assets/consent.md @@ -3,10 +3,10 @@ Before we begin, please read the following consent form carefully and thereafter complete the section below. ## Purpose of the Study -Taking valuable feedback and assessment from sighted community is crucial as a forst evaluation step for our generated text descriptions. The sole purpose of the study is to evaluate our text description for UpSet plots. +Taking valuable feedback and assessment from the sighted community is crucial as a first evaluation step for our generated text descriptions. The sole purpose of the study is to evaluate our text description for UpSet plots. ## Tasks -The study contains three visualization contents: Visualization only (a UpSet plot), Text only (text description of a UpSet plot), Visualization and Text (a UpSet plot and text description of that plot). For each visualization content, we ask you 12 questions regarding understanding the content. The questions contain various types such as single-choice, checkbox, long text, likert scale, etc. After answering the questions for each content, the participant will take part in a post study survey. Each participant should take approximately 20-25 minutes to complete the questionnaire. +The study contains three visualization contents: Visualization only (an UpSet plot), Text only (text description of an UpSet plot), Visualization and Text (an UpSet plot and text description of that plot). For each visualization content, we ask you 12 questions regarding understanding the content. The questions contain various types such as single-choice, checkbox, long text, likert scale, etc. After answering the questions for each content, the participant will take part in a post-study survey. Each participant should take approximately 20-25 minutes to complete the questionnaire. Each participant in this study is assigned a random ID for analyses. As such, your participation will remain anonymous, and your responses will not be used to identify you. diff --git a/public/Upset-Alttext-User-Survey/assets/introduction.md b/public/Upset-Alttext-User-Survey/assets/introduction.md index 0aab81f9b..fa5556d63 100644 --- a/public/Upset-Alttext-User-Survey/assets/introduction.md +++ b/public/Upset-Alttext-User-Survey/assets/introduction.md @@ -2,12 +2,14 @@ Welcome to our study! -In this study, you will encounter questions about **data visualizations and equivalent text describing the content** of data visualizations. We will use UpSet plots, a set visualization technique similar to Venn diagrams as an example. +In this study, you will encounter questions about **data visualizations and equivalent text describing the content** of data visualizations. We will use UpSet plots, a set visualization technique similar to Venn diagrams, as an example. -Our research aims to **make data visualizations more accessible to people with visual impairments** such as low vision, residual vision, or no vision. We want to understand whether text can convey equal amounts of informations as a chart. +Our research aims to **make data visualizations more accessible to people with visual impairments** such as low vision, residual vision, or no vision. We want to understand whether text can convey equal amounts of information as a chart. We first will introduce what an UpSet plot is, why is it used, how you can interpret an UpSet plot. -To ensure you are comfortable with the actual question-answer session, there will be a training section where you will have up to **three attempts** to answer the questions correctly. If you do no pass the training on your third try, **you can not proceed to the study** and **your answer will not be accepted**. In that case, **we will not be able to pay you** and **you will be automatically redirected to the finish page**. \ No newline at end of file +> +**⚠️ WARNING:** +To ensure you have engaged with the tutorial, there will be a training section where you will have up to **three attempts** to answer the questions correctly. If you do not pass the training on your third try, **you will not be able to proceed to the study** and **your answers will not be accepted**. In that case, **we will not be able to pay you**. \ No newline at end of file diff --git a/public/Upset-Alttext-User-Survey/assets/training1.md b/public/Upset-Alttext-User-Survey/assets/training1.md index 23bc72063..6b54296a0 100644 --- a/public/Upset-Alttext-User-Survey/assets/training1.md +++ b/public/Upset-Alttext-User-Survey/assets/training1.md @@ -23,7 +23,7 @@ Here you see an example for "Visualization Only". In the sidebar, you can see so This UpSet plot shows movies and their genres. A movie can have multiple genres. The sets are movie genres. The items are movies. The intersections show how many movies have exactly the same genres.
- Movie Genres + Movie Genres
diff --git a/public/Upset-Alttext-User-Survey/assets/upset.md b/public/Upset-Alttext-User-Survey/assets/upset.md index 76f5c3ebf..ed7680801 100644 --- a/public/Upset-Alttext-User-Survey/assets/upset.md +++ b/public/Upset-Alttext-User-Survey/assets/upset.md @@ -2,7 +2,7 @@ # What Is UpSet? -UpSet visualizes set data, as it would be usually displayed in a Venn diagram. However, unlike Venn diagrams, UpSet works for more than three sets. +UpSet visualizes set data, as it would usually be displayed in a Venn diagram. However, unlike Venn diagrams, UpSet works for more than three sets. ## UpSet Explained @@ -13,7 +13,7 @@ UpSet plots the intersections of a set as a matrix, as shown in the following fi
-Below you can see examples of how these intersections correspond to the segments in a Venn diagram. The first row in the figure is completely empty – it corresponds to all the elements that are in none of the sets. The green (third) row corresponds to the elements that are only in set B, (not in A or C). The orange (fifth) row represents elements that are shared by sets A and B, but not with C. Finally, the last (violet) row represents the elements shared between all sets. +Below, you can see examples of how these intersections correspond to the segments in a Venn diagram. The first row in the figure is completely empty – it corresponds to all the elements that are in none of the sets. The green (third) row corresponds to the elements that are only in set B, (not in A or C). The orange (fifth) row represents elements that are shared by sets A and B, but not with C. Finally, the last (violet) row represents the elements shared between all sets. Explaining the intersections in UpSet @@ -31,7 +31,7 @@ The matrix is also very useful because it can be sorted in various ways. A commo Sorting by cardinality in UpSet -These are the basiscs of UpSet! Next, we'll look at an example like you will see in the study. +These are the basics of UpSet! Next, we'll look at an example like the ones you will see in the study. ## Interpreting UpSet Plots @@ -41,20 +41,20 @@ The following UpSet plot shows movie genres as intersecting sets.
### Sets and Set Sizes: -In this UpSet plot the genres Drama, Comedy, Thrillar, Crime, and Fantasy are the sets. The bars on top indicate that Drama is a large set, but Fantasy is small. +In this UpSet plot, the genres Drama, Comedy, Thriller, Crime, and Fantasy are the sets. The bars on top indicate that Drama is a large set, but Fantasy is small. -In the study, **you will be asked to classify this distribution** using your judgement. Are they: +In the study, **you will be asked to classify this distribution** using your judgment. Are they: * Roughly equal, * Diverging a bit, or * Diverging a lot. - For this example the correct answer is "Diverging a lot". + For this example, the correct answer is "Diverging a lot". ### Intersection and Intersection Sizes: In this UpSet plot, there are 20 intersections that are made up of different sets. -You will be asked about the make-up of these intersection. The categories are +You will be asked about the make-up of these intersections. The categories are * an intersection is made up of **no set**; as shown in the third row with size 854 here (movies that don't have any of the genres shown). * an intersection can only contain a **single set**, like the largest one (Drama, with 1198 movies) * an intersection can contain **2-3 sets**, like the "Drama-Comedy" intersection with size 212. diff --git a/public/Upset-Alttext-User-Survey/config.json b/public/Upset-Alttext-User-Survey/config.json index e189e0471..86da934ef 100644 --- a/public/Upset-Alttext-User-Survey/config.json +++ b/public/Upset-Alttext-User-Survey/config.json @@ -1977,611 +1977,6 @@ "baseComponent": "TandV3", "path": "Upset-Alttext-User-Survey/assets/tennisTextandVis.md" }, - "GoT-Vis-Q3": { - "baseComponent": "VO-image1", - "path": "Upset-Alttext-User-Survey/assets/GoTVis.md" - }, - "GoT-Vis-Q1": { - "baseComponent": "VO-image2", - "path": "Upset-Alttext-User-Survey/assets/GoTVis.md", - "response": [ - { - "id": "voq1", - "prompt": "How many sets are shown in the upset plot?", - "required": true, - "location": "sidebar", - "type": "numerical", - "placeholder": "Please enter your answer here", - "min": 0, - "max": 100 - - }, - { - "id": "voq2", - "prompt": "What is the largest set?", - "required": true, - "location": "sidebar", - "type": "dropdown", - "placeholder": "Please select an answer", - "options": [ - { - "label": "Baratheon", - "value": "Baratheon" - }, - { - "label": "Lannister", - "value": "Lannister" - }, - { - "label": "Stark", - "value": "Stark" - }, - { - "label": "Targarayen", - "value": "Targarayen" - }, - { - - "label": "Female", - "value": "Female" - }, - { - "label": "Male", - "value": "Male" - }, - { - "label": "royal", - "value": "royal" - } - ] - }, - { - "id": "voq3", - "prompt": "What is the largest intersection?", - "secondaryText": "Select all Sets in the Intersection", - "required": true, - "location": "sidebar", - "type": "checkbox", - "options": [ - { - "label": "Baratheon", - "value": "Baratheon" - }, - { - "label": "Lannister", - "value": "Lannister" - }, - { - "label": "Stark", - "value": "Stark" - }, - { - "label": "Targarayen", - "value": "Targarayen" - }, - { - - "label": "Female", - "value": "Female" - }, - { - "label": "Male", - "value": "Male" - }, - { - "label": "royal", - "value": "royal" - }, - { - "label": "Empty Intersection (no sets)", - "value": "Empty Intersection" - } - ] - }, - { - "id": "voq4", - "prompt": "How large is the largest intersection?", - "required": true, - "location": "sidebar", - "type": "numerical", - "placeholder": "Please enter your answer here", - "min": 0, - "max": 1000000000 - }, - { - "id": "voq5", - "prompt": "How many sets make up the largest intersection?", - "required": true, - "location": "sidebar", - "type": "dropdown", - "placeholder": "Please choose your answer", - "options": [ - { - "label": "It is only of a single set", - "value": "It is only of a single set" - }, - { - "label": "It is the intersection of 2-3 sets", - "value": "It is the intersection of 2-3 sets" - }, - { - "label": "It is the intersection of many sets", - "value": "It is the intersection of many sets" - }, - { - "label": "It is the empty intersection (no sets)", - "value": "It is the empty intersection (no sets)" - } - - ] - }, - { - "id": "voq6", - "prompt": "How similar are the set sizes?", - "required": true, - "location": "sidebar", - "type": "dropdown", - "placeholder": "Please choose your answer", - "options": [ - { - "label": "Roughly equal (all the set sizes look similar)", - "value": "Roughly equal" - }, - { - "label": "Diverging a bit (the sizes of the largest set and the smallest set don't differ largely)", - "value": "Diverging a bit" - }, - { - "label": "Diverging a lot (the sizes of the largest set and the smallest set differ largely)", - "value": "Diverging a lot" - } - ] - }, - { - "id": "voq7", - "prompt": "Is the largest set present in the largest intersection?", - "required": true, - "location": "sidebar", - "type": "radio", - "options": [ - { - "label": "Yes", - "value": "Yes" - }, - { - "label": "No", - "value": "No" - } - ] - }, - { - "id": "voq8", - "prompt": "Is the all-set intersection (intersection having all the sets) present? ", - "required": true, - "location": "sidebar", - "type": "radio", - "options": [ - { - "label": "Yes", - "value": "Yes" - }, - { - "label": "No", - "value": "No" - } - - ] - } - ] - }, - "GoT-Vis-Q2": { - "baseComponent": "VO-image3", - "path": "Upset-Alttext-User-Survey/assets/GoTVis.md" - }, - "GoT-Text-Q3": { - "baseComponent": "TO-text1", - "path": "Upset-Alttext-User-Survey/assets/GoText.md" - }, - "GoT-Text-Q1": { - "baseComponent": "TO-text2", - "path": "Upset-Alttext-User-Survey/assets/GoText.md", - "response": [ - { - "id": "voq1", - "prompt": "How many sets are shown in the description?", - "required": true, - "location": "sidebar", - "type": "numerical", - "placeholder": "Please enter your answer here", - "min": 0, - "max": 100 - - }, - { - "id": "voq2", - "prompt": "What is the largest set?", - "required": true, - "location": "sidebar", - "type": "dropdown", - "placeholder": "Please select an answer", - "options": [ - { - "label": "Baratheon", - "value": "Baratheon" - }, - { - "label": "Lannister", - "value": "Lannister" - }, - { - "label": "Stark", - "value": "Stark" - }, - { - "label": "Targarayen", - "value": "Targarayen" - }, - { - - "label": "Female", - "value": "Female" - }, - { - "label": "Male", - "value": "Male" - }, - { - "label": "royal", - "value": "royal" - } - ] - - }, - { - "id": "voq3", - "prompt": "What is the largest intersection?", - "secondaryText": "Select all Sets in the Intersection", - "required": true, - "location": "sidebar", - "type": "checkbox", - "options": [ - { - "label": "Baratheon", - "value": "Baratheon" - }, - { - "label": "Lannister", - "value": "Lannister" - }, - { - "label": "Stark", - "value": "Stark" - }, - { - "label": "Targarayen", - "value": "Targarayen" - }, - { - - "label": "Female", - "value": "Female" - }, - { - "label": "Male", - "value": "Male" - }, - { - "label": "royal", - "value": "royal" - }, - { - "label": "Empty Intersection (no sets)", - "value": "Empty Intersection" - } - ] - }, - { - "id": "voq4", - "prompt": "How large is the largest intersection?", - "required": true, - "location": "sidebar", - "type": "numerical", - "placeholder": "Please enter your answer here", - "min": 0, - "max": 1000000000 - }, - { - "id": "voq5", - "prompt": "How many sets make up the largest intersection?", - "required": true, - "location": "sidebar", - "type": "dropdown", - "placeholder": "Please choose your answer", - "options": [ - { - "label": "It is only of a single set", - "value": "It is only of a single set" - }, - { - "label": "It is the intersection of 2-3 sets", - "value": "It is the intersection of 2-3 sets" - }, - { - "label": "It is the intersection of many sets", - "value": "It is the intersection of many sets" - }, - { - "label": "It is the empty intersection (no sets)", - "value": "It is the empty intersection (no sets)" - } - - ] - }, - { - "id": "voq6", - "prompt": "How similar are the set sizes?", - "required": true, - "location": "sidebar", - "type": "dropdown", - "placeholder": "Please choose your answer", - "options": [ - { - "label": "Roughly equal (all the set sizes look similar)", - "value": "Roughly equal" - }, - { - "label": "Diverging a bit (the sizes of the largest set and the smallest set don't differ largely)", - "value": "Diverging a bit" - }, - { - "label": "Diverging a lot (the sizes of the largest set and the smallest set differ largely)", - "value": "Diverging a lot" - } - ] - }, - { - "id": "voq7", - "prompt": "Is the largest set present in the largest intersection?", - "required": true, - "location": "sidebar", - "type": "radio", - "options": [ - { - "label": "Yes", - "value": "Yes" - }, - { - "label": "No", - "value": "No" - } - ] - }, - { - "id": "voq8", - "prompt": "Is the all-set intersection (intersection having all the sets) present? ", - "required": true, - "location": "sidebar", - "type": "radio", - "options": [ - { - "label": "Yes", - "value": "Yes" - }, - { - "label": "No", - "value": "No" - } - - ] - } - ] - - }, - "GoT-Text-Q2": { - "baseComponent": "TO-text3", - "path": "Upset-Alttext-User-Survey/assets/GoText.md" - }, - "GoT-Both-Q3": { - "baseComponent": "TandV1", - "path": "Upset-Alttext-User-Survey/assets/GoTextandVis.md" - }, - "GoT-Both-Q1": { - "baseComponent": "TandV2", - "path": "Upset-Alttext-User-Survey/assets/GoTextandVis.md", - "response": [ - { - "id": "voq1", - "prompt": "How many sets are there?", - "required": true, - "location": "sidebar", - "type": "numerical", - "placeholder": "Please enter your answer here", - "min": 0, - "max": 100 - - }, - { - "id": "voq2", - "prompt": "What is the largest set?", - "required": true, - "location": "sidebar", - "type": "dropdown", - "placeholder": "Please select an answer", - "options": [ - { - "label": "Baratheon", - "value": "Baratheon" - }, - { - "label": "Lannister", - "value": "Lannister" - }, - { - "label": "Stark", - "value": "Stark" - }, - { - "label": "Targarayen", - "value": "Targarayen" - }, - { - - "label": "Female", - "value": "Female" - }, - { - "label": "Male", - "value": "Male" - }, - { - "label": "royal", - "value": "royal" - } - ] - }, - { - "id": "voq3", - "prompt": "What is the largest intersection?", - "secondaryText": "Select all Sets in the Intersection", - "required": true, - "location": "sidebar", - "type": "checkbox", - "options": [ - { - "label": "Baratheon", - "value": "Baratheon" - }, - { - "label": "Lannister", - "value": "Lannister" - }, - { - "label": "Stark", - "value": "Stark" - }, - { - "label": "Targarayen", - "value": "Targarayen" - }, - { - - "label": "Female", - "value": "Female" - }, - { - "label": "Male", - "value": "Male" - }, - { - "label": "royal", - "value": "royal" - }, - { - "label": "Empty Intersection (no sets)", - "value": "Empty Intersection" - } - ] - }, - { - "id": "voq4", - "prompt": "How large is the largest intersection?", - "required": true, - "location": "sidebar", - "type": "numerical", - "placeholder": "Please enter your answer here", - "min": 0, - "max": 1000000000 - }, - { - "id": "voq5", - "prompt": "How many sets make up the largest intersection?", - "required": true, - "location": "sidebar", - "type": "dropdown", - "placeholder": "Please choose your answer", - "options": [ - { - "label": "It is only of a single set", - "value": "It is only of a single set" - }, - { - "label": "It is the intersection of 2-3 sets", - "value": "It is the intersection of 2-3 sets" - }, - { - "label": "It is the intersection of many sets", - "value": "It is the intersection of many sets" - }, - { - "label": "It is the empty intersection (no sets)", - "value": "It is the empty intersection (no sets)" - } - - ] - }, - { - "id": "voq6", - "prompt": "How similar are the set sizes?", - "required": true, - "location": "sidebar", - "type": "dropdown", - "placeholder": "Please choose your answer", - "options": [ - { - "label": "Roughly equal (all the set sizes look similar)", - "value": "Roughly equal" - }, - { - "label": "Diverging a bit (the sizes of the largest set and the smallest set don't differ largely)", - "value": "Diverging a bit" - }, - { - "label": "Diverging a lot (the sizes of the largest set and the smallest set differ largely)", - "value": "Diverging a lot" - } - ] - }, - { - "id": "voq7", - "prompt": "Is the largest set present in the largest intersection?", - "required": true, - "location": "sidebar", - "type": "radio", - "options": [ - { - "label": "Yes", - "value": "Yes" - }, - { - "label": "No", - "value": "No" - } - ] - }, - { - "id": "voq8", - "prompt": "Is the all-set intersection (intersection having all the sets) present? ", - "required": true, - "location": "sidebar", - "type": "radio", - "options": [ - { - "label": "Yes", - "value": "Yes" - }, - { - "label": "No", - "value": "No" - } - - ] - } - ] - }, - "GoT-Both-Q2": { - "baseComponent": "TandV3", - "path": "Upset-Alttext-User-Survey/assets/GoTextandVis.md" - }, "Organizations-Vis-Q3": { "baseComponent": "VO-image1", "path": "Upset-Alttext-User-Survey/assets/orgVis.md"