Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8243 improve language controlled vocab #10481

Merged
merged 13 commits into from
May 22, 2024
Merged

8243 improve language controlled vocab #10481

Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
8796a1d
pushing the language controlled vocab additions suggested in #10197 i…
landreev Feb 12, 2024
56f3b4b
Merge branch 'develop' into 8243-improve-language-controlled-vocab
landreev Feb 14, 2024
4f8a74c
Alternative long spellings for the Slovene and Kyrgyz languages. #8243
landreev Feb 15, 2024
e3aa0e8
Merge branch 'develop' into 8243-improve-language-controlled-vocab
landreev Apr 10, 2024
f2f1ce1
Dropping the "macrolanguage" from the names of 2 language families, p…
landreev Apr 10, 2024
5acb13a
Addresses the cases of languages with the names like "Navajo, Navaho"…
landreev Apr 10, 2024
6324477
Fixed the tabs in the entries for Catalan and Haitian Creole; added t…
landreev Apr 10, 2024
091ad21
One extra alternative name for Nuosu (from the ISO 893-2 articles on …
landreev Apr 10, 2024
d7a727f
a few minor cosmetic improvements (rearranging the 3 letter codes in …
landreev Apr 10, 2024
8072d91
a release note. #8243
landreev Apr 12, 2024
a84d577
Merge branch 'develop' into 8243-improve-language-controlled-vocab
landreev Apr 12, 2024
93fb1f5
Merge branch 'develop' into 8243-improve-language-controlled-vocab
landreev May 15, 2024
1073333
Populated the identifier column in the Language CV, to make future up…
landreev May 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions doc/release-notes/8243-improve-language-controlled-vocab.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
The Controlled Vocabuary Values list for the metadata field Language in the Citation block has been improved, with some missing two- and three-letter ISO 639 codes added, as well as more alternative names for some of the languages, making all these extra language identifiers importable.

To be added to the 6.3 release instructions:

Update the Citation block, to incorporate the improved controlled vocabulary for language [plus whatever other improvements may be made to the block in other PRs]:

```
wget https://raw.githubusercontent.com/IQSS/dataverse/v6.3/scripts/api/data/metadatablocks/citation.tsv
curl http://localhost:8080/api/admin/datasetfield/load -H "Content-type: text/tab-separated-values" -X POST --upload-file citation.tsv
```

64 changes: 32 additions & 32 deletions scripts/api/data/metadatablocks/citation.tsv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@
authorIdentifierScheme DAI 5
authorIdentifierScheme ResearcherID 6
authorIdentifierScheme ScopusID 7
language Abkhaz 0
language Abkhaz 0 abk ab
language Afar 1 aar aa
language Afrikaans 2 afr af
language Akan 3 aka ak
Expand All @@ -156,17 +156,17 @@
language Bashkir 15 bak ba
language Basque 16 eus baq eu
language Belarusian 17 bel be
language Bengali, Bangla 18 ben bn
language Bengali, Bangla 18 ben bn Bengali Bangla
language Bihari 19 bih bh
language Bislama 20 bis bi
language Bosnian 21 bos bs
language Breton 22 bre br
language Bulgarian 23 bul bg
language Burmese 24 mya bur my
language Catalan,Valencian 25 cat ca
language Catalan,Valencian 25 cat ca Catalan Valencian
language Chamorro 26 cha ch
language Chechen 27 che ce
language Chichewa, Chewa, Nyanja 28 nya ny
language Chichewa, Chewa, Nyanja 28 nya ny Chichewa Chewa Nyanja
language Chinese 29 zho chi zh
language Chuvash 30 chv cv
language Cornish 31 cor kw
Expand All @@ -175,7 +175,7 @@
language Croatian 34 hrv src hr
language Czech 35 ces cze cs
language Danish 36 dan da
language Divehi, Dhivehi, Maldivian 37 div dv
language Divehi, Dhivehi, Maldivian 37 div dv Divehi Dhivehi Maldivian
language Dutch 38 nld dut nl
language Dzongkha 39 dzo dz
language English 40 eng en
Expand All @@ -186,14 +186,14 @@
language Fijian 45 fij fj
language Finnish 46 fin fi
language French 47 fra fre fr
language Fula, Fulah, Pulaar, Pular 48 ful ff
language Fula, Fulah, Pulaar, Pular 48 ful ff Fula Fulah Pulaar Pular
language Galician 49 glg gl
language Georgian 50 kat geo ka
language German 51 deu ger de
language Greek (modern) 52 gre ell el
language Greek (modern) 52 ell gre el Greek
language Guaraní 53 grn gn
language Gujarati 54 guj gu
language Haitian, Haitian Creole 55 hat ht
language Haitian, Haitian Creole 55 hat ht Haitian Haitian Creole
language Hausa 56 hau ha
language Hebrew (modern) 57 heb he
language Herero 58 her hz
Expand All @@ -212,24 +212,24 @@
language Inuktitut 71 iku iu
language Japanese 72 jpn ja
language Javanese 73 jav jv
language Kalaallisut, Greenlandic 74 kal kl
language Kalaallisut, Greenlandic 74 kal kl Kalaallisut Greenlandic
language Kannada 75 kan kn
language Kanuri 76 kau kr
language Kashmiri 77 kas ks
language Kazakh 78 kaz kk
language Khmer 79 khm km
language Kikuyu, Gikuyu 80 kik ki
language Kikuyu, Gikuyu 80 kik ki Kikuyu Gikuyu
language Kinyarwanda 81 kin rw
language Kyrgyz 82
language Kyrgyz 82 kir ky Kirghiz
language Komi 83 kom kv
language Kongo 84 kon kg
language Korean 85 kor ko
language Kurdish 86 kur ku
language Kwanyama, Kuanyama 87 kua kj
language Kwanyama, Kuanyama 87 kua kj Kwanyama Kuanyama
language Latin 88 lat la
language Luxembourgish, Letzeburgesch 89 ltz lb
language Luxembourgish, Letzeburgesch 89 ltz lb Luxembourgish Letzeburgesch
language Ganda 90 lug lg
language Limburgish, Limburgan, Limburger 91 lim li
language Limburgish, Limburgan, Limburger 91 lim li Limburgish Limburgan Limburger
language Lingala 92 lin ln
language Lao 93 lao lo
language Lithuanian 94 lit lt
Expand All @@ -238,35 +238,35 @@
language Manx 97 glv gv
language Macedonian 98 mkd mac mk
language Malagasy 99 mlg mg
language Malay 100 may msa ms
language Malay 100 msa may ms
language Malayalam 101 mal ml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a conversation going on where Youn Noh at Yale is proposing these changes:

ORIG:
	language	Malayalam		101	mal	ml

NEW:
	language	Malayalam	mal	4016	mal	mal	ml

Should we get this change in as well?

Please see https://groups.google.com/g/dataverse-community/c/rGkaHuzA2x4/m/GeyOF1kpAgAJ

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pdurbin The change above doesn't make much sense, so no, we certainly don't want it as is. I'm going to add a comment in the google group thread reiterating an earlier suggestion for the author to take a closer look at the opening comment in this PR, since it answers some of the questions asked there.

language Maltese 102 mlt mt
language Māori 103 mao mri mi
language Māori 103 mri mao mi Maori
language Marathi (Marāṭhī) 104 mar mr
language Marshallese 105 mah mh
language Mixtepec Mixtec 106 mix
language Mongolian 107 mon mn
language Nauru 108 nau na
language Navajo, Navaho 109 nav nv
language Navajo, Navaho 109 nav nv Navajo Navaho
language Northern Ndebele 110 nde nd
language Nepali 111 nep ne
language Ndonga 112 ndo ng
language Norwegian Bokmål 113 nob nb
language Norwegian Nynorsk 114 nno nn
language Norwegian 115 nor no
language Nuosu 116
language Nuosu 116 iii ii Sichuan Yi
language Southern Ndebele 117 nbl nr
language Occitan 118 oci oc
language Ojibwe, Ojibwa 119 oji oj
language Ojibwe, Ojibwa 119 oji oj Ojibwe Ojibwa
language Old Church Slavonic,Church Slavonic,Old Bulgarian 120 chu cu
language Oromo 121 orm om
language Oriya 122 ori or
language Ossetian, Ossetic 123 oss os
language Panjabi, Punjabi 124 pan pa
language Ossetian, Ossetic 123 oss os Ossetian Ossetic
language Panjabi, Punjabi 124 pan pa Panjabi Punjabi
language Pāli 125 pli pi
language Persian (Farsi) 126 per fas fa
language Persian (Farsi) 126 fas per fa
language Polish 127 pol pl
language Pashto, Pushto 128 pus ps
language Pashto, Pushto 128 pus ps Pashto Pushto
language Portuguese 129 por pt
language Quechua 130 que qu
language Romansh 131 roh rm
Expand All @@ -280,14 +280,14 @@
language Samoan 139 smo sm
language Sango 140 sag sg
language Serbian 141 srp scc sr
language Scottish Gaelic, Gaelic 142 gla gd
language Scottish Gaelic, Gaelic 142 gla gd Scottish Gaelic Gaelic
language Shona 143 sna sn
language Sinhala, Sinhalese 144 sin si
language Sinhala, Sinhalese 144 sin si Sinhala Sinhalese
language Slovak 145 slk slo sk
language Slovene 146 slv sl
language Slovene 146 slv sl Slovenian
language Somali 147 som so
language Southern Sotho 148 sot st
language Spanish, Castilian 149 spa es
language Spanish, Castilian 149 spa es Spanish Castilian
language Sundanese 150 sun su
language Swahili 151 swa sw
language Swati 152 ssw ss
Expand All @@ -297,17 +297,17 @@
language Tajik 156 tgk tg
language Thai 157 tha th
language Tigrinya 158 tir ti
language Tibetan Standard, Tibetan, Central 159 tib bod bo
language Tibetan Standard, Tibetan, Central 159 bod tib bo Tibetan Standard Tibetan Central
language Turkmen 160 tuk tk
language Tagalog 161 tgl tl
language Tswana 162 tsn tn
language Tonga (Tonga Islands) 163 ton to
language Tonga (Tonga Islands) 163 ton to Tonga
language Turkish 164 tur tr
language Tsonga 165 tso ts
language Tatar 166 tat tt
language Twi 167 twi tw
language Tahitian 168 tah ty
language Uyghur, Uighur 169 uig ug
language Uyghur, Uighur 169 uig ug Uyghur Uighur
language Ukrainian 170 ukr uk
language Urdu 171 urd ur
language Uzbek 172 uzb uz
Expand All @@ -321,6 +321,6 @@
language Xhosa 180 xho xh
language Yiddish 181 yid yi
language Yoruba 182 yor yo
language Zhuang, Chuang 183 zha za
language Zulu 184 zul zu
language Zhuang, Chuang 183 zha za Zhuang Chuang
language Zulu 184 zul zu
language Not applicable 185
Loading