Closes #593 | Add Dataloader XQuAD-R #601
Conversation
akhdanfadh
requested review from
holylovenia,
SamuelCahyawijaya,
sabilmakbar,
jamesjaya,
yongzx,
gentaiscool,
ljvmiranda921,
jensan-1,
danjohnvelasco,
MJonibek and
tellarin
as code owners
holylovenia
requested review from
khelli07 and
muhammadravi251001
and removed request for
tellarin,
gentaiscool,
jamesjaya,
SamuelCahyawijaya,
ljvmiranda921,
holylovenia,
MJonibek,
danjohnvelasco,
jensan-1,
sabilmakbar,
yongzx and
khelli07
There was a problem hiding this comment.
The test passed successfully, and the results from load_dataset() perfectly matched. Just one minor suggestion below.
| # coding=utf-8 | ||
| # Copyright 2022 The HuggingFace Datasets Authors and the current dataset script contributor. | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. |
There was a problem hiding this comment.
Remove this comment block
There was a problem hiding this comment.
May I know why?
There was a problem hiding this comment.
I guess for a unified format consistent with the rest of the submitted and approved dataloaders, as seen in my implementation in this dataloader and the discussion below: #609 (comment). Previously, my code style was the same as yours (including that comment-block-header), but after receiving that comment, I resolved it to maintain the unified format.
| yield key, { | ||
| "id": str(key), | ||
| "question_id": qa["id"], | ||
| # "document_id": None, |
There was a problem hiding this comment.
Sorry for the small nitpick, but can you remove this comment line?
|
Since both recommendations have no impact on the dataloader implementation (just for code aesthetics to remove unused comments), I will merge this PR to |
Closes #593
There are 2 subsets. Configs will look like this:
xquadr_vie_source,xquadr_tha_seacrowd_qa, etc. When testing, passxquadr_<subset>to the--subset_idparameter.I will add a new PR for adding the QAR task once this PR is checked, and remove the relevant file from this PR.
Checkbox
seacrowd/sea_datasets/my_dataset/my_dataset.py(please use only lowercase and underscore for dataset naming)._CITATION,_DATASETNAME,_DESCRIPTION,_HOMEPAGE,_LICENSE,_URLs,_SUPPORTED_TASKS,_SOURCE_VERSION, and_SEACROWD_VERSIONvariables._info(),_split_generators()and_generate_examples()in dataloader script.BUILDER_CONFIGSclass attribute is a list with at least oneSEACrowdConfigfor the source schema and one for a seacrowd schema.datasets.load_datasetfunction.python -m tests.test_seacrowd seacrowd/sea_datasets/<my_dataset>/<my_dataset>.py.