Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add SMS-WSJ RETURNN datasets #116

Open
wants to merge 47 commits into
base: main
Choose a base branch
from
Open

add SMS-WSJ RETURNN datasets #116

wants to merge 47 commits into from

Commits on Feb 2, 2023

  1. add SMS-WSJ RETURNN datasets

    @vieting
    vieting committed Feb 2, 2023
    Configuration menu
    Copy the full SHA
    59edbeb View commit details
    Browse the repository at this point in the history

  2. black formatting

    @vieting
    vieting committed Feb 2, 2023
    Configuration menu
    Copy the full SHA
    d95f335 View commit details
    Browse the repository at this point in the history

  3. remove hard coded label for padding

    @vieting
    vieting committed Feb 2, 2023
    Configuration menu
    Copy the full SHA
    2da7e59 View commit details
    Browse the repository at this point in the history

  4. add SmsWsjMixtureEarlyBpeDataset

    @vieting
    vieting committed Feb 2, 2023
    Configuration menu
    Copy the full SHA
    7bfa8d4 View commit details
    Browse the repository at this point in the history

  5. black formatting

    @vieting
    vieting committed Feb 2, 2023
    Configuration menu
    Copy the full SHA
    8c539ea View commit details
    Browse the repository at this point in the history

Commits on Feb 3, 2023

  1. fix data_types

    @vieting
    vieting committed Feb 3, 2023
    Configuration menu
    Copy the full SHA
    9ad34aa View commit details
    Browse the repository at this point in the history

Commits on Feb 6, 2023

  1. bpe add data types

    @vieting
    vieting committed Feb 6, 2023
    Configuration menu
    Copy the full SHA
    ac0718c View commit details
    Browse the repository at this point in the history

  2. allow original alignment

    @vieting
    vieting committed Feb 6, 2023
    Configuration menu
    Copy the full SHA
    5c0ea9e View commit details
    Browse the repository at this point in the history

  3. black

    @vieting
    vieting committed Feb 6, 2023
    Configuration menu
    Copy the full SHA
    ff2f0ea View commit details
    Browse the repository at this point in the history

Commits on Feb 7, 2023

  1. Update common/datasets/sms_wsj/returnn_datasets.py

    @vieting
    vieting authored Feb 7, 2023
    Configuration menu
    Copy the full SHA
    44774a8 View commit details
    Browse the repository at this point in the history

  2. Update common/datasets/sms_wsj/returnn_datasets.py 

    Co-authored-by: SimBe195 <[email protected]>
    @vieting @SimBe195
    vieting and SimBe195 authored Feb 7, 2023
    Configuration menu
    Copy the full SHA
    eed5171 View commit details
    Browse the repository at this point in the history

  3. Update common/datasets/sms_wsj/returnn_datasets.py 

    Co-authored-by: SimBe195 <[email protected]>
    @vieting @SimBe195
    vieting and SimBe195 authored Feb 7, 2023
    Configuration menu
    Copy the full SHA
    1db9771 View commit details
    Browse the repository at this point in the history

  4. Update common/datasets/sms_wsj/returnn_datasets.py 

    Co-authored-by: SimBe195 <[email protected]>
    @vieting @SimBe195
    vieting and SimBe195 authored Feb 7, 2023
    Configuration menu
    Copy the full SHA
    10a13cf View commit details
    Browse the repository at this point in the history

  5. Update common/datasets/sms_wsj/returnn_datasets.py 

    Co-authored-by: SimBe195 <[email protected]>
    @vieting @SimBe195
    vieting and SimBe195 authored Feb 7, 2023
    Configuration menu
    Copy the full SHA
    f7ef64a View commit details
    Browse the repository at this point in the history

  6. Update common/datasets/sms_wsj/returnn_datasets.py 

    Co-authored-by: SimBe195 <[email protected]>
    @vieting @SimBe195
    vieting and SimBe195 authored Feb 7, 2023
    Configuration menu
    Copy the full SHA
    0f4d363 View commit details
    Browse the repository at this point in the history

  7. Update common/datasets/sms_wsj/returnn_datasets.py 

    Co-authored-by: SimBe195 <[email protected]>
    @vieting @SimBe195
    vieting and SimBe195 authored Feb 7, 2023
    Configuration menu
    Copy the full SHA
    c64c6fc View commit details
    Browse the repository at this point in the history

  8. review comments

    @vieting
    vieting committed Feb 7, 2023
    Configuration menu
    Copy the full SHA
    f39ebda View commit details
    Browse the repository at this point in the history

  9. black

    @vieting
    vieting committed Feb 7, 2023
    Configuration menu
    Copy the full SHA
    8b7ad02 View commit details
    Browse the repository at this point in the history

Commits on Feb 9, 2023

  1. _segment_to_rasr str input

    @vieting
    vieting committed Feb 9, 2023
    Configuration menu
    Copy the full SHA
    f736a50 View commit details
    Browse the repository at this point in the history

  2. sms wsj add init

    @vieting
    vieting committed Feb 9, 2023
    Configuration menu
    Copy the full SHA
    4445b29 View commit details
    Browse the repository at this point in the history

  3. caching: use local files if available

    @vieting
    vieting committed Feb 9, 2023
    Configuration menu
    Copy the full SHA
    19d033e View commit details
    Browse the repository at this point in the history

  4. non standard imports inside classes

    @vieting
    vieting committed Feb 9, 2023
    Configuration menu
    Copy the full SHA
    ac70e9d View commit details
    Browse the repository at this point in the history

  5. black

    @vieting
    vieting committed Feb 9, 2023
    Configuration menu
    Copy the full SHA
    30a90f7 View commit details
    Browse the repository at this point in the history

  6. update of super() usage

    @vieting
    vieting committed Feb 9, 2023
    Configuration menu
    Copy the full SHA
    dcce15a View commit details
    Browse the repository at this point in the history

  7. simplify buffer logic

    @vieting
    vieting committed Feb 9, 2023
    Configuration menu
    Copy the full SHA
    e80e210 View commit details
    Browse the repository at this point in the history

  8. print to log

    @vieting
    vieting committed Feb 9, 2023
    Configuration menu
    Copy the full SHA
    f0a0fd8 View commit details
    Browse the repository at this point in the history

  9. avoid name RASR

    @vieting
    vieting committed Feb 9, 2023
    Configuration menu
    Copy the full SHA
    f2aac11 View commit details
    Browse the repository at this point in the history

  10. simons review comments

    @vieting
    vieting committed Feb 9, 2023
    Configuration menu
    Copy the full SHA
    d55bd5a View commit details
    Browse the repository at this point in the history

Commits on Feb 10, 2023

  1. update caching logic

    @vieting
    vieting committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    2f0c484 View commit details
    Browse the repository at this point in the history

  2. returnn log

    @vieting
    vieting committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    4074af4 View commit details
    Browse the repository at this point in the history

  3. pad start int

    @vieting
    vieting committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    c188b82 View commit details
    Browse the repository at this point in the history

  4. log unzipping command

    @vieting
    vieting committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    439da8f View commit details
    Browse the repository at this point in the history

  5. rename rasr variable

    @vieting
    vieting committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    f58844a View commit details
    Browse the repository at this point in the history

  6. use sequence buffer class

    @vieting
    vieting committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    caa18a3 View commit details
    Browse the repository at this point in the history

  7. black

    @vieting
    vieting committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    55ff943 View commit details
    Browse the repository at this point in the history

Commits on Feb 13, 2023

  1. chmod -f

    @vieting
    vieting committed Feb 13, 2023
    Configuration menu
    Copy the full SHA
    8ecf218 View commit details
    Browse the repository at this point in the history

  2. chmod force exit 0

    @vieting
    vieting committed Feb 13, 2023
    Configuration menu
    Copy the full SHA
    cab1c3c View commit details
    Browse the repository at this point in the history

  3. read data from zip instead of unzipping

    @vieting
    vieting committed Feb 13, 2023
    Configuration menu
    Copy the full SHA
    15ebbdc View commit details
    Browse the repository at this point in the history

  4. cleanup

    @vieting
    vieting committed Feb 13, 2023
    Configuration menu
    Copy the full SHA
    b051045 View commit details
    Browse the repository at this point in the history

  5. black

    @vieting
    vieting committed Feb 13, 2023
    Configuration menu
    Copy the full SHA
    3823e2b View commit details
    Browse the repository at this point in the history

Commits on Feb 14, 2023

  1. fix buffer logic

    @vieting
    vieting committed Feb 14, 2023
    Configuration menu
    Copy the full SHA
    b8c3811 View commit details
    Browse the repository at this point in the history

  2. explicit num_outputs required

    @vieting
    vieting committed Feb 14, 2023
    Configuration menu
    Copy the full SHA
    fca6ea3 View commit details
    Browse the repository at this point in the history

  3. prefetch buffer size

    @vieting
    vieting committed Feb 14, 2023
    Configuration menu
    Copy the full SHA
    45fb9a0 View commit details
    Browse the repository at this point in the history

  4. fix init seq order

    @vieting
    vieting committed Feb 14, 2023
    Configuration menu
    Copy the full SHA
    c2cb0a0 View commit details
    Browse the repository at this point in the history

  5. allow shuffling

    @vieting
    vieting committed Feb 14, 2023
    Configuration menu
    Copy the full SHA
    a7fc519 View commit details
    Browse the repository at this point in the history

Commits on Feb 15, 2023

  1. try reading from zip 5 times

    @vieting
    vieting committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    ac50834 View commit details
    Browse the repository at this point in the history

  2. black

    @vieting
    vieting committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    48ad8b5 View commit details
    Browse the repository at this point in the history