Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🔨 Dataset Additions: CSV data, custom validation set, dataset filtering and splitting support #2239

Open
wants to merge 48 commits into
base: feature/v2
Choose a base branch
from
Open

🔨 Dataset Additions: CSV data, custom validation set, dataset filtering and splitting support #2239

wants to merge 48 commits into from

Commits on Aug 7, 2024

  1. re-order data __init__

    @samet-akcay
    samet-akcay committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    c35804f View commit details
    Browse the repository at this point in the history

  2. Added dataset filter and split

    @samet-akcay
    samet-akcay committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    5780541 View commit details
    Browse the repository at this point in the history

  3. Add the base dataset and datamodule implementations

    @samet-akcay
    samet-akcay committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    a139f4f View commit details
    Browse the repository at this point in the history

  4. Add conftest to create sample classification and segmentation datafra… 

    …mes to test filter and split classes
    @samet-akcay
    samet-akcay committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    11492a0 View commit details
    Browse the repository at this point in the history

  5. Edited video datamodule

    @samet-akcay
    samet-akcay committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    1ad059c View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2024

  1. Add csv data documentation

    @samet-akcay
    samet-akcay committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    e631aa0 View commit details
    Browse the repository at this point in the history

  2. Fix a bug in datamodule to address when train/val/test datasets are p… 

    …rovided
    @samet-akcay
    samet-akcay committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    9e53c4c View commit details
    Browse the repository at this point in the history

  3. Fix a bug in resolve_split_mode function

    @samet-akcay
    samet-akcay committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    f45c7ec View commit details
    Browse the repository at this point in the history

  4. Add CSV dataset and datamodules

    @samet-akcay
    samet-akcay committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    28a6244 View commit details
    Browse the repository at this point in the history

  5. add relative import to anomalib.data

    @samet-akcay
    samet-akcay committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    e46bb37 View commit details
    Browse the repository at this point in the history

  6. Add clone and copy method to anomalib dataset

    @samet-akcay
    samet-akcay committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    3aa0ac2 View commit details
    Browse the repository at this point in the history

  7. Add csv tests

    @samet-akcay
    samet-akcay committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    1565bbf View commit details
    Browse the repository at this point in the history

  8. Change csv dataset assignment logic

    @samet-akcay
    samet-akcay committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    591e2ed View commit details
    Browse the repository at this point in the history

  9. Update the mvtec logic

    @samet-akcay
    samet-akcay committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    ad8271b View commit details
    Browse the repository at this point in the history

  10. Reflect the changes in image datasets.

    @samet-akcay
    samet-akcay committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    a19b7f1 View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2024

  1. Add clear cache option to avoid using old samples

    @samet-akcay
    samet-akcay committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    7bc8439 View commit details
    Browse the repository at this point in the history

  2. Modify split_by_ratio logic

    @samet-akcay
    samet-akcay committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    374d360 View commit details
    Browse the repository at this point in the history

  3. Modify _process_train_only_scenario

    @samet-akcay
    samet-akcay committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    df64670 View commit details
    Browse the repository at this point in the history

  4. Update the Folder datamodule

    @samet-akcay
    samet-akcay committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    751ed3c View commit details
    Browse the repository at this point in the history

  5. Update src/anomalib/data/base/datamodule.py 

    Co-authored-by: Dick Ameln <[email protected]>
    @samet-akcay @djdameln
    samet-akcay and djdameln authored Aug 13, 2024
    Configuration menu
    Copy the full SHA
    ceb9675 View commit details
    Browse the repository at this point in the history

  6. pre-commit

    @samet-akcay
    samet-akcay committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    e7e955c View commit details
    Browse the repository at this point in the history

  7. Remove unused old code

    @samet-akcay
    samet-akcay committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    64fd3f3 View commit details
    Browse the repository at this point in the history

  8. Refactor CSV dataset assignment logic

    @samet-akcay
    samet-akcay committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    4dc7d7e View commit details
    Browse the repository at this point in the history

  9. Update folder notebook

    @samet-akcay
    samet-akcay committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    8a7d87a View commit details
    Browse the repository at this point in the history

  10. Add CSV notebook

    @samet-akcay
    samet-akcay committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    73fcd41 View commit details
    Browse the repository at this point in the history

  11. chore: Add CSV datamodule notebook and update folder notebook

    @samet-akcay
    samet-akcay committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    3b91977 View commit details
    Browse the repository at this point in the history

  12. Add csv link to the sidebar in docs.

    @samet-akcay
    samet-akcay committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    a21332a View commit details
    Browse the repository at this point in the history

  13. Add more informative deprecation warning in resolvesplitmode. Change … 

    …logic in split-by-ratio
    @samet-akcay
    samet-akcay committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    5e0a088 View commit details
    Browse the repository at this point in the history

  14. Address PR comments.

    @samet-akcay
    samet-akcay committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    a992f75 View commit details
    Browse the repository at this point in the history

Commits on Aug 20, 2024

  1. Fix the csv tests

    @samet-akcay
    samet-akcay committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    5e7de57 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into feature/add-custom-validation-set-support

    @samet-akcay
    samet-akcay authored Aug 20, 2024
    Configuration menu
    Copy the full SHA
    5eaa516 View commit details
    Browse the repository at this point in the history

  3. refactor: Update folder.yaml configuration file

    @samet-akcay
    samet-akcay committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    f7603a1 View commit details
    Browse the repository at this point in the history

  4. remove normal_test_dir arg from tests

    @samet-akcay
    samet-akcay committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    74ccb06 View commit details
    Browse the repository at this point in the history

  5. remove normal_test_dir arg from tests

    @samet-akcay
    samet-akcay committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    9f58d18 View commit details
    Browse the repository at this point in the history

  6. Reflect the new changes in video datamodules 

    Signed-off-by: Samet Akcay <[email protected]>
    @samet-akcay
    samet-akcay committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    b871b9e View commit details
    Browse the repository at this point in the history

  7. Modify the tests to conform the new datamodule format 

    Signed-off-by: Samet Akcay <[email protected]>
    @samet-akcay
    samet-akcay committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    ebfaf7f View commit details
    Browse the repository at this point in the history

  8. Modify the tests to conform the new datamodule format 

    Signed-off-by: Samet Akcay <[email protected]>
    @samet-akcay
    samet-akcay committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    ce8b5fa View commit details
    Browse the repository at this point in the history

  9. Merge branch 'feature/add-custom-validation-set-support' of github.co… 

    …m:samet-akcay/anomalib into feature/add-custom-validation-set-support
    @samet-akcay
    samet-akcay committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    b3fc488 View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2024

  1. Write the csv file to the notebooks directory.

    @samet-akcay
    samet-akcay committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    c575d6a View commit details
    Browse the repository at this point in the history

Commits on Nov 27, 2024

  1. Resolve merge conflicts 

    Signed-off-by: Samet Akcay <[email protected]>
    @samet-akcay
    samet-akcay committed Nov 27, 2024
    Configuration menu
    Copy the full SHA
    8e905e8 View commit details
    Browse the repository at this point in the history

  2. Resolve merge conflicts 

    Signed-off-by: Samet Akcay <[email protected]>
    @samet-akcay
    samet-akcay committed Nov 27, 2024
    Configuration menu
    Copy the full SHA
    c44f377 View commit details
    Browse the repository at this point in the history

  3. Update the tests 

    Signed-off-by: Samet Akcay <[email protected]>
    @samet-akcay
    samet-akcay committed Nov 27, 2024
    Configuration menu
    Copy the full SHA
    ddede9e View commit details
    Browse the repository at this point in the history

  4. Add the new split logic to kolektor 

    Signed-off-by: Samet Akcay <[email protected]>
    @samet-akcay
    samet-akcay committed Nov 27, 2024
    Configuration menu
    Copy the full SHA
    489e96c View commit details
    Browse the repository at this point in the history

  5. fix pre-commit 

    Signed-off-by: Samet Akcay <[email protected]>
    @samet-akcay
    samet-akcay committed Nov 27, 2024
    Configuration menu
    Copy the full SHA
    e98bada View commit details
    Browse the repository at this point in the history

  6. Modify make_kolektor_dataset function 

    Signed-off-by: Samet Akcay <[email protected]>
    @samet-akcay
    samet-akcay committed Nov 27, 2024
    Configuration menu
    Copy the full SHA
    17b6923 View commit details
    Browse the repository at this point in the history

  7. Refactor validation split configurations across multiple YAML files 

    - Updated validation split modes to 'auto' and set ratios to 'null' in avenue.yaml, btech.yaml, datumaro.yaml, shanghaitech.yaml, and ucsd_ped.yaml.
- Changed test split modes to 'predefined' and set test split ratios to 'null' in btech.yaml, kolektor.yaml, mvtec.yaml, and visa.yaml.
- Adjusted the val_split_ratio in folder.yaml to '0.5' for consistency.

These changes standardize the configuration settings for validation and test splits across various datasets, enhancing maintainability and clarity.
    @samet-akcay
    samet-akcay committed Nov 27, 2024
    Configuration menu
    Copy the full SHA
    b169aeb View commit details
    Browse the repository at this point in the history

  8. Merge branch 'feature/v2' of github.com:openvinotoolkit/anomalib into… 

    … feature/add-custom-validation-set-support
    @samet-akcay
    samet-akcay committed Nov 27, 2024
    Configuration menu
    Copy the full SHA
    5973e20 View commit details
    Browse the repository at this point in the history

Commits on Nov 29, 2024

  1. Properly assign val data transforms 

    Signed-off-by: Samet Akcay <[email protected]>
    @samet-akcay
    samet-akcay committed Nov 29, 2024
    Configuration menu
    Copy the full SHA
    529e0ad View commit details
    Browse the repository at this point in the history