Skip to content

Commit

Permalink
solved error of missing few samples due to integer division of parall…
Browse files Browse the repository at this point in the history
…el worker_chunksize
  • Loading branch information
Saiyam26 committed Dec 9, 2024
1 parent 16ad8c6 commit e88f9a1
Showing 1 changed file with 4 additions and 2 deletions.
6 changes: 4 additions & 2 deletions scalr/utils/file_utils.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
"""This file contains functions related to file read-write."""

import json
from math import ceil
import os
from os import path
from typing import Union
Expand Down Expand Up @@ -146,8 +147,9 @@ def transform_and_write_data(data: AnnData, chunk_number: int):

write_data(data, path.join(dirpath, f'{chunk_number}.h5ad'))

worker_chunksize = (sample_chunksize //
num_workers) if num_workers else sample_chunksize
worker_chunksize = int(
ceil(sample_chunksize /
num_workers)) if num_workers else sample_chunksize

# Execute parallel jobs for transformation and witing of data.
# In case of `num_workers = None`, single process is used.
Expand Down

0 comments on commit e88f9a1

Please sign in to comment.