CacheDataset in script not working, pt 2

[ctpn]

Hello,

I am having a very similar issue as #1616, i.e. I can use monai.data.CacheDataset with my data in the colab notebook, but cannot in my script. In script, monai.data.CacheDataset does not work, but monai.data.Dataset does. The author of #1616 resolved the issue by updating to MONAI v 0.4.0, but I am already running v 0.4.0. The Exception raised in monai/transforms/utils.py seems to be because data is not a list or tuple, but instead is of class dict.

Error:

train_ds = monai.data.CacheDataset(data=datalist_train, transform=train_trans, cache_rate=2, num_workers=num_workers) 0%| | 0/7 [00:00<?, ?it/s] Traceback (most recent call last): File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/monai/transforms/utils.py", line 361, in apply_transform return transform(data) File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/monai/transforms/utility/dictionary.py", line 133, in __call__ d[key] = self.adder(d[key]) File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/monai/transforms/utility/array.py", line 118, in __call__ return img[None] TypeError: string indices must be integers The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3437, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File "<ipython-input-50-3971aeffabde>", line 1, in <module> train_ds = monai.data.CacheDataset(data=datalist_train, transform=train_trans, cache_rate=2, num_workers=num_workers) File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/monai/data/dataset.py", line 492, in __init__ self._cache: List = self._fill_cache() File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/monai/data/dataset.py", line 501, in _fill_cache return list(tqdm(p.imap(self._load_cache_item, range(self.cache_num)), total=self.cache_num)) File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/tqdm/std.py", line 1129, in __iter__ for obj in iterable: File "/usr/lib/python3.8/multiprocessing/pool.py", line 868, in next raise value File "/usr/lib/python3.8/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, **kwds)) File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/monai/data/dataset.py", line 516, in _load_cache_item item = apply_transform(_transform, item) File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/monai/transforms/utils.py", line 364, in apply_transform raise RuntimeError(f"applying transform {transform}") from e RuntimeError: applying transform <monai.transforms.utility.dictionary.AddChanneld object at 0x7f586599f760> 

Potentially useful info:

type(datalist_train) Out[52]: list type(train_trans) Out[53]: monai.transforms.compose.Compose 

Furthermore, if I do use monai.data.Dataset --> train_loader = monai.data.Dataloader(...) in my script, then try to run iterate through train_loader, i.e.

for batch_data in train_loader: step += 1 inputs, labels = ( batch_data["image"].to(device), batch_data["label"].to(device), ) 

...
I get a similar traceback. I get the same Exception for temp = iter(train_loader):

Traceback (most recent call last): File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3437, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File "<ipython-input-70-0c2346cfc244>", line 1, in <module> for batch_data in train_loader: File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 363, in __next__ data = self._next_data() File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 989, in _next_data return self._process_data(data) File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1014, in _process_data data.reraise() File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/torch/_utils.py", line 395, in reraise raise self.exc_type(msg) RuntimeError: Caught RuntimeError in DataLoader worker process 0. Original Traceback (most recent call last): File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/monai/transforms/utils.py", line 361, in apply_transform return transform(data) File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/monai/transforms/utility/dictionary.py", line 133, in __call__ d[key] = self.adder(d[key]) File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/monai/transforms/utility/array.py", line 118, in __call__ return img[None] TypeError: string indices must be integers The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/monai/transforms/utils.py", line 361, in apply_transform return transform(data) File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/monai/transforms/compose.py", line 236, in __call__ input_ = apply_transform(_transform, input_) File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/monai/transforms/utils.py", line 364, in apply_transform raise RuntimeError(f"applying transform {transform}") from e RuntimeError: applying transform <monai.transforms.utility.dictionary.AddChanneld object at 0x7f586599f760> The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 185, in _worker_loop data = fetcher.fetch(index) File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp> data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/monai/data/dataset.py", line 69, in __getitem__ data = apply_transform(self.transform, data) File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/monai/transforms/utils.py", line 364, in apply_transform raise RuntimeError(f"applying transform {transform}") from e RuntimeError: applying transform <monai.transforms.compose.Compose object at 0x7f5864259c40> type of data is: <class 'dict'> type of data is: <class 'dict'> 

num_workers is currently set at 1, but same error occurs when it is at 0.

Am stumped. Any advice / direction would be greatly appreciated, thank you!

[rijobro]

Do you think you could create a minimum example that demonstrates your problem? You can use one of the freely available datasets, something like this:

from glob import glob import os from monai.apps import download_and_extract from monai.transforms import <transforms> from monai.data import CacheDataset, DataLoader, Dataset # get data directory = os.environ.get("MONAI_DATA_DIRECTORY") root_dir = tempfile.mkdtemp() if directory is None else os.path.expanduser(directory) print(root_dir) task = "Task09_Spleen" resource = "https://msd-for-monai.s3-us-west-2.amazonaws.com/" + task + ".tar" compressed_file = os.path.join(root_dir, task + ".tar") data_dir = os.path.join(root_dir, task) download_and_extract(resource, compressed_file, root_dir) # get images images = sorted(glob(os.path.join(data_dir, "imagesTr", "*.nii.gz"))) labels = sorted(glob(os.path.join(data_dir, "labelsTr", "*.nii.gz"))) data_dicts = [{"image": image, "label": label} for image, label in zip(images, labels)] # transforms  transform = Compose([ <transforms> ]) ds = CacheDataset(data=data_dicts, transform=transforms, ...) # any other args dl = DataLoader(ds, batch_size=?, shuffle=?, num_workers=?) for _ in dl: pass

[ctpn]

Thanks very much for your reply! I figured it out using the code that you provided. Turns out I was missing LoadImaged from my transforms... I'm so sorry. At some point I think the LoadNiftid transform was removed and I never replaced it properly. Thanks again for the support, much appreciated!

---

More details in case it's helpful to anyone else:

The code rijobro provided, with my parameters:

from glob import glob import os import tempfile from monai.apps import download_and_extract from monai.transforms import Compose, AddChanneld, NormalizeIntensityd, ScaleIntensityRanged, RandCropByPosNegLabeld, Spacingd, Orientationd, ToTensord, LoadImaged, CropForegroundd from monai.data import CacheDataset, DataLoader, Dataset # get data directory = os.environ.get("MONAI_DATA_DIRECTORY") root_dir = tempfile.mkdtemp() if directory is None else os.path.expanduser(directory) print(root_dir) task = "Task09_Spleen" resource = "https://msd-for-monai.s3-us-west-2.amazonaws.com/" + task + ".tar" compressed_file = os.path.join(root_dir, task + ".tar") data_dir = os.path.join(root_dir, task) download_and_extract(resource, compressed_file, root_dir) # get images images = sorted(glob(os.path.join(data_dir, "imagesTr", "*.nii.gz"))) labels = sorted(glob(os.path.join(data_dir, "labelsTr", "*.nii.gz"))) data_dicts = [{"image": image, "label": label} for image, label in zip(images, labels)] # transforms key_list = ['image', 'label'] '''transform = Compose([ AddChanneld(keys=key_list), Spacingd(keys=key_list, pixdim=(1., 1., 1.)), # assuming isotropic here Orientationd(keys=key_list, axcodes='RAS'), RandCropByPosNegLabeld(keys=key_list, label_key='seg', spatial_size=(96, 96, 96), pos=1, neg=1, num_samples=3, image_key='img', image_threshold=0) #ToTensord(keys=key_list) ])''' # ds = CacheDataset(data=data_dicts, transform=transforms, ...) # any other args\ ds = CacheDataset(data=data_dicts, transform=transform, cache_rate=1.0, num_workers=1) ds = Dataset(data=data_dicts, transform=transform) #dl = DataLoader(ds, batch_size=?, shuffle=?, num_workers=?) dl = DataLoader(ds, batch_size=1, shuffle=True, num_workers=1) #dl = DataLoader(ds, batch_size=1, shuffle=True, num_workers=1, worker_init_fn=worker_init_fn) for _ in dl: pass 

ds loads with Dataset, but not with CacheDataset. The error code when using CacheDataset without LoadImaged in transform:

ds = CacheDataset(data=data_dicts, transform=transform, cache_rate=1.0, num_workers=1) 0%| | 0/41 [00:00<?, ?it/s]type of data is: <class 'dict'> 0%| | 0/41 [00:00<?, ?it/s]<class 'dict'> Traceback (most recent call last): File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/monai/transforms/utils.py", line 361, in apply_transform return transform(data) File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/monai/transforms/utility/dictionary.py", line 133, in __call__ d[key] = self.adder(d[key]) File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/monai/transforms/utility/array.py", line 118, in __call__ return img[None] TypeError: string indices must be integers The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3437, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File "<ipython-input-32-53ff2beee332>", line 1, in <module> ds = CacheDataset(data=data_dicts, transform=transform, cache_rate=1.0, num_workers=1) File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/monai/data/dataset.py", line 492, in __init__ self._cache: List = self._fill_cache() File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/monai/data/dataset.py", line 501, in _fill_cache return list(tqdm(p.imap(self._load_cache_item, range(self.cache_num)), total=self.cache_num)) File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/tqdm/std.py", line 1129, in __iter__ for obj in iterable: File "/usr/lib/python3.8/multiprocessing/pool.py", line 868, in next raise value File "/usr/lib/python3.8/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, **kwds)) File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/monai/data/dataset.py", line 516, in _load_cache_item item = apply_transform(_transform, item) File "/home/charissa/PycharmProjects/vesselNN_v2/venv-vesselnn2/lib/python3.8/site-packages/monai/transforms/utils.py", line 364, in apply_transform raise RuntimeError(f"applying transform {transform}") from e RuntimeError: applying transform <monai.transforms.utility.dictionary.AddChanneld object at 0x7fd366c638b0> 

Using CacheDataset with transforms AddChanneld, Spacingd, Orientationd, and/or ToTensord will throw this error, but not with RandCropByPosNegLabeld.

[rijobro]

Ok that makes sense. I guess the error message here wasn't very helpful, maybe that's something we can improve! @wyli?