Python Forum

I am trying to make the below code work since morning. I have read tutorials on google about python, I have done 0 progress. Can you help me with this error:

Using TensorFlow backend. Traceback (most recent call last): File "source_code_modified.py", line 65, in <module> dict1 = pickle.load(f1,encoding='bytes') TypeError: file must have 'read' and 'readline' attributes

The code that explodes is this part:

class load_train_data: os.open("/home/mojito/Desktop/CNN/datasets/training_data/images", os.O_RDONLY) def __enter__(self): return self def __exit__(self, type, value, traceback): pass class load_test_data: os.open("/home/mojito/Desktop/CNN/datasets/test_data/images",os.O_RDONLY) def __enter__(self): return self def __exit__(self, type, value, traceback): pass with load_train_data() as f1: dict1 = pickle.load(f1,encoding='bytes')

I tried to fix this with the code below and I got a new error:

Using TensorFlow backend. Traceback (most recent call last): File "source_code_modified.py", line 74, in <module> with open_train_data() as f1: File "source_code_modified.py", line 47, in open_train_data return open('/home/mojito/Desktop/CNN/datasets/training_data/images','rb') IsADirectoryError: [Errno 21] Is a directory: '/home/mojito/Desktop/CNN/datasets/training_data/images'

the code explodes in this part:

def open_train_data(): return open('/home/mojito/Desktop/CNN/datasets/training_data/images','rb') <--- explodes here def open_test_data(): return open('/home/mojito/Desktop/CNN/datasets/test_data/images','rb') with open_train_data() as f1: dict1 = pickle.load(f1) <--- explodes here

What I am trying to do is to load a custom dataset of images for a CNN stored to my PC in the format (trainX, trainY), (testX, testY) = ...

Another question: Can I transform the folder with the images to a pickle (.P) file?

What it looks like you are trying to do is 'open' a directory to get all the images, so you can use them elsewhere.
I would recommend the glob module for that.

import glob def open_train_data(): return glob.glob('/home/mojito/Desktop/CNN/datasets/training_data/images/*') #the '*' is a 'wildcard' and it means to match one or more characters for f in open_train_data(): dict1 = pickle.load(f) ###do whatever else you need to

(Jun-12-2020, 09:01 AM)DreamingInsanity Wrote: [ -> ]What it looks like you are trying to do is 'open' a directory to get all the images, so you can use them elsewhere.
I would recommend the glob module for that.

import glob def open_train_data(): return glob.glob('/home/mojito/Desktop/CNN/datasets/training_data/images/*') #the '*' is a 'wildcard' and it means to match one or more characters for f in open_train_data(): dict1 = pickle.load(f) ###do whatever else you need to

I ran the code you posted. I got this error:

Using TensorFlow backend. Traceback (most recent call last): File "source_code_modified.py", line 78, in <module> dict1 = pickle.load(f) TypeError: file must have 'read' and 'readline' attributes

It explodes at this point:

dict1 = pickle.load(f)

Sorry, my mistake - it should work if you do this:

import glob def open_train_data(): return glob.glob('/home/mojito/Desktop/CNN/datasets/training_data/images/*') #the '*' is a 'wildcard' and it means to match one or more characters for f in open_train_data(): dict1 = pickle.load(open(f, 'rb')) ###do whatever else you need to

I forgot to open the file.

(Jun-12-2020, 10:20 AM)DreamingInsanity Wrote: [ -> ]Sorry, my mistake - it should work if you do this:

import glob def open_train_data(): return glob.glob('/home/mojito/Desktop/CNN/datasets/training_data/images/*') #the '*' is a 'wildcard' and it means to match one or more characters for f in open_train_data(): dict1 = pickle.load(open(f, 'rb')) ###do whatever else you need to

I forgot to open the file.

I am getting this error:

Using TensorFlow backend. Traceback (most recent call last): File "source_code_modified.py", line 80, in <module> dict1 = pickle.load(open(f, 'rb')) _pickle.UnpicklingError: A load persistent id instruction was encountered, but no persistent_load function was specified.

I am trying to solve it though, but I thought posting it for more help. Thank you...

Any idea?

(Jun-12-2020, 05:05 PM)hobbyist Wrote: [ -> ]Any idea?

It seems pickle.load() is for use when unpickling. If you are trying to pickle an image, you want to use pickle.dumps() which returns a pickled object.
If you want to save the pickled object to a file, you could use pickle.dump(f, open("myfile.data", "wb"))