Skip to content

Conversation

@kshitizrimal
Copy link
Contributor

I have created Image classification tutorial using TF.Keras and ImageDataGenerator class, of which I am an owner. Please have a look and if everything is fine with the tutorial, I would like to make this pull request.

@googlebot
Copy link

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here (e.g. I signed it!) and we'll verify it.


What to do if you already signed the CLA

Individual signers
Corporate signers
@kshitizrimal
Copy link
Contributor Author

I signed it!

@googlebot
Copy link

CLAs look good, thanks!

@lamberta
Copy link
Member

lamberta commented Oct 5, 2018

Very nice, thank you!
Colab preview here: http://colab.research.google.com/github/kshitizrimal/docs/blob/image-classification/site/en/tutorials/keras/image_classification.ipynb

A couple of quick things I see:

  • Please delete the .DS_Store and .ipynb_checkpoints/ files that crept into this PR.
  • We're saving notebooks without output (output is rendered when imported into tensorflow.org)
  • I think we want to keep the tutorials/keras collection limited to the examples from Deep Learning with Python, so maybe we can look at /tutorials/images for this?
  • In Colab, save notebook with left table of contents open
  • We'll need to add an Apache 2 license
@kshitizrimal
Copy link
Contributor Author

Ok thank you .. i will do that

@kshitizrimal
Copy link
Contributor Author

@lamberta about images in your 3rd point.. in the tutorials/images section there are use cases for CIFAR-10 and ImageNet.. does that mean i will have to only use either of them? or meant something else?

@kshitizrimal
Copy link
Contributor Author

I re-checked the book and even in the book, the image dataset used is of cats vs dogs, but there, a local directory path is used for that, but its the same dataset i used here in the tutorial only located in remote server so that anyone can download and use.

@lamberta
Copy link
Member

lamberta commented Oct 5, 2018

We're fine with using other datasets, we just don't want to host them on tensorflow.org :)
If the license is agreeable, we can download it from somewhere—we just need to make it clear where it's coming from.

@kshitizrimal
Copy link
Contributor Author

The link to the images I used came from one of the notebook available in Google official github repo. Here is the link:
https://github.com/google/eng-edu/blob/master/ml/pc/exercises/image_classification_part1.ipynb

Even in this notebook it is only mentioned that the original dataset is from kaggle:
https://www.kaggle.com/c/dogs-vs-cats
Where dataset is publicly available.

Will this work?

@lamberta
Copy link
Member

lamberta commented Oct 6, 2018

Thank you for the context.

We need to discuss some things and come back to this. One issue is the use of the the keras-preprocessing package for your ImageDataGenerator. tf.keras.preprocessing and tf.keras.applications are not actually in tf.keras but are wrappers around these external packages. They work, they're accessible from tf.keras, but they live elsewhere and are not subject to the same API stability guarantees.

While this is definitely useful, and I think a lot of people will find it helpful, we can't quite present this as an official example. But, maybe we can create a community section (or something like that) ... but then that opens a different set of questions.

We'll discuss further and see if we can come up with something. Thanks

Copy link
Member

@lamberta lamberta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to discuss how keras-preprocessing and keras-applications examples fit in.

@kshitizrimal
Copy link
Contributor Author

Sure, I am happy to replace ImageDataGenerator with tf.data api if required. Please let me know what other changes you might need in this notebook and I am happy to do so.

@kshitizrimal
Copy link
Contributor Author

@lamberta if I need to remove keras.preprocessing, I am considering using opencv and numpy to first load and resize the data and use tf.data api to make the generators.. so just wanted to check if this approach is acceptable for this tutorial?

@random-forests
Copy link
Member

Re: the ImageDataGenerator living elsewhere, my vote is to move this into core. Data augmentation (especially for images) is something basically every developer will need to do.

Do we have an canonical utility with similar functionality? If not, I suggest we use these, they're proven. If they need to be rewritten to be backed by tf.data or similar, worth it.

Also in favor of adding this to our official tutorials, rather than a community section, after a bit more cleanup.

@random-forests
Copy link
Member

@kshitizrimal, please hold off on re-writing that (hoping we can find a standard solution, so everyone doesn't need to roll their own for a common task like this).

@kshitizrimal
Copy link
Contributor Author

@kshitizrimal, please hold off on re-writing that (hoping we can find a standard solution, so everyone doesn't need to roll their own for a common task like this).

@random-forests sure, just let me know when its all sort out and I will make the changes accordingly.

@MarkDaoust
Copy link
Member

Sorry about the delay here.

We have talked with the owners, and they agree that if tf.keras.preprocessing is the easiest way to do these things, we should show it.

We might re-organize/edit the content a little (maybe break it into smaller documents). But let's not do that in this PR.

I'm going to merge this, It won't be visible on the site immediately but I'll post back here when it is.

@MarkDaoust MarkDaoust merged commit 3b14882 into tensorflow:master Oct 30, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

5 participants