|
43 | 43 | "source": [ |
44 | 44 | "# Graph regularization for sentiment classification using synthesized graphs\n", |
45 | 45 | "\n", |
46 | | - "<table class=\"tfo-notebook-buttons\" align=\"left\">\n", |
47 | | - " <td>\n", |
48 | | - " <a target=\"_blank\" href=\"https://www.tensorflow.org/neural_structured_learning/tutorials/graph_keras_lstm_imdb\"><img src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" />View on TensorFlow.org</a>\n", |
49 | | - " </td>\n", |
50 | | - " <td>\n", |
51 | | - " <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/neural-structured-learning/blob/master/g3doc/tutorials/graph_keras_lstm_imdb.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n", |
52 | | - " </td>\n", |
53 | | - " <td>\n", |
54 | | - " <a target=\"_blank\" href=\"https://github.com/tensorflow/neural-structured-learning/blob/master/g3doc/tutorials/graph_keras_lstm_imdb.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n", |
55 | | - " </td>\n", |
56 | | - "</table>" |
| 46 | + "\u003ctable class=\"tfo-notebook-buttons\" align=\"left\"\u003e\n", |
| 47 | + " \u003ctd\u003e\n", |
| 48 | + " \u003ca target=\"_blank\" href=\"https://www.tensorflow.org/neural_structured_learning/tutorials/graph_keras_lstm_imdb\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" /\u003eView on TensorFlow.org\u003c/a\u003e\n", |
| 49 | + " \u003c/td\u003e\n", |
| 50 | + " \u003ctd\u003e\n", |
| 51 | + " \u003ca target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/neural-structured-learning/blob/master/g3doc/tutorials/graph_keras_lstm_imdb.ipynb\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" /\u003eRun in Google Colab\u003c/a\u003e\n", |
| 52 | + " \u003c/td\u003e\n", |
| 53 | + " \u003ctd\u003e\n", |
| 54 | + " \u003ca target=\"_blank\" href=\"https://github.com/tensorflow/neural-structured-learning/blob/master/g3doc/tutorials/graph_keras_lstm_imdb.ipynb\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" /\u003eView source on GitHub\u003c/a\u003e\n", |
| 55 | + " \u003c/td\u003e\n", |
| 56 | + "\u003c/table\u003e" |
57 | 57 | ] |
58 | 58 | }, |
59 | 59 | { |
|
344 | 344 | "\n", |
345 | 345 | " # The first indices are reserved\n", |
346 | 346 | " word_index = {k: (v + 3) for k, v in word_index.items()}\n", |
347 | | - " word_index['<PAD>'] = 0\n", |
348 | | - " word_index['<START>'] = 1\n", |
349 | | - " word_index['<UNK>'] = 2 # unknown\n", |
350 | | - " word_index['<UNUSED>'] = 3\n", |
| 347 | + " word_index['\u003cPAD\u003e'] = 0\n", |
| 348 | + " word_index['\u003cSTART\u003e'] = 1\n", |
| 349 | + " word_index['\u003cUNK\u003e'] = 2 # unknown\n", |
| 350 | + " word_index['\u003cUNUSED\u003e'] = 3\n", |
351 | 351 | " return dict((value, key) for (key, value) in word_index.items())\n", |
352 | 352 | "\n", |
353 | 353 | "reverse_word_index = build_reverse_word_index()\n", |
|
569 | 569 | "source": [ |
570 | 570 | "## Sample features\n", |
571 | 571 | "\n", |
572 | | - "We create sample features for our problem in the `tf.train.Example`s format and\n", |
573 | | - "persist them in the `TFRecord` format. Each sample will include the following\n", |
574 | | - "three features:\n", |
| 572 | + "We create sample features for our problem using the `tf.train.Example` format\n", |
| 573 | + "and persist them in the `TFRecord` format. Each sample will include the\n", |
| 574 | + "following three features:\n", |
575 | 575 | "\n", |
576 | 576 | "1. **id**: The node ID of the sample.\n", |
577 | 577 | "2. **words**: An int64 list containing word IDs.\n", |
|
774 | 774 | "source": [ |
775 | 775 | "### Prepare the data\n", |
776 | 776 | "\n", |
777 | | - "The reviews\u2014the arrays of integers\u2014must be converted to tensors before being fed\n", |
| 777 | + "The reviews—the arrays of integers—must be converted to tensors before being fed\n", |
778 | 778 | "into the neural network. This conversion can be done a couple of ways:\n", |
779 | 779 | "\n", |
780 | 780 | "* Convert the arrays into vectors of `0`s and `1`s indicating word occurrence,\n", |
781 | | - " similar to a one-hot encoding. For example, the sequence `[3, 5]` would become a `10000`-dimensional vector that is all zeros except for indices `3` and `5`, which are ones. Then, make this the first layer in our network\u2014a `Dense` layer\u2014that can handle floating point vector data. This approach is memory intensive, though, requiring a `num_words * num_reviews` size matrix.\n", |
| 781 | + " similar to a one-hot encoding. For example, the sequence `[3, 5]` would become a `10000`-dimensional vector that is all zeros except for indices `3` and `5`, which are ones. Then, make this the first layer in our network—a `Dense` layer—that can handle floating point vector data. This approach is memory intensive, though, requiring a `num_words * num_reviews` size matrix.\n", |
782 | 782 | "\n", |
783 | 783 | "* Alternatively, we can pad the arrays so they all have the same length, then\n", |
784 | 784 | " create an integer tensor of shape `max_length * num_reviews`. We can use an\n", |
|
885 | 885 | "source": [ |
886 | 886 | "### Build the model\n", |
887 | 887 | "\n", |
888 | | - "A neural network is created by stacking layers\u2014this requires two main architectural decisions:\n", |
| 888 | + "A neural network is created by stacking layers—this requires two main architectural decisions:\n", |
889 | 889 | "\n", |
890 | 890 | "* How many layers to use in the model?\n", |
891 | 891 | "* How many *hidden units* to use for each layer?\n", |
|
982 | 982 | "If a model has more hidden units (a higher-dimensional representation space),\n", |
983 | 983 | "and/or more layers, then the network can learn more complex representations.\n", |
984 | 984 | "However, it makes the network more computationally expensive and may lead to\n", |
985 | | - "learning unwanted patterns\u2014patterns that improve performance on training data\n", |
| 985 | + "learning unwanted patterns—patterns that improve performance on training data\n", |
986 | 986 | "but not on the test data. This is called *overfitting*." |
987 | 987 | ] |
988 | 988 | }, |
|
1204 | 1204 | "source": [ |
1205 | 1205 | "Notice the training loss *decreases* with each epoch and the training accuracy\n", |
1206 | 1206 | "*increases* with each epoch. This is expected when using a gradient descent\n", |
1207 | | - "optimization\u2014it should minimize the desired quantity on every iteration." |
| 1207 | + "optimization—it should minimize the desired quantity on every iteration." |
1208 | 1208 | ] |
1209 | 1209 | }, |
1210 | 1210 | { |
|
0 commit comments