|
12 | 12 | # See the License for the specific language governing permissions and
|
13 | 13 | # limitations under the License.
|
14 | 14 | # ==============================================================================
|
15 |
| -"""##Ops for evaluation metrics and summary statistics. |
| 15 | +"""Ops for evaluation metrics and summary statistics. |
16 | 16 |
|
17 |
| -### API |
18 |
| -
|
19 |
| -This module provides functions for computing streaming metrics: metrics computed |
20 |
| -on dynamically valued `Tensors`. Each metric declaration returns a |
21 |
| -"value_tensor", an idempotent operation that returns the current value of the |
22 |
| -metric, and an "update_op", an operation that accumulates the information |
23 |
| -from the current value of the `Tensors` being measured as well as returns the |
24 |
| -value of the "value_tensor". |
25 |
| -
|
26 |
| -To use any of these metrics, one need only declare the metric, call `update_op` |
27 |
| -repeatedly to accumulate data over the desired number of `Tensor` values (often |
28 |
| -each one is a single batch) and finally evaluate the value_tensor. For example, |
29 |
| -to use the `streaming_mean`: |
30 |
| -
|
31 |
| -```python |
32 |
| -value = ... |
33 |
| -mean_value, update_op = tf.contrib.metrics.streaming_mean(values) |
34 |
| -sess.run(tf.local_variables_initializer()) |
35 |
| -
|
36 |
| -for i in range(number_of_batches): |
37 |
| - print('Mean after batch %d: %f' % (i, update_op.eval()) |
38 |
| -print('Final Mean: %f' % mean_value.eval()) |
39 |
| -``` |
40 |
| -
|
41 |
| -Each metric function adds nodes to the graph that hold the state necessary to |
42 |
| -compute the value of the metric as well as a set of operations that actually |
43 |
| -perform the computation. Every metric evaluation is composed of three steps |
44 |
| -
|
45 |
| -* Initialization: initializing the metric state. |
46 |
| -* Aggregation: updating the values of the metric state. |
47 |
| -* Finalization: computing the final metric value. |
48 |
| -
|
49 |
| -In the above example, calling streaming_mean creates a pair of state variables |
50 |
| -that will contain (1) the running sum and (2) the count of the number of samples |
51 |
| -in the sum. Because the streaming metrics use local variables, |
52 |
| -the Initialization stage is performed by running the op returned |
53 |
| -by `tf.local_variables_initializer()`. It sets the sum and count variables to |
54 |
| -zero. |
55 |
| -
|
56 |
| -Next, Aggregation is performed by examining the current state of `values` |
57 |
| -and incrementing the state variables appropriately. This step is executed by |
58 |
| -running the `update_op` returned by the metric. |
59 |
| -
|
60 |
| -Finally, finalization is performed by evaluating the "value_tensor" |
61 |
| -
|
62 |
| -In practice, we commonly want to evaluate across many batches and multiple |
63 |
| -metrics. To do so, we need only run the metric computation operations multiple |
64 |
| -times: |
65 |
| -
|
66 |
| -```python |
67 |
| -labels = ... |
68 |
| -predictions = ... |
69 |
| -accuracy, update_op_acc = tf.contrib.metrics.streaming_accuracy( |
70 |
| - labels, predictions) |
71 |
| -error, update_op_error = tf.contrib.metrics.streaming_mean_absolute_error( |
72 |
| - labels, predictions) |
73 |
| -
|
74 |
| -sess.run(tf.local_variables_initializer()) |
75 |
| -for batch in range(num_batches): |
76 |
| - sess.run([update_op_acc, update_op_error]) |
77 |
| -
|
78 |
| -accuracy, mean_absolute_error = sess.run([accuracy, mean_absolute_error]) |
79 |
| -``` |
80 |
| -
|
81 |
| -Note that when evaluating the same metric multiple times on different inputs, |
82 |
| -one must specify the scope of each metric to avoid accumulating the results |
83 |
| -together: |
84 |
| -
|
85 |
| -```python |
86 |
| -labels = ... |
87 |
| -predictions0 = ... |
88 |
| -predictions1 = ... |
89 |
| -
|
90 |
| -accuracy0 = tf.contrib.metrics.accuracy(labels, predictions0, name='preds0') |
91 |
| -accuracy1 = tf.contrib.metrics.accuracy(labels, predictions1, name='preds1') |
92 |
| -``` |
93 |
| -
|
94 |
| -Certain metrics, such as streaming_mean or streaming_accuracy, can be weighted |
95 |
| -via a `weights` argument. The `weights` tensor must be the same size as the |
96 |
| -labels and predictions tensors and results in a weighted average of the metric. |
97 |
| -
|
98 |
| -## Metric `Ops` |
| 17 | +See the @{$python/contrib.metrics} guide. |
99 | 18 |
|
100 | 19 | @@streaming_accuracy
|
101 | 20 | @@streaming_mean
|
|
130 | 49 | @@streaming_true_negatives_at_thresholds
|
131 | 50 | @@streaming_true_positives
|
132 | 51 | @@streaming_true_positives_at_thresholds
|
133 |
| -
|
134 | 52 | @@auc_using_histogram
|
135 |
| -
|
136 | 53 | @@accuracy
|
137 |
| -
|
138 | 54 | @@aggregate_metrics
|
139 | 55 | @@aggregate_metric_map
|
140 |
| -
|
141 | 56 | @@confusion_matrix
|
142 |
| -
|
143 |
| -## Set `Ops` |
144 |
| -
|
145 | 57 | @@set_difference
|
146 | 58 | @@set_intersection
|
147 | 59 | @@set_size
|
|
0 commit comments