tf.train.SequenceExample

A SequenceExample represents a sequence of features and some context.

It can be thought of as a proto-implementation of the following python type:

Feature = Union[List[bytes], List[int64], List[float]] class SequenceExample(typing.NamedTuple): context: Dict[str, Feature] feature_lists: Dict[str, List[Feature]]

To implement this as protos it's broken up into sub-messages as follows:

# tf.train.Feature Feature = Union[List[bytes], List[int64], List[float]] # tf.train.FeatureList FeatureList = List[Feature] # tf.train.FeatureLists FeatureLists = Dict[str, FeatureList] # tf.train.SequenceExample class SequenceExample(typing.NamedTuple): context: Dict[str, Feature] feature_lists: FeatureLists

To parse a SequenceExample in TensorFlow refer to the tf.io.parse_sequence_example function.

The context contains features which apply to the entire example. The feature_lists contain a key, value map where each key is associated with a repeated set of tf.train.Features (a tf.train.FeatureList). A FeatureList represents the values of a feature identified by its key over time / frames.

Below is a SequenceExample for a movie recommendation application recording a sequence of ratings by a user. The time-independent features ("locale", "age", "favorites") describing the user are part of the context. The sequence of movies the user rated are part of the feature_lists. For each movie in the sequence we have information on its name and actors and the user's rating. This information is recorded in three separate feature_lists. In the example below there are only two movies. All three feature_lists, namely "movie_ratings", "movie_names", and "actors" have a feature value for both movies. Note, that "actors" is itself a bytes_list with multiple strings per movie.

 context: { feature: { key : "locale" value: { bytes_list: { value: [ "pt_BR" ] } } } feature: { key : "age" value: { float_list: { value: [ 19.0 ] } } } feature: { key : "favorites" value: { bytes_list: { value: [ "Majesty Rose", "Savannah Outen", "One Direction" ] } } } } feature_lists: { feature_list: { key : "movie_ratings" value: { feature: { float_list: { value: [ 4.5 ] } } feature: { float_list: { value: [ 5.0 ] } } } } feature_list: { key : "movie_names" value: { feature: { bytes_list: { value: [ "The Shawshank Redemption" ] } } feature: { bytes_list: { value: [ "Fight Club" ] } } } } feature_list: { key : "actors" value: { feature: { bytes_list: { value: [ "Tim Robbins", "Morgan Freeman" ] } } feature: { bytes_list: { value: [ "Brad Pitt", "Edward Norton", "Helena Bonham Carter" ] } } } } }

A conformant SequenceExample data set obeys the following conventions:

context:

  • All conformant context features K must obey the same conventions as a conformant Example's features (see above).

feature_lists:

  • A FeatureList L may be missing in an example; it is up to the parser configuration to determine if this is allowed or considered an empty list (zero length).
  • If a FeatureList L exists, it may be empty (zero length).
  • If a FeatureList L is non-empty, all features within the FeatureList must have the same data type T. Even across SequenceExamples, the type T of the FeatureList identified by the same key must be the same. An entry without any values may serve as an empty feature.
  • If a FeatureList L is non-empty, it is up to the parser configuration to determine if all features within the FeatureList must have the same size. The same holds for this FeatureList across multiple examples.
  • For sequence modeling (example), the feature lists represent a sequence of frames. In this scenario, all FeatureLists in a SequenceExample have the same number of Feature messages, so that the i-th element in each FeatureList is part of the i-th frame (or time step).

Examples of conformant and non-conformant examples' FeatureLists:

Conformant FeatureLists:

 feature_lists: { feature_list: { key: "movie_ratings" value: { feature: { float_list: { value: [ 4.5 ] } } feature: { float_list: { value: [ 5.0 ] } } } } }

Non-conformant FeatureLists (mismatched types):

 feature_lists: { feature_list: { key: "movie_ratings" value: { feature: { float_list: { value: [ 4.5 ] } } feature: { int64_list: { value: [ 5 ] } } } } }

Conditionally conformant FeatureLists, the parser configuration determines if the feature sizes must match:

 feature_lists: { feature_list: { key: "movie_ratings" value: { feature: { float_list: { value: [ 4.5 ] } } feature: { float_list: { value: [ 5.0, 6.0 ] } } } } }

Examples of conformant and non-conformant SequenceExamples:

Conformant pair of SequenceExample:

 feature_lists: { feature_list: { key: "movie_ratings" value: { feature: { float_list: { value: [ 4.5 ] } } feature: { float_list: { value: [ 5.0 ] } } } } } feature_lists: { feature_list: { key: "movie_ratings" value: { feature: { float_list: { value: [ 4.5 ] } } feature: { float_list: { value: [ 5.0 ] } } feature: { float_list: { value: [ 2.0 ] } } } } }

Conformant pair of SequenceExamples:

 feature_lists: { feature_list: { key: "movie_ratings" value: { feature: { float_list: { value: [ 4.5 ] } } feature: { float_list: { value: [ 5.0 ] } } } } } feature_lists: { feature_list: { key: "movie_ratings" value: { } } }

Conditionally conformant pair of SequenceExamples, the parser configuration determines if the second feature_lists is consistent (zero-length) or invalid (missing "movie_ratings"):

 feature_lists: { feature_list: { key: "movie_ratings" value: { feature: { float_list: { value: [ 4.5 ] } } feature: { float_list: { value: [ 5.0 ] } } } } } feature_lists: { }

Non-conformant pair of SequenceExamples (mismatched types):

 feature_lists: { feature_list: { key: "movie_ratings" value: { feature: { float_list: { value: [ 4.5 ] } } feature: { float_list: { value: [ 5.0 ] } } } } } feature_lists: { feature_list: { key: "movie_ratings" value: { feature: { int64_list: { value: [ 4 ] } } feature: { int64_list: { value: [ 5 ] } } feature: { int64_list: { value: [ 2 ] } } } } }

Conditionally conformant pair of SequenceExamples; the parser configuration determines if the feature sizes must match:

 feature_lists: { feature_list: { key: "movie_ratings" value: { feature: { float_list: { value: [ 4.5 ] } } feature: { float_list: { value: [ 5.0 ] } } } } } feature_lists: { feature_list: { key: "movie_ratings" value: { feature: { float_list: { value: [ 4.0 ] } } feature: { float_list: { value: [ 5.0, 3.0 ] } } } }

context Features context
feature_lists FeatureLists feature_lists