Skip to content
This repository was archived by the owner on Jul 7, 2023. It is now read-only.
This repository was archived by the owner on Jul 7, 2023. It is now read-only.

[question] - using custom vocabulary #850

@jestjest

Description

@jestjest

Are there any helpful posts or requirements into how to use tensor2tensor with a custom vocabulary? It's for a translation problem.

For example, do we need to include and as the first two lines in the vocabulary file, and UNK at the end?

I'm following the example from https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/data_generators/translate_ende.py and it seems like generate_samples will create a vocabulary file from a temporary one already?

Thank you.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions