OpenNMT-py Translation Server – Update [2018-09-13]
I made a simple REST server to use OpenNMT-py models.
I] How it works?
The idea behind the translation server is to make a entry point for translation with multiple models.
A configuration file (default: ./available_models/conf.json) will describes such models, with the path of the checkpoint files along with other parameters.
Configuration:
-
models_root: (opt) folder containing model checkpoints, [default:./available_models] -
models: list of objects such as :-
id: (opt) manually assign an id (int), [default: value from counter] -
name: (opt) assing a name (str) -
model: (required) path to checkpoint file i.e.*.pt -
timeout: (opt) interval (seconds) before unloading, reset at each translation using the model -
load: (opt) whether to load the model at start [default: False] -
on_timeout: (opt) what to do on timeout:unloadremoves everything;to_cputransfer the model to RAM (from GPU memory) this is faster to reload but takes RAM. -
opt: (opt) dict of translation options (see./translate.py) -
tokenizer: (opt) set tokenizer options (if any), such as:-
type: (str) value in{sentencepiece, pyonmttok}. -
model: (str) path to tokenizer model
-
-
Example
{ "models_root": "./available_models", "models": [ { "id": 100, "model": "model_0.pt", "timeout": 600, "on_timeout": "to_cpu", "load": true, "opt": { "gpu": 0, "beam_size": 5 }, "tokenizer": { "type": "sentencepiece", "model": "wmtenfr.model" } },{ "model": "model_0.light.pt", "timeout": -1, "on_timeout": "unload", "model_root": "../other_models", "opt": { "batch_size": 1, "beam_size": 10 } } ] } II] Start the server
0) Get the code
The translation server has been merged into onmt-py master branch.
Keep in line with master for last fix / improvements.
1) Install flask
pip install flask 2) Put some models
mkdir available_models/ cp $path_to_my_model available_models 3) start the server
export IP="0.0.0.0" export PORT=5000 export URL_ROOT="/translator" export CONFIG="./available_models/conf.json" # NOTE that these parameters are optionnal # here, we explicitely set to default values python server.py --ip $IP --port $PORT --url_root $URL_ROOT --config $CONFIG III] API Usage
0) set the hostname
export HOST="127.0.0.1" 1) list models
curl http://$HOST:$PORT$URL_ROOT/models Result (example):
"available": [ "wmt14.en-de_acc_69.22_ppl_4.33_e9.pt", "wmt14.en-de_acc_69.22_ppl_4.33_e9.light.pt" ], "loaded": [] } 2) Translate
(this example involves subwords)
curl -i -X POST -H "Content-Type: application/json" \ -d '[{"src": "this is a test for model 0", "id": 0}]' \ http://$HOST:$PORT$URL_ROOT/translate Result:
{ "model_id": 0, "result": "\u2581die \u2581Formen kant en \u2581( K \u00f6r ner ) \u2581des \u2581Stahl g u\u00df form .\n", "status": "ok", "time": { "total": 8.510261535644531, "translation": 8.509992599487305, "writing_src": 0.0002689361572265625 } }