Skip to content

Commit 6be9576

Browse files
lupeskojspisak
lupesko
authored and
jspisak
committed
Added a tutorial for serving ONNX model with MXNet Model Server (onnx#36)
1 parent bed3a2c commit 6be9576

File tree

3 files changed

+225
-0
lines changed

3 files changed

+225
-0
lines changed

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
11
.ipynb_checkpoints
22
/tutorials/output/*
33
/tutorials/output/*/
4+
.DS_Store
5+
.idea/
46
!/tutorials/output/README.md

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@
1919
* [Converting SuperResolution model from PyTorch to Caffe2 and deploying on mobile device](tutorials/PytorchCaffe2SuperResolution.ipynb)
2020
* [Transferring SqueezeNet from PyTorch to Caffe2 and to Android app](tutorials/PytorchCaffe2MobileSqueezeNet.ipynb)
2121
* [Serving PyTorch Models on AWS Lambda with Caffe2 & ONNX](https://machinelearnings.co/serving-pytorch-models-on-aws-lambda-with-caffe2-onnx-7b096806cfac)
22+
* [Serving ONNX models with MXNet Model Server](tutorials/ONNXMXNetServer.ipynb)
2223

2324
## ONNX tools
2425

‎tutorials/ONNXMXNetServer.ipynb‎

Lines changed: 222 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,222 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# Serving ONNX model with MXNet Model Server\n",
8+
"\n",
9+
"This tutorial dmeonstrates how to serve an ONNX model with MXNet Model Server. We'll use a pre-trained SqueezeNet model from ONNX model zoo. The same example can be easily applied to other ONNX models.\n",
10+
"\n",
11+
"Frameworks used in this tutorial:\n",
12+
"* [MXNet Model Server](https://github.com/awslabs/mxnet-model-server) that uses [MXNet](http://mxnet.io)\n",
13+
"* [ONNX](https://onnx.ai)"
14+
]
15+
},
16+
{
17+
"cell_type": "markdown",
18+
"metadata": {},
19+
"source": [
20+
"## Installing pre-requisites:\n",
21+
"Next we'll install the pre-requisites:\n",
22+
"* [ONNX](https://github.com/onnx/onnx)\n",
23+
"* [MXNetModelServer](https://github.com/awslabs/mxnet-model-server)"
24+
]
25+
},
26+
{
27+
"cell_type": "code",
28+
"execution_count": null,
29+
"metadata": {},
30+
"outputs": [],
31+
"source": [
32+
"!conda install -y -c conda-forge onnx\n",
33+
"!pip install mxnet-model-server"
34+
]
35+
},
36+
{
37+
"cell_type": "markdown",
38+
"metadata": {},
39+
"source": [
40+
"## Downloading a pre-trained ONNX model\n",
41+
"\n",
42+
"Let's go ahead and download a aqueezenet onnx model into a new directory."
43+
]
44+
},
45+
{
46+
"cell_type": "code",
47+
"execution_count": null,
48+
"metadata": {
49+
"scrolled": true
50+
},
51+
"outputs": [],
52+
"source": [
53+
"!mkdir squeezenet\n",
54+
"%cd squeezenet\n",
55+
"!curl -O https://s3.amazonaws.com/model-server/models/onnx-squeezenet/squeezenet.onnx"
56+
]
57+
},
58+
{
59+
"cell_type": "markdown",
60+
"metadata": {},
61+
"source": [
62+
"## Inspecting the ONNX model\n",
63+
"\n",
64+
"Let's make sure the exported ONNX model is well formed"
65+
]
66+
},
67+
{
68+
"cell_type": "code",
69+
"execution_count": null,
70+
"metadata": {},
71+
"outputs": [],
72+
"source": [
73+
"import onnx\n",
74+
"\n",
75+
"# Load the ONNX model\n",
76+
"model = onnx.load(\"squeezenet.onnx\")\n",
77+
"\n",
78+
"# Check that the IR is well formed, identified issues will error out\n",
79+
"onnx.checker.check_model(model)"
80+
]
81+
},
82+
{
83+
"cell_type": "markdown",
84+
"metadata": {
85+
"collapsed": true
86+
},
87+
"source": [
88+
"## Packaging the ONNX model for serving with MXNet Model Server (MMS)\n",
89+
"\n",
90+
"To serve the ONNX model with MMS, we will first need to prepare some files that will be bundled into a **Model Archive**. \n",
91+
"The Model Archive containes everything MMS needs to setup endpoints and serve the model. \n",
92+
"\n",
93+
"The files needed are:\n",
94+
"* squeezenet.onnx - the ONNX model file\n",
95+
"* signature.json - defining the input and output of the model\n",
96+
"* synset.txt - defining the set of classes the model was trained on, and returned by the model"
97+
]
98+
},
99+
{
100+
"cell_type": "code",
101+
"execution_count": null,
102+
"metadata": {},
103+
"outputs": [],
104+
"source": [
105+
"# Let's go ahead and download the files we need:\n",
106+
"!curl -O https://s3.amazonaws.com/model-server/models/onnx-squeezenet/signature.json\n",
107+
"!curl -O https://s3.amazonaws.com/model-server/models/onnx-squeezenet/synset.txt"
108+
]
109+
},
110+
{
111+
"cell_type": "code",
112+
"execution_count": null,
113+
"metadata": {},
114+
"outputs": [],
115+
"source": [
116+
"# Let's take a peek into the **signature.json** file:\n",
117+
"!cat signature.json"
118+
]
119+
},
120+
{
121+
"cell_type": "code",
122+
"execution_count": null,
123+
"metadata": {},
124+
"outputs": [],
125+
"source": [
126+
"# Let's take a peek into the synset.txt file:\n",
127+
"!head -n 10 synset.txt"
128+
]
129+
},
130+
{
131+
"cell_type": "code",
132+
"execution_count": null,
133+
"metadata": {},
134+
"outputs": [],
135+
"source": [
136+
"# Let's package everything up into a Model Archive bundle\n",
137+
"!mxnet-model-export --model-name squeezenet --model-path .\n",
138+
"!ls -l squeezenet.model"
139+
]
140+
},
141+
{
142+
"cell_type": "markdown",
143+
"metadata": {},
144+
"source": [
145+
"## Serving the Model Archive with MXNet Model Server\n",
146+
"Now that we have the **Model Archive**, we can start the server and ask it to setup HTTP endpoints to serve the model, emit real-time operational metrics and more.\n",
147+
"\n",
148+
"We'll also test the server's endpoints."
149+
]
150+
},
151+
{
152+
"cell_type": "code",
153+
"execution_count": null,
154+
"metadata": {},
155+
"outputs": [],
156+
"source": [
157+
"# Spawning a new process to run the server\n",
158+
"import subprocess as sp\n",
159+
"server = sp.Popen(\"mxnet-model-server --models squeezenet=squeezenet.model\", shell=True)"
160+
]
161+
},
162+
{
163+
"cell_type": "code",
164+
"execution_count": null,
165+
"metadata": {},
166+
"outputs": [],
167+
"source": [
168+
"# Check out the health endpoint\n",
169+
"!curl http://127.0.0.1:8080/ping"
170+
]
171+
},
172+
{
173+
"cell_type": "code",
174+
"execution_count": null,
175+
"metadata": {},
176+
"outputs": [],
177+
"source": [
178+
"# Download an image Trying out image classification\n",
179+
"!curl -O https://s3.amazonaws.com/model-server/inputs/kitten.jpg\n",
180+
"!curl -X POST http://127.0.0.1:8080/squeezenet/predict -F \"input_0=@kitten.jpg\""
181+
]
182+
},
183+
{
184+
"cell_type": "code",
185+
"execution_count": null,
186+
"metadata": {},
187+
"outputs": [],
188+
"source": [
189+
"# Lastly, we'll terminate the server\n",
190+
"server.terminate()"
191+
]
192+
},
193+
{
194+
"cell_type": "code",
195+
"execution_count": null,
196+
"metadata": {},
197+
"outputs": [],
198+
"source": []
199+
}
200+
],
201+
"metadata": {
202+
"kernelspec": {
203+
"display_name": "Python 3",
204+
"language": "python",
205+
"name": "python3"
206+
},
207+
"language_info": {
208+
"codemirror_mode": {
209+
"name": "ipython",
210+
"version": 3
211+
},
212+
"file_extension": ".py",
213+
"mimetype": "text/x-python",
214+
"name": "python",
215+
"nbconvert_exporter": "python",
216+
"pygments_lexer": "ipython3",
217+
"version": "3.6.4"
218+
}
219+
},
220+
"nbformat": 4,
221+
"nbformat_minor": 2
222+
}

0 commit comments

Comments
 (0)