Merge branch 'master' into rolling-update

Author: deliahu

README.md

@@ -15,14 +15,13 @@ Cortex is an open source platform for deploying machine learning models as produ
 
 ## Key features
 
-* **Multi framework:** Cortex supports TensorFlow, PyTorch, scikit-learn, XGBoost, and more.
-* **Autoscaling:** Cortex automatically scales APIs to handle production workloads.
-* **CPU / GPU support:** Cortex can run inference on CPU or GPU infrastructure.
-* **Spot instances:** Cortex supports EC2 spot instances.
-* **Rolling updates:** Cortex updates deployed APIs without any downtime.
-* **Log streaming:** Cortex streams logs from deployed models to your CLI.
-* **Prediction monitoring:** Cortex monitors network metrics and tracks predictions.
-* **Minimal configuration:** Cortex deployments are defined in a single `cortex.yaml` file.
+* **Multi framework:** deploy TensorFlow, PyTorch, scikit-learn, and other models.
+* **Autoscaling:** automatically scale APIs to handle production workloads.
+* **CPU / GPU support:** run inference on CPU or GPU instances.
+* **Spot instances:** save money with EC2 spot instances.
+* **Rolling updates:** update deployed APIs with no downtime.
+* **Log streaming:** stream logs from deployed models to your CLI.
+* **Prediction monitoring:** monitor API performance and track predictions.
 
 <br>
 
@@ -135,6 +134,8 @@ Cortex is an open source alternative to serving models with SageMaker or buildin
 
 The CLI sends configuration and code to the cluster every time you run `cortex deploy`. Each model is loaded into a Docker container, along with any Python packages and request handling code. The model is exposed as a web service using Elastic Load Balancing (ELB), TensorFlow Serving, and ONNX Runtime. The containers are orchestrated on Elastic Kubernetes Service (EKS) while logs and metrics are streamed to CloudWatch.
 
+**Note:** Cortex manages its own Kubernetes cluster so that end-to-end functionality like request-based autoscaling, GPU support, and spot instance management can work out of the box without any DevOps work.
+
 <br>
 
 ## Examples of Cortex deployments

examples/tensorflow/license-plate-reader/predictor_crnn.py

@@ -31,9 +31,14 @@ def predict(self, payload):
  except ValueError:
  # exception can occur when the images are too small
  prediction_groups = []
+
+ image_list = []
  for img_predictions in prediction_groups:
+ boxes_per_image = []
  for predictions in img_predictions:
- predictions = tuple([predictions[0], predictions[1].tolist()])
- lps = {"license-plates": prediction_groups}
+ boxes_per_image.append([predictions[0], predictions[1].tolist()])
+ image_list.append(boxes_per_image)
+
+ lps = {"license-plates": image_list}
 
  return lps

examples/tensorflow/license-plate-reader/predictor_yolo.py

@@ -40,6 +40,8 @@ def predict(self, payload):
  # package the response
  response = {"boxes": []}
  for box in boxes:
- response["boxes"].append([box.xmin, box.ymin, box.xmax, box.ymax, box.c, box.classes])
+ response["boxes"].append(
+ [box.xmin, box.ymin, box.xmax, box.ymax, float(box.c), box.classes.tolist()]
+ )
 
  return response