Skip to content

Commit 3aa487c

Browse files
Merge pull request #5 from HowardRiddiough/update-readme
Updated README
2 parents 8b3d5c9 + d5e16a5 commit 3aa487c

File tree

1 file changed

+6
-6
lines changed

1 file changed

+6
-6
lines changed

README.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -11,18 +11,18 @@ Making predictions in PySpark using sophistaicated python ml is unlocked using o
1111

1212
`spark_predict` is a wrapper around a `pandas_udf`, a wrapper is used to enable a python ml model to be passed to the `pandas_udf`.
1313

14-
def spark_predict(model, *cols) -> pyspark.sql.column:
15-
"""This function deploys python ml in PySpark using the `predict` method of the `model` parameter.
14+
def spark_predict(model, cols) -> pyspark.sql.column:
15+
"""This function deploys python ml in PySpark using the `predict` method of `model.
1616

1717
Args:
1818
model: python ml model with sklearn API
19-
*cols (list-like): Features used for predictions, required to be present as columns in the spark DataFrame used to make predictions.
19+
cols (list-like): Features used for predictions, required to be present as columns in the spark DataFrame used to make predictions.
2020
"""
2121
@sf.pandas_udf(returnType=DoubleType())
2222
def predict_pandas_udf(*cols):
2323
# cols will be a tuple of pandas.Series here.
24-
X = pd.concat(cols, axis=1)
25-
return pd.Series(model.predict(X))
24+
x = pd.concat(cols, axis=1)
25+
return pd.Series(model.predict(x))
2626

2727
return predict_pandas_udf(*cols)
2828

@@ -40,4 +40,4 @@ The code used in the [deploying-python-ml-in-pyspark](deploying-python-ml-in-pys
4040

4141
# Further Reading
4242
- The code used in is based on the excellent excellent blog post ["Prediction at Scale with scikit-learn and PySpark Pandas UDFs"](https://medium.com/civis-analytics/prediction-at-scale-with-scikit-learn-and-pyspark-pandas-udfs-51d5ebfb2cd8) written by **Michael Heilman**.
43-
- [sklearn](https://scikit-learn.org/stable/auto_examples/compose/plot_column_transformer_mixed_types.html) has more information on column transformers with mixed types.
43+
- [sklearn](https://scikit-learn.org/stable/auto_examples/compose/plot_column_transformer_mixed_types.html) has more information on column transformers with mixed types.

0 commit comments

Comments
 (0)