Add Ground Truth Streaming notebooks #1617
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
| Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
trenton reviewed Oct 14, 2020
| @@ -0,0 +1,1718 @@ | |||
| { | |||
There was a problem hiding this comment.
| @@ -0,0 +1,1718 @@ | |||
| { | |||
| @@ -0,0 +1,1718 @@ | |||
| { | |||
There was a problem hiding this comment.
| @@ -0,0 +1,1718 @@ | |||
| { | |||
There was a problem hiding this comment.
Contributor Author
There was a problem hiding this comment.
Removed all .format() code and replace with f-strings
added 2 commits October 15, 2020 17:14
* Replace .format with f-strings * Added pip sagemaker isntall * Download image from public url * Minor comments
jjbuck approved these changes Oct 16, 2020
jjbuck approved these changes Oct 16, 2020
jjbuck approved these changes Oct 16, 2020
TEChopra1000 approved these changes Oct 16, 2020
ajaykarpur added a commit that referenced this pull request Dec 1, 2020
* GluonCV YoloV3 Darknet53 example training and inference with Neo (#1266) * upgrade MNIST experiment notebook to SDK v2 (#1576) * GluonCV YoloV3 Darknet53 example minor fixes (#1582) * Code cell type corrected. Removed empty cell * Unzip datasets if not available in the notebook's folder * fix invalid json in MNIST notetook (#1594) * Kkoppolu inference examples (#1587) * Compilation examples changes for new inference containers Update examples for PyTorch - to use the new inference containers - Use SageMaker 2.x * Clear outputs Clear outputs in the notebook * Fix typo Fix typo in text box * Undo change to iterations in old way Undo change to iterations in old way * Code Review feedback Organize imports Code Review feedback * CR Use new inference containers for both uncompiled and compiled flows. * CR Remove incorrect code comments * Update versions of torch and torchvision Co-authored-by: EC2 Default User <ec2-user@ip-172-16-66-130.us-west-2.compute.internal> * add template notebook (#1570) * add template notebook * resolve comments * Bump tensorflow (#1574) Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 1.13.1 to 1.15.4. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](tensorflow/tensorflow@v1.13.1...v1.15.4) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * mxnet_mnist.ipynb fix (#1597) * Update mxnet_mnist.ipynb Set notebook to default to CPU training * Update mxnet_mnist.ipynb * updated birds dataset download source (#1593) * fix pandas errors in notebooks (#1490) * Refactor the Debugger detect_stalled_training_job_and_stop.ipynb notebook (#1592) * publish BYOC with Debugger notebook * some test change * revert the kernel names in the metadata * fix typos * incorporate feedback * incorporate comments * pin to pysdk v1 * remove installation output logs * refactor the stalled training job notebook * remove unnecessary module imports / minor fix * incorporate feedback * minor fix * fix typo * minor fix * fix unfinished sentence * incorporate feedback * minor fix Co-authored-by: Miyoung Choi <cmiyoung@amazon.com> * Make RL training compatible with PyTorch (#1520) * Make RLEstimator() PyTorch compatible & modify cartpole notebook * set use_pytorch to False by default * minor refactor; check in first unit test * indent correction * Verify sagemaker SDK version (#1606) * updating mxnet_mnist notebook (#1588) * updating mxnet_mnist notebook * typo fix * refactoring * refactored mnist.py * updated bucket paths in the notebook for better organization * notebook updated to handle sdk upgrade Co-authored-by: EC2 Default User <ec2-user@ip-172-16-24-233.us-west-2.compute.internal> Co-authored-by: EC2 Default User <ec2-user@ip-172-16-29-127.us-west-2.compute.internal> * fixing Model Package ARNs and removing region specific dependency (#1611) * fixing Model Package ARNs and removing region specific dependency * Adding a disclaimer on reference notebooks Co-authored-by: kwwaikar <kwwaikar@3c22fb50dcae.ant.amazon.com> * Fix: add 'import tensorflow as tf' required by _save_tf_model (#1560) Co-authored-by: Felipe Antunes <felipe.antunes@wildlifestudios.com> * Update xgboost churn neo example for sagemaker v2 (#1591) * Update xgboost churn neo example for sagemaker v2 * Remove use of latest version * Add sagemaker installation command and remove duplicate import * Use sagemaker pysdk v2 * Add setup and cleanup steps * clear output * Revert kernel metadata Co-authored-by: Nikhil Kulkarni <nikhilsk@amazon.com> * Add integration tests using Papermill library for RL notebooks. List of notebooks covered in the tests: (#1580) 1. rl_cartpole_coach/rl_cartpole_coach_gymEnv.ipynb 2. rl_cartpole_ray/rl_cartpole_ray_gymEnv.ipynb Co-authored-by: Akash Goel <goelakas@amazon.com> * Delete KernelExplainerWrapper and remove importing LogitLink and IdentityLink (#1603) * update-neo-mxnet-notebooks (#1625) * update-neo-mxnet-notebooks * refactoring and typo fixes * Add Ground Truth Streaming notebooks (#1617) * Add Ground Truth Streaming notebooks * Made below changes * Replace .format with f-strings * Added pip sagemaker isntall * Download image from public url * Minor comments * Minor f-string updates to chained notebook Co-authored-by: Gopalakrishna, Priyanka <gopalakp@lab126.com> * Added downgrade to SDK 1.72 and edited the text. Verified notebook runs through with no errors. (#1633) * Add SDK version rollback code. (#1634) * Running tests in parallel for RL notebooks. (#1624) Co-authored-by: Akash Goel <goelakas@amazon.com> * fix: resolve breaking changes of neo container, adding `softmax_label` to `compile_model` (#1635) * Fixes #902 (#1632) * fix probability out of bound * fixed probability out of bound * cleared the notebook output * fix of probabilities out of bound * adding an example for Linear Learner regression use case with abalone dataset and input csv format (#1622) * infra: add PR buildspec (#1642) * add notebook instance buildspec * Update HPO_Analyze_TuningJob_Results.ipynb on where to retrieve a HP job (#1637) * Update HPO_Analyze_TuningJob_Results.ipynb Adding instructions on where to find the hyperparameter jobs needed as input. * Update hyperparameter_tuning/analyze_results/HPO_Analyze_TuningJob_Results.ipynb Co-authored-by: Aaron Markham <markhama@amazon.com> * infra: update buildspec (#1649) * update buildspec * terminate early if no notebooks in PR * reformat command * move conditional to build phase as one command * removing object2vec_multilabel_genre_classification.ipynb (#1648) * adding preprocessing tabular data notebooks * incorporating changes * incorporating changes * incorporating changes * incorporating few changes * minor fix to persist sagemaker version * minor fix to persist sagemaker version * removing notebook Co-authored-by: Ajay Karpur <akarpur@amazon.com> * fix: move the Tensorflow import in coach_launcher.py inside the _save_tf_model fn (#1652) Co-authored-by: Akash Goel <goelakas@amazon.com> * delete extra common folder inside rl_game_server_autopilot/sagemaker directory (#1653) Co-authored-by: Akash Goel <goelakas@amazon.com> * Removed pip install, edited for clarity, tested on JupyterLab (#1660) * doc: fix typos in PyTorch CIFAR-10 notebook (#1650) * fix typos in PyTorch CIFAR-10 notebook * deliberately raise error to test PR build * Revert "deliberately raise error to test PR build" This reverts commit 7c2bac3. * Update mm byo (#1663) * Added note that nb won't run in studio, add note about kernel and sdk version testing details * changed kernel metadata back to conda_mxnet_p36 * Removed conda command to install s3fs. (#1659) * change: updated for sagemaker python sdk 2.x (#1667) * min_df was larger than max_df and outside of the acceptable range of 0.0-1.0 (#1601) * min_df was larger than max_df and outside of the acceptable range of 0.0 to 1.0. This gave me an error but changing the min_df to 0.2 or 0.02 resolved the error. It is unclear if the author intended min_df to be 0.2 or 0.02. * Update ntm_20newsgroups_topic_model.ipynb remove output and changed min_df to a likely better default of 0.2 Co-authored-by: Aaron Markham <markhama@amazon.com> * Neo pytorch inf1 notebook (#1583) * Add Neo notebook for PT model on Inf1 * Change target to inf1 * resolve comments * Add revert sm version * Add multiple cores instruction and fix revert sagemaker version * polish instructions * one more polish * make sm version at least 2.11.0 * change to upgrade only * remove fixed pytorch version Co-authored-by: EC2 Default User <ec2-user@ip-172-16-72-244.us-west-2.compute.internal> Co-authored-by: EC2 Default User <ec2-user@ip-172-16-43-69.us-west-2.compute.internal> Co-authored-by: Aaron Markham <markhama@amazon.com> * Update generate_example_data.py (#1077) Added code solution for Bug in the Multinomial lines: theta = np.asarray(theta).astype('float64') theta = theta / np.sum(theta) and lines: topic_word_distribution = np.asarray(topic_word_distribution).astype('float64') topic_word_distribution = topic_word_distribution / np.sum(topic_word_distribution) Co-authored-by: Aaron Markham <markhama@amazon.com> * Fix boolean argument parsing (#1681) * Fixed predictions showing as array of False instead of a single True or False value (#1679) * Fixed predictions matched showing as array of False instead of showing whether prediction is correct (True or False). * Fixed predictions matched showing as array of False * Fixed predictions showing as array of False instead of a single True or False * Dev branch (#1688) * Adding new project gpt-2 * Reviewed. Reset Kernel. * made fix to reflect region names in model_package_arns * Minor notebook content rearrangement * fixed region-specific arns * Update README.md Added description for new project 'creative-writing-using-gpt-2-text-generation' under 'using_model_packages' * Update README.md added description for new project 'creative-writing-using-gpt-2-text-generation' under 'aws_marketplace/using_model_packages' Co-authored-by: Alex Ignatov <aignatov@amazon.com> * fix: use image_uris module for retrieval (#1698) * added autogluon v0.0.14 support, changed the build method (#1640) * added autogluon v0.0.14 support, changed the build method * changed the bash execution Co-authored-by: Eric Johnson <65414824+metrizable@users.noreply.github.com> * added data ingestion notebooks (#1602) * added data ingestion notebooks data ingestion notebooks v1 * Added image for Athena and Redshift notebook Added images displayed in two data ingestion notebooks -- Athena and Redshift * Text Data Pre-processing Notebook New notebook added for text data pre-processing, feedback incorporated * Include Data Aggregation to text data ingestion (S3) include the text data aggregation content to the text data ingestion notebook * Modified Data Ingestion Notebooks and Text preprocessing Notebooks Modified all seven (7) data ingestion and text preprocessing notebooks to incorporate feedback * Modified the image data ingestion notebook Added some note to downloading COCO dataset from online resources * updated all the links in the notebooks links to notebooks are changed to relative links; links to videos are removed for now and can be added later. Citations to data sources and existing aws notebooks are added. * modified some links that were not working modified links that's not working (refer to another folder) * Modified 012 for running error Removed a typo in 012 * updated SageMaker SDK, clear output, added data downloading added data downloading to the beginning of each notebook; update SageMaker SDK at the beginning of each notebook; output cleared. * Modified packages used in notebooks modified packages used in 011, 012, 02, 04 and text data pre-processing. Co-authored-by: ZoeMa <yizhuoma@amazon.com> Co-authored-by: Talia <31782251+TEChopra1000@users.noreply.github.com> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: Ajay Karpur <akarpur@amazon.com> * * Add framework_version to SKLearn estimator (#1716) Co-authored-by: Sean Morgan <seanmrgn@amazon.com> * Fix autopilot_customer_churn.ipynb notebook for Sagemaker V2 SDK (#1699) * Fix notebook for Sagemaker V2 SDK * revert account change Co-authored-by: Michele Ricciardi <mriccia@amazon.com> * Notebook fixed and cleaned (#1726) * Notebook fixed and cleaned * Comment reformatted * Fixed notebooks for errors due to syntax change and cleaned notebooks (#1723) * Revert "Fixed notebooks for errors due to syntax change and cleaned notebooks (#1723)" (#1730) This reverts commit e691349. * Revert "Notebook fixed and cleaned (#1726)" (#1732) This reverts commit b68acb4. * Sample notebook fix 2 (#1675) * Reducing the random hpo resource values We've specified the total number of training jobs to be only 20 and the maximum number of parallel jobs to be 2. * Edited the text to be consistent with the new parameter values. With the new parameter values, this notebook now runs without error. * fixed typo fixed a typo * Updated Neo compilation notebook for GluonCV Yolo example (#1638) * Updated Neo compilation notebook for GluonCV Yolo example * Minor fixes to comments and logging Co-authored-by: Eric Johnson <65414824+metrizable@users.noreply.github.com> Co-authored-by: Ajay Karpur <akarpur@amazon.com> * Fixed malformed TensorFlow estimator declaration. (#1628) * Fixed malformed TensorFlow estimator declaration. * Removed extraneous output. Co-authored-by: Eric Johnson <65414824+metrizable@users.noreply.github.com> * logx=False plots data as User_Score is <=10 (#1265) logx=True doesn't seem appropriate since User_Score is <=10 the plot shows nothing Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: Ajay Karpur <akarpur@amazon.com> * Update detect_stalled_training_job_and_stop.ipynb (#1735) * Updated sagemaker attribute configurations for V2 SDK support (#1636) Co-authored-by: Aaron Markham <markhama@amazon.com> * Update Batch Transform - breast cancer prediction with high level SDK.ipynb (#1138) Fix a small bug. Before specifying content_type='text/csv' in sm_transformer.transform, I get error that "Loading libsvm data failed with Exception, please ensure data is in libsvm format: <class 'ValueError'>" Co-authored-by: Aaron Markham <markhama@amazon.com> * Edit xgboost_customer_churn_studio.ipynb (#1060) Co-authored-by: Aaron Markham <markhama@amazon.com> * added a feature selection notebook (#1664) * added a feature selection notebook * addressed comments and renamed files for CI * used model.model_data to index last trained model in s3 * added pip sagemaker>=2.15.0 * add lineage example notebooks (#90) * add example notebook skeleton for fairness and explainability (#91) Co-authored-by: Xinyu Liu <liuxinyu@amazon.com> Co-authored-by: Bartek Pawlik <bartek.pawlik@outlook.com> Co-authored-by: Dana Benson <31262102+danabens@users.noreply.github.com> Co-authored-by: Krishna Chaitanya Koppolu <71738025+kkoppolu1@users.noreply.github.com> Co-authored-by: EC2 Default User <ec2-user@ip-172-16-66-130.us-west-2.compute.internal> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: IvyBazan <45951687+IvyBazan@users.noreply.github.com> Co-authored-by: chenonit <72450093+chenonit@users.noreply.github.com> Co-authored-by: Valentin Flunkert <vafl@users.noreply.github.com> Co-authored-by: Miyoung <myoung8739@gmail.com> Co-authored-by: Miyoung Choi <cmiyoung@amazon.com> Co-authored-by: Anna Luo <45078924+annaluo676@users.noreply.github.com> Co-authored-by: Pratyush Bagaria <prats13.live@gmail.com> Co-authored-by: EC2 Default User <ec2-user@ip-172-16-24-233.us-west-2.compute.internal> Co-authored-by: EC2 Default User <ec2-user@ip-172-16-29-127.us-west-2.compute.internal> Co-authored-by: Kanchan Waikar <36546813+kwwaikar@users.noreply.github.com> Co-authored-by: kwwaikar <kwwaikar@3c22fb50dcae.ant.amazon.com> Co-authored-by: Felipe Antunes <felipe.antunes@me.com> Co-authored-by: Felipe Antunes <felipe.antunes@wildlifestudios.com> Co-authored-by: Nikhil Kulkarni <knikhil29@gmail.com> Co-authored-by: Nikhil Kulkarni <nikhilsk@amazon.com> Co-authored-by: Akash Goel <goelakash93@gmail.com> Co-authored-by: Akash Goel <goelakas@amazon.com> Co-authored-by: Somnath Sarkar <sosarkar@amazon.com> Co-authored-by: gopalakp <72235203+gopalakp@users.noreply.github.com> Co-authored-by: Gopalakrishna, Priyanka <gopalakp@lab126.com> Co-authored-by: Laren-AWS <57545972+Laren-AWS@users.noreply.github.com> Co-authored-by: Chuyang <chuyangdeng@gmail.com> Co-authored-by: Hongshan Li <lihongshan8128@gmail.com> Co-authored-by: moagaber <47145559+moagaber@users.noreply.github.com> Co-authored-by: Roald Bradley Severtson <rseverts@amazon.com> Co-authored-by: Paul B <paul-cb@users.noreply.github.com> Co-authored-by: Eric Slesar <34587362+eslesar-aws@users.noreply.github.com> Co-authored-by: PaulC-AWS <pccornel@users.noreply.github.com> Co-authored-by: Corvus LEE <51771215+corvuslee@users.noreply.github.com> Co-authored-by: aserfass <65733011+aserfass@users.noreply.github.com> Co-authored-by: minlu1021 <minlu1021@users.noreply.github.com> Co-authored-by: EC2 Default User <ec2-user@ip-172-16-72-244.us-west-2.compute.internal> Co-authored-by: EC2 Default User <ec2-user@ip-172-16-43-69.us-west-2.compute.internal> Co-authored-by: hbono2019 <D18129993@mytudublin.ie> Co-authored-by: H. Furkan Bozkurt <hfurkanbozkurt@gmail.com> Co-authored-by: Eitan Sela <eitan.sela@gmail.com> Co-authored-by: awsmrud <71855151+awsmrud@users.noreply.github.com> Co-authored-by: Alex Ignatov <aignatov@amazon.com> Co-authored-by: Eric Johnson <65414824+metrizable@users.noreply.github.com> Co-authored-by: Yohei Nakayama <25813762+yoheigon@users.noreply.github.com> Co-authored-by: ZoeMa <ym910@nyu.edu> Co-authored-by: ZoeMa <yizhuoma@amazon.com> Co-authored-by: Talia <31782251+TEChopra1000@users.noreply.github.com> Co-authored-by: Sean Morgan <seanmorgan91@gmail.com> Co-authored-by: Sean Morgan <seanmrgn@amazon.com> Co-authored-by: Michele Ricciardi <info@michelericciardi.co.uk> Co-authored-by: Michele Ricciardi <mriccia@amazon.com> Co-authored-by: vivekmadan2 <53404938+vivekmadan2@users.noreply.github.com> Co-authored-by: playphil <66652335+playphil@users.noreply.github.com> Co-authored-by: Gili Nachum <gilinachum@gmail.com> Co-authored-by: sdoyle <sdoyle88@users.noreply.github.com> Co-authored-by: fyang1234 <33530337+fyang1234@users.noreply.github.com> Co-authored-by: annbech <19807786+annbech@users.noreply.github.com> Co-authored-by: Xinyu <59369929+xinyu7030@users.noreply.github.com> Co-authored-by: Xinyu Liu <liuxinyu@amazon.com>
ajaykarpur added a commit that referenced this pull request Dec 1, 2020
* GluonCV YoloV3 Darknet53 example training and inference with Neo (#1266) * upgrade MNIST experiment notebook to SDK v2 (#1576) * GluonCV YoloV3 Darknet53 example minor fixes (#1582) * Code cell type corrected. Removed empty cell * Unzip datasets if not available in the notebook's folder * fix invalid json in MNIST notetook (#1594) * Kkoppolu inference examples (#1587) * Compilation examples changes for new inference containers Update examples for PyTorch - to use the new inference containers - Use SageMaker 2.x * Clear outputs Clear outputs in the notebook * Fix typo Fix typo in text box * Undo change to iterations in old way Undo change to iterations in old way * Code Review feedback Organize imports Code Review feedback * CR Use new inference containers for both uncompiled and compiled flows. * CR Remove incorrect code comments * Update versions of torch and torchvision Co-authored-by: EC2 Default User <ec2-user@ip-172-16-66-130.us-west-2.compute.internal> * add template notebook (#1570) * add template notebook * resolve comments * Bump tensorflow (#1574) Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 1.13.1 to 1.15.4. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](tensorflow/tensorflow@v1.13.1...v1.15.4) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * mxnet_mnist.ipynb fix (#1597) * Update mxnet_mnist.ipynb Set notebook to default to CPU training * Update mxnet_mnist.ipynb * updated birds dataset download source (#1593) * fix pandas errors in notebooks (#1490) * Refactor the Debugger detect_stalled_training_job_and_stop.ipynb notebook (#1592) * publish BYOC with Debugger notebook * some test change * revert the kernel names in the metadata * fix typos * incorporate feedback * incorporate comments * pin to pysdk v1 * remove installation output logs * refactor the stalled training job notebook * remove unnecessary module imports / minor fix * incorporate feedback * minor fix * fix typo * minor fix * fix unfinished sentence * incorporate feedback * minor fix Co-authored-by: Miyoung Choi <cmiyoung@amazon.com> * Make RL training compatible with PyTorch (#1520) * Make RLEstimator() PyTorch compatible & modify cartpole notebook * set use_pytorch to False by default * minor refactor; check in first unit test * indent correction * Verify sagemaker SDK version (#1606) * updating mxnet_mnist notebook (#1588) * updating mxnet_mnist notebook * typo fix * refactoring * refactored mnist.py * updated bucket paths in the notebook for better organization * notebook updated to handle sdk upgrade Co-authored-by: EC2 Default User <ec2-user@ip-172-16-24-233.us-west-2.compute.internal> Co-authored-by: EC2 Default User <ec2-user@ip-172-16-29-127.us-west-2.compute.internal> * fixing Model Package ARNs and removing region specific dependency (#1611) * fixing Model Package ARNs and removing region specific dependency * Adding a disclaimer on reference notebooks Co-authored-by: kwwaikar <kwwaikar@3c22fb50dcae.ant.amazon.com> * Fix: add 'import tensorflow as tf' required by _save_tf_model (#1560) Co-authored-by: Felipe Antunes <felipe.antunes@wildlifestudios.com> * Update xgboost churn neo example for sagemaker v2 (#1591) * Update xgboost churn neo example for sagemaker v2 * Remove use of latest version * Add sagemaker installation command and remove duplicate import * Use sagemaker pysdk v2 * Add setup and cleanup steps * clear output * Revert kernel metadata Co-authored-by: Nikhil Kulkarni <nikhilsk@amazon.com> * Add integration tests using Papermill library for RL notebooks. List of notebooks covered in the tests: (#1580) 1. rl_cartpole_coach/rl_cartpole_coach_gymEnv.ipynb 2. rl_cartpole_ray/rl_cartpole_ray_gymEnv.ipynb Co-authored-by: Akash Goel <goelakas@amazon.com> * Delete KernelExplainerWrapper and remove importing LogitLink and IdentityLink (#1603) * update-neo-mxnet-notebooks (#1625) * update-neo-mxnet-notebooks * refactoring and typo fixes * Add Ground Truth Streaming notebooks (#1617) * Add Ground Truth Streaming notebooks * Made below changes * Replace .format with f-strings * Added pip sagemaker isntall * Download image from public url * Minor comments * Minor f-string updates to chained notebook Co-authored-by: Gopalakrishna, Priyanka <gopalakp@lab126.com> * Added downgrade to SDK 1.72 and edited the text. Verified notebook runs through with no errors. (#1633) * Add SDK version rollback code. (#1634) * Running tests in parallel for RL notebooks. (#1624) Co-authored-by: Akash Goel <goelakas@amazon.com> * fix: resolve breaking changes of neo container, adding `softmax_label` to `compile_model` (#1635) * Fixes #902 (#1632) * fix probability out of bound * fixed probability out of bound * cleared the notebook output * fix of probabilities out of bound * adding an example for Linear Learner regression use case with abalone dataset and input csv format (#1622) * infra: add PR buildspec (#1642) * add notebook instance buildspec * Update HPO_Analyze_TuningJob_Results.ipynb on where to retrieve a HP job (#1637) * Update HPO_Analyze_TuningJob_Results.ipynb Adding instructions on where to find the hyperparameter jobs needed as input. * Update hyperparameter_tuning/analyze_results/HPO_Analyze_TuningJob_Results.ipynb Co-authored-by: Aaron Markham <markhama@amazon.com> * infra: update buildspec (#1649) * update buildspec * terminate early if no notebooks in PR * reformat command * move conditional to build phase as one command * removing object2vec_multilabel_genre_classification.ipynb (#1648) * adding preprocessing tabular data notebooks * incorporating changes * incorporating changes * incorporating changes * incorporating few changes * minor fix to persist sagemaker version * minor fix to persist sagemaker version * removing notebook Co-authored-by: Ajay Karpur <akarpur@amazon.com> * fix: move the Tensorflow import in coach_launcher.py inside the _save_tf_model fn (#1652) Co-authored-by: Akash Goel <goelakas@amazon.com> * delete extra common folder inside rl_game_server_autopilot/sagemaker directory (#1653) Co-authored-by: Akash Goel <goelakas@amazon.com> * Removed pip install, edited for clarity, tested on JupyterLab (#1660) * doc: fix typos in PyTorch CIFAR-10 notebook (#1650) * fix typos in PyTorch CIFAR-10 notebook * deliberately raise error to test PR build * Revert "deliberately raise error to test PR build" This reverts commit 7c2bac3. * Update mm byo (#1663) * Added note that nb won't run in studio, add note about kernel and sdk version testing details * changed kernel metadata back to conda_mxnet_p36 * Removed conda command to install s3fs. (#1659) * change: updated for sagemaker python sdk 2.x (#1667) * min_df was larger than max_df and outside of the acceptable range of 0.0-1.0 (#1601) * min_df was larger than max_df and outside of the acceptable range of 0.0 to 1.0. This gave me an error but changing the min_df to 0.2 or 0.02 resolved the error. It is unclear if the author intended min_df to be 0.2 or 0.02. * Update ntm_20newsgroups_topic_model.ipynb remove output and changed min_df to a likely better default of 0.2 Co-authored-by: Aaron Markham <markhama@amazon.com> * Neo pytorch inf1 notebook (#1583) * Add Neo notebook for PT model on Inf1 * Change target to inf1 * resolve comments * Add revert sm version * Add multiple cores instruction and fix revert sagemaker version * polish instructions * one more polish * make sm version at least 2.11.0 * change to upgrade only * remove fixed pytorch version Co-authored-by: EC2 Default User <ec2-user@ip-172-16-72-244.us-west-2.compute.internal> Co-authored-by: EC2 Default User <ec2-user@ip-172-16-43-69.us-west-2.compute.internal> Co-authored-by: Aaron Markham <markhama@amazon.com> * Update generate_example_data.py (#1077) Added code solution for Bug in the Multinomial lines: theta = np.asarray(theta).astype('float64') theta = theta / np.sum(theta) and lines: topic_word_distribution = np.asarray(topic_word_distribution).astype('float64') topic_word_distribution = topic_word_distribution / np.sum(topic_word_distribution) Co-authored-by: Aaron Markham <markhama@amazon.com> * Fix boolean argument parsing (#1681) * Fixed predictions showing as array of False instead of a single True or False value (#1679) * Fixed predictions matched showing as array of False instead of showing whether prediction is correct (True or False). * Fixed predictions matched showing as array of False * Fixed predictions showing as array of False instead of a single True or False * Dev branch (#1688) * Adding new project gpt-2 * Reviewed. Reset Kernel. * made fix to reflect region names in model_package_arns * Minor notebook content rearrangement * fixed region-specific arns * Update README.md Added description for new project 'creative-writing-using-gpt-2-text-generation' under 'using_model_packages' * Update README.md added description for new project 'creative-writing-using-gpt-2-text-generation' under 'aws_marketplace/using_model_packages' Co-authored-by: Alex Ignatov <aignatov@amazon.com> * fix: use image_uris module for retrieval (#1698) * added autogluon v0.0.14 support, changed the build method (#1640) * added autogluon v0.0.14 support, changed the build method * changed the bash execution Co-authored-by: Eric Johnson <65414824+metrizable@users.noreply.github.com> * added data ingestion notebooks (#1602) * added data ingestion notebooks data ingestion notebooks v1 * Added image for Athena and Redshift notebook Added images displayed in two data ingestion notebooks -- Athena and Redshift * Text Data Pre-processing Notebook New notebook added for text data pre-processing, feedback incorporated * Include Data Aggregation to text data ingestion (S3) include the text data aggregation content to the text data ingestion notebook * Modified Data Ingestion Notebooks and Text preprocessing Notebooks Modified all seven (7) data ingestion and text preprocessing notebooks to incorporate feedback * Modified the image data ingestion notebook Added some note to downloading COCO dataset from online resources * updated all the links in the notebooks links to notebooks are changed to relative links; links to videos are removed for now and can be added later. Citations to data sources and existing aws notebooks are added. * modified some links that were not working modified links that's not working (refer to another folder) * Modified 012 for running error Removed a typo in 012 * updated SageMaker SDK, clear output, added data downloading added data downloading to the beginning of each notebook; update SageMaker SDK at the beginning of each notebook; output cleared. * Modified packages used in notebooks modified packages used in 011, 012, 02, 04 and text data pre-processing. Co-authored-by: ZoeMa <yizhuoma@amazon.com> Co-authored-by: Talia <31782251+TEChopra1000@users.noreply.github.com> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: Ajay Karpur <akarpur@amazon.com> * * Add framework_version to SKLearn estimator (#1716) Co-authored-by: Sean Morgan <seanmrgn@amazon.com> * Fix autopilot_customer_churn.ipynb notebook for Sagemaker V2 SDK (#1699) * Fix notebook for Sagemaker V2 SDK * revert account change Co-authored-by: Michele Ricciardi <mriccia@amazon.com> * Notebook fixed and cleaned (#1726) * Notebook fixed and cleaned * Comment reformatted * Fixed notebooks for errors due to syntax change and cleaned notebooks (#1723) * Revert "Fixed notebooks for errors due to syntax change and cleaned notebooks (#1723)" (#1730) This reverts commit e691349. * Revert "Notebook fixed and cleaned (#1726)" (#1732) This reverts commit b68acb4. * Sample notebook fix 2 (#1675) * Reducing the random hpo resource values We've specified the total number of training jobs to be only 20 and the maximum number of parallel jobs to be 2. * Edited the text to be consistent with the new parameter values. With the new parameter values, this notebook now runs without error. * fixed typo fixed a typo * Updated Neo compilation notebook for GluonCV Yolo example (#1638) * Updated Neo compilation notebook for GluonCV Yolo example * Minor fixes to comments and logging Co-authored-by: Eric Johnson <65414824+metrizable@users.noreply.github.com> Co-authored-by: Ajay Karpur <akarpur@amazon.com> * Fixed malformed TensorFlow estimator declaration. (#1628) * Fixed malformed TensorFlow estimator declaration. * Removed extraneous output. Co-authored-by: Eric Johnson <65414824+metrizable@users.noreply.github.com> * logx=False plots data as User_Score is <=10 (#1265) logx=True doesn't seem appropriate since User_Score is <=10 the plot shows nothing Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: Ajay Karpur <akarpur@amazon.com> * Update detect_stalled_training_job_and_stop.ipynb (#1735) * Updated sagemaker attribute configurations for V2 SDK support (#1636) Co-authored-by: Aaron Markham <markhama@amazon.com> * Update Batch Transform - breast cancer prediction with high level SDK.ipynb (#1138) Fix a small bug. Before specifying content_type='text/csv' in sm_transformer.transform, I get error that "Loading libsvm data failed with Exception, please ensure data is in libsvm format: <class 'ValueError'>" Co-authored-by: Aaron Markham <markhama@amazon.com> * Edit xgboost_customer_churn_studio.ipynb (#1060) Co-authored-by: Aaron Markham <markhama@amazon.com> * added a feature selection notebook (#1664) * added a feature selection notebook * addressed comments and renamed files for CI * used model.model_data to index last trained model in s3 * added pip sagemaker>=2.15.0 * add lineage example notebooks (#90) * add example notebook skeleton for fairness and explainability (#91) Co-authored-by: Xinyu Liu <liuxinyu@amazon.com> Co-authored-by: Bartek Pawlik <bartek.pawlik@outlook.com> Co-authored-by: Dana Benson <31262102+danabens@users.noreply.github.com> Co-authored-by: Krishna Chaitanya Koppolu <71738025+kkoppolu1@users.noreply.github.com> Co-authored-by: EC2 Default User <ec2-user@ip-172-16-66-130.us-west-2.compute.internal> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: IvyBazan <45951687+IvyBazan@users.noreply.github.com> Co-authored-by: chenonit <72450093+chenonit@users.noreply.github.com> Co-authored-by: Valentin Flunkert <vafl@users.noreply.github.com> Co-authored-by: Miyoung <myoung8739@gmail.com> Co-authored-by: Miyoung Choi <cmiyoung@amazon.com> Co-authored-by: Anna Luo <45078924+annaluo676@users.noreply.github.com> Co-authored-by: Pratyush Bagaria <prats13.live@gmail.com> Co-authored-by: EC2 Default User <ec2-user@ip-172-16-24-233.us-west-2.compute.internal> Co-authored-by: EC2 Default User <ec2-user@ip-172-16-29-127.us-west-2.compute.internal> Co-authored-by: Kanchan Waikar <36546813+kwwaikar@users.noreply.github.com> Co-authored-by: kwwaikar <kwwaikar@3c22fb50dcae.ant.amazon.com> Co-authored-by: Felipe Antunes <felipe.antunes@me.com> Co-authored-by: Felipe Antunes <felipe.antunes@wildlifestudios.com> Co-authored-by: Nikhil Kulkarni <knikhil29@gmail.com> Co-authored-by: Nikhil Kulkarni <nikhilsk@amazon.com> Co-authored-by: Akash Goel <goelakash93@gmail.com> Co-authored-by: Akash Goel <goelakas@amazon.com> Co-authored-by: Somnath Sarkar <sosarkar@amazon.com> Co-authored-by: gopalakp <72235203+gopalakp@users.noreply.github.com> Co-authored-by: Gopalakrishna, Priyanka <gopalakp@lab126.com> Co-authored-by: Laren-AWS <57545972+Laren-AWS@users.noreply.github.com> Co-authored-by: Chuyang <chuyangdeng@gmail.com> Co-authored-by: Hongshan Li <lihongshan8128@gmail.com> Co-authored-by: moagaber <47145559+moagaber@users.noreply.github.com> Co-authored-by: Roald Bradley Severtson <rseverts@amazon.com> Co-authored-by: Paul B <paul-cb@users.noreply.github.com> Co-authored-by: Eric Slesar <34587362+eslesar-aws@users.noreply.github.com> Co-authored-by: PaulC-AWS <pccornel@users.noreply.github.com> Co-authored-by: Corvus LEE <51771215+corvuslee@users.noreply.github.com> Co-authored-by: aserfass <65733011+aserfass@users.noreply.github.com> Co-authored-by: minlu1021 <minlu1021@users.noreply.github.com> Co-authored-by: EC2 Default User <ec2-user@ip-172-16-72-244.us-west-2.compute.internal> Co-authored-by: EC2 Default User <ec2-user@ip-172-16-43-69.us-west-2.compute.internal> Co-authored-by: hbono2019 <D18129993@mytudublin.ie> Co-authored-by: H. Furkan Bozkurt <hfurkanbozkurt@gmail.com> Co-authored-by: Eitan Sela <eitan.sela@gmail.com> Co-authored-by: awsmrud <71855151+awsmrud@users.noreply.github.com> Co-authored-by: Alex Ignatov <aignatov@amazon.com> Co-authored-by: Eric Johnson <65414824+metrizable@users.noreply.github.com> Co-authored-by: Yohei Nakayama <25813762+yoheigon@users.noreply.github.com> Co-authored-by: ZoeMa <ym910@nyu.edu> Co-authored-by: ZoeMa <yizhuoma@amazon.com> Co-authored-by: Talia <31782251+TEChopra1000@users.noreply.github.com> Co-authored-by: Sean Morgan <seanmorgan91@gmail.com> Co-authored-by: Sean Morgan <seanmrgn@amazon.com> Co-authored-by: Michele Ricciardi <info@michelericciardi.co.uk> Co-authored-by: Michele Ricciardi <mriccia@amazon.com> Co-authored-by: vivekmadan2 <53404938+vivekmadan2@users.noreply.github.com> Co-authored-by: playphil <66652335+playphil@users.noreply.github.com> Co-authored-by: Gili Nachum <gilinachum@gmail.com> Co-authored-by: sdoyle <sdoyle88@users.noreply.github.com> Co-authored-by: fyang1234 <33530337+fyang1234@users.noreply.github.com> Co-authored-by: annbech <19807786+annbech@users.noreply.github.com> Co-authored-by: Xinyu <59369929+xinyu7030@users.noreply.github.com> Co-authored-by: Xinyu Liu <liuxinyu@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.