Refactoring

Author: bansalkanav

Module 5 - MLOPs/3. Experiment Tracking and Model Management/experiment_tracking_with_mlflow.ipynb

@@ -10,7 +10,9 @@
  "\n",
  "## **Key Features:**\n",
  "1. Experiment Tracking\n",
- "2. Model Registry"
+ "2. Model Registry\n",
+ "\n",
+ "<img src=\"img/tracking_experiments.PNG\">\n"
  ]
  },
  {
@@ -222,7 +224,7 @@
  },
  {
  "cell_type": "code",
- "execution_count": 5,
+ "execution_count": 7,
  "metadata": {},
  "outputs": [],
  "source": [
@@ -244,7 +246,7 @@
  },
  {
  "cell_type": "code",
- "execution_count": 6,
+ "execution_count": 8,
  "metadata": {},
  "outputs": [],
  "source": [
@@ -255,7 +257,7 @@
  },
  {
  "cell_type": "code",
- "execution_count": 7,
+ "execution_count": 9,
  "metadata": {},
  "outputs": [],
  "source": [
@@ -266,7 +268,7 @@
  },
  {
  "cell_type": "code",
- "execution_count": 8,
+ "execution_count": 10,
  "metadata": {},
  "outputs": [
  {
@@ -303,21 +305,30 @@
  "# Initialize the auto logger\n",
  "# max_tuning_runs=None will make sure that all the runs are recorded.\n",
  "# By default top 5 runs will be recorded for each experiment\n",
- "```"
+ "```\n",
+ "**Step 3 - Start the experiment run**\n",
+ "```python\n",
+ "with mlflow.start_run() as run:\n",
+ " clf.fit(X_train, y_train)\n",
+ "```\n",
+ "\n",
+ "\n",
+ "\n",
+ "<img src=\"img/tracking_experiments_hyperparameters.JPG\">"
  ]
  },
  {
  "cell_type": "code",
- "execution_count": 9,
+ "execution_count": 11,
  "metadata": {},
  "outputs": [
  {
  "data": {
  "text/plain": [
- "<Experiment: artifact_location='file:///C:/Users/DELL/Desktop/github/bansalkanav/Machine_Learning_and_Deep_Learning/Module%205%20-%20MLOPs/3.%20Experiment%20Tracking%20and%20Model%20Management/mlruns/947285828145926172', creation_time=1710911744020, experiment_id='947285828145926172', last_update_time=1710911744020, lifecycle_stage='active', name='iris_species_prediction', tags={}>"
+ "<Experiment: artifact_location='file:///C:/Users/DELL/Desktop/github/bansalkanav/Machine_Learning_and_Deep_Learning/Module%205%20-%20MLOPs/3.%20Experiment%20Tracking%20and%20Model%20Management/mlruns/315136114113215422', creation_time=1710940361468, experiment_id='315136114113215422', last_update_time=1710940361468, lifecycle_stage='active', name='iris_species_prediction', tags={}>"
  ]
  },
- "execution_count": 9,
+ "execution_count": 11,
  "metadata": {},
  "output_type": "execute_result"
  }
@@ -330,7 +341,7 @@
  },
  {
  "cell_type": "code",
- "execution_count": 10,
+ "execution_count": 12,
  "metadata": {},
  "outputs": [],
  "source": [
@@ -355,14 +366,14 @@
  },
  {
  "cell_type": "code",
- "execution_count": 11,
+ "execution_count": 13,
  "metadata": {},
  "outputs": [
  {
  "name": "stderr",
  "output_type": "stream",
  "text": [
- "2024/03/20 12:59:41 WARNING mlflow.utils.git_utils: Failed to import Git (the Git executable is probably not on your PATH), so Git SHA is not available. Error: Failed to initialize: Bad git executable.\n",
+ "2024/03/20 18:47:40 WARNING mlflow.utils.git_utils: Failed to import Git (the Git executable is probably not on your PATH), so Git SHA is not available. Error: Failed to initialize: Bad git executable.\n",
  "The git executable must be specified in one of the following ways:\n",
  " - be included in your $PATH\n",
  " - be set via $GIT_PYTHON_GIT_EXECUTABLE\n",
@@ -386,8 +397,8 @@
  "output_type": "stream",
  "text": [
  "Fitting 5 folds for each of 54 candidates, totalling 270 fits\n",
- "CPU times: total: 20.2 s\n",
- "Wall time: 26.2 s\n"
+ "CPU times: total: 31 s\n",
+ "Wall time: 40.8 s\n"
  ]
  }
  ],
@@ -419,7 +430,7 @@
  },
  {
  "cell_type": "code",
- "execution_count": 12,
+ "execution_count": 14,
  "metadata": {},
  "outputs": [],
  "source": [
@@ -454,16 +465,16 @@
  },
  {
  "cell_type": "code",
- "execution_count": 13,
+ "execution_count": 15,
  "metadata": {},
  "outputs": [
  {
  "name": "stdout",
  "output_type": "stream",
  "text": [
  "Fitting 5 folds for each of 60 candidates, totalling 300 fits\n",
- "CPU times: total: 19 s\n",
- "Wall time: 18.3 s\n"
+ "CPU times: total: 22.3 s\n",
+ "Wall time: 31.5 s\n"
  ]
  }
  ],
@@ -495,7 +506,7 @@
  },
  {
  "cell_type": "code",
- "execution_count": 14,
+ "execution_count": 16,
  "metadata": {},
  "outputs": [],
  "source": [
@@ -595,7 +606,7 @@
  },
  {
  "cell_type": "code",
- "execution_count": 15,
+ "execution_count": 17,
  "metadata": {},
  "outputs": [
  {
@@ -604,43 +615,43 @@
  "text": [
  "********** knn **********\n",
  "Fitting 5 folds for each of 54 candidates, totalling 270 fits\n",
- "CPU times: total: 18.4 s\n",
- "Wall time: 18.3 s\n",
+ "CPU times: total: 27 s\n",
+ "Wall time: 36.8 s\n",
  "Train Score: 0.9644268774703558\n",
  "Test Score: 0.9736842105263158\n",
  "\n",
  "********** svc **********\n",
  "Fitting 5 folds for each of 60 candidates, totalling 300 fits\n",
- "CPU times: total: 18.3 s\n",
- "Wall time: 17 s\n",
+ "CPU times: total: 24 s\n",
+ "Wall time: 26.6 s\n",
  "Train Score: 0.9644268774703558\n",
  "Test Score: 0.9736842105263158\n",
  "\n",
  "********** logistic_regression **********\n",
  "Fitting 5 folds for each of 30 candidates, totalling 150 fits\n",
- "CPU times: total: 10.9 s\n",
- "Wall time: 12.4 s\n",
+ "CPU times: total: 14.6 s\n",
+ "Wall time: 23.2 s\n",
  "Train Score: 0.9640316205533598\n",
  "Test Score: 0.9736842105263158\n",
  "\n",
  "********** random_forest **********\n",
  "Fitting 5 folds for each of 6 candidates, totalling 30 fits\n",
- "CPU times: total: 11.2 s\n",
- "Wall time: 16.8 s\n",
+ "CPU times: total: 24.9 s\n",
+ "Wall time: 36.5 s\n",
  "Train Score: 0.9553359683794467\n",
  "Test Score: 0.9736842105263158\n",
  "\n",
  "********** decision_tree **********\n",
  "Fitting 5 folds for each of 6 candidates, totalling 30 fits\n",
- "CPU times: total: 2.3 s\n",
- "Wall time: 8.02 s\n",
+ "CPU times: total: 3.66 s\n",
+ "Wall time: 15.2 s\n",
  "Train Score: 0.9640316205533598\n",
  "Test Score: 0.9736842105263158\n",
  "\n",
  "********** naive_bayes **********\n",
  "Fitting 5 folds for each of 2 candidates, totalling 10 fits\n",
- "CPU times: total: 969 ms\n",
- "Wall time: 6.9 s\n",
+ "CPU times: total: 1.97 s\n",
+ "Wall time: 15.2 s\n",
  "Train Score: 0.9557312252964426\n",
  "Test Score: 1.0\n",
  "\n"
@@ -894,7 +905,10 @@
  "3. **Production**: These versions are actively serving users in live environments.\n",
  " - The \"Production\" tag refers to versions of software or code that are actively running in a live environment and serving end-users or customers.\n",
  " - Production versions are expected to be stable, performant, and reliable, as they are handling real-world traffic and interactions.\n",
- " - Changes to production versions often follow strict release procedures and may involve deployment strategies such as blue-green deployment or canary releases to minimize disruptions."
+ " - Changes to production versions often follow strict release procedures and may involve deployment strategies such as blue-green deployment or canary releases to minimize disruptions.\n",
+ " \n",
+ "\n",
+ "<img src=\"img/model_management.PNG\">\n"
  ]
  }
  ],

Module 5 - MLOPs/3. Experiment Tracking and Model Management/img/tracking_experiments_hyperparameters.JPG

@@ -1,11 +1,5 @@
 # Managing Machine Learning Workflows using Prefect 2.0
 
-### In this repository, you will find three versions of app
-
-> `version_1` - Breaking the Jupyter Notebook to Python Script (Basic Code without workflow management) 
-> `version_2` - Code with Prefect Workflow - Defining the workflow and running them 
-> `version_3` - Deployment and Scheduling tasks
-
 
 ### Why Prefect?
 - Python based open source tool 
@@ -41,21 +35,25 @@ Check the prefect version:
 
 ### Running Prefect Dashboard
 
-> `$ prefect orion start` 
+> `$ prefect server start` 
 
 ```
-___ ___ ___ ___ ___ ___ _____ ___ ___ ___ ___ _ _
-| _ \ _ \ __| __| __/ __|_ _| / _ \| _ \_ _/ _ \| \| |
-| _/ / _|| _|| _| (__ | | | (_) | /| | (_) | .` |
-|_| |_|_\___|_| |___\___| |_| \___/|_|_\___\___/|_|\_|
+ ___ ___ ___ ___ ___ ___ _____
+| _ \ _ \ __| __| __/ __|_ _|
+| _/ / _|| _|| _| (__ | |
+|_| |_|_\___|_| |___\___| |_|
+
 Configure Prefect to communicate with the server with:
+
  prefect config set PREFECT_API_URL=http://127.0.0.1:4200/api
+
 View the API reference documentation at http://127.0.0.1:4200/docs
-Check out the dashboard at http://127.0.0.1:4200/
 
+Check out the dashboard at http://127.0.0.1:4200
 ```
+***
 
-**Note - In Windows OS, if your path contains spaces, it will generate error (as mentioned below) when you try to run prefect orion.**
+**Note - In one of the earliest update of Prefect Orion, in Windows OS, if your path contains spaces, it will generate error (as mentioned below) when you try to run prefect orion. Sharing this so that you know what it is if you see it.**
 
 ```
 ___ ___ ___ ___ ___ ___ _____ ___ ___ ___ ___ _ _
@@ -74,28 +72,7 @@ Error: Got unexpected extra argument (prefect.orion.api.server:create_app)
 Orion stopped!
 ```
 
-### Deployment of Prefect Flow
-
-- `work_queue_name` is used to submit the deployment to the a specific work queue.
-- You don't need to create a work queue before using the work queue. A work queue will be created if it doesn't exist.
-
-```python
-from prefect.deployments import Deployment
-from prefect.orion.schemas.schedules import IntervalSchedule
-from datetime import timedelta
 
-deployment = Deployment.build_from_flow(
- flow=main,
- name="model_training",
- schedule=IntervalSchedule(interval=timedelta(minutes=5)),
- work_queue_name="ml"
-)
+### Make your code schedulable
 
-deployment.apply()
-```
-
-### Running an Agent
-
-```
-$ prefect agent start --work-queue "ml"
-```
+Check the .ipynb file for details.