Update ML demos for 23c and OML4Py 2.0

machine-learning/notebooks/sql/README.md

@@ -19,7 +19,7 @@ The specific denomination "21c or 23c" in the name of the file means that the al
 
  See [Oracle Machine Learning Notebooks - Get Started](https://docs.oracle.com/en/database/oracle/machine-learning/oml-notebooks/) for more information.
 
-Last updated on: April 2023
+Last updated on: June 2023
 
 #### Copyright (c) 2023 Oracle Corporation and/or its affilitiates.
 

machine-learning/sql/23c/README.md

@@ -0,0 +1,59 @@
+# Oracle Database 23c Machine Learning for SQL examples
+Oracle Machine Learning for SQL is part of the Oracle Machine Learning family of technologies, and is the fastest way to access the heart of the Machine Learning inside every Oracle Database.
+
+Following Oracle's philosophy of moving the algorithms and not the data, the execution of these 30+ parallel, scalable in-Database implementations of machine learning algorithms minimizes or eliminates data movement, achieving scalability, security, and accelerated time-to-model deployment. 
+
+Oracle also provides easy mechanism for running R-extensible scripts directly inside the Oracle Database.
+
+**The following are a list of demos containing SQL code for learning about (OML4SQL)**
+
+Files on the current folder:
+ * dmsh.sql - Setup - Creates Demo Views and Tables
+ * dmshgrants.sql - Setup - Grants Privileges to Users
+ * oml4sql-anomaly-detection-1class-svm.sql - Anomaly Detection - One Class SVM Algorithm
+ * oml4sql-association-rules.sql - Association Rules - Apriori Algorithm
+ * oml4sql-attribute-importance.sql - Attribute Importance - "EXPLAIN" MDL function
+ * oml4sql-classification-decision-tree.sql - Classification - Decision Tree Algorithm
+ * oml4sql-classification-glm.sql - Classification - Generalized Linear Model Algorithm
+ * oml4sql-classification-naive-bayes.sql - Classification - Naïve Bayes Algorithm
+ * oml4sql-classification-neural-networks.sql - Classification - Neural Networks Algorithm
+ * oml4sql-classification-random-forest.sql - Classification - Random Forest Algorithm
+ * oml4sql-classification-regression-xgboost.sql - Classification and Regression - XGBoost Algorithm
+ * oml4sql-classification-svm.sql - Classification - SVM Algorithm
+ * oml4sql-classification-text-mining-svm.sql - Classification - SVM Algorithm with Text Mining
+ * oml4sql-clustering-expectation-maximization.sql - Clustering - Expectation-Maximization Algorithm
+ * oml4sql-clustering-kmeans-star-schema.sql - Clustering - K-Means Algorithm
+ * oml4sql-clustering-kmeans.sql - Clustering - K-Means Algorithm
+ * oml4sql-clustering-ocluster.sql - Clustering - O-Cluster Algorithm
+ * oml4sql-cross-validation-decision-tree.sql - Cross-Validation - Decision Tree Algorithm
+ * oml4sql-feature-extraction-cur.sql - Feature and Row Extraction - CUR Decomposition Algorithm
+ * oml4sql-feature-extraction-nmf.sql - Feature Extraction - Non-Negative Matrix Factorization Algorithm
+ * oml4sql-feature-extraction-svd.sql - Feature Extraction - Singular Value Decomposition
+ * oml4sql-feature-extraction-text-mining-esa.sql - Feature Extraction - ESA Algorithm for Text Mining
+ * oml4sql-feature-extraction-text-mining-nmf.sql - Feature Extraction - NMF Algorithm with Text Mining
+ * oml4sql-feature-extraction-text-term-extraction.sql - Feature Extraction - Text Term Extraction
+ * oml4sql-partitioned-models-svm.sql - Partitioned Models - Support Vector Machine Algorithm
+ * oml4sql-r-extensible-algorithm-registration.sql - OML R Extensible - Algorithm Registration
+ * oml4sql-r-extensible-association-rules.sql - OML R Extensible - Association Rules Algorithm
+ * oml4sql-r-extensible-attribute-importance-via-rf.sql - OML R Extensible - Attribute Importance via RF Algorithm
+ * oml4sql-r-extensible-glm.sql - OML R Extensible - Generalized Linear Model Algorithm
+ * oml4sql-r-extensible-kmeans.sql - OML R Extensible - K-Means Algorithm
+ * oml4sql-r-extensible-principal-components.sql - OML R Extensible - Principal Components Algorithm
+ * oml4sql-r-extensible-regression-neural-networks.sql - OML R Extensible - Regression - Neural Networks Algorithm
+ * oml4sql-r-extensible-regression-tree.sql - OML R Extensible - Regression Tree Algorithm
+ * oml4sql-regression-glm.sql - Regression - Generalized Linear Model Algorithm
+ * oml4sql-regression-neural-networks.sql - Regression - Neural Networks Algorithm
+ * oml4sql-regression-random-forest.sql - Regression - Random Forest Algorithm
+ * oml4sql-regression-svm.sql - Regression - SVM Algorithm
+ * oml4sql-singular-value-decomposition.sql - SVD Algorithm
+ * oml4sql-survival-analysis-xgboost.sql - Survival Analysis with AFT (Accelerated Failure Time) via XGBoost Algorithm
+ * oml4sql-time-series-esm-auto-model-search.sql - Time Series - Exponential Smoothing with Auto Model Search
+ * oml4sql-time-series-exponential-smoothing.sql - Time Series - Exponential Smoothing Algorithm
+ * oml4sql-time-series-mset.sql - Time Series - MSET Multivariate State Estimation Technique Algorithm
+ * oml4sql-time-series-regression-dataset.sql - Time Series Regression - Dataset
+ * oml4sql-time-series-regression.sql - Time Series Regression - Using a combination of ESM and GLM algorithms
+
+#### Copyright (c) 2023 Oracle Corporation and its affiliates
+
+##### [The Universal Permissive License (UPL), Version 1.0](https://oss.oracle.com/licenses/upl/)
+

machine-learning/sql/23c/dmsh.sql

@@ -0,0 +1,150 @@
+-----------------------------------------------------------------------
+-- Oracle Machine Learning for SQL (OML4SQL) 23c
+-- 
+-- Setup - Creates Demo Views and Tables - dmsh.sql
+-- 
+-- Copyright (c) 2023 Oracle Corporation and/or its affilitiates.
+--
+-- The Universal Permissive License (UPL), Version 1.0
+--
+-- https://oss.oracle.com/licenses/upl/
+-----------------------------------------------------------------------
+--
+--
+-- dmsh.sql
+--
+--
+-- NAME
+-- dmsh.sql
+--
+-- DESCRIPTION
+-- This script creates views and tables using SH data
+-- in the schema of the data mining user. These tables/views
+-- are the datasets used by the Oracle Data Mining demo programs.
+-- This script also creates a text policy for text mining.
+-- NOTES
+-- The script assumes that the full SH schema is already created and the
+-- necessary SELECTs have been granted (See dmshgrants.sql). This script runs in 
+-- the schema of the data mining user.
+-- mining_data_*_v views : Used for mining (no text)
+-- mining_*_text views: Used for mining with text 
+--
+--------------------------------------------------------------------------------
+--
+-- Creates data mining views on SH data
+--
+
+--------------------------------------------------------------------------------
+-- View to join and filter data
+-- CUST_YEAR_OF_BIRTH column is transformed to an AGE
+CREATE OR REPLACE VIEW mining_data AS
+SELECT
+ a.CUST_ID, a.CUST_GENDER,
+ 2003-a.CUST_YEAR_OF_BIRTH AGE,
+ a.CUST_MARITAL_STATUS, c.COUNTRY_NAME, a.CUST_INCOME_LEVEL, b.EDUCATION,
+ b.OCCUPATION, b.HOUSEHOLD_SIZE, b.YRS_RESIDENCE, b.AFFINITY_CARD,
+ b.BULK_PACK_DISKETTES, b.FLAT_PANEL_MONITOR, b.HOME_THEATER_PACKAGE,
+ b.BOOKKEEPING_APPLICATION, b.PRINTER_SUPPLIES, b.Y_BOX_GAMES,
+ b.os_doc_set_kanji, b.comments 
+FROM
+ sh.customers a,
+ sh.supplementary_demographics b,
+ sh.countries c
+WHERE
+ a.CUST_ID = b.CUST_ID
+ AND a.country_id = c.country_id
+ AND a.cust_id between 100001 and 104500;
+
+--------------------------------------------------------------------------------
+-- Build, test, and apply views (with text)
+-- Build, test, and apply datasets are made non-overlapping by using
+-- a predicate on cust_id.
+
+CREATE OR REPLACE VIEW mining_build_text AS
+SELECT *
+FROM mining_data
+WHERE cust_id between 101501 and 103000;
+
+CREATE OR REPLACE VIEW mining_test_text AS
+SELECT *
+FROM mining_data
+WHERE cust_id between 103001 and 104500;
+
+CREATE OR REPLACE VIEW mining_apply_text AS
+SELECT *
+FROM mining_data
+WHERE cust_id between 100001 and 101500;
+
+--------------------------------------------------------------------------------
+-- Build, test, and apply views
+-- Same as above, but no text - COMMENTS column removed
+
+CREATE OR REPLACE VIEW mining_data_build_v AS
+SELECT CUST_ID, CUST_GENDER, AGE, CUST_MARITAL_STATUS, COUNTRY_NAME,
+ CUST_INCOME_LEVEL, EDUCATION, OCCUPATION, HOUSEHOLD_SIZE,
+ YRS_RESIDENCE, AFFINITY_CARD, BULK_PACK_DISKETTES, FLAT_PANEL_MONITOR,
+ HOME_THEATER_PACKAGE, BOOKKEEPING_APPLICATION, PRINTER_SUPPLIES,
+ Y_BOX_GAMES, OS_DOC_SET_KANJI
+FROM mining_build_text;
+
+CREATE OR REPLACE VIEW mining_data_test_v AS
+SELECT CUST_ID, CUST_GENDER, AGE, CUST_MARITAL_STATUS, COUNTRY_NAME,
+ CUST_INCOME_LEVEL, EDUCATION, OCCUPATION, HOUSEHOLD_SIZE,
+ YRS_RESIDENCE, AFFINITY_CARD, BULK_PACK_DISKETTES, FLAT_PANEL_MONITOR,
+ HOME_THEATER_PACKAGE, BOOKKEEPING_APPLICATION, PRINTER_SUPPLIES,
+ Y_BOX_GAMES, OS_DOC_SET_KANJI
+FROM mining_test_text;
+
+CREATE OR REPLACE VIEW mining_data_apply_v AS
+SELECT CUST_ID, CUST_GENDER, AGE, CUST_MARITAL_STATUS, COUNTRY_NAME,
+ CUST_INCOME_LEVEL, EDUCATION, OCCUPATION, HOUSEHOLD_SIZE,
+ YRS_RESIDENCE, AFFINITY_CARD, BULK_PACK_DISKETTES, FLAT_PANEL_MONITOR,
+ HOME_THEATER_PACKAGE, BOOKKEEPING_APPLICATION, PRINTER_SUPPLIES,
+ Y_BOX_GAMES, OS_DOC_SET_KANJI
+FROM mining_apply_text;
+
+--------------------------------------------------------------------------------
+-- Data for one class model
+-- Only data for positive affinity card (one class) is used.
+
+CREATE OR REPLACE VIEW mining_data_one_class_v AS
+SELECT CUST_ID, CUST_GENDER, AGE, CUST_MARITAL_STATUS, COUNTRY_NAME,
+ CUST_INCOME_LEVEL, EDUCATION, OCCUPATION, HOUSEHOLD_SIZE, YRS_RESIDENCE
+FROM mining_data_build_v
+WHERE affinity_card = 1;
+
+--------------------------------------------------------------------------------
+--
+-- Create view mining_data_build_parallel_v with a parallel hint
+--
+--------------------------------------------------------------------------------
+CREATE or REPLACE VIEW mining_data_build_parallel_v AS SELECT /*+ parallel (4)*/ * FROM mining_data_build_v; 
+
+--------------------------------------------------------------------------------
+--
+-- Create view mining_data_one_class_pv with a parallel hint
+--
+--------------------------------------------------------------------------------
+CREATE or REPLACE VIEW mining_data_one_class_pv AS SELECT /*+ parallel (4)*/ * FROM mining_data_one_class_v;
+
+--------------------------------------------------------------------------------
+--
+-- Create view mining_data_test_parallel_v with a parallel hint
+--
+--------------------------------------------------------------------------------
+CREATE or REPLACE VIEW mining_data_test_parallel_v AS SELECT /*+ parallel (4)*/ * FROM mining_data_test_v;
+
+--------------------------------------------------------------------------------
+--
+-- Create view mining_data_apply_parallel_v with a parallel hint
+--
+--------------------------------------------------------------------------------
+CREATE or REPLACE VIEW mining_data_apply_parallel_v AS SELECT /*+ parallel (4)*/ * FROM mining_data_apply_v;
+
+--------------------------------------------------------------------------------
+--
+-- Create view mining_test_text_parallel with a parallel hint
+--
+--------------------------------------------------------------------------------
+CREATE or REPLACE VIEW mining_test_text_parallel AS SELECT /*+ parallel (4)*/ * FROM mining_test_text;
+

machine-learning/sql/23c/dmshgrants.sql

@@ -0,0 +1,54 @@
+-----------------------------------------------------------------------
+-- Oracle Machine Learning for SQL (OML4SQL) 23c
+-- 
+-- Setup - Grants Privileges to Users - dmshgrants.sql
+-- 
+-- Copyright (c) 2023 Oracle Corporation and/or its affilitiates.
+--
+-- The Universal Permissive License (UPL), Version 1.0
+--
+-- https://oss.oracle.com/licenses/upl/
+-----------------------------------------------------------------------
+--
+--
+-- dmshgrants.sql
+--
+-- Copyright (c) 2001, 2010, 2021, Oracle and/or its affiliates. 
+-- All rights reserved. 
+--
+-- NAME
+-- dmshgrants.sql
+--
+-- DESCRIPTION
+-- This script grants SELECT on SH tables and SYS privileges 
+-- required to run the Oracle Data Mining demo programs
+-- 
+-- The script is to be run in SYS account
+--
+-- NOTES
+-- &&1 Name of the DM user
+--
+--------------------------------------------------------------------------------
+DEFINE DMUSER = &&1 
+
+grant create session to &DMUSER
+/
+grant create table to &DMUSER
+/
+grant create view to &DMUSER
+/
+grant create mining model to &DMUSER
+/
+grant execute on ctxsys.ctx_ddl to &DMUSER
+/
+
+GRANT SELECT ON sh.customers TO &DMUSER
+/
+GRANT SELECT ON sh.sales TO &DMUSER
+/
+GRANT SELECT ON sh.products TO &DMUSER
+/
+GRANT SELECT ON sh.supplementary_demographics TO &DMUSER
+/
+GRANT SELECT ON sh.countries TO &DMUSER
+/