Skip to content

Commit 13e7ffa

Browse files
SireInsectusSireInsectus
SireInsectus
authored and
SireInsectus
committed
Publishing v2.2.5
1 parent 91f0f7c commit 13e7ffa

File tree

10 files changed

+28
-31
lines changed

10 files changed

+28
-31
lines changed

‎ASP 1 - Introductions/ASP 1.1L - Explore Datasets Lab.py‎

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -28,19 +28,18 @@
2828

2929
# COMMAND ----------
3030

31-
# MAGIC %md ### 1. List data files in DBFS using magic commands
32-
# MAGIC Use a magic command to display files located in the DBFS directory: **`dbfs:/user`**
31+
# MAGIC %md ### 1. List files in DBFS using magic commands
32+
# MAGIC Use a magic command to display files located in the DBFS directory: **`dbfs:/mnt/dbacademy-users/`**
3333
# MAGIC
3434
# MAGIC <img src="https://files.training.databricks.com/images/icon_hint_32.png" alt="Hint"> You should see several user directories including your own. Depending on your permissions, you may see only your user directory.
3535

3636
# COMMAND ----------
3737

38-
# TODO
39-
<FILL_IN>
38+
# <FILL_IN>
4039

4140
# COMMAND ----------
4241

43-
# MAGIC %md ### 2. List data files in DBFS using dbutils
42+
# MAGIC %md ### 2. List files in DBFS using dbutils
4443
# MAGIC - Use **`dbutils`** to get the files at the directory above and assign it to the variable **`files`**
4544
# MAGIC - Use the Databricks display() function to display the contents in **`files`**
4645
# MAGIC
@@ -49,7 +48,7 @@
4948
# COMMAND ----------
5049

5150
# TODO
52-
files = dbutils.FILL_IN
51+
files = FILL_IN
5352
display(files)
5453

5554
# COMMAND ----------

‎ASP 2 - Spark Core/ASP 2.1 - Spark SQL.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -120,7 +120,7 @@
120120
# COMMAND ----------
121121

122122
# MAGIC %md
123-
# MAGIC Below are several additional methods we can use to create DataFrames. All of these can be found in the <a href="https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.sql.SparkSession.html" target="_blank">documentation</a> for **`SparkSession`**.
123+
# MAGIC Below are several additional methods we can use to create DataFrames. All of these can be found in the <a href="https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/spark_session.html" target="_blank">documentation</a> for **`SparkSession`**.
124124
# MAGIC
125125
# MAGIC #### **`SparkSession`** Methods
126126
# MAGIC | Method | Description |

‎ASP 2 - Spark Core/ASP 2.2 - Reader & Writer.py‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -16,12 +16,12 @@
1616
# MAGIC 1. Write DataFrame to a Delta table
1717
# MAGIC
1818
# MAGIC ##### Methods
19-
# MAGIC - <a href="https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql.html#input-and-output" target="_blank">DataFrameReader</a>: **`csv`**, **`json`**, **`option`**, **`schema`**
20-
# MAGIC - <a href="https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql.html#input-and-output" target="_blank">DataFrameWriter</a>: **`mode`**, **`option`**, **`parquet`**, **`format`**, **`saveAsTable`**
21-
# MAGIC - <a href="https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.sql.types.StructType.html#pyspark.sql.types.StructType" target="_blank">StructType</a>: **`toDDL`**
19+
# MAGIC - <a href="https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrameReader.html" target="_blank">DataFrameReader</a>: **`csv`**, **`json`**, **`option`**, **`schema`**
20+
# MAGIC - <a href="https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrameWriter.html" target="_blank">DataFrameWriter</a>: **`mode`**, **`option`**, **`parquet`**, **`format`**, **`saveAsTable`**
21+
# MAGIC - <a href="https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.types.StructType.html?highlight=structtype#pyspark.sql.types.StructType" target="_blank">StructType</a>: **`toDDL`**
2222
# MAGIC
2323
# MAGIC ##### Spark Types
24-
# MAGIC - <a href="https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql.html#data-types" target="_blank">Types</a>: **`ArrayType`**, **`DoubleType`**, **`IntegerType`**, **`LongType`**, **`StringType`**, **`StructType`**, **`StructField`**
24+
# MAGIC - <a href="https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/data_types.html" target="_blank">Types</a>: **`ArrayType`**, **`DoubleType`**, **`IntegerType`**, **`LongType`**, **`StringType`**, **`StructType`**, **`StructField`**
2525

2626
# COMMAND ----------
2727

‎Includes/_common.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
# Databricks notebook source
22
# MAGIC %pip install \
33
# MAGIC git+https://github.com/databricks-academy/dbacademy-gems@e8183eed9481624f25b34436810cf6666b4438c0 \
4-
# MAGIC git+https://github.com/databricks-academy/dbacademy-rest@9f11b96e8d7e641f6141c7da0acd90a975075a9f \
5-
# MAGIC git+https://github.com/databricks-academy/dbacademy-helper@fb5e3e2a3f367209a43fe7f7c477602c8aa28f24 \
4+
# MAGIC git+https://github.com/databricks-academy/dbacademy-rest@96351a554ed00b664e7c2d97249805df5611be06 \
5+
# MAGIC git+https://github.com/databricks-academy/dbacademy-helper@3a3a27b743283e24fe52f1f03ecc963d896f108e \
66
# MAGIC --quiet --disable-pip-version-check
77

88
# COMMAND ----------

‎Solutions/ASP 1 - Introductions/ASP 1.1L - Explore Datasets Lab.py‎

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -28,20 +28,18 @@
2828

2929
# COMMAND ----------
3030

31-
# MAGIC %md ### 1. List data files in DBFS using magic commands
32-
# MAGIC Use a magic command to display files located in the DBFS directory: **`dbfs:/user`**
31+
# MAGIC %md ### 1. List files in DBFS using magic commands
32+
# MAGIC Use a magic command to display files located in the DBFS directory: **`dbfs:/mnt/dbacademy-users/`**
3333
# MAGIC
3434
# MAGIC <img src="https://files.training.databricks.com/images/icon_hint_32.png" alt="Hint"> You should see several user directories including your own. Depending on your permissions, you may see only your user directory.
3535

3636
# COMMAND ----------
3737

38-
# ANSWER
39-
# Note: since this answer relies on the %fs magic, you'll need to remove both comment lines before this cell will run!
40-
%fs ls dbfs:/user
38+
# <FILL_IN>
4139

4240
# COMMAND ----------
4341

44-
# MAGIC %md ### 2. List data files in DBFS using dbutils
42+
# MAGIC %md ### 2. List files in DBFS using dbutils
4543
# MAGIC - Use **`dbutils`** to get the files at the directory above and assign it to the variable **`files`**
4644
# MAGIC - Use the Databricks display() function to display the contents in **`files`**
4745
# MAGIC
@@ -50,7 +48,7 @@
5048
# COMMAND ----------
5149

5250
# ANSWER
53-
files = dbutils.fs.ls("dbfs:/user")
51+
files = dbutils.fs.ls("dbfs:/mnt/dbacademy-users/")
5452
display(files)
5553

5654
# COMMAND ----------

‎Solutions/ASP 2 - Spark Core/ASP 2.1 - Spark SQL.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -120,7 +120,7 @@
120120
# COMMAND ----------
121121

122122
# MAGIC %md
123-
# MAGIC Below are several additional methods we can use to create DataFrames. All of these can be found in the <a href="https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.sql.SparkSession.html" target="_blank">documentation</a> for **`SparkSession`**.
123+
# MAGIC Below are several additional methods we can use to create DataFrames. All of these can be found in the <a href="https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/spark_session.html" target="_blank">documentation</a> for **`SparkSession`**.
124124
# MAGIC
125125
# MAGIC #### **`SparkSession`** Methods
126126
# MAGIC | Method | Description |

‎Solutions/ASP 2 - Spark Core/ASP 2.2 - Reader & Writer.py‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -16,12 +16,12 @@
1616
# MAGIC 1. Write DataFrame to a Delta table
1717
# MAGIC
1818
# MAGIC ##### Methods
19-
# MAGIC - <a href="https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql.html#input-and-output" target="_blank">DataFrameReader</a>: **`csv`**, **`json`**, **`option`**, **`schema`**
20-
# MAGIC - <a href="https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql.html#input-and-output" target="_blank">DataFrameWriter</a>: **`mode`**, **`option`**, **`parquet`**, **`format`**, **`saveAsTable`**
21-
# MAGIC - <a href="https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.sql.types.StructType.html#pyspark.sql.types.StructType" target="_blank">StructType</a>: **`toDDL`**
19+
# MAGIC - <a href="https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrameReader.html" target="_blank">DataFrameReader</a>: **`csv`**, **`json`**, **`option`**, **`schema`**
20+
# MAGIC - <a href="https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrameWriter.html" target="_blank">DataFrameWriter</a>: **`mode`**, **`option`**, **`parquet`**, **`format`**, **`saveAsTable`**
21+
# MAGIC - <a href="https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.types.StructType.html?highlight=structtype#pyspark.sql.types.StructType" target="_blank">StructType</a>: **`toDDL`**
2222
# MAGIC
2323
# MAGIC ##### Spark Types
24-
# MAGIC - <a href="https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql.html#data-types" target="_blank">Types</a>: **`ArrayType`**, **`DoubleType`**, **`IntegerType`**, **`LongType`**, **`StringType`**, **`StructType`**, **`StructField`**
24+
# MAGIC - <a href="https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/data_types.html" target="_blank">Types</a>: **`ArrayType`**, **`DoubleType`**, **`IntegerType`**, **`LongType`**, **`StringType`**, **`StructType`**, **`StructField`**
2525

2626
# COMMAND ----------
2727

‎Solutions/Includes/_common.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
# Databricks notebook source
22
# MAGIC %pip install \
33
# MAGIC git+https://github.com/databricks-academy/dbacademy-gems@e8183eed9481624f25b34436810cf6666b4438c0 \
4-
# MAGIC git+https://github.com/databricks-academy/dbacademy-rest@9f11b96e8d7e641f6141c7da0acd90a975075a9f \
5-
# MAGIC git+https://github.com/databricks-academy/dbacademy-helper@fb5e3e2a3f367209a43fe7f7c477602c8aa28f24 \
4+
# MAGIC git+https://github.com/databricks-academy/dbacademy-rest@96351a554ed00b664e7c2d97249805df5611be06 \
5+
# MAGIC git+https://github.com/databricks-academy/dbacademy-helper@3a3a27b743283e24fe52f1f03ecc963d896f108e \
66
# MAGIC --quiet --disable-pip-version-check
77

88
# COMMAND ----------

‎Solutions/Version Info.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,8 @@
1010
# MAGIC %md # Project Information
1111
# MAGIC
1212
# MAGIC * Name: **Apache Spark Programming with Databricks**
13-
# MAGIC * Version: **2.2.4**
14-
# MAGIC * Built On: **Sep 14, 2022 at 19:16:26 UTC**
13+
# MAGIC * Version: **2.2.5**
14+
# MAGIC * Built On: **Sep 23, 2022 at 18:19:26 UTC**
1515

1616
# COMMAND ----------
1717

‎Version Info.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,8 @@
1010
# MAGIC %md # Project Information
1111
# MAGIC
1212
# MAGIC * Name: **Apache Spark Programming with Databricks**
13-
# MAGIC * Version: **2.2.4**
14-
# MAGIC * Built On: **Sep 14, 2022 at 19:16:26 UTC**
13+
# MAGIC * Version: **2.2.5**
14+
# MAGIC * Built On: **Sep 23, 2022 at 18:19:26 UTC**
1515

1616
# COMMAND ----------
1717

0 commit comments

Comments
 (0)