Skip to content

Commit 5ce4d39

Browse files
SireInsectusSireInsectus
SireInsectus
authored and
SireInsectus
committed
Publishing v2.0.0
1 parent 7622960 commit 5ce4d39

File tree

66 files changed

+1369
-1363
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

66 files changed

+1369
-1363
lines changed

‎Apache-Spark-Programming-with-Databricks/ASP 0 - Course Agenda.py‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -14,13 +14,12 @@
1414
# COMMAND ----------
1515

1616
# MAGIC %md
17+
# MAGIC # Day 1
1718
# MAGIC ## Introductions
1819
# MAGIC * [ASP 1.1 - Databricks Platform]($./ASP 1 - Introductions/ASP 1.1 - Databricks Platform)
1920
# MAGIC * [ASP 1.1L - Explore Datasets Lab]($./ASP 1 - Introductions/Labs/ASP 1.1L - Explore Datasets Lab)
20-
21-
# COMMAND ----------
22-
23-
# MAGIC %md
21+
# MAGIC
22+
# MAGIC
2423
# MAGIC ## Spark Core
2524
# MAGIC * [ASP 2.1 - Spark SQL]($./ASP 2 - Spark Core/ASP 2.1 - Spark SQL)
2625
# MAGIC * [ASP 2.2 - Reader & Writer]($./ASP 2 - Spark Core/ASP 2.2 - Reader & Writer)
@@ -32,6 +31,7 @@
3231
# COMMAND ----------
3332

3433
# MAGIC %md
34+
# MAGIC # Day 2
3535
# MAGIC ## Functions
3636
# MAGIC * [ASP 3.1 - Aggregation]($./ASP 3 - Functions/ASP 3.1 - Aggregation)
3737
# MAGIC * [ASP 3.1L - Revenue by Traffic Lab]($./ASP 3 - Functions/Labs/ASP 3.1L - Revenue by Traffic Lab)
@@ -47,6 +47,7 @@
4747
# COMMAND ----------
4848

4949
# MAGIC %md
50+
# MAGIC # Day 3
5051
# MAGIC ## Performance
5152
# MAGIC * [ASP 4.1 - Query Optimization]($./ASP 4 - Performance/ASP 4.1 - Query Optimization)
5253
# MAGIC * [ASP 4.2 - Partitioning]($./ASP 4 - Performance/ASP 4.2 - Partitioning)
@@ -55,15 +56,14 @@
5556
# COMMAND ----------
5657

5758
# MAGIC %md
59+
# MAGIC # Day 4
5860
# MAGIC ## Streaming
5961
# MAGIC * [ASP 5.1 - Streaming Query]($./ASP 5 - Streaming/ASP 5.1 - Streaming Query)
6062
# MAGIC * [ASP 5.1L - Coupon Sales Lab]($./ASP 5 - Streaming/Labs/ASP 5.1L - Coupon Sales Lab)
6163
# MAGIC * [ASP 5.2L - Hourly Activity by Traffic Lab]($./ASP 5 - Streaming/Labs/ASP 5.2L - Hourly Activity by Traffic Lab)
6264
# MAGIC * [ASP 5.3L - Activity by Traffic Lab]($./ASP 5 - Streaming/Labs/ASP 5.3L - Activity by Traffic Lab)
63-
64-
# COMMAND ----------
65-
66-
# MAGIC %md
65+
# MAGIC
66+
# MAGIC
6767
# MAGIC ## Delta Lake
6868
# MAGIC * [ASP 6.1 - Delta Lake]($./ASP 6 - Delta Lake/ASP 6.1 - Delta Lake)
6969
# MAGIC * [ASP 6.1L - Delta Lake Lab]($./ASP 6 - Delta Lake/Labs/ASP 6.1L - Delta Lake Lab)

‎Apache-Spark-Programming-with-Databricks/ASP 1 - Introductions/ASP 1.1 - Databricks Platform.py‎

Lines changed: 19 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -23,16 +23,16 @@
2323
# MAGIC
2424
# MAGIC
2525
# MAGIC ##### Databricks Notebook Utilities
26-
# MAGIC - <a href="https://docs.databricks.com/notebooks/notebooks-use.html#language-magic" target="_blank">Magic commands</a>: `%python`, `%scala`, `%sql`, `%r`, `%sh`, `%md`
27-
# MAGIC - <a href="https://docs.databricks.com/dev-tools/databricks-utils.html" target="_blank">DBUtils</a>: `dbutils.fs` (`%fs`), `dbutils.notebooks` (`%run`), `dbutils.widgets`
28-
# MAGIC - <a href="https://docs.databricks.com/notebooks/visualizations/index.html" target="_blank">Visualization</a>: `display`, `displayHTML`
26+
# MAGIC - <a href="https://docs.databricks.com/notebooks/notebooks-use.html#language-magic" target="_blank">Magic commands</a>: **`%python`**, **`%scala`**, **`%sql`**, **`%r`**, **`%sh`**, **`%md`**
27+
# MAGIC - <a href="https://docs.databricks.com/dev-tools/databricks-utils.html" target="_blank">DBUtils</a>: **`dbutils.fs`** **(`%fs`)**, **`dbutils.notebooks`** **(`%run`)**, **`dbutils.widgets`**
28+
# MAGIC - <a href="https://docs.databricks.com/notebooks/visualizations/index.html" target="_blank">Visualization</a>: **`display`**, **`displayHTML`**
2929

3030
# COMMAND ----------
3131

3232
# MAGIC %md ### Setup
3333
# MAGIC Run classroom setup to mount Databricks training datasets and create your own database for BedBricks.
3434
# MAGIC
35-
# MAGIC Use the `%run` magic command to run another notebook within a notebook
35+
# MAGIC Use the **`%run`** magic command to run another notebook within a notebook
3636

3737
# COMMAND ----------
3838

@@ -49,7 +49,7 @@
4949

5050
# COMMAND ----------
5151

52-
# MAGIC %md Run language specified by language magic commands: `%python`, `%scala`, `%sql`, `%r`
52+
# MAGIC %md Run language specified by language magic commands: **`%python`**, **`%scala`**, **`%sql`**, **`%r`**
5353

5454
# COMMAND ----------
5555

@@ -73,15 +73,15 @@
7373

7474
# COMMAND ----------
7575

76-
# MAGIC %md Run shell commands on the driver using the magic command: `%sh`
76+
# MAGIC %md Run shell commands on the driver using the magic command: **`%sh`**
7777

7878
# COMMAND ----------
7979

8080
# MAGIC %sh ps | grep 'java'
8181

8282
# COMMAND ----------
8383

84-
# MAGIC %md Render HTML using the function: `displayHTML` (available in Python, Scala, and R)
84+
# MAGIC %md Render HTML using the function: **`displayHTML`** (available in Python, Scala, and R)
8585

8686
# COMMAND ----------
8787

@@ -92,9 +92,9 @@
9292

9393
# MAGIC %md
9494
# MAGIC ## Create documentation cells
95-
# MAGIC Render cell as <a href="https://www.markdownguide.org/cheat-sheet/" target="_blank">Markdown</a> using the magic command: `%md`
95+
# MAGIC Render cell as <a href="https://www.markdownguide.org/cheat-sheet/" target="_blank">Markdown</a> using the magic command: **`%md`**
9696
# MAGIC
97-
# MAGIC Below are some examples of how you can use Markdown to format documentation. Click this cell and press `Enter` to view the underlying Markdown syntax.
97+
# MAGIC Below are some examples of how you can use Markdown to format documentation. Click this cell and press **`Enter`** to view the underlying Markdown syntax.
9898
# MAGIC
9999
# MAGIC
100100
# MAGIC # Heading 1
@@ -141,7 +141,7 @@
141141
# MAGIC %md ## Access DBFS (Databricks File System)
142142
# MAGIC The <a href="https://docs.databricks.com/data/databricks-file-system.html" target="_blank">Databricks File System</a> (DBFS) is a virtual file system that allows you to treat cloud object storage as though it were local files and directories on the cluster.
143143
# MAGIC
144-
# MAGIC Run file system commands on DBFS using the magic command: `%fs`
144+
# MAGIC Run file system commands on DBFS using the magic command: **`%fs`**
145145

146146
# COMMAND ----------
147147

@@ -161,7 +161,7 @@
161161

162162
# COMMAND ----------
163163

164-
# MAGIC %md `%fs` is shorthand for the <a href="https://docs.databricks.com/dev-tools/databricks-utils.html" target="_blank">DBUtils</a> module: `dbutils.fs`
164+
# MAGIC %md `%fs` is shorthand for the <a href="https://docs.databricks.com/dev-tools/databricks-utils.html" target="_blank">DBUtils</a> module: **`dbutils.fs`**
165165

166166
# COMMAND ----------
167167

@@ -195,7 +195,7 @@
195195

196196
# COMMAND ----------
197197

198-
files = dbutils.fs.ls(eventsPath)
198+
files = dbutils.fs.ls(events_path)
199199
display(files)
200200

201201
# COMMAND ----------
@@ -209,12 +209,12 @@
209209

210210
# COMMAND ----------
211211

212-
spark.sql(f"SET c.events_path = {eventsPath}")
212+
spark.sql(f"SET c.events_path = {events_path}")
213213

214214
# COMMAND ----------
215215

216216
# MAGIC %md ## Create table
217-
# MAGIC Run <a href="https://docs.databricks.com/spark/latest/spark-sql/language-manual/index.html#sql-reference" target="_blank">Databricks SQL Commands</a> to create a table named `events` using BedBricks event files on DBFS.
217+
# MAGIC Run <a href="https://docs.databricks.com/spark/latest/spark-sql/language-manual/index.html#sql-reference" target="_blank">Databricks SQL Commands</a> to create a table named **`events`** using BedBricks event files on DBFS.
218218

219219
# COMMAND ----------
220220

@@ -227,7 +227,7 @@
227227

228228
# COMMAND ----------
229229

230-
print(databaseName)
230+
print(database_name)
231231

232232
# COMMAND ----------
233233

@@ -236,7 +236,7 @@
236236
# COMMAND ----------
237237

238238
# MAGIC %md ## Query table and plot results
239-
# MAGIC Use SQL to query the `events` table
239+
# MAGIC Use SQL to query the **`events`** table
240240

241241
# COMMAND ----------
242242

@@ -268,7 +268,7 @@
268268

269269
# COMMAND ----------
270270

271-
# MAGIC %md Access the current value of the widget using the function `getArgument`
271+
# MAGIC %md Access the current value of the widget using the function **`getArgument`**
272272

273273
# COMMAND ----------
274274

@@ -288,7 +288,7 @@
288288

289289
# COMMAND ----------
290290

291-
# MAGIC %md To create widgets in Python, Scala, and R, use the DBUtils module: `dbutils.widgets`
291+
# MAGIC %md To create widgets in Python, Scala, and R, use the DBUtils module: **`dbutils.widgets`**
292292

293293
# COMMAND ----------
294294

@@ -297,7 +297,7 @@
297297

298298
# COMMAND ----------
299299

300-
# MAGIC %md Access the current value of the widget using the `dbutils.widgets` function `get`
300+
# MAGIC %md Access the current value of the widget using the **`dbutils.widgets`** function **`get`**
301301

302302
# COMMAND ----------
303303

‎Apache-Spark-Programming-with-Databricks/ASP 1 - Introductions/Labs/ASP 1.1L - Explore Datasets Lab.py‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
# MAGIC
1515
# MAGIC ### BedBricks Case Study
1616
# MAGIC This course uses a case study that explores clickstream data for the online mattress retailer, BedBricks.
17-
# MAGIC You are an analyst at BedBricks working with the following datasets: `events`, `sales`, `users`, and `products`.
17+
# MAGIC You are an analyst at BedBricks working with the following datasets: **`events`**, **`sales`**, **`users`**, and **`products`**.
1818
# MAGIC
1919
# MAGIC ##### Tasks
2020
# MAGIC 1. View data files in DBFS using magic commands
@@ -31,7 +31,7 @@
3131
# MAGIC %md ### 1. List data files in DBFS using magic commands
3232
# MAGIC Use a magic command to display files located in the DBFS directory: **`dbfs:/databricks-datasets`**
3333
# MAGIC
34-
# MAGIC <img src="https://files.training.databricks.com/images/icon_hint_32.png" alt="Hint"> You should see four items: `events`, `products`, `sales`, `users`
34+
# MAGIC <img src="https://files.training.databricks.com/images/icon_hint_32.png" alt="Hint"> You should see four items: **`events`**, **`products`**, **`sales`**, **`users`**
3535

3636
# COMMAND ----------
3737

@@ -44,7 +44,7 @@
4444
# MAGIC - Use **`dbutils`** to get the files at the directory above and save it to the variable **`files`**
4545
# MAGIC - Use the Databricks display() function to display the contents in **`files`**
4646
# MAGIC
47-
# MAGIC <img src="https://files.training.databricks.com/images/icon_hint_32.png" alt="Hint"> You should see four items: `events`, `items`, `sales`, `users`
47+
# MAGIC <img src="https://files.training.databricks.com/images/icon_hint_32.png" alt="Hint"> You should see four items: **`events`**, **`items`**, **`sales`**, **`users`**
4848

4949
# COMMAND ----------
5050

@@ -74,7 +74,7 @@
7474
# COMMAND ----------
7575

7676
# MAGIC %md ### 4. Execute SQL to explore BedBricks datasets
77-
# MAGIC Run SQL queries on the `products`, `sales`, and `events` tables to answer the following questions.
77+
# MAGIC Run SQL queries on the **`products`**, **`sales`**, and **`events`** tables to answer the following questions.
7878
# MAGIC - What products are available for purchase at BedBricks?
7979
# MAGIC - What is the average purchase revenue for a transaction at BedBricks?
8080
# MAGIC - What types of events are recorded on the BedBricks website?

0 commit comments

Comments
 (0)