Skip to content

Commit 387c37d

Browse files
authored
Update avro-parquet.scala
1 parent 28d2605 commit 387c37d

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

avro-parquet.scala

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,9 @@ sqlContext.setConf("spark.sql.avro.compression.codec", "snappy")
55
sqlContext.setConf("spark.sql.avro.deflate.level", "5")
66
sqlContext.setConf("spark.sql.parquet.compression.codec", "snappy")
77

8-
val ratings_all = sqlContext.read.format("com.databricks.spark.csv").option("header", "true").option("inferSchema", "true").load("/input/ratings-all/ratings-all.csv")
8+
val ratings_all = sqlContext.read.format("com.databricks.spark.csv").option("header", "true").option("inferSchema", "true").load("/data/ratings-all/ratings-all.csv")
99

1010
//if we want that parquet/avro files are saved in a single partition we do it like this
11-
ratings_all.coalesce(1).saveAsParquetFile("/input/ratings_all_parquet")
12-
ratings_all.coalesce(1).write.format("com.databricks.spark.avro").save("/input/ratings_all_avro")
11+
ratings_all.coalesce(1).saveAsParquetFile("/data/ratings_all_parquet")
12+
ratings_all.coalesce(1).write.format("com.databricks.spark.avro").save("/data/ratings_all_avro")
1313

0 commit comments

Comments
 (0)