Skip to content

Commit

Permalink
Fix typo
Browse files Browse the repository at this point in the history
  • Loading branch information
murilo committed Sep 5, 2020
1 parent 673b0b8 commit 9b270c8
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion spark-dev-env/docker-img-volume/src/IOController.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ def create_pyspark_session(app_name='my_app', aws_secrets:dict=aws_secrets) -> S
.config('spark.sql.codegen.wholeStage', False) \
.getOrCreate()

# setups AWS S3 connection
# configures AWS S3 connection
sc=spark.sparkContext
hadoop_conf=sc._jsc.hadoopConfiguration()
hadoop_conf.set("fs.s3n.impl", "org.apache.hadoop.fs.s3native.NativeS3FileSystem")
Expand Down

0 comments on commit 9b270c8

Please sign in to comment.