Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

executing .py files #3

Open
MeghaAgarwal06 opened this issue Jul 16, 2021 · 0 comments
Open

executing .py files #3

MeghaAgarwal06 opened this issue Jul 16, 2021 · 0 comments

Comments

@MeghaAgarwal06
Copy link

Hi Sir, @LearningJournal
I have the clone of the project and want to execute the .py files using spark-submit command , but facing an error. Can you guide me through on how can I execute these scripts in spark env.

command:./bin/spark-submit /home/ubuntu/Spark-Programming-In-Python-master/01-HelloSpark/HelloSpark.py

the error logs are pasted below for reference:

21/07/16 14:01:56 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Traceback (most recent call last):
File "/home/ubuntu/anaconda3/lib/python3.8/configparser.py", line 846, in items
d.update(self._sections[section])
KeyError: 'SPARK_APP_CONFIGS'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/ubuntu/Spark-Programming-In-Python-master/01-HelloSpark/HelloSpark.py", line 7, in
conf = get_spark_app_config()
File "/home/ubuntu/Spark-Programming-In-Python-master/01-HelloSpark/lib/utils.py", line 25, in get_spark_app_config
for (key, val) in config.items("SPARK_APP_CONFIGS"):
File "/home/ubuntu/anaconda3/lib/python3.8/configparser.py", line 849, in items
raise NoSectionError(section)
configparser.NoSectionError: No section: 'SPARK_APP_CONFIGS'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant