PySpark Project

Mikko Saukkoriipi

Distributed Data Infrastructure
University of Helsinki
Fall 2020

For the first data set (data-1.txt), provide the value of the median of the data set. You should provide the exact median, not an approximation, and sorting the complete data set is not an acceptable solution.
The second data set (data-2.txt) contains the matrix A. Your task is to calculate $A*A^{T}*A$ and provide the resulting matrix as your answer in the same format as the input matrix.

Name		Name	Last commit message	Last commit date
Latest commit History 18 Commits
PySpark_matrix.py		PySpark_matrix.py
PySpark_median.py		PySpark_median.py
Python_matrix.ipynb		Python_matrix.ipynb
Python_median_test.ipynb		Python_median_test.ipynb
README.md		README.md
Spark_project_report.pdf		Spark_project_report.pdf
data-1-sample.txt		data-1-sample.txt
data-2-sample.txt		data-2-sample.txt

Provide feedback