Error using package in Databricks

[zakipatel]

I am getting the following error:

'module' object has no attribute 'view keys

I am running python 2.7.10, and installed using pip install spark-df-profiling in Databricks (Spark 2.0)

I am able to import the module, but when I pass a data frame to the profiler, i get the above error (see attached)

[julioasotodv]

Hi zakipatel:

Yes, you're right. It looks like the problem is that the version of the default six library installed in Databricks is very old:

import six
six.__version__

Still, I don't know whether you can upgrade the version of a Python package in Databricks...

[julioasotodv]

According to Databricks docs, you just have to create a new library, and in the Pypi name write six==1.10.0. Then re-attach your notebook to the cluster (either that or restart it), and it will work.

On the other hand, I am looking how can be HTML displayed nicely in Databricks Notebooks...

This should work for you:

import spark_df_profiling
df = sqlContext.createDataFrame([["2",True,None,"8"],
                                 ["2",False,None,"8"],
                                 ["2",True,"5","7"]], ["a","b","c","d"])
rep = spark_df_profiling.ProfileReport(df)
displayHTML(rep.html)

But since JQuery is not being loaded, it doesn't look good at all. I will try to reach Databricks guys and come back with a solution...

[zakipatel]

Thanks! I created the new library as you instructed, and the error is resolved. The displayHTML also works, however, the output is not as nicely formatted, and the toggle details does not work. There is a a way to make javaScript libraries available to DisplayHTML in Databricks (either by putting the library on the FileStore or using https in the source link for the Jquery lib import).

Is there a recommended way to resolve the UI layout in Databricks that I can try?