Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Koalas and pandas read csv result is different #2206

Open
tommyhj217 opened this issue Oct 27, 2021 · 1 comment
Open

Koalas and pandas read csv result is different #2206

tommyhj217 opened this issue Oct 27, 2021 · 1 comment

Comments

@tommyhj217
Copy link

tommyhj217 commented Oct 27, 2021

I tired to read csv.
And koalas and pandas show different result.

Below is one column
"[""a"",""b""]"

Pandas returned below result
image

But koalas returned 2 columns
image

I already tried escape='"'. It also showed same result.

I thought pandas result is right so I want to get same result by koalas.
Thanks
please solve this issue

@itholic
Copy link
Contributor

itholic commented Dec 9, 2021

escape='"' measure seems like working for me??

CSV

# test.csv
col1,col2
"[""a"",""b""]"

Python code

# pandas
>>> pd.read_csv
        col1  col2
0  ["a","b"]   NaN
# Koalas
>>> ks.read_csv("test.csv", escape='"')
        col1  col2
0  ["a","b"]  None

pandas and Koalas show me the same result with escape='"'

If it's still not working, could you give me a more detailed context of your situation ??

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants