-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for product types #251
Conversation
Codecov Report
@@ Coverage Diff @@
## main #251 +/- ##
==========================================
+ Coverage 97.28% 97.29% +0.01%
==========================================
Files 58 58
Lines 1028 1107 +79
Branches 10 10
==========================================
+ Hits 1000 1077 +77
- Misses 28 30 +2
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
429f150
to
50d5dd2
Compare
@alfonsorr @eruizalo los dos |
Description
The main motivation for this PR is increasing the support for product types (i.e. case classes) in doric. This mainly includes:
User
instance from a string containing name and age).RowColumns
).Some other changes in the
LiteralSparkType
andSparkType
type classes have been also included. For instance:nullable
field. This is for ensuring that the datatype calculated by doric (statically, through implicits), matches exactly the datatype calculated reflectively by Spark.DateType
,TimeStampType
, ...typedlit
in order to serialize product types (and combinations thereof) out-of-the-box.Related Issues
Resolves:
Partially solves:
Related but NOT SOLVED by this PR:
How Has This Been Tested?
Tests for Spark and Literal Spark types have been divded into four major specs: