tech.tablesaw.columns.strings.NoKeysAvailableException #1159
Replies: 5 comments 3 replies
-
|
The string column is dictionary encoded. It starts with a byte encoding,
and then when you have more than 255 entries, it should switch to shorts,
and then if that's not enough, it should re-encode using ints. You should
be good up to Integer.MAX_INT number of unique entries. If it's not doing
that, there is a bug. I would need to see a stack trace and code to debug.
…On Wed, Oct 12, 2022 at 6:22 AM SMVitality ***@***.***> wrote:
Our monitoring system is showing that
tech.tablesaw.columns.strings.NoKeysAvailableException are being thrown
during our application processing with the Message:
*String column can only contain 255 unique values. Column has more.*
After a bit of digging I can see that the
public StringColumn append(String value) method is handling this exception
so it doesn't bubble back up into our own code
ie -- *we never knew this was happening*
My Application is reading in a CSV file containing ~30K rows of data
My understanding was that Tablesaw would easily deal with this sort of
size of Table
??Have I mis-understood something fundamental
Thanks
—
Reply to this email directly, view it on GitHub
<#1159>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AA2FPATNOYJM67DELHOX4ELWC2GPXANCNFSM6AAAAAARDEMY54>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
To be clear, is the application crashing or just handling the exception and continuing to work? |
Beta Was this translation helpful? Give feedback.
-
|
Application continues (not crashing) The concern is that we are fooling ourselves -ie we have ONLY processed 255 rows out of ~30K |
Beta Was this translation helpful? Give feedback.
-
|
??Is the following sufficient Exception: Thanks |
Beta Was this translation helpful? Give feedback.
-
|
FYI |
Beta Was this translation helpful? Give feedback.
-
|
I don’t think this is a bug.
Perhaps you can set your monitoring to ignore handled exceptions inside
tablesaw? They are not meant for external consumption.
…On Wed, Oct 12, 2022 at 8:40 AM SMVitality ***@***.***> wrote:
FYI
tech.tablesaw
tablesaw-core
0.42.0
—
Reply to this email directly, view it on GitHub
<#1159 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AA2FPAWU4FWSDJM6EOF6OK3WC2WSHANCNFSM6AAAAAARDEMY54>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
OK thanks for taking a look |
Beta Was this translation helpful? Give feedback.
-
|
It always has. You should be able to see it in your own data.
…On Fri, Oct 14, 2022 at 2:46 AM SMVitality ***@***.***> wrote:
OK thanks for taking a look
So can you confirm that it is operating over >255 rows within the Column
object
—
Reply to this email directly, view it on GitHub
<#1159 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AA2FPAW66GXDPKERZGRXOSLWDD6UXANCNFSM6AAAAAARDEMY54>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
Our monitoring system is showing that tech.tablesaw.columns.strings.NoKeysAvailableException are being thrown during our application processing with the Message:
String column can only contain 255 unique values. Column has more.
After a bit of digging I can see that the
public StringColumn append(String value) method is handling this exception so it doesn't bubble back up into our own code
ie -- we never knew this was happening
My Application is reading in a CSV file containing ~30K rows of data
My understanding was that Tablesaw would easily deal with this sort of size of Table
??Have I mis-understood something fundamental
Thanks
Beta Was this translation helpful? Give feedback.
All reactions