Could a batchSize parameter be added to insertTable?

[katy-sadowski]

Would it be possible to add a batchSize parameter (like the one in dbWriteTable) to insertTable?

Use case: I'd like to provide the option to specify a tempEmulationSchema, and to insert data in batches.

[schuemie]

The aim of the insertTable() function is to work with as little user hand-holding across all supported platforms. It currently uses a hard-coded batch size of 10,000 rows. Is this not working for you?

[katy-sadowski]

ah! i missed that! that works for me, though maybe couldn't hurt to add it as a param with default value 10000 for more flexibility/clarity?

[schuemie]

Well, the behavior can actually be different depending on the database platform, and of course it is a whole different story when you enable bulk loading. The insertTable() function should just be smart enough to figure out how to upload your data. If you're running into cases where it's not, let me know and we can fix the behavior.

[katy-sadowski]

makes sense. it worked like a charm as-is in Snowflake - i hadn't even tried it before because i thought it wasn't possible to specify the batch size and assumed the default was 1 😂 🤦