Running out of memory when pushing Andromeda tables to external databases using bulk import

[schuemie]

Noticed while trying to push the concept_relationship table to Postgres. It probably has to with writing the Andromeda table to CSV prior to calling the PG bulk upload tool

[schuemie]

This is actually harder than I thought. insertTable() does some operations on the data, like (optionally) converting column names to snake_case, and inferring column types for the table creation SQL, that don't work out-of-the-box on Andromeda tables. For now, insertTable() explicitly converts Andromeda tables to data frames and throws a warning when it does.

For larger tables, for now the best option is to do the batching outside of insertTable().