BigQuery: provide option to omit 'insertId' for 'insert_rows' and 'insert_rows_json'.

[mryerse]

Streaming inserts to BigQuery can achieve 1GB per second but only when insertId is omitted. Currently both methods from the google-cloud-bigquery client that implement the BigQuery insertAll REST API method, insert_rows and insert_rows_json, automatically add insertId preventing anyone from making use of the higher throughput limits.

Proposing that an argument be added to one or both of these methods like "insertId=True" (defaulting to true). When false, uuid is not used to add insert IDs. Furthermore, documentation is updated to clearly articulate that omitting insertId can result in duplicate records upon retry of failed API calls according to this guide.

[tseaver]

@tswast Can you please comment?

[tswast]

This is already supported for insert_rows_json. You can explicitly pass in an iterable of Nones for the row_ids argument. Probably worth keeping this issue open to make a sample demonstrating that, though.

client.insert_rows_json(table, json_rows, row_ids=[None]*len(json_rows))

[tswast]

Oh, it also should work for insert_rows, as it passes any additional keyword arguments to insert_rows_json.

[tseaver]

@tswast That seems like an odd DX to have to document / defend. Instead, maybe we could special-case row_ids=None to expand to that? E.g.:

client.insert_rows_json(table, json_rows, row_ids=None)

later

Ugh, backward compat would require changing the default value for row_ids to some marker object, which would then expand to UUIDs as currently happens for None. Or else we would need to define a marker object which signalled "expand to array of None".

[tswast]

It's only a handful of people who'll want to trade more duplicate rows to get to 1 GB of writes per second. I think the current default of populating row IDs with UUID when row_id=None is the correct choice.

We could potentially make a special marker value, but I don't see all that much benefit. At a minimum, we should do the following, though.

• Expand our unit tests to ensure that a sequence of None values continues to work for the insert_rows*() methods.
• Add a code sample demonstrating use of insert_rows_json with row_ids=[None]*len(rows) for use on the page where we document how to achieve the higher 1GB / s write limit.