Skip to content

BigQuery: Allow partial schema in load_table_from_dataframe #8140

@tswast

Description

@tswast

As discussed in #8105 (comment) and implemented in pandas-gbq at googleapis/python-bigquery-pandas#218, often times the default schema detection works well enough for pandas -> BQ, and only one or two of many columns need to be overridden.

Example:

df = pandas.DataFrame({... lots of columns ..., "needs_manual_schema": [...]})
job_config = bigquery.LoadJobConfig(schema=[
    bigquery.SchemaField("needs_manual_schema", "BOOLEAN"),
])
client.load_table_from_dataframe(
    df, "my_dataset.my_table", job_config=job_config,
)
# All columns from df are uploaded, but only df["needs_manual_schema"] is
# coerced into a specific type (BOOLEAN).

Metadata

Metadata

Assignees

Labels

api: bigqueryIssues related to the BigQuery API.type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions