Skip to content

com.google.cloud.bigquery.Job.getQueryResults fails on job that writes to destination table in different project #4934

@nfallen

Description

@nfallen

The failure occurs on a query job with a destination table in a different project than the project in which the query runs:

Builder builder = QueryJobConfiguration.newBuilder(query);
builder.setDestinationTable(
TableId.of("destproject", "mydataset", "mytable"));

Say the query runs in srcproject.

The job succeeds and the destination table of the job is correct, destproject.mydataset.mytable.

However, calling com.google.cloud.bigquery.Job.getQueryResults on the completed job leads to a failing call to tabledata.list:

Stack trace:
com.google.cloud.bigquery.BigQueryException: Not found: Table srcproject:mydataset.mytable
at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.translate(HttpBigQueryRpc.java:100) ~[google-cloud-bigquery-1.59.0.jar:1.59.0]
at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.listTableData(HttpBigQueryRpc.java:326) ~[google-cloud-bigquery-1.59.0.jar:1.59.0]
at com.google.cloud.bigquery.BigQueryImpl$17.call(BigQueryImpl.java:640) ~[google-cloud-bigquery-1.59.0.jar:1.59.0]
at com.google.cloud.bigquery.BigQueryImpl$17.call(BigQueryImpl.java:635) ~[google-cloud-bigquery-1.59.0.jar:1.59.0]
at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105) ~[gax-1.35.1.jar:1.35.1]
at com.google.cloud.RetryHelper.run(RetryHelper.java:76) ~[google-cloud-core-1.59.0.jar:1.59.0]
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50) ~[google-cloud-core-1.59.0.jar:1.59.0]
at com.google.cloud.bigquery.BigQueryImpl.listTableData(BigQueryImpl.java:634) ~[google-cloud-bigquery-1.59.0.jar:1.59.0]
at com.google.cloud.bigquery.BigQueryImpl.listTableData(BigQueryImpl.java:619) ~[google-cloud-bigquery-1.59.0.jar:1.59.0]
at com.google.cloud.bigquery.Job.getQueryResults(Job.java:312) ~[google-cloud-bigquery-1.59.0.jar:1.59.0]
...

Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 404 Not Found
{
"code" : 404,
"errors" : [ {
"domain" : "global",
"message" : "Not found: Table srcproject.mydataset.mytable",
"reason" : "notFound"
} ],
"message" : "Not found: Table srcproject.mydataset.mytable",
"status" : "NOT_FOUND"
}

It looks like tabledata.list is being called with the wrong project id. I believe this could be due to the following line of code: https://github.com/googleapis/google-cloud-java/blob/master/google-cloud-clients/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryImpl.java#L658 which looks like it will override the project id in the TableId with the user's default project.

Metadata

Metadata

Assignees

Labels

api: bigqueryIssues related to the BigQuery API.priority: p2Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions