[BEAM-9795] Support custom avro DatumWriters when writing to BigQuery

[steveniemitz]

There are cases where we'd like to further control the user record -> avro process. For example, I might not want to write generic records, but instead use my own DatumWriter instance to do the avro writing.

This PR simply allows further customizing the AvroRowWriter that is created during pipeline construction, allowing the user to hook into the DatumWriter construction.

R: @pabloem @chamikaramj

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Choose reviewer(s) and mention them in a comment (R: @username).
• Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

Post-Commit Tests Status (on master branch)

Lang	SDK	Apex	Dataflow	Flink	Gearpump	Samza	Spark
Go		---	---		---	---
Java
Python		---			---	---
XLang	---	---	---		---	---

Pre-Commit Tests Status (on master branch)

---	Java	Python	Go	Website
Non-portable
Portable	---		---	---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

[steveniemitz]

hm, looks like the precommit is failing with:

13:01:36 FAILURE: Build failed with an exception.
13:01:36 
13:01:36 * What went wrong:
13:01:36 Execution failed for task ':sdks:java:container:generateThirdPartyLicenses'.
13:01:36 > Process 'command './sdks/java/container/license_scripts/license_script.sh'' finished with non-zero exit value 1

which hopefully my change didn't break ;)

[pabloem]

Hi Steve! thanks for the feature. I'll take a look hopefully by friday

[steveniemitz]

thanks!

[steveniemitz]

bump?

[pabloem]

ah yes sorry Steve. Thanks for the contribution. This looks good to me. I'll just rerun precomits to get a green run, and run postcommits as well.

[pabloem]

Run Java PostCommit

[pabloem]

Run Java PreCommit

[steveniemitz]

thanks! I'm going to have a similar PR for AvroIO coming up, do you know who's best to review that?

edit: hmm looks like this pre-commit broke again w/ the license error, do I need to rebase this?

[pabloem]

since it's only that task that failed, we can merge anyway.

For AvroIO - I think @iemejia may be a good candidate.

[pabloem]

thanks @steveniemitz !