parquet-cascading Add Parquet Avro Scheme

[asfimport]

At the moment using parquet-cascading writing Parquet files when using Avro as the object format is not possible as a Schema is not yet provided.

As part of this ticket, a ParquetAvroScheme will be introduced, and also a local.ParquetAvroScheme , so that from Cascading / Scalding developers can read / write and test in both --hdfs and --local mode Parquet-Avro files.

Reporter: Antonios Chalkiopoulos / @Antwnis
Assignee: Antonios Chalkiopoulos / @Antwnis

_{Note: This issue was originally created as PARQUET-198. Please see the migration documentation for further details.}

[asfimport]

Julien Le Dem / @julienledem:
There are two options for adding this support.
As Parquet-cascading will start pulling dependencies for all the parquet modules we can either:

• make all these dependencies "provided" and let users add the dependency they want (parquet-avro or parquet-thrift, ...)
• make a sub module for each: parquet-cascading-avro, parquet-cascading-thrift, ...
  CC @rdblue @isnotinvain

[asfimport]

Alex Levenson / @isnotinvain:
I'd rather do sub-modules if we can – provided dependencies are not easily discovered by users.

[asfimport]

Ryan Blue / @rdblue:
I agree, submodules sound like a better solution.

[asfimport]

Andre Schumacher:
I added a first implementation neither based on submodules nor on provided dependencies (I needed something that works right now). I agree that probably it would be best to keep the respective Avro and Thrift dependencies separate, so maybe separate submodules could be the way to go.

Would be great if somebody would have a look at my PR: https://github.com/apache/incubator-parquet-mr/pull/151

[asfimport]

Julien Le Dem / @julienledem:
It looks like there are 2 PR for this:
https://github.com/apache/incubator-parquet-mr/pull/122
https://github.com/apache/incubator-parquet-mr/pull/151
Does someone want to move it to a separate submodule?

[asfimport]

Andre Schumacher:
I did not know about PR122 but it seems the approach is a bit different. In any case I will change my PR so it will be in a separate submodule.

[asfimport]

Andre Schumacher:
Reshuffled the code into its own submodule and added a few more tests. Would be great if you could have a look.

[asfimport]

Andre Schumacher:
New link to the PR due to the incubator graduation (congrats, btw): #151