Collect additional metadata while writing manifets

[aokolnychyi]

In order to avoid reading manifests to get stats in #675, we need to collect additional metadata when writing manifests. The minimum required information (num added/deleted records) can be gathered easily. The main question is how to store it.

Option 1

We can store the new metadata as we store the number of added/deleted/existing files.

Benefits:

• We can always retrieve the additional information by reading manifests.

Drawbacks:

• Affects what we actually store on disk.
• Potentially increases the metadata size.

Option 2

We can introduce a serializable field of type Map<String, String> but don't store that on disk. Only instances of ManifestFile created via ManifestWriter will always contain all needed properties.

Benefits:

• The metadata on disk doesn't change.
• The metadata size stays the same.

Drawbacks

• We cannot retrieve the additional information by reading manifests.

[aokolnychyi]

@rdblue, how did you envision future extensions when you wrote GenericManifestFile and what do you think makes sense in this case?

[rdblue]

Does it make sense for ManifestFile to have stats like row-count, added-row-count, and deleted-row-count? I think those are the main ones that we care about that are not available because manifests already have file counts. I think it may make sense to add those to ManifestFile. 3 more longs aren't going to be a ton of space in the manifest list.

[aokolnychyi]

This was done.