[BEAM-5] Add Flink Runner#12

Closed

mxm wants to merge 149 commits intoapache:masterfrom

mxm:master

Commits on Mar 4, 2016

Initial commit

StephanEwen
authored and
mxm
committed
Update .gitignore

StephanEwen
authored and
mxm
committed
Initial stubs

StephanEwen
authored and
mxm
committed
change to latest dataflow sdk version
mxm
committed
Parallel Do Function
mxm
committed
GroupByKeyOnly
mxm
committed
change flink version to 0.8.0
mxm
committed
working WordCount and TFIDF
mxm
committed
Add WordCountITCase

aljoscha
authored and
mxm
committed
Rename Flink Runner to FlinkPipelineRunner, make it obey options

aljoscha
authored and
mxm
committed
Move type related stuff to types package. Add Javadoc

aljoscha
authored and
mxm
committed
Rename utils to wrappers, add Javadoc

aljoscha
authored and
mxm
committed
Rename package example to examples

aljoscha
authored and
mxm
committed
Move Flink Functions into package functions, normalize naming scheme

aljoscha
authored and
mxm
committed
Add comments for missing Flink Translators, some cleanup in translators

aljoscha
authored and
mxm
committed
Add warnings for features not yet implemented

aljoscha
authored and
mxm
committed
Normalize DataSet names in FlinkTransformTranslators

aljoscha
authored and
mxm
committed
Fix bug in TextIO.Read translation, parallelism not correctly set

aljoscha
authored and
mxm
committed
Add sideOutput support and ConsoleIO.Write transform

aljoscha
authored and
mxm
committed
Add translation for Combine.perKey

aljoscha
authored and
mxm
committed
Add TfIdfITCase

aljoscha
authored and
mxm
committed
Add RemoveDuplicates ITCases

aljoscha
authored and
mxm
committed
Change Create translation to serialize to byte arrays

aljoscha
authored and
mxm
committed
Add JoinExamplesITCase

aljoscha
authored and
mxm
committed
Add non-working (for-now) WikipediaTopSessionsITCase

aljoscha
authored and
mxm
committed
Properly serialize/deserialize PipelineOptions from/to JSON

aljoscha
authored and
mxm
committed
Add InputFormat wrapper for Source, along with ITCase

aljoscha
authored and
mxm
committed
Add Avro support with ITCase

aljoscha
authored and
mxm
committed
removed unused textPath variable
mxm
committed
run example with FlinkLocalRunner
mxm
committed
parse tree: print transform name
mxm
committed
add WordCountJoin test (coGroup)
mxm
committed
add WordCountJoin3Test (three dimensional reduce)
mxm
committed
optimize CoGroupByKey by using Flink's coGroup
mxm
committed
use only one switch for translating composite transforms
mxm
committed
implement CoderComperator for Flink's coGroup
mxm
committed
coGroup optimization: disable until pull request is resolved
mxm
committed
Change version to 1-SNAPSHOT

aljoscha
authored and
mxm
committed
Update Javadoc on CoderComparator and KvCoderComparator

aljoscha
authored and
mxm
committed
Change Serializers and Comparators to reuse byte arrays

aljoscha
authored and
mxm
committed
Add warning about KeyedState in DoFn, which is not supported

aljoscha
authored and
mxm
committed
Rename WordCount Example, Use Mavne Shade to bundle

aljoscha
authored and
mxm
committed
Fix CoderComparators to correctly compare key lengths

aljoscha
authored and
mxm
committed
Add dependency-reduced-pom.xml to .gitignore

aljoscha
authored and
mxm
committed
Correctly forward EOFExceptions from CoderTypeSerializer

aljoscha
authored and
mxm
committed
Fix mismatch between DataInput and InputStream in DataInputViewWrapper

aljoscha
authored and
mxm
committed
Add Fake Grouping Key for KV GroupByKey translation

aljoscha
authored and
mxm
committed
Change Flink Grouping Reducer to not materialize values

aljoscha
authored and
mxm
committed
Add Partial Reduce Translation using MapPartition

aljoscha
authored and
mxm
committed
Move to Flink 0.9-SNAPSHOT add Combine based on MapPartition

aljoscha
authored and
mxm
committed
Rename CoGroupKeyedListAggregator to match Naming Scheme

aljoscha
authored and
mxm
committed
enable CoGroupByKey translator which was disabled due to pending pull request
mxm
committed
integrate now available Google Dataflow classes
mxm
committed
integrate partialGroupReduce operator to properly optimize Combine.perKey
mxm
committed
fix bug that would incorrectly match elements in the coGroup
mxm
committed
implement a rudimentary normalized key comparator
mxm
committed
Optimize CoderSerializer and CoderComparator

aljoscha
authored and
mxm
committed
fix buffer overflow
mxm
committed
add README file
mxm
committed
add license headers
mxm
committed
change name of Flink's new GroupCombine operator
mxm
committed
properly integrate Flink's new GroupCombine operator
mxm
committed
README: remove install Flink part due to availability of maven dependency
mxm
committed
fix null value issue in Create translator
mxm
committed
KvCoderComperator: set the offset correctly
mxm
committed
KvCoderComperator: set the correct context for the encoding
mxm
committed
Maven: remove shade plugin
mxm
committed
improve execution of WordCount example
mxm
committed
README: add additional note about cluster setup
mxm
committed
Flink master changes: rename FlatCombineFunction to GroupCombineFunction
mxm
committed
adapt Flink Dataflow to run with the latest DataflowJavaSDK
mxm
committed
instead of using LATEST, pin to a version of the DataflowJavaSDK
mxm
committed
Pinned version for SDK too

mbalassi
authored and
mxm
committed
Updated Aggregator wrappers to match new Flink Accumulator behaviour

mbalassi
authored and
mxm
committed
adapt to the latest Flink master
mxm
committed
[streaming] add a hint to the streaming branch in the readme
mxm
committed
add a Travis config file to enable continuous integration
mxm
committed
travis: modify config file to adjust jdk version and test command
mxm
committed
travis: remove openjdk8 since it is not supported
mxm
committed
bump Flink version to 0.9.1
mxm
committed
change version number to 0.1
mxm
committed
remove unused serializer
mxm
committed
update to Dataflow SDK 1.0.0
mxm
committed
fix FlinkRunnerResult for new PipelineResult
mxm
committed
fix UnionCoder
mxm
committed
adapt to 1.0.0 changes (handle AppliedPTransform)
mxm
committed
fix renaming of ReadSource to Read.Bounded
mxm
committed
adapt to the API changes: get types from context
mxm
committed
adapt to new windowing semantics
mxm
committed
update ConsoleIO
mxm
committed
remove useless job
mxm
committed
fix FlinkPipelineOptions
mxm
committed
fix aggregators
mxm
committed
fix accumulators
mxm
committed
unify transform input and output getters/setters
mxm
committed
re-enable and fix Read.Bounded
mxm
committed
fix AvroITCase
mxm
committed
re-enable TfIdfITCase
mxm
committed
update UnionCoder
mxm
committed
properly translate nested composite transforms
mxm
committed
translate composite transform GroupByKey
mxm
committed
fix WordCount IT cases
mxm
committed
fix WordCount example program
mxm
committed
add missing interface methods for DoFn wrapper
mxm
committed
fix TfIdf example
mxm
committed
add SideInputITCase
mxm
committed
add MayBeEmptyITCase
mxm
committed
fix WordCount comment
mxm
committed
use correct output type
mxm
committed
disable Wikipedia test (for now)
mxm
committed
move these examples to test and convert them to test cases eventually
mxm
committed
move JoinExamples to util
mxm
committed
created it case Flattenize
mxm
committed
optimize imports
mxm
committed
create ParDoMultiOutput IT case
mxm
committed
port to Flink 0.10.0
mxm
committed
Enforce Java >= 1.7, Optimize Imports, Java 7 code compatibility

smarthi
authored and
mxm
committed
make Avro type extraction more explicit
mxm
committed
[maven] correct license and formatting
mxm
committed
[tests] refactor ReadSourceITCase
mxm
committed
[runner] add translator for Write.Bound for custom sinks
mxm
committed
[sink] generate unique id for writer initialization
mxm
committed
[readme] add a section on how to submit cluster programs
mxm
committed
[readme] add hint on how to submit jar to cluster
mxm
committed
[runner] add streaming support with checkpointing

kl0u
authored and
mxm
committed
[readme] add hint that streaming support is available
mxm
committed
[readme] update to reflect the current state
mxm
committed
[cleanup] various small improvements

smarthi
authored and
mxm
committed
[cleanup] remove obsolete code
mxm
committed
[tests] add streaming mode to TestPipeline
mxm
committed
[runner] add Create transform
mxm
committed
[tests] integrate Wikipedia session test
mxm
committed
Rearranging the code and renaming certain classes.

kl0u
authored and
mxm
committed
Adds javadocs.

kl0u
authored and
mxm
committed
Fixes Void handling

kl0u
authored and
mxm
committed
Track dataflow 1.0 -> 1.5 changes

Mark Shields
authored and
mxm
committed
[runner] some small fixes for 1.5
mxm
committed
Fixes the GroupAlsoByWindowTest.

kl0u
authored and
mxm
committed
[travis] install snapshot version of SDK before running CI
mxm
committed
[tests] suppress unnecessary log output
mxm
committed
[maven] add project for Runners
mxm
committed
[flink] adjust root pom.xml to Beam
mxm
committed
[flink] convert tabs to 2 spaces
mxm
committed
[flink] change package namespace
mxm
committed
[flink] adjust directories according to package name
mxm
committed
[flink] update license headers
mxm
committed
[flink] update README
mxm
committed
[flink] [cleanup] delete .gitignore
mxm
committed
[travis] go through all Maven phases (except deploy)
mxm
committed