Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
102 changes: 102 additions & 0 deletions docs/_blog/2019-12-09-announcing-the-release-of-apache-samza--1.3.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
---
layout: blog
title: Announcing the release of Apache Samza 1.3.0
icon: git-pull-request
authors:
- name: Hai Lu
website:
image:
excerpt_separator: <!--more-->
---

<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->

# **Announcing the release of Apache Samza 1.3.0**


<!--more-->

**IMPORTANT NOTE**: We may introduce a **backward incompatible changes regarding samza job submission** in the future 1.4 release. Details can be found on [SEP-23: Simplify Job Runner](https://cwiki.apache.org/confluence/display/SAMZA/SEP-23%3A+Simplify+Job+Runner)

We’re thrilled to announce the release of Apache Samza 1.3.0.

Today Samza forms the backbone of hundreds of real-time production applications across a multitude of companies, such as LinkedIn, VMWare, Slack, Redfin among many others. Samza provides leading support for large-scale stateful stream processing with:

* First class support for local state (with RocksDB store). This allows a stateful application to scale up to 1.1 Million events/sec on a single machine with SSD.

* Support for incremental checkpointing of state instead of full snapshots. This enables Samza to scale to applications with very large state.

* A fully asynchronous programming model that makes parallelizing remote calls efficient and effortless.

* High level API for expressing complex stream processing pipelines in a few lines of code.

* Beam Samza Runner that marries Beam’s best in class support for EventTime based windowed processing and sophisticated triggering with Samza’s stable and scalable stateful processing model.

* A fully pluggable model for input sources (e.g. Kafka, Kinesis, DynamoDB streams etc.) and output systems (HDFS, Kafka, ElastiCache etc.).

* A Table API that provides a common abstraction for accessing remote or local databases and allowing developers are able to "join" an input event stream with such a Table.

* Flexible deployment model for running the applications in any hosting environment and with cluster managers other than YARN.

* Features like canaries, upgrades and rollbacks that support extremely large deployments with minimal downtime.

### **New Features, Upgrades and Bug Fixes:**
This release brings the following features, upgrades, and capabilities (highlights):

* Startpoint support improvement

* Samza SQL improvement

* Table API improvement

* Miscellaneous bug fixes

Full list of the jiras addressed in this release can be found [here](https://issues.apache.org/jira/browse/SAMZA-2354?jql=project%20%3D%20%22SAMZA%22%20and%20fixVersion%20in%20(1.3)).
### **Upgrading your application to Apache Samza 1.3.0**

### Startpoint support improvement
[SAMZA-2201](https://issues.apache.org/jira/browse/SAMZA-2201) Startpoints - Integrate fan out with job coordinators

[SAMZA-2215](https://issues.apache.org/jira/browse/SAMZA-2215) StartpointManager fix for previous CoordinatorStreamStore refactor

[SAMZA-2220](https://issues.apache.org/jira/browse/SAMZA-2220) Startpoints - Fully encapsulate resolution of starting offsets in OffsetManager

### Samza SQL improvement
[SAMZA-2234](https://issues.apache.org/jira/browse/SAMZA-2234) Samza SQL : Provide access to Samza context to the Sama SQL UDFs

[SAMZA-2313](https://issues.apache.org/jira/browse/SAMZA-2313) Samza-sql: Add validation for Samza sql statements

[SAMZA-2354](https://issues.apache.org/jira/browse/SAMZA-2354) Improve UDF discovery in samza-sql

#### Table API improvement
[SAMZA-2191](https://issues.apache.org/jira/browse/SAMZA-2191) support batching for remote tables

[SAMZA-2200](https://issues.apache.org/jira/browse/SAMZA-2200) update table sendTo() and join() operation to accept additional arguments

[SAMZA-2219](https://issues.apache.org/jira/browse/SAMZA-2219) Add a dummy table read function

[SAMZA-2309](https://issues.apache.org/jira/browse/SAMZA-2309) Remote table descriptor requires read function

#### Miscellaneous bug fixing
[SAMZA-2198](https://issues.apache.org/jira/browse/SAMZA-2198) containers process always takes task.shutdown.ms to shut down

[SAMZA-2293](https://issues.apache.org/jira/browse/SAMZA-2293) Propagate the watermark future to StreamOperatorTask correctly

### Sources downloads
A source download of Samza 1.3.0 is available [here](https://dist.apache.org/repos/dist/release/samza/1.3.0/), and is also available in Apache’s Maven repository. See Samza’s download [page](https://samza.apache.org/startup/download/) for details and Samza’s feature preview for new features.

2 changes: 1 addition & 1 deletion docs/_config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ exclude: [_docs]
baseurl: http://samza.apache.org
version: latest
# this is the version you will go if you click 'switch version' in "latest" pages.
latest-release: '1.2.0'
latest-release: '1.3.0'
collections:
menu:
output: false
Expand Down
93 changes: 93 additions & 0 deletions docs/_releases/1.3.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
---
version: '1.3'
order: 130
layout: page
menu_title: '1.3'
title: Apache Samza 1.3 <a href="/learn/documentation/1.3.0/"> [Docs] </a>
---
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->

**IMPORTANT NOTE**: We may introduce a **backward incompatible changes regarding samza job submission** in the future 1.4 release. Details can be found on [SEP-23: Simplify Job Runner](https://cwiki.apache.org/confluence/display/SAMZA/SEP-23%3A+Simplify+Job+Runner)

We’re thrilled to announce the release of Apache Samza 1.3.0.

Today Samza forms the backbone of hundreds of real-time production applications across a multitude of companies, such as LinkedIn, VMWare, Slack, Redfin among many others. Samza provides leading support for large-scale stateful stream processing with:

* First class support for local state (with RocksDB store). This allows a stateful application to scale up to 1.1 Million events/sec on a single machine with SSD.

* Support for incremental checkpointing of state instead of full snapshots. This enables Samza to scale to applications with very large state.

* A fully asynchronous programming model that makes parallelizing remote calls efficient and effortless.

* High level API for expressing complex stream processing pipelines in a few lines of code.

* Beam Samza Runner that marries Beam’s best in class support for EventTime based windowed processing and sophisticated triggering with Samza’s stable and scalable stateful processing model.

* A fully pluggable model for input sources (e.g. Kafka, Kinesis, DynamoDB streams etc.) and output systems (HDFS, Kafka, ElastiCache etc.).

* A Table API that provides a common abstraction for accessing remote or local databases and allowing developers are able to "join" an input event stream with such a Table.

* Flexible deployment model for running the applications in any hosting environment and with cluster managers other than YARN.

* Features like canaries, upgrades and rollbacks that support extremely large deployments with minimal downtime.

### **New Features, Upgrades and Bug Fixes:**
This release brings the following features, upgrades, and capabilities (highlights):

* Startpoint support improvement

* Samza SQL improvement

* Table API improvement

* Miscellaneous bug fixes

Full list of the jiras addressed in this release can be found [here](https://issues.apache.org/jira/browse/SAMZA-2354?jql=project%20%3D%20%22SAMZA%22%20and%20fixVersion%20in%20(1.3)).
### **Upgrading your application to Apache Samza 1.3.0**

### Startpoint support improvement
[SAMZA-2201](https://issues.apache.org/jira/browse/SAMZA-2201) Startpoints - Integrate fan out with job coordinators

[SAMZA-2215](https://issues.apache.org/jira/browse/SAMZA-2215) StartpointManager fix for previous CoordinatorStreamStore refactor

[SAMZA-2220](https://issues.apache.org/jira/browse/SAMZA-2220) Startpoints - Fully encapsulate resolution of starting offsets in OffsetManager

### Samza SQL improvement
[SAMZA-2234](https://issues.apache.org/jira/browse/SAMZA-2234) Samza SQL : Provide access to Samza context to the Sama SQL UDFs

[SAMZA-2313](https://issues.apache.org/jira/browse/SAMZA-2313) Samza-sql: Add validation for Samza sql statements

[SAMZA-2354](https://issues.apache.org/jira/browse/SAMZA-2354) Improve UDF discovery in samza-sql

#### Table API improvement
[SAMZA-2191](https://issues.apache.org/jira/browse/SAMZA-2191) support batching for remote tables

[SAMZA-2200](https://issues.apache.org/jira/browse/SAMZA-2200) update table sendTo() and join() operation to accept additional arguments

[SAMZA-2219](https://issues.apache.org/jira/browse/SAMZA-2219) Add a dummy table read function

[SAMZA-2309](https://issues.apache.org/jira/browse/SAMZA-2309) Remote table descriptor requires read function

#### Miscellaneous bug fixing
[SAMZA-2198](https://issues.apache.org/jira/browse/SAMZA-2198) containers process always takes task.shutdown.ms to shut down

[SAMZA-2293](https://issues.apache.org/jira/browse/SAMZA-2293) Propagate the watermark future to StreamOperatorTask correctly

### Sources downloads
A source download of Samza 1.3.0 is available [here](https://dist.apache.org/repos/dist/release/samza/1.3.0/), and is also available in Apache’s Maven repository. See Samza’s download [page](https://samza.apache.org/startup/download/) for details and Samza’s feature preview for new features.

8 changes: 8 additions & 0 deletions docs/archive/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,14 @@ <h4 id="latest">Latest Release</h4>
<li><a href="../startup/hello-samza/latest">Hello Samza</a></li>
</ul>

<h4 id="1.3">1.3 Release</h4>

<ul class="documentation-list">
<li><a href="../learn/documentation/1.3.0">Documentation</a></li>
<li><a href="../learn/tutorials/1.3.0">Tutorials</a></li>
<li><a href="../startup/hello-samza/1.3.0">Hello Samza</a></li>
</ul>

<h4 id="1.2">1.2 Release</h4>

<ul class="documentation-list">
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ Then, you can continue w/ the following command in hello-samza project:
{% highlight bash %}
mvn clean package
mkdir -p deploy/samza
tar -xvf ./target/hello-samza-1.2.0-SNAPSHOT-dist.tar.gz -C deploy/samza
tar -xvf ./target/hello-samza-1.4.0-SNAPSHOT-dist.tar.gz -C deploy/samza
{% endhighlight %}

### Run a Samza Application
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ With the environment setup complete, let us move on to building the hello-samza
{% highlight bash %}
mvn clean package
mkdir -p deploy/samza
tar -xvf ./target/hello-samza-1.3.0-SNAPSHOT-dist.tar.gz -C deploy/samza
tar -xvf ./target/hello-samza-1.4.0-SNAPSHOT-dist.tar.gz -C deploy/samza
{% endhighlight %}

We are now all set to deploy the application locally.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ Run the following commands:
{% highlight bash %}
cd samza-rest/build/distributions/
mkdir -p deploy/samza-rest
tar -xvf ./samza-rest-1.3.0-SNAPSHOT.tgz -C deploy/samza-rest
tar -xvf ./samza-rest-1.4.0-SNAPSHOT.tgz -C deploy/samza-rest
{% endhighlight %}

#### Configure the Installations Path
Expand Down
18 changes: 10 additions & 8 deletions docs/startup/download/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,15 @@ Starting from 2016, Samza will begin requiring JDK8 or higher. Please see [this

Samza tools package contains command line tools that user can run to use Samza and it's input/output systems.

* [samza-tools_2.11-1.3.0.tgz](http://www-us.apache.org/dist/samza/1.3.0/samza-tools_2.11-1.3.0.tgz)
* [samza-tools_2.11-1.2.0.tgz](http://www-us.apache.org/dist/samza/1.2.0/samza-tools_2.11-1.2.0.tgz)
* [samza-tools_2.11-1.1.0.tgz](http://www-us.apache.org/dist/samza/1.1.0/samza-tools_2.11-1.1.0.tgz)
* [samza-tools-1.0.0.tgz](http://www-us.apache.org/dist/samza/1.0.0/samza-tools-1.0.0.tgz)
* [samza-tools-0.14.1.tgz](http://www-us.apache.org/dist/samza/0.14.1/samza-tools-0.14.1.tgz)

### Source Releases

* [samza-sources-1.3.0.tgz](http://www.apache.org/dyn/closer.lua/samza/1.3.0)
* [samza-sources-1.2.0.tgz](http://www.apache.org/dyn/closer.lua/samza/1.2.0)
* [samza-sources-1.1.0.tgz](http://www.apache.org/dyn/closer.lua/samza/1.1.0)
* [samza-sources-1.0.0.tgz](http://www.apache.org/dyn/closer.lua/samza/1.0.0)
Expand Down Expand Up @@ -67,50 +69,50 @@ A Maven-based Samza project can pull in all required dependencies Samza dependen
<dependency>
<setId>org.apache.samza</setId>
<artifactId>samza-api</artifactId>
<version>1.2.0</version>
<version>1.3.0</version>
</dependency>
<dependency>
<setId>org.apache.samza</setId>
<artifactId>samza-core_2.11</artifactId>
<version>1.2.0</version>
<version>1.3.0</version>
<scope>runtime</scope>
</dependency>
<dependency>
<setId>org.apache.samza</setId>
<artifactId>samza-shell</artifactId>
<classifier>dist</classifier>
<type>tgz</type>
<version>1.2.0</version>
<version>1.3.0</version>
<scope>runtime</scope>
</dependency>
<dependency>
<setId>org.apache.samza</setId>
<artifactId>samza-yarn_2.11</artifactId>
<version>1.2.0</version>
<version>1.3.0</version>
<scope>runtime</scope>
</dependency>
<dependency>
<setId>org.apache.samza</setId>
<artifactId>samza-kv_2.11</artifactId>
<version>1.2.0</version>
<version>1.3.0</version>
<scope>runtime</scope>
</dependency>
<dependency>
<setId>org.apache.samza</setId>
<artifactId>samza-kv-rocksdb_2.11</artifactId>
<version>1.2.0</version>
<version>1.3.0</version>
<scope>runtime</scope>
</dependency>
<dependency>
<setId>org.apache.samza</setId>
<artifactId>samza-kv-inmemory_2.11</artifactId>
<version>1.2.0</version>
<version>1.3.0</version>
<scope>runtime</scope>
</dependency>
<dependency>
<setId>org.apache.samza</setId>
<artifactId>samza-kafka_2.11</artifactId>
<version>1.2.0</version>
<version>1.3.0</version>
<scope>runtime</scope>
</dependency>
{% endhighlight %}
Expand Down
2 changes: 1 addition & 1 deletion docs/startup/hello-samza/versioned/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ Then, you can continue w/ the following command in hello-samza project:
{% highlight bash %}
mvn clean package
mkdir -p deploy/samza
tar -xvf ./target/hello-samza-1.3.0-SNAPSHOT-dist.tar.gz -C deploy/samza
tar -xvf ./target/hello-samza-1.4.0-SNAPSHOT-dist.tar.gz -C deploy/samza
{% endhighlight %}

### Run a Samza Job
Expand Down