Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,057 changes: 648 additions & 409 deletions BREEZE.rst

Large diffs are not rendered by default.

23 changes: 22 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ Use Airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The
- [Requirements](#requirements)
- [Getting started](#getting-started)
- [Installing from PyPI](#installing-from-pypi)
- [Building customized production images](#building-customized-production-images)
- [Beyond the Horizon](#beyond-the-horizon)
- [Principles](#principles)
- [User Interface](#user-interface)
Expand All @@ -67,7 +68,7 @@ Apache Airflow is tested with:
* Sqlite - latest stable (it is used mainly for development purpose)
* Kubernetes - 1.16.2, 1.17.0

### Stable version (1.10.10)
### Stable version (1.10.11)

* Python versions: 2.7, 3.5, 3.6, 3.7
* Postgres DB: 9.6, 10
Expand Down Expand Up @@ -118,6 +119,26 @@ pip install apache-airflow[postgres,gcp]==1.10.10 \
--constraint https://raw.githubusercontent.com/apache/airflow/1.10.10/requirements/requirements-python3.7.txt
```

## Building customized production images

In order to use Airlfow in Docker Compose or Kubernetes, you might need to use or build production images
of Apache Airflow. The community provides two types of support for the production images:

* We provide pre-build relesed version of production image in PyPI build from released
sources of Apache Airflow - shortly after release. Those images are available in the DockerHub.
You can pull those images via `docker pull apache/airflow:<VERSION>-pythonX.Y` - version is the
version number (for example 1.10.11). Additionally `docker pull apache/airflow` will pull latest
stable version of the image with default python version (currently 3.6)

* In `master` branch of Airflow and in `v1-10-stable` branch we provide Dockerfiles and accompanying
files that allow to build your own customized version of the Airflow Production image. The instructions
on how to build your own image with additional dependencies (if needed) are provided in the
[IMAGES.rst](IMAGES.rst#production-images) if you want to build it using `docker build` command or in
[BREEZE.rst](BREEZE.rst#building-production-images) to use Breeze tool which easier interface,
auto-complete, and accompanying screencast video. Note, that while it is possible to use master
branch to build images for released Airflow versions, it might at times get broken so you should
rather rely on building your own images from the v1-10-stable branch.

## Beyond the Horizon

Airflow **is not** a data streaming solution. Tasks do not move data from
Expand Down
1 change: 1 addition & 0 deletions breeze-complete
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ _BREEZE_ALLOWED_POSTGRES_VERSIONS="9.6 10"
_BREEZE_ALLOWED_KIND_OPERATIONS="start stop restart status deploy test shell"

_BREEZE_ALLOWED_INSTALL_AIRFLOW_VERSIONS=$(cat <<-EOF
1.10.11
1.10.10
1.10.9
1.10.8
Expand Down
49 changes: 49 additions & 0 deletions images/breeze/add_overlay.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
#!/bin/bash

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

# shellcheck disable=SC2034
time_breeze="44:37"
time_breeze_installation=" 3:04"
time_breeze_running_tests=" 2:06"
time_breeze_select_backend_python=" 7:15"
time_breeze_using_tmux=" 2:34"
time_breeze_using_exec=" 1:35"
time_breeze_cloud_tools=" 1:55"
time_breeze_integrations=" 3:20"
time_breeze_build_images=" 1:48"
time_breeze_build_images_prod=" 1:30"
time_breeze_build_images_released_versions=" 1:30"
time_breeze_static_checks=" 1:24"
time_breeze_build_docs=" 1:04"
time_breeze_generate_requirements=" 1:37"
time_breeze_initialize_virtualenv=" 2:53"
time_breeze_kubernetes_tests=" 9:06"
time_breeze_stop=" 1:37"

for i in breeze*
do
variable_name="time_${i%.*}"
displayed_time="${!variable_name}"
convert "$i" -strokewidth 0 -fill "rgba( 220, 220, 220 , 0.5 )" \
-draw "rectangle 1620,980 1920,1080" \
-fill "rgba( 255, 255, 255 , 1 )" \
-draw "path 'M 1650,1010 L 1650,1050 L 1690,1030 Z'" \
-pointsize 54 -draw "text 1745,1050 '${displayed_time}'" \
"overlayed_${i}"
done
Binary file added images/breeze/breeze.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/breeze/breeze_build_docs.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/breeze/breeze_build_images.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/breeze/breeze_build_images_prod.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/breeze/breeze_cloud_tools.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/breeze/breeze_generate_requirements.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/breeze/breeze_initialize_virtualenv.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/breeze/breeze_installation.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/breeze/breeze_integrations.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/breeze/breeze_kubernetes_tests.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/breeze/breeze_running_tests.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/breeze/breeze_select_backend_python.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/breeze/breeze_static_checks.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/breeze/breeze_stop.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/breeze/breeze_using_exec.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/breeze/breeze_using_tmux.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/breeze/overlayed_breeze.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/breeze/overlayed_breeze_build_docs.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/breeze/overlayed_breeze_build_images.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/breeze/overlayed_breeze_cloud_tools.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/breeze/overlayed_breeze_installation.png
Binary file added images/breeze/overlayed_breeze_integrations.png
Binary file added images/breeze/overlayed_breeze_running_tests.png
Binary file added images/breeze/overlayed_breeze_stop.png
Binary file added images/breeze/overlayed_breeze_using_exec.png
Binary file added images/breeze/overlayed_breeze_using_tmux.png