Skip to content
This repository was archived by the owner on Nov 24, 2025. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 2 additions & 79 deletions build/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,83 +18,6 @@
-->


# Rpm Build Instructions
# Build instructions

## Using `docker-compose`

We are moving toward using a docker-based build system. This eliminates the need to maintain a local installation with all the
build tools as well as ensuring that you are using the same versions as we use for testing.

These are the versions of these tools we are using:
* docker 1.12.2
* docker-compose 1.8.1

You can build from any repository/branch combination, but that repository must be available to `git clone ...`. Note that in the
following `docker-compose` commands, you can limit building to one or more sub-projects by supplying arguments at the end. If none
are supplied, then *all* will be run.

Starting at the top-level of your trafficcontrol git clone (e.g. `~/src/trafficcontrol`):

> cd infrastructure/docker/build
> docker-compose build traffic_ops_build traffic_monitor_build ...
> GITREPO=https://github.com/username/trafficcontrol BRANCH=mybranch docker-compose up traffic_ops_build traffic_monitor_build ...

The resulting `.rpm` files will be created in the `artifacts` directory.


## Building the old-fashioned way

rpm files for all sub-projects can be built using the file `build/build.sh`. If this script is given parameters, it will build only
those projects specified on the command line, e.g. `$ ./build/build.sh traffic_ops`. The prerequisites for each sub-project are
listed below.

These build scripts depend on the text in the __VERSION__ file along with the __BUILD_NUMBER__ described below to name each rpm.

The build scripts use environment variables to control how the build is done. These have sensible defaults listed below, and it is
recommended to not override them:
* __WORKSPACE__
- defaults to the top level of the traffic_control directory. The _dist_ and _rpmbuild_ directories are created in this
directory during the rpm build process.
* __BUILD_NUMBER__
- generates build number from the number of commits on the current git branch followed by the 8 character short commit hash of
the last commit on the branch.This number is used to create the rpm version, e.g. _traffic_ops.1.2.0.1723.a18e2bb7_.

At the conclusion of the build, all rpms are copied into the __$WORKSPACE/dist__ directory.

## Prerequisites for building:

### all sub-projects

* CentOS 6.x
* rpmbuild (yum install rpm-build)
* git 1.7.12 or higher

#### traffic_ops:
* go 1.7 or higher

#### traffic_stats:
* go 1.7 or higher

#### traffic_monitor and traffic_router:
* java-1.8.0-openjdk and java-1.8.0-openjdk-devel
* apache-maven 3.3.1 or higher

#### traffic_monitor_golang:
* go 1.7 or higher

#### traffic_portal
* npm (yum install npm)
* grunt (npm install -g grunt)

# Docker build instructions

__Building using `docker` is experimental at this time and has not been fully vetted.__

Dockerfiles for each sub-project are located in the build directory (e.g. `traffic_ops/build/Dockerfile`)

## Optionally set these environment variables to control the source to start with:
* `GITREPO` (default is `https://github.com/Comcast/traffic_control`) and `BRANCH` (default is master).

> export GITHUB_REPO=https://github.com/myuser/traffic_control
> export BRANCH=feature/my-new-feature
> ./build/docker-build.sh
See https://traffic-control-cdn.readthedocs.io/en/latest/development/building.html or [`/docs/source/development/building.rst`](https://github.com/apache/trafficcontrol/blob/master/docs/source/development/building.rst) for instructions for building Apache Traffic Control.
2 changes: 1 addition & 1 deletion cache-config/testing/docker/variables.env
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ CDNCONF=/opt/traffic_ops/app/conf/cdn.conf
CERT_COUNTRY=US
CERT_STATE=Colorado
CERT_CITY=Denver
CERT_COMPANY=NotComcast
CERT_COMPANY=Kabletown
DBCONF=/opt/traffic_ops/app/db/dbconf.yml
DATABASECONF=/opt/traffic_ops/app/conf/production/database.conf
POSTGRES_HOME=/usr/pgsql-13
Expand Down
2 changes: 1 addition & 1 deletion infrastructure/cdn-in-a-box/variables.env
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ X509_CA_NAME=CIAB-CA
X509_CA_COUNTRY=US
X509_CA_STATE=Colorado
X509_CA_CITY=Denver
X509_CA_COMPANY=NotComcast
X509_CA_COMPANY=Kabletown
X509_CA_ORG=CDN-in-a-Box
X509_CA_ORGUNIT=CDN-in-a-Box
X509_CA_EMAIL=no-reply@infra.ciab.test
Expand Down
8 changes: 4 additions & 4 deletions infrastructure/docker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ You can create block RAM devices on Linux with, for example, `sudo modprobe brd
Privileged containers
---------------------

The Traffic Server containers must be run with `--privileged` and `--cap-add NET_BIND_SERVICE` in order for Apache Traffic Server to bind on port 80. If you don't want to run privileged containers, you can try modifying the containers to run on another port (you will also need to update their ports in the Traffic Ops server entry, update the Traffic Ops `server_ports` parameters, re-generate the CRConfig in Traffic Ops, and re-run the `ort` script on the caches). However, there are known issues with using caches on ports other than 80. See [issue #1134](https://github.com/Comcast/traffic_control/issues/1134).
The Traffic Server containers must be run with `--privileged` and `--cap-add NET_BIND_SERVICE` in order for Apache Traffic Server to bind on port 80. If you don't want to run privileged containers, you can try modifying the containers to run on another port (you will also need to update their ports in the Traffic Ops server entry, update the Traffic Ops `server_ports` parameters, re-generate the CRConfig in Traffic Ops, and re-run the `ort` script on the caches).

Example
--------
Expand Down Expand Up @@ -112,9 +112,9 @@ docker network create cdnet

sudo docker run -it --publish 192.0.2.100:3306:3306 --name c23-to-db --hostname c23-to-db --net cdnet --env MYSQL_ROOT_PASS=secretrootpass --env IP=192.0.2.100 mysql:5.5

sudo docker run -it --publish 192.0.2.101:443:443 --name c23-to-01 --hostname c23-to-01 --net cdnet --env MYSQL_IP=c23-to-db.example.net --env MYSQL_PORT=3306 --env MYSQL_ROOT_PASS=secretrootpass --env MYSQL_TRAFFIC_OPS_PASS=supersecretpassword --env ADMIN_USER=superroot --env ADMIN_PASS=supersecreterpassward --env CERT_COUNTRY=US --env CERT_STATE=Colorado --env CERT_CITY=Denver --env CERT_COMPANY=NotComcast --env TRAFFIC_VAULT_PASS=marginallylesssecret --env IP=192.0.2.101 --env DOMAIN=c23.example.net traffic_ops:1.4
sudo docker run -it --publish 192.0.2.101:443:443 --name c23-to-01 --hostname c23-to-01 --net cdnet --env MYSQL_IP=c23-to-db.example.net --env MYSQL_PORT=3306 --env MYSQL_ROOT_PASS=secretrootpass --env MYSQL_TRAFFIC_OPS_PASS=supersecretpassword --env ADMIN_USER=superroot --env ADMIN_PASS=supersecreterpassward --env CERT_COUNTRY=US --env CERT_STATE=Colorado --env CERT_CITY=Denver --env CERT_COMPANY=Kabletown --env TRAFFIC_VAULT_PASS=marginallylesssecret --env IP=192.0.2.101 --env DOMAIN=c23.example.net traffic_ops:1.4

sudo docker run -it --publish 192.0.2.102:8088:8088 --name c23-tv-01 --hostname c23-tv-01 --net cdnet --env ADMIN_PASS=riakadminsecret --env USER_PASS=marginallylesssecret --env CERT_COUNTRY=US --env CERT_STATE=Colorado --env CERT_CITY=Denver --env CERT_COMPANY=NotComcast --env TRAFFIC_OPS_URI=https://c23-to-01.example.net --env TRAFFIC_OPS_USER=superroot --env TRAFFIC_OPS_PASS=supersecreterpassward --env DOMAIN=example.net --env IP=192.0.2.102 --env GATEWAY=192.0.2.161 traffic_vault:1.4
sudo docker run -it --publish 192.0.2.102:8088:8088 --name c23-tv-01 --hostname c23-tv-01 --net cdnet --env ADMIN_PASS=riakadminsecret --env USER_PASS=marginallylesssecret --env CERT_COUNTRY=US --env CERT_STATE=Colorado --env CERT_CITY=Denver --env CERT_COMPANY=Kabletown --env TRAFFIC_OPS_URI=https://c23-to-01.example.net --env TRAFFIC_OPS_USER=superroot --env TRAFFIC_OPS_PASS=supersecreterpassward --env DOMAIN=example.net --env IP=192.0.2.102 --env GATEWAY=192.0.2.161 traffic_vault:1.4

sudo docker run -it --publish 192.0.2.104:80:80 --name c23-atsec-01 --hostname c23-atsec-01 --net cdnet --privileged --cap-add NET_BIND_SERVICE --device /dev/ram0:/dev/ram0 --device /dev/ram1:/dev/ram1 --env TRAFFIC_OPS_URI=https://c23-to-01.example.net --env TRAFFIC_OPS_USER=superroot --env TRAFFIC_OPS_PASS=supersecreterpassward --env DOMAIN=example.net --env IP=192.0.2.104 --env GATEWAY=192.0.2.161 traffic_server_edge:1.4

Expand Down Expand Up @@ -149,7 +149,7 @@ curl -v -k -H "Content-Type: application/x-www-form-urlencoded" -H "Cookie: mojo

sudo docker run -it --publish 192.0.2.110:80:80 --publish 192.0.2.110:3333:3333 --publish 192.0.2.110:53:53 --publish 192.0.2.110:53:53/udp --name c23-tr-01 --hostname c23-tr-01 --net cdnet --env TRAFFIC_OPS_URI=https://c23-to-01.example.net --env TRAFFIC_OPS_USER=superroot --env TRAFFIC_OPS_PASS=supersecreterpassward --env TRAFFIC_MONITORS="c23-tm-01.example.net:80;c23-tm-02.example.net:80" --env ORIGIN_URI="http://c23-hotair-01.example.net" --env DOMAIN=example.net --env IP=192.0.2.110 --env GATEWAY=192.0.2.161 traffic_router:1.4

sudo docker run -it --publish 192.0.2.111:8083:8083 --publish 192.0.2.111:8086:8086 --name c23-ts-01 --hostname c23-ts-01 --net cdnet --env TRAFFIC_OPS_URI=https://c23-to-01.example.net --env TRAFFIC_OPS_USER=superroot --env TRAFFIC_OPS_PASS=supersecreterpassward --env CERT_COUNTRY=US --env CERT_STATE=Colorado --env CERT_CITY=Denver --env CERT_COMPANY=NotComcast --env DOMAIN=example.net --env IP=192.0.2.111 --env GATEWAY=192.0.2.161 traffic_stats:1.4
sudo docker run -it --publish 192.0.2.111:8083:8083 --publish 192.0.2.111:8086:8086 --name c23-ts-01 --hostname c23-ts-01 --net cdnet --env TRAFFIC_OPS_URI=https://c23-to-01.example.net --env TRAFFIC_OPS_USER=superroot --env TRAFFIC_OPS_PASS=supersecreterpassward --env CERT_COUNTRY=US --env CERT_STATE=Colorado --env CERT_CITY=Denver --env CERT_COMPANY=Kabletown --env DOMAIN=example.net --env IP=192.0.2.111 --env GATEWAY=192.0.2.161 traffic_stats:1.4

# Edge and Mid caches must be created first to add themselves as servers to Traffic Ops.
# Once Traffic Ops has all the servers and the CRConfig is generated (by the Traffic Router container), we must re-run the ort script on them.
Expand Down
2 changes: 1 addition & 1 deletion infrastructure/docker/traffic_ops/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
#
# docker run --name my-traffic-ops-mysql --hostname my-traffic-ops-mysql --net cdnet --env MYSQL_ROOT_PASSWORD=secretrootpass --detach mysql:5.5
#
# docker run --name my-traffic-ops --hostname my-traffic-ops --net cdnet --publish 443:443 --env MYSQL_IP=my-traffic-ops-mysql --env MYSQL_PORT=3306 --env MYSQL_ROOT_PASS=secretrootpass --env MYSQL_TRAFFIC_OPS_PASS=supersecretpassword --env ADMIN_USER=superroot --env ADMIN_PASS=supersecreterpassward --env CERT_COUNTRY=US --env CERT_STATE=Colorado --env CERT_CITY=Denver --env CERT_COMPANY=NotComcast --env TRAFFIC_VAULT_PASS=marginallylesssecret --env DOMAIN=cdnet --detach traffic_ops:1.5.1
# docker run --name my-traffic-ops --hostname my-traffic-ops --net cdnet --publish 443:443 --env MYSQL_IP=my-traffic-ops-mysql --env MYSQL_PORT=3306 --env MYSQL_ROOT_PASS=secretrootpass --env MYSQL_TRAFFIC_OPS_PASS=supersecretpassword --env ADMIN_USER=superroot --env ADMIN_PASS=supersecreterpassward --env CERT_COUNTRY=US --env CERT_STATE=Colorado --env CERT_CITY=Denver --env CERT_COMPANY=Kabletown --env TRAFFIC_VAULT_PASS=marginallylesssecret --env DOMAIN=cdnet --detach traffic_ops:1.5.1

FROM centos/systemd
MAINTAINER dev@trafficcontrol.apache.org
Expand Down
2 changes: 1 addition & 1 deletion infrastructure/docker/traffic_ops/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ services:
CERT_COUNTRY: US
CERT_STATE: Colorado
CERT_CITY: Denver
CERT_COMPANY: NotComcast
CERT_COMPANY: Kabletown
TRAFFIC_VAULT_PASS: tvsecret
DOMAIN: trafficops_default
ports:
Expand Down
2 changes: 1 addition & 1 deletion infrastructure/docker/traffic_server_edge/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ RUN curl -O http://traffic-control-cdn.net/downloads/trafficserver-5.3.2-599.089
RUN yum install -y trafficserver-5.3.2-599.089d585.el6.x86_64.rpm

RUN mkdir /opt/ort
RUN cd /opt/ort && curl -O https://raw.githubusercontent.com/Comcast/traffic_control/RELEASE-1.4.0-RC0/traffic_ops/bin/traffic_ops_ort.pl
RUN cd /opt/ort && curl -LO https://github.com/apache/trafficcontrol/raw/RELEASE-1.4.0-RC0/traffic_ops/bin/traffic_ops_ort.pl
RUN chmod 777 /opt/ort/traffic_ops_ort.pl

RUN curl -O http://traffic-control-cdn.net/downloads/astats_over_http-1.2-8.el6.x86_64.rpm
Expand Down
2 changes: 1 addition & 1 deletion infrastructure/docker/traffic_server_mid/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ RUN curl -O http://traffic-control-cdn.net/downloads/trafficserver-5.3.2-599.089
RUN yum install -y trafficserver-5.3.2-599.089d585.el6.x86_64.rpm

RUN mkdir /opt/ort
RUN cd /opt/ort && curl -O https://raw.githubusercontent.com/Comcast/traffic_control/RELEASE-1.4.0-RC0/traffic_ops/bin/traffic_ops_ort.pl
RUN cd /opt/ort && curl -LO https://github.com/apache/trafficcontrol/raw/RELEASE-1.4.0-RC0/traffic_ops/bin/traffic_ops_ort.pl
RUN chmod +x /opt/ort/traffic_ops_ort.pl
RUN yum install -y "perl(JSON)"
RUN curl -O http://traffic-control-cdn.net/downloads/astats_over_http-1.2-8.el6.x86_64.rpm
Expand Down
2 changes: 1 addition & 1 deletion infrastructure/docker/traffic_stats/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
# Example Build and Run:
# docker build --rm --tag traffic_stats:1.6 Traffic_Stats
#
# docker run --name my-traffic-stats --hostname my-traffic-stats --net cdnet --env TRAFFIC_OPS_URI=http://my-traffic-ops:3000 --env TRAFFIC_OPS_USER=superroot --env TRAFFIC_OPS_PASS=supersecreterpassward --env CERT_COUNTRY=US --env CERT_STATE=Colorado --env CERT_CITY=Denver --env CERT_COMPANY=NotComcast --detach traffic_stats:1.4
# docker run --name my-traffic-stats --hostname my-traffic-stats --net cdnet --env TRAFFIC_OPS_URI=http://my-traffic-ops:3000 --env TRAFFIC_OPS_USER=superroot --env TRAFFIC_OPS_PASS=supersecreterpassward --env CERT_COUNTRY=US --env CERT_STATE=Colorado --env CERT_CITY=Denver --env CERT_COMPANY=Kabletown --detach traffic_stats:1.4

FROM centos:6.6
MAINTAINER dev@trafficcontrol.apache.org
Expand Down
2 changes: 1 addition & 1 deletion infrastructure/docker/traffic_vault/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@

# Example Build and Run:
# docker build --rm --tag traffic_vault:1.6.0 traffic_vault
# docker run --name my-traffic-vault --hostname my-traffic-vault --net cdnet --env ADMIN_PASS=riakadminsecret --env USER_PASS=marginallylesssecret --env CERT_COUNTRY=US --env CERT_STATE=Colorado --env CERT_CITY=Denver --env CERT_COMPANY=NotComcast --env TRAFFIC_OPS_URI=http://my-traffic-ops:3000 --env TRAFFIC_OPS_USER=superroot --env TRAFFIC_OPS_PASS=supersecreterpassward --env DOMAIN=cdnet --detach traffic_vault:1.6.0
# docker run --name my-traffic-vault --hostname my-traffic-vault --net cdnet --env ADMIN_PASS=riakadminsecret --env USER_PASS=marginallylesssecret --env CERT_COUNTRY=US --env CERT_STATE=Colorado --env CERT_CITY=Denver --env CERT_COMPANY=Kabletown --env TRAFFIC_OPS_URI=http://my-traffic-ops:3000 --env TRAFFIC_OPS_USER=superroot --env TRAFFIC_OPS_PASS=supersecreterpassward --env DOMAIN=cdnet --detach traffic_vault:1.6.0

FROM centos:6.6
MAINTAINER dev@trafficcontrol.apache.org
Expand Down
7 changes: 3 additions & 4 deletions misc/git/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,10 @@ A collection of useful pre-commit hooks can be found in the `pre-commit-hooks` d

#### Installing pre-commit hooks

In the `github.com/Comcast/traffic_control/.git/hooks` directory, create a symbolic link to the `pre-commit` executable contained in this directory.
In the `$GOPATH/src/github.com/apache/trafficcontrol/` directory, create a symbolic link from the `pre-commit` executable in this directory to the `.git/hooks/` directory:

```bash
cd github.com/Comcast/traffic_control/.git/hooks
ln -s ../../misc/git/pre-commit .
```shell
ln -s ../../misc/git/pre-commit .git/hooks/
```

Now, all executables in the `pre-commit-hooks` directory will be run on commit.
Expand Down
19 changes: 8 additions & 11 deletions misc/git/pre-commit-hooks/01-gofmt
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,14 @@
#
# This script does not handle file names that contain spaces.

gofiles=$(git diff --cached --name-only --diff-filter=ACM | grep '.go$')
[ -z "$gofiles" ] && exit 0
unformatted_files_count="$(git ls-files '**.go' | grep -ve^vendor | xargs gofmt -l | wc -l)"
if [[ "$unformatted_files_count" == 0 ]]; then
exit 0
fi

unformatted=$(gofmt -l $gofiles)
[ -z "$unformatted" ] && exit 0

# Some files are not gofmt'd. Print message and fail.

echo >&2 "Go files must be formatted with gofmt. Please run:"
for fn in $unformatted; do
echo >&2 " gofmt -w $PWD/$fn"
done
<<GOFMT_MESSAGE cat >&2
${unformatted_files} go files must be formatted with \`go fmt\`. Please run:
go fmt ./...
GOFMT_MESSAGE

exit 1
22 changes: 5 additions & 17 deletions misc/git/pre-commit-hooks/02-govet
Original file line number Diff line number Diff line change
Expand Up @@ -21,20 +21,8 @@
# git go vet pre-commit hook
#
# Runs go vet

# Executes go vet on the non-external packages (ie packages from github)
__DIR__="$(cd "$(dirname "${0}")"; echo $(pwd))"
source "$__DIR__/pkgs"

vet=$(go vet $PKGS 2>&1)
[ $? == 0 ] && exit 0

# There are go vet errors. Print them and fail.

# remove the 'exit status 1' messages
failing=$(echo "$vet" | grep -v "exit status")

echo >&2 "'go vet' errors. Please fix the following:"
echo >&2 "$failing"

exit 1
go vet ./...
if ! go vet ./...; then
echo go vet hook failed >&2
exit 1
fi
Original file line number Diff line number Diff line change
Expand Up @@ -17,25 +17,10 @@
# specific language governing permissions and limitations
# under the License.

# git golint pre-commit hook
# golangci-lint pre-commit hook
#
# Runs golint

# Executes tests on the non-external packages (ie packages from github)
__DIR__="$(cd "$(dirname "${0}")"; echo $(pwd))"
source "$__DIR__/pkgs"

lints=""
for pkg in $PKGS
do
lints="$lints$(golint $pkg 2>&1)"
done

[ -z "$lints" ] && exit 0

# There are golint errors. Print them and fail.

echo >&2 "'golint' errors. Please fix the following:"
echo >&2 "$lints"

exit 1
# Runs golangci-lint
if ! golangci-lint run; then
echo golangci-lint hook failed >&2
exit 1
fi
45 changes: 24 additions & 21 deletions misc/git/pre-commit-hooks/99-gotest
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/bin/sh
#!/usr/bin/env bash

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
Expand All @@ -19,24 +19,27 @@

# git gotest pre-commit hook
#
# Runs go tests and also ensures that packages with no tests compile, ie
# the tests dir.
# Directories to exclude
exclude=(
cache-config/testing/ort-tests
cache-config/testing/ort-tests
.github/actions/license-file-coverage
test/router/dnssec
traffic_monitor/tests/_integration
traffic_ops/testing/api
vendor
)

# Executes tests on the non-external packages (ie packages from github)
__DIR__="$(cd "$(dirname "${0}")"; echo $(pwd))"
source "$__DIR__/pkgs"

# run tests. go test outputs compilation errors to STDERR, which is why it's redirected.
tests=$(go test $PKGS -timeout=10s 2>&1)
[ $? == 0 ] && exit 0

# There are failing tests. Packages with compilation errors start with #, failures with FAIL. Print them and fail.
failing=$(echo "$tests" | grep '^\#\|^FAIL' | awk '{print $2}' | grep -ve '^$' | sort -u)

echo >&2 "There are failing tests or compilation errors. Please fix these packages:"
for pkg in $failing
do
echo >&2 " go test -v -short $pkg"
done

exit 1
set -o pipefail;
# Runs go tests
if
! git ls-files '**.go' |
grep -vE "$(printf '^%s|' "${exclude[@]}")^$" |
xargs dirname |
uniq |
sed 's|^|./|' |
xargs go test;
then
echo go test hook failed >&2
exit 1
fi
28 changes: 0 additions & 28 deletions misc/git/pre-commit-hooks/pkgs

This file was deleted.