Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 100 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
version: 2
jobs:
build:
working_directory: ~/go/src/github.com/docker/swarmkit
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this use $GOPATH, or is that not setup when this is used? (otherwise we could use $HOME/go/src/.... for consistency?)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately apparently in the new circle version environment variables aren’t interpolated in the working_directory entry :|

environment:
# Needed to install go
OS: linux
ARCH: amd64
GOVERSION: 1.10.3
# Needed to install protoc
PROTOC: https://github.com/google/protobuf/releases/download/v3.5.0/protoc-3.5.0-linux-x86_64.zip

# Note(cyli): We create a tmpfs mount to be used for temporary files created by tests
# to mitigate the excessive I/O latencies that sometimes cause the tests to fail.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the latency due to files being written in the container's (CoW) filesystem, or just due to physical disk instead of memory (tmpfs) being used? Wondering if a VOLUME /some/path would be resolving the latency sufficiently.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We did this in CircleCI V1 as well due to those issues, so possibly just whatever is running the CI's disk may be too slow?

# See https://github.com/docker/swarmkit/pull/2254.

# There is no way to mount tmpfs volumes in the docker executor, so we are using
# the machine executor. However, this incur a performance overhead
# (https://discuss.circleci.com/t/using-docker-compose-in-2-0/9492/4)
# and in the future could incur additional pricing changes
# (https://circleci.com/docs/2.0/executor-types/#using-machine).

# One possible hack is the following:

# /dev/shm in the container is tmpfs although files in /dev/shm are not executable.
# If we specify TMPDIR=/dev/shm, /dev/shm will be used by our tests, which call
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what I was going to suggest, but it seems you've already tried it.

# ioutil.TempDir/ioutil.TempFile, to write temporary files.
# We can also specify GOTMPDIR=/tmp or some other non-tmpfs directory
# (see https://golang.org/doc/go1.10#goroot) - this is the directory in which the
# go tool itself will put temporarily compiled test executables, etc.

# However, using this hack still resulted in occasional WAL test failures,
# so it seems like it does not work, or there may be some other failure.
# It may be something to explore again if the penalty for using the machine
# executor becomes unacceptable.

machine: true

steps:
- checkout

# This would not be needed if we used a golang docker image
- run:
name: Install go
command: |
sudo rm -rf /usr/local/go
rm -rf "$GOPATH"
curl -fsSL -o "$HOME/go.tar.gz" "https://storage.googleapis.com/golang/go$GOVERSION.$OS-$ARCH.tar.gz"
sudo tar -C /usr/local -xzf "$HOME/go.tar.gz"

- run:
name: Output debugging information
command: |
go version
env

- run:
name: Install protoc
command: |
curl -fsSL -o "$HOME/$(basename $PROTOC)" "$PROTOC"
unzip -o "$HOME/$(basename $PROTOC)" -d "$HOME"
sudo cp -R "$HOME/include/google" /usr/local/include
sudo chmod 777 -R /usr/local/include/google
sudo cp -R "$HOME/bin/protoc" /usr/local/bin
sudo chmod 777 /usr/local/bin/protoc

- run:
name: Install test/lint dependencies
command: make setup

- run:
name: Validate dependency vendoring
command: |
git fetch origin
if test -n "`git diff --stat=1000 origin/master | grep -E \"^[[:space:]]*vendor\"`"; then
make dep-validate;
fi

# The GOPATH setting would not be needed if we used the golang docker image
- run:
name: Compile/lint/vet/protobuf validation
command: GOPATH="$HOME/go" make check binaries checkprotos

- run:
name: Run unit tests
command: |
sudo mkdir /tmpfs
sudo mount -t tmpfs tmpfs /tmpfs
sudo chown 1000:1000 /tmpfs
GOPATH="$HOME/go" TMPDIR=/tmpfs make coverage

- run:
name: Run integration tests
command: |
# TMPFS has already been set up previously in the unit test step
GOPATH="$HOME/go" TMPDIR=/tmpfs make coverage-integration

- run:
name: Push coverage info to codecov.io
command: bash <(curl -fsSL https://codecov.io/bash)
71 changes: 0 additions & 71 deletions circle.yml

This file was deleted.