From 1b40b8599b3fc034e610d51afbf1d7caa2fffff3 Mon Sep 17 00:00:00 2001 From: bfallonf Date: Mon, 25 May 2015 16:25:27 +1000 Subject: [PATCH 1/4] initial edits --- architecture/core_objects/builds.adoc | 88 ++++++++++++++------------- 1 file changed, 47 insertions(+), 41 deletions(-) diff --git a/architecture/core_objects/builds.adoc b/architecture/core_objects/builds.adoc index 4c8629419785..fed4d7dd39b0 100644 --- a/architecture/core_objects/builds.adoc +++ b/architecture/core_objects/builds.adoc @@ -70,7 +70,7 @@ Speed:: Adding layers as part of a *_Dockerfile_* can be slow. With STI, the assemble process can perform a large number of complex operations without creating a new layer at each step. In addition, STI scripts can be written to re-use dependencies stored in a previous version of the application image rather -than re-downloading them each time the build is run. +than having to download them each time the build is run. Patchability:: If an underlying image needs to be patched due to a security issue, OpenShift can use STI to rebuild your application on top of the patched @@ -80,11 +80,11 @@ Operational efficiency:: By restricting build operations instead of allowing arbitrary actions such as in a *_Dockerfile_*, the PaaS operator can avoid accidental or intentional abuses of the build system. -Operational security:: Allowing users to build arbitrary an *_Dockerfile_* -exposes the host system to root privilege escalation by a malicious user because -the entire docker build process is run as a user with docker privileges. STI -restricts the operations performed as a root user, and can run the scripts as an -individual user +Operational security:: Building an arbitrary *_Dockerfile_* exposes the host +system to root privilege escalation by a malicious user because the entire +docker build process is run as a user with docker privileges. STI restricts the +operations performed as a root user, and can run the scripts as an individual +user. User efficiency:: STI prevents developers from falling into a trap of performing arbitrary `yum install` type operations during their application build, which @@ -95,15 +95,17 @@ leverage for your applications. [#custom-build] == Custom Build -Custom builds are the most sophisticated version of builds, allowing developers to define a builder image which is responsible for the entire process of the build. The custom builder image is a plain Docker image within which the author embeds the logic of the desired build process, such as building RPMs or building -base Docker images. +Custom builds allow developers to define the builder image that is responsible +for the entire process of the build. The custom builder image is a plain Docker +image were the author embeds the logic of the desired build process, such as +building RPMs or building base Docker images. [#using-docker-credentials-for-pushing-and-pulling-images] == Using Docker Credentials for Pushing and Pulling Images -In case you want to push the output image into a private Docker Registry or pull -the builder image from the private Docker Registry that requires authentication -or Docker Hub, you have to supply the `.dockercfg` file with valid Docker +In order to push the output image into a private Docker Registry or pull the +builder image from the private Docker Registry that requires authentication or +Docker Hub, supply the `.dockercfg` file with valid Docker Registry credentials. The *_.dockercfg_* JSON file usually exists in your home directory and has @@ -125,14 +127,14 @@ following format: <3> Email address for the login. ==== -You can define multiple Docker registries entries in this file. You can also add -authentication entries to this file by running the `docker login` command. The -file will be created if it does not exist. +You can define multiple Docker registry entries in this file. Alternatively, you +can also add authentication entries to this file by running the `docker login` +command. The file will be created if it does not exist. Kubernetes provides the https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/design/secrets.md[Secret] -resource, which you can use to store your passwords and configuration. You must -create the `*Secret*` first before builds can use your *_.dockercfg_* file for +resource, which you can use to store your configuration and passwords. You must +first create the `*Secret*` before builds can use your *_.dockercfg_* file for pushing the output image. The `*data*` field for the `*Secret*` object must contain the `*dockercfg*` key with the value set to the base64-encoded content of the *_.dockercfg_* file. For example: @@ -154,8 +156,7 @@ of the *_.dockercfg_* file. For example: ---- ==== -To create the `*Secret*` from a *_secret.json_* file, for example, you can use -the following command: +To create the `*Secret*` from a *_secret.json_* file: ==== @@ -168,7 +169,8 @@ Once you have the `*Secret*` created, you can add a `PushSecret` field into `Output` section inside the `BuildConfig` and set it to the name of the `*Secret*` that you created, in this case `*dockerhub*`. -``` +==== +---- "parameters": { "output": { "to": { @@ -179,12 +181,14 @@ Once you have the `*Secret*` created, you can add a `PushSecret` field into } } } -``` +---- +==== For pulling the builder Docker image from a private Docker registry, you have to specify the `PullSecret` field, which is part of the build strategy definition: -``` +==== +---- { "strategy": { "stiStrategy": { @@ -199,16 +203,17 @@ the `PullSecret` field, which is part of the build strategy definition: "type": "STI" } } -``` +---- +==== [#using-private-repositories-for-builds] == Using private repositories for builds -If you want to build your application from a private repository you must -supply valid credentials. Currently only SSH key based authentication is supported. -The keys used to access that repository live in `$HOME/.ssh/` directory. -They are usually named `id_dsa.pub`, `id_ecdsa.pub`, `id_ed25519.pub` or `id_rsa.pub`. -If you don't have any, you can generate them with the following command: +To build your application from a private repository you must supply valid +credentials. Currently only SSH key based authentication is supported. The keys +used to access that repository are in the `$HOME/.ssh/` directory, and are +usually named `id_dsa.pub`, `id_ecdsa.pub`, `id_ed25519.pub` or `id_rsa.pub`. +You can generate SSH key credentials with the following command: ==== @@ -217,18 +222,18 @@ $ ssh-keygen -t rsa -C "your_email@example.com" ---- ==== -Once that's done you're going to get two files: one containing your public key (as -explained above) and one containing a corresponding private key (one of `id_dsa`, `id_ecdsa`, -`id_ed25519` or `id_rsa`). With both of these in place you should consult your -source control management (SCM) system's manual on how to upload the public key. -The private one will be used to access your private repository. +Two files will be created: the public key (as explained above) and a +corresponding private key (one of `id_dsa`, `id_ecdsa`, `id_ed25519` or +`id_rsa`). With both of these in place you should consult your source control +management (SCM) system's manual on how to upload the public key. The private +key will be used to access your private repository. Kubernetes provides the https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/design/secrets.md[Secret] -resource, which you can use to store your keys. You must create the `*Secret*` first -before builds can use your ssh key for accessing your private repository. -The `*data*` field for the `*Secret*` object must contain your private key -with the value set to the base64-encoded content of that file. For example: +resource, which you can use to store your keys. You must create the `*Secret*` +first before builds can use your ssh key for accessing your private repository. +The `*data*` field for the `*Secret*` object must contain your private key with +the value set to the base64-encoded content of that file. For example: ==== @@ -238,8 +243,8 @@ $ base64 -w 0 $HOME/.ssh/id_rsa ---- ==== -Copy the value returned from `base64` command and paste it into `ssh-privatekey` field -in `*_secret.json_*` file: +Copy the value returned from the above command and paste it into the +`ssh-privatekey` field in `*_secret.json_*` file: ==== @@ -258,7 +263,8 @@ in `*_secret.json_*` file: ---- ==== -You can then create the `*Secret*` from the *_secret.json_* file using the following command: +You can then create the `*Secret*` from the *_secret.json_* file using the +following command: ==== @@ -267,8 +273,8 @@ $ osc create -f secret.json ---- ==== -Once you have the `*Secret*` created, you can add a `SourceSecret` field into -`Source` section inside the `BuildConfig` and set it to the name of the +Once you have created the `*Secret*`, you can add a `SourceSecret` field into +the `Source` section inside the `BuildConfig` and set it to the name of the `*Secret*` that you created, in this case `*scmsecret*`: ==== From 8915cbc2037accda680149ee6a5e05f8de3b874b Mon Sep 17 00:00:00 2001 From: bfallonf Date: Tue, 26 May 2015 11:52:51 +1000 Subject: [PATCH 2/4] edits to architecture/core_objects/builds file --- architecture/core_objects/builds.adoc | 164 +++++++++++++------------- 1 file changed, 83 insertions(+), 81 deletions(-) diff --git a/architecture/core_objects/builds.adoc b/architecture/core_objects/builds.adoc index fed4d7dd39b0..a6809d18b579 100644 --- a/architecture/core_objects/builds.adoc +++ b/architecture/core_objects/builds.adoc @@ -10,11 +10,16 @@ toc::[] == Overview -A build is a process of transforming input parameters, typically source code, into a resulting object, typically a runnable image. +A build is the end result of transforming input parameters, typically source code, into a resulting object, typically a runnable image. + +For a list of build commands, see the +link:../../dev_guide/builds.html[Developer's Guide]. == BuildConfig -The `BuildConfig` object is the definition of the entire build process. It consists of the following elements: +The `BuildConfig` object is the definition of the entire build process, and is dictated by link:../../dev_guide/builds.html#build-triggers[build triggers]. + +//// [horizontal] triggers:: Define policies used for automatically invoking builds: GitHub webhooks::: GitHub specific webhooks that specify which repository @@ -34,6 +39,9 @@ details. output::: Describes the resulting image name, tag, and registry to which the image should be pushed. resources::: Describes the resource limits (cpu, memory) that should be used by pods that run a build. +//// + +== Build Strategies There are three available link:openshift_model.html#build-strategies[build strategies]: @@ -41,74 +49,70 @@ There are three available link:openshift_model.html#build-strategies[build strat * link:#sti-build[STI build] * link:#custom-build[Custom build] -The resulting object depends on which builder is used to create the image. +The resulting object depends on the builder used to create the image. [#docker-build] -== Docker Build -Docker builds invoke the plain https://docs.docker.com/reference/commandline/cli/#build[docker build] command, and therefore expect a repository with a `Dockerfile` and all required directories. +=== Docker Build +Docker builds invoke the plain https://docs.docker.com/reference/commandline/cli/#build[docker build command], and therefore expect a repository with a *_Dockerfile_* and all required directories. [#sti-build] -== STI Build +=== STI Build link:../../creating_images/sti.html[Source-to-image (STI)] is a tool for building reproducible Docker images. It produces ready-to-run images by injecting a user source into a docker image and assembling a new docker image. The new image incorporates the base image and built source, and is ready to use -with the `docker run` command. STI supports incremental builds which re-use +with the `docker run` command. STI supports incremental builds, which re-use previously downloaded dependencies, previously built artifacts, etc. -*What are the goals of STI?* +The advantages of STI include: [horizontal] -Image flexibility:: STI allows you to use almost any existing Docker image as -the base for your application. STI scripts can be written to layer application -code onto almost any existing Docker image, so you can take advantage of the -existing ecosystem. Note that currently STI relies on `tar` and `untar` to -inject application source so the image needs to be able to process tarred -content. - -Speed:: Adding layers as part of a *_Dockerfile_* can be slow. With STI, the -assemble process can perform a large number of complex operations without -creating a new layer at each step. In addition, STI scripts can be written to -re-use dependencies stored in a previous version of the application image rather -than having to download them each time the build is run. - -Patchability:: If an underlying image needs to be patched due to a security -issue, OpenShift can use STI to rebuild your application on top of the patched -builder image. +Image flexibility:: STI scripts can be written to layer application code onto +almost any existing Docker image, taking advantage of the existing ecosystem. +Note that, currently, STI relies on `tar` and `untar` to inject application +source, so the image needs to be able to process tarred content. + +Speed:: With STI, the assemble process can perform a large number of complex +operations without creating a new layer at each step, resulting in a fast +process. In addition, STI scripts can be written to re-use dependencies stored +in a previous version of the application image rather than having to download +them each time the build is run. + +Patchability:: If an underlying image needs a patch due to a security issue, +OpenShift can use STI to rebuild your application on top of the patched builder +image. Operational efficiency:: By restricting build operations instead of allowing -arbitrary actions such as in a *_Dockerfile_*, the PaaS operator can avoid +arbitrary actions, such as in a *_Dockerfile_*, the PaaS operator can avoid accidental or intentional abuses of the build system. Operational security:: Building an arbitrary *_Dockerfile_* exposes the host -system to root privilege escalation by a malicious user because the entire -docker build process is run as a user with docker privileges. STI restricts the -operations performed as a root user, and can run the scripts as an individual -user. +system to root privilege escalation. This can be exploited by a malicious user +because the entire docker build process is run as a user with docker privileges. +STI restricts the operations performed as a root user, and can run the scripts +as an individual user. -User efficiency:: STI prevents developers from falling into a trap of performing -arbitrary `yum install` type operations during their application build, which -would result in slow development iteration. +User efficiency:: STI prevents developers from performing arbitrary `yum +install` type operations during their application build, which results in slow +development iteration. -Ecosystem:: Encourages a shared ecosystem of images with best practices you can -leverage for your applications. +Ecosystem:: STI encourages a shared ecosystem of images where you can leverage +best practices for your applications. [#custom-build] -== Custom Build -Custom builds allow developers to define the builder image that is responsible -for the entire process of the build. The custom builder image is a plain Docker -image were the author embeds the logic of the desired build process, such as -building RPMs or building base Docker images. +=== Custom Build +Custom builds allow developers to define the specific builder image responsible +for the entire build process. The custom builder image is a plain Docker image +with embedded build process logic, such as building RPMs or building base Docker +images. [#using-docker-credentials-for-pushing-and-pulling-images] == Using Docker Credentials for Pushing and Pulling Images -In order to push the output image into a private Docker Registry or pull the -builder image from the private Docker Registry that requires authentication or -Docker Hub, supply the `.dockercfg` file with valid Docker -Registry credentials. +Supply the `.dockercfg` file with valid Docker Registry credentials in order to push the output image into a private Docker Registry or pull the +builder image from the private Docker Registry that requires authentication. -The *_.dockercfg_* JSON file usually exists in your home directory and has +The *_.dockercfg_* JSON file exists in your home directory by default and has following format: ==== @@ -117,7 +121,7 @@ following format: { "https://index.docker.io/v1/": { <1> "auth": "YWRfbGzhcGU6R2labnRib21ifTE=", <2> - "email": "foo@bar.com" <3> + "email": "user@example.com" <3> } } ---- @@ -133,11 +137,19 @@ command. The file will be created if it does not exist. Kubernetes provides the https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/design/secrets.md[Secret] -resource, which you can use to store your configuration and passwords. You must +resource, which is used to store your configuration and passwords. You must first create the `*Secret*` before builds can use your *_.dockercfg_* file for -pushing the output image. The `*data*` field for the `*Secret*` object must -contain the `*dockercfg*` key with the value set to the base64-encoded content -of the *_.dockercfg_* file. For example: +pushing the output image: + +==== +---- +$ osc create -f secret.json +---- +==== + +The `*data*` field for the `*Secret*` object must contain the `*dockercfg*` key +with the value set to the base64-encoded content of the *_.dockercfg_* file. For +example: ==== @@ -156,18 +168,9 @@ of the *_.dockercfg_* file. For example: ---- ==== -To create the `*Secret*` from a *_secret.json_* file: - -==== - ----- -$ osc create -f secret.json ----- -==== - -Once you have the `*Secret*` created, you can add a `PushSecret` field into -`Output` section inside the `BuildConfig` and set it to the name of the -`*Secret*` that you created, in this case `*dockerhub*`. +Once you have the `*Secret*` created, you can add a `PushSecret` field into the +`Output` section of the `BuildConfig` and set it to the name of the `*Secret*` +that you created, which in the above example is `*dockerhub*`: ==== ---- @@ -184,8 +187,8 @@ Once you have the `*Secret*` created, you can add a `PushSecret` field into ---- ==== -For pulling the builder Docker image from a private Docker registry, you have to specify -the `PullSecret` field, which is part of the build strategy definition: +Pull the builder Docker image from a private Docker registry by specifying the +`PullSecret` field, which is part of the build strategy definition: ==== ---- @@ -207,13 +210,13 @@ the `PullSecret` field, which is part of the build strategy definition: ==== [#using-private-repositories-for-builds] -== Using private repositories for builds +== Using Private Repositories for Builds -To build your application from a private repository you must supply valid -credentials. Currently only SSH key based authentication is supported. The keys -used to access that repository are in the `$HOME/.ssh/` directory, and are -usually named `id_dsa.pub`, `id_ecdsa.pub`, `id_ed25519.pub` or `id_rsa.pub`. -You can generate SSH key credentials with the following command: +Supply valid credentials to build an application from a private repository. +Currently, only SSH key based authentication is supported. The repository keys +are located in the `$HOME/.ssh/` directory, and are named `id_dsa.pub`, +`id_ecdsa.pub`, `id_ed25519.pub` or `id_rsa.pub` by default. Generate SSH key +credentials with the following command: ==== @@ -228,12 +231,12 @@ corresponding private key (one of `id_dsa`, `id_ecdsa`, `id_ed25519` or management (SCM) system's manual on how to upload the public key. The private key will be used to access your private repository. -Kubernetes provides the +The https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/design/secrets.md[Secret] -resource, which you can use to store your keys. You must create the `*Secret*` -first before builds can use your ssh key for accessing your private repository. -The `*data*` field for the `*Secret*` object must contain your private key with -the value set to the base64-encoded content of that file. For example: +resource is used to store your keys. Create the `*Secret*` first before using +the ssh key to access the private repository. The `*data*` field for the +`*Secret*` object must contain your private key with the value set to the +base64-encoded content of that file: ==== @@ -243,7 +246,7 @@ $ base64 -w 0 $HOME/.ssh/id_rsa ---- ==== -Copy the value returned from the above command and paste it into the +Copy the value returned from the above command and place it into the `ssh-privatekey` field in `*_secret.json_*` file: ==== @@ -263,19 +266,18 @@ Copy the value returned from the above command and paste it into the ---- ==== -You can then create the `*Secret*` from the *_secret.json_* file using the -following command: +Then, create the `*Secret*` from the *_secret.json_* file using the following +command: ==== - ---- $ osc create -f secret.json ---- ==== -Once you have created the `*Secret*`, you can add a `SourceSecret` field into -the `Source` section inside the `BuildConfig` and set it to the name of the -`*Secret*` that you created, in this case `*scmsecret*`: +Add a `SourceSecret` field into the `Source` section inside the `BuildConfig` +and set it to the name of the `*Secret*` that you created, in this case +`*scmsecret*`: ==== From e5c1516ca481685ce66cb1ee52e5ec897978c7d4 Mon Sep 17 00:00:00 2001 From: bfallonf Date: Mon, 1 Jun 2015 16:31:14 +1000 Subject: [PATCH 3/4] added bc example --- architecture/core_objects/builds.adoc | 80 ++++++++++++++++++++------- 1 file changed, 60 insertions(+), 20 deletions(-) diff --git a/architecture/core_objects/builds.adoc b/architecture/core_objects/builds.adoc index a6809d18b579..4f0be960ed71 100644 --- a/architecture/core_objects/builds.adoc +++ b/architecture/core_objects/builds.adoc @@ -16,30 +16,70 @@ For a list of build commands, see the link:../../dev_guide/builds.html[Developer's Guide]. == BuildConfig -The `BuildConfig` object is the definition of the entire build process, and is dictated by link:../../dev_guide/builds.html#build-triggers[build triggers]. +The `BuildConfig` object is the definition of the entire build process, and is dictated by link:../../dev_guide/builds.html#build-triggers[build triggers]: +==== +---- +apiVersion: v1beta3 +kind: BuildConfig +metadata: + creationTimestamp: 2015-06-01T05:54:04Z + labels: + name: ruby-sample-build + template: application-template-stibuild + name: ruby-sample-build + namespace: ruby + resourceVersion: "7246" + selfLink: /osapi/v1beta3/namespaces/ruby/buildconfigs/ruby-sample-build + uid: 9c3d9693-0822-11e5-9deb-080027c5bfa9 +spec: + output: <1> + to: + kind: ImageStreamTag + name: origin-ruby-sample:latest + resources: {} <2> + source: <3> + git: + uri: git://github.com/openshift/ruby-hello-world.git + type: Git + strategy: <4> + sourceStrategy: + from: + kind: ImageStreamTag + name: ruby-20-centos7:latest + incremental: true + type: Source + triggers: + - github: <5> + secret: secret101 + type: github + - generic: <6> + secret: secret101 + type: generic + - imageChange: <7> + lastTriggeredImageID: openshift/ruby-20-centos7:latest + type: imageChange +status: + lastVersion: 1 -//// -[horizontal] -triggers:: Define policies used for automatically invoking builds: -GitHub webhooks::: GitHub specific webhooks that specify which repository +---- +<1> *output* describes the resulting image name, tag, and registry to which the +image should be pushed. +<2> *resources* describes the resource limits (cpu, memory) that should be used +by pods that run a build. +<3> *source* describes the SCM used to locate the sources. Currently only +supports Git. +<4> *strategy* describes which build type is invoked along with build type +specific details. +<5> *gitHub* triggers are Github-specific webhooks that specify which repository changes, such as a new commit, should invoke a new build. This trigger is specific to the GitHub API. -generic webhooks::: Similar to GitHub webhooks in that they invoke a new build -whenever it gets a notification. The difference is its payload is slightly -different than GitHub's. -image change::: Defines a trigger which is invoked upon availability of a new -image in the specified ImageRepository. - -parameters:: -source::: Describes the SCM used to locate the sources. Currently only supports -Git. -strategy::: Describes which build type is invoked along with build type specific -details. -output::: Describes the resulting image name, tag, and registry to which the -image should be pushed. -resources::: Describes the resource limits (cpu, memory) that should be used by pods that run a build. -//// +<6> *generic* triggers are similar to GitHub webhooks in that they invoke a new +build when notified, but different in that the payload is slightly different +than GitHub's. +<7> *imageChange* triggers define a trigger which is invoked upon availability +of a new image in the specified ImageRepository. +==== == Build Strategies From 2858cbb4715c037e4f5628fe805d3382818e99e0 Mon Sep 17 00:00:00 2001 From: bfallonf Date: Tue, 2 Jun 2015 14:08:44 +1000 Subject: [PATCH 4/4] edits after dev suggestion --- architecture/core_objects/builds.adoc | 20 ++++++++------------ 1 file changed, 8 insertions(+), 12 deletions(-) diff --git a/architecture/core_objects/builds.adoc b/architecture/core_objects/builds.adoc index 4f0be960ed71..6e8b049f1628 100644 --- a/architecture/core_objects/builds.adoc +++ b/architecture/core_objects/builds.adoc @@ -23,7 +23,6 @@ The `BuildConfig` object is the definition of the entire build process, and is d apiVersion: v1beta3 kind: BuildConfig metadata: - creationTimestamp: 2015-06-01T05:54:04Z labels: name: ruby-sample-build template: application-template-stibuild @@ -59,11 +58,9 @@ spec: - imageChange: <7> lastTriggeredImageID: openshift/ruby-20-centos7:latest type: imageChange -status: - lastVersion: 1 ---- -<1> *output* describes the resulting image name, tag, and registry to which the +<1> *output* describes the resulting ImageStream source to which the image should be pushed. <2> *resources* describes the resource limits (cpu, memory) that should be used by pods that run a build. @@ -78,7 +75,7 @@ specific to the GitHub API. build when notified, but different in that the payload is slightly different than GitHub's. <7> *imageChange* triggers define a trigger which is invoked upon availability -of a new image in the specified ImageRepository. +of a new image in the specified ImageStream. ==== == Build Strategies @@ -109,18 +106,17 @@ The advantages of STI include: [horizontal] Image flexibility:: STI scripts can be written to layer application code onto almost any existing Docker image, taking advantage of the existing ecosystem. -Note that, currently, STI relies on `tar` and `untar` to inject application +Note that, currently, STI relies on `tar` to inject application source, so the image needs to be able to process tarred content. Speed:: With STI, the assemble process can perform a large number of complex operations without creating a new layer at each step, resulting in a fast -process. In addition, STI scripts can be written to re-use dependencies stored -in a previous version of the application image rather than having to download -them each time the build is run. +process. In addition, STI scripts can be written to re-use artifacts stored in a +previous version of the application image rather than having to download or +build them each time the build is run. -Patchability:: If an underlying image needs a patch due to a security issue, -OpenShift can use STI to rebuild your application on top of the patched builder -image. +Patchability:: STI allows you to rebuild the application consistently if an +underlying image needs a patch due to a security issue Operational efficiency:: By restricting build operations instead of allowing arbitrary actions, such as in a *_Dockerfile_*, the PaaS operator can avoid