From cde8aa2726626ba44afb31e49ce3e6bde50f236f Mon Sep 17 00:00:00 2001 From: Jason Boxman Date: Wed, 25 Sep 2019 14:30:05 -0400 Subject: [PATCH] Clarify and expand content for additional networks - https://github.com/openshift/openshift-docs/pull/16349 --- _topic_map.yml | 8 + modules/nw-multus-add-pod.adoc | 50 ++++- modules/nw-multus-bridge-object.adoc | 81 +++++-- modules/nw-multus-create-network.adoc | 40 ++-- modules/nw-multus-delete-network.adoc | 5 +- modules/nw-multus-edit-network.adoc | 10 +- modules/nw-multus-host-device-object.adoc | 68 ++++-- modules/nw-multus-ipam-object.adoc | 200 +++++++++++------- modules/nw-multus-ipvlan-object.adoc | 73 +++++-- modules/nw-multus-macvlan-object.adoc | 44 ++-- modules/nw-multus-remove-pod.adoc | 49 +++-- .../multiple-networks/attaching-pod.adoc | 10 + .../multiple-networks/configuring-bridge.adoc | 7 +- .../configuring-host-device.adoc | 7 +- .../multiple-networks/configuring-ipvlan.adoc | 7 +- .../configuring-macvlan.adoc | 7 +- .../edit-additional-network.adoc | 11 + .../remove-additional-network.adoc | 10 + .../multiple-networks/removing-pod.adoc | 10 + .../understanding-multiple-networks.adoc | 33 +-- 20 files changed, 512 insertions(+), 218 deletions(-) create mode 100644 networking/multiple-networks/attaching-pod.adoc create mode 100644 networking/multiple-networks/edit-additional-network.adoc create mode 100644 networking/multiple-networks/remove-additional-network.adoc create mode 100644 networking/multiple-networks/removing-pod.adoc diff --git a/_topic_map.yml b/_topic_map.yml index b6f90e2a3214..1f4fb7aa27e2 100644 --- a/_topic_map.yml +++ b/_topic_map.yml @@ -299,6 +299,10 @@ Topics: Topics: - Name: Understanding multiple networks File: understanding-multiple-networks + - Name: Attaching a Pod to an additional network + File: attaching-pod + - Name: Removing a Pod from an additional network + File: removing-pod - Name: Configuring a bridge network File: configuring-bridge - Name: Configuring a macvlan network @@ -309,6 +313,10 @@ Topics: File: configuring-host-device - Name: Configuring SR-IOV File: configuring-sr-iov + - Name: Editing an additional network + File: edit-additional-network + - Name: Removing an additional network + File: remove-additional-network - Name: OpenShift SDN Dir: openshift-sdn Topics: diff --git a/modules/nw-multus-add-pod.adoc b/modules/nw-multus-add-pod.adoc index 8a3e78bde70d..7e6c5f7759e3 100644 --- a/modules/nw-multus-add-pod.adoc +++ b/modules/nw-multus-add-pod.adoc @@ -1,14 +1,16 @@ // Module included in the following assemblies: // +// * networking/multiple-networks/attaching-pod.adoc [id="nw-multus-add-pod_{context}"] = Adding a Pod to an additional network -You can add a Pod to an additional network. {product-title} will continue using -the primary network for normal cluster related network traffic. +You can add a Pod to an additional network. The Pod continues to send normal +cluster related network traffic over the default network. .Prerequisites +* The Pod must be in the same namespace as the additional network. * Install the OpenShift Command-line Interface (CLI), commonly known as `oc`. * You must log in to the cluster. @@ -26,8 +28,7 @@ $ oc edit pod . In the Pod resource definition, add the `k8s.v1.cni.cncf.io/networks` parameter to the Pod `metadata` mapping. The `k8s.v1.cni.cncf.io/networks` -accepts a comma separated string of one or more `AdditionalNetworkDefinition` -names: +accepts a comma separated string of one or more NetworkAttachmentDefinition Custom Resource (CR) names: + [source,yaml] ---- @@ -59,8 +60,47 @@ spec: ---- . Optional: Confirm that the annotation exists in the Pod CR by running the -following command: +following command. Replace `` with the name of the Pod. + ---- $ oc describe pod ---- ++ +In the following example, the `example-pod` Pod is attached to the `net1` +additional network: ++ +---- +$ oc get pod example-pod -o yaml +apiVersion: v1 +kind: Pod +metadata: + annotations: + k8s.v1.cni.cncf.io/networks: macvlan-bridge + k8s.v1.cni.cncf.io/networks-status: |- <1> + [{ + "name": "openshift-sdn", + "interface": "eth0", + "ips": [ + "10.128.2.14" + ], + "default": true, + "dns": {} + },{ + "name": "macvlan-bridge", + "interface": "net1", + "ips": [ + "20.2.2.100" + ], + "mac": "22:2f:60:a5:f8:00", + "dns": {} + }] + name: example-pod + namespace: default +spec: + ... +status: + ... +---- +<1> The `k8s.v1.cni.cncf.io/networks-status` parameter is a JSON array of +objects. Each object describes the status of an additional network attached +to the Pod. The annotation value is stored as a plain text value. diff --git a/modules/nw-multus-bridge-object.adoc b/modules/nw-multus-bridge-object.adoc index 5b7adf1853ff..a3d4a8d24b85 100644 --- a/modules/nw-multus-bridge-object.adoc +++ b/modules/nw-multus-bridge-object.adoc @@ -1,26 +1,44 @@ // Module included in the following assemblies: // +// * networking/multiple-networks/configuring-bridge.adoc [id="nw-multus-bridge-object_{context}"] = Configuration for bridge -// TODO - duplicated in ipvlan, copy changes from there to here. +The configuration for an additional network attachment that uses the bridge +Container Network Interface (CNI) plug-in is provided in two parts: -.bridge CNI plug-in YAML configuration +* Cluster Network Operator (CNO) configuration +* CNI plug-in configuration + +The CNO configuration specifies the name for the additional network attachment +and the namespace to create the attachment in. The plug-in +is configured by a JSON object specified by the `rawCNIConfig` parameter in +the CNO configuration. + +The following YAML describes the configuration parameters for the CNO: + +.Cluster Network Operator YAML configuration [source,yaml] ---- name: <1> namespace: <2> -rawCNIConfig: '' <3> +rawCNIConfig: '{ <3> + ... +}' type: Raw ---- -<1> `name`: Specify the name of the `NetworkAttachmentDefinition` created from -the `rawCNIConfig` JSON object. +<1> Specify a name for the additional network attachment that you are +creating. The name must be unique within the specified `namespace`. + +<2> Specify the namespace to create the network attachment in. If +you do not specify a value, then the `default` namespace is used. -<2> `namespace`: Specify the namespace to create the network attachment in. If -a value is not specified, the `default` namespace is used. +<3> Specify the CNI plug-in configuration in JSON format, which +is based on the following template. -<3> `rawCNIConfig`: Specify the CNI plug-in configuration. +The following object describes the configuration parameters for the bridge CNI +plug-in: .bridge CNI plug-in JSON configuration object [source,json] @@ -43,42 +61,65 @@ a value is not specified, the `default` namespace is used. "mtu": <11> } ---- -<1> `name`: Specify the name of the `NetworkAttachmentDefinition`. +<1> Specify the value for the `name` parameter you provided previously for +the CNO configuration. -<2> `bridge`: Specify the name of the virtual bridge to use. If the bridge +<2> Specify the name of the virtual bridge to use. If the bridge interface does not exist on the host, it is created. The default value is `cni0`. -<3> `ipam`: Specify a configuration object for the ipam CNI plug-in. The plug-in +<3> Specify a configuration object for the ipam CNI plug-in. The plug-in manages IP address assignment for the network attachment definition. -<4> `ipMasq`: Set to `true` to enable IP masquerading for traffic that leaves the +<4> Set to `true` to enable IP masquerading for traffic that leaves the virtual network. The source IP address for all traffic is rewritten to the bridge's IP address. If the bridge does not have an IP address, this setting has no effect. The default value is `false`. -<5> `isGateway`: Set to `true` to assign an IP address to the bridge. The +<5> Set to `true` to assign an IP address to the bridge. The default value is `false`. -<6> `isDefaultGateway`: Set to `true` to configure the bridge as the default +<6> Set to `true` to configure the bridge as the default gateway for the virtual network. The default value is `false`. If `isDefaultGateway` is set to `true`, then `isGateway` is also set to `true` automatically. -<7> `forceAddress`: Set to `true` to allow assignment of a previously assigned +<7> Set to `true` to allow assignment of a previously assigned IP address to the virtual bridge. When set to `false`, if an IPv4 address or an IPv6 address from overlapping subsets is assigned to the virtual bridge, an error occurs. The default value is `false`. -<8> `hairpinMode`: Set to `true` to allow the virtual bridge to send an ethernet +<8> Set to `true` to allow the virtual bridge to send an ethernet frame back through the virtual port it was received on. This mode is also known as _reflective relay_. The default value is `false`. -<9> `promiscMode`: Set to `true` to enable promiscuous mode on the bridge. The +<9> Set to `true` to enable promiscuous mode on the bridge. The default value is `false`. -<10> `vlan`: Specify a virtual LAN (VLAN) tag as an integer value. By default, +<10> Specify a virtual LAN (VLAN) tag as an integer value. By default, no VLAN tag is assigned. -<11> `mtu`: Set the maximum transmission unit (MTU) to the specified value. The -default value is automatically chosen by the kernel. +<11> Set the maximum transmission unit (MTU) to the specified value. The +default value is automatically set by the kernel. + +[id="nw-multus-bridge-config-example_{context}"] +== bridge configuration example + +The following example configures an additional network named `bridge-net`: + +[source,yaml] +---- +name: bridge-net +type: Raw +rawCNIConfig: '{ <1> + "cniVersion": "0.3.1", + "type": "bridge", + "master": "eth1", + "isGateway": true, + "vlan": 2, + "ipam": { + "type": "dhcp" + } +}' +---- +<1> The CNI configuration object is specified as a YAML string. diff --git a/modules/nw-multus-create-network.adoc b/modules/nw-multus-create-network.adoc index 7ce17c64f7c8..f65ee203810e 100644 --- a/modules/nw-multus-create-network.adoc +++ b/modules/nw-multus-create-network.adoc @@ -1,5 +1,9 @@ // Module included in the following assemblies: // +// * networking/multiple-networks/configuring-macvlan.adoc +// * networking/multiple-networks/configuring-ipvlan.adoc +// * networking/multiple-networks/configuring-bridge.adoc +// * networking/multiple-networks/configuring-host-device.adoc // Configuring Multus plug-ins using the Cluster Network Operator // is nearly identical in each case. @@ -26,11 +30,11 @@ endif::[] The Cluster Network Operator (CNO) manages additional network definitions. When you specify an additional network to create, the CNO creates the -`NetworkAttachmentDefinition` Custom Resource (CR) automatically. +NetworkAttachmentDefinition Custom Resource (CR) automatically. [IMPORTANT] ==== -Do not edit the `NetworkAttachmentDefinition` CRs that the Cluster Network +Do not edit the NetworkAttachmentDefinition CRs that the Cluster Network Operator manages. Doing so might disrupt network traffic on your additional network. ==== @@ -44,14 +48,14 @@ network. To create an additional network for your cluster, complete the following steps: -. Run the following command to edit the CNO CR: +. Edit the CNO CR by running the following command: + ---- $ oc edit networks.operator.openshift.io cluster ---- . Modify the CR that you are creating by adding the configuration for the -additional network you are creating. +additional network you are creating, as in the following example CR. + ifdef::yaml[] The following YAML configures the {plugin} CNI plug-in: @@ -68,7 +72,10 @@ spec: type: SimpleMacvlan simpleMacvlanConfig: ipamConfig: - type: DHCP + type: static + staticIPAMConfig: + addresses: + - address: 10.1.1.0/24 ---- endif::yaml[] ifdef::json[] @@ -91,8 +98,13 @@ spec: "type": "{plugin}", "master": "eth1", "ipam": { - "type": "dhcp" - } + "type": "static", + "addresses": [ + { + "address": "191.168.1.1/24" + } + ] + } }' ---- endif::[] @@ -131,17 +143,21 @@ spec: "master": "eth1", "mode": "l2", "ipam": { - "type": "dhcp" - } + "type": "static", + "addresses": [ + { + "address": "191.168.1.1/24" + } + ] + } }' ---- endif::[] -<1> `additionalNetworks`: Specify the configuration for the additional network -attachment definition. +<1> Specify the configuration for the additional network attachment definition. . Save your changes and quit the text editor to commit your changes. -. Optional: Confirm that the CNO created the `NetworkAttachmentDefinition` CR by +. Optional: Confirm that the CNO created the NetworkAttachmentDefinition CR by running the following command. There might be a delay before the CNO creates the CR. + diff --git a/modules/nw-multus-delete-network.adoc b/modules/nw-multus-delete-network.adoc index 92bb1b106897..b99bd90d1e24 100644 --- a/modules/nw-multus-delete-network.adoc +++ b/modules/nw-multus-delete-network.adoc @@ -1,5 +1,6 @@ // Module included in the following assemblies: // +// * networking/multiple-networks/remove-additional-network.adoc [id="nw-multus-delete-network_{context}"] = Removing an additional network attachment definition @@ -24,8 +25,8 @@ For more information, see link:https://bugzilla.redhat.com/show_bug.cgi?id=17559 To remove an additional network from your cluster, complete the following steps: -. Run the following command to edit the Cluster Network Operator (CNO) in your -default text editor: +. Edit the Cluster Network Operator (CNO) in your default text editor by running +the following command: + ---- $ oc edit networks.operator.openshift.io cluster diff --git a/modules/nw-multus-edit-network.adoc b/modules/nw-multus-edit-network.adoc index 556d47e19d3a..65a2ada5eee7 100644 --- a/modules/nw-multus-edit-network.adoc +++ b/modules/nw-multus-edit-network.adoc @@ -1,14 +1,16 @@ // Module included in the following assemblies: // +// * networking/multiple-networks/edit-additional-network.adoc [id="nw-multus-edit-network_{context}"] = Modifying an additional network attachment definition As a cluster administrator, you can make changes to an existing additional -network. +network. Any existing Pods attached to the additional network will not be updated. .Prerequisites +* You have configured an additional network for your cluster. * Install the OpenShift Command-line Interface (CLI), commonly known as `oc`. * Log in as a user with `cluster-admin` privileges. @@ -28,17 +30,17 @@ your changes. . Save your changes and quit the text editor to commit your changes. -. Optional: Confirm that the CNO updated the `NetworkAttachmentDefinition` CR by +. Optional: Confirm that the CNO updated the NetworkAttachmentDefinition CR by running the following command. Replace `` with the name of the additional network to display. There might be a delay before the CNO updates the -`NetworkAttachmentDefinition` CR to reflect your changes. +NetworkAttachmentDefinition CR to reflect your changes. + ---- $ oc get network-attachment-definitions -o yaml ---- + For example, the following console output displays a -`NetworkAttachmentDefinition` that is named `net1`: +NetworkAttachmentDefinition that is named `net1`: + ---- $ oc get network-attachment-definitions net1 -o go-template='{{printf "%s\n" .spec.config}}' diff --git a/modules/nw-multus-host-device-object.adoc b/modules/nw-multus-host-device-object.adoc index c76de0a58bbc..ccb611d42428 100644 --- a/modules/nw-multus-host-device-object.adoc +++ b/modules/nw-multus-host-device-object.adoc @@ -1,30 +1,47 @@ // Module included in the following assemblies: // +// * networking/multiple-networks/configuring-host-device.adoc [id="nw-multus-host-device-object_{context}"] = Configuration for host-device -// TODO - consider refactoring this into its own module as it is used -// by both ipvlan and bridge CNI plug-ins. +The configuration for an additional network attachment that uses the host-device +Container Network Interface (CNI) plug-in is provided in two parts: -.host-device CNI plug-in YAML configuration +* Cluster Network Operator (CNO) configuration +* CNI plug-in configuration + +The CNO configuration specifies the name for the additional network attachment +and the namespace to create the attachment in. The plug-in +is configured by a JSON object specified by the `rawCNIConfig` parameter in +the CNO configuration. + +The following YAML describes the configuration parameters for the CNO: + +.Cluster Network Operator YAML configuration [source,yaml] ---- name: <1> namespace: <2> -rawCNIConfig: '' <3> +rawCNIConfig: '{ <3> + ... +}' type: Raw ---- -<1> `name`: Specify the name of the `NetworkAttachmentDefinition` created from -the `rawCNIConfig` JSON object. +<1> Specify a name for the additional network attachment that you are +creating. The name must be unique within the specified `namespace`. + +<2> Specify the namespace to create the network attachment in. If +you do not specify a value, then the `default` namespace is used. -<2> `namespace`: Specify the namespace to create the network attachment in. If -a value is not specified, the `default` namespace is used. +<3> Specify the CNI plug-in configuration in JSON format, which +is based on the following template. -<3> `rawCNIConfig`: Specify the CNI plug-in configuration. +IMPORTANT: Specify your network device by setting only one of the +following parameters: `device`, `hwaddr`, `kernelpath`, or `pciBusID`. -IMPORTANT: You can specify your network device using only one of the following -parameters: `device`, `hwaddr`, `kernelpath`, or `pciBusID`. +The following object describes the configuration parameters for the host-device CNI +plug-in: .host-device CNI plug-in JSON configuration object [source,json] @@ -39,14 +56,31 @@ parameters: `device`, `hwaddr`, `kernelpath`, or `pciBusID`. "pciBusID": "" <5> } ---- -<1> `name`: Specify the name of the `NetworkAttachmentDefinition`. +<1> Specify the value for the `name` parameter you provided previously for +the CNO configuration. -<2> `device`: Specify the name of the device, such as `eth0`. +<2> Specify the name of the device, such as `eth0`. -<3> `hwaddr`: Specify the device hardware MAC address. +<3> Specify the device hardware MAC address. -<4> `kernelpath`: Specify the Linux kernel device path, such as +<4> Specify the Linux kernel device path, such as `/sys/devices/pci0000:00/0000:00:1f.6`. -<5> `pciBusID`: Specify the PCI address of the network device, such as -`0000:00:1f.6`. +<5> Specify the PCI address of the network device, such as `0000:00:1f.6`. + +[id="nw-multus-hostdev-config-example_{context}"] +== host-device configuration example + +The following example configures an additional network named `hostdev-net`: + +[source,yaml] +---- +name: hostdev-net +type: Raw +rawCNIConfig: '{ <1> + "cniVersion": "0.3.1", + "type": "host-device", + "device": "eth1" +}' +---- +<1> The CNI configuration object is specified as a YAML string. diff --git a/modules/nw-multus-ipam-object.adoc b/modules/nw-multus-ipam-object.adoc index 498b7e916c6d..64fb5ceae146 100644 --- a/modules/nw-multus-ipam-object.adoc +++ b/modules/nw-multus-ipam-object.adoc @@ -1,5 +1,8 @@ // Module included in the following assemblies: // +// * networking/multiple-networks/configuring-macvlan.adoc +// * networking/multiple-networks/configuring-ipvlan.adoc +// * networking/multiple-networks/configuring-bridge.adoc // Because the Cluster Network Operator abstracts the configuration for // Macvlan, including IPAM configuration, this must be provided as YAML @@ -13,41 +16,22 @@ ifeval::["{context}" != "configuring-macvlan"] :json: endif::[] -:type: pass:q[Specify `static` to configure the plug-in to manage IP address \ -assignment. Specify `DHCP` to allow a DHCP server to manage IP \ -address assignment. You cannot specify any additional parameters if you \ -specify a value of `DHCP`.] - -:addresses: pass:q[that define IP addresses to assign to the virtual \ -interface. Both IPv4 and IPv6 IP addresses are supported.] - -:address: pass:q[A block of IP addresses specified in CIDR format to assign \ -to Pods on a worker node.] - -:address-gateway: pass:q[The default gateway to route egress network traffic to.] - -:routes: pass:q[describing routes to configure inside the Pod.] - -:route: pass:q[The IP address range in CIDR format.] - -:route-gateway: pass:q[The gateway to use to route network traffic to.] - -:dns: pass:q[The DNS configuration. Optional.] - -:nameservers: pass:q[one or more IP addresses for to send DNS queries to.] - -:domain: pass:q[The default domain to append to a host name. For example, if the \ -domain is set to `example.com`, a DNS lookup query for `example-host` will be \ -rewritten as `example-host.example.com`.] - -:search: pass:q[An array of domain names to append to an unqualified host name, \ -such as `example-host`, during a DNS lookup query.] - [id="nw-multus-ipam-object_{context}"] = Configuration for ipam CNI plug-in The IP address management (IPAM) CNI plug-in manages IP address assignment for -other CNI plug-ins. +other CNI plug-ins. You can configure ipam for either static IP address +assignment or dynamic IP address assignment by using DHCP. The DHCP server you +specify must be reachable from the additional network. + +[IMPORTANT] +==== +In {product-title} 4.2.0, if you attach a Pod to an additional network that uses +DHCP for IP address management, the Pod will fail to start. +This will be fixed in a future release. +For more information, see link:https://bugzilla.redhat.com/show_bug.cgi?id=1754686[BZ#1754686]. +==== + ifdef::json[] The following JSON configuration object describes the parameters that you can set. @@ -61,10 +45,11 @@ IMPORTANT: If you set the `type` parameter to the `DHCP` value, you cannot set any other parameters. ifdef::json[] -.IPAM CNI plug-in JSON configuration object +.ipam CNI plug-in JSON configuration object [source,json] ---- { + "ipam": { "type": "", <1> "addresses": [ <2> { @@ -83,29 +68,70 @@ ifdef::json[] "domain": "", <10> "search": [""] <11> } + } } ---- -<1> `type`: {type} +<1> Specify `static` to configure the plug-in to manage IP address assignment. +Specify `DHCP` to allow a DHCP server to manage IP address assignment. You +cannot specify any additional parameters if you specify a value of `DHCP`. + +<2> An array of that define IP addresses to assign to the virtual +interface. Both IPv4 and IPv6 IP addresses are supported. + +<3> A block of IP addresses that you specify in CIDR format to assign +to Pods on a worker node, such as `10.1.1.0/24`. + +<4> The default gateway to route egress network traffic to. + +<5> An array describing routes to configure inside the Pod. -<2> `addresses`: An array of {addresses} +<6> The IP address range in CIDR format. -<3> `address`: {address} +<7> The gateway to use to route network traffic to. -<4> `gateway`: {address-gateway} +<8> The DNS configuration. Optional. -<5> `routes`: An array {routes} +<9> An of array of one or more IP addresses for to send DNS queries to. -<6> `dst`: {route} +<10> The default domain to append to a host name. For example, if the +domain is set to `example.com`, a DNS lookup query for `example-host` will be +rewritten as `example-host.example.com`. -<7> `gw`: {route-gateway} +<11> An array of domain names to append to an unqualified host name, +such as `example-host`, during a DNS lookup query. -<8> `dns`: {dns} +[id="nw-multus-static-example_{context}"] +== Static IP address assignment configuration example + +You can configure ipam for static IP address assignment: + +[source,json] +---- +{ + "ipam": { + "type": "static", + "addresses": [ + { + "address": "191.168.1.1/24" + } + ] + } +} +---- -<9> `nameservers`: An of array of {nameservers} +[id="nw-multus-dynamic-example_{context}"] +== Dynamic IP address assignment configuration example -<10> `domain`: {domain} +You can configure ipam for DHCP: -<11> `search`: {search} +[source,json] +---- +{ + "ipam": { + "type": "DHCP" + } +} +---- endif::json[] // YAML uses collection and mapping to describe arrays and objects @@ -114,57 +140,80 @@ ifdef::yaml[] .ipam CNI plug-in YAML configuration object [source,yaml] ---- -ipamConfig: <1> - type: <2> - ... <3> +ipamConfig: + type: <1> + ... <2> ---- -<1> `ipamConfig`: The ipam configuration. - -<2> `type`: {type} +<1> Specify `static` to configure the plug-in to manage IP address assignment. +Specify `DHCP` to allow a DHCP server to manage IP address assignment. You +cannot specify any additional parameters if you specify a value of `DHCP`. -<3> If you set the `type` parameter to `static`, then provide the +<2> If you set the `type` parameter to `static`, then provide the `staticIPAMConfig` parameter. +[id="nw-multus-static-config_{context}"] == Static ipam configuration YAML +The following YAML describes a configuration for static IP address assignment: + .Static ipam configuration YAML [source,yaml] ---- -staticIPAMConfig: - addresses: <1> - - address:
<2> - gateway: <3> - routes: <4> - - destination: <5> - gateway: <6> - dns: <7> - nameservers: <8> - - - domain: <9> - search: <10> - - +ipamConfig: + type: static + staticIPAMConfig: + addresses: <1> + - address:
<2> + gateway: <3> + routes: <4> + - destination: <5> + gateway: <6> + dns: <7> + nameservers: <8> + - + domain: <9> + search: <10> + - ---- -<1> `addresses`: A collection of mappings {addresses} +<1> A collection of mappings that define IP addresses to assign to the virtual +interface. Both IPv4 and IPv6 IP addresses are supported. + +<2> A block of IP addresses that you specify in CIDR format to assign +to Pods on a worker node, such as `10.1.1.0/24`. -<2> `address`: {address} +<3> The default gateway to route egress network traffic to. -<3> `gateway`: {address-gateway} +<4> A collection of mappings describing routes to configure inside the Pod. -<4> `routes`: A collection of mappings {routes} +<5> The IP address range in CIDR format. -<5> `destination`: {route} +<6> The gateway to use to route network traffic to. -<6> `gateway`: {route-gateway} +<7> The DNS configuration. Optional. -<7> `dns`: {dns} +<8> A collection of one or more IP addresses for to send DNS queries to. -<8> `nameservers`: A collection of {nameservers} +<9> The default domain to append to a host name. For example, if the +domain is set to `example.com`, a DNS lookup query for `example-host` will be +rewritten as `example-host.example.com`. -<9> `domain`: {domain} +<10> An array of domain names to append to an unqualified host name, +such as `example-host`, during a DNS lookup query. -<10> `search`: {search} +[id="nw-multus-dynamic-config_{context}"] +== Dynamic ipam configuration YAML -== Example ipam configuration YAML +The following YAML describes a configuration for static IP address assignment: + +.Dynamic ipam configuration YAML +[source,yaml] +---- +ipamConfig: + type: DHCP +---- + +[id="nw-multus-static-example-yaml_{context}"] +== Static IP address assignment configuration example The following example shows an ipam configuration for static IP addresses: @@ -189,6 +238,9 @@ ipamConfig: - testdomain2.example ---- +[id="nw-multus-dynamic-example-yaml_{context}"] +== Dynamic IP address assignment configuration example + The following example shows an ipam configuration for DHCP: [source,yaml] diff --git a/modules/nw-multus-ipvlan-object.adoc b/modules/nw-multus-ipvlan-object.adoc index 703c8d4610b6..0975a85b31ac 100644 --- a/modules/nw-multus-ipvlan-object.adoc +++ b/modules/nw-multus-ipvlan-object.adoc @@ -1,33 +1,44 @@ // Module included in the following assemblies: // +// * networking/multiple-networks/configuring-ipvlan.adoc [id="nw-multus-ipvlan-object_{context}"] = Configuration for ipvlan -// TODO - consider refactoring this into its own module as it is used -// by both ipvlan and bridge CNI plug-ins. +The configuration for an additional network attachment that uses the ipvlan +Container Network Interface (CNI) plug-in is provided in two parts: -.ipvlan CNI plug-in YAML configuration +* Cluster Network Operator (CNO) configuration +* CNI plug-in configuration + +The CNO configuration specifies the name for the additional network attachment +and the namespace to create the attachment in. The plug-in +is configured by a JSON object specified by the `rawCNIConfig` parameter in +the CNO configuration. + +The following YAML describes the configuration parameters for the CNO: + +.Cluster Network Operator YAML configuration [source,yaml] ---- name: <1> namespace: <2> -rawCNIConfig: '' <3> +rawCNIConfig: '{ <3> + ... +}' type: Raw ---- -<1> `name`: Specify the name of the `NetworkAttachmentDefinition` created from -the `rawCNIConfig` JSON object. - -<2> `namespace`: Specify the namespace to create the network attachment in. If -a value is not specified, the `default` namespace is used. +<1> Specify a name for the additional network attachment that you are +creating. The name must be unique within the specified `namespace`. -<3> `rawCNIConfig`: Specify the CNI plug-in configuration. +<2> Specify the namespace to create the network attachment in. If +you do not specify a value, then the `default` namespace is used. -//// -TODO - +<3> Specify the CNI plug-in configuration in JSON format, which +is based on the following template. -- `mode` needs an extended discussion; l2 might not work on all cloud providers -//// +The following object describes the configuration parameters for the ipvlan CNI +plug-in: .ipvlan CNI plug-in JSON configuration object [source,json] @@ -44,17 +55,39 @@ TODO - } } ---- -<1> `name`: Specify the name of the `NetworkAttachmentDefinition`. +<1> Specify the value for the `name` parameter you provided previously for +the CNO configuration. -<2> `mode`: Specify the operating mode for the virtual network. The value must +<2> Specify the operating mode for the virtual network. The value must be `l2`, `l3`, or `l3s`. The default value is `l2`. -<3> `master`: Specify the ethernet interface to associate with the network +<3> Specify the ethernet interface to associate with the network attachment. If a `master` is not specified, the interface for the default network route is used. -<4> `mtu`: Set the maximum transmission unit (MTU) to the specified value. The -default value is automatically chosen by the kernel. +<4> Set the maximum transmission unit (MTU) to the specified value. The +default value is automatically set by the kernel. -<5> `ipam`: Specify a configuration object for the ipam CNI plug-in. The plug-in +<5> Specify a configuration object for the ipam CNI plug-in. The plug-in manages IP address assignment for the attachment definition. + +[id="nw-multus-ipvlan-config-example_{context}"] +== ipvlan configuration example + +The following example configures an additional network named `ipvlan-net`: + +[source,yaml] +---- +name: ipvlan-net +type: Raw +rawCNIConfig: '{ <1> + "cniVersion": "0.3.1", + "type": "ipvlan", + "master": "eth1", + "mode": "l3", + "ipam": { + "type": "dhcp" + } +}' +---- +<1> The CNI configuration object is specified as a YAML string. diff --git a/modules/nw-multus-macvlan-object.adoc b/modules/nw-multus-macvlan-object.adoc index 877845775180..d8cff35b8cf3 100644 --- a/modules/nw-multus-macvlan-object.adoc +++ b/modules/nw-multus-macvlan-object.adoc @@ -1,5 +1,6 @@ // Module included in the following assemblies: // +// * networking/multiple-networks/configuring-macvlan.adoc [id="nw-multus-macvlan-object_{context}"] = Configuration for macvlan CNI plug-in @@ -13,32 +14,43 @@ Container Network Interface (CNI) plug-in: name: <1> namespace: <2> type: SimpleMacvlan -simpleMacvlanConfig: <3> - master: <4> - mode: <5> - mtu: <6> - ipamConfig: <7> +simpleMacvlanConfig: + master: <3> + mode: <4> + mtu: <5> + ipamConfig: <6> ... ---- -<1> `name`: Specify a unique name for the additional network attachment. The -associated `NetworkAttachmentDefinition` will share this name. +<1> Specify a name for the additional network attachment that you are +creating. The name must be unique within the specified `namespace`. -<2> `namespace`: Specify the namespace to create the network attachment in. If +<2> Specify the namespace to create the network attachment in. If a value is not specified, the `default` namespace is used. -<3> `simpleMacvlanConfig`: Provide the configuration for the macvlan CNI -plug-in. - -<4> `master`: The ethernet interface to associate with the virtual interface. If +<3> The ethernet interface to associate with the virtual interface. If a value for `master` is not specified, then the host system's primary ethernet interface is used. -<5> `mode`: Configures traffic visibility on the virtual network. Must be either +<4> Configures traffic visibility on the virtual network. Must be either `bridge`, `passthru`, `private`, or `vepa`. If a value for `mode` is not provided, the default value is `bridge`. -<6> `mtu`: Set the maximum transmission unit (MTU) to the specified value. The -default value is automatically chosen by the kernel. +<5> Set the maximum transmission unit (MTU) to the specified value. The +default value is automatically set by the kernel. -<7> `ipamConfig`: Specify a configuration object for the ipam CNI plug-in. The +<6> Specify a configuration object for the ipam CNI plug-in. The plug-in manages IP address assignment for the attachment definition. + +[id="nw-multus-macvlan-config-example_{context}"] +== macvlan configuration example + +The following example configures an additional network named `macvlan-net`: + +[source,yaml] +---- +name: macvlan-net +type: SimpleMacvlan +simpleMacvlanConfig: + ipamConfig: + type: DHCP +---- diff --git a/modules/nw-multus-remove-pod.adoc b/modules/nw-multus-remove-pod.adoc index 7af6f6717c63..70697290e1d7 100644 --- a/modules/nw-multus-remove-pod.adoc +++ b/modules/nw-multus-remove-pod.adoc @@ -1,5 +1,6 @@ // Module included in the following assemblies: // +// * networking/multiple-networks/removing-pod.adoc [id="nw-multus-remove-pod_{context}"] = Removing a Pod from an additional network @@ -8,10 +9,10 @@ You can remove a Pod from an additional network. .Prerequisites +* You have configured an additional network for your cluster. +* You have an additional network attached to the Pod. * Install the OpenShift Command-line Interface (CLI), commonly known as `oc`. * You must log in to the cluster. -* You must have configured an additional network for your cluster. -* You must have a Pod attached to an additional network. .Procedure @@ -25,7 +26,7 @@ $ oc edit pod ---- . Update the `annotations` mapping to remove the additional network from the -Pod: +Pod by performing one of the following actions: * To remove all additional networks from a Pod, remove the `k8s.v1.cni.cncf.io/networks` parameter from the Pod resource definition as in @@ -47,20 +48,34 @@ spec: * To remove a specific additional network from a Pod, update the `k8s.v1.cni.cncf.io/networks` parameter by removing the name of the -`AdditionalNetworkAttachment` for the additional network. In the following -example, the additional network `net1` was removed: +NetworkAttachmentDefinition for the additional network. + +. Optional: Confirm that the Pod is no longer attached to the additional network +by running the following command. Replace `` with the name of the Pod. + -[source,yaml] ---- -apiVersion: v1 -kind: Pod -metadata: - name: example-pod - annotations: - k8s.v1.cni.cncf.io/networks: net2 -spec: - containers: - - name: example-pod - command: ["/bin/bash", "-c", "sleep 2000000000000"] - image: centos/tools +$ oc describe pod +---- ++ +In the following example, the `example-pod` Pod is attached only to the default +cluster network. ++ +---- +$ oc describe pod example-pod + +Name: example-pod +... +Annotations: k8s.v1.cni.cncf.io/networks-status: + [{ + "name": "openshift-sdn", + "interface": "eth0", + "ips": [ + "10.131.0.13" + ], + "default": true, <1> + "dns": {} + }] +Status: Running +... ---- +<1> Only the default cluster network is attached to the Pod. diff --git a/networking/multiple-networks/attaching-pod.adoc b/networking/multiple-networks/attaching-pod.adoc new file mode 100644 index 000000000000..f879ac307124 --- /dev/null +++ b/networking/multiple-networks/attaching-pod.adoc @@ -0,0 +1,10 @@ +[id="attaching-pod"] += Attaching a Pod to an additional network +include::modules/common-attributes.adoc[] +:context: attaching-pod + +toc::[] + +As a cluster user you can attach a Pod to an additional network. + +include::modules/nw-multus-add-pod.adoc[leveloffset=+1] diff --git a/networking/multiple-networks/configuring-bridge.adoc b/networking/multiple-networks/configuring-bridge.adoc index e33177c9b0af..a0c77c6c3bd3 100644 --- a/networking/multiple-networks/configuring-bridge.adoc +++ b/networking/multiple-networks/configuring-bridge.adoc @@ -15,8 +15,7 @@ assigned an IP address on the additional network. include::modules/nw-multus-create-network.adoc[leveloffset=+1] include::modules/nw-multus-bridge-object.adoc[leveloffset=+2] include::modules/nw-multus-ipam-object.adoc[leveloffset=+2] -include::modules/nw-multus-add-pod.adoc[leveloffset=+1] -include::modules/nw-multus-remove-pod.adoc[leveloffset=+1] -include::modules/nw-multus-edit-network.adoc[leveloffset=+1] -include::modules/nw-multus-delete-network.adoc[leveloffset=+1] +.Next steps + +* xref:../../networking/multiple-networks/attaching-pod.adoc#attaching-pod[Attach a Pod to an additional network]. diff --git a/networking/multiple-networks/configuring-host-device.adoc b/networking/multiple-networks/configuring-host-device.adoc index 38e9b31c7923..124b1f41ef7d 100644 --- a/networking/multiple-networks/configuring-host-device.adoc +++ b/networking/multiple-networks/configuring-host-device.adoc @@ -12,8 +12,7 @@ namespace into the Pod's network namespace. include::modules/nw-multus-create-network.adoc[leveloffset=+1] include::modules/nw-multus-host-device-object.adoc[leveloffset=+2] -include::modules/nw-multus-add-pod.adoc[leveloffset=+1] -include::modules/nw-multus-remove-pod.adoc[leveloffset=+1] -include::modules/nw-multus-edit-network.adoc[leveloffset=+1] -include::modules/nw-multus-delete-network.adoc[leveloffset=+1] +.Next steps + +* xref:../../networking/multiple-networks/attaching-pod.adoc#attaching-pod[Attach a Pod to an additional network]. diff --git a/networking/multiple-networks/configuring-ipvlan.adoc b/networking/multiple-networks/configuring-ipvlan.adoc index fc9e5d70a3f7..7684976c3936 100644 --- a/networking/multiple-networks/configuring-ipvlan.adoc +++ b/networking/multiple-networks/configuring-ipvlan.adoc @@ -17,8 +17,7 @@ specify. include::modules/nw-multus-create-network.adoc[leveloffset=+1] include::modules/nw-multus-ipvlan-object.adoc[leveloffset=+2] include::modules/nw-multus-ipam-object.adoc[leveloffset=+2] -include::modules/nw-multus-add-pod.adoc[leveloffset=+1] -include::modules/nw-multus-remove-pod.adoc[leveloffset=+1] -include::modules/nw-multus-edit-network.adoc[leveloffset=+1] -include::modules/nw-multus-delete-network.adoc[leveloffset=+1] +.Next steps + +* xref:../../networking/multiple-networks/attaching-pod.adoc#attaching-pod[Attach a Pod to an additional network]. diff --git a/networking/multiple-networks/configuring-macvlan.adoc b/networking/multiple-networks/configuring-macvlan.adoc index 92215d046fac..747c3c50202b 100644 --- a/networking/multiple-networks/configuring-macvlan.adoc +++ b/networking/multiple-networks/configuring-macvlan.adoc @@ -19,8 +19,7 @@ compatible with the security polices of your cloud provider. include::modules/nw-multus-create-network.adoc[leveloffset=+1] include::modules/nw-multus-macvlan-object.adoc[leveloffset=+2] include::modules/nw-multus-ipam-object.adoc[leveloffset=+2] -include::modules/nw-multus-add-pod.adoc[leveloffset=+1] -include::modules/nw-multus-remove-pod.adoc[leveloffset=+1] -include::modules/nw-multus-edit-network.adoc[leveloffset=+1] -include::modules/nw-multus-delete-network.adoc[leveloffset=+1] +.Next steps + +* xref:../../networking/multiple-networks/attaching-pod.adoc#attaching-pod[Attach a Pod to an additional network]. diff --git a/networking/multiple-networks/edit-additional-network.adoc b/networking/multiple-networks/edit-additional-network.adoc new file mode 100644 index 000000000000..e5d7a5009f9a --- /dev/null +++ b/networking/multiple-networks/edit-additional-network.adoc @@ -0,0 +1,11 @@ +[id="edit-additional-network"] += Editing an additional network +include::modules/common-attributes.adoc[] +:context: edit-additional-network + +toc::[] + +As a cluster administrator you can modify the configuration for an existing +additional network. + +include::modules/nw-multus-edit-network.adoc[leveloffset=+1] diff --git a/networking/multiple-networks/remove-additional-network.adoc b/networking/multiple-networks/remove-additional-network.adoc new file mode 100644 index 000000000000..683ceaa21f7d --- /dev/null +++ b/networking/multiple-networks/remove-additional-network.adoc @@ -0,0 +1,10 @@ +[id="remove-additional-network"] += Removing an additional network +include::modules/common-attributes.adoc[] +:context: remove-additional-network + +toc::[] + +As a cluster administrator you can remove an additional network attachment. + +include::modules/nw-multus-delete-network.adoc[leveloffset=+1] diff --git a/networking/multiple-networks/removing-pod.adoc b/networking/multiple-networks/removing-pod.adoc new file mode 100644 index 000000000000..6c95de0e4283 --- /dev/null +++ b/networking/multiple-networks/removing-pod.adoc @@ -0,0 +1,10 @@ +[id="removing-pod"] += Removing a Pod from an additional network +include::modules/common-attributes.adoc[] +:context: removing-pod + +toc::[] + +As a cluster user you can remove a Pod from an additional network. + +include::modules/nw-multus-remove-pod.adoc[leveloffset=+1] diff --git a/networking/multiple-networks/understanding-multiple-networks.adoc b/networking/multiple-networks/understanding-multiple-networks.adoc index c09c9cf7d349..5fa0e3d7da7d 100644 --- a/networking/multiple-networks/understanding-multiple-networks.adoc +++ b/networking/multiple-networks/understanding-multiple-networks.adoc @@ -17,6 +17,9 @@ additional network for your cluster, depending on your needs. This gives you flexibility when you configure Pods that deliver network functionality, such as switching or routing. +[id="additional-network-considerations"] +== Usage scenarios for an additional network + You can use an additional network in situations where network isolation is needed, including data plane and control plane separation. Isolating network traffic is useful for the following performance and security reasons: @@ -27,6 +30,21 @@ Security:: You can send sensitive traffic onto a network plane that is managed specifically for security considerations, and you can separate private data that must not be shared between tenants or customers. +All of the Pods in the cluster still use the cluster-wide default network +to maintain connectivity across the cluster. Every Pod has an `eth0` interface +that is attached to the cluster-wide Pod network. You can view the interfaces +for a Pod by using the `oc exec -it \-- ip a` command. If you +add additional network interfaces that use Multus CNI, they are named `net1`, +`net2`, ..., `netN`. + +To attach additional network interfaces to a Pod, you must create configurations +that define how the interfaces are attached. You specify each interface by using +a Custom Resource (CR) that has a `NetworkAttachmentDefinition` type. A CNI +configuration inside each of these CRs defines how that interface is created. + +[id="additional-networks-provided"] +== Additional networks in {product-title} + {product-title} provides the following CNI plug-ins for creating additional networks in your cluster: @@ -48,18 +66,3 @@ networks in your cluster: allows Pods to attach to a virtual function (VF) interface on SR-IOV capable hardware on the host system. -//// - -All of the Pods in the cluster still use the cluster-wide default network -to maintain connectivity across the cluster. Every Pod has an `eth0` interface -that is attached to the cluster-wide Pod network. You can view the interfaces -for a Pod by using the `oc exec -it \-- ip a` command. If you -add additional network interfaces that use Multus CNI, they are named `net1`, -`net2`, ..., `netN`. - -To attach additional network interfaces to a Pod, you must create configurations -that define how the interfaces are attached. You specify each interface by using -a Custom Resource (CR) that has a `NetworkAttachmentDefinition` type. A CNI -configuration inside each of these CRs defines how that interface is created. - -////