[BUG] Interfaces show DOWN

[havok4u]

Describe the bug

When adding interfaces to polycube, they show down and connectivity doesn't work

To Reproduce

sudo ip netns add myapp0
sudo ip netns add myapp1

Create interfaces and up the links

sudo ip link add veth1 type veth peer name veth2
sudo ip link add veth3 type veth peer name veth4
for i in 1 2 3 4
do
sudo ip link set veth$i up
done
sudo ip link set veth2 netns myapp0
sudo ip link set veth4 netns myapp1

Add to Polycube

polycubectl br0 ports add veth1
polycubectl br0 ports add veth3

Set IP address to namespaces

sudo ip netns exec myapp0 ip addr add 10.1.1.2/24 dev veth2
sudo ip netns exec myapp1 ip addr add 10.1.1.3/24 dev veth4

Up interfaces in namespaces if required

sudo ip netns exec myapp0 sudo ip link set veth2 up
sudo ip netns exec myapp0 ip addr show
1: lo: mtu 65536 qdisc noop state DOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
14: veth2@if15: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 2e:a9:da:40:ec:af brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 10.1.1.2/24 scope global veth2
valid_lft forever preferred_lft forever
inet6 fe80::2ca9:daff:fe40:ecaf/64 scope link
valid_lft forever preferred_lft forever
sudo ip netns exec myapp1 sudo ip link set veth4 up
sudo ip netns exec myapp1 ip addr show
1: lo: mtu 65536 qdisc noop state DOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
16: veth4@if17: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 3a:49:95:d0:5a:bf brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 10.1.1.3/24 scope global veth4
valid_lft forever preferred_lft forever
inet6 fe80::3849:95ff:fed0:5abf/64 scope link
valid_lft forever preferred_lft forever

Verify veth1 and veth3

ip link show veth1
15: veth1@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 86:83:02:57:b4:1b brd ff:ff:ff:ff:ff:ff link-netnsid 0

ip link show veth3
17: veth3@if16: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether e2:75:0c:f1:47:1c brd ff:ff:ff:ff:ff:ff link-netnsid 1

Ping

sudo ip netns exec myapp1 ping 10.1.1.2 -c 5
PING 10.1.1.2 (10.1.1.2) 56(84) bytes of data.

--- 10.1.1.2 ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 4100ms

Show Bridge

polycubectl br0 show
name: br0
uuid: 91081d7c-b164-4487-a161-3876467d338d
service-name: bridge
type: TC
loglevel: INFO
shadow: false
span: false
stp-enabled: false
mac: f2:f6:ae:20:3a:dc
fdb:
aging-time: 300

ports:
name uuid status peer mac mode
veth1 14720cdf-8ae2-45aa-8dad-8420e1432943 DOWN ae:8f:a6:15:f3:a3 access
veth3 ef0e0094-12aa-4fdd-96be-09326317f2d0 DOWN 36:18:34:87:f3:97 access

[mauriciovasquezbernal]

Ì think you are missing to set the peer.

polycubectl br0 ports add veth1 peer=veth1
polycubectl br0 ports add veth3 peer=veth3

[havok4u]

I guess I would be curious to know why would you do that when the port is already added? Regardless, it did not work.

$ polycubectl br0 ports add veth1 peer=veth1
Port veth1 already exists
$ polycubectl br0 ports add veth3 peer=veth3
Port veth3 already exists

$ polycubectl br0 show
name: br0
uuid: 91081d7c-b164-4487-a161-3876467d338d
service-name: bridge
type: TC
loglevel: INFO
shadow: false
span: false
stp-enabled: false
mac: f2:f6:ae:20:3a:dc
fdb:
aging-time: 300

ports:
name uuid status peer mac mode
veth1 14720cdf-8ae2-45aa-8dad-8420e1432943 DOWN ae:8f:a6:15:f3:a3 access
veth3 ef0e0094-12aa-4fdd-96be-09326317f2d0 DOWN 36:18:34:87:f3:97 access

[havok4u]

Some missing information here, this is running in a KVM on a machine. The KVM OS Info is as follows:
$ uname -a
Linux xdptest1 4.15.0-55-generic #60-Ubuntu SMP Tue Jul 2 18:22:20 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

$ cat /etc/os-release
NAME="Ubuntu"
VERSION="18.04.2 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.2 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

$ kvm-ok
INFO: /dev/kvm exists
KVM acceleration can be used

The host machine OS is as follows:
$ uname -a
Linux node41 4.4.0-154-generic #181-Ubuntu SMP Tue Jun 25 05:29:03 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

$ cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.6 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.6 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

[havok4u]

BTW, Just to reiterate, I tried what @mauriciovasquezbernal suggested, it did not work. Also wanted to show that the two interfaces from the OS level are up, but obviously down from the polycube standpoint:
$ ip link show veth1
15: veth1@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 86:83:02:57:b4:1b brd ff:ff:ff:ff:ff:ff link-netnsid 0
$ ip link show veth1
15: veth1@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 86:83:02:57:b4:1b brd ff:ff:ff:ff:ff:ff link-netnsid 0

[havok4u]

@mauriciovasquezbernal Sorry, I didn't remove the interfaces before I performed your commands. I deleted the interfaces and added them back in the way you instructed and they are up.
$ polycubectl br0 ports show
mac mode name peer status uuid
aa:f0:84:c6:db:42 access veth1 veth1 UP 6f1f1a31-3bbe-46ed-ad2c-183d0feae485
8a:72:c0:80:75:a8 access veth3 veth3 UP 222a1540-7c84-4b54-b552-dd429ad80b40

I think we are ok to close this case, but I would like to understand why we would need to add the peer for just and interface? Also, if this is the case, can we get a fail and error message if we try to add the interface without defining a peer. Thanks

[goldenrye]

$ polycubectl br0 ports add veth1 peer=veth1
The first "veth1" is the name of port -- an attribute of bridge br0, and the second "veth1" refers to the veth1 interface in the host, to avoid the confusion I think it is better to configure like the following command:
$ polycubectl br0 ports add p1 peer=veth1

We cannot fail the configuration without peer parameter because the port not necessary has a interface peer, for example if a bridge connects to a router, either the bridge or the router's port has to be configured without a peer because the other peer may not configure yet.

[mauriciovasquezbernal]

@havok4u I think this https://polycube.readthedocs.io/en/latest/cubes.html can clarify your doubts about the peer parameter in the ports.

I am closing this as this is really not an issue.

btw, @acloudiator I think the good first issue label should only be added when the issue is confirmed, otherwise a newcomer could get confused trying to solve something that is already working.

[havok4u]

Awesome the readthedocs works. I'll read over it. Currently having issues making the bridge mode of xdp_drv work. When I use it, nothing passes. Also the xdp_skb has been sketchy if it works every time as well. I'll look to see if you address that in the readthedocs. Thank

…

On Tue, Aug 20, 2019 at 8:29 PM Mauricio Vásquez ***@***.***> wrote: @havok4u <https://github.com/havok4u> I think this https://polycube.readthedocs.io/en/latest/cubes.html can clarify your doubts about the peer parameter in the ports. I am closing this as this is really not an issue. btw, @acloudiator <https://github.com/acloudiator> I think the good first issue label should only be added when the issue is confirmed, otherwise a newcomer could get confused trying to solve something that is already working. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#202?email_source=notifications&email_token=AA2VWWDDA2PLOK35PIHH7HTQFSK55A5CNFSM4II2AZO2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4YEOCI#issuecomment-523257609>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA2VWWALUXZFOF2CFAI67HTQFSK55ANCNFSM4II2AZOQ> .

[frisso]

@sebymiano I remember that you tested the bridge in XDP mode. Did you note any issues? Suggestions?

[havok4u]

So when in polycube TC bridge mode, all works fine. When I go into xdp_skb mode the namespaces do not communicate other than pings (also cannot tcpdump the other veth pair. Is XDP_SKB mode ok to use for veths? The XDP_DRV mode I can see doesn't do veths (which makes sense), so I put the Physical 10Gs in the bridge, I'll have to setup an external traffic gen to test there. Problem is I can see one 10G going on that bridge with veths or taps (the guest apps), but when I put a veth in an xdp_drv bridge it fails to put them in. Below is my test results: *LINUBRIDGE* *-------------------------------------------------------------------------------------------------* # ip netns exec myapp1 ip -4 -o addr show 10: veth2 inet 10.1.1.2/24 scope global veth2\ valid_lft forever preferred_lft forever # ip netns exec myapp2 ip -4 -o addr show 12: veth4 inet 10.1.1.3/24 scope global veth4\ valid_lft forever preferred_lft forever root@node42:/home/tepkes# ip netns exec myapp1 ping 10.1.1.3 PING 10.1.1.3 (10.1.1.3) 56(84) bytes of data. 64 bytes from 10.1.1.3: icmp_seq=1 ttl=64 time=0.064 ms 64 bytes from 10.1.1.3: icmp_seq=2 ttl=64 time=0.044 ms ^C --- 10.1.1.3 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1013ms # brctl show bridge name bridge id STP enabled interfaces bridge0 8000.4a06cb3e461d yes bridge0-nic veth1 veth3 *POLYCUBE TC* *---------------------------------------------------------------------------------------------* # polycubectl br0 ports add veth1 peer=veth1 root@node42:/home/tepkes# polycubectl br0 ports add veth3 peer=veth3 root@node42:/home/tepkes# polycubectl br0 show name: br0 uuid: 369064f8-724b-49ab-b9ac-d0aa2963d6fb service-name: bridge type: TC loglevel: INFO shadow: false span: false stp-enabled: false mac: 7e:92:80:10:02:c0 fdb: aging-time: 300 ports: name uuid status peer mac mode veth1 5a58e1fa-1034-47c0-97d4-d852000efceb UP veth1 86:dd:0d:c5:39:1c access veth3 704e5da9-c3f5-4e3a-b9d8-96958893100e UP veth3 86:a9:65:e8:b5:34 access # ip netns exec myapp1 ping 10.1.1.3 PING 10.1.1.3 (10.1.1.3) 56(84) bytes of data. 64 bytes from 10.1.1.3: icmp_seq=1 ttl=64 time=0.423 ms 64 bytes from 10.1.1.3: icmp_seq=2 ttl=64 time=0.055 ms 64 bytes from 10.1.1.3: icmp_seq=3 ttl=64 time=0.050 ms ^C --- 10.1.1.3 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2025ms rtt min/avg/max/mdev = 0.050/0.176/0.423/0.174 ms *NOTES:* # tcpdump -ni veth1 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on veth1, link-type EN10MB (Ethernet), capture size 262144 bytes 15:50:40.600598 IP 10.1.1.2 > 10.1.1.3: ICMP echo request, id 2947, seq 9, length 64 15:50:40.600643 IP 10.1.1.3 > 10.1.1.2: ICMP echo reply, id 2947, seq 9, length 64 15:50:41.624614 IP 10.1.1.2 > 10.1.1.3: ICMP echo request, id 2947, seq 10, length 64 15:50:41.624651 IP 10.1.1.3 > 10.1.1.2: ICMP echo reply, id 2947, seq 10, length 64 15:50:42.648608 IP 10.1.1.2 > 10.1.1.3: ICMP echo request, id 2947, seq 11, length 64 15:50:42.648644 IP 10.1.1.3 > 10.1.1.2: ICMP echo reply, id 2947, seq 11, length 64 ^C 6 packets captured 6 packets received by filter 0 packets dropped by kernel # tcpdump -ni veth3 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on veth3, link-type EN10MB (Ethernet), capture size 262144 bytes 15:50:48.792624 IP 10.1.1.2 > 10.1.1.3: ICMP echo request, id 2947, seq 17, length 64 15:50:48.792651 IP 10.1.1.3 > 10.1.1.2: ICMP echo reply, id 2947, seq 17, length 64 15:50:49.816613 IP 10.1.1.2 > 10.1.1.3: ICMP echo request, id 2947, seq 18, length 64 15:50:49.816634 IP 10.1.1.3 > 10.1.1.2: ICMP echo reply, id 2947, seq 18, length 64 # ip netns exec myapp1 iperf3 -c 10.1.1.3 Connecting to host 10.1.1.3, port 5201 [ 4] local 10.1.1.2 port 33824 connected to 10.1.1.3 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 4.02 GBytes 34.5 Gbits/sec 0 720 KBytes [ 4] 1.00-2.00 sec 4.14 GBytes 35.6 Gbits/sec 0 720 KBytes [ 4] 2.00-3.00 sec 4.16 GBytes 35.8 Gbits/sec 0 720 KBytes [ 4] 3.00-4.00 sec 4.19 GBytes 36.0 Gbits/sec 0 720 KBytes [ 4] 4.00-5.00 sec 4.23 GBytes 36.4 Gbits/sec 0 720 KBytes [ 4] 5.00-6.00 sec 4.17 GBytes 35.8 Gbits/sec 0 720 KBytes [ 4] 6.00-7.00 sec 4.16 GBytes 35.8 Gbits/sec 0 720 KBytes [ 4] 7.00-8.00 sec 4.16 GBytes 35.7 Gbits/sec 0 720 KBytes [ 4] 8.00-9.00 sec 4.16 GBytes 35.7 Gbits/sec 0 720 KBytes [ 4] 9.00-10.00 sec 4.17 GBytes 35.8 Gbits/sec 0 720 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 41.6 GBytes 35.7 Gbits/sec 0 sender [ 4] 0.00-10.00 sec 41.6 GBytes 35.7 Gbits/sec receiver iperf Done. # ip netns exec myapp2 iperf3 -s -B 10.1.1.3 ----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- Accepted connection from 10.1.1.2, port 33822 [ 5] local 10.1.1.3 port 5201 connected to 10.1.1.2 port 33824 [ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 3.85 GBytes 33.1 Gbits/sec [ 5] 1.00-2.00 sec 4.14 GBytes 35.5 Gbits/sec [ 5] 2.00-3.00 sec 4.19 GBytes 36.0 Gbits/sec [ 5] 3.00-4.00 sec 4.16 GBytes 35.7 Gbits/sec [ 5] 4.00-5.00 sec 4.23 GBytes 36.4 Gbits/sec [ 5] 5.00-6.00 sec 4.17 GBytes 35.8 Gbits/sec [ 5] 6.00-7.00 sec 4.16 GBytes 35.7 Gbits/sec [ 5] 7.00-8.00 sec 4.16 GBytes 35.8 Gbits/sec [ 5] 8.00-9.00 sec 4.16 GBytes 35.8 Gbits/sec [ 5] 9.00-10.00 sec 4.16 GBytes 35.8 Gbits/sec [ 5] 10.00-10.04 sec 178 MBytes 37.3 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 5] 0.00-10.04 sec 0.00 Bytes 0.00 bits/sec sender [ 5] 0.00-10.04 sec 41.6 GBytes 35.6 Gbits/sec receiver ----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- *POLYCUBE XDP_SKB* *----------------------------------------------------------------------------------------------* # polycubectl br1 ports add veth1 peer=veth1 # polycubectl br1 ports add veth3 peer=veth3 # polycubectl br1 show name: br1 uuid: ec949119-7e9c-42a3-b8e7-158f21e65757 service-name: bridge type: XDP_SKB loglevel: INFO shadow: false span: false stp-enabled: false mac: 92:a0:ca:24:95:8c fdb: aging-time: 300 ports: name uuid status peer mac mode veth1 f662f905-4f52-4b2a-96ff-afd178685854 UP veth1 92:a3:bf:af:86:97 access veth3 58e3fd10-6a2b-49d6-ad8b-56797849df2d UP veth3 36:c1:70:ca:11:d1 access # ip netns exec myapp1 ping 10.1.1.3 PING 10.1.1.3 (10.1.1.3) 56(84) bytes of data. 64 bytes from 10.1.1.3: icmp_seq=1 ttl=64 time=0.326 ms 64 bytes from 10.1.1.3: icmp_seq=2 ttl=64 time=0.050 ms 64 bytes from 10.1.1.3: icmp_seq=3 ttl=64 time=0.048 ms 64 bytes from 10.1.1.3: icmp_seq=4 ttl=64 time=0.048 ms ^C --- 10.1.1.3 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3073ms rtt min/avg/max/mdev = 0.048/0.118/0.326/0.120 ms *NOTE*: tcpdump on interface veth1 and veth3 shows nothing even though pings are making it through and IPERF3 does not work at all.

…

On Fri, Aug 23, 2019 at 1:13 PM Fulvio Risso ***@***.***> wrote: @sebymiano <https://github.com/sebymiano> I remember that you tested the bridge in XDP mode. Did you note any issues? Suggestions? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#202?email_source=notifications&email_token=AA2VWWC45H5OCU2CBUXBRMDQGASEDA5CNFSM4II2AZO2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5A6IDQ#issuecomment-524411918>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA2VWWAILK6OIVW32AXJ3S3QGASEDANCNFSM4II2AZOQ> .

[sebymiano]

@frisso Unfortunately, I have always tested the simplebridge service in either the TC or XDP_DRV mode, but not in the XDP_SKB mode.
@havok4u Could you please try the same test with the simplebridge? Just to understand in the problem is within the service or something else.

[havok4u]

I will do that. Give me a few and I'll get you some results. Thanks

…

On Sat, Aug 24, 2019 at 12:19 PM Sebastiano Miano ***@***.***> wrote: @frisso <https://github.com/frisso> Unfortunately, I have always tested the simplebridge service in either the TC or XDP_DRV mode, but not in the XDP_SKB mode. @havok4u <https://github.com/havok4u> Could you please try the same test with the simplebridge? Just to understand in the problem is within the service or something else. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#202?email_source=notifications&email_token=AA2VWWBP2JJWUHLFUJ4TPNDQGFUQTA5CNFSM4II2AZO2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5CEEHA#issuecomment-524567068>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA2VWWAVMBZZY6DKOMXBZRLQGFUQTANCNFSM4II2AZOQ> .