Updating IPAM config with results from HNS create network call. #2429

pradipd · 2019-08-07T21:33:01Z

In windows HNS manages IPAM. If the user does not specify a subnet, HNS will choose one
for them. However, in order for the IPAM to show up in the output of "docker inspect",
we need to update the network IPAMv4Config field.

Signed-off-by: Pradip Dhara pradipd@microsoft.com

pradipd · 2019-08-07T21:46:54Z

This is a fix for moby/moby#38358. However, this is a fix for only the original issue. I.e. The default docker NAT network is missing.

This root cause is that when the user does not specify a subnet, windows (HNS) will chose one for them. However, we do not show the subnet/gateway in the output of docker inspect. However, the subnet and gateway are present and will work. Unfortunately any scripts that parse the output of "docker inspect nat" will fail.

Another simpler way to repro this issue is:
On windows server 2019.
docker network create -d nat mynat
docker inspect mynat

The output of docker inspect mynat will show
"Config": [ { "Subnet": "0.0.0.0/0" } ]

pradipd · 2019-08-07T21:56:22Z

Regarding the solution, I investigated @arkodg's suggesting of using docker's ipam and getting rid of the window's ipam plugin (https://github.com/docker/libnetwork/blob/master/ipams/windowsipam/windowsipam.go).
It may be possible to make that work (pradipd/moby@4a8e6fc) but it still requires some more investigation. And, I was told it may not work on server 2016. So, I went with this approach of updating the network object from the driver. I realize that this may not be ideal, however, I don't think we are the first to need the driver to update the network configuration:
https://github.com/pradipd/moby/blob/master/vendor/github.com/docker/libnetwork/controller.go#L836
We could also follow that model. But, I felt the callback was cleaner.

pradipd · 2019-08-07T21:56:39Z

@arkodg @mavenugo PTAL.

drivers/windows/windows.go

network.go

arkodg · 2019-08-13T01:36:30Z

@pradipd, would it be possible to add an API to HNS to GET a free subnet or even check if a subnet is free or not (Windows IPAM in libnetwork could pass some random value), then there would be a clear separation between the Driver and the Network

pradipd · 2019-08-22T20:48:58Z

Sorry for the delayed response. I was OOF.

pradipd · 2019-08-22T20:51:52Z

@arkodg: Adding an API would require us to backport the change to 2016. I don't think this issue is important enough to backport to 2016. There are workarounds that can unblock users.

pradipd · 2019-08-22T21:16:37Z

Oops. Sorry. Did not mean to resolve conversation. Will try and reopen

arkodg · 2019-08-22T21:58:51Z

@pradipd is it possible to use this CMD or the underlying API - https://docs.microsoft.com/en-us/powershell/module/ipamserver/find-ipamfreeaddress?view=win10-ps

pradipd · 2019-08-22T22:17:48Z

No. That API is for a different windows server role/feature that is not involved with HNS. I.e. that API is for when you setup your Windows Server as a DHCP server.

selansen · 2019-08-23T16:28:44Z

Could you pls rebase and submit again?
latest code base fixes the CI failure .

In windows HNS manages IPAM. If the user does not specify a subnet, HNS will choose one for them. However, in order for the IPAM to show up in the output of "docker inspect", we need to update the network IPAMv4Config field. Signed-off-by: Pradip Dhara <pradipd@microsoft.com>

daschott · 2019-09-12T18:03:14Z

@pradipd @arkodg @selansen is there anything else needed here?

selansen

LGTM

selansen · 2019-09-12T19:34:35Z

@arkodg PTAL .

arkodg · 2019-09-12T21:17:13Z

network.go

 	return nil
 }

+func (n *network) UpdateIpamConfig(ipV4Data []driverapi.IPAMData) {


Can we make this generic to ipv4 and ipv6

I just undid that change based off feedback from @selansen . Windows wasn't using it, so, we decided we can add it later when we (or some other driver) needs it.

I mean windows wasn't passing any IPv6 data in using UpdateIpamConfig()

I still stick by my comment. if we are not using it, we dont need it. It will confuse everyone who is looking into the code after few months wondering why is this code exists if we are not using it.

selansen · 2019-09-13T15:19:03Z

Thanks a lot @arkodg @pradipd

full diff: moby/libnetwork@92d1fbe...96bcc0d changes included: - moby/libnetwork#2429 Updating IPAM config with results from HNS create network call - addresses moby#38358 - moby/libnetwork#2450 Always configure iptables forward policy - related to moby#14041 and moby/libnetwork#1526 Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

full diff: moby/libnetwork@92d1fbe...96bcc0d changes included: - moby/libnetwork#2429 Updating IPAM config with results from HNS create network call - addresses moby/moby#38358 - moby/libnetwork#2450 Always configure iptables forward policy - related to moby/moby#14041 and moby/libnetwork#1526 Signed-off-by: Sebastiaan van Stijn <github@gone.nl> Upstream-commit: 75477f0b3c77f2108a6b5586dbc246c52b479941 Component: engine

full diff: moby/libnetwork@92d1fbe...96bcc0d changes included: - moby/libnetwork#2429 Updating IPAM config with results from HNS create network call - addresses moby#38358 - moby/libnetwork#2450 Always configure iptables forward policy - related to moby#14041 and moby/libnetwork#1526 Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 75477f0) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

full diff: moby/libnetwork@92d1fbe...96bcc0d changes included: - moby/libnetwork#2429 Updating IPAM config with results from HNS create network call - addresses moby/moby#38358 - moby/libnetwork#2450 Always configure iptables forward policy - related to moby/moby#14041 and moby/libnetwork#1526 Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 75477f0b3c77f2108a6b5586dbc246c52b479941) Signed-off-by: Sebastiaan van Stijn <github@gone.nl> Upstream-commit: 559be42fc26048f4069de64f84202803a113413a Component: engine

full diff: moby/libnetwork@92d1fbe...96bcc0d changes included: - moby/libnetwork#2429 Updating IPAM config with results from HNS create network call - addresses moby#38358 - moby/libnetwork#2450 Always configure iptables forward policy - related to moby#14041 and moby/libnetwork#1526 Signed-off-by: Sebastiaan van Stijn <github@gone.nl> Signed-off-by: zach <Zachary.Joyner@linux.com>

subbunori · 2019-10-31T21:02:37Z

Hi, How to know if this change is release in latest docker enterprise version?

arkodg · 2019-10-31T21:22:06Z

@subbunori can you please update to latest 19.03 version and this fix should be present
here are the commits in the 19.03 branch - https://github.com/docker/libnetwork/commits/bump_19.03

selansen reviewed Aug 11, 2019

View reviewed changes

drivers/windows/windows.go Outdated Show resolved Hide resolved

selansen reviewed Aug 11, 2019

View reviewed changes

network.go Outdated Show resolved Hide resolved

This was referenced Aug 22, 2019

Cannot configure fixed-cidr in Docker for Windows via Whaley UI docker/for-win#625

Closed

Default Docker NAT network gateway is missing on Windows Server 2019 moby/moby#38358

Closed

pradipd force-pushed the windows-nosubnet branch from aaaaf5a to c6c6036 Compare August 23, 2019 05:43

pradipd force-pushed the windows-nosubnet branch from c6c6036 to 64f88ad Compare August 23, 2019 16:58

selansen approved these changes Sep 12, 2019

View reviewed changes

arkodg suggested changes Sep 12, 2019

View reviewed changes

arkodg approved these changes Sep 13, 2019

View reviewed changes

selansen merged commit 45c7102 into moby:master Sep 13, 2019

thaJeztah mentioned this pull request Sep 20, 2019

bump libnetwork to 96bcc0dae898308ed659c5095526788a602f4726 moby/moby#39967

Merged

thaJeztah mentioned this pull request Sep 20, 2019

[19.03 backport] bump libnetwork to 96bcc0dae898308ed659c5095526788a602f4726 docker-archive/engine#372

Merged

olljanat mentioned this pull request Jul 16, 2025

libnetwork: Use generic IPAM also in Windows moby/moby#50427

Open

Updating IPAM config with results from HNS create network call. #2429

Updating IPAM config with results from HNS create network call. #2429

Uh oh!

Conversation

pradipd commented Aug 7, 2019

Uh oh!

pradipd commented Aug 7, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pradipd commented Aug 7, 2019

Uh oh!

pradipd commented Aug 7, 2019

Uh oh!

Uh oh!

Uh oh!

arkodg commented Aug 13, 2019

Uh oh!

pradipd commented Aug 22, 2019

Uh oh!

pradipd commented Aug 22, 2019

Uh oh!

pradipd commented Aug 22, 2019

Uh oh!

arkodg commented Aug 22, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pradipd commented Aug 22, 2019

Uh oh!

selansen commented Aug 23, 2019

Uh oh!

daschott commented Sep 12, 2019

Uh oh!

selansen left a comment

Choose a reason for hiding this comment

Uh oh!

selansen commented Sep 12, 2019

Uh oh!

arkodg Sep 12, 2019

Choose a reason for hiding this comment

Uh oh!

pradipd Sep 13, 2019

Choose a reason for hiding this comment

Uh oh!

pradipd Sep 13, 2019

Choose a reason for hiding this comment

Uh oh!

selansen Sep 13, 2019

Choose a reason for hiding this comment

Uh oh!

selansen commented Sep 13, 2019

Uh oh!

subbunori commented Oct 31, 2019

Uh oh!

arkodg commented Oct 31, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

pradipd commented Aug 7, 2019 •

edited

Loading

arkodg commented Aug 22, 2019 •

edited

Loading