Allow drivers to get the name along with uuid.

[shettyg]

The user passes a "name" to libnetwork apis (for e.g., a network name or an endpoint name.). The libnetwork does not pass the same name to the driver but passes a uuid instead. IMO, the "name" is an information from the user that should ideally reach the driver. But currently libnetwork blocks it.

It would be nice if libnetwork passes this information to the driver too along with the uuid. This way, the driver can keep a relationship between the uuid that libnetwork generates and the name that the user provides.

Usecases:
The usecases is for hybrid networks where a docker container and a VM and a physical machine could be reachable to each other. IMO, this provides a good transition path from current workloads to pure-docker workloads. In this case, a network would have to be pre-created outside docker. But we can make docker aware of this network with 'docker network create'.

1. A pool of endpoints can be pre-created outside docker with names that identify them. When that name is passed to libnetwork api, the driver does not receive it. There is no way for the driver to relate to the uuid that it receives as it is generated by docker. One could argue that the information can be passed with labels. Unfortunately, the "join" api does not pass the same label to the driver. So this makes the "join" api unusable in such cases and all the functionalities have to remain in driver.CreateEndpoint.

Even assuming that the driver has to live with this restriction, a 'docker network join' would look unwieldy and confusing looking something like:
docker network join CONTAINER NETWORK --label ENDPOINT=OS-UUID --label NETWORK=OS-UUID

[mapuri]

The libnetwork does not pass the same name to the driver but passes a uuid instead.

I think there is a plan to pass driver specific config/info as part of network create in form of labels #106 (comment). Because it might not just be network name that driver may need. But I am wondering if network name can just always be passed as one of the generic labels. That shall address this right?

[shettyg]

@mapuri
Yes. A 'network-create' does have the ability to receive labels. But a driver.Join() does not receive the labels provided when you call "docker network join". (Please see: #133 for more details). It instead receives the labels that you provide with 'docker create container'

Irrespective of the resolution to the driver.Join() issue, I do not see a need to supply the name twice to the command line when that information is already available with libnetwork.

[mapuri]

Yes. A 'network-create' does have the ability to receive labels. But a driver.Join() does not receive the labels provided when you call "docker network join". (Please see: #133 for more details). It instead receives the labels that you provide with 'docker create container'

I should have been more elaborate of my thought process. I meant the uuid for the network and it's corresponding info like name and other labels will be passed just once to the driver at time of network create i.e. when user issues 'docker network create'. So driver can map the libnetwork's uuid to corresponding name and other labels. Similar logic can apply for endpoints create as well.

Irrespective of the resolution to the driver.Join() issue, I do not see a need to supply the name twice to the command line when that information is already available with libnetwork.

If my understanding above is correct, once a network/endpoint has been created libnetwork can just keep passing the uuids to the driver (for instance during join) and driver will have enough info to fetch the other details from it's own data structures. And there is no need to supply the name and other labels passed at time of create again to the driver, right?

As for passing names to the command-line instead of uuids, I think wherever possible it is better to allow passing either names (and internally map them to uuids in libnetwork before calling into driver) or uuids. That shall keep the cli consistent with rest of docker cli that can consume both names and uuids today.

Let me know if I am missing something.

[shettyg]

@mapuri
What you suggest is possible. According to @mrjana, endpointcreate can become portable across hosts. So the driver has to maintain the relationship between docker generated uuid and name across hosts too. I would like to avoid managing and maintaining this relationship. I foresee it becoming a pain in the neck.

[shettyg]

@mrjana @mavenugo
This is hopefully my last outstanding issue that I need some clarity on while writing a driver. I would like it if you guys provide your thoughts.

[mrjana]

@shettyg The name that is assigned to the network and endpoint objects are not long lasting i.e they can be replaced by something else by the user for the same object. That is why we don't pass this along to keep the driver api simple. Also this way we can make sure that the objects are referred to in a consistent manner anywhere in the cluster. If you would like to do some optimization based on names you can pass some reference as a label but please keep in mind that libnetwork will not do anything to ensure consistency of these labels across different objects i.e if you assign the same label to two different objects libnetwork will happily accept that. It's up to whoever manages the labels to ensure consistency if that is what the semantics that they want for any particular label.

[erikh]

The problem for us is that we associate network names with multihost definitions in some cases; this will cause libnetwork to give us a UUID for each host that might be different, if we provided our own centralized state implementation (which we wish to do).

I think we should have this freedom. We're free to discard the data if it's meaningless. I'm pretty sure we could poll the client API for this data if this is omitted, which is a pretty significant hack.

[mapuri]

+1 for passing supplied names to the driver as part of network and endpoint creation. The usecase is the scenario where the network driver get's some of it's information from a network controller (which will mostly not be aware of libnetwork generated UUIDs), so passing the name to the driver helps the driver fetch more state from the controller.

BTW once #222 is available, this can be achieved by using labels as well but then name information will look redundant. For instance,

docker network create --label driver.network-name=foo --label driver.other-label=abc foo

v/s

docker network create --label driver.other-label=abc foo

@shettyg

So the driver has to maintain the relationship between docker generated uuid and name across hosts too. I would like to avoid managing and maintaining this relationship. I foresee it becoming a pain in the neck.

I understand, but the driver might need to manage other relationships as well (like mapping of UUIDs to other driver-specific labels). So I am not sure if treating name as a special case and passing it again to the driver as part of Join() is required.

@erikh

this will cause libnetwork to give us a UUID for each host that might be different

From @mavenugo reply here #129 (comment) I think libnetwork shall ensure the UUID to name mapping are seen to be same on all hosts.

[shettyg]

From @mavenugo reply here #129 (comment) I think libnetwork shall ensure the UUID to name mapping are seen to be same on all hosts.

@mapuri
As I understand, @mavengo is using etcd to store that information across hosts. I may be a little ignorant here, but from what I know etcd scales up to 9 hosts right now (with good redundancy capabilities) with a goal of 50 hosts. For use cases, wherein we want to scale this to a lot more hosts, we can no longer depend on uuid generated by libnetwork. So the 'name' provided by the user becomes important.

[mavenugo]

@shettyg @mapuri thanks for speaking for me :-), but am not using etcd :-).
But, I don't think that matters to this discussion. UUID is by definition is universal and unique and we use it for identifying the network.

[shettyg]

@mavenugo
Oops, sorry about that.
For the main question here, will the 'name' be available to libnetwork across hosts too? Since labels passed to Join and CreateEndpoint are supposed to be different, would it makes sense that libnetwork provides the name to the underlying driver to solve this issue? Do you have any other recommendations?

[jainvipin]

@mavenugo @mrjana - I understand the worry about uniqueness, regardless if the name is what docker network accepted, it would make sense to me to have a correlation in the driver, especially if a higher level orchestrator is going to ensure that network names and endpoint names are unique.

If it is not too difficult to add, I'd +1 @shettyg request to allow network/endpoint names be passed to the driver.

[mrjana]

@shettyg @erikh @jainvipin @mapuri There have been lots of questions regarding why name is not passed down as part of the driver api. I will try to answer the question with some explanation on what our thinking here is and let us know if it makes sense for you guys.

When we started exposing plugin interfaces to the ecosystem the thinking was to expose interfaces at every extension point with only the absolutely necessary bits exposed at that extension point. The aim was not to create a hugely all-encompassing interface through which a full fledged application can be built. The reason is very simple. We wanted to provide enough flexibility to the end-user to pick and choose different solutions for different extension points. If we had one big interface then the user can only choose one ecosystem solution for all the layers. This is exactly the same as golang's interface philosophy which recommends to keep the interface to be as minimal as possible and if need be create multiple such interfaces.

Coming back to network driver plugin the interface it represents is to provide a driver to plumb the network path based on user request. The driver can plumb the network path using various different networking technologies as it wishes but the scope should only be about plumbing the network path. To do this job the only constructs that it needs are network ids/endpoint ids etc as long as they are guaranteed to be unique across the network(which they are).

If we pass down the name along with the id then we are tying in the semantics of the id and the name to be one and the same. Which is absolutely not true. The lifetime of an id and the name are very different. And if we decide to provide a flexibility to the user in the future to have a different scope to the name then it would not be possible if we couple it with the id. On top of all this the name is not required by the driver to do it's plumbing job. Our philosophy is if something is not needed for an interface to do it's job then it should not be provided because once we provide it, it is much difficult to remove it.

The name is a construct that is needed only in the management layer of the solution. So if there is an extension point for the management layer in the future then the name would definitely be a candidate to be passed down in that layer.

Ultimately all of this thinking is focussed on the end user and the need for providing consistent user experience. If we define an interface in a far too wide scope or if we enable plugins to assign different semantics to a particular construct then imo it ultimately leads to incoherent user experience.

Hope this clarifies our position on this matter and why we are reluctant to pass down the name via the driver api.