Skip to content

[skip ci] Define REST API handlers for inspecting VCHs#6318

Merged
zjs merged 1 commit intovmware:feature/vic-machine-servicefrom
zjs:topic/vic-machine-service-impl-inspect
Sep 22, 2017
Merged

[skip ci] Define REST API handlers for inspecting VCHs#6318
zjs merged 1 commit intovmware:feature/vic-machine-servicefrom
zjs:topic/vic-machine-service-impl-inspect

Conversation

@zjs
Copy link
Member

@zjs zjs commented Sep 13, 2017

Introduce a pair of handlers for accessing information about the VCH with a given identifier within a vSphere target or datacenter.

There are ways in which this work is incomplete, marked by TODO:

  • Malformed managed object references are ignored
  • Malformed or unexpected PKI material is ignored

I think these are acceptable for now. We can address proper handling of invalid data returned by lower layers later, in a holistic way.

Resolves #6029.

@zjs zjs self-assigned this Sep 13, 2017
@jzt
Copy link
Contributor

jzt commented Sep 13, 2017

Lots of these files are in need of copyright headers.

@jzt
Copy link
Contributor

jzt commented Sep 13, 2017

@zjs I was thinking back on the portlayer when looking at this and wondering if we couldn't also benefit here from a refactor in order to organize the handlers into discrete chunks. Its a little different here since the portlayer is broken into each of its different component handlers (containers, images, storage, etc), but we could perhaps set configure_vic_machine.go up in a similar scheme. We could have a list of handlers like this and basically call configure for each one. That way maybe we could have a much cleaner configure_vic_machine.go with a collection of files by operation, like: vch_get.go, vch_list.go, vch_create.go, etc.

Probably not necessary now but maybe if we start thinking about it we can avoid a big refactor later on down the line.

@zjs
Copy link
Member Author

zjs commented Sep 14, 2017

We could have a list of handlers like this and basically call configure for each one.

When I started writing configure_vic_machine.go, I noticed that there seemed to be multiple approaches used by different projects. I followed the pattern Ivan recommended.

The impression I got was that the portlayer approach ended up being unnecessarily complex; it makes the configure_*.go file a bit shorter, but it's much less clear which handler will be invoked for a given API call since that information is now spread out across all of the Configure methods.

I'm not attached to any specific pattern, and there may be advantages to the portlayer approach I'm not aware of.

// GET /container/target/{target}/datacenter/{datacenter}/vch
api.GetTargetTargetDatacenterDatacenterVchHandler = &handlers.VCHDatacenterListGet{}

// POST /container/target/{target}/datacenter/{datacenter}/vch
Copy link
Contributor

@jzt jzt Sep 14, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor, but maybe we could edit the spec such that the /container/target/{target}/datacenter/{datacenter}/vch pattern becomes something like /container/target/{address}/datacenter/{name}/vch so we end up with method names like api.GetTargetAddressDatacenterNameVchHandler instead of api.GetTargetTargetDatacenterDatacenterVchHandler.

Again, pretty minor - not worth the time spent on it if you don't see it as a long term improvement.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really don't like the method names, and am open to changing the spec.

I worry that including vague things like {name} or {id} in the path might limit us in the future. If we start with a path like /foo/{name} and then want to add a sub-resource bar, we can't have /foo/{name}/bar/{name}, so we're stuck being inconsistent (with something like /foo/{name}/bar/{barName}) or refactoring (to something like /foo/{fooName}/bar/{barName}) and likely breaking clients using generated bindings.

To avoid painting ourselves into a corner like that, I think we'd want to qualify any generally-applicable parameters up front. (So {address} is probably okay as-is, but {name} or {id} should probably be qualified.)

I also think readability of the generated documentation is at least as important as readability of the generated go code, so we should keep that in mind as we consider changes.

Given that, maybe we should compare both the generated documentation and generate method names for the current style (/foo/{foo}) with some potential new styles like /foo/{fooId}?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with your concerns around {name} and {id}, especially in cases where you could supply a name or an id (datacenters/datastores for example?). I like the /foo/{fooID} suggestion as fooID can imply whatever it is you are using to identify a particular foo, and is not restricted to being a name or a UUID or whatever, but can be anything.

@zjs zjs force-pushed the topic/vic-machine-service-impl-inspect branch from 7747037 to 3105095 Compare September 14, 2017 17:38
@zjs
Copy link
Member Author

zjs commented Sep 14, 2017

Lots of these files are in need of copyright headers.

This should be fixed across all of my pending PRs.

@jzt
Copy link
Contributor

jzt commented Sep 14, 2017

@zjs once this is merged I will use it to build from when I take one of the tasks from the epic.

@zjs zjs force-pushed the topic/vic-machine-service-impl-inspect branch from 3105095 to 9100c2b Compare September 14, 2017 21:10
@zjs zjs requested a review from andrewtchin September 14, 2017 21:12
@zjs zjs force-pushed the topic/vic-machine-service-impl-inspect branch from 9100c2b to 17196ea Compare September 14, 2017 21:43
@zjs zjs requested a review from emlin September 14, 2017 22:55
@zjs zjs force-pushed the topic/vic-machine-service-impl-inspect branch from 61b715d to ead37fd Compare September 14, 2017 23:52

d.ID = params.VchID

vch, err := getVCH(params.HTTPRequest.Context(), d)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now portlayer is moving to trace.NewOperation to track different operation, to help log analysis. So suggest to add that into vic-machine service as well

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a good pattern to follow. I filed a separate issue for it so that we can make sure to fix this across the board: #6374 (several handlers have already been committed).

func (h *VCHGet) Handle(params operations.GetTargetTargetVchVchIDParams, principal interface{}) middleware.Responder {
d := buildData(
url.URL{Host: params.Target},
principal.(Credentials).user,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is wondering how the principal is set. is that the credential to login vic-machine service?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, we've only implemented basic authentication (where a username and password are passed via an Authorization header using the Basic scheme: https://tools.ietf.org/html/rfc7617)

Relevant code:

Implementing an additional type of authentication is tracked by #6033

Shares: asShares(d.ResourceLimits.VCHMemoryShares),
},
Resource: &models.ManagedObject{
ID: vchConfig.Container.ComputeResources[0].String(), // TODO: This doesn't seem to be an ID?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's resource pool reference object, *types.ManagedObjectReference

model.Network = &models.VCHNetwork{
Bridge: &models.VCHNetworkBridge{
PortGroup: &models.ManagedObject{
ID: vchConfig.Network.BridgeNetwork,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you test it? feels it's not right
you might reference here: https://github.com/vmware/vic/blob/master/lib/install/validate/config_to_data.go#L366
You could also reference the code in the above file for how to read back VCH configuration

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! It looks like it's always returning "bridge", which happened to be the name of my bridge network, so I didn't notice.

@zjs zjs force-pushed the topic/vic-machine-service-impl-inspect branch from 7d310e2 to f683764 Compare September 15, 2017 22:01
"github.com/docker/docker/opts"
"github.com/go-openapi/runtime/middleware"
"github.com/go-openapi/strfmt"
"github.com/vmware/govmomi/vim25/types"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gofmt will complain that there should be a space above this line separating vmware imports from external imports.

Copy link
Contributor

@jzt jzt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, after some minor changes.


d.ID = params.VchID

vch, err := getVCH(params.HTTPRequest.Context(), d)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we wrap parmas.HTTPRequest.Context() in a trace.NewOperation() we can chase these calls through the logs easier when debugging:

op := trace.NewOperation(params.HTTPRequest.Context(), "vch: %s", params.VchID)
vch, err := getVCH(op, d)

You can then use it to log with op.Infof(), op.Debugf(), etc.

func NewOperation(ctx context.Context, format string, args ...interface{}) Operation {


d.ID = params.VchID

vch, err := getVCH(params.HTTPRequest.Context(), d)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, re: trace.Operation

Sockets: int64(d.NumCPUs),
},
OperationsCredentials: &models.VCHEndpointOperationsCredentials{
User: vchConfig.Connection.Username,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth excluding username as well?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think including the operations usename is useful, and potentially important.

If the VCH suddenly stops working, being able to look at it and say "Oh, it's using my account, and I just changed the password!" or "Oh, it was using Joe's account, but... we just fired Joe and deactivated his account." might be helpful. (Or, alternatively, to verify that it's using a service account, as expected, and not Joe's account.)

model.Runtime.PowerState = string(powerState)

if public := vchConfig.ExecutorConfig.Networks["public"]; public != nil {
if public_ip := public.Assigned.IP; public_ip != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/public_ip/publicIP

if !vchConfig.HostCertificate.IsNil() {
docker_port = fmt.Sprintf("%d", opts.DefaultTLSHTTPPort)
} else {
docker_port = fmt.Sprintf("%d", opts.DefaultHTTPPort)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/docker_port/dockerPort

(golint will complain about using underscores in var names)


func (h *VCHListGet) Handle(params operations.GetTargetTargetVchParams, principal interface{}) middleware.Responder {
d := buildData(
d, err := buildData(params.HTTPRequest.Context(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, re: trace.Operation

@jzt
Copy link
Contributor

jzt commented Sep 15, 2017

@zjs Sorry, disregard my comments about trace.NewOperation.. just saw this.

d.Thumbprint = *thumbprint
}

if datacenter != nil {
Copy link
Contributor

@emlin emlin Sep 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm... is this necessary? saw this line: https://github.com/vmware/vic/pull/6318/files#diff-adbd7d5a586ec7aeba3724c39fc6a462R78
what I'm thinking is that even if datacenter is nil, you might want to validate target, the following logic could still be inside of this condition

Copy link
Member Author

@zjs zjs Sep 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This block translates a datacenter id (what we want the API to use) into a datacenter path (what the rest of the product uses). The validateTarget method is called here with an incomplete data object and the validator which is returned is only used within this block. (We can't look up the datacenter path if we have an invalid target or credentials.)

In all cases, including when datacenter is nil, validateTarget is called later to validate the full data object returned by this method.

return validator, nil
}

// Copied from list.go, and appears to be present other places. TODO: deduplicate
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also appears at inspect.go, should be unified

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree this should be unified, but don't think this PR is the right place to do that, which is why I've left this TODO.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

saw the todo, just add one comment for another duplication.

return fmt.Sprintf("Unknown: %s", err)
}
if upgrading {
return "Upgrade in progress"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it could also be configure in progress, see the method in inspect.go

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It sounds like we have a bug then; this method is copied as-is from list.go, which wasn't updated in #4847 when the reference to configure was added to inspect.go's version of this method.

We can sort that our when we address the above TODO and de-duplicate the code between inspect.go, list.go, and common.go. At the latest, this would happen as a part of #6032 (which will be resolved before this feature branch is merged to master).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure

Introduce a pair of handlers for accessing information about the VCH
with a given identifier within a vSphere target or named datacenter.

There are ways in which this work is incomplete, marked by TODO:
 - We accept datacenters by name, not ID
 - The name of the VCH's compute resource is returned, not its ID
 - The names of portgroups used by the VCH are returned, not the IDs
 - Malformed or unexpected PKI material is ignored
@zjs zjs force-pushed the topic/vic-machine-service-impl-inspect branch from 23ed830 to a6d7f53 Compare September 22, 2017 18:06
@zjs zjs merged commit 92aa233 into vmware:feature/vic-machine-service Sep 22, 2017
@zjs zjs deleted the topic/vic-machine-service-impl-inspect branch September 22, 2017 18:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants