cli can't talk to external kubernetes

[phemmer]

I built the code from PR #797 in attempt to run openshift on top of GKE. Openshift starts, but cli commands which need to talk to kubernetes fail.

For example, trying to launch the hello-openshift example:

# openshift cli create -f hello-pod.json
F0130 23:50:19.119121     236 create.go:75] pods "" not found

From discussion with @smarterclayton on IRC, this error is because the cli needs to talk to both openshift & kubernetes, but they exist as separate services, and the client doesn't support this.

Full reproducible example (needs image built with PR #797):

cid=$(docker run -d --privileged -v /var/run/docker.sock:/var/run/docker.sock --net=host openshift/origin start master --kubernetes=http://localhost:8080 --etcd=localhost)
docker logs -f $cid 2>&1 | grep -qm1 'Started OpenShift API'
docker exec $cid openshift cli --kubeconfig=/var/lib/openshift/openshift.local.certificates/admin/.kubeconfig create -f https://raw.githubusercontent.com/openshift/origin/master/examples/hello-openshift/hello-pod.json

[phemmer]

From IRC:

18:27 <qubit> trying to `cli create` on the hello-openshift example just results in `create.go:75] pods "" not found`. But I'll tackle this issue later.
18:29 <claytonc> qubit: i think that's a known gap
18:29 <claytonc> there's no way to tell the cli to use two different servers right now
18:30 <claytonc> qubit: so you can't use the cli to talk to kube and openshift at the same time
18:34 <qubit> ah. I'm not familiar with what goes on during a `create`, but from what you're saying, it needs to make api calls to both openshift & kubernetes. If this is indeed the case, then wouldn't this need to be added to support running with a non-bundled 
              kubernetes.
18:34 <claytonc> yes it would - there were two approaches i was considering - passthrough proxy (which owuld allow double auth, and kube master to delegate auth to openshift as well)
18:35 <claytonc> and then the standard double config
18:35 <claytonc> where kubeconfig would contain two sets of clients/certs/etc
18:35 <claytonc> that's mostly there
18:35 <claytonc> just some plumbing
18:39 <qubit> the first option feels cleaner to me. having the client talk to both openshift, and the thing that runs behind openshift (kubernetes) doesn't feel right.
18:41 <claytonc> qubit: yeah - the extra hop perf wise always sucks a bit
18:42 <claytonc> openshift as microservices shouldn't have to proxy - there's a long discussion in kube about how we register services in such a way that kubectl can handle them generically
18:42 <claytonc> most of the support is there, it'sj ust that last step of registration
18:42 <claytonc> that would be the "i want the features without the paas"
18:42 <claytonc> side of the house
18:45 <qubit> claytonc: is there an open issue I can track for this? Or should I create one?
18:46 <claytonc> qubit: please do - it's not written down anywhere so that's as good an any

As noted in the chat, I'm in favor of having openshift proxy the kubernetes calls. Having the client talk to both feels dirty, and would make the configuration more complex.

[smarterclayton]

Proxy gives a big advantage of either doing no auth pass through, or optionally filtering api requests. So someone can have a kube cluster, stand up openshift, let openshift impose ACL on top of Kube, and keep using core resources.

[phemmer]

I added a quick and dirty proxy which seems to solve both this issue, and the console issue in #820.

https://github.com/phemmer/openshift-origin/commit/25507025d697a588091820f622164939f155389a

Haven't submitted as a PR as it's probably not release quality.

[smarterclayton]

Thanks - I'll take a look tomorrow. We probably want to put more sophisticated behavior (auth passthrough and delegation) around it eventually, but I want to get the base scenario up and working, and this is really close to that.

On Feb 1, 2015, at 5:25 PM, Patrick Hemmer notifications@github.com wrote:

I added a quick and dirty proxy which seems to solve both this issue, and the console issue in #820.

phemmer@2550702

Haven't submitted as a PR as it's probably not release quality.

—
Reply to this email directly or view it on GitHub.

[smarterclayton]

#839 contains a commit very similar to yours, but I reorganized some of the other code so we can ensure calls get authorized at our level before being passed down.

[phemmer]

Closing since #839 has landed.