Run on windows by bypassing dockerpty.

[dopry]

This is a proposal for a way to implement windows support for compose in the near term.

A primary blocker to support for running on windows in dockerpty's dependency on fcntl to lock the file descriptors for the pty streams. This is only used when docker-compose is ran in attached mode. This PR removes attached mode support on windows. The assumption is that you could simply use docker itself to attach to the containers as a windows user.

I have tested docker build, run, and up with some simple projects that do not use volumes and all seems to work as expected.

The caveat here being that volumes on windows generally don't work well with vagrant, docker-machine, docker, and those same limitations apply to compose.

The upside to this simple patch is it allows me to use docker-compose to build and start environments as long as I am creating containers that do not depend on volumes. I'll be using it for local development and experimentation.

I'm not quite sure how to go about implementing tests. I'm not sure what is required from a CI perspective to get windows testing in place and how best to approach disambiguating the platform in tests and build pipeline.

I'm happy to collaborate with anyone who is willing to work on this feature branch and help implement full windows support for compose, although my ability to work further it is will likely be limited to testing and debugging additional changes.

[wendal]

it looks great!

[ahmetb]

👏

[friism]

Yes!

[dopry]

I ran into an uncaught exception when the docker host environment variables weren't set and docker-compose tried to connect to the local unix socket. I added an error message to recommend users to docker-machine env --help.

I currently have a fairly complex django environment running with postgres, redis, rabbitmq, django app, django celery worker and haproxy. I've tested scaling the workers and app containers successfully. links are working properly between containers in my compose environment.

Everything seems to be working fine. I'm very happy to have compose on windows.

[colltoaction]

What's the way to try this? Tried the following and got a an ImportError with the module Queue:

pip install --upgrade git+https://github.com/dopry/compose.git@windows

[dopry]

I am unable to reproduce the ImportError for Queue

Both of the following work for me.

git clone https://github.com/dopry/compose.git
cd  compose
python setup.py develop 

pip install --upgrade git+https://github.com/dopry/compose.git@windows

[colltoaction]

Maybe I messed up my Python installations. I have pypy2 and python3. Are both supported?

[dopry]

As far as i know python 3 is not officially supported in docker-composer. I'm testing on python 2.7.

[aanand]

Thanks for this - I've been investigating Windows binaries for the past few days and have hit on the exact same problem.

It would, however, be quite rubbish to release a binary that can't do an interactive docker-compose run. See #1051 (comment) for where I'm currently at.

[dopry]

To be honest I've never used docker-compose run. I use build, up, and
scale. I'm not even sure why docker-compose includes it instead of
recommending to use the native docker client to attach to containers. It's
duplicate functionality in the ecosystem of tools. I personally doubt
anyone would miss it.

On Tue, Aug 25, 2015 at 8:58 AM, Aanand Prasad notifications@github.com
wrote:

Thanks for this - I've been investigating Windows binaries for the past
few days and have hit on the exact same problem.

It would, however, be quite rubbish to release a binary that can't do an
interactive docker-compose run. See #1051 (comment)
#1051 (comment)
for where I'm currently at.

—
Reply to this email directly or view it on GitHub
#1900 (comment).

Darrel O'Pry
The Spry Group, LLC.
http://www.spry-group.com
718-355-9767 x101

[ahmetb]

+1 for the comment above.

[aanand]

Lots of people use docker-compose run. Even if we wanted to remove it, we can't do so without a proper deprecation plan.

[dopry]

well it doesn't run on windows at all right now, so technically you're not
deprecating anything.

On Tue, Aug 25, 2015 at 1:01 PM, Aanand Prasad notifications@github.com
wrote:

Lots of people use docker-compose run. Even if we wanted to remove it, we
can't do so without a proper deprecation plan.

—
Reply to this email directly or view it on GitHub
#1900 (comment).

Darrel O'Pry
The Spry Group, LLC.
http://www.spry-group.com
718-355-9767 x101

[colltoaction]

I usually run docker-compose run when I'm starting my day. I run the
webserver, database, etc. and kill them when I'm done.

On Tue, Aug 25, 2015 at 2:08 PM dopry notifications@github.com wrote:

well it doesn't run on windows at all right now, so technically you're not
deprecating anything.

On Tue, Aug 25, 2015 at 1:01 PM, Aanand Prasad notifications@github.com
wrote:

Lots of people use docker-compose run. Even if we wanted to remove it, we
can't do so without a proper deprecation plan.

—
Reply to this email directly or view it on GitHub
#1900 (comment).

Darrel O'Pry
The Spry Group, LLC.
http://www.spry-group.com
718-355-9767 x101

—

Reply to this email directly or view it on GitHub
#1900 (comment).

[dopry]

So you're doing that on windows?

On Tue, Aug 25, 2015 at 1:21 PM, Martín Coll notifications@github.com
wrote:

I usually run docker-compose run when I'm starting my day. I run the
webserver, database, etc. and kill them when I'm done.

On Tue, Aug 25, 2015 at 2:08 PM dopry notifications@github.com wrote:

well it doesn't run on windows at all right now, so technically you're
not
deprecating anything.

On Tue, Aug 25, 2015 at 1:01 PM, Aanand Prasad <notifications@github.com

wrote:

Lots of people use docker-compose run. Even if we wanted to remove it,
we
can't do so without a proper deprecation plan.

—
Reply to this email directly or view it on GitHub
#1900 (comment).

Darrel O'Pry
The Spry Group, LLC.
http://www.spry-group.com
718-355-9767 x101

—

Reply to this email directly or view it on GitHub
#1900 (comment).

—
Reply to this email directly or view it on GitHub
#1900 (comment).

Darrel O'Pry
The Spry Group, LLC.
http://www.spry-group.com
718-355-9767 x101

[colltoaction]

I'd love to do that. I SSH into the machine and use docker-compose from
inside there. I'm basically not using the docker client from Windows until
#1085 gets fixed :)

[dopry]

So why do you explicitly use docker-compose run over docker-compose up when
developing? Are you running in interactive terminal mode? Do you use it?
Why for? Is it better than using docker exec/attach ?

On Tue, Aug 25, 2015 at 1:39 PM, Martín Coll notifications@github.com
wrote:

I'd love to do that. I SSH into the machine and use docker-compose from
inside there. I'm basically not using the docker client from Windows until
this issue gets fixed :)

On Tue, Aug 25, 2015 at 2:38 PM dopry notifications@github.com wrote:

So you're doing that on windows?

On Tue, Aug 25, 2015 at 1:21 PM, Martín Coll notifications@github.com
wrote:

I usually run docker-compose run when I'm starting my day. I run the
webserver, database, etc. and kill them when I'm done.

On Tue, Aug 25, 2015 at 2:08 PM dopry notifications@github.com
wrote:

well it doesn't run on windows at all right now, so technically
you're
not
deprecating anything.

On Tue, Aug 25, 2015 at 1:01 PM, Aanand Prasad <
notifications@github.com

wrote:

Lots of people use docker-compose run. Even if we wanted to remove
it,
we
can't do so without a proper deprecation plan.

—
Reply to this email directly or view it on GitHub
<
#1900 (comment)
.

Darrel O'Pry
The Spry Group, LLC.
http://www.spry-group.com
718-355-9767 x101

—

Reply to this email directly or view it on GitHub
<#1900 (comment)
.

—
Reply to this email directly or view it on GitHub
#1900 (comment).

Darrel O'Pry
The Spry Group, LLC.
http://www.spry-group.com
718-355-9767 x101

—
Reply to this email directly or view it on GitHub
#1900 (comment).

—
Reply to this email directly or view it on GitHub
#1900 (comment).

Darrel O'Pry
The Spry Group, LLC.
http://www.spry-group.com
718-355-9767 x101

[colltoaction]

First, I have no idea how to use exec/attach. Now that that's clear, I use run because it spins up a couple of containers that I don't usually want to be running, all in one command.
Also, I'm able to just Ctrl+C to stop them. I'm able to see their logs from the console (e.g., requests arriving to my php-fpm server).

[dopry]

@tinchou, You can use docker compose up [service name] to start a subset of
services in much the same way as run. Is there a reason you favor
docker-compose run [servicename] to docker-compose up [servicename]?

On Tue, Aug 25, 2015 at 1:56 PM, Martín Coll notifications@github.com
wrote:

First, I have no idea how to use exec/attach. Now that that's clear, I use
run because it spins up a couple of containers that I don't usually
want to be running, all in one command.
Also, I'm able to just Ctrl+C to stop them. I'm able to see their logs
from the console (e.g., requests arriving to my php-fpm server).

—
Reply to this email directly or view it on GitHub
#1900 (comment).

Darrel O'Pry
The Spry Group, LLC.
http://www.spry-group.com
718-355-9767 x101

[colltoaction]

@dopry

I'm able to just Ctrl+C to stop them. I'm able to see their logs from the console (e.g., requests arriving to my php-fpm server).

[dopry]

@tinchou, all of those things work with docker-compose up [servicename]... docker compose run is actually meant to run commands on all the hosts implementing a service.. ie) you want to restart a service on all of them or some similar action.

[dopry]

@aanand, how do I resolve ERROR: for 5 b'Conflict. The name "custom-container" is already in use by container b2dd5be30e09. You have to delete (or rename) that container to be able to reuse that name.' It looks like a problem in the test suite/build environment, not the PR itself.

[dnephin]

I've noticed that test "Test that calling scale on a service that has a custom container name" has been flaking a few times today against 1.8.1. I've created #1922 to investigate it further. I agree I don't think it's related to your branch.

I'll start another run of this branch.

[dopry]

@dnephin, still looks like an upstream build/ci issue.

I give up. @aanand has already indicated the approach is rubbish... so I'll just go use https://github.com/rancher/rancher-compose instead, since they're not running around chasing their tails.

[ahmetb]

I think this patch should have been accepted. This fixes a lot of things and does not ship a crappy experience at all. It enables >90% of the functionality for the Windows users (who have no other option today) and the terminal problems can be fixed over time. I don't think there's anything wrong with shipping docker-compose run disabled on Windows.

For instance, the Windows port of Docker client was initially in a similar shape. We shipped something where the console/attach functionality didn't really work well but it still did a lot of things for the users. Literally nobody complained about it and everybody was so happy to use the Windows client to do docker build/run/ps/stop/start etc. We did this early on (around April 2015) and lots of people have been using it since then.

I think we need maintainers' input here @aanand @bfirsh, please comment if you find this PR useful and acceptable at all.

[friism]

+1, docker-compose on Windows without run is better than nothing.

[bfirsh]

+1 I think this is a good first step.

[HCKgit]

+1