Lifecycle hook doesn't execute when callilng service.Up in the API

[chris-gputrader]

I've been pounding my head against this and am not sure if I'm not doing something incorrect or if its the codebase but lifecycle hooks don't seem to get executed when calling the API directly. I have the following example code to test this.

package main

import (
	"context"
	"flag"
	"log"
	"os"
	"time"

	"github.com/compose-spec/compose-go/v2/loader"
	"github.com/compose-spec/compose-go/v2/types"
	"github.com/docker/cli/cli/command"
	"github.com/docker/cli/cli/flags"
	"github.com/docker/compose/v2/pkg/api"
	"github.com/docker/compose/v2/pkg/compose"
)

func main() {
	// Add command line flag for config file
	configFile := flag.String("config", "compose.yaml", "Path to compose config file (yaml or json)")
	flag.Parse()

	// Read the configuration file
	bytes, err := os.ReadFile(*configFile)
	if err != nil {
		log.Fatalf("Failed to read config file: %v", err)
	}

	// Setup Docker client config
	dockerCli, err := command.NewDockerCli()
	if err != nil {
		log.Fatalf("Failed to create docker cli: %v", err)
	}

	opts := flags.NewClientOptions()
	if err := dockerCli.Initialize(opts); err != nil {
		log.Fatalf("Failed to initialize docker cli: %v", err)
	}

	// Create compose service
	service := compose.NewComposeService(dockerCli)

	// Load the project from config
	project, err := loader.Load(types.ConfigDetails{
		ConfigFiles: []types.ConfigFile{
			{
				Content: bytes,
			},
		},
		Environment: types.NewMapping(os.Environ()),
	})
	if err != nil {
		log.Fatalf("Failed to load project: %v", err)
	}

	// Setup options
	options := api.UpOptions{
		Create: api.CreateOptions{
			RemoveOrphans: true,
		},
		Start: api.StartOptions{
			Wait: true,
		},
	}

	// Create context with timeout
	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
	defer cancel()

	// Pull the images
	if err := service.Pull(ctx, project, api.PullOptions{}); err != nil {
		log.Fatalf("Failed to pull images: %v", err)
	}

	//log the project object
	log.Printf("Project: %v\n", project)

	// Start the project
	if err := service.Up(ctx, project, options); err != nil {
		log.Fatalf("Failed to start project: %v", err)
	}

	log.Println("Services started successfully!")
}

Here is a simple compose.yml to test with:

services:
  alpine:
    image: alpine
    command: sleep 1000000
    post_start:
      - command: /bin/sh -c "echo somecontent > /root/file.txt"      
        privileged: true

The container gets created but the post_start hook never runs.

Any ideas or am I just calling this wrong?

[maggie44]

#11210 (comment)

[maggie44]

It would be worth checking this to make sure it creates the services properly. docker inspect will help indicate if there are any differences between this approach and using docker compose up directly. It you do explore it further, do come back and post any updates as they will help me too.

https://gist.github.com/maggie44/98d870da0961b61c32f6e55407697f76

[chris-gputrader]

It would be worth checking this to make sure it creates the services properly. docker inspect will help indicate if there are any differences between this approach and using docker compose up directly. It you do explore it further, do come back and post any updates as they will help me too.

https://gist.github.com/maggie44/98d870da0961b61c32f6e55407697f76

Thanks, I updated that gist and included a lifecycle hook 'post_start' and verified that the container did not run the lifecycle hook, so something is not right here.

[chris-gputrader]

Not sure if this has something to do with it but just looking through the code from the change to include lifecycle hooks: #12166

It appears as though no changes were made to up.go which might be why the hooks are being ignored, changes were made to start.go and others.

[maggie44]

I assume it is working when using the docker compose binary on the same host (i.e. using docker compose up instead of this custom code)?

[chris-gputrader]

Yes here is the code that I used to test:

package main

import (
	"context"
	"fmt"
	"log/slog"
	"os"

	"github.com/compose-spec/compose-go/v2/loader"
	"github.com/compose-spec/compose-go/v2/types"
	"github.com/compose-spec/compose-go/v2/utils"
	"github.com/docker/cli/cli/command"
	"github.com/docker/cli/cli/flags"
	"github.com/docker/compose/v2/pkg/api"
	"github.com/docker/compose/v2/pkg/compose"
)

const exampleCompose = `
services:
  example:
    image: nginx:latest # Use the latest Nginx image
    container_name: example
    environment:
      - FOO=${SET_ME} # Use the environment variable SET_ME
      - BAR=${AND_ME} # Use the environment variable AND_ME
    ports:
      - "8080:80" # Map port 80 in the container to port 8080 on the host
    post_start:
      - command: /bin/sh -c "echo somecontent > /root/file.txt"
    networks:
      - webnet # Connect the web service to the webnet network
     
networks:
  webnet: # Define a custom network for communication between services
`

func main() {
	err := run()
	if err != nil {
		slog.Error(err.Error())
		os.Exit(1)
	}
}

func run() error {
	// Create a Docker client
	cli, err := command.NewDockerCli()
	if err != nil {
		return err
	}

	// Initialize the Docker client with the default options
	err = cli.Initialize(flags.NewClientOptions())
	if err != nil {
		return err
	}

	// Create the compose API service instance with the Docker cli
	composeService := compose.NewComposeService(cli)

	// Set the environment variables
	envVars := []string{
		"SET_ME=ok",
		"AND_ME=thanks",
	}

	// Convert []string of env vars to map
	envVarMap := utils.GetAsEqualsMap(envVars)

	configDetails := types.ConfigDetails{
		ConfigFiles: []types.ConfigFile{
			{
				Content: []byte(exampleCompose),
			},
		},
		Environment: envVarMap,
	}

	// Create a new project from the compose config details
	project, err := loader.LoadWithContext(context.TODO(), configDetails,
		func(opts *loader.Options) {
			// We are passing in our own environment
			opts.SkipResolveEnvironment = true
			// Set the compose project name
			opts.SetProjectName("example", true)
		})
	if err != nil {
		fmt.Println("Error loading project: ", err)
		return err
	}

	// Optional: Drops networks/volumes/secrets/configs that are not referenced by active services
	project = project.WithoutUnnecessaryResources()

	// Show what is about to be started
	compiledYAML, err := project.MarshalYAML()
	if err != nil {
		return err
	}

	fmt.Println("Compose file to be started:")
	fmt.Println(string(compiledYAML))

	// Optional: configurations
	opts := api.UpOptions{
		Create: api.CreateOptions{
			RemoveOrphans: true,
		},
		Start: api.StartOptions{},
	}

	// Sets the labels required for Docker to recognise that these are Compose services
	for i, s := range project.Services {
		s.CustomLabels = map[string]string{
			api.ProjectLabel: project.Name,
			api.ServiceLabel: s.Name,
			api.VersionLabel: api.ComposeVersion,
			api.OneoffLabel:  "False",
		}

		project.Services[i] = s
	}

	// Run the up command
	err = composeService.Up(context.TODO(), project, opts)
	if err != nil {
		return err
	}

	return nil
}

The container will run fine but "/root/file.txt" does not exist. If I run docker compose up with that yml the hook runs and I see "/root/file.txt" as expected.

[maggie44]

Try adding project to the api.StartOptions:

Start: api.StartOptions{
	Project: project,
},

[chris-gputrader]

Try adding project to the api.StartOptions:

Start: api.StartOptions{
	Project: project,
},

Thanks so much, that solved the issue!

[chris-gputrader]

Sorry to reopen, curious if api.DownOptions would also need to have the Project passed in to execute pre_stop hooks?

[maggie44]

Sorry to reopen, curious if api.DownOptions would also need to have the Project passed in to execute pre_stop hooks?

Did you try it? I would suspect that if projectName is set then it isn't necessary.

...
composeService := compose.NewComposeService(cli)
err = composeService.Down(ctx, projectName, api.DownOptions{RemoveOrphans: true, Volumes: false})

[ndeloof]

Just a quick note: docker/compose is not a library, and despite we have an api package for legacy reasons, using it programmatically is "at your own risks"

[ndeloof]

DownOptions would also need to have the Project passed in to execute pre_stop hooks?

it does. hooks are defined by Project they're not known by the runtime resources - hooks are not a docker engine feature. if you don't pass a Project, no hook will be executed.