Fix CloudFoundry tagger collector

[zippolyte]

What does this PR do?

Fixes container collection in cloud foundry

Motivation

• Do not make the liveness probe fail in case of network issue — Part #2 #7605 changed the interface for a Puller but didn't change the signature in garden_main.go
• Use Env AD to choose which providers to trigger in different components (tagger, hostname, host tags...) #8521 changed the collector detection mechanism that could introduce some flakiness depending on the order the process are started during a cloud foundry deployment. The retry is needed

[ahmed-mez]

Any reason why this didn't work as expected? Should we rather fix detectCloudFoundry() ? (just curious about the bug, we can ship this fix as is in 7.30 and fix detectCloudFoundry in 7.31 ofc)

[zippolyte]

This tests the connection to the socket of the garden process. But if it's not started yet at the time the agent starts, then we skip collection and there's no retry ever. The correct logic is in the cloudfoundry.GetGardenUtil() function just below this, that returns a retriable error.

[ahmed-mez]

Thanks for the explanation - in any case let's backlog something about this to fix detectCloudFoundry in 7.31 🙇

[vboulineau]

Can we expect the socket to exist even if Garden process is not started yet?
It could be easily modified in detectCloudFoundry, otherwise it's useless and should be removed

[zippolyte]

No the socket won't exist if the garden process is not started yet.
What's the use case for the detectCloudFoundry given we already have the Detect method ?

[zippolyte]

Ah ! Then we should be able to check the config option cloud_foundry to check if we're even in a cloud foundry environment

datadog-agent/pkg/config/config.go

Line 590 in 29ad37f

config.BindEnvAndSetDefault("cloud_foundry", false)

[ahmed-mez]

cool thanks! a proper fix would be simple then 💯

[zippolyte]

Should I make it for 7.30 instead of this quick and dirty one ?

[vboulineau]

Ideally speaking we would want to auto set cloud_foundry based on some system/environment/files/whatever that the Agent could auto-detect (that's the purpose of detectCloudFoundry).
If not possible we can reverse it and use cloud_foundry as an indicator to activate the Cloud Foundry features.

I think it'd be better to fix it in 7.30.

[zippolyte]

Well that flag in the config is the detector then. It's automatically set in the configuration when the agent is installed in a cloud foundry environment.
I'll make a PR