make docker operators always use DockerHook for API calls

[Taragolis]

Right now DockerHook only partially uses in DockerOperator and DockerSwarmOperator.

This PR refactor some parts

1. Drop raising errors for most of the missing arguments during constructing hook. In Docker SDK all of them optional.
2. Move out all expensive call from __init__, e.g. obtain connection.
3. docker_conn_id now optional. If user not required login to docker Registry this field might be set to None.
   Previously this functional sits in DockerOperator
4. Cache connection to docker server/host.
5. Move out constructing TLSConfig from DockerOperator to DockerHook

This kind of integration will allow in the future move all complex logic from Operators to Hook and make them reusable.

Also this PR pre-requirements for #26162

[Taragolis]

BTW, may be someone know why docker-provider has min airflow version 2.4?

[eladkal]

BTW, may be someone know why docker-provider has min airflow version 2.4?

Because of ExternalPythonOperator #25780

[Taragolis]

BTW, may be someone know why docker-provider has min airflow version 2.4?

Because of ExternalPythonOperator #25780

I can't find any information why it should affect docker provider but maybe I blind

Also we have this lines, which make sense if we support 2.3+ and look redundant if 2.4+

airflow/airflow/providers/docker/decorators/docker.py

Lines 32 to 37 in 4e545c6

	try:
	from airflow.utils.decorators import remove_task_decorator
	# This can be removed after we move to Airflow 2.4+
	except ImportError:
	from airflow.utils.python_virtualenv import remove_task_decorator

[potiuk]

I can't find any information why it should affect docker provider but maybe I blind

It looks like it was a mistake. We should get it down to 2.3.0 in the next release:

We already have a code added in the same PR that handles pre-2.4 case - https://github.com/apache/airflow/pull/25780/files#diff-1042a900e2f4c7b044453f7229033b8787d35abe66b14d0e9392fee8b1a714ffR30

So >=2.3.0 should be fine.

[potiuk]

I think it was just failing with 2.3 and I first bumped it to 2.4 and then added the if and forgot to move it down to 2.3.0

[potiuk]

Very nice refactor.

However I have one request - could we also handle tls parameter In Hook possible to be passed as string for backwards compatibility? Seems that it should be very easy to implement and not that unreasonable even (Hook will be then a bit more flexible on what cold be passed to tls. And we will avoid having to bump major version of the provider? Seems like all other changes are backwards compatible.

[Taragolis]

However I have one request - could we also handle tls parameter In Hook possible to be passed as string for backwards compatibility?

@potiuk, I will have a look again (today last day of my holidays) however based on docker-py code pass tls parameter as string no make sense at least in docker>=5.0.3 (our requirements). It will interpreters as bool(tls) if it not an instance of TLSConfig

Internally we do not check type of this argument (if not count mypy), so if user pass string and this somehow work it will work.

[potiuk]

tls parameter as string no make sense at least in docker>=5.0.3 (our requirements)

Ah. Then it makes perfect sense - it's really a bugfix not a backwards-compatibility probiem then.

[Taragolis]

Ah. Then it makes perfect sense - it's really a bugfix not a backwards-compatibility probiem then.

maybe somewhere in the past tls_config could be a string (I didn't check) but the main issue that Hook previously do not use in DockerOperator and others and even when it use then tls_config constructed from multiple separate parameters.