Skip to content

Add configuration option for base Python image for Docker files#524

Merged
rbs-jacob merged 2 commits into
masterfrom
docker/config-file-python-image
Nov 22, 2024
Merged

Add configuration option for base Python image for Docker files#524
rbs-jacob merged 2 commits into
masterfrom
docker/config-file-python-image

Conversation

@alchzh
Copy link
Copy Markdown
Contributor

@alchzh alchzh commented Nov 21, 2024

Python 3.8 is now officially EoL. I've been testing how OFRAK does with newer python versions in the maintenance/newer-python-test-fixes branch. This change just lets us build Docker images off images of newer Python versions.

One sentence summary of this PR (This should go in the CHANGELOG!)
Add configuration option for base Python image for Docker files.

Link to Related Issue(s)

Please describe the changes in your request.

  • Adds a python_image configuration option to docker yaml files. Defaults to the current 3.8-bullseye image for now.
  • Changes the pip and jefferson install to not depend on specific version

Anyone you think should look at this, specifically?

Comment thread ofrak_core/Dockerstub Outdated
@alchzh
Copy link
Copy Markdown
Contributor Author

alchzh commented Nov 22, 2024

I ran into a problem where the new setuptools editable installs are broken with our directory layout. Directories in the working directory are now imported before editable installed packages, so running e.g. import ofrak_type from a folder with the parent ofrak_type directory is broken. See pypa/setuptools#3557 and https://setuptools.pypa.io/en/latest/userguide/development_mode.html for details.

$ docker run -it redballoonsecurity/ofrak/dev-3.13 bash
Using service wrapper: yajsw-beta-13.01
Installing Ghidra Server...
Traceback (most recent call last):
  File "<frozen runpy>", line 189, in _run_module_as_main
  File "<frozen runpy>", line 148, in _get_module_details
  File "<frozen runpy>", line 112, in _get_module_details
  File "/ofrak_core/ofrak/__init__.py", line 8, in <module>
    from ofrak.ofrak_context import OFRAK, OFRAKContext
  File "/ofrak_core/ofrak/ofrak_context.py", line 13, in <module>
    from ofrak_type import InvalidStateError
ImportError: cannot import name 'InvalidStateError' from 'ofrak_type' (unknown location)

I see a couple solutions to this problem, not sure which one is the best

  • add --config-settings editable_mode=compat to any pip install -e command
  • Require older setuptools in pyproject.toml
[build-system]
requires = ["setuptools == 63"]
build-backend = "setuptools.build_meta"

@rbs-jacob
Copy link
Copy Markdown
Member

I ran into a problem where the new setuptools editable installs are broken with our directory layout. Directories in the working directory are now imported before editable installed packages, so running e.g. import ofrak_type from a folder with the parent ofrak_type directory is broken.

Is this only happening because the default working directory in the Docker images we build is the root directory? Would changing the WORKDIR to /root/ as the last step in the Dockerfile assembled from Dockerstub files be an alternative way to address this?

@rbs-jacob rbs-jacob merged commit e601a5e into master Nov 22, 2024
@whyitfor whyitfor deleted the docker/config-file-python-image branch November 25, 2024 10:24
@ANogin
Copy link
Copy Markdown
Contributor

ANogin commented Dec 4, 2024

I ran into a problem where the new setuptools editable installs are broken with our directory layout. Directories in the working directory are now imported before editable installed packages, so running e.g. import ofrak_type from a folder with the parent ofrak_type directory is broken. See pypa/setuptools#3557 and https://setuptools.pypa.io/en/latest/userguide/development_mode.html for details.

$ docker run -it redballoonsecurity/ofrak/dev-3.13 bash
Using service wrapper: yajsw-beta-13.01
Installing Ghidra Server...
Traceback (most recent call last):
  File "<frozen runpy>", line 189, in _run_module_as_main
  File "<frozen runpy>", line 148, in _get_module_details
  File "<frozen runpy>", line 112, in _get_module_details
  File "/ofrak_core/ofrak/__init__.py", line 8, in <module>
    from ofrak.ofrak_context import OFRAK, OFRAKContext
  File "/ofrak_core/ofrak/ofrak_context.py", line 13, in <module>
    from ofrak_type import InvalidStateError
ImportError: cannot import name 'InvalidStateError' from 'ofrak_type' (unknown location)

I see a couple solutions to this problem, not sure which one is the best

  • add --config-settings editable_mode=compat to any pip install -e command
  • Require older setuptools in pyproject.toml
[build-system]
requires = ["setuptools == 63"]
build-backend = "setuptools.build_meta"

@alchzh There is also python/mypy#13392; I ended up using export SETUPTOOLS_ENABLE_FEATURES="legacy-editable" in [stale] https://github.com/redballoonsecurity/ofrak/pull/314/files#diff-9d7ea1b163b7b4693bbc13b0dd627154521dc56ad997cd3b1a7747e413916027

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants