AttributeError: 'str' object has no attribute 'get'

[schorlton-bugseq]

Thanks for your hard work on this tool! Running v8.4.0 via official docker image:

multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
                    ^^^^^^^^^^^^^^^^^^^
  File "/megalinter/MegaLinter.py", line 51, in run_linters
    linter.run(run_commands_before_linters=False, run_commands_after_linters=False)
  File "/megalinter/Linter.py", line 819, in run
    pre_post_factory.run_linter_pre_commands(
  File "/megalinter/pre_post_factory.py", line 48, in run_linter_pre_commands
    return run_commands(
           ^^^^^^^^^^^^^
  File "/megalinter/pre_post_factory.py", line 97, in run_commands
    pre_command_result = run_command(command_info, log_key, mega_linter, linter)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/megalinter/pre_post_factory.py", line 105, in run_command
    if command_info.get("cwd", "root") == "workspace":
       ^^^^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'get'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/megalinter/run.py", line 14, in <module>
    linter.run()
  File "/megalinter/MegaLinter.py", line 252, in run
    self.process_linters_parallel(active_linters, linters_do_fixes)
  File "/megalinter/MegaLinter.py", line 391, in process_linters_parallel
    updated_linters = pool_result.get()
                      ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 774, in get
    raise self._value
AttributeError: 'str' object has no attribute 'get'

Any ideas? v8.1.0 was the last version I was on and the same repo worked. Thanks!

[nvuillam]

Mmm this is in pre-post commands code... @bdovaz any idea ? :)

[nvuillam]

@schorlton-bugseq please can you share your PRE_COMMANDS config ? (Or even your whole .mega-linter.yml file ? )

[bdovaz]

Without @schorlton-bugseq config I would not be able to tell...

I have looked and seems that we have this test:

https://github.com/oxsecurity/megalinter/blob/main/megalinter/tests/test_megalinter/pre_post_test.py

https://github.com/oxsecurity/megalinter/tree/main/.automation/test/pre-post-test

That's why I thought that at least this test (which already existed before) covered the possible regressions for my PR but I see it doesn't?

[dtrinf]

Thanks for your hard work on this tool! Running v8.4.0 via official docker image:

multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
^^^^^^^^^^^^^^^^^^^
File "/megalinter/MegaLinter.py", line 51, in run_linters
linter.run(run_commands_before_linters=False, run_commands_after_linters=False)
File "/megalinter/Linter.py", line 819, in run
pre_post_factory.run_linter_pre_commands(
File "/megalinter/pre_post_factory.py", line 48, in run_linter_pre_commands
return run_commands(
^^^^^^^^^^^^^
File "/megalinter/pre_post_factory.py", line 97, in run_commands
pre_command_result = run_command(command_info, log_key, mega_linter, linter)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/megalinter/pre_post_factory.py", line 105, in run_command
if command_info.get("cwd", "root") == "workspace":
^^^^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'get'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/megalinter/run.py", line 14, in
linter.run()
File "/megalinter/MegaLinter.py", line 252, in run
self.process_linters_parallel(active_linters, linters_do_fixes)
File "/megalinter/MegaLinter.py", line 391, in process_linters_parallel
updated_linters = pool_result.get()
^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 774, in get
raise self._value
AttributeError: 'str' object has no attribute 'get'

Any ideas? v8.1.0 was the last version I was on and the same repo worked. Thanks!

I'm also facing the same problem, the previous working version for me was:

- Image Creation Date: 2024-11-23T10:43:56Z
- Image Revision: 1fc052d03c7a43c78fe0fee19c9d648b749e0c01
- Image Version: v8.3.0

[bdovaz]

@schorlton-bugseq please can you share your PRE_COMMANDS config ? (Or even your whole .mega-linter.yml file ? )

Can any of you who are affected by it, put what @nvuillam asks? Thank you.

[dtrinf]

@schorlton-bugseq please can you share your PRE_COMMANDS config ? (Or even your whole .mega-linter.yml file ? )

Can any of you who are affected by it, put what @nvuillam asks? Thank you.

Sure, here you have my .mega-linter.yml file

# https://megalinter.io/latest/config-file/
# https://megalinter.io/latest/config-activation/
---
ENABLE_LINTERS:
  - TERRAFORM_TERRAFORM_FMT
  - TERRAFORM_TFLINT
# DISABLE_ERRORS_LINTERS:

## Terraform ##
TERRAFORM_TERRAFORM_FMT_DISABLE_ERRORS: false # break build on terraform formatting errors
TERRAFORM_FILTER_REGEX_INCLUDE: (deploy/terraform/continuous)
TERRAFORM_TFLINT_ARGUMENTS: --color

## Generic ##
EXCLUDED_DIRECTORIES: ["dasltools"]
FORMATTERS_DISABLE_ERRORS: false #break build on ALL formatting errors
VALIDATE_ALL_CODEBASE: true
CLEAR_REPORT_FOLDER: true
SHOW_SKIPPED_LINTERS: false
SECURED_ENV_VARIABLES:
  - AWS_ACCESS_KEY_ID
  - AWS_SECRET_ACCESS_KEY
  - AWS_SESSION_TOKEN

[bdovaz]

@nvuillam looking at that file it looks like it has to do when you do NOT have pre/post commands.

[nvuillam]

Damn :o
So work around woukd be to add a dummy pre-command, meanwhile @bdovaz do u have availability to reproduce & fix the bug so i can release a patch tonight or tomorrow ?

[pjungermann]

Maybe, it is (also) related to the bump to Python 3.12? 🤔

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/megalinter/run.py", line 14, in <module>
    linter.run()
  File "/megalinter/MegaLinter.py", line 252, in run
    self.process_linters_parallel(active_linters, linters_do_fixes)
  File "/megalinter/MegaLinter.py", line 391, in process_linters_parallel
    updated_linters = pool_result.get()
                      ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 774, in get
    raise self._value
AttributeError: 'str' object has no attribute 'get'

For me, the error always happens after the execution of the first linter.

However, it does not happen/can be mitigated with

PARALLEL: false

I tried adding pre-/post-commands but that didn't help.

ENABLE_LINTERS:
  - REPOSITORY_CHECKOV
  - REPOSITORY_TRIVY

PRE_COMMANDS:
  - command: echo "Temporary mitigation for https://github.com/oxsecurity/megalinter/issues/4591"
    continue_if_failed: true
    cwd: workspace

POST_COMMANDS:
  - command: echo "Temporary mitigation for https://github.com/oxsecurity/megalinter/issues/4591"
    continue_if_failed: true
    cwd: workspace

REPOSITORY_PRE_COMMANDS:
  - command: echo "Temporary mitigation for https://github.com/oxsecurity/megalinter/issues/4591"
    continue_if_failed: true
    cwd: workspace

REPOSITORY_POST_COMMANDS:
  - command: echo "Temporary mitigation for https://github.com/oxsecurity/megalinter/issues/4591"
    continue_if_failed: true
    cwd: workspace

REPOSITORY_CHECKOV_PRE_COMMANDS:
  - command: echo "Temporary mitigation for https://github.com/oxsecurity/megalinter/issues/4591"
    continue_if_failed: true
    cwd: workspace

REPOSITORY_CHECKOV_POST_COMMANDS:
  - command: echo "Temporary mitigation for https://github.com/oxsecurity/megalinter/issues/4591"
    continue_if_failed: true
    cwd: workspace


REPOSITORY_TRIVY_PRE_COMMANDS:
  - command: echo "Temporary mitigation for https://github.com/oxsecurity/megalinter/issues/4591"
    continue_if_failed: true
    cwd: workspace

REPOSITORY_TRIVY_POST_COMMANDS:
  - command: echo "Temporary mitigation for https://github.com/oxsecurity/megalinter/issues/4591"
    continue_if_failed: true
    cwd: workspace

[fostermarcp]

Also running into this error on

• Image Creation Date: 2025-01-26T16:10:00Z
• Image Revision: f90c800
• Image Version: v8.4.0

# global settings
APPLY_FIXES: none
FLAVOR_SUGGESTIONS: false
SHOW_ELAPSED_TIME: true
ENABLE_LINTERS:
  - TERRAFORM_TFLINT
  - TERRAFORM_TERRASCAN
  - REPOSITORY_GITLEAKS 
  - REPOSITORY_TRIVY
  - YAML_YAMLLINT
  - SQL_SQLFLUFF
PRINT_ALPACA: false
VALIDATE_ALL_CODEBASE: false
PRINT_ALL_FILES: false
FILTER_REGEX_EXCLUDE: '(\.vscode/|\.mega-linter.yml|aks-ds*********|aks-aa*********/|\.terraform/)'
EXCLUDED_DIRECTORIES:
  - aks-ds********* # Exclude
  - aks-aa*********v # Exclude

## logging settings
LOG_LEVEL: INFO 
CLEAR_REPORT_FOLDER: true

## reporting settings
SARIF_REPORTER: true
AZURE_COMMENT_REPORTER: true
AZURE_COMMENT_REPORTER_LINKS_TYPE: build
REPORTERS_MARKDOWN_TYPE: simple
CONFIG_REPORTER: false
EMAIL_REPORTER: false
TAP_REPORTER: false

## linter settings
TERRAFORM_TFLINT_PRE_COMMANDS:
  - command: tflint --init

[pjungermann]

I realized it didn't happen for all flavors we use.

For the failing one (security), we have the following in a default/base config we extend:

REPOSITORY_CHECKOV_PRE_COMMANDS:
  - command: some_command

Removing this pre-command helped as well.

[nvuillam]

@bdovaz I think i found the bug :)

+= can be used to append 2 lists, but for a simple element it probably doesn't work and create some unusable string instead

I replaced by append() , PR on the way