Inventory stored as a python script printing it, parsed with a yaml parser

[JulienPalard]

ISSUE TYPE

• Bug Report

SUMMARY

Erreur parsing scm-project-imported yml inventory.

ENVIRONMENT

• AWX version: 3.0.1.0
• AWX install method: docker on linux
• Ansible version: 2.7.7
• Operating System: Debian 9.8
• Web Browser: Firefox 65.0

STEPS TO REPRODUCE

• Create a git repo with a yml inventory
• Create a project from a git repo
• Create an inventory
• Source the inventory from the project
• Create a job template using this inventory

ACTUAL RESULTS

Enter passphrase for /tmp/awx_20_1hyd0bnn/credential_4: 
Identity added: /tmp/awx_20_1hyd0bnn/credential_4 (awx@awx1)
SSH password: 
 [WARNING]:  * Failed to parse /tmp/awx_20_1hyd0bnn/tmpekssw087 with yaml
plugin: Syntax Error while loading YAML.   did not find expected key  The error
appears to have been in '/tmp/awx_20_1hyd0bnn/tmpekssw087': line 3, column 89,
but may be elsewhere in the file depending on the exact syntax problem.  The
offending line appears to be:  # -*- coding: utf-8 -*- print('{"all": {"vars":
{"ansible_python_interpreter": "/usr/bin/python3"}, "hosts": []}, " [... my whole inventory here, redacted for readability and privacy...]"}}}}')
^ here We could be wrong, but this one looks like it might be an issue with
unbalanced quotes.  If starting a value with a quote, make sure the line ends
with the same set of quotes.  For instance this arbitrary example:      foo:
"bad" "wolf"  Could be written as:      foo: '"bad" "wolf"'

 [WARNING]: Unable to parse /tmp/awx_20_1hyd0bnn/tmpekssw087 as an inventory
source

ERROR! No inventory was parsed, please check your configuration and options.

ADDITIONAL INFORMATION

The problem looks like the inventory was stored as a python script printing the inventory, and is parsed using a yaml parser.

[AlanCoding]

Is your YAML inventory in the same directory as your playbook?

Inside this directory, is there an ansible.cfg?

Within your custom config, did you set that the enabled inventory plugins to be only yaml?

[JulienPalard]

Hi @AlanCoding,

Is your YAML inventory in the same directory as your playbook?

No my inventories are "hidden" in a directory.

Inside this directory, is there an ansible.cfg?

At the root of my hierarchy, not near the inventories.

Within your custom config, did you set that the enabled inventory plugins to be only yaml?

I think so but I'm not sure, relevant parts of my inventory:

[defaults]
inventory      = inventories/dev/inventory.yml
[inventory]
enable_plugins = yaml

And just in case I was not clear in my previous answers, here is my hiearchy (simplified for readability, bot not that much so you can still understand it):

$ tree
.
├── ansible.cfg
├── files
│   ├── android.keystore
│   └── ssh_known_hosts
├── group_vars
│   ├── all
│   │   └── vars  # A few encrypted variables 
│   ├── devs.yml  # whole-file encrypted
│   ├── gitlabs.yml  # whole-file encrypted
├── inventories
│   ├── dev
│   │   └── inventory.yml  # Moved here due to https://github.com/ansible/awx/issues/223
│   └── prod
│       └── inventory.yml  # Same
├── library
│   └── venv.py
├── Makefile
├── playbook-backups.yml
├── playbook-devs.yml  # and a few others ^^
├── roles
│   ├── adminer
│   │   ├── handlers
│   │   │   └── main.yml
│   │   ├── tasks
│   │   │   ├── handlers
│   │   │   └── main.yml
│   │   └── templates
│   │       └── adminer.conf.j2
# And a few other roles

[AlanCoding]

IIRC, the ansible.cfg will be read from any parent directory. Even if that's not the case, IIRC, the root of the project is the current working directory for running jobs.

Anyway, this appears to be exactly what I thought.

Your expected behavior is directly at odds with what was being addressed in:

#2313

Specifying ANSIBLE_INVENTORY_ENABLED='script' ends up causing issues for playbooks that themselves invoke -m shell -a 'ansible-playbook ...'

You, on the other hand, are expecting to set an inventory plugin for the inventory import, and (reasonably) run a playbook from the same repo. Doing so, you apply the same config for the inventory import as the playbook run. This is perfectly fine for the inventory import, and fine when you run from the CLI, but we change the type of inventory being used when we do a job run, because the AWX server vendors an internal form of inventory, sourced from the database, which is the script type.

Ping @ryanpetrello @jakemcdermott who were involved in the playbook-invoking-playbook expectation.

There is no theoretical way to solve this problem. I strongly considered if we could use ANSIBLE_CONFIG to point to a config used just for that playbook run, and we cannot, because it results in the same problem that the environment, and thus the config, persists to any subprocess.

To ever resolve this, core would need to have some ephemeral way to specify config options, such that it would only apply to that playbook run, and not other playbook runs invoked within the same environment.

Even the "auto" plugin can't solve this problem for us, because in cases like yours it's not even enabled.

[JulienPalard]

I would appreciate if the long term solution for this issue goes towards "unifying" CLI and AWX behavior. My current user experience of AWX (single day test) is I have to change things that work in CLI so they work in AWX (like "moving my inventory in a sub directory so I can use vaults in group_vars"), which I think is not optimal user experience.

[AlanCoding]

I should have linked #2713

I would appreciate if the long term solution for this issue goes towards "unifying" CLI and AWX behavior.

I understand entirely.

There is not any such proposal that would work for this at the moment. We have a couple of changes to inventory coming that should help, and the whole-file encrypted value issue is now solvable, after we've iterated with Ansible core's implementation.

It would be very cool to change out SCM inventory update-on-launch behavior to (instead of performing an inventory update) take place within the inventory loader phase of the playbook. Some people (such as @abadger) have thrown around the idea an inventory loading callback which we could hook into. I believe that particular approach is non-viable because in AWX we hook into the messaging system, and the inventory data is much too large to send through that system, in addition to lacking a callback. I believe we need a more local form of inter-process communication to have the task update inventory in-place. If we did that, the current mechanism of providing multiple inventories in ansible-playbook calls would be plenty sufficient. The current system based around ansible-inventory is rather fundamentally at odds with how many projects are structured.

[JulienPalard]

In the short term, does moving back to an "ini-like" inventory would make my current setup work?

Maybe also document that AWX is not supporting yaml inventories for the moment to avoid some other people being hit by this one?

[AlanCoding]

For the matter of resolving your specific case, I don't see why you couldn't do:

[defaults]
inventory      = inventories/dev/inventory.yml
[inventory]
enable_plugins = yaml,script

In your config file. It is astronomically unlikely that the script plugin would have false positives, mistaking yaml-like inventory for script-like inventory. This should make your case work without any regressions. You could even remove the [inventory] entry, probably with no ill effects.

This does reveal a detail of the AWX internal implementation, that's the general issue here.

[invizus]

I have slightly similar issue caused by below config in ansible.cfg:

[inventory]
enable_plugins = ini, yaml

Removed this, then AWX is able to run templated jobs.
But my inventories are in ini format.

[shanemcd]

Seems like there are some workarounds identified above. Going to close this but please open a new issue or reach out on the mailing list if you see this again.