AuTest: Traffic Dump Schema Verification Fails with jsonschema 4.0.0

[bneradt]

I notice with the 4.x versions of jsonschema, the traffic_dump AuTests fail because the jsonschema verification fails.

$ autest -D gold_tests/ --sandbox /tmp/sb --clean=none --ats-bin ~/build/ts_asf_master_fix_TSSslSecretSet_for_stored_ssl_contexts/bin/ -f traffic_dump
Running Test traffic_dump:.F Failed                                                                            
                                                                                                                                                     
Generating Report: --------------                                                                              
 Test: traffic_dump: Failed                                                                                                   
    File: traffic_dump.test.py                                                                                 
    Directory: /home/bneradt/src/ts_asf_master_fix_TSSslSecretSet_for_stored_ssl_contexts/tests/gold_tests/pluginTest/traffic_dump
...
   Run: Verify the json content of the first session: Failed                                                                                                                                                                                                                                                      
     Setting up : Copying 'verify_replay.py' to directory '/tmp/sb/traffic_dump' as 'verify_replay.py'' - Passed                                                                                                                                                                                                  
     Test : Checking that True == _isRunningAfter - Passed                                                                                                                                                                                                                                                        
        Reason: Returned Value: True == _isRunningAfter                                                                                                                                                                                                                                                           
     Starting TestRun 1-tr : No Issues found - Passed                                                                                                                                                                                                                                                             
        Reason: Started!                                                                                                                                                                                                                                                                                          
     Process: Default: Failed                                                                                                                                                                                                                                                                                     
       Test : Checking that ReturnCode == 0 - Failed                                                                                                                                                                                                                                                              
          Reason: Returned Value 1 != 0

If I pin the jsonschema version to the latest 3.x release (3.2.0), the test passes.

For those seeking a workaround, it can be pinned like so:

$ cd tests

$ pipenv run pip install --no-input jsonschema==3.2.0
Collecting jsonschema==3.2.0
  Using cached jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
Requirement already satisfied: six>=1.11.0 in /home/bneradt/.local/share/virtualenvs/tests-RWaVYdAG/lib/python3.8/site-packages (from jsonschema==3.2.0) (1.16.0)
Requirement already satisfied: attrs>=17.4.0 in /home/bneradt/.local/share/virtualenvs/tests-RWaVYdAG/lib/python3.8/site-packages (from jsonschema==3.2.0) (21.2.0)
Requirement already satisfied: pyrsistent>=0.14.0 in /home/bneradt/.local/share/virtualenvs/tests-RWaVYdAG/lib64/python3.8/site-packages (from jsonschema==3.2.0) (0.18.0)
Requirement already satisfied: setuptools in /home/bneradt/.local/share/virtualenvs/tests-RWaVYdAG/lib/python3.8/site-packages (from jsonschema==3.2.0) (57.0.0)
Installing collected packages: jsonschema
  Attempting uninstall: jsonschema
    Found existing installation: jsonschema 4.0.1
    Uninstalling jsonschema-4.0.1:
      Successfully uninstalled jsonschema-4.0.1
Successfully installed jsonschema-3.2.0

$ pipenv run pip freeze | grep jsonschema
jsonschema==3.2.0

[bneradt]

The actual jsonschema failure can be seen in the output of 1-tr-Default, the test run which runs the verification:

$ bash _output/1-tr-Default/replay.sh 
_output/1-tr-Default/replay.sh: line 24: __git_ps1: command not found
Traceback (most recent call last):
  File "verify_replay.py", line 446, in <module>
    sys.exit(main())
  File "verify_replay.py", line 394, in main
    if not validate_json(schema_json, replay_json):
  File "verify_replay.py", line 53, in validate_json
    jsonschema.validate(instance=replay_json, schema=schema_json)
  File "/home/bneradt/.local/share/virtualenvs/tests-RWaVYdAG/lib/python3.8/site-packages/jsonschema/validators.py", line 963, in validate
    cls.check_schema(schema)
  File "/home/bneradt/.local/share/virtualenvs/tests-RWaVYdAG/lib/python3.8/site-packages/jsonschema/validators.py", line 183, in check_schema
    raise exceptions.SchemaError.create_from(error)
jsonschema.exceptions.SchemaError: [{'description': 'Name of the field.', 'type': 'string'}, {'description': 'Value of the field.', 'type': 'string'}] is not of type 'object', 'boolean'

Failed validating 'type' in metaschema['properties']['definitions']['additionalProperties']['$dynamicRef']['allOf'][1]['properties']['properties']['additionalProperties']['$dynamicRef']['allOf'][1]['properties']['items']['$dynamicRef']['allOf'][1]['properties']['items']['$dynamicRef']['allOf'][0]:
    {'$defs': {'anchorString': {'pattern': '^[A-Za-z_][-A-Za-z0-9._]*$',
                                'type': 'string'},
               'uriReferenceString': {'format': 'uri-reference',
                                      'type': 'string'},
               'uriString': {'format': 'uri', 'type': 'string'}},
     '$dynamicAnchor': 'meta',
     '$id': 'https://json-schema.org/draft/2020-12/meta/core',
     '$schema': 'https://json-schema.org/draft/2020-12/schema',
     '$vocabulary': {'https://json-schema.org/draft/2020-12/vocab/core': True},
     'properties': {'$anchor': {'$ref': '#/$defs/anchorString'},
                    '$comment': {'type': 'string'},
                    '$defs': {'additionalProperties': {'$dynamicRef': '#meta'},
                              'type': 'object'},
                    '$dynamicAnchor': {'$ref': '#/$defs/anchorString'},
                    '$dynamicRef': {'$ref': '#/$defs/uriReferenceString'},
                    '$id': {'$comment': 'Non-empty fragments not allowed.',
                            '$ref': '#/$defs/uriReferenceString',
                            'pattern': '^[^#]*#?$'},
                    '$ref': {'$ref': '#/$defs/uriReferenceString'},
                    '$schema': {'$ref': '#/$defs/uriString'},
                    '$vocabulary': {'additionalProperties': {'type': 'boolean'},
                                    'propertyNames': {'$ref': '#/$defs/uriString'},
                                    'type': 'object'}},
     'title': 'Core vocabulary meta-schema',
     'type': ['object', 'boolean']}

On schema['definitions']['header-fields']['properties']['fields']['items']['items']:
    [{'description': 'Name of the field.', 'type': 'string'},
     {'description': 'Value of the field.', 'type': 'string'}]

[bneradt]

Fixed by #8370.