include any non-default settings from OF config.ini file in the report

[jpiesing]

Although it is not encouraged, organisations running the tests can edit the configuration files.
One likely reason for this is an implementation that fails tests due to dropped frames - the configuration file can be edited to permit dropped frames at the start, in the middle or at the end.
Service provides may accept this.
When someone submits a test report to a service provider / platform, this should include any non-default values for the configuration.

[louaybassbouss]

@jpiesing in the deploy project we have the option to override the default tests configuration --> https://github.com/cta-wave/dpctf-deploy#mapping-new-content-into-the-container . We can update the text in section to make it more clear. wdyt?

[jpiesing]

Adding @yanj-github.

This relates to the config.ini from the OF. Specifically settings like this.

[TOLERANCES]
start_frame_num_tolerance = 0
end_frame_num_tolerance = 0
mid_frame_num_tolerance = 10
splice_start_frame_num_tolerance = 0
splice_end_frame_num_tolerance = 0

Perhaps the OF can send this file to the test runner which could append it at the end of the report? Alternatively just the differences from the default could be appended - which would be clearer for service providers or platforms reading the report.

[louaybassbouss]

I propose an extension of the JSON WPT report structure to include additional parameters:

1. Date/Time when the test is run and when observation is run
2. OF configuration from config.ini

Something like this:

{
    "datetime_test_run": "<UTC Date Time>",
    "datetime_observation": "<UTC Date Time>",
    "observation_config": {
          "start_frame_num_tolerance": 0,
           ....
    }
}

[yanj-github]

@louaybassbouss One question does these additional parameters one set per test or one set per test session (may contain more then one test) please?

[yanj-github]

@jpiesing OF already embeded the tolereances to the result messge.
eg: "message": " First frame found is 5, expected to start from 1. First frame number tolerance is 0. Mid frame number tolerance is 10. Following frames are missing: 12 Total of missing frames is 5."
Relevent tolerence presented to the related observation. such as start end tolerence for every sample rendered observation and startup delay tolerence noted for start up delay observation.
e.g: "Maximum permitted startup delay is 120ms.The presentation start up delay is xxxms"

I this good enough for you or you still want this to be added in different structure please?

[louaybassbouss]

@louaybassbouss One question does these additional parameters one set per test or one set per test session (may contain more then one test) please?

@yanj-github my idea is to add these parameters to the same JSON file you upload via the Results API. This means for each group of tests in a folder.

[jpiesing]

October 12th meeting
OK as proposed. Eurofins to implement in the OF first.

[yanj-github]

@louaybassbouss
Will TR include the following entries with default value to the result file please?
"datetime_observation": "",
"observation_config": {}

Not sure whether OF will need to add "datetime_observation" and "observation_config" entry if so which level?
Or OF just looking for "datetime_observation" and "observation_config" and change the value please?