Getting constant retry failures during value table generation execution takes over 1 hour and still fails

[rahilmehta-dev]

I ran AutoRestTest on the Podman API and I'm getting constant retry failures during the value table generation phase. The execution ran for over 1 hour and then stopped. I'm not sure what's causing this or what I should do to fix it.

What I'm Seeing

During execution, almost every operation shows retry failures:

Building value table generation for operation: VolumeCreateLibpod
Attempting retry due to failed response...
Attempting retry due to failed response...
Attempting retry due to failed response...
Completed value table generation for operation: VolumeCreateLibpod

This pattern repeats for nearly all operations. The process continues for over 1 hour before stopping.

My Configuration

I only changed two settings in configurations.toml:

1. Spec file location: Changed to point to Podman API spec
2. Time duration: Set to 60 seconds

Reproduction

I've tried running this twice with the same configuration and got the same result both times.

Environment:

• AutoRestTest version: [latest from main branch]
• API: Podman
• Configuration: Default settings except spec location and time_duration

Why am I getting all these "failed response" retries during the value table generation phase? Is this related to the OpenAI API or the Podman API itself? Additionally, why does the table generation phase execution take over 1 hour - is this normal? The process stopped without any output or completion message. What should I do to resolve this issue?

For convenience, I’ve attached link which has log and OpenAPI specification

[tylerstennett]

Hi Rahil,

I understand the frustration. The Attempting retry due to failed response... message you're seeing is actually expected behavior — it's part of the value table generation phase, where AutoRestTest uses server responses to enhance its value generation and create valid query parameters and request bodies. This isn't an error with the specification or the Podman API, but I agree the message can be misleading, so I've removed it.

Regarding execution time: the time_duration option in the [request_generation] section of configurations.toml only applies to the reinforcement learning portion of AutoRestTest, not the preprocessing phase where value table generation occurs. The length you're experiencing is simply a result of the number of operations for the Podman API combined with HTTP latency during requests.

To help with this, I've added the ability to parallelize value table construction in commit 0b634cf (accessible in the main branch). See the README.md and the updated configurations file for instructions on enabling parallelization and specifying the number of workers.

Since table generation can take a while, I'd also encourage enabling cached value tables for future runs.

I'm closing this issue as resolved — feel free to reopen if you run into any issues with the changes!