Freeway environment doesn't end episode on collision — is this intended?

[asyncs]

I was watching the video of the Freeway environment on the website and noticed that the episode doesn’t end when the agent collides with cars.

Is this the intended behavior? It feels a bit off, since collisions usually mark failure in similar environments. Might be worth changing, as it could give the wrong impression about how robust the agent actually is.

Just wanted to flag it in case it's unintentional.

Thanks!

[Hadrien-Cr]

Hi, i wrote freeway. In the original atari game, there are 2 modes of collision. easy: the car bumps. hard; the player respawns after collision (but doesnt reset the env). so i implemented both of these (difficulty {0,1})

I agree that the trained policy is colliding a lot and it seems a bit off. Tweaking the reward or the size of the sprites would help: the player in this freeway is twice bigger

[Anri-Lombard]

I looked into this and the current behavior seems intentional for RL training purposes. When the player collides with a car, they get stunned for 40 ticks and optionally receive a -0.01 penalty (when using dense rewards). On difficulty 1, the player also resets to the starting position. The episode only ends when the 136-second timer expires.

This is different from environments like Asteroids where collision immediately terminates the episode. The stun-based approach likely provides more consistent episode lengths and allows the agent to learn recovery behavior rather than just binary success/failure.

Is this the intended design? If there's interest in changing the behavior, I could add an optional terminate_on_collision flag to give users a choice.