Improvements to removeCheckoutDir

[DrJosh9000]

Description

Ensure that, when removing all files and directories for a cleanup, that directories with weird permissions (e.g. lacking execute bit) are cleaned up.

Context

#2941

Changes

• Add hardRemoveAll helper function that tests for permissions problems after failing to os.RemoveAll, in a loop
• Replace some calls to os.RemoveAll with a call to hardRemoveAll

Testing

• Tests have run locally (with go test ./...). Buildkite employees may check this if the pipeline has run automatically.
• Code is formatted (with go fmt ./...)