Step 0: Make sure that starter_code is in your PYTHONPATH.
Step 1: Put your environment in the EnvRegistry() in starter_code/env_config.py
Step 2:
python starter_code/run.py --alg-name <alg-name> --env-name <env_name>
where <alg-name> can be ppo, a2c, or vpg
For example, you can run python starter_code/run.py --alg-name ppo --env-name CartPole-v0