Simple Ship AI is a simplified copy of shipAI (https://github.com/jmpf2018/ShipAI) reinforcement learning module based on the OpenAI Gym and Keras-RL projects. This module presents a simulation environment for navigating a ship in a channel (Restricted Waters). The purpose of the navigation task is to keep the ship in the midline of the channel. With ShipAI you can:
- Design Reinforced Learning (RL) algorithms to control ship maneuver movements.
- Visualize the Ship movement Behavior over a channel.
- Train, test and validate your RL agent.
The main purpose of the navigation task was defined as follows:
Use the rudder and propulsion controls to perform a defined linear navigation path along a channel
The states chosen for the application of RL in the task of the ship were as follows:
Where is the distance from the center of mass of the ship to the guideline;
is the horizontal speed of the ship in its center of mass (in the direction of the guideline;
is the vertical speed of the ship in its center of mass (perpendicular to the guideline);
is the angular velocity of the ship.
The reward function is defined as:
Where:
The action are the input parameters for controlling the ship maneuver movement. The forces that make the ship controllable are the rudder and propulsion forces. They have the vector form of , where
is the dimensionless rudder command and
the dimensionless propulsion command, such that
and
. These parameters have a direct proportional relation with the rudder angle and the propulsion
- Make sure you have all necessary modules installed:
pip install TensorFlowpip install Keraspip install gympip install keras-rlpip install Shapelypip install numpypip install scipy
- Clone or download the repository:
git clone https://github.com/jmpf2018/ShipAIcd ShipAI
- Run DQN or DDPG RL algorithms:
python dqn_keras_rl.pyorpython ddpg_keras_rl.py
4.After that RL will start to run:
5.Finally in the end of the process the vizualization screen you start:
-
You can change the ship dynamics and adapt for your needs in: simulator.py
-
You can change the reward and state design in: ship_env.py
-
You can change the network hyperparameter in ddpg.py or dqn.py
-
You can evaluate the learing process and the states evolution in evaluate_learning.py
To read our paper check this link.
@misc{shipAI,
author = {Jonathas Pereira and Rodrigo Pereira Abou Rejaili},
title = {SimpleShipAI},
year = {2018},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/jmpf2018/ShipAI}},
}