Skip to content

Commit

Permalink
Merge pull request #1 from harisankar95/migrate-to-gymnasium
Browse files Browse the repository at this point in the history
Migrate to gymnasium
  • Loading branch information
harisankar95 authored Dec 29, 2023
2 parents 137af30 + 5aac004 commit 72c13a2
Show file tree
Hide file tree
Showing 27 changed files with 1,040 additions and 401 deletions.
12 changes: 2 additions & 10 deletions .github/workflows/publish-pypi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4
with:
Expand All @@ -20,19 +20,11 @@ jobs:
python -m pip install --upgrade pip
pip install setuptools wheel twine
- name: Build and publish to Test PyPI
env:
TWINE_USERNAME: ${{ secrets.TEST_PYPI_USERNAME }}
TWINE_PASSWORD: ${{ secrets.TEST_PYPI_PASSWORD }}
run: |
python setup.py sdist bdist_wheel
twine check dist/*
twine upload --repository testpypi dist/*
- name: Build and publish to PyPI
env:
TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
run: |
python setup.py sdist bdist_wheel
twine check dist/*
twine upload dist/*
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: CI
name: Test

on:
push:
Expand All @@ -17,7 +17,7 @@ jobs:

steps:
- name: Checkout code
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
Expand All @@ -26,7 +26,8 @@ jobs:

- name: Install dependencies
run: |
pip install setuptools==65.5.0
python -m pip install --upgrade pip
pip install setuptools wheel twine
pip install -e .[dev]
- name: Pylint
Expand All @@ -43,7 +44,7 @@ jobs:

- name: Pytest
run: |
coverage run -m pytest tests/
coverage run --source voxelgym2D -m pytest tests/
coverage report
- name: Build docs
Expand Down
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
MIT License

Copyright (c) 2023 Hari
Copyright (c) 2023 Harisankar Babu

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
Expand Down
55 changes: 38 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,37 +1,44 @@
# voxelgym2D

[![MIT License](https://img.shields.io/github/license/harisankar95/voxelgym2D)](LICENSE)
[![PyPI](https://img.shields.io/pypi/v/voxelgym2D)](https://pypi.org/project/voxelgym2D/)
[![Pipeline](https://github.com/harisankar95/voxelgym2D/actions/workflows/test-main.yml/badge.svg?branch=main)](https://github.com/harisankar95/voxelgym2D/actions/workflows/test-main.yml)
[![codestyle](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)

A gym environment for voxel/grid based reinforcement learning for path planning.

<div id="solution-table">
<table>
<tr>
<td style="padding:10px">
<img src="https://github.com/harisankar95/voxelgym2D/raw/main/resources/solution_1.gif" width="375"/>
</td>
<tr>
<td style="padding:10px">
<img src="https://github.com/harisankar95/voxelgym2D/raw/main/resources/solution_1.gif" width="375"/>
</td>
<td style="padding:10px">
<img src="https://github.com/harisankar95/voxelgym2D/raw/main/resources/solution_2.gif" width="375"/>
<img src="https://github.com/harisankar95/voxelgym2D/raw/main/resources/solution_2.gif" width="375"/>
</td>
</tr>
</table>
</div>

## Results with [SB3](https://github.com/DLR-RM/stable-baselines3) (v1.6.2) : PPO :smile:

Here are the results of training a PPO agent on the `onestep-v0` using the example [here](examples/train_ppo.py). Below you will find the episode reward and episode length over steps during training. As the agent learns, the episode reward increases and the episode length reduces are the agent learns to identify the goal and reach it in the shortest possible path.

<div id="solution-table">
<table>
<tr>
<td style="padding:10px">
<img src="https://github.com/harisankar95/voxelgym2D/raw/main/resources/ep_reward.png" width="375"/>
</td>
<tr>
<td style="padding:10px">
<img src="https://github.com/harisankar95/voxelgym2D/raw/main/resources/ep_reward.png" width="375"/>
</td>
<td style="padding:10px">
<img src="https://github.com/harisankar95/voxelgym2D/raw/main/resources/ep_length.png" width="375"/>
<img src="https://github.com/harisankar95/voxelgym2D/raw/main/resources/ep_length.png" width="375"/>
</td>
</tr>
</table>
</div>


## Installation

```bash
pip install git+https://github.com/harisankar95/voxelgym2D.git
```
Expand All @@ -43,7 +50,9 @@ pip install voxelgym2D
```

## Development

To install the package in development mode, run the following command in the root directory of the repository:

```bash
git clone https://github.com/harisankar95/voxelgym2D.git ~/path/to/repo
cd ~/path/to/repo
Expand All @@ -54,41 +63,53 @@ pip install -e .[dev,sb3]
```

## Usage

```python
import voxelgym2D
import gym
import gymnasium as gym

env = gym.make("voxelgym2D:onestep-v0")
env.reset()
observation, info = env.reset(seed=123456)
env.render()
```

## Examples

The examples can be found [here](examples).

## License

This project is licensed under the terms of the [MIT license](LICENSE).

## Documentation

The documentation can be found [here](https://harisankar95.github.io/voxelgym2D/).

## Changelog

### 0.1.0

- Initial release of voxelgym 2D environments tested with stable_baselines 3 (v1.6.2) and python 3.8

### 0.2.0

- Available on PyPI, sphinx documentation in work

### 0.3.0

- Migration to gymnasium
- Agent can now be reset to multiple start positions

## TODO

- [x] Add 2D environments
- [ ] Test with gym 0.26.2
- [ ] Add documentation

## Known issues
- [ ] Currently only supports gym==0.21.0 :neutral_face:, hence setuptools==65.5.0 is required to install gym.

## Contributing

Contributions are welcome! Please open an issue or a pull request.

## References

- [OpenAI Gym](https://arxiv.org/abs/1606.01540)
- [Stable Baselines 3](http://jmlr.org/papers/v22/20-1364.html)
28 changes: 28 additions & 0 deletions docs/INSTALL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Installation

## PyPI

Install the package from PyPI using pip:

```bash
pip install voxelgym2D
```

## GitHub

```bash
pip install git+https://github.com/harisankar95/voxelgym2D.git
```

## For development purpose use editable mode

To install the package in development mode, run the following command in the root directory of the repository:

```bash
git clone https://github.com/harisankar95/voxelgym2D.git ~/path/to/repo
cd ~/path/to/repo
pip install -e .[dev]

# to aditionally install stable_baselines 3 and pytorch (optional)
pip install -e .[dev,sb3]
```
16 changes: 16 additions & 0 deletions docs/INTRO.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Voxelgym2D

A gym environment for voxel/grid based reinforcement learning for path planning.

<div id="solution-table">
<table>
<tr>
<td style="padding:10px">
<img src="https://github.com/harisankar95/voxelgym2D/raw/main/resources/solution_1.gif" width="375"/>
</td>
<td style="padding:10px">
<img src="https://github.com/harisankar95/voxelgym2D/raw/main/resources/solution_2.gif" width="375"/>
</td>
</tr>
</table>
</div>
21 changes: 21 additions & 0 deletions docs/USAGE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Examples

For usage examples with detailed descriptions take a look at the [examples](https://github.com/harisankar95/voxelgym2D/tree/main/examples/) folder.

## Basic usage

```python
import gymnasium as gym

env = gym.make("voxelgym2D:onestep-v0")
observation, info = env.reset(seed=123456)

done = False
while not done:
action = env.action_space.sample() # agent policy that uses the observation and info
observation, reward, terminated, truncated, info = env.step(action)

done = terminated or truncated
env.render()

env.close()
77 changes: 77 additions & 0 deletions docs/_static/custom.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
/* Based on Stable Baselines 3 theme
* https://github.com/DLR-RM/stable-baselines3/
* */
:root {
--main-bg-color: #B6C8DB;
--link-color: #6DB59F;
}

/* Header fonts y */
h1,
h2,
.rst-content .toctree-wrapper p.caption,
h3,
h4,
h5,
h6,
legend,
p.caption {
font-family: "Lato", "proxima-nova", "Helvetica Neue", Arial, sans-serif;
}


/* Docs background */
.wy-side-nav-search {
background-color: var(--main-bg-color);
}

/* Mobile version */
.wy-nav-top {
background-color: var(--main-bg-color);
}

/* Change link colors (except for the menu) */
a {
color: var(--link-color);
}

a:hover {
color: #798EA9;
}

.wy-menu a {
color: #b3b3b3;
}

.wy-menu a:hover {
color: #b3b3b3;
}

a.icon.icon-home {
color: #b3b3b3;
}

.version {
color: var(--link-color) !important;
}


/* Make code blocks have a background */
.codeblock,
pre.literal-block,
.rst-content .literal-block,
.rst-content pre.literal-block,
div[class^='highlight'] {
background: #FFFFFF;
;
}

/* Change style of types in the docstrings .rst-content .field-list */
.field-list .xref.py.docutils,
.field-list code.docutils,
.field-list .docutils.literal.notranslate {
border: None;
padding-left: 0;
padding-right: 0;
color: #404040;
}
27 changes: 27 additions & 0 deletions docs/_templates/versions.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
{%- if current_version %}
<div class="rst-versions" data-toggle="rst-versions" role="note" aria-label="versions">
<span class="rst-current-version" data-toggle="rst-current-version">
<span class="fa fa-book"> Other Versions</span>
v: {{ current_version.name }}
<span class="fa fa-caret-down"></span>
</span>
<div class="rst-other-versions">
{%- if versions.tags %}
<dl>
<dt>Tags</dt>
{%- for item in versions.tags %}
<dd><a href="{{ item.url }}">{{ item.name }}</a></dd>
{%- endfor %}
</dl>
{%- endif %}
{%- if versions.branches %}
<dl>
<dt>Branches</dt>
{%- for item in versions.branches %}
<dd><a href="{{ item.url }}">{{ item.name }}</a></dd>
{%- endfor %}
</dl>
{%- endif %}
</div>
</div>
{%- endif %}
Loading

0 comments on commit 72c13a2

Please sign in to comment.