v0.11.1

What's Changed

• fix the weight of CatCMA by @ha-mano in #184
• Bump the version up to v0.11.1 by @c-bata in #185

Full Changelog: v0.11.0...v0.11.1

PyPI: https://pypi.org/project/cmaes/0.11.1/

v0.11.0

Highlights

CatCMA [Hamano+, GECCO2024]

arXiv: https://arxiv.org/pdf/2405.09962

CatCMA is a method for mixed-category optimization problems, which is the problem of simultaneously optimizing continuous and categorical variables. CatCMA employs the joint probability distribution of multivariate Gaussian and categorical distributions as the search distribution.

Usage is like below:

import numpy as np
from cmaes import CatCMA


def sphere_com(x, c):
    dim_co = len(x)
    dim_ca = len(c)
    if dim_co < 2:
        raise ValueError("dimension must be greater one")
    sphere = sum(x * x)
    com = dim_ca - sum(c[:, 0])
    return sphere + com


def rosenbrock_clo(x, c):
    dim_co = len(x)
    dim_ca = len(c)
    if dim_co < 2:
        raise ValueError("dimension must be greater one")
    rosenbrock = sum(100 * (x[:-1] ** 2 - x[1:]) ** 2 + (x[:-1] - 1) ** 2)
    clo = dim_ca - (c[:, 0].argmin() + c[:, 0].prod() * dim_ca)
    return rosenbrock + clo


def mc_proximity(x, c, cat_num):
    dim_co = len(x)
    dim_ca = len(c)
    if dim_co < 2:
        raise ValueError("dimension must be greater one")
    if dim_co != dim_ca:
        raise ValueError(
            "number of dimensions of continuous and categorical variables "
            "must be equal in mc_proximity"
        )

    c_index = np.argmax(c, axis=1) / cat_num
    return sum((x - c_index) ** 2) + sum(c_index)


if __name__ == "__main__":
    cont_dim = 5
    cat_dim = 5
    cat_num = np.array([3, 4, 5, 5, 5])
    # cat_num = 3 * np.ones(cat_dim, dtype=np.int64)
    optimizer = CatCMA(mean=3.0 * np.ones(cont_dim), sigma=1.0, cat_num=cat_num)

    for generation in range(200):
        solutions = []
        for _ in range(optimizer.population_size):
            x, c = optimizer.ask()
            value = mc_proximity(x, c, cat_num)
            if generation % 10 == 0:
                print(f"#{generation} {value}")
            solutions.append(((x, c), value))
        optimizer.tell(solutions)

        if optimizer.should_stop():
            break

What's Changed

• Add support for Python 3.12 by @c-bata in #153
• Remove setup.py and use build module by @c-bata in #154
• Fix CI failures by @c-bata in #158
• get mean by @nomuramasahir0 in #159
• add question template by @nomuramasahir0 in #162
• fix sigma setting by @nomuramasahir0 in #160
• Add GitHub action setting for continuous benchmark by @c-bata in #168
• fix typo by @nomuramasahir0 in #169
• fix BIPOP-CMA in visualization by @nomuramasahir0 in #170
• update readme by @nomuramasahir0 in #171
• update by @nomuramasahir0 in #172
• fix the old_sigma assertion when lr_adapt=True by @Kreyparion in #174
• remove kurobako dependency by @nomuramasahir0 in #175
• Support for numpy v2.0 by @porink0424 in #177
• catcma (GECCO2024) by @ha-mano in #178
• fix CatCMA by @ha-mano in #179
• Update README.md by @ha-mano in #181
• Bump the version up to v0.11.0 by @c-bata in #183

New Contributors

• @Kreyparion made their first contribution in #174
• @porink0424 made their first contribution in #177
• @ha-mano made their first contribution in #178

Full Changelog: v0.10.0...v0.11.0

v0.10.0

What's Changed

• add DX-NES-IC by @nomuramasahir0 in #149
• xNES implementation by @nomuramasahir0 in #150
• add LRA-CMA-ES by @nomuramasahir0 in #151
• Bump the version up to v0.10.0 by @c-bata in #152

Full Changelog: v0.9.1...v0.10.0

• PyPI: https://github.com/CyberAgentAILab/cmaes/blob/main/.github/workflows/pypi-publish.yml

v0.9.1

What's Changed

• Remove tox.ini by @c-bata in #131
• Fix a broken link to Optuna's documentation by @c-bata in #132
• Drop Python 3.6 support. by @c-bata in #130
• Reuse CMA inside CMAwM by @knshnb in #133
• Add rng related methods by @knshnb in #135
• Fix correction of out-of-range continuous params of CMAwM by @knshnb in #134
• Fix correction of out-of-range discrete params of CMAwM by @knshnb in #136
• Avoid to use typing.List, typing.Dict, and typing.Tuple. by @c-bata in #139
• Check feasibility of sampled discrete parameters in CMAwM by @knshnb in #140
• Refactor CMAwM by @knshnb in #141
• Add a test case for no discrete spaces. by @c-bata in #143
• Allow no discrete spaces in CMAwM by @knshnb in #142
• Remove warnings in CMAwM class by @c-bata in #144
• Revert handling of infeasible discrete parameters by @knshnb in #145
• Bump the version up to v0.9.1 by @c-bata in #138

Full Changelog: v0.9.0...v0.9.1

v0.9.0

Highlights

CMA-ES with Margin is now available. It introduces a lower bound on the marginal probability associated with each discrete dimension so that samples can avoid being fixed to a single point. It can be applied to mixed spaces of continuous (float) and discrete (including integer and binary). This algorithm is proposed by Hamano, Saito, @nomuramasahir0 (a maintainer of this library), and Shirakawa, has been nominated as best paper at GECCO'22 ENUM track.

R. Hamano, S. Saito, M. Nomura, S. Shirakawa, CMA-ES with Margin: Lower-Bounding Marginal Probability for Mixed-Integer Black-Box Optimization, GECCO, 2022.

CMA-ES	CMA-ESwM

The above figures are taken from EvoConJP/CMA-ES_with_Margin.

Please check out the following examples for the usage.

• cmaes_with_margin_binary.py
• cmaes_with_margin_integer.py

What's Changed

• Running benchmark of Warm Starting CMA-ES on GitHub Actions. by @c-bata in #99
• Validate bounds domain contains mean by @c-bata in #100
• Fix overflow errors uncovered by Coverage-guided Fuzzing. by @c-bata in #104
• Fuzzing for sep-CMA-ES by @c-bata in #105
• Set license_file on setup.cfg by @c-bata in #106
• fix sep-CMA description by @nomuramasahir0 in #107
• Temporarily disable a GitHub action for kurobako benchmarks by @c-bata in #113
• Fix mutable by @nomuramasahir0 in #112
• Run tests with Python 3.10 by @c-bata in #109
• Update author and maintainer package info. by @c-bata in #116
• Introduce some related projects on README by @c-bata in #118
• Migrate the project metadata to pyproject.toml by @c-bata in #119
• Revert #119 to support Python 3.6. by @c-bata in #122
• Support CMA-ES with Margin. by @knshnb in #121
• Add integer examples for CMA-ES with Margin by @nomuramasahir0 in #125
• Support Python 3.11 by @c-bata in #123
• Add README of CMA-ES with margin by @knshnb in #124
• Follow-up #126: Remove Scipy dependency by @c-bata in #127
• Remove SciPy dependency by @amylase in #126
• Use gh instead of ghr by @c-bata in #128
• Bump the version up to v0.9.0 by @c-bata in #129

New Contributors

• @knshnb made their first contribution in #121
• @amylase made their first contribution in #126

References

Full Changelog: v0.8.2...v0.9.0

v0.8.2

CHANGES

• Fix dimensions of Warm starting CMA-ES (#98).
  • Thank you @Yibinjiang for reporting the bug.

v0.8.1

CHANGES

• Unset version constraint of numpy.
• Remove extra_requires for development.

v0.8.0

CHANGES

New features

Warm-starting CMA-ES is now available. It estimates a promising distribution, then generates parameters of the multivariate gaussian distribution used for the initialization of CMA-ES, so that you can exploit optimization results from a similar optimization task. This algorithm is proposed by @nmasahiro, a maintainer of this library, and accepted at AAAI 2021.

Rot Ellipsoid	Ellipsoid

• Masahiro Nomura, Shuhei Watanabe, Youhei Akimoto, Yoshihiko Ozaki, Masaki Onishi. “Warm Starting CMA-ES for Hyperparameter Optimization”, AAAI. 2021.

Link

• PyPI: https://pypi.org/project/cmaes/0.8.0/

v0.7.1

CHANGES

• Support Python 3.9 (#84)
• Refactor constants definitions (#85)
• Add tox.ini (#86)
• Remove numpy from setup_requires (#90)

v0.7.0

CHANGES

New features

Separable CMA-ES is added at #82. It accelerates the search by ignoring the dependency of variables. This is inefficient if there is a strong dependency between variables; however, this technique significantly improves the performance if the dependency can be ignored.

Benchmark

• R. Ros, N. Hansen. A Simple Modification in CMA-ES Achieving Linear Time and Space Complexity, PPSN, 2008.

Dependency

• Drop Python 3.5 support (#74)

Links

• PyPI: https://pypi.org/project/cmaes/0.7.0/