Possible bug in ablation

[Michael190502]

Hi,

I think I found a possible bug in the ablation file.

Firstly I think after the following line

DeepCAVE/deepcave/evaluators/ablation.py

Line 128 in 58d6801

inc_cost, _ = self._model.predict(np.array([incumbent_encode]))

inc_cost = inc_cost[0] is missing (which is there for def_cost). This though only affects the comparison between these two values a few lines later I think.

Secondly the ablation path analysis did not work correctly for me when I had only categorical hyperparameters. I traced the problem to the ablation file where "encode_config" is used instead of "get_array". This resulted, in my case at least, in an encoding normalised to [0,1] instead of the number of the choice of the categorical hyperparameter. This resulted in the predicted performance of the default configuration being equal to the predicted performance of the incumbent.

It worked correctly when I replaced "encode_config" with "get_array".

Please let me know if this is something specific to my setup or if I did something wrong. I will append an image comparing the encoding of "get_array" to "encode_config" for clarity.

[mwever]

Hi @Michael190502 ,

thanks for reporting this issue, we will have a look at it!