Upload failed runs

[DanielKuehn87]

Is it currently possible to upload failed runs using R?
I'd like to know for which parameter-settings my mlr runs fail.

Seems like OpenML should be able to handle it: openml/OpenML#424

[joaquinvanschoren]

Yes this is supported on the server

On Fri, 23 Jun 2017 at 10:48, DanielKuehn87 ***@***.***> wrote: Is it currently possible to upload failed runs using R? I'd like to know for which parameter-settings my mlr runs fail. Seems like OpenML should be able to handle it: openml/OpenML#424 <openml/OpenML#424> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#371>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABpQVyDyLs-D8j2ZPk5rkRvHtgo_yq78ks5sG3v1gaJpZM4ODTKM> .

-- Thank you, Joaquin

[giuseppec]

Unfortunately, the R package currently does not support this (at least this is not tested). Would you be willing to help here to start a PR?

[joaquinvanschoren]

Also, if you want to *download* failed runs, there is an option 'show_errors' that you must set to true, like this: https://www.openml.org/api/v1/run/list/uploader/86/show_errors/true This is not documented ATM

On Fri, Jun 23, 2017 at 1:10 PM Giuseppe Casalicchio < ***@***.***> wrote: Unfortunately, the R package currently does not support this (at least this is not tested). Would you be willing to help here to start a PR? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#371 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABpQV_doo4xmjLcdkuUkuoqFUW4lJ1ySks5sG50hgaJpZM4ODTKM> .

-- Thank you, Joaquin

[DanielKuehn87]

@joaquinvanschoren Where is the documentation for the upload of the failed runs? Is it just uploadrun without any predictions? https://www.openml.org/api_docs/#!/run/post_run
I'll have a look, if I can manage to integrate it into the R interface.

[giuseppec]

Yeah, I think you need to make sure that runTaskMlr produces a mlr FailureModel object and try to upload the run without predictions (but using the error message of the FailureModel ).

[joaquinvanschoren]

I don't think this is explicitly documented, but the XSD shows how you can upload an error message with a run:
https://www.openml.org/api/v1/xsd/openml.run.upload