Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mobile: be prepared for failure during experiment creation #2558

Closed
bassosimone opened this issue Oct 11, 2023 · 3 comments · Fixed by ooni/probe-android#622 or ooni/probe-ios#534
Closed
Assignees
Labels

Comments

@bassosimone
Copy link
Contributor

In engine <= 3.18, you can always create an experiment if the experiment name exists. In engine >= 3.19, this is no longer the case, because of ooni/probe-cli#1355. For example, the "echcheck" experiment cannot be created unless the check-in API has authorized us to do so in the last 24h. The cli code is already handling this case correctly and there seems to be no database side effect. We need to make sure the same holds for the mobile apps.

@aanorbel
Copy link
Member

Supported as part of ooni/probe-android#622 .
This also introduces a situation where the test suite can be successful and still has a list of errors.
See the example below for a sample experimental test suite.

echcheck: experiment not enabled by check-in API

torsf: experiment not enabled by check-in API
riseupvpn: experiment not enabled by check-in API

I propose we come up with a way to inform the users that some experiments were not started.
We have had issues reported of experiments not being run when they are enabled in the test options #2204

cc: @bassosimone , @agrabeli , @hellais

@aanorbel aanorbel linked a pull request Oct 16, 2023 that will close this issue
@bassosimone
Copy link
Contributor Author

@aanorbel the engine communicates errors to the mobile apps using strings. We can use the "experiment not enabled by check-in API" string to know that an experiment was disabled by a OONI backend policy. If we choose to go down this road, please, let me know, so that I can document the specific error string as precious for the apps and thus increase the probability that it would not be changed by mistake when refactoring.

@aanorbel
Copy link
Member

Since there is currently no way to inform users about the OONI backend policy disabling a specific test, I propose we discuss it in the next front-end meeting so a decision can be taken.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants