Skip to content

Commit

Permalink
preserve consumer-provided "runOnJobQueue" value when set to True (#23)
Browse files Browse the repository at this point in the history
* preserve consumer-provided "runOnJobQueue" value when set to True

Previously, the "runOnJobQueue" parameter was always overwritten with False, even if a consumer explicitly passed True. This behavior prevented consumers from skipping the initial attempt without the job queue, even when it was certain their query would go to the job queue.

By using setdefault(), the parameter is only set to False if it is not already provided. This change allows consumers who know their query requires the job queue to bypass the first attempt and directly push the job to the queue.

* add logic for skipping first query and query directly over job queue
  • Loading branch information
huwylphimet authored Jan 8, 2025
1 parent fd43fe4 commit fa95cc5
Showing 1 changed file with 18 additions and 14 deletions.
32 changes: 18 additions & 14 deletions meteoblue_dataset_sdk/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -169,21 +169,25 @@ async def _query_raw(self, params: dict):
:return: ClientResponse object from aiohttp lib
"""

# always try to execute without job queue first:
params["runOnJobQueue"] = False
# always try to execute without job queue by default:
params.setdefault("runOnJobQueue", False)
async with aiohttp.ClientSession() as session:
# Try to run the job directly
# In case the API throws an error, try to run it on a job queue
try:
url = self._config.query_url.format(self._config.api_key)
async with self._fetch(
session, "POST", url, body_dict=params
) as response:
yield response
except ApiError as error:
# Run on a job queue in case the api throws this error
if error.message != "This job must be executed on a job-queue":
raise
if not params.get("runOnJobQueue"):
# Try to run the job directly
# In case the API throws an error, try to run it on a job queue
try:
url = self._config.query_url.format(self._config.api_key)
async with self._fetch(
session, "POST", url, body_dict=params
) as response:
yield response
except ApiError as error:
# Run on a job queue in case the api throws this error
if error.message != "This job must be executed on a job-queue":
raise
async with self._run_on_job_queue(session, params) as response:
yield response
else:
async with self._run_on_job_queue(session, params) as response:
yield response

Expand Down

0 comments on commit fa95cc5

Please sign in to comment.