You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The challenge is, there is no straightforward way of telling how many results a flow will produce, given that individual steps or parallel steps can create multiple outputs for any one input. Thus, while it is easy to count how many run results are already complete, we cannot easily translate this to 100%.
For a meaningful progress bar, we need to come up with progress estimates for all steps & propagate these along the flow. I.e. once we know the number of inputs, we estimate the number of outputs of every step to be 1, then update that estimate if the input/output ratio starts to be different. in most cases this will lead to a reasonable progress bar. In some cases, estimates may need to be updated and progress can go from say 80% back to 60%.
We should implement this as an optional argument to the run function, called progress_bar, which defaults to False. This approach ensures that the progress bar is only shown when it is actually useful, avoiding unnecessary clutter when the pipeline runs quickly due to cached results or because the user is testing out a simple pipeline.
The text was updated successfully, but these errors were encountered:
The challenge is, there is no straightforward way of telling how many results a flow will produce, given that individual steps or parallel steps can create multiple outputs for any one input. Thus, while it is easy to count how many run results are already complete, we cannot easily translate this to 100%.
For a meaningful progress bar, we need to come up with progress estimates for all steps & propagate these along the flow. I.e. once we know the number of inputs, we estimate the number of outputs of every step to be 1, then update that estimate if the input/output ratio starts to be different. in most cases this will lead to a reasonable progress bar. In some cases, estimates may need to be updated and progress can go from say 80% back to 60%.
We should implement this as an optional argument to the
run
function, calledprogress_bar
, which defaults toFalse
. This approach ensures that the progress bar is only shown when it is actually useful, avoiding unnecessary clutter when the pipeline runs quickly due to cached results or because the user is testing out a simple pipeline.The text was updated successfully, but these errors were encountered: