[PRE REVIEW]: WorkflowMaker, a visual workflow software tool for the easy integration of self-developed console applications #7735
Comments
editorialbot
added
pre-review
Track: 7 (CSISM)
|
Hello human, I'm @editorialbot, a robot that can help you with some common editorial tasks. For a list of things I can do to help you, just type: For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type: |
|
Software report: Commit count by author: |
|
Paper file info: 📄 Wordcount for ✅ The paper includes a |
|
License info: ✅ License found: |
|
|
👋 @jose-navarro - thanks for your submission.
|
|
Dear Daniel, WorkflowMaker may help researchers when these have implemented the solutions to some particular problems using console applications and there is a notable need to combine these to check new solutions (algorithms, for instance) or to obtain different kind of results. For instance, we have several software systems where this paradigm applies. I’ll describe the typical situations that we face when researchers work with our GEMMA system. This one is a trajectory determination system. As the name states, this system intends to determine the trajectory of some object (typically, vehicles) using observations from inputs originating in a wide variety of sensors, such as, for instance, GPS receivers, LiDAR scanners or 2D/ 3D cameras, among many others. A trajectory, on the other side, is nothing but a set of positions (x,y,z) and attitudes (the three rotational angles), so computing a trajectory is computing the set of successive positions that define it. To determine the trajectory there are many possibilities: for instance, computing the positions delivered by each sensor independently and then try to obtain a final result by using some kind (which one, there may be many) of integration algorithm providing a global solution taking all the partial ones into account; or, on the contrary, it is possible to consider the raw observations coming from all sensors at once to, from these data, compute directly a set of positions. But, maybe, one of the sensors is too noisy, so integrating all raw observations from the very beginning may introduce errors, so, why not integrate at once only a raw observations coming from a subset of sensors, and then filter and compute separately the position delivered by the noisy one, and finally, integrate everything with said integration algorithm? On the other side… who says that the algorithm to integrate the positions computed from several sensors I talked about above is the best? What if we try a new one implementing the latest scientific contributions in the area? Will it work better than the one we have? Or it will work only better for this or that sensor only? More on this. assume that we’ve discovered that some new sensor is rather noisy, so maybe applying some kind of filtering to its output before it is used to generate positions will help to improve the results. What filter is best? A Kalman one? Any other? Or a combination of several of these? But we still can make things more complicated: should the trajectory determined in real time or may we process the data later at the lab? Is our system able to switch the interface to obtain the sensor data from real-time to batch? In real time we need some application(s) able to capture the output of the sensors; when we process data in batch mode we just need file readers for each specific data format instead. And still more variables: let’s assume that we want to evaluate if some new sensor is good enough to reach some performance (quality level, positions with a standard deviation below some threshold). The problem, in many situations, is that the sensor is too expensive and buying it before being sure that it will be good enough for the problem is not an option. In these cases it is possible to simulate the raw observations that the sensor would produce by means of a signal generator (providing that the mathematical characterization of the sensor is available, either because the manufacturer made it public or because other colleagues computed it). In this case, we would replace the batch processing of pre-recorded observations or the module able to read the data produced on real time by the sensor by this signal simulator, and then try to compute the positions to see if their quality is the expected one, thus deciding whether to buy it or not. I could go further but I think that the situation should be clear now. As you can see from the description above, many different algorithms – and thus, console applications – may be identified. For instance, we may have an application to apply a Kalman filter; or a software able to read the data recorded by some optical sensor; or an application to derive positions from the observations coming from a LiDAR sensor, or a signal simulator for the newest GPS receiver or... Of course, it would be possible to prepare a monolithic application able to deal with all these situations (we did something like this in the past, and now it is dead software), but this implies that any change, any new feature, must be incorporated into the code, thus requiring very specialized researchers with the necessary skills to develop and maintain the software. Moreover, it is necessary to implement all the possible combination of sub-algorithms that the user may decide to run (as for instance: real time or batch, apply a filter or not, integrate observations to a single position or compute separate positions and the integrate these, …) This, clearly, is a solution lacking flexibility and requiring very specialized personnel. On the other hand, if these specific algorithms are implemented as independent console applications, testing a combination of the situations described above leads to combining the corresponding console applications. A new feature (such a new sensor reader, or an improved data filter) should also be implemented as another console application. This fosters flexibility and simplicity. Of course, developers are still needed, but the complexity of the problems to tackle is smaller and the time needed to have new software ready or maintain old one is shorter. Thus, and assuming that console applications are a solution good enough to face this kind of situations, there is a still a problem remaining. When the number of applications to combine starts to grow, it will be necessary to run the corresponding console applications one by one, in the appropriate order, keeping track of input and output files for each of them as well as of the keyboard parameters these need. This is a time-consuming and prone to errors process. A solution would be to write scripts to implement the required workflows (obviously, one script per each workflow to check). But again, some programming skills are needed to do this, so researchers with no programming skills are out of the equation. And here’s where WorkflowMaker may help. As stated in the paper, adhering to a very simple set of restrictions when developing console applications, makes possible to integrate them in workflows created with this tool. That is, their interface (input & output files as well as keyboard parameters) must be standardized. This is something that a researcher / developer able to deal with Kalman filters or GPS receivers can do very easily. Then, and thanks to this standardization, it is possible to formally describe these applications creating the so-called toolkits (tool: ToolkitEditor). And at this point is where researcher with programming skills are not needed any more. Other researchers may then DRAW workflows combining the tools (console applications) defined in the toolkit, inserting them in the canvas and connecting their outputs to the inputs of other tasks. In this way, it is possible to define as many workflows as desired to test all the situations described above, very quickly and in a visual way. This is the task of WorkflowEditor. The key point here is that once that a workflow has been designed, the researcher only needs to provide the names of the input and output files (not those of the intermediate ones, these are named automatically) as well as the values of the so-called keyboard parameters by means of WorkflowLauncher. Once all these values have been input, it is possible to generate the necessary script (either for Linux or Windows) which researchers only needs to run to process their data. I beg your pardon for such a long explanation, but I thought that it was very important to state how WorkflowMaker could help researchers in their daily work. I would have included it in the paper if the size limitations would not be so hard. At any rate, I hope that it is enough to clarify the role played by WorkflowMaker in research environments. On the other side, I’ll correct the erroneous doi entries. Note, however, that these correspond to papers published either in journals or conferences. There are many other references, pointing to web pages, that have no doi since there is none for them! |
|
@editorialbot check references |
|
|
@jose-navarro - thanks. Don't worry about references without DOIs, but do please fix the 3 references that have invalid DOIs still (the leading \ before a _ isn't part of the DOI). Additionally, I will be asking the JOSS editors to take a look at this to ensure that it is research software as defined by JOSS. Your response above is helpful, but I want other editors to weigh in. You should hear back in a week or two. |
|
@editorialbot check references |
|
@editorialbot query scope |
|
Submission flagged for editorial review. |
|
@editorialbot check references |
|
1 similar comment
|
|
@editorialbot generate pdf |
Submitting author: @jose-navarro (José Antonio Navarro Esteban)
Repository: https://github.com/jose-navarro/workflowmaker
Branch with paper.md (empty if default branch):
Version: v1.1.3
Editor: Pending
Reviewers: Pending
Managing EiC: Daniel S. Katz
Status
Status badge code:
Author instructions
Thanks for submitting your paper to JOSS @jose-navarro. Currently, there isn't a JOSS editor assigned to your paper.
@jose-navarro if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). You can search the list of people that have already agreed to review and may be suitable for this submission.
Editor instructions
The JOSS submission bot @editorialbot is here to help you find and assign reviewers and start the main review. To find out what @editorialbot can do for you type:
The text was updated successfully, but these errors were encountered: