Make apps launch cartesian controller locally?

[PeterBowman]

I've recently committed a few control apps that launch an instance of CartesianControlClient and expect to find a CartesianControlServer wrapping a specific implementation somewhere in the YARP network. That is, communication between the app and the controller is held through YARP ports, regardless of whether both are executing on the same machine.

Question 1: could this impose a delay depending on network latency?

Question 2: if so, could it be minimized by making the app open an instance of the controller (e.g. BasicCartesianControl) and access it directly instead of proxying through the client device? Check whether the local carrier would fit this particular use case.

Question 3: assuming that we decide to launch both the app and the controller bundle on the same machine separately (that is, involving the CCC/CCS pair), could we benefit from switching to the shmem carrier?

Question 4: is it possible for the CartesianControlClient to automate some of these choices?

Attaching diagrams for two apps, bigger rectangles are associated to YARP devices.

---

keyboardController

---

streamingDeviceController

[jgvictores]

Some (somewhat historically, hopefully helpful) comments on this:

Question 1: could this impose a delay depending on network latency?

Not sure if I understand this question, but taking advantage of this place to add a comment: AFAIK, both TCP and UDP comms on the same machine should go through the local loop-back (lo) interface, which turns out to be pretty fast and not depend on what other interfaces (e.g. eth0 or wlan0) are doing.

Question 2: if so, could it be minimized by making the app open an instance of the controller (e.g. BasicCartesianControl) and access it directly instead of proxying through the client device? Check whether the local carrier would fit this particular use case.

Originally, on ASIBOT, we used to launch the full stack all with local devices: application-load_local->cartesianbot-load_local->ravebot/canbot. The reason we stopped doing this was with TEO, where we had multiple limbs and thus several instances of BasicCartesianController, which forces splitting and a certain paradigm shift. There are reasons for maintaining everything decoupled (reading console output, seeing what breaks at what level), but for ASIBOT the full stack launching is feasible and efficient in terms of speed (direct calls are obviously faster than comms).

The local carrier looks like a different approach. It does look good and more maintained than shmem. Would require some testing.

Question 3: assuming that we decide to launch both the app and the controller bundle on the same machine separately (that is, involving the CCC/CCS pair), could we benefit from switching to the shmem carrier?

My experience with shmem was that it did not improve communication times. In fact, communications on a single Windows machine seemed to be slower (:sweat:). Again, would ultimately require some testing (but I'd take this experience into account as a first approach).

Question 4: is it possible for the CartesianControlClient to automate some of these choices?

An original intention of yarpmanager was to have some cluster load management. The idea was to have a map of resources, and have the machine reason "okay, I have enough resources on the robot CPU, I can launch the kinematics there", or the opposite case "let's send these computations to that machine with a good GPU". Not very sure if that aspect ever matured, but I do believe it would be a good idea to put that logic on the yarpmanager side rather than on any specific module (though it would be pretty cool to see CartesianControlClient doing that sort of thing 😄).

PS: BTW, I cannot find the issue, but you asked the purpose/extent of BasicCartesianController. I guess it could be renamed to something like OneLimbCartesianController to express it should be capable of anything that requires a single limb (i.e. anything except walking with two legs, or whole-body motion control).

[jgvictores]

PS: I updated my comment slightly just to put answers in line with questions.
PPS: Cool diagrams! Are they uploaded anywhere? 🙂

[PeterBowman]

Thanks for your valuable answers, @jgvictores!

The first one relieves many of my concerns around performance - it confirms my suspicions that network comms are somewhat smart enough to optimize local connections (i.e. same machine, as opposed to two machines speaking to eachother). Such benefits may ultimately outweigh any reason to ever use shmem, as answered at Q3.

Ad Q2: I was thinking about ASIBOT, too. Wondering how could we measure send/receive rates or rate the overall performance. Maybe the iCub team has already stumbled upon this matter?

Ad Q4: note to myself to investigate yarpmanager's handling of resource availability: http://www.yarp.it/yarpmanager.html#resource.

PPS: Cool diagrams! Are they uploaded anywhere? 🙂

Sent link through Google Drive, you may use them freely.

[PeterBowman]

PS: BTW, I cannot find the issue, but you asked the purpose/extent of BasicCartesianController. I guess it could be renamed to something like OneLimbCartesianController to express it should be capable of anything that requires a single limb (i.e. anything except walking with two legs, or whole-body motion control).

Let's discuss this at #124.

[PeterBowman]

YARP 3.4 will feature new carriers for efficiently communicating with a process on the same machine using a unix socket: robotology/yarp#2329.

[PeterBowman]

The best performance will be achieved with direct device instantiation, i.e. not involving YARP network wrappers (and hence ports). If not possible (but that's a design issue), use the unix_stream carrier. For communication between different machines and heavy bandwidth usage (should not be the case), mind the new yarp repeat command (introduced in YARP 3.5, unreleased AToW, see robotology/yarp#2434).