Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adds support for configuring hardware objects with YAML files #1062

Draft
wants to merge 53 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
2dcdad7
First step in moving to compatible XML-YML transition status
rhfogh May 19, 2024
077274b
Second step in moving to compatible XML-YML transition status
rhfogh May 21, 2024
ddb3f3a
Third step in moving to compatible XML-YML transition status
rhfogh May 22, 2024
93ed233
Fourth step in moving to compatible XML-YML transition status. Config…
rhfogh May 23, 2024
e3a050e
Branch now runs in mock mode. Added export of yml config files
rhfogh May 23, 2024
a081325
Branch now runs in mock mode. Added export of yml config files
rhfogh May 30, 2024
0d7e3fe
Bug fix - and added warning for unrecognised role
rhfogh May 30, 2024
198792f
Bug fix - and added warning for unrecognised role
rhfogh May 30, 2024
b4ab9b0
Updated access to elements
marcus-oscarsson May 30, 2024
2570e23
Added harvester to beamline object
marcus-oscarsson May 30, 2024
4e0a888
yaml: update edna workflow to new API
elmjag Jul 1, 2024
1bc736d
yaml: include YAML loaded HWOs into 'hardware_objects' dictionary
elmjag Jul 1, 2024
79bae0f
adds dynamic 'export yaml config' feature
marcus-oscarsson Jun 27, 2024
9ddb967
Bug fix: Added '#' in front of ''%YAML 1.2'
rhfogh Jul 5, 2024
cdd4404
Changed YAML default indentation to yaml.indent(mapping=2, sequence=4…
rhfogh Jul 5, 2024
b13a606
reshuffle code for populating _config attribute for XML configs
elmjag Aug 2, 2024
caba802
fix issue with skipped calls to init() for YAML HWOBJs
elmjag Aug 12, 2024
8bbbd5d
update Session HWOBJ to work with both YAML and XML configs
elmjag Aug 2, 2024
c6b17ff
docs: update and expand 'Configuration files' section
elmjag Aug 20, 2024
845963e
correct deprecation warning fix suggestion
elmjag Aug 12, 2024
f6bea5d
docs: XML to YAML config migration document
elmjag Jul 5, 2024
ad2c3ec
'emulate' get_object_by_role() access for YAML HWOBJs
elmjag Aug 12, 2024
fcf1231
handle missing YAML configure files
elmjag Aug 15, 2024
f1e0cff
prevent re-loading same YAML config file twice
elmjag Aug 27, 2024
3166090
make HWR.beamline.acquisition_limit_values work again
elmjag Aug 19, 2024
0359411
port AbstractDetector to use get_property()
elmjag Aug 12, 2024
221f12a
port beam releated HWOBJS to work with YAML
elmjag Aug 13, 2024
4f2c684
make some diffractometer related HWOBJs work with YAML
elmjag Aug 14, 2024
0021b47
make some sample changer related HWOBJs work with YAML
elmjag Aug 14, 2024
43c5f99
remove proxy 'camera' attribute on SampleView HWOBJs
elmjag Aug 15, 2024
6af0b18
refactor initialization code of MDCameraMockup
elmjag Aug 15, 2024
6bf2325
make BeamlineActions HWOBJs work with YAML
elmjag Aug 15, 2024
7b0c4fb
add detector_distance attribute to detector mockup HWOBJ
elmjag Aug 21, 2024
7004ff6
tests: remove test on HardwareObjectNode.__getattr__ method
elmjag Aug 16, 2024
cfe658b
tests: remove checks for outdated HardwareObjectNode.__setattr__() be…
elmjag Aug 16, 2024
a95a77f
tests: remove test on HardwareObjectNode.has_object() method
elmjag Aug 16, 2024
1aa983e
tests: move print_log() method test to new class
elmjag Aug 16, 2024
485e364
tests: use new method name of HardwareObjectNode._objects_names()
elmjag Aug 16, 2024
66099bd
tests: update a couple of BaseHardwareObjects tests
elmjag Aug 16, 2024
a859ce8
tests: fix procedure tests
elmjag Aug 16, 2024
fbaf2cb
tests: update test YAML configs format
elmjag Aug 16, 2024
0866295
tests: fix a command container test
elmjag Aug 16, 2024
e06cb35
tests: update MAXIV MachInfo tests
elmjag Sep 3, 2024
1ba2bc2
make a couple of mock HWOBJs work with YAML
elmjag Sep 4, 2024
666e4de
docs: add section on removed set_property()
elmjag Sep 16, 2024
1c741e4
port CentringMath HWOBJ to work with the yaml config API changes
elmjag Sep 12, 2024
f392ad5
export YAML configuration to '.yaml' files
elmjag Oct 3, 2024
043d2e0
docs: add section on deprecated <tangoname> tag
elmjag Oct 15, 2024
28f6cf6
restructure slightly the docs strings in CommandContainer module
elmjag Oct 14, 2024
7f71d38
add docs on commands and channels YAML configuration format
elmjag Sep 27, 2024
ed87ac8
adds support for configuring commands and channels using YAML
elmjag Sep 27, 2024
76fb9ae
ISPyB client: handle boolean loginTranslate property correctly
elmjag Oct 16, 2024
02a5b97
remove GenericDiffractometer's 'zoom' proxy attribute
elmjag Oct 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions deprecated/HardwareObjects/Camera.py
Original file line number Diff line number Diff line change
Expand Up @@ -482,15 +482,15 @@ def takeSnapshot(self, *args, **kwargs):
# img.save(*args)
except Exception:
logging.getLogger("HWR").exception(
"%s: could not save snapshot", self.name()
"%s: could not save snapshot", self.id
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wondering where the id attribute gets assigned and I noticed that .name() is sometimes replaced by .name and sometimes with id

)
else:
if len(args):
try:
img.save(*args)
except Exception:
logging.getLogger("HWR").exception(
"%s: could not save snapshot", self.name()
"%s: could not save snapshot", self.id
)
else:
return True
Expand All @@ -499,7 +499,7 @@ def takeSnapshot(self, *args, **kwargs):
else:
logging.getLogger("HWR").error(
"%s: could not take snapshot: sorry PIL is not available :-(",
self.name(),
self.id,
)
return False

Expand Down Expand Up @@ -833,15 +833,15 @@ def takeSnapshot(self, *args):
# img.save(*args)
except Exception:
logging.getLogger("HWR").exception(
"%s: could not save snapshot", self.name()
"%s: could not save snapshot", self.id
)
else:
if len(args):
try:
img.save(*args)
except Exception:
logging.getLogger("HWR").exception(
"%s: could not save snapshot", self.name()
"%s: could not save snapshot", self.id
)
else:
return True
Expand All @@ -850,7 +850,7 @@ def takeSnapshot(self, *args):
else:
logging.getLogger("HWR").error(
"%s: could not take snapshot: sorry PIL is not available :-(",
self.name(),
self.id,
)
return False

Expand Down
4 changes: 2 additions & 2 deletions deprecated/HardwareObjects/NamedState.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,13 +65,13 @@ def connect_notify(self, signal):
self.emit(signal, (self.get_state(),))

def stateChanged(self, channelValue):
logging.info("hw NamedState %s. got new value %s" % (self.name(), channelValue))
logging.info("hw NamedState %s. got new value %s" % (self.id, channelValue))
self.set_is_ready(True)
self.emit("stateChanged", (self.get_state(),))

def hardwareStateChanged(self, channelValue):
logging.info(
"hw NamedState %s. Hardware state is now %s" % (self.name(), channelValue)
"hw NamedState %s. Hardware state is now %s" % (self.id, channelValue)
)
self.hdw_state = channelValue
self.emit("hardwareStateChanged", (self.hdw_state,))
Expand Down
297 changes: 297 additions & 0 deletions docs/source/dev/commands_channels.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,297 @@
# Commands and Channels

The mxcubecore provides a hardware object-level abstraction
for communicating using various flavors of control system protocols.
A hardware object can utilize instances of
[`CommandObject`](#mxcubecore.CommandContainer.CommandObject) and
[`ChannelObject`](#mxcubecore.CommandContainer.ChannelObject) objects.
These objects provide a uniform API for accessing a specific control system.
Mxcubecore provides support for using protocols such as
[_Tango_](https://www.tango-controls.org/),
[_EPICS_](https://docs.epics-controls.org),
_exporter_ and more.

The `CommandObject` and `ChannelObject` objects can be created using the `add_command()` and `add_channel()` methods of hardware objects.
Another option is to specify them in the hardware object's configuration file.
If `CommandObject` and `ChannelObject` are specified in the configuration file,
the specified objects will be automatically created during hardware object initialization.

## Configuration files format

The general format for specifying `CommandObject` and `ChannelObject` objects is as follows:

```yaml
class: <hardware-object-class>
<protocol>: # protocol in use, tango / exporter / epics / etc
<end-point>: # tango device / exporter address / EPICS prefix / etc
commands:
<command-1-name>: # command's MXCuBE name
<config-prop-1>: <val-1>
<config-prop-2>: <val-2>
channels:
<channel-1-name>: # channel's MXCuBE Name
<config-prop-1>: <val-1>
<config-prop-2>: <val-2>
```

The `CommandObject` and `ChannelObject` specification are grouped by the protocol they are using.
Each protocol have its own dedicated section in the configuration file.
The semantics for the protocol are similar but protocol-specific, see below for details.

Currently, the following protocols can be configured using YAML configuration files:

- [Tango](#tango-protocol)
- [exporter](#exporter-protocol)
- [EPICS](#epics-protocol)

## Tango Protocol

The format for specifying _tango_ `CommandObject` and `ChannelObject` objects is as follows:

```yaml
class: <hardware-object-class>
tango:
<tango-device-name>:
commands:
<command-1-name>:
<config-prop-1>: <val-1>
<command-2-name>:
<config-prop-1>: <val-1>
channels:
<channel-1-name>:
<config-prop-1>: <val-1>
<config-prop-2>: <val-2>
<channel-2-name>:
<config-prop-1>: <val-1>
```

`<tango-device-name>` specifies the tango device to use.
Multiple `<tango-device-name>` sections can be specified, in order to use different tango devices.
Each `<tango-device-name>` contains optional `commands` and `channels` sections.
These sections specify `CommandObject` and `ChannelObject` object to create using the `<tango-device-name>` tango device.

### Commands

`commands` is a dictionary where each key specifies a `CommandObject` object.
The key defines the MXCuBE name for the command.
The values specify an optional dictionary with configuration properties for the `CommandObject` object.
The following configuration properties are supported:

| property | purpose | default |
|----------|--------------------|---------------------|
| name | tango command name | MXCuBE command name |

### Channels

`channels` is a dictionary where each key specifies a `ChannelObject` object.
The key defines the MXCuBE name for the channel.
The values specify an optional dictionary with configuration properties for the `ChannelObject` object.
The following configuration properties are supported:

| property | purpose | default |
|----------------|------------------------------------|---------------------|
| attribute | tango attribute name | MXCuBE channel name |
| polling_period | polling periodicity, milliseconds | polling is disabled |
fabcor-maxiv marked this conversation as resolved.
Show resolved Hide resolved
| timeout | tango device timeout, milliseconds | 10000 |

By default, a tango `ChannelObject` object will use tango attribute change event, in order to receive new attribute values.
For this to work, the tango device must send the change events for the attribute.
For cases where such events are not sent, the attribute polling can be enabled.
If `polling_period` property is specified, MXCuBE will poll the tango attribute with specified periodicity.

### Example

Below is an example of a hardware object that specifies Tango commands and channels.

```yaml
class: MyTango
tango:
some/tango/device:
commands:
Open:
Close:
Reset:
name: Reboot
channels:
State:
Volume:
attribute: currentVolume
polling_period: 1024
```

In the above example, commands `Open`, `Close` and `Reset` as well as channels `State` and `Volume` are configured.
All command and channel objects are bound to the commands and attributes of the _some/tango/device_ tango device.

`Open` and `Close` commands are bound to _Open_ and _Close_ Tango commands.
The `Reset` has a configuration property that binds it to _Reboot_ tango command.

The `State` channel will be mapped to _State_ attribute of the Tango device.
Its value will be updated via Tango change events.

The `Volume` channel will be mapped to the _currentVolume_ attribute of the tango device.
The _currentVolume_ attribute's value will be polled every 1024 milliseconds.

## Exporter Protocol

The format for specifying _exporter_ `CommandObject` and `ChannelObject` objects is as follows:

```yaml
class: <hardware-object-class>
exporter:
<exporter-address>:
commands:
<command-1-name>:
<config-prop-1>: <val-1>
<command-2-name>:
<config-prop-1>: <val-1>
channels:
<channel-1-name>:
<config-prop-1>: <val-1>
<channel-2-name>:
<config-prop-1>: <val-1>
```

`<exporter-address>` specifies the exporter address to use.
Multiple `<exporter-address>` sections can be specified to use devices at different addresses.
Each `<exporter-address>` contains optional `commands` and `channels` sections.
These sections specify `CommandObject` and `ChannelObject` objects to create using the `<exporter-address>` tango device.

`<exporter-address>` specifies the exporter's host and port number.
It has the following format: `<host>:<port>`.
`<host>` is the host name or IP address to use.
`<port>` is the TCP port number to use.
Note that, due to YAML parsing rules, you need to use quotes when specifying the exporter address.
Below is an example of an exporter address that can be used in a YAML configuration file:

```
"foo.example.com:9001"
```

### Commands

`commands` is a dictionary where each key specifies a `CommandObject` object.
The key defines the MXCuBE name for the command.
The values specify an optional dictionary with configuration properties for the `CommandObject` object.
The following configuration properties are supported:

| property | purpose | default |
|----------|-----------------------|---------------------|
| name | exporter command name | MXCuBE command name |

### Channels

`channels` is a dictionary where each key specifies a `ChannelObject` object.
The key defines the MXCuBE name for the channel.
The values specify an optional dictionary with configuration properties for the `ChannelObject` object.
The following configuration properties are supported:

| property | purpose | default |
|-----------|--------------------------|---------------------|
| attribute | exporter attribute name | MXCuBE channel name |

### Example

Below is an example of a hardware object that specifies exporter commands and channels.

```yaml
class: MyExporter
exporter:
"foo.example.com:9001":
commands:
Open:
Close:
Reset:
name: Reboot
channels:
State:
Volume:
attribute: currentVolume
```

In the above example, commands `Open`, `Close` and `Reset` as well as `State` and `Volume` channels are configured.
All command and channel objects are bound to the exporter host _foo.example.com_ at port _9001_.

`Open` and `Close` commands are bound to _Open_ and _Close_ exporter commands.
The `Reset` has a configuration property that binds it to the _Reboot_ exporter command.

The `State` channel will be mapped to _State_ exporter attribute.
The `Volume` channel will be mapped to _currentVolume_ exporter attribute.

## EPICS Protocol

The format for specifying _EPICS_ `ChannelObject` objects is as follows:

```yaml
class: EpicsCommunicator
epics:
<prefix>:
channels:
<channel-1-name>:
<config-prop-1>: <val-1>
<config-prop-2>: <val-2>
<channel-2-name>:
<config-prop-1>: <val-1>
```

`<prefix>` specifies the EPICS PV prefix to use for that section.
Multiple `<prefix>` sections can be specified, in case not all channels share a common prefix.
Each `<prefix>` contains a `channels` section, which specifies `ChannelObject` objects to create.

It is also possible to use the empty string, `""`, as the prefix.
This is useful in cases where none of the channels share a common prefix.
See [below](#pv-names) for details on how channel PV names are determined.

### Channels

`channels` is a dictionary where each key specifies a `ChannelObject` object.
The key defines the MXCuBE name for the channel.
The values specify an optional dictionary with configuration properties for the `ChannelObject` object.
The following configuration properties are supported:

| property | purpose | default |
|----------------|-----------------------|---------------------|
| suffix | PV name suffix | MXCuBE channel name |
| polling_period | polling periodicity | |

#### PV names

The PV name of a channel is determined by concatenating its section's prefix and the specified `suffix`.
If no suffix is specified, the channel's MXCuBE name is used in place of the `suffix`.
Observe an example configuration below:

```yaml
class: EpicsCommunicator
epics:
"FOO:B:":
channels:
State:
suffix: pv_1.STAT
Vol:
suffix: volume.VAL
Freq:
```

Her we have an `FOO:B:` prefix specified, with channels `State`, `Vol` and `Freq`.
The `State` channel will use `FOO:B:pv_1.STAT` PV name, specified by section's prefix and the `suffix` configuration property.
The `Vol` channel's PV name will be`FOO:B:volume.VAL`.
The `Freq` channel's PV name becomes `FOO:B:Freq`, specified by section's prefix and channel's MXCuBE name.

### Example

Below is an example of a hardware object that specifies EPICS channels.

```yaml
class: EpicsCommunicator
epics:
"MNC:B:PB04.":
channels:
State:
Volume:
suffix: vlm
polling_period: 512
```

In the above example channels `State` and `Volume` are configured.
The `State` channel will be mapped to PV name _MNC:B:PB04.State_.
The `Volume` channel will be mapped to PV name _MNC:B:PB04.vlm_.
For `Volume` channel, polling will be enabled.
Loading
Loading