We’ve now worked with an Automation Engine object that represents a virtual machine, and we’ve called one of its methods to add a custom attribute to the VM.
In this chapter we’ll take a deeper look at these Automation Engine objects, and at some of the technology that exists behind the scenes in Rails when we run automation scripts. It is useful background information, but can be skipped on first read if required.
Firstly, by way of reassurance…
We do not need to know Ruby on Rails to write ManageIQ Automation scripts.
It can, however, be useful to have an appreciation of Rails models, and how the Automation Engine encapsulates these as Ruby objects that we can program with. The objects represent the things that we are typically interested in when we write automation code, such as VMs, Clusters, Guest Applications, or even Provisioning Requests.
The Ruby scripts that we write are just plain Ruby 2.2, although the Active Support core extensions to Ruby are available if we wish to use them.
|
Note
|
The Active Support extensions can make our lives easier. For example rather than adding math to our automation script to work out the number of seconds in a two-month time span (perhaps to specify a VM retirement period), we can just specify 2.months. |
Our automation scripts access Ruby objects, made available to us by the Automation Engine via the $evm variable ($evm is described in more detail in [evm-and-the-workspace]). Behind the scenes these are Rails objects, which is why having some understanding of Rails can help our investigation into how we can use these objects to our maximum benefit.
Rails is a Model-View-Controller (MVC) application [1]
The model is a representation of the underlying, logical structure of data from the database (which in the case of ManageIQ is PostgreSQL). When writing automation scripts, we work with models extensively (although we may not necessarily realise it).
Rails models are called active records. They always have a singular CamelCase name (e.g. GuestApplication), and their corresponding database tables have a plural snake_case name (e.g. guest_applications).
Active record associations link the models together in one-to-many or one-to-one relationships that allow us to traverse objects.
We can illustrate this by looking at some of the Rails code that defines the Host (i.e. Hypervisor) active record:
class Host < ActiveRecord::Base
...
belongs_to :ext_management_system, :foreign_key => "ems_id"
belongs_to :ems_cluster
has_one :operating_system, :dependent => :destroy
has_one :hardware, :dependent => :destroy
has_many :vms_and_templates, :dependent => :nullify
has_many :vms
...We see that there are several associations from a host object, including to the cluster that it’s a member of, and to the virtual machines that run on that host.
Although these associations are defined in Rails, they are available to us when we work with the corresponding service model objects from the Automation Engine (see Service Models).
Rails does a lot of things to make our lives easier, including dynamically creating helper methods. The most useful ones to us as ManageIQ automation scripters are the find_by_columnname methods.
owner = $evm.vmdb('user').find_by_id(ownerid.to_i)
vm = $evm.vmdb('vm').find_by_name(vm_name)We can .find_by_ any column name in a database table. For example in PostgreSQL we can look at the services table that represents services created via a service catalog.
vmdb_production=# \d services
Table "public.services"
Column | Type | Modifiers
----------------------+-----------------------------+---------------------------
id | bigint | not null default nextva...
name | character varying(255) |
description | character varying(255) |
guid | character varying(255) |
type | character varying(255) |
service_template_id | bigint |
options | text |
display | boolean
...
We see that there is a description column, so if we wanted we could call:
$evm.vmdb('service').find_by_description('My New Service')|
Tip
|
Don’t try searching the ManageIQ sources for def find_by_description, these are not statically defined methods and so don’t exist in the ManageIQ code. In a future version of ManageIQ they will be deprecated in favour of a more current Rails-like syntax using where, for example: $evm.vmdb('service').where(:description =>'My New Service')+ |
We saw earlier that Rails data models are called active records. We can’t access these directly from an automation script, but fortunately most of the useful ones are made available to us as Automation Engine service model objects.
The objects that we work with in the Automation Engine are all service models; instances of an MiqAeService class that abstract and make available to us their corresponding Rails active record.
For example if we’re working with a User object (representing a person, such as the owner of a virtual machine), we might access that object in our script via $evm.root['user']. This is actually an instance of an MiqAeServiceUser class, which represents the corresponding Rails User Active Record. There are service model objects representing all of the things that we need to work with when we write automation scripts. These include the traditional components in our infrastructure such as virtual machines, hypervisor clusters, operating systems or ethernet adapters, but also the intangible objects such as provisioning requests or automation tasks.
All of the MiqAeService* objects extend a common MiqAeServiceModelBase class that contains some common methods available to all objects, such as:
.tagged_with?(category, name) .tags(category = nil) .tag_assign(tag)
Many of the service model objects have several levels of superclass, for example:
MiqAeServiceMiqProvisionRedhatViaPxe <
MiqAeServiceMiqProvisionRedhat <
MiqAeServiceMiqProvision <
MiqAeServiceMiqRequestTask <
MiqAeServiceModelBase
The following list shows the class definition for some of the ManageIQ Botvinnik MiqAeService model classes, showing their immediate superclass:
class MiqAeServiceAuthentication < MiqAeServiceModelBase class MiqAeServiceAuthPrivateKey < MiqAeServiceAuthentication class MiqAeServiceAuthKeyPairCloud < MiqAeServiceAuthPrivateKey class MiqAeServiceAuthKeyPairOpenstack < MiqAeServiceAuthKeyPairCloud ... class MiqAeServiceAutomationRequest < MiqAeServiceMiqRequest class MiqAeServiceAutomationTask < MiqAeServiceMiqRequestTask ... class MiqAeServiceAvailabilityZone < MiqAeServiceModelBase class MiqAeServiceAvailabilityZoneAmazon < MiqAeServiceAvailabilityZone class MiqAeServiceAvailabilityZoneOpenstack < MiqAeServiceAvailabilityZone ... class MiqAeServiceHost < MiqAeServiceModelBase class MiqAeServiceHostMicrosoft < MiqAeServiceHost class MiqAeServiceHostOpenstackInfra < MiqAeServiceHost class MiqAeServiceHostRedhat < MiqAeServiceHost class MiqAeServiceHostVmware < MiqAeServiceHost class MiqAeServiceHostVmwareEsx < MiqAeServiceHostVmware ... class MiqAeServiceMiqProvision < MiqAeServiceMiqProvisionTask class MiqAeServiceMiqProvisionAmazon < MiqAeServiceMiqProvisionCloud class MiqAeServiceMiqProvisionCloud < MiqAeServiceMiqProvision class MiqAeServiceMiqProvisionConfiguredSystemRequest < MiqAeServiceMiqRequest class MiqAeServiceMiqProvisionMicrosoft < MiqAeServiceMiqProvision class MiqAeServiceMiqProvisionOpenstack < MiqAeServiceMiqProvisionCloud class MiqAeServiceMiqProvisionRedhat < MiqAeServiceMiqProvision class MiqAeServiceMiqProvisionRedhatViaIso < MiqAeServiceMiqProvisionRedhat class MiqAeServiceMiqProvisionRedhatViaPxe < MiqAeServiceMiqProvisionRedhat class MiqAeServiceMiqProvisionRequest < MiqAeServiceMiqRequest class MiqAeServiceMiqProvisionRequestTemplate < MiqAeServiceMiqProvisionRequest class MiqAeServiceMiqProvisionTask < MiqAeServiceMiqRequestTask ... class MiqAeServiceServiceTemplateProvisionTask < MiqAeServiceMiqRequestTask ... class MiqAeServiceVmOrTemplate < MiqAeServiceModelBase class MiqAeServiceVm < MiqAeServiceVmOrTemplate class MiqAeServiceVmCloud < MiqAeServiceVm class MiqAeServiceVmInfra < MiqAeServiceVm class MiqAeServiceVmMicrosoft < MiqAeServiceVmInfra class MiqAeServiceVmOpenstack < MiqAeServiceVmCloud class MiqAeServiceVmAmazon < MiqAeServiceVmCloud class MiqAeServiceVmRedhat < MiqAeServiceVmInfra
The service model objects that the Automation Engine makes available to us have four properties that we frequently work with, attributes, virtual columns, associations and methods.
Just like any other Ruby object, the service model objects that we work with have attributes that we often use. A service model object represents a record in a database table, and the object’s attributes correspond to the columns in the table for that record.
For example, some attributes for a RHEV Host (i.e. Hypervisor) object (MiqAeServiceHostRedhat), with typical values, are:
host.connection_state = connected
host.created_on = 2014-11-13 17:53:34 UTC
host.ems_cluster_id = 1000000000001
host.ems_id = 1000000000001
host.ems_ref = /api/hosts/b959325b-c667-4e3a-a52e-fd936c225a1a
host.ems_ref_obj = /api/hosts/b959325b-c667-4e3a-a52e-fd936c225a1a
host.guid = fcea82c8-6b5d-11e4-98ac-001a4aa01599
host.hostname = 192.168.1.224
host.hyperthreading = nil
host.id = 1000000000001
host.ipaddress = 192.168.1.224
host.last_perf_capture_on = 2015-06-05 10:25:46 UTC
host.name = rhelh03.bit63.net
host.power_state = on
host.settings = {:autoscan=>false, :inherit_mgt_tags=>false, :scan_frequency=>0}
host.smart = 1
host.type = HostRedhat
host.uid_ems = b959325b-c667-4e3a-a52e-fd936c225a1a
host.updated_on = 2015-06-05 10:43:00 UTC
host.vmm_product = rhel
host.vmm_vendor = RedHat
We can enumerate an object’s attributes using:
this_object.attributes.each do |key, value|In addition to the standard object attributes (which correspond to 'real' database columns), Rails dynamically adds a number of virtual columns to many of the service models.
|
Note
|
A virtual column is a computed database column that is not physically stored in the table. Virtual columns often contain more dynamic values than attributes, such as the number of VMs currently running on a hypervisor. |
Some virtual columns for our same RHEV Host object, with typical values, are:
host.authentication_status = Valid host.derived_memory_used_avg_over_time_period = 790.1026640002773 host.derived_memory_used_high_over_time_period = 2586.493300608264 host.derived_memory_used_low_over_time_period = 0 host.os_image_name = linux_generic host.platform = linux host.ram_size = 15821 host.region_description = Region 1 host.region_number = 1 host.total_cores = 4 host.total_vcpus = 4 host.v_owning_cluster = Default host.v_total_miq_templates = 0 host.v_total_storages = 3 host.v_total_vms = 7
We access theses virtual columns just as we would access attributes, using "object.virtual_column_name" syntax. If we want to enumerate through all of an object’s virtual columns getting the corresponding values, we must use .send, specifying the virtual column name, like so:
this_object.virtual_column_names.each do |virtual_column_name|
virtual_column_value = this_object.send(virtual_column_name)We saw earlier that there are associations between many of the Active Records (and hence service models), and we use these extensively when scripting.
For example we can discover more about the hardware of our virtual machine (VM) by following associations between the VM object (MiqAeServiceVmRedhat), and its Hardware and GuestDevice objects (MiqAeServiceHardware and MiqAeServiceGuestDevice), as follows:
hardware = $evm.root['vm'].hardware
hardware.guest_devices.each do |guest_device|
if guest_device.device_type == "ethernet"
nic_name = guest_device.device_name
end
endFortunately we don’t need to know anything about the Active Records or service models behind the scenes, we just magically follow the association. See [investigative-debugging] to find out what associations there are to follow.
Continuing our exploration of our RHEV Host object (MiqAeServiceHostRedhat), the associations available to this object are:
host.datacenter host.directories host.ems_cluster host.ems_events host.ems_folder host.ext_management_system host.files host.guest_applications host.hardware host.lans host.operating_system host.storages host.switches host.vms
We can enumerate an object’s associations using:
this_object.associations.each do |association|Most of the objects that we work with have useful methods defined that we can use, either in their own class or one of their parent superclasses. For example the methods available to call for our RHEV Host object (MiqAeServiceHostRedhat) are:
host.authentication_password host.authentication_userid host.credentials host.current_cpu_usage host.current_memory_headroom host.current_memory_usage host.custom_get host.custom_keys host.custom_set host.domain host.ems_custom_get host.ems_custom_keys host.ems_custom_set host.event_log_threshold? host.get_realtime_metric host.scan host.ssh_exec host.tagged_with? host.tags host.tag_assign
Enumerating a service model object’s methods is more challenging, because the actual object that we want to enumerate is running in the Automation Engine on the remote side of a dRuby call (see below), and all we have is the local DRb::DRbObject accessible from $evm. We can use method_missing, but we get returned the entire method list, which includes attribute names, virtual column names, association names, superclass methods, and so on.
this_object.method_missing(:class).instance_methodsThe Automation Engine runs in a ManageIQ worker thread, and it launches one of our automation scripts by spawning it as a child Ruby process. We can see this from the command line using ps to check the PID of the worker processes and its children:
\_ /var/www/miq/vmdb/lib/workers/bin/worker.rb | \_ /opt/rh/rh-ruby22/root/usr/bin/ruby <-- automation script running
An automation script runs in its own process space, but it must somehow access the service model objects that reside in the Automation Engine process. It does this using Distributed Ruby.
We can use rake evm:status to see which workers are running on a ManageIQ appliance:
vmdb bin/rake evm:status ... Worker Type | Status | -------------------------------------------------------------------+---------+ ManageIQ::Providers::Redhat::InfraManager::EventCatcher | started | ManageIQ::Providers::Redhat::InfraManager::MetricsCollectorWorker | started | ManageIQ::Providers::Redhat::InfraManager::MetricsCollectorWorker | started | ManageIQ::Providers::Redhat::InfraManager::RefreshWorker | started | MiqEmsMetricsProcessorWorker | started | MiqEmsMetricsProcessorWorker | started | MiqEventHandler | started | MiqGenericWorker | started | MiqGenericWorker | started | MiqPriorityWorker | started | MiqPriorityWorker | started | MiqReportingWorker | started | MiqReportingWorker | started | MiqScheduleWorker | started | MiqSmartProxyWorker | started | MiqSmartProxyWorker | started | MiqUiWorker | started | MiqWebServiceWorker | started |
Distributed Ruby (dRuby) is a distributed client-server object system that allows a client Ruby process to call methods on a Ruby object located in another (server) Ruby process. This can even be on another machine.
The object in the remote dRuby server process is locally represented in the dRuby client by an instance of a DRb::DRbObject object. In the case of an automation script, this object is our $evm variable.
The Automation Engine cleverly handles everything for us. When it runs our automation script, the Engine sets up the dRuby session automatically, and we access all of the service model objects seamlesssly via $evm in our script. Behind the scenes the dRuby library handles the TCP/IP socket communication with the dRuby server in the Automation Engine worker.
We gain insight into this if we examine some of these $evm objects using object_walker, for example:
$evm.root['user'] => #<MiqAeMethodService::MiqAeServiceUser:0x0000000c5431c8> \
(type: DRb::DRbObject, URI: druby://127.0.0.1:38842)
Although the use of dRuby mostly transparent to us, it can occasionally produce unexpected results. Perhaps we are hoping to find some useful user-related method that we can call on our user object, which we know we can access as $evm.root['user']. We might try to call a standard Ruby method such as:
$evm.root['user'].instance_methodsIf we were to do this we would actually get a list of the instance methods for the local DRb::DRbObject object, rather than the remote MiqAeServiceUser service model; probably not what we want.
When we get more adventurous in our scripting, we also occasionally get a DRb::DRbUnknown object returned to us, indicating that the class of the object is unknown in our dRuby client’s namespace.
This chapter has given us some good insight into the Rails active records that ManageIQ uses internally to represent our virtual infrastructure, and how these are made available to us as service model objects. We’ve also seen how these service model objects have four specific properties that we frequently make use of: attributes, virtual columns, associations and methods.
Masatoshi Seki: The dRuby Book