qos_capacity.adoc

Capacity-controller QoS Register Interface

Controllers, such as cache controllers, that support capacity allocation and usage monitoring provide a memory-mapped capacity-controller QoS register interface.

Table 1. Capacity-controller QoS register layout

Offset	Name	Size	Description	Optional?
0	cc_capabilities	8	Capabilities	No
8	cc_mon_ctl	8	Usage monitoring control	Yes
16	cc_mon_ctr_val	8	Monitoring counter value	Yes
24	cc_alloc_ctl	8	Capacity allocation control	Yes
32	cc_block_mask	M * 8	Capacity block mask	Yes

The size and offset in Capacity-controller QoS register layout are specified in bytes.

The size of the cc_block_mask register is determined by the NCBLKS field of the cc_capabilities register but is always a multiple of 8 bytes.

The reset value is 0 for the following registers fields.

• cc_mon_ctl - BUSY field

• cc_alloc_ctl - BUSY field

The reset value is UNSPECIFIED for all other registers and/or fields.

The capacity controllers at reset must allocate all available capacity to RCID value of 0. When the capacity controller supports capacity allocation per access-type, then all available capacity is shared by all the access-type for RCID=0. The capacity allocation for all other RCID values is UNSPECIFIED. The capacity controller behavior for handling a request with a non-zero RCID value before configuring the capacity controller with capacity allocation for that RCID is UNSPECIFIED.

Capacity-controller capabilities (cc_capabilities)

The cc_capabilities register is a read-only register that holds the capacity-controller capabilities.

Capacity-controller capabilities register fields

{reg: [
  {bits:  8, name: 'VER'},
  {bits: 16, name: 'NCBLKS'},
  {bits:  1, name: 'FRCID'},
  {bits: 39, name: 'WPRI'},
], config:{lanes: 4, hspace:1024}}

The VER field holds the version of the specification implemented by the capacity controller. The low nibble is used to hold the minor version of the specification and the upper nibble is used to hold the major version of the specification. For example, an implementation that supports version 1.0 of the specification reports 0x10.

The NCBLKS field holds the total number of allocatable capacity blocks in the controller. The capacity represented by an allocatable capacity block is UNSPECIFIED. The capacity controllers support allocating capacity in multiples of an allocatable capacity block.

Note	For example, a cache controller that supports capacity allocation by ways may report the number of ways as the number of allocatable capacity blocks.

If FRCID is 1, the controller supports an operation to flush and deallocate the capacity blocks occupied by an RCID.

Capacity usage monitoring control (cc_mon_ctl)

The cc_mon_ctl register is used to control monitoring of capacity usage by a MCID. When the controller does not support capacity usage monitoring the cc_mon_ctl register is read-only zero.

Capacity usage monitoring control (cc_mon_ctl)

{reg: [
  {bits:  5, name: 'OP'},
  {bits:  3, name: 'AT'},
  {bits: 12, name: 'MCID'},
  {bits:  8, name: 'EVT_ID'},
  {bits:  1, name: 'ATV'},
  {bits:  3, name: 'WPRI'},
  {bits:  7, name: 'STATUS'},
  {bits:  1, name: 'BUSY'},
  {bits: 24, name: 'WPRI'},
], config:{lanes: 2, hspace:1024}}

Capacity controllers that support capacity usage monitoring implement a usage monitoring counter for each supported MCID. The usage monitoring counter may be configured to count a monitoring event. When an event matching the event configured for the MCID occurs then the monitoring counter is updated. The event matching may optionally be filtered by the access-type.

The OP, AT, ATV, `MCID, and EVT_ID fields of the register are WARL fields.

The OP field is used to instruct the controller to perform an operation listed in Capacity usage monitoring operations (OP). The supported operations of the OP field include configuring an event to count in the monitoring counter and obtaining a snapshot of the counter value.

The EVT_ID field is used to program the identifier of the event to count in the monitoring counter selected by MCID. The AT field is used to program the access-type to count, and its validity is indicated by the ATV field. When ATV is 0, the counter counts requests with all access-types, and the AT value is ignored.

When the EVT_ID for a MCID is programmed with a non-zero and legal value, the counter is reset to 0 and starts counting matching events for requests with the matching MCID and AT (if ATV is 1). However, if the EVT_ID is configured as 0, the counter retains its current value but stops counting.

A controller that does not support monitoring by access-type can hardwire the ATV and the AT fields to 0, indicating that the counter counts requests with all access-types.

Table 2. Capacity usage monitoring operations (OP)

Operation	Encoding	Description
—	0	Reserved for future standard use.
CONFIG_EVENT	1	Configure the counter selected by MCID to count the event selected by EVT_ID, AT, and ATV. The EVT_ID encodings are listed in Capacity usage monitoring event ID (EVT_ID).
READ_COUNTER	2	Snapshot the value of the counter selected by MCID into cc_mon_ctr_val register. The EVT_ID, AT, and ATV fields are not used by this operation.
—	3-23	Reserved for future standard use.
—	24-31	Designated for custom use.

Table 3. Capacity usage monitoring event ID (EVT_ID)

Event ID	Encoding	Description
None	0	Counter does not count and retains its value.
Occupancy	1	Counter is incremented by 1 when a request with a matching MCID and AT allocates a unit of capacity. The counter is decremented by 1 when a unit of capacity is de-allocated.
—	2-127	Reserved for future standard use.
—	128-256	Designated for custom use.

When the cc_mon_ctl register is written, the controller may need to perform several actions that may not complete synchronously with the write. A write to the cc_mon_ctl sets the read-only BUSY bit to 1 indicating the controller is performing the requested operation. When the BUSY bit reads 0, the operation is complete, and the read-only STATUS field provides a status value (see cc_mon_ctl.STATUS field encodings for details). Written values to the BUSY and the STATUS fields are ignored. An implementation that can complete the operation synchronously with the write may hardwire the BUSY bit to 0. The state of the BUSY bit, when not hardwired to 0, shall only change in response to a write to the register. The STATUS field remains valid until a subsequent write to the cc_mon_ctl register.

Table 4. cc_mon_ctl.STATUS field encodings

STATUS	Description
0	Reserved
1	The operation was successfully completed.
2	An invalid operation (OP) was requested.
3	An operation was requested for an invalid MCID.
4	An operation was requested for an invalid EVT_ID.
5	An operation was requested for an invalid AT.
6-63	Reserved for future standard use.
64-127	Designated for custom use.

When the BUSY bit is set to 1, the behavior of writes to the cc_mon_ctl is UNSPECIFIED. Some implementations may ignore the second write, while others may perform the operation determined by the second write. To ensure proper operation, software must first verify that the BUSY bit is 0 before writing the cc_mon_ctl register.

Capacity usage monitoring counter value (cc_mon_ctr_val)

The cc_mon_ctr_val is a read-only register that holds a snapshot of the counter selected by the READ_COUNTER operation. When the controller does not support capacity usage monitoring, the cc_mon_ctr_val register always reads as zero.

Capacity usage monitoring counter value (cc_mon_ctr_val)

{reg: [
  {bits:  63, name: 'CTR'},
  {bits:   1, name: 'INV'},
], config:{lanes: 2, hspace:1024}}

The counter is valid if the INV field is 0. The counter may be marked INV if it underflows or the controller, for UNSPECIFIED reasons determine the count to be not valid. The counters marked INV may become valid in future.

Note

A counter may underflow when capacity is de-allocated following a reset of the counter to 0. This may be due to the MCID being reallocated to a new workload while the capacity controller still holds capacity allocated by the workload to which the MCID was previously allocated. The counter value should typically stabilize to reflect the capacity usage of the new workload after the workload has executed for a short duration following the counter reset.

Note

Some implementations may not store the MCID of the request that caused the capacity to be allocated with every unit of capacity in the controller to optimize on the storage overheads. Such controllers may in turn rely on statistical sampling to report the capacity usage by tagging only a subset of the capacity units. Set-sampling is a technique commonly used in caches to estimate the cache occupancy with a relatively small sample size. The basic idea behind set-sampling is to select a subset of the cache sets and monitor only those sets. By keeping track of the hits and misses in the monitored sets, it is possible to estimate the overall cache occupancy with a high degree of accuracy. The size of the subset needed to obtain accurate estimates depends on various factors, such as the size of the cache, the cache access patterns, and the desired accuracy level. Research cite:[SSAMPLE] has shown that set-sampling can provide statistically accurate estimates with a relatively small sample size, such as 10% or less, depending on the cache properties and sampling technique used. When the controller has not observed enough samples to provide an accurate value in the monitoring counter, it may report the counter as being INV until more accurate measurements are available. This helps to prevent inaccurate or misleading data from being used in capacity planning or other decision-making processes.

Capacity allocation control (cc_alloc_ctl)

The cc_alloc_ctl register is used to configure allocation of capacity to an RCID per access-type (AT). The RCID and AT fields in this register are WARL. If a controller does not support capacity allocation then this register is read-only zero. If the controller does not support capacity allocation per access-type then the AT field is read-only zero.

Capacity allocation control (cc_alloc_ctl)

{reg: [
  {bits:  5, name: 'OP'},
  {bits:  3, name: 'AT'},
  {bits: 12, name: 'RCID'},
  {bits: 12, name: 'WPRI'},
  {bits:  7, name: 'STATUS'},
  {bits:  1, name: 'BUSY'},
  {bits: 24, name: 'WPRI'},
], config:{lanes: 2, hspace:1024}}

The OP, AT, and RCID are WARL fields.

The OP field used to instruct the capacity controller to perform an operation listed in Capacity allocation operations (OP). Some operations require specifying the capacity blocks to operate on. The capacity blocks, in the form of a bitmask, for such operations are specified in the cc_block_mask register. To request operations that need a capacity block mask to be specified, software must first program the cc_block_mask register and then request the operation using the cc_alloc_ctl register.

Table 5. Capacity allocation operations (OP)

Operation	Encoding	Description
—	0	Reserved for future standard use.
CONFIG_LIMIT	1	Configure a capacity allocation for requests by RCID and of access-type AT. The capacity allocation is specified in the cc_block_mask register.
READ_LIMIT	2	Read back the previously configured capacity allocation for requests by RCID and of access-type AT. The configured allocation is returned as a bit-mask in the cc_block_mask register on successful completion of the operation.
FLUSH_RCID	3	Deallocate the capacity used by the specified RCID and access-type AT. This operation is supported if the capabilities.FRCID bit is 1.+ The cc_block_mask register is not used for this operation.+ The configured capacity allocation is not changed by this operation.
—	4-23	Reserved for future standard use.
—	24-31	Designated for custom use.

Capacity controllers enumerate the allocatable capacity blocks in the NCBLKS field of the cc_capabilities register. The cc_block_mask register is programmed with a bit-mask where each bit represents a capacity block for the operation.

A capacity allocation must be configured for each supported access-type by the controller. An implementation that does not support capacity allocation per access-type may hardwire the AT field to 0 and associate use the same capacity allocation configuration for requests with all access-types. When capacity allocation per access-type is supported, identical limits may be configured for two or more access-types if different capacity allocation per access-type is not required. If capacity is not allocated for each access-type supported by the controller, the behavior is UNSPECIFIED.

Note

A cache controller that supports capacity allocation indicates the number of allocatable capacity blocks in cc_capabilities.NCBLKS field. For example, let’s consider a cache with NCBLKS=8. In this example, the RCID=5 has been allocated capacity blocks numbered 0 and 1 for requests with access-type AT=0, and has been allocated capacity blocks numbered 2 for requests with access-type AT=1. The RCID=3 in this example has been allocated capacity blocks numbered 3 and 4 for both AT=0 and AT=1 access-types as separate capacity allocation by access-type is not required for this workload. Further in this example, the RCID=6 has been configured with the same capacity block allocations as RCID=3. This implies that they share a common capacity allocation in this cache but may have been associated with different RCID to allow differentiated treatment in another capacity and/or bandwidth controller. 7 6 5 4 3 2 1 0 RCID=3, AT=0 0 0 0 1 1 0 0 0 RCID=3, AT=1 0 0 0 1 1 0 0 0 RCID=5, AT=0 0 0 0 0 0 0 1 1 RCID=5, AT=1 0 0 0 0 0 1 0 0 RCID=6, AT=0 0 0 0 1 1 0 0 0 RCID=6, AT=1 0 0 0 1 1 0 0 0

The FLUSH_RCID operation may incur a long latency to complete. New requests to the controller by the RCID being flushed are allowed. Additionally, the controller is allowed to deallocate capacity that was allocated after the operation was initiated.

Note

For cache controllers, the FLUSH_RCID operation may perfom an operation similar to that performed by the RISC-V CBO.FLUSH instruction on each cache block that is part of the allocation configured for the RCID. The FLUSH_RCID operation can be used as part of reclaiming a previously allocated RCID and associating it with a new workload. When such a reallocation is performed, the capacity controllers may have capacity allocated by the old workload and thus for a short warmup duration the capacity controller may be enforcing capacity allocation limits that reflect the usage by the old workload. Such warmup durations are typically not statistically significant, but if that is not desired, then the FLUSH_RCID operation can be used to flush and evict capacity allocated by the old workload.

When the cc_alloc_ctl register is written, the controller may need to perform several actions that may not complete synchronously with the write. A write to the cc_alloc_ctl sets the read-only BUSY bit to 1 indicating the controller is performing the requested operation. When the BUSY bit reads 0, the operation is complete, and the read-only STATUS field provides a status value (cc_alloc_ctl.STATUS field encodings) of the requested operation. Values written to the BUSY and the STATUS fields are always ignored. An implementation that can complete the operation synchronously with the write may hardwire the BUSY bit to 0. The state of the BUSY bit, when not hardwired to 0, shall only change in response to a write to the register. The STATUS field remains valid until a subsequent write to the cc_alloc_ctl register.

Table 6. cc_alloc_ctl.STATUS field encodings

STATUS	Description
0	Reserved
1	The operation was successfully completed.
2	An invalid or unsupported operation (OP) requested.
3	An operation was requested for an invalid RCID.
4	An operation was requested for an invalid AT.
5	An invalid capacity block mask was specified.
6-63	Reserved for future standard use.
64-127	Designated for custom use.

When the BUSY bit is set to 1, the behavior of writes to the cc_alloc_ctl register or to the cc_block_mask register is UNSPECIFIED. Some implementations may ignore the second write and others may perform the operation determined by the second write. To ensure proper operation, software must verify that BUSY bit is 0 before writing the cc_alloc_ctl register or the cc_block_mask register.

Capacity block mask (cc_block_mask)

The cc_block_mask is a WARL register. If the controller does not support capacity allocation i.e. NCBLKS is 0, then this register is read-only 0.

The register has NCBLKS bits each corresponding to one allocatable capacity block in the controller. The width of this register is variable but always a multiple of 64 bits. The bitmap width in bits (BMW) is determined in equation (1). The division operation in this equation is an integer division.

\[\begin{equation} BMW = \lfloor{\frac{NCBLKS + 63}{64}}\rfloor \times 64 \end{equation}\]

Bits NCBLKS-1:0 are read-write, and the bits BMW-1:NCBLKS are read-only and have a value of 0.

The process of configuring capacity allocation for an RCID and AT begins by programming the cc_block_mask register with a bit-mask that identifies the capacity blocks to be allocated. Next, the cc_alloc_ctl register is written to request a CONFIG_LIMIT operation for the RCID and AT. Once a capacity allocation limit has been established, a request may be allocated capacity in the capacity blocks allocated to the RCID and AT associated with the request. It is important to note that at least one capacity block must be allocated using cc_block_mask when allocating capacity, or else the operation will fail with STATUS=5. Overlapping capacity block masks among RCID and/or AT are allowed to be configured.

Note

A set-associative cache controller that supports capacity allocation by ways can advertise NCBLKS as the number of ways per set in the cache. To Allocate capacity in such a cache for an RCID and AT, a subset of ways must be selected and mask of the selected ways must be programmed in cc_block_mask when requesting the CONFIG_LIMIT operation.

To read the capacity allocation limit for an RCID and AT, the controller provides the READ_LIMIT operation which can be requested by writing to the cc_alloc_ctl register. Upon successful completion of the operation, the cc_block_mask register holds the configured capacity allocation limit.