Skip to content

Latest commit

 

History

History
470 lines (404 loc) · 20.1 KB

qos_bandwidth.adoc

File metadata and controls

470 lines (404 loc) · 20.1 KB

Bandwidth-controller QoS Register Interface

Controllers, such as memory controllers, that support bandwidth allocation and bandwidth usage monitoring provide a memory-mapped bandwidth-controller QoS register interface.

Table 1. Bandwidth-controller QoS register layout
Offset Name Size Description Optional?

0

bc_capabilities

8

Capabilities

No

8

bc_mon_ctl

8

Usage monitoring control

Yes

16

bc_mon_ctr_val

8

Monitoring counter value

Yes

24

bc_alloc_ctl

8

Bandwidth allocation control

Yes

32

bc_bw_alloc

8

Bandwidth allocation

Yes

The reset value is 0 for the following registers fields.

  • bc_mon_ctl - BUSY field

  • bc_alloc_ctl - BUSY field

The reset value is UNSPECIFIED for all other registers and/or fields.

The bandwidth controllers at reset must allocate all available bandwidth to RCID value of 0. When the bandwidth controller supports bandwidth allocation per access-type, the access-type value of 0 of RCID=0 is allocated all available bandwidth, while all other access-types associated with that RCID share the bandwidth allocation with AT=0. The bandwidth allocation for all other RCID values is UNSPECIFIED. The bandwidth controller behavior in handling a request with a non-zero RCID value before configuring the bandwidth controller with bandwidth allocation for that RCID is also UNSPECIFIED.

Capabilities (bc_capabilities)

The bc_capabilities register is a read-only register that holds the bandwidth-controller capabilities.

Capabilities register fields
{reg: [
  {bits:  8, name: 'VER'},
  {bits: 16, name: 'NBWBLKS'},
  {bits:  8, name: 'WPRI'},
  {bits: 16, name: 'MRBWB'},
  {bits: 16, name: 'WPRI'},
], config:{lanes: 2, hspace:1024}}

The VER field holds the version of the specification implemented by the bandwidth controller. The low nibble is used to hold the minor version of the specification and the upper nibble is used to hold the major version of the specification. For example, an implementation that supports version 1.0 of the specification reports 0x10.

The NBWBLKS field holds the total number of available bandwidth blocks in the controller. The bandwidth represented by each bandwidth block is UNSPECIFIED. The bandwidth controller supports reserving bandwidth in multiples of a bandwidth block, which enables proportional allocation of bandwidth.

Bandwidth controllers may limit the maximum bandwidth that may be reserved to a value smaller than NBWBLKS. The MRBWB field reports the maximum number of bandwidth blocks that can be reserved.

Note

The bandwidth controller needs to meter the bandwidth usage by a workload to determine if it is exceeding its allocations and, if necessary, take necessary measures to throttle the workload’s bandwidth usage. Therefore, the instantaneous bandwidth used by a workload may either exceed or fall short of the configured allocation. QoS capabilities are statistical in nature and are typically designed to enforce the configured bandwidth over larger time windows. By not allowing all available bandwidth blocks to be reserved for allocation, the bandwidth controller can handle such transient inaccuracies.

Bandwidth usage monitoring control (bc_mon_ctl)

The bc_mon_ctl register is used to control monitoring of bandwidth usage by a MCID. When the controller does not support bandwidth usage monitoring, the bc_mon_ctl register is read-only and hardwired to zero.

Bandwidth usage monitoring control (bc_mon_ctl)
{reg: [
  {bits:  5, name: 'OP'},
  {bits:  3, name: 'AT'},
  {bits: 12, name: 'MCID'},
  {bits:  8, name: 'EVT_ID'},
  {bits:  1, name: 'ATV'},
  {bits:  3, name: 'WPRI'},
  {bits:  7, name: 'STATUS'},
  {bits:  1, name: 'BUSY'},
  {bits: 24, name: 'WPRI'},
], config:{lanes: 2, hspace:1024}}

Bandwidth controllers that support bandwidth usage monitoring implement a usage monitoring counter for each supported MCID. The usage monitoring counter may be configured to count a monitoring event. When an event matching the event configured for the MCID occurs then the monitoring counter is updated. The event matching may optionally be filtered by the access-type. The monitoring counter for bandwidth usage counts the number of bytes transferred by requests matching the monitoring event as the requests go past the monitoring point.

The OP, AT, MCID, and EVT_ID fields of the register are WARL fields.

The OP field is used to instruct the controller to perform an operation listed in Usage monitoring operations (OP). The supported operations of the OP field include configuring an event to count in the monitoring counter and obtaining a snapshot of the counter value.

The EVT_ID field is used to program the identifier of the event to count in the monitoring counter selected by MCID. The AT field is used to program the access-type to count, and its validity is indicated by the ATV field. When ATV is 0, the counter counts requests with all access-types, and the AT value is ignored.

When the EVT_ID for a MCID is programmed with a non-zero and legal value, the counter is reset to 0 and starts counting matching events for requests with the matching MCID and AT (if ATV is 1). However, if the EVT_ID is configured as 0, the counter retains its current value but stops counting.

A controller that does not support monitoring by access-type can hardwire the ATV and the AT fields to 0, indicating that the counter counts requests with all access-types.

Table 2. Usage monitoring operations (OP)
Operation Encoding Description

 — 

0

Reserved for future standard use.

CONFIG_EVENT

1

Configure the counter selected by MCID to count the event selected by EVT_ID, AT, and ATV. The EVT_ID encodings are listed in Bandwidth monitoring event ID (EVT_ID).

READ_COUNTER

2

Snapshot the value of the counter selected by MCID into bc_mon_ctr_val register. The EVT_ID, AT, and ATV fields are not used by this operation.

 — 

3-23

Reserved for future standard use.

 — 

24-31

Designated for custom use.

Table 3. Bandwidth monitoring event ID (EVT_ID)
Event ID Encoding Description

None

0

Counter does not count and retains its value.

Total Read and Write byte count

1

Counter is incremented by the number of bytes transferred by a matching read or write request as the requests go past the monitor.

Total Read byte count

2

Counter is incremented by the number of bytes transferred by a matching read request as the requests go past the monitor.

Total Write byte count

3

Counter is incremented by the number of bytes transferred by a matching write request as the requests go past the monitor.

 — 

4-127

Reserved for future standard use.

 — 

128-256

Designated for custom use.

When the bc_mon_ctl register is written, the controller may need to perform several actions that may not complete synchronously with the write. A write to the bc_mon_ctl sets the read-only BUSY bit to 1 indicating the controller is performing the requested operation. When the BUSY bit reads 0, the operation is complete, and the read-only STATUS field provides a status value (see bc_mon_ctl.STATUS field encodings for details). Written values to the BUSY and the STATUS fields are ignored. An implementation that can complete the operation synchronously with the write may hardwire the BUSY bit to 0. The state of the BUSY bit, when not hardwired to 0, shall only change in response to a write to the register. The STATUS field remains valid until a subsequent write to the bc_mon_ctl register.

Table 4. bc_mon_ctl.STATUS field encodings
STATUS Description

0

Reserved

1

The operation was successfully completed.

2

An invalid operation (OP) was requested.

3

An operation was requested for an invalid MCID.

4

An operation was requested for an invalid EVT_ID.

5

An operation was requested for an invalid AT.

6-63

Reserved for future standard use.

64-127

Designated for custom use.

When the BUSY bit is set to 1, the behavior of writes to the bc_mon_ctl is UNSPECIFIED. Some implementations may ignore the second write, while others may perform the operation determined by the second write. To ensure proper operation, software must first verify that the BUSY bit is 0 before writing the bc_mon_ctl register.

Bandwidth monitoring counter value (bc_mon_ctr_val)

The bc_mon_ctr_val is a read-only register that holds a snapshot of the counter selected by a READ_COUNTER operation. When the controller does not support bandwidth usage monitoring, the bc_mon_ctr_val register always reads as zero.

Bandwidth monitoring counter value (bc_mon_ctr_val)
{reg: [
  {bits:  62, name: 'CTR'},
  {bits:   1, name: 'INV'},
  {bits:   1, name: 'OVF'},
], config:{lanes: 2, hspace:1024}}

The counter is valid if the INV field is 0. The counter may be marked INV if, for UNSPECIFIED reasons, the controllerdetermine the count to be not valid. Such counters may become valid in the future. Additionally, if an unsigned integer overflow of the counter occurs, then the OVF bit is set to 1.

Note

A counter may be marked as INV if the controller has not been able to establish an accurate counter value for the monitored event.

The counter provides the number of bytes transferred by requests matching the EVT_ID as they go past the monitoring point. A bandwidth value may be determined by reading the byte count value at two instances of time T1 and T2, see equation (1). If the value of the counter at time T1 was B1, and at time T2 is B2, then the bandwidth can be calculated as follows. The frequency of the time source is represented by \(T_{freq}\).

\[\begin{equation} Bandwidth = T_{freq} \times \frac{ B2 - B1 }{T2 - T1} \end{equation}\]

The width of the counter is UNSPECIFIED.

Note

While the width of the counter is UNSPECIFIED, it is recommended to be wide enough to prevent more than one overflow per sample when the sampling frequency is 1 Hz.

If an overflow was detected then software may discard that sample and reset the counter and overflow indication by reprogramming the event using CONFIG_EVENT operation.

Bandwidth allocation control (bc_alloc_ctl)

The bc_alloc_ctl register is used to control the allocation of bandwidth to an RCID per AT. If a controller does not support bandwidth allocation, then the register is read-only zero. If the controller does not support bandwidth allocation per access-type, then the AT field is also read-only zero.

Bandwidth allocation control (bc_alloc_ctl)
{reg: [
  {bits:  5, name: 'OP'},
  {bits:  3, name: 'AT'},
  {bits: 12, name: 'RCID'},
  {bits: 12, name: 'WPRI'},
  {bits:  7, name: 'STATUS'},
  {bits:  1, name: 'BUSY'},
  {bits: 24, name: 'WPRI'},
], config:{lanes: 2, hspace:1024}}

The OP field instructs the bandwidth controller to perform an operation listed in Bandwidth allocation operations (OP). The bc_alloc_ctl register is used in conjunction with the bc_bw_alloc register to perform bandwidth allocation operations. If the requested operation uses the operands configured in bc_bw_alloc, software must first program the bc_bw_alloc register with the operands for the operation before requesting the operation.

Table 5. Bandwidth allocation operations (OP)
Operation Encoding Description

 — 

0

Reserved for future standard use.

CONFIG_LIMIT

1

Establishes reserved bandwidth allocation for requests by RCID and of access-type AT. The bandwidth allocation is specified in bc_bw_alloc register.

READ_LIMIT

2

Reads back the previously configured bandwidth allocation for requests by RCID and of access-type AT. The current configured allocation is written to bc_bw_alloc register on completion of the operation.

 — 

3-23

Reserved for future standard use.

 — 

24-31

Designated for custom use.

A bandwidth allocation must be configured for each access-type supported by the controller. When differentiated bandwidth allocation based on access-type is not required, one of the access-types may be designated to hold a default bandwidth allocation, and the other access-types can be configured to share the allocation with the default access-type. If bandwidth is not allocated for each access-type supported by the controller, the behavior is UNSPECIFIED.

When the bc_alloc_ctl register is written, the controller may need to perform several actions that may not complete synchronously with the write. A write to the bc_alloc_ctl sets the read-only BUSY bit to 1 indicating the controller is performing the requested operation. When the BUSY bit reads 0, the operation is complete, and the read-only STATUS field provides a status value (see bc_alloc_ctl.STATUS field encodings for details). Written values to the BUSY and the STATUS fields are ignored. An implementation that can complete the operation synchronously with the write may hardwire the BUSY bit to 0. The state of the BUSY bit, when not hardwired to 0, shall only change in response to a write to the register. The STATUS field remains valid until a subsequent write to the bc_alloc_ctl register.

Table 6. bc_alloc_ctl.STATUS field encodings
STATUS Description

0

Reserved

1

The operation was successfully completed.

2

An invalid operation (OP) was requested.

3

An operation was requested for an invalid RCID.

4

An operation was requested for an invalid AT.

5

An invalid or unsupported reserved bandwidth block was requested.

6-63

Reserved for future standard use.

64-127

Designated for custom use.

Bandwidth allocation configuration (bc_bw_alloc)

The bc_bw_alloc is used to program reserved bandwidth blocks (Rbwb) for an RCID for requests of access-type AT using the CONFIG_LIMIT operation. If a controller does not support bandwidth allocation, then the bc_bw_alloc register is read-only zero.

The bc_bw_alloc holds the previously configured reserved bandwidth blocks for an RCID and AT on successful completion of the READ_LIMIT operation.

Bandwidth is allocated in multiples of bandwidth blocks, and the value in Rbwb must be at least 1 and must not exceed MRBWB. Otherwise, the CONFIG_LIMIT operation fails with STATUS=5. Additionally, the sum of Rbwb allocated across all RCID`s must not exceed `MRBWB, or the CONFIG_LIMIT operation fails with STATUS=5.

Bandwidth allocation configuration (bc_bw_alloc)
{reg: [
  {bits: 16, name: 'Rbwb'},
  {bits:  4, name: 'WPRI'},
  {bits:  8, name: 'Mweight'},
  {bits:  3, name: 'sharedAT'},
  {bits:  1, name: 'useShared'},
  {bits: 32, name: 'WPRI'},
], config:{lanes: 4, hspace:1024}}

The Rbwb, Mweight, sharedAT, and useShared are all WARL fields.

Bandwidth allocation is typically enforced by the bandwidth controller over finite accounting windows. The process involves measuring the bandwidth consumption over an accounting window and using the measured bandwidth to determine if an RCID is exceeding its bandwidth allocations for each access-types. The specifics of how the accounting window is implemented are UNSPECIFIED, but is expected to provide a statistically accurate control of the bandwidth usage over a few accounting intervals.

The Rbwb represents the bandwidth that is made available to a RCID for requests matching AT, even when all other RCID are using their full allocation of bandwidth.

If there is non-reserved or unused bandwidth available in an accounting interval, RCID`s may compete for additional bandwidth. The non-reserved or unused bandwidth is proportionately shared among the competing RCIDs using the configured `Mweight parameter, which is a number between 0 and 255. A larger weight implies a greater fraction of the bandwidth. A weight of 0 implies that the configured limit is a hard limit, and the use of unused or non-reserved bandwidth is not allowed.

The sharing of non-reserved bandwidth is not differentiated by access-type. Therefore, the Mweight parameter must be programmed identically for all access-types. If this parameter is programmed differently for each access-type, then the controller may use the parameter configured for any of the access-types, but the behavior is otherwise well defined.

When the Mweight parameter is not set to 0, the amount of unused bandwidth allocated to RCID=x during contention with another RCID that is also permitted to use unused bandwidth is determined by dividing the Mweight of RCID=x by the sum of the Mweight of all other contending RCID`s. This ratio `P is determined as in equation (2).

\[\begin{equation} P = \frac{Mweight_{x}}{\sum_{r=1}^{r=n} Mweight_{r}} \end{equation}\]
Note

The bandwidth enforcement is typically work-conserving, meaning that it allows unused bandwidth to be used by requestors enabled to use it even if they have consumed their Rbwb.

When contending for unused bandwidth, the weighted share is typically computed among the `RCID`s that are actively generating requests in that accounting interval and have a non-zero weight programmed.

If unique bandwidth allocation is not required for an access-type, then the useShared parameter may be set to 1 for a CONFIG_LIMIT operation. When useShared is set to 1, the sharedAT field specifies the access-type with which the bandwidth allocation is shared by the access-type in bc_alloc_ctl.AT. In this case, the Rbwb and Mweight fields are ignored, and the configurations of the access-type in sharedAT are applied. If the access-type specified by sharedAT does not have unique bandwidth allocation, meaning that it has not been configured with useShared=0, then the behavior is UNSPECIFIED.

The useShared and sharedAT fields are reserved if the bandwidth controller does not support bandwidth allocation per access-type.

Note

When unique bandwidth allocation for an access-type is not required then one or more access-types may be configured with a shared bandwidth allocation. For example, consider a bandwidth controller that supports 3 access-types. The access-type 0 and 1 of RCID 3 are configured with unique bandwidth allocations and the access-type 2 is configured to share bandwidth allocation with access-type 1. The example configuration is illustrated as follows:

Rbwb Mweight useShared sharedAT

RCID=3, AT=0

100

16

0

N/A

RCID=3, AT=1

50

16

0

N/A

RCID=3, AT=2

N/A

N/A

1

1