Question about Layout broadcasting and codomain.

[cloudhan]

I will use array as a substitute of Tensor, think of it as np.array

1. Is cute::Layout algebra compatible with array broadcasting? Say

   a = np.ones([4,3,2])
   b = np.ones([3,2])
   # then b can be treated as an array of shape=(4,3,2) and stride=(0,2,1)
   # then the inner product of coord (of broadcasted array shape) and stride will be valid array offset.
   a + b  # is valid due to broadcasting

   then is Layout (4,3,2):(0,2,1) in cute::Layout algebra? This will be critical if we want to unifiy elementwise ops under cute.

2. as per

   cutlass/media/docs/cute/02_layout_operations.md

   Line 186 in 7c04f95

   *Cosize*: Size of the function's codomain (not necessarily the range); for a layout A, A(size(A) - 1) + 1. (Here, we use size(A) - 1 as a 1-D logical coordinate input.)

   codomain (not necessarily the range)

   Then in what cases, the codomain is not the range of the function (the function here is also known as Layout)?

[ccecka]

1. Yes, stride-0 is supported in both the Layout and the algebra. It is very useful and can be used to "broadcast" a vector of data into a matrix like you suggest.
2. I believe that's referring to something like Layout (2,2):(1,4), where the range of the function is the set {0,1,4,5}, the codomain is the set Z6, and the cosize of the function is 6. In general, the range is the retraction of the codomain.