BH prep

[pgkeller]

This is a list of todos for BH.
Must to run anything:

Phase1 for BH bring up - target 5/2

@abhullar-tt

• Update to use latest UMD #8453 bump umd version up to latest
• Enable ARCH_NAME=Blackhole to compile in Metal #8530 ARCH_NAME = blackhole
• Add Blackhole soc desc and core desc yamls #8560 BH needs SOC Descriptor (two parts: UMD part which is common + Metal-specific part? )

Reem

• Enable jit compile for BH #8735 compile kernels for BH, including new llk paths (In-progress - PR posted)

Almeet/David

• [Blackhole bringup] Disable L1 mem cache #8702 Disable L1 memory cache / coalescing
• [Blackhole bringup] Writing GO signal in stream registers #8703 --> opting to keeping GO signals in L1 (will need memory fences)
• [Blackhole bringup] Remove IRAM specific paths for NCRISC  #8705 remove NCRISC IRAM copy, halt/resume path - make work like trisc
• [Blackhole bringup] Handle difference sequence of deaserting ncrisc/trisc* on BH #8985
• [Blackhole bringup] Wider addresses in NOC cmd buffer programming sequence #8704 NOC command buffer programming sequence has changed to support wider addresses - noc_nonblocking_api
• [Blackhole bringup] Account for 64B DRAM read alignment and 16B DRAM write alignment #8706 NOC requires 64 byte alignment for DRAM reads. 16 byte writes are ok (need to fix on metal side)
• Bring up watcher (needs memory fence, handle BH NOC alignment) --> memory fences added when l1 cache is enabled
• Enable BH L1 cache #10865 Memory fences for RISCV shared access to L1 (e.g., GO messages) (L1 data cache and write coalescing)
• #12187: Enabling cmd buffer fifo on bh #12475
• [Blackhole bringup] Investigate workaround for ETH iCache flush #8707 ETH need iCache flush workaround
• [Blackhole xfunc] Support harvesting columns #8741
• Explore using interrupts on BH #16650 Interrupt support (on eth only on risc0)
• Non-rectangular multicast support (L shaped and strided)
• NoC traffic counters Enable use of NoC traffic counters for debug support on BH #16649
• Validate whether BH issuing 4 tensix instructions at once is default enabled #16651 Exercise feature where up to 4 tensix instructions can be issued at once (does this come for "free"?)
• [Blackhole] Enable accessing riscv on DRAM cores #10974 1x riscv on DRAM
• 2x riscv on Eth
• Multicast to ethernet cores --> decided to not do this (might revisit)
• Use 16 deep CMD buffer FIFOs performantly (convo with Djordje about programming NOC on BH) --> will not be enabling this feature based on NoC team feedback because of HW bugs

---

OLD NOTES BELOW:

Infra Flow - Versim scramble

1. bring up versim in gitlab
2. Getting stuff to build on public github
3. fork to gitlab and test on versim

Development flow (MVP: metal running slow dispatch with a few ops) -

1. Do the build stuff (make blackhole arch available and ability to build), fork into gitlab, validate versim submodule build.
2. bring up runtime with slow dispatch and do metal "hello-world" in the github
3. bring up simple ops with simple config (single core, native numeric) in the github
4. bring up less simple ops in the github
5. bring up key features/tool (watcher) + bring in key fixes (64 byte alignment) in the github

note: if we can staff (2) - (5) pretty soon, you should try to test on versim; if not, we should do it on the cards.
note: this flow will give us versim as a backup platform in case things don't work on the cards - but development and testing on both side (github/gitlab) is very cumbersome.

TODO:

• Need a runtime guy to bring up runtime and address the runtime related items.
• Need to define simple operators configuration

Phase2 for BH 30-day milestone & Open Source BH SW - target 5/17
Metal Goal - Single Tensix OP
[ ] Versim on CI
[ ] MatMul workload to stress-test single Tensix core

Phase3 for BH 60-day milestone - target 6/17
Metal goal - Multi-Tensix OP
[ ] MatMul workload to stress-test Multi Tensix cores
[ ]

** To be prioritized --> **
[ ] ? NOC/tensix shared access (need to enumerate)
[ ] Eth IRAM? TBD if BH has IRAM on ethernet. If true - need changes for Eth support

Performance changes/new features (not required to run):
[ ] NOC has a RISC-NOC command fifo which allows more non-blocking transactions in flight (legacy interface still works)

• NOC non-rectangular multicast
• how to use interrupts w/ metallium
• risc vector units

SFPU/I Optimizations / New Features:

• scoreboarding, can remove NOPs
• use new insns: arcip, aexp, conditionals
• explore RISC access to dst_reg and if/how to expose

Debug/analysis:

• NOC traffic counters to help find network congestion

[aliuTT]

For performance changes:

• using both ethernet risc in bi directional dataflow. Would require some stress testing to make sure the eth hardware works as expected.
• try dram risc

[mo-tenstorrent]

At some point I overheard BH has interrupts support, at least on BRISC, if I remember correctly. Should confirm its functionality.

[DrJessop]

"64B-alignment for reads" Just wanted to make sure not a typo, since in GS/WH it's 32B-alignment.

[DrJessop]

Also, I recall there being some issues with there being a relatively small NOC max packet size of 8KB. Wondering if BH will have the same limitation.

[pgkeller]

"64B-alignment for reads" Just wanted to make sure not a typo, since in GS/WH it's 32B-alignment.

not a typo, needs to change

[tt-rkim]

Some risks I see from infra / CI side:

• Versim cannot run on cloud network / metal CI unfortunately... we need some serious rethinking around CI / dev for this
• Similarly, this means versim-related source files cannot be checked into metal GH repo
• Unknown timelines from syseng for HW delivery - both non-ethernet and ethernet
• Fuzz testing our infra scripts + knowledge + syseng tools on BM BH systems
• Availability of BH in cloud

[TT-billteng]

perhaps we need to fork TTMetal on gitlab side and make necessary modifications

[jliangTT]

Versim cannot run on cloud network / metal CI unfortunately... we need some serious rethinking around CI / dev for this

my 2c: versim as a stop-gap to prefetch some the software development to build/bring up our sw stack and some unit test until hardware comes back. Once the hardware comes back, testing / development will be brought up on hardware and versim should be less relevant. As such, CI on versim may be throw away work.

[jliangTT]

per @rtawfik01 earlier on the current coverage on versim for Buda -

Unary datacopy -> start with single tile, single core, bfloat16, then test more combinations i.e more tiles, different dataformats, etc
Unary SFPU ops
Eltwise binary ops
Reduce ops
Matrix multiply/Convs
Here we can start more testing combination ops/graphs -> layernorms, softmaxes, feedforwards, etc

With buda, we tested all kernels, but we started with the list above
And always try every op with the simplest scenario (bfloat 16, single core, single tile), then add combinations as the tests pass

Also, Reem is getting the llk submodule ready and we should let her know when we complete bulding metal with the current blackhole arch compile through metal stack.

[abhullar-tt]

TODOs as of 31/07/2024

• [Blackhole xfunc] Get faster BH machines #10976
• Run microbenchmarks to make sure they are functional on current BH dev machines
• Run microbenchmarks on faster BHs to get accurate BW measurements
• Run MMs and Convs that saturate BW
• Op/LLK parity

FYI @davorchap

[abhullar-tt]

Most of the items from the initial list have been completed. Outstanding work has been broken out into separate issues and tracked in https://github.com/orgs/tenstorrent/projects/50/views/1