[Feature] Beam Search

[laixinn]

Motivation & Modifications

Support beam search, RFC in #3032 .

TODO:

• enable beam search by server args (currently using environment variables)
• support the overlap mode
• support jump forward
• use trition kernel to accelerate indices overwritten for kv cache

Result & Reproduce

1. Accuracy: test/srt/test_beam_search.py: line 259, test_beam_search_accuracy_by_offline_mmlu
2. MMLU Score: test/srt/test_beam_search.py: line 336, test_beam_search_memory_leak_via_mmlu
3. Efficiency (will not run in unit test by default): line 301, bench_beam_search_overhead_by_offline_mmlu

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling.