Add endpoint for file support, purely to speed up processing of input_embeds.

[RinRin-32]

Motivation

While input_embeds is now supported, I find that for what I'm using it for, normally requesting through /generate wasn't enough. I was spending around 2.5 seconds on average until the request was even handled in tokenizer manager. With this in mind, I suspect that the main issue came around possible overhead and such. To overcome said issue, I figured sending said input_embeds as a file likely would reduce the issue, doing so I managed to shaved off around 2 seconds from my use case.

Please contribute or comment your idea of a proper unit test for this application.

Modifications

Added endpoint in server.py to handle direct file transfer.

Checklist

• Format your code according to the Contributor Guide.
• Add unit tests as outlined in the Contributor Guide.
• Update documentation as needed, including docstrings or example tutorials.

[merrymercy]

You can add a test case here https://github.com/sgl-project/sglang/blob/main/test/srt/test_input_embeddings.py

[RinRin-32]

@merrymercy
My use case with this right now is to serve our LLM based service.

It's based on Gemma 2 2b where I launch it with the following

python -m sglang.launch_server --model-path <model-path> --port 30000 --disable-radix --disable-cuda-graph-padding --disable-jump-forward --disable-mla

While it's faster with the new endpoint I can't quit get it as close as how the model can perform on simple implementation. Is there's anything else I can do to get faster response? Perhaps there's something I overlooked while making the changes in this PR, I'm especially curious about whether fastapi implementation made in sglang can be improved.

[RinRin-32]

I skipped checking for param from file name because using the overlap scheduler the sampling params are ignored anyway.