arslantu/qwen_deploy
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

init🎉:

Browse Source
This commit is contained in:
arslantu
2024-04-06 10:10:57 +08:00
commit ccfb462fb8
6 changed files with 74 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docker/start.sh Normal file
View File

@ -0,0 +1 @@
python -m vllm.entrypoints.openai.api_server --dtype bfloat16 --model ${MODEL_PATH} --max-model-len ${MAX_MODEL_LEN} --host ${HOST} --port ${PORT} --api-key ${API_KEY}