Kupereka mapangidwe a AI kuchokera ku NVIDIA Tesla P40 (24 GB VRAM, Pascal architecture): LLMs, mapangidwe azithunzi ndi zina zoyembekezeka. Ma seva a GPU sakupezekapo; tiuzeni kuti mudziwe pamenepo.
$ pip install vllm && vllm serve meta-llama/Llama-3-8B-Instruct
# Mfano wa kukhazikitsa lamulo
GPU inference server imasunga mtundu woyikidwa pa GPU ndipo imayankha mafunso ofunikira kwambiri kuposa CPU.Kuphatikiza apo, GPU inference server imasunga mtundu woyikidwa pa GPU ndipo imayankha mafunso ofunikira kwambiri kuposa CPU.
GPU imapita ku seva yanu yokha, kotero latency siyikhudzana ndi ntchito za GPU za makasitomala ena.
Kugwiritsa ntchito Ollama, llama.cpp, Triton kapena API yanu yapadera. Ena mwa ma frameworks, monga vLLM, amafuna GPUs atsopano kuposa Pascal P40.
Kuyendetsa ma seva angapo m'mbuyo mwa woyang'anira wanu wapadera ngati GPU imodzi siyokwanira.
Kugwiritsa ntchito imodzi mwa mitundu yapakati kapena mitundu ingapo yazing'ono.
AI Inference akhoza kugwiritsa ntchito NVIDIA GPU kuti afulumizitse ntchito yake. VPS.org GPU ma seva si angathe kulandira; tiuzeni kuti mudziwe pamene iwo ndi. AI Inference ndi GPU-kufulumira ntchito ntchito. onetsetsani kuti mapulogalamu anu ali CUDA thandizo ndi kuti wanu woyendetsa / runtime zigawo kugwirizana zosowa ntchito ntchito kwa AI Inference.
GPU ma seva si opezeka kuti aperekebe. Tsopano mungagwiritse ntchito AI Inference pa CPU yachikhalidwe, komwe imagwira ntchito mofulumira kuposa pa GPU: SSH mu ndi kuyendetsa pip install vllm && vllm serve meta-llama/Llama-3-8B-Instruct.
VRAM zosowa kwa AI Inference zimadalira mtundu wa mtundu ndi gulu; onetsetsani ntchito zamakono za ntchitoyi za anthu amakono.
Kugulitsa kwa GPU kudzatulutsidwa pamene ma seva a GPU angagwiritsidwe ntchito. Tiuzeni ngati mukufuna kumvetsetsa pamene izi zikuchitika.
Yai. Kawirikawiri VPS.org server amachokera ndi wonse root kulowa, kotero inu mukhoza kuyendetsa ena zipangizo mbali AI Inference m'ma RAM ndi kusungirako server.
Gawo lililonse VPS.org wothandizira amachokera ndi wonse root SSH kulowa, kotero inu mukhoza kukhazikitsa oyendetsa, mabuku ndi AI Inference koma inu muyenera.
Sichoncho. Palibe wothandizira amafika ndi CUDA runtime kapena NVIDIA woyendetsa preinstalled, ndi GPU othandizira sizilipo kuti aperekebe. Mumakhazikitsa AI Inference zojambula zofunikira nokha.
Ndikoyenera. VPS.org ma seva ndi olimba olimba, osati opanda pake: AI Inference mapangidwe, zosankha ndi deta zimakhazikika pa SSD yosungira server mpaka mutachotsa.
Kusunga ntchito deta pa SSD seva kwa kulowa mosavuta pamene AI Inference kunyamula, ndi kukopera kumaliza artifacts (mawuwo, chiyambi, embeddings) ku osungirako kunja VPS.org; snapshots ndi zosunga zobwezeretsera azikhala pamodzi osungirako gulu.
Mukhoza kuwonjezera iliyonse VPS.org wothandizira kuti malire lalikulu kuchokera ku gulu la kuwongolera; resize imayambanso ndi AI Inference kukhazikitsa wanu amapita. GPU othandizira si opezekabe.
Yes. Backups ndi wolipira add-on (tsiku ndi tsiku + 30% ya mtengo wa ndondomeko, anasunga masiku 7; tsiku ndi tsiku + 20%, anasunga masiku 28) ndi snapshots ndi ufulu, amene ndi zothandiza pamaso AI Inference m'mbuyo m'mbuyo.
Mutha kulumikizana ndi thandizo m'masiku 30 a kugula kwanu koyamba kuti mupemphe kuti mubwezere ndalama. GPU servers sizikugulitsidwabe, chifukwa chake palibe GPU order.
Kutha kwa GPU ndi kocheperako. Tiuzeni za ntchito yanu ndipo tidzakuyankha ndi kuchuluka kwa ndalama zomwe zilipo.