Tikukonza ma GPU ma seva ndi NVIDIA Tesla P40 (24 GB VRAM, Pascal architecture) kuti tigwiritse ntchito LLMs otsegulira monga Llama, Mistral ndi Gemma ndi Ollama. Ma GPU ma seva si akhoza kusinthidwa; tiuzeni kuti mudziwidwe pamenepo.
$ curl -fsSL https://ollama.com/install.sh | sh && ollama run llama3
# Mfano wa kukhazikitsa lamulo
Ollama ndi chida cholimba choyendetsa mapangidwe amtundu waukulu pa kompyuta yanu. Pa GPU server, mapangidwe amayenda pa GPU osati pa CPU, zomwe zimapangitsa kuti mayankho aziyenda mofulumira.
Kuthamanga Llama, Mistral, Gemma, Phi, Qwen ndi ambiri ena mafano ndi lamulo limodzi.
Amagwirizana ndi 7B mpaka 13B mafano mofulumira ndi m'madera 30B mafano pamene quantized.70B mafano sayenera pa 24 GB katundu wina pa khalidwe quantization.
Mafunso ndi deta amakhala pa seva yanu, popanda zopempha za API zachitatu.
Ollama amasonyeza OpenAI-ogwirizana endpoint kuti ambiri opezekapo angathe kugwiritsa ntchito.
Ollama angagwiritse ntchito NVIDIA GPU kuti afulumizitse ntchito yake. VPS.org GPU ma seva si apezeka kuti apereke; tiuzeni kuti mudziwe pamene iwo ndi. Ollama ndi koyamba LLM-inference / chat workload. Mudzafuna mwachangu kulowa kwaulere kuchokera pa diski kupita ku kumbukirani ndi VRAM yokwanira kwa model plus context window.
GPU ma seva si opezeka kuti aperekebe. Tsopano mungagwiritse ntchito Ollama pa CPU yachikhalidwe, komwe imagwira ntchito mofulumira kuposa pa GPU: SSH mu ndi kuyendetsa curl -fsSL https://ollama.com/install.sh | sh && ollama run llama3.
LLM inference VRAM scales ndi ma parameters a model: 7B model imafuna pafupifupi 5 mpaka 8 GB ya VRAM ndi 13B model pafupifupi 10 mpaka 14 GB, pomwe 70B-class models imafunikira GPUs angapo kapena kulemera kwamphamvu.
Kugulitsa kwa GPU kudzatulutsidwa pamene ma seva a GPU angagwiritsidwe ntchito. Tiuzeni ngati mukufuna kumvetsetsa pamene izi zikuchitika.
Yai. Kawirikawiri VPS.org server amachokera ndi wonse root kulowa, kotero inu mukhoza kuyendetsa ena zipangizo mbali Ollama m'ma RAM ndi kusungirako server.
Gawo lililonse VPS.org wothandizira amachokera ndi wonse root SSH kulowa, kotero inu mukhoza kukhazikitsa oyendetsa, mabuku ndi Ollama koma inu muyenera.
Sichoncho. Palibe wothandizira amafika ndi CUDA runtime kapena NVIDIA woyendetsa preinstalled, ndi GPU othandizira sizilipo kuti aperekebe. Mumakhazikitsa Ollama zojambula zofunikira nokha.
Ndikoyenera. VPS.org ma seva ndi olimba olimba, osati opanda pake: Ollama mapangidwe, zosankha ndi deta zimakhazikika pa SSD yosungira server mpaka mutachotsa.
Kusunga ntchito deta pa SSD seva kwa kulowa mosavuta pamene Ollama kunyamula, ndi kukopera kumaliza artifacts (mawuwo, chiyambi, embeddings) ku osungirako kunja VPS.org; snapshots ndi zosunga zobwezeretsera azikhala pamodzi osungirako gulu.
Mukhoza kuwonjezera iliyonse VPS.org wothandizira kuti malire lalikulu kuchokera ku gulu la kuwongolera; resize imayambanso ndi Ollama kukhazikitsa wanu amapita. GPU othandizira si opezekabe.
Yes. Backups ndi wolipira add-on (tsiku ndi tsiku + 30% ya mtengo wa ndondomeko, anasunga masiku 7; tsiku ndi tsiku + 20%, anasunga masiku 28) ndi snapshots ndi ufulu, amene ndi zothandiza pamaso Ollama m'mbuyo m'mbuyo.
Mutha kulumikizana ndi thandizo m'masiku 30 a kugula kwanu koyamba kuti mupemphe kuti mubwezere ndalama. GPU servers sizikugulitsidwabe, chifukwa chake palibe GPU order.
Kutha kwa GPU ndi kocheperako. Tiuzeni za ntchito yanu ndipo tidzakuyankha ndi kuchuluka kwa ndalama zomwe zilipo.