Silungiselela iiseva ze-GPU nge-NVIDIA Tesla P40 (24 GB VRAM, uyilo lwePascal) ukuqhubela phambili i-LLMs ezivulekileyo ezinjenge-Llama, i-Mistral ne-Gemma nge-Ollama. Iiseva ze-GPU azikwazi ukuthengwa; qhagamshelana nathi ukuze uqinisekiswe xa zikhona.
$ curl -fsSL https://ollama.com/install.sh | sh && ollama run llama3
# Umzekelo wofakelo lomyalelo
Ollama yisixhobo esilula sokusebenza kweemodeli zolwimi olukhulu kwikhompyutha yakho. Kwiseva ye GPU imodeli isebenza kwi GPU endaweni ye CPU, eyenza izimvo zikhawuleze kakhulu.
Thumela i Llama, i Mistral, i Gemma, i Phi, i Qwen nezinye iimodyuli ezininzi ngomyalelo omnye.
Ilungele i7B ukuya kwi13B imodeli ngokukhululekileyo kunye ne-30B imodeli xa i-quantized. I-70B imodeli ayifani kwikhadi elinye le-24 GB kumanqanaba aqhelekileyo oku-quantization.
Ii-prompts ne-data zihlala kwiseva yakho, ngaphandle kokunye unxibelelwano lwe-API.
Ollama ibonisa i OpenAI-ehambelanayo nendawo yesiphelo eninzi yabaxhasi abakhoyo banokusetyenziswa.
Ollama inokusebenzisa i NVIDIA GPU ukukhawulezisa umsebenzi wayo. VPS.org GPU abancedisi abafumanekanga ukucela ngoku; qhagamshelana nathi ukuva xa bekhona. Ollama kukuqala LLM-inference / incoko umsebenzi wobunzima. Uzakufuna ukufunda okukhawulezayo okunokwenzeka-ukufikelela ukusuka kwi diski ukuya kwimemori kunye ne VRAM eyaneleyo yemodeli kunye ne window yemeko.
Abancedisi be-GPU abafumanekanga ukucela ngoku. Namhlanje ungaqhuba i-Ollama kwi-CPU eqhelekileyo, apho isebenza khona ngokucothayo kune-GPU: SSH ngaphakathi kwaye uqhube i-curl -fsSL https://ollama.com/install.sh | sh && ollama run llama3.
I-LLM ichaza ukuba i-VRAM iphakama ngeeparamitha zemodeli: imodeli ye-7B ifuna malunga ne-5 ukuya kwi-8 GB ye-VRAM nemodeli ye-13B malunga ne-10 ukuya kwi-14 GB, ngelixa imodeli ye-70B-class ifuna i-GPUs ezininzi okanye ukulinganisa okunzima.
Ixabiso le-GPU liza kupapashwa xa iiseva ze-GPU zingafunyanwa. Xhulumanisa nathi ukuba ufuna ukuva xa oku kwenzeka.
Ewe. Isincedisi ngasinye se-VPS.org sifika sinee-root access ezipheleleyo, ngoko ungaqhuba ezinye izixhobo kunye ne-Ollama ngaphakathi kwe-RAM yeseva kunye nokugcina.
Yonke i VPS.org umncedisi ifika nge root SSH unikezelo olupheleleyo, ngoko ungafaka abaqhubi, iincwadi ze Ollama nokuba ufuna njani.
Hayi. Akukho mncedisi ufika nexesha lokuqhuba le-CUDA okanye umqhubi we-NVIDIA ofakwe ngaphambili, kwaye abancedisi be-GPU abafumanekanga ukucela ngoku. Ufaka uguqulelo lwe-Ollama olufunayo ngokwakho.
Ewe. Abancedisi be-VPS.org basebenza njengezimatshini ezibonakalayo eziqhubekayo, azikho iziganeko ezikhawulezayo: iimodyuli ze-Ollama, iinkqubo zokumiselwa kwenkqubo kunye nedata zihlala kwi-SSD yokugcina yeseva de uzicime.
Gcina i-data esebenza kwi-SSD yomncedisi ukufikelela ngokukhawuleza ngexesha lokusebenza kwe-Ollama, kwaye ukope i-artifacts egqityiweyo (isisindo, u Generations, i-embeddings) ukugcinwa ngaphandle kwe-VPS.org; iifoto ezikhawulezayo kunye nokhuseleko luhlala kwiqela elinye lokugcina.
Unganyusa nayiphi na iseva ye-VPS.org ibe yinkqubo enkulu ukusuka kwiqela lenjongo yolawulo; ubungakanani obutsha buqala kwakhona kwaye ukufaka kwe-Ollama kugqitha. Iiseva ze-GPU azifumanekanga ngoku.
Ewe. Iindawo zokugcina zihlawulwe zidityaniswe (ngosuku +30% yexabiso lenkqubo, igcinwe iintsuku ezi-7; ngeveki +20%, igcinwe iintsuku ezi-28) kwaye iifoto ezikhawulezayo zikhululekile, eziluncedo phambi kwe Ollama ende iqhuba. Zonke zihlala kwiqela lokugcina elifanayo njengeseva.
Nxibelelana noncedo ngaphakathi kweentsuku ezingama-30 zokuthengiswa kwakho kokuqala ukuze ucele ubuyiselo. Iiseva ze-GPU azithengwanga, ngoko akukho mfuneko yokuthumela i-GPU.
Ubungakanani be-GPU bumiselwe. Sixelele malunga nobunzima bomsebenzi wakho kwaye siza kuphendula ngokufumaneka nokuthengiswa.