Silungiselela amaseva we-GPU nge-NVIDIA Tesla P40 (24 GB VRAM, Pascal architecture) ukuqhuba ama-LLMs avulekile afana ne-Llama, Mistral ne-Gemma nge-Ollama. Amaseva we-GPU angeke akwazi ukuthengwa; xhumana nathi ukuze uqinisekiswe uma bekhona.
$ curl -fsSL https://ollama.com/install.sh | sh && ollama run llama3
# Isibonelo sokufaka umyalo
Ollama yithuluzi elincane lokuqhuba amamodeli elimi elikhulu kwikhompyutha yakho. Ku-GPU server imodeli isebenza ku-GPU endaweni ye-CPU, okukwenza uphendule ngokushesha.
Thola i-Llama, i-Mistral, i-Gemma, i-Phi, i-Qwen nezinye izinhlobo eziningi ngemiyalelo eyodwa.
Ilungele 7B kuya ku-13B amamodeli ngokunethezeka futhi eduze kwamamodeli we-30B uma ku-quantized. Amamodeli we-70B angeke alungele ikhadi elilodwa le-24 GB emazingeni ajwayelekile we-quantization.
Izinhlamvu nedatha zihlala kwiseva yakho, ngaphandle kokushaya kwe-API kwengxenye yesithathu.
Ollama ibonisa isiphetho esihambisana ne-OpenAI esisetshenziswa izisebenzi eziningi ezikhona.
Ollama ingasebenzisa i-NVIDIA GPU ukusheshisa umsebenzi wayo. VPS.org amaseva we-GPU akhona ukucela; xhumana nathi ukulalela uma bekhona. Ollama kubaluleke kakhulu i-LLM-inference / ukuxhumana komsebenzi. Uzofuna ukufundela okukhawulelwe okuzenzakalelayo kusuka ku-disk kuya ku-memory kanye ne-VRAM eyanele yemodeli kanye nefasitela lesimo.
Amaseva we-GPU akhona ukucela okwamanje. Namuhla ungasebenzisa i-Ollama ku-CPU ejwayelekile, lapho isebenza khona ngokucophelela kunakwi-GPU: SSH ngaphakathi bese usebenzisa i-curl -fsSL https://ollama.com/install.sh | sh && ollama run llama3.
LLM inference VRAM scales with model parameters: a 7B model needs roughly 5 to 8 GB of VRAM and a 13B model roughly 10 to 14 GB, while 70B-class models need several GPUs or heavy quantification.
Intengo ye-GPU izoshicilelwa uma izisebenzisi ze-GPU zingasatshalaliswa. Xhumana nathi uma ufuna ukulalela uma lokhu kwenzeke.
Yebo. Yonke iseva ye-VPS.org ifika nge-root access egcwele, ngakho ungasebenzisa ezinye izisetshenziswa kanye ne-Ollama ngaphakathi kwe-RAM ne-storage yeseva.
Yonke isisebenzisi se VPS.org sifika sigcwele ukungena kwe-root SSH, ngakho ungafaka ama-driver, ama-library ne Ollama noma ngabe ufuna.
Akukho umsekeli ofika ngesikhathi sokusebenza se-CUDA noma umqhubi we-NVIDIA ofakwe ngaphambili, futhi ama-servers we-GPU atholakalanga ukuthenga okwamanje. Ungafaka izibuyekezo Ollama ezidingayo wena.
Yebo. Amaseva we-VPS.org asebenza ngama-machines aqhubekayo, hhayi izimo ezihambayo: amamodeli we-Ollama, izilungiselelo nedatha zihlala ku-SSD yokugcina yeseva kuze kube ucima.
Gcina idatha esebenza kuseva ye-SSD ukuze ufinyelele ngokushesha ngesikhathi se-Ollama, futhi ukope izitho eziqediwe (izitho, izizukulwane, izingeniso) ukugcinwa ngaphandle kwe-VPS.org; ama-snapshots nama-backup ahlala ku-storage cluster efanayo.
Ungakhuphula noma iyiphi iseva ye-VPS.org ibe yi-plan enkulu kusuka kwindawo yokulawula; ukushintsha ubukhulu kuqala kabusha futhi ukufakwa kwe-Ollama kudlulisa. Amaseva we-GPU akhona.
Yebo. Amafayela okugcina akhokhelwa angeza-on (ngesonto +30% yentengo yephrojekthi, igcinwe 7 izinsuku; ngeviki +20%, igcinwe 28 izinsuku) futhi ama-snapshots amahhala, okusiza ngaphambi kokuhamba isikhathi eside Ollama. Kokubili kuhlale ku-indawo yokugcina efanayo njengeseva.
Xhumana nosizo ngaphakathi kwezinsuku ezingama-30 zokuthenga kwakho okukuqala ukuze ucele ukubuyiselwa imali. Amaseva we-GPU athengiswe, ngakho-ke akukho sinqumo se-GPU esihlobene.
Ikhono le-GPU lilinganiselwe. Sitshele ngomsebenzi wakho futhi sizophendula ngokutholakala nokuthengwa.