I-Llama model ye-Meta ehlala yona e-NVIDIA Tesla P40 (24 GB VRAM, Pascal architecture). Ilungele i-8B model egcwele ukuthembeka kanye ne-large model uma i-quantized. Amaseva we-GPU angeke akwazi uku-oda; xhumana nathi ukuze uqinisekiswe uma ekhona.
$ curl -fsSL https://ollama.com/install.sh | sh && ollama run llama3
# Isibonelo sokufaka umyalo
I-Llama yimindeni ye-Meta yezinhlamvu ezitholakalayo ezivulekile. Ukusebenzisa i-Llama ku-GPU yakho yesisebenzisi sigcina izicelo nedatha ifihlakele.
Isebenza amamodeli we-8B ku-24 GB we-VRAM. Amamodeli we-70B alungele ikhadi elilodwa le-24 GB kuma-leveli ajwayelekile wokwahlukanisa.
Izinhlamvu zakho nedatha zihlala kwiseva yakho, ngaphandle kokushaya kwe-API kwengxenye yesithathu.
Sebenzisa i-GGUF ne-lama.cpp noma i-Ollama, noma isisindo se-PyTorch esisemthethweni.
Sebenzisa amamodeli nge Ollama noma i-lama.cpp server ye-OpenAI-compatible endpoint.
LLaMA ingasebenzisa i-NVIDIA GPU ukusheshisa umsebenzi wayo. VPS.org amaseva we-GPU akhona ukucela; xhumana nathi ukulalela uma bekhona. LLaMA yi-GPU-ekhawulelwe umsebenzi ojwayelekile. Qinisekisa ukuthi isofthiwe yakho inezixhaso ze-CUDA futhi ukuthi umshayeli wakho / ama-runtime versions alingana nezidingo zomsebenzi we-LLaMA.
Amaseva we-GPU akhona ukucela okwamanje. Namuhla ungasebenzisa i-LLaMA ku-CPU ejwayelekile, lapho isebenza khona ngokucophelela kunakwi-GPU: SSH ngaphakathi bese usebenzisa i-curl -fsSL https://ollama.com/install.sh | sh && ollama run llama3.
Izidingo ze-VRAM ze-LLaMA zixhomekeke kumodeli kanye nobukhulu beqembu; hlola idatabase yephrojekthi ngezibalo zamanje.
Intengo ye-GPU izoshicilelwa uma izisebenzisi ze-GPU zingasatshalaliswa. Xhumana nathi uma ufuna ukulalela uma lokhu kwenzeke.
Yebo. Yonke iseva ye-VPS.org ifika nge-root access egcwele, ngakho ungasebenzisa ezinye izisetshenziswa kanye ne-LLaMA ngaphakathi kwe-RAM ne-storage yeseva.
Yonke isisebenzisi se VPS.org sifika sigcwele ukungena kwe-root SSH, ngakho ungafaka ama-driver, ama-library ne LLaMA noma ngabe ufuna.
Akukho umsekeli ofika ngesikhathi sokusebenza se-CUDA noma umqhubi we-NVIDIA ofakwe ngaphambili, futhi ama-servers we-GPU atholakalanga ukuthenga okwamanje. Ungafaka izibuyekezo LLaMA ezidingayo wena.
Yebo. Amaseva we-VPS.org asebenza ngama-machines aqhubekayo, hhayi izimo ezihambayo: amamodeli we-LLaMA, izilungiselelo nedatha zihlala ku-SSD yokugcina yeseva kuze kube ucima.
Gcina idatha esebenza kuseva ye-SSD ukuze ufinyelele ngokushesha ngesikhathi se-LLaMA, futhi ukope izitho eziqediwe (izitho, izizukulwane, izingeniso) ukugcinwa ngaphandle kwe-VPS.org; ama-snapshots nama-backup ahlala ku-storage cluster efanayo.
Ungakhuphula noma iyiphi iseva ye-VPS.org ibe yi-plan enkulu kusuka kwindawo yokulawula; ukushintsha ubukhulu kuqala kabusha futhi ukufakwa kwe-LLaMA kudlulisa. Amaseva we-GPU akhona.
Yebo. Amafayela okugcina akhokhelwa angeza-on (ngesonto +30% yentengo yephrojekthi, igcinwe 7 izinsuku; ngeviki +20%, igcinwe 28 izinsuku) futhi ama-snapshots amahhala, okusiza ngaphambi kokuhamba isikhathi eside LLaMA. Kokubili kuhlale ku-indawo yokugcina efanayo njengeseva.
Xhumana nosizo ngaphakathi kwezinsuku ezingama-30 zokuthenga kwakho okukuqala ukuze ucele ukubuyiselwa imali. Amaseva we-GPU athengiswe, ngakho-ke akukho sinqumo se-GPU esihlobene.
Ikhono le-GPU lilinganiselwe. Sitshele ngomsebenzi wakho futhi sizophendula ngokutholakala nokuthengwa.