Sebenzisa amamodeli we-AI kusuka ku-NVIDIA Tesla P40 (24 GB VRAM, Pascal architecture): LLMs, amamodeli wesithombe nezinye izibikezelo. Amaseva we-GPU angeke akwazi uku-oda; xhumana nathi ukuze uqinisekiswe uma bekhona.
$ pip install vllm && vllm serve meta-llama/Llama-3-8B-Instruct
# Isibonelo sokufaka umyalo
Ukuqonda kwe-AI kusho ukuthi usebenza ngemodeli eziqeqeshiwe ukuze ukhiqize izibikezelo. Umsekeli wokuqonda kwe-GPU ugcina imodeli efakwe ku-GPU futhi uphendula izicelo ngokushesha kune-CPU.
I-GPU idluliswa kuseva yakho kuphela, ngakho-ke ukulibazisa akuhlobene namanye amakhasimende wemisebenzi ye-GPU.
Sebenzisa i-Ollama, i-lama.cpp, i-Triton noma i-API yakho. Ezinye izimo, ezifana ne-vLLM, zidinga ama-GPUs amasha kunawo wonke ama-Pascal P40.
Sebenzisa izisebenzisi eziningi ezingaphansi kokufaka isilinganisi sakho uma i-GPU eyodwa ingalungile.
Sebenzisa imodeli eyodwa ephakathi noma encane.
AI Inference ingasebenzisa i-NVIDIA GPU ukusheshisa umsebenzi wayo. VPS.org amaseva we-GPU akhona ukucela; xhumana nathi ukulalela uma bekhona. AI Inference yi-GPU-ekhawulelwe umsebenzi ojwayelekile. Qinisekisa ukuthi isofthiwe yakho inezixhaso ze-CUDA futhi ukuthi umshayeli wakho / ama-runtime versions alingana nezidingo zomsebenzi we-AI Inference.
Amaseva we-GPU akhona ukucela okwamanje. Namuhla ungasebenzisa i-AI Inference ku-CPU ejwayelekile, lapho isebenza khona ngokucophelela kunakwi-GPU: SSH ngaphakathi bese usebenzisa i-pip install vllm && vllm serve meta-llama/Llama-3-8B-Instruct.
Izidingo ze-VRAM ze-AI Inference zixhomekeke kumodeli kanye nobukhulu beqembu; hlola idatabase yephrojekthi ngezibalo zamanje.
Intengo ye-GPU izoshicilelwa uma izisebenzisi ze-GPU zingasatshalaliswa. Xhumana nathi uma ufuna ukulalela uma lokhu kwenzeke.
Yebo. Yonke iseva ye-VPS.org ifika nge-root access egcwele, ngakho ungasebenzisa ezinye izisetshenziswa kanye ne-AI Inference ngaphakathi kwe-RAM ne-storage yeseva.
Yonke isisebenzisi se VPS.org sifika sigcwele ukungena kwe-root SSH, ngakho ungafaka ama-driver, ama-library ne AI Inference noma ngabe ufuna.
Akukho umsekeli ofika ngesikhathi sokusebenza se-CUDA noma umqhubi we-NVIDIA ofakwe ngaphambili, futhi ama-servers we-GPU atholakalanga ukuthenga okwamanje. Ungafaka izibuyekezo AI Inference ezidingayo wena.
Yebo. Amaseva we-VPS.org asebenza ngama-machines aqhubekayo, hhayi izimo ezihambayo: amamodeli we-AI Inference, izilungiselelo nedatha zihlala ku-SSD yokugcina yeseva kuze kube ucima.
Gcina idatha esebenza kuseva ye-SSD ukuze ufinyelele ngokushesha ngesikhathi se-AI Inference, futhi ukope izitho eziqediwe (izitho, izizukulwane, izingeniso) ukugcinwa ngaphandle kwe-VPS.org; ama-snapshots nama-backup ahlala ku-storage cluster efanayo.
Ungakhuphula noma iyiphi iseva ye-VPS.org ibe yi-plan enkulu kusuka kwindawo yokulawula; ukushintsha ubukhulu kuqala kabusha futhi ukufakwa kwe-AI Inference kudlulisa. Amaseva we-GPU akhona.
Yebo. Amafayela okugcina akhokhelwa angeza-on (ngesonto +30% yentengo yephrojekthi, igcinwe 7 izinsuku; ngeviki +20%, igcinwe 28 izinsuku) futhi ama-snapshots amahhala, okusiza ngaphambi kokuhamba isikhathi eside AI Inference. Kokubili kuhlale ku-indawo yokugcina efanayo njengeseva.
Xhumana nosizo ngaphakathi kwezinsuku ezingama-30 zokuthenga kwakho okukuqala ukuze ucele ukubuyiselwa imali. Amaseva we-GPU athengiswe, ngakho-ke akukho sinqumo se-GPU esihlobene.
Ikhono le-GPU lilinganiselwe. Sitshele ngomsebenzi wakho futhi sizophendula ngokutholakala nokuthengwa.