Ukubonelela ngeemodeli ze-AI ukusuka kwi-NVIDIA Tesla P40 (24 GB VRAM, uyilo lwePascal): i-LLMs, iimodyuli zomfanekiso nezinye izilumkiso. Abancedisi be-GPU abakwazi ukuthengwa; qhagamshelana nathi ukuze uqinisekiswe xa bekhona.
$ pip install vllm && vllm serve meta-llama/Llama-3-8B-Instruct
# Umzekelo wofakelo lomyalelo
AI inference ithetha ukuba uqhuba imodeli eqeqeshiweyo ukudala izigqibo. Umncedisi we-GPU inference ugcina imodeli elayishwe kwi-GPU kwaye uphendula izicelo ngokukhawuleza kune-CPU.
I-GPU idlula kwisincedisi sakho kuphela, ngoko ke ukuphuma kwexesha alichaphazeli imisebenzi ye-GPU yabanye abathengi.
Sebenzisa i-Ollama, i-lama.cpp, i-Triton okanye i-API yakho. Ezinye iifayili, ezinjenge-vLLM, zifuna i-GPUs entsha kune-Pascal P40.
Sebenzisa abancedisi abaninzi ngasemva kolawulo lwakho oluzinzileyo ukuba i-GPU enye ayinyanzelekanga.
Unikezele ngemodeli enye ephakathi okanye ezininzi ezincinci.
AI Inference inokusebenzisa i NVIDIA GPU ukukhawulezisa umsebenzi wayo. VPS.org GPU abancedisi abafumanekanga ukucela ngoku; qhagamshelana nathi ukuva xa bekhona. AI Inference yi GPU- ekhawulezayo yomsebenzi oqhelekileyo. Qinisekisa ukuba udweliso lwenkqubo luxhaswe yi CUDA kwaye ukuba umqhubi wakho / uguqulelo lwexesha lokusebenza luhambelana neemfuno zomsebenzi womsebenzi we AI Inference.
Abancedisi be-GPU abafumanekanga ukucela ngoku. Namhlanje ungaqhuba i-AI Inference kwi-CPU eqhelekileyo, apho isebenza khona ngokucothayo kune-GPU: SSH ngaphakathi kwaye uqhube i-pip install vllm && vllm serve meta-llama/Llama-3-8B-Instruct.
Iimfuneko ze-VRAM ze-AI Inference zixhomekeke kwimodeli kunye nobungakanani beqela; khangela ushicilelo lweprojekthi yemifanekiso yangoku.
Ixabiso le-GPU liza kupapashwa xa iiseva ze-GPU zingafunyanwa. Xhulumanisa nathi ukuba ufuna ukuva xa oku kwenzeka.
Ewe. Isincedisi ngasinye se-VPS.org sifika sinee-root access ezipheleleyo, ngoko ungaqhuba ezinye izixhobo kunye ne-AI Inference ngaphakathi kwe-RAM yeseva kunye nokugcina.
Yonke i VPS.org umncedisi ifika nge root SSH unikezelo olupheleleyo, ngoko ungafaka abaqhubi, iincwadi ze AI Inference nokuba ufuna njani.
Hayi. Akukho mncedisi ufika nexesha lokuqhuba le-CUDA okanye umqhubi we-NVIDIA ofakwe ngaphambili, kwaye abancedisi be-GPU abafumanekanga ukucela ngoku. Ufaka uguqulelo lwe-AI Inference olufunayo ngokwakho.
Ewe. Abancedisi be-VPS.org basebenza njengezimatshini ezibonakalayo eziqhubekayo, azikho iziganeko ezikhawulezayo: iimodyuli ze-AI Inference, iinkqubo zokumiselwa kwenkqubo kunye nedata zihlala kwi-SSD yokugcina yeseva de uzicime.
Gcina i-data esebenza kwi-SSD yomncedisi ukufikelela ngokukhawuleza ngexesha lokusebenza kwe-AI Inference, kwaye ukope i-artifacts egqityiweyo (isisindo, u Generations, i-embeddings) ukugcinwa ngaphandle kwe-VPS.org; iifoto ezikhawulezayo kunye nokhuseleko luhlala kwiqela elinye lokugcina.
Unganyusa nayiphi na iseva ye-VPS.org ibe yinkqubo enkulu ukusuka kwiqela lenjongo yolawulo; ubungakanani obutsha buqala kwakhona kwaye ukufaka kwe-AI Inference kugqitha. Iiseva ze-GPU azifumanekanga ngoku.
Ewe. Iindawo zokugcina zihlawulwe zidityaniswe (ngosuku +30% yexabiso lenkqubo, igcinwe iintsuku ezi-7; ngeveki +20%, igcinwe iintsuku ezi-28) kwaye iifoto ezikhawulezayo zikhululekile, eziluncedo phambi kwe AI Inference ende iqhuba. Zonke zihlala kwiqela lokugcina elifanayo njengeseva.
Nxibelelana noncedo ngaphakathi kweentsuku ezingama-30 zokuthengiswa kwakho kokuqala ukuze ucele ubuyiselo. Iiseva ze-GPU azithengwanga, ngoko akukho mfuneko yokuthumela i-GPU.
Ubungakanani be-GPU bumiselwe. Sixelele malunga nobunzima bomsebenzi wakho kwaye siza kuphendula ngokufumaneka nokuthengiswa.