Re ntse re hlophisa li-server tsa GPU le NVIDIA Tesla P40 (24 GB VRAM, moralo oa Pascal) bakeng sa ho sebetsa ka LLMs tse bulehileng tse kang Llama, Mistral le Gemma le Ollama. Li-server tsa GPU ha li ka odara; ikopanye le rona ho tsebisoa ha li le teng.
$ curl -fsSL https://ollama.com/install.sh | sh && ollama run llama3
# Mohlala oa molaoli oa ho kenya
Ollama ke sesebelisoa se bonolo sa ho sebetsana le liteko tsa puo e kholo ka mochini oa hau. Ka mofani oa GPU, liteko li sebetsa ka GPU eseng CPU, e etsang hore likarabo li potlake haholo.
Ea Llama, Mistral, Gemma, Phi, Qwen le li-model tse ling tse ngata ka taelo e le 'ngoe.
E loketse li-models tsa 7B ho 13B ka mokhoa o sireletsehileng le kantle ho li-models tsa 30B ha li quantised. Li-models tsa 70B ha li loketse karete e le 'ngoe ea 24 GB ka litekanyetso tsa quantification tse tloaelehileng.
Lipotso le data li lula ho seva ea hau, ntle le ho ngola API ea motho oa boraro.
E na le li-terminals tse ngata tse sebetsang ka mokhoa o ts'oanang le tse ling tse ngata tse sebetsang ka mokhoa o ts'oanang.
Ollama e ka sebelisa NVIDIA GPU ho potlakisa mosebetsi oa eona. VPS.org GPU servers don't exist to order yet; contact us to hear when they are. Ollama is primarily a LLM-inference / chat workload. You'll want fast random-access reading from disk to memory and enough VRAM for the model plus context window.
Li-server tsa GPU ha li fumanehe ho rekisoa. Kajeno u ka kenya Ollama ka qeto ea CPU e tloaelehileng, moo e sebetsang ka potlako ho feta GPU: SSH kahare'me u kenya curl -fsSL https://ollama.com/install.sh | sh && ollama run llama3.
Mofuta oa 7B o hloka ~ 5-8 GB VRAM, 13B ~ 10-14 GB, 70B o hloka multi-GPU kapa quantification. 24 GB ea rona Tesla P40 e sebetsa ka mokhoa o sireletsehileng li-models tsa 7B-13B ka ho hlaka le li-models tsa 30B-class le INT8 quantification.
The GPU priceing e tla phatlalatsoa ha li-server tsa GPU li ka rekisoa. Ikopanye le rona haeba u batla ho tseba ha e etsahala.
Ha ho joalo. Motsamaisi VPS.org o mong le o mong o na le ho kena ka ho feletseng ho root, ka hona o ka kenya lisebelisoa tse ling hammoho le Ollama ka RAM ea motsamaisi le polokelo.
Motsamaisi oa VPS.org o ka fumana ho kena ka SSH root, ka hona o ka kenya drivers, libs le Ollama leha o hloka.
Ha ho server e nang le CUDA runtime kapa NVIDIA driver e kentsoeng pele, le servers tsa GPU ha li fumanehe ho rekisoa. U kenya li-versions Ollama li hloka ho u sebeletsa.
Ha ho joalo. Li-server tsa VPS.org ke li-virtual machines tse tsoelang pele, eseng li-instantane tse sa feleng: li-models tsa Ollama, li-configurations le data li lula ho SSD storage ea server ho fihlela u li tlosa.
E boloka data e sebetsang ka SSD ea mosebeletsi bakeng sa ho kena ka potlako nakong ea Ollama e tsamaeang, le ho kopitsa artifacts e felile (mafura, li-generations, embeddings) ho boloka ka ntle ho VPS.org; snapshots le backups ba lula ka ho boloka cluster e tšoanang.
U ka ntlafatsa mosebedisi leha e le ofe oa VPS.org ho ea ka qeto e kholo ho tloha paneleng ea taolo; ho fetola boholo ho e qala hape'me ho kenya Ollama ho e nka. Mosebedisi oa GPU ha o fumanehe.
Ee. Backups ke ho lefa add-on (ka letsatsi + 30% ea theko ea qeto, e bolokiloe 7 letsatsi; beke le beke + 20%, e bolokiloe 28 letsatsi) le snapshots ke mahala, e leng molemo pele Ollama e telele e tsamaea. Bothata ba lula ka ho tšoanang ho boloka cluster joaloka server.
Ikopanye le ts'ehetso ka matsatsi a 30 a ho reka ho qala ho kopa ho khutlisa. Liserver tsa GPU ha li romelloe, ka hona ha ho na taelo ea GPU e amanang.
GPU e na le bokhoni bo fokolang. Re bolella ka boima ba mosebetsi oa hau'me re tla u araba ka ho fumaneha le theko.