Muna shirya masu ba da sabis na GPU tare da NVIDIA Tesla P40 (24 GB VRAM, Pascal architecture) don gudanar da LLMs masu sauki kamar Llama, Mistral da Gemma tare da Ollama. Masu ba da sabis na GPU ba su da iyaka; tuntuɓi mu don a sanar da mu lokacin da suke.
$ curl -fsSL https://ollama.com/install.sh | sh && ollama run llama3
# Umarnin shigar da misali
Ollama wani kayan aiki ne mai sauƙi don tafiyar da manyan nau'ikan harshe a kan kwamfutarka. A kan mai ba da sabis na GPU, nau'in yana tafiyar da GPU a matsayin CPU, wanda ke sa amsawa su zama da sauri.
Yi amfani da umarnin daya don rufe Llama, Mistral, Gemma, Phi, Qwen da sauran nau'ikan da yawa.
Fits 7B zuwa 13B models comfortably da kusa da 30B models lokacin da kwatanta. 70B models ba su dace da ɗaya 24 GB katin a matsakaicin matsakaicin matsakaici.
Ka tuna cewa bayananka ba su taba barin mai ba da sabis ɗinka ba. Ka yi cikakken iko kan tsarin AI na ka.
Ollama na nuna wani OpenAI-compatible endpoint wanda masu amfani da yawa na yanzu za su iya amfani da shi.
Ollama na iya amfani da NVIDIA GPU don saurin aiki. VPS.org GPU masu aiki ba su da sauki don a umarce su ba; tuntuɓi mu don jin lokacin da suke. Ollama shine farkon LLM-inference / chat workload. Za ka so karantawa masu sauri na random-access daga diski zuwa ajiya da VRAM mai yawa don siffar da kuma tagar yanayin.
An kasa samun masu ba da sabis na GPU don a umarce su. A yau zaka iya tafiyar da Ollama akan wani shirin CPU na yau da kullum, inda zai yi aiki da sauri fiye da a kan GPU: SSH cikin kuma ka yi tafiyar da curl -fsSL https://ollama.com/install.sh | sh && ollama run llama3.
LLM inference VRAM scales tare da model parameters: wani 7B model yana buƙatar kusan 5 zuwa 8 GB na VRAM da 13B model kusan 10 zuwa 14 GB, yayin da 70B-class models suna buƙatar GPUs da yawa ko kwantiza mai nauyi.
Za'a wallafa farashin GPU idan za'a iya umarnin masu ba da sabis na GPU. Ka tuntuɓi mu idan kana so ka ji lokacin da wannan ya faru.
Yanzu, duk mai ba da sabis na VPS.org yana da damar shiga mai amfani da shi, don haka zaka iya tafiyar da wasu kayan aiki tare da Ollama cikin RAM da ajiyar mai ba da sabis.
Every VPS.org server comes with full root SSH access, so you can install drivers, libraries and Ollama however you need.
Ba. Babu mai ba da sabis da ya zo da wani CUDA runtime ko NVIDIA mai sarrafa da aka shigar da shi a gaba, kuma masu ba da sabis na GPU ba su da sauki a umarce su ba har yanzu. Za ka iya shigar da sifofin da Ollama ke bukata kai tsaye.
Ya. VPS.org servers are persistent virtual machines, not ephemeral instances: Ollama models, configurations and data stay on the server's SSD storage until you delete them.
Ka riƙe bayanan aiki a kan SSD na mai ba da sabis domin samun damar sauri yayin da ake tafiyar da Ollama, kuma ka kwafe artifacts da aka gama (matsaloli, shekaru, embeddings) zuwa ajiyar waje VPS.org; snapshots da backups suna zama a kan ƙungiya mai ajiyar guda.
Za ka iya canza kowace masarauta VPS.org zuwa wani shirin da ya fi girma daga fanel na kulawa; sake girman zai sake fara shi kuma shigarwarka ta Ollama za ta yi ta. Masarautar GPU ba ta da sauki ba har yanzu.
Yes. Backups are a paid add-on (daily +30% of the plan price, kept 7 days; weekly +20%, kept 28 days) and snapshots are free, which is useful before long Ollama runs. Both stay on the same storage cluster as the server.
Ka tuntubi goyon baya cikin kwanaki 30 na farkon siyarka don neman dawo da kuɗi. Ba a sayar da masu watsa shirye-shiryen GPU ba har yanzu, don haka babu umarnin GPU da aka haɗa.
GPU na da iyakacin ƙarfin aiki. Ka gaya mana game da nauyin aikinka kuma za mu amsa da damar da farashin.