Recently, both OpenAI Translator and NextChat have begun to support large-scale language models running locally in Ollama, which adds a new way of playing for "newbies" enthusiasts.
Moreover, the launch of Ollama on Windows (preview version) has completely subverted the way of AI development on Windows devices. It has guided a clear path for explorers in the field of AI and ordinary "test players".
Ollama is a groundbreaking artificial intelligence (AI) and machine learning (ML) tool platform that dramatically simplifies the development and use of AI models.
In the technical community, the hardware configuration and environment construction of AI models have always been a thorny issue, and Ollama emerged to solve such critical needs:
Among many AI tools, Ollama stands out with the following key advantages. These features not only highlight its uniqueness, but also solve the most common problems encountered by AI developers and enthusiasts:
Through these carefully designed features, Ollama not only solves common problems in AI development, but also allows more people to easily access and apply advanced AI technology, greatly expanding the application prospects of AI.
Welcome to the new era of AI and ML! Next, we'll take you through every step of getting started, and we'll also provide some practical code and command examples to make sure you have a smooth journey.
1Visit the Ollama Windows Preview page and download the OllamaSetup.exe
installation program.
2 Double-click the file and click "Install" to start the installation.
3After the installation is completed, you can start using Ollama on Windows. Isn’t it very simple?
To launch Ollama and get an open source AI model from the model library, follow these steps:
1 Click the Ollama icon in the "Start" menu. After running, an icon will reside in the taskbar tray.
2 Right-click the taskbar icon and select "View log" to open the command line window.
3Execute the following command to run Ollama and load the model:
ollama run [modelname]
After executing the above command, Ollama will start to initialize and automatically pull and load the selected model from the Ollama model library. Once it's ready, you can send it instructions and it will understand and respond using the chosen model.
Remember to replace the modelname
name with the name of the model to be run. Commonly used ones are:
Model | parameter | size | Installation command | Publishing Organization |
---|---|---|---|---|
Llama 2 | 7B | 3.8GB | ollama run llama2 |
Meta |
Code Llama | 7B | 3.8GB | ollama run codellama |
Meta |
Llama 2 13B | 13B | 7.3GB | ollama run llama2:13b |
Meta |
Llama 2 70B | 70B | 39GB | ollama run llama2:70b |
Meta |
Mistral | 7B | 4.1GB | ollama run mistral |
Mistral AI |
mixtral | 8x7b | 26GB | ollama run mixtral:8x7b |
Mistral AI |
Phi-2 | 2.7B | 1.7GB | ollama run phi |
Microsoft Research |
LLaVA | 7B | 4.5GB | ollama run llava |
Microsoft Research Columbia University Wisconsin |
Gemma 2B | 2B | 1.4GB | ollama run gemma:2b |
|
Gemma 7B | 7B | 4.8GB | ollama run gemma:7b |
|
Qwen 4B | 4B | 2.3GB | ollama run qwen:4b |
Alibaba |
Qwen 7B | 7B | 4.5GB | ollama run qwen:7b |
Alibaba |
Qwen 14B | 14B | 8.2GB | ollama run qwen:14b |
Alibaba |
运行 7B 至少需要 8GB 内存,运行 13B 至少需要 16GB 内存。
如前所述,Ollama 支持通过各种各样的开源模型来完成不同的任务,下面就来看看怎么使用。
ollama run llava1.6
Ollama 会使用你选择的模型来分析这张图片,并给你一些结果,比如图片的内容和分类,图片是否有修改,或者其他的分析等等(取决于所使用的模型)。
我们不可能只通过命令行来使用,将应用程序连接到 Ollama API 是一个非常重要的步骤。这样就可以把 AI 的功能整合到自己的软件里,或者在 OpenAI Translator 和 NextChat 这类的前端工具中进行调用。
以下是如何连接和使用 Ollama API 的步骤:
http://localhost:11434
,可以在安装 Ollama 的系统中直接调用。1右击点击任务栏图标,选择「Quit Ollama」退出后台运行。
2使用Windows + R
快捷键打开「运行」对话框,输出以下命令,然后按Ctrl + Shift + Enter
以管理员权限启动「环境变量」。
C:Windowssystem32rundll32.exe sysdm.cpl, EditEnvironmentVariables
3要更改侦听地址和端口,可以添加以下环境变量:
OLLAMA_HOST
:8000
只填写端口号可以同时侦听(所有) IPv4 和 IPv6 的:8000
端口。
要使用 IPv6,需要 Ollama 0.0.20 或更新版本。
4如果安装了多个模型,可以通过OLLAMA_MODELS
变量名来指定默认模型。
5更改完之后,重新运行 Ollama。然后在浏览器中测试访问,验证更改是否成功。
6示例 API 调用: 要使用 Ollama API,可以在自己的程序里发送 HTTP 请求。下面是在「终端」里使用curl
命令给 Gemma 模型发送文字提示的例子:
curl http://192.168.100.10:8000/api/generate -d '{ "model": "gemma:7b", "prompt": "天空为什么是蓝色的?" }'
返回响应的格式,目前只支持 Json 格式。
# 查看 Ollama 版本 ollama -v # 查看已安装的模型 ollama list # 删除指定模型 ollama rm [modelname] # 模型存储路径 # C:Users\.ollamamodels
按照上述步骤,并参考命令示例,你可以在 Windows 上尽情体验 Ollama 的强大功能。不管是在命令行中直接下达指令,通过 API 将 AI 模型集成到你的软件当中,还是通过前端套壳,Ollama 的大门都已经为你敞开。
要让 Ollama 在 Windows 上充分发挥最大潜力,需要注意以下几点最佳实践和技巧,这将帮助你优化性能并解决一些常见问题:
In this tutorial, we learned how to install and use Ollama on Windows, including installing Ollama, executing basic commands, using the Ollama model library, and connecting to Ollama through the API. I recommend you dig into Ollama and try out a variety of different models.
Ollama has unlimited potential, and with it, you can achieve more!
The above is the detailed content of Windows on Ollama: A new tool for running large language models (LLM) locally. For more information, please follow other related articles on the PHP Chinese website!