How to use Olllama models and open webui

Operating System ZimaOS
1

I have ollama and open webui on different computers so I thought why not on Zima OS
but is no documentation how to install a model and have open webui app see it, the ollama is the gpu app
one person said
(from SSH)

run sudo docker ps
copy the ID for ollama
run sudo docker exec

“docker exec” requires at least 2 arguments.
See ‘docker exec --help’.

Usage: docker exec [OPTIONS] CONTAINER COMMAND [ARG…]

Execute a command in a running container

No one on the discord channel has helped or knows how to do this, and why is there no documentation

SSH reports also a lot things are not installed so what are the purpose of the apps if they don’t install

On windows or linux docker you can install both and in open webui olllama run modelname but not Zimaos app

I really hope someone can help maybe i need another NAS server Zima is not ready yet

i ashared the error

2

https://www.youtube.com/watch?v=88h0XVBNVlk Can this help you？

3

no when i try like he did to download a model in open webui nothing happens
also i noted his zima os sees his gpu mine doesnt

4

If you want open webui works with a seperate Ollama, then this is helpful:

On ZimaOS, for now, you need to remove the current ollama or opwn WebUI first and install the nvidia GPU and at last you install the Open WebUi to leverage the GPU.

1、remove possible installed Open WebUI
2、install the GPU
3、install the Open WebUI

PS.
Removal of pre-installed open-webui app at first (to use the GPU) may be fixed in the future.

5

This doese not work, you have apps that do nothing, open webui you cannot add ollama models

I hope this helps someone else
I intalled linux mint , then made sure it recognized the Nvidia gpu
All through the linux terminal:
Chatgpt helped me get docker to use the gpu
then i installed casaos because zimaos is standalone and still not working i hope someday zima will let us install like casaos
anyways after that I installed with terminal ollama and then open webui
now in open webui i can download a model

6

In fact, you can pull models directly in Open WebUI, which includes Ollama itself. This is shown in our video:

7

@mytechweb_41856 - I have ‘Ollama-Nvidia’ and ‘Open WebUI’ apps on ZimaOS installed on an old Lenovo server. Ollama is using my 3060 GPU and Open WebUI sees the Ollama instance on the server. I can access the server from any browser on my LAN via Open WebUI port 3050 and select a model from Ollama in the Open WebUI interface. It all works very well for me for text inputs. I have not set up speech inputs as I’d need to set up certificates within my browser. Therefore, I have a separate machine with Home Assistant running on it that accesses the Ollama on the server for speech chat LLM queries.
If this is the sort of set up that you are after, let me know and I will share the settings I have in the hope that you may find them useful.
Regards, Denzil.

1 Like
8

Would love your setup

9

Intel GPU does not works with Open WebUI, how can it be fixed?

10

Thanks Animatco. Whilst not ‘cheap’ it was relatively inexpensive for the amount of opportunity that having the ZimaOS environment brings. The Lenovo P520 was about £250 used and a new RTX3060 was about £250 in the UK. ZimaOS installed flawlessly and has been running nearly 24/7 for the last 9 months. With the app library on ZimaOS pretty extensive and growing, there are ample opportunities to experiment with all sorts of home-labbing and self-hosting. I’ve moved my iCloud photos to Immich on my ZimaOS and have it all backed up on a separate drive. My music collection is in Jellyfin, again, all self-hosted. My notes are in Joplin. The in-built Files app is great and super fast to drag/drop and share large amounts of data between machines on my home network. Plus the Ollama AI stuff is super useful for everyday information from fixing my car, providing me with tech support, helping the kids with their homework etc. Not paying Apple £30/mth or OpenAI £20/mth means that the entire cost of the hardware and system is paid for within 10mths.
My only issue is that I’ve not been able to support the ZimaOS team by buying some hardware from them yet - I feel a bit bad about that! - but, when I can find a use case and have some cash free (that the kids haven’t got their hands on!) I’ll try and pick up one of their Zima blades to have a play with. Once you start this home-lab stuff, it’s pretty addictive and a lot easier to work with than trying to run Docker in a Linux CLI!
Good luck with your home-lab journey :slight_smile:

11

Sorry Mtbx, I have no experience with an Intel card. If you have not seen this:GPU Expansion | Zimaspace Docs - it may be helpful. Good luck.