Just sharing with my testing of my second Zimaboard 2 with installed Aoostar eGPU and 4060ti 8gb graphics card. Using Ollama Nvidia GPU app.
How effective was it?
The initial prompt takes almost a minute to warm up before responding.
I am planning to buy a 4060 to 16GB is it worth buying for local LLM or should I look for other options.
Try to look for other options. The only reason Iām using 4060ti 8gb at the moment because its a spare from my old computer.
Hi, I have tried something similar with the same eGPU setup. But I have problems getting responses. I am loading a small model, and accessing it through WebUI, all installed through the app store. A query to the LLM starts thinking, the thinking output is visible, but then I get a time out error. When I try another query after re-starting both processes, the dot just pulses, and there is no output. Any ideas about how to fix this?


