I’ve been using Ollama with Open WeUI as the frontend for quite awhile now and it’s been a pretty great combo. The Open WebUI team releases what seems like nearly weekly updates adding great new features all the time. It works amazing with Ollama as the backend inference server, and I love Open WebUi’s Docker / Watchtower setup which makes updates to Open WebUI completely automatic.
Now that my RAG chat setup is working well, I decided that I wanted to make it securely remotely accessible from my phone.
I have a Firewalla Gold firewall appliance at home and I know that it has a super simple to set up OpenVPN server built in, so I thought that would be a good way to securely connect to my Ollama / Open WebUi server (much more securely than just using basic port forwarding). It’s important to note that this is a VPN server (not a client).
I followed the instructions in this video; https://youtu.be/edoRRBGTja0?si=JNgpVjA0Qk_UPF9_ to setup the OpenVPN server on my Firewalla appliance. If you don’t have a Firewalla, OpenVPN server is also offered on many home routers, I’m sure there are easy instructions for every router that supports it. It’s free to use and doesn’t cost anything.
After I set up the server, I then downloaded the OpenVPN app for iOS (free) and installed the profile / certificate as described in the video.
Once connected to the OpenVPN server via the OpenVPN server app, I’m effectively securely inside my home network. Now, I just open a browser on my phone and put the internal IP of my Open WebUI server, add port 3000 at the end of the IP, and I’m now using my local Open WebUI server from my phone over a private and encrypted VPN tunnel.
I then choose the “save to Home Screen” option in Safari and it saves it as an icon on my home screen. Now Open WebUI looks and functions completely like a native iPhone application (modal with no browser bar).
Anytime I want to use my private Open WebUi, I just open the OpenVPN iOS app, tap connect, and then open the Open WebUI app. That’s it!
I can upload docs directly from my phone and use them in RAG prompts and it’s all encrypted and private thanks to the OpenVPN server. The response time from Open WebUI in this remote mode is great, it’s just like I was on the box locally. The whole OpenVPN set up process took like 5 minutes and I did it all on my phone. Note: be sure when you’re testing it out in your phone that you’re not already on your internal network (turn off WiFi and only use cellular connection) when you connect from the OpenVPN app.
Has anyone else implemented a similar setup? Is there a better, more secure implementation out there for accessing local RAG chat remotely?
💬 Discussion r/LocalLLaMA (11 points, 9 commentaires)