Loading…
12
I swapped our support chatbot from a big hosted model to a small open one running on a $400 mini PC in our Denver office
Six weeks back I had to pick between paying about $800 a month for an API plan or running an open model myself on a cheap mini PC we already owned. I went with the local box because our ticket volume is small and the data stays inside the office. It took two weekends to tune the prompts and the answers got sharper once I fed it our old resolved tickets. The hosted setup was smarter out of the gate, but for this job the gap was not worth the bill. Anyone here running small models on their own hardware for real support work, and what broke for you?
0 comments
Log in to join the discussion
Log In0 Comments
No comments yet
Be the first to share your thoughts on this discussion.