Confirm before forwarding to LLM and native Home Assistant forwarding

krove
Oct 2026
0 posts
0 likes
Member
Confirm before forwarding to LLM and native Home Assistant forwarding
Posted Fri Nov 06th 16:56• Last edited Fri Nov 06th 16:59

I like the ability to forward commands that are not understood to an LLM, it gives Sophia both speed and flexibility however at the moment the only way to know when Sophia is forwarding to AI is to set a specific command. It would be preferable if instead it would ask something like "I didn't understand, would you like me to ask custom ai name here?" That way it is clear when your commands were misunderstood.

On the topic of forwarding to LLM, it would be nice if instead of having to select an LLM from the available list and putting in the api key, you selected a home assistant voice assistant pipeline to fallback to. For instance, if you had 2 Home Assistant assist pipelines, one for Sophia NLU, and another for ollama, with Sophia as the default, when you ask the voice assistant for something, if it fails it then goes to the fallback pipeline which is Ollama. This would allow all your devices to be exposed (which is not possible at the moment), as well as giving access to any LLM setup through Home Assistant, not just the LLMs built into the Sophia NLU HA App. It also makes maintaining the code easier as you no longer need to update the app when companies (probably google) change how their API works.

taunt7872
Nov 2026
2 posts
0 likes
Member
Posted Sat Nov 07th 05:29

Like button doesn't seem to be working for some reason - so I'm commenting to say "This is a great idea - would love to see it!" :)

Though I do like that Sophia handles the LLM interaction instead of HASS - I'm quite frustrated with the call/response time when HA calls a local model, almost entirely due to all of the context it provides and the model has to parse. Would love to have the LLM be specifically for "outside of this house" questions. Though I suppose having the handoff function isn't a bad thing so long as the current method isn't abandoned in favor of it 😅

Matt
Apr 2025
94 posts
0 likes
Member
Posted Sat Nov 07th 06:41

I don't want to do that as I know it would cause a decent amount of confusion and support tickets. That, and it defeats the purpose of Sophia.

If any sentences are not working as expected, right after sending the sentences in send in "no, that's wrong". That will log the sentence, and once you have all the desired sentences logged visit the Log menu of the Sophia web interface and hit that upload button. Then drop me an e-mail to let me know you have pending misinterpretatoins you'd like me to look at, and I'll happily do so and get back to you.

Quick Reply

You must be logged in to reply to this thread.

Login Register