Nicolas Mowen is switching to a fully local assist backed by local first + llama.cpp (previously Ollama) solution for a voice assistant.
I will share the steps I took to get to where I am today, the decisions I made and why they were the best for my use case specifically.
I have tested a wide variety of hardware from a 3050 to a 3090, most modern discrete GPUs can be used for local assist effectively, it just depends on your expectations of capability and speed for what hardware is required.
I am running Home Assistant on my UnRaid NAS, specs are not really important as it has nothing to do with HA Voice.
Voice Server Hardware is a Beelink MiniPC with USB4 (the exact model isn’t important as long as it has USB4) and a USB4 eGPU enclosure.
Various models are tested for effectiveness.
The End Result
I definitely would not recommend this for the average Home Assistant user, IMO a lot of patience and research is needed to understand particular problems and work towards a solution, and I imagine we will run into more problems as we continue to use these. I am certainly not done, but that is the beauty of this solution – most aspects of it can be tuned.
The goal has been met though, overall we have a more enjoyable voice assistant that runs locally without privacy concerns, and our core tasks are handled reliably.
Check out the details in the post here.
from Adafruit Industries – Makers, hackers, artists, designers and engineers! https://ift.tt/L1x2mZW
via IFTTT
Комментариев нет:
Отправить комментарий