I had a fun time chatting about my local AI experiments with Leo Laporte on his Intelligent Machines show. This show used to be called “This Week in Google.” Also on the show was Father Robert Ballecer (who I bump into at CES frequently) and Jeff Jarvis who I first met when I was working on my journalism startup back in 2009.
Leo was most interested in my latest Local AI experiment that involved re-purposing an old Nvidia Tesla V100 data center card into something that can work on a desktop PC. You can check out my video on that topic here!
I also talked about how good local AI has become for practical applications like document analysis or managing personal data. Local models are effective, grounded, and largely free of hallucinations seen in older models. My favorites at the moment are Gemma 31B and Qwen 3.8 27B. Both are “dense” models that run at usable speeds (around 20-25 tk/s) even with large 96k context windows.
One of the projects I’m working on is building a virtual “Chief of Staff” that manages all of the balls I have in the air running my business, serving my community on the school board and a few other activities I’m involved with. That will be the subject of an upcoming video once I have it working the way I’d like it to work. Stay tuned!