Geekmaster 👽:system76:<p>In case anyone wants to run their own <a href="https://ioc.exchange/tags/LLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LLM</span></a> on their personal devices, I can suggest <a href="https://ioc.exchange/tags/lmstudio" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>lmstudio</span></a> It allows you to download any major model that's out there to your own laptop/desktop, and then you can use it's Graphical Interface to interact with it in a familiar interface environment, as you would <a href="https://ioc.exchange/tags/Claude" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Claude</span></a> or <a href="https://ioc.exchange/tags/ChatGPT" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ChatGPT</span></a> or <a href="https://ioc.exchange/tags/Gemini" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Gemini</span></a>. It will also let you know if your device has enough resources (power) to run a specific model variant: <a href="https://lmstudio.ai/" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">lmstudio.ai/</span><span class="invisible"></span></a></p><p>For those of you who love the Terminal (aka: Command Line Interface or <a href="https://ioc.exchange/tags/CLI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>CLI</span></a>), check out <a href="https://ollama.com/" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">ollama.com/</span><span class="invisible"></span></a></p><p>For starters, I suggest the <a href="https://ioc.exchange/tags/gemma3n" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>gemma3n</span></a> model (works great on tablets, laptops, or phones), or <a href="https://ioc.exchange/tags/llama3" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>llama3</span></a>.1 for the most common interactions. While most of us geeks will have very powerful personal systems, or servers in our home racks, most people do not have high-end systems/devices. The gemma3n model is lightweight, very powerful, and a solid general purpose LLM.</p><p>What's the benefit of running LLM's locally? <a href="https://ioc.exchange/tags/Privacy" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Privacy</span></a> is a big one (it's running on your local machine, not a cloud server) - so you can ask it questions against sensitive business data, PHI/PII, etc.. You can also run it <a href="https://ioc.exchange/tags/offline" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>offline</span></a> (no Internet connection required) so if you wanted to <a href="https://ioc.exchange/tags/airgap" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>airgap</span></a> your interactions, or play with it on vacation (on a plane, lost in the back country, etc.), you can absolutely do that - even with the Deepseek model.</p><p>I started playing with both LM Studio and ollama myself - been asking models basic questions like "what is the capital of Italy" and also more complex questions like "write me a <a href="https://ioc.exchange/tags/powershell" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>powershell</span></a> script to add users of a specific OU to a number of Security Groups within AD" and so far, it's been very accurate. The PowerShell script llama3.1 provided worked out of the box (after I revised variables to match my environment).</p><p><a href="https://ioc.exchange/tags/AIForward" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AIForward</span></a> <a href="https://ioc.exchange/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AI</span></a> <a href="https://ioc.exchange/tags/AiResearch" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AiResearch</span></a> <a href="https://ioc.exchange/tags/FunWithAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>FunWithAI</span></a></p>