I’ve recently been experimenting with how generative AI can support Linux debugging. The experience was both impressive and frustrating — depending on whether I was diagnosing or actually fixing a problem.
It started with my Raspberry Pi, which had repeated Wi-Fi connection issues. At first, I suspected the Wi-Fi adapter, but I wasn’t entirely sure. Network issues could be anywhere: in the driver itself, the network setting, the hardware, the access point, interferences between the computer and the access point …
As it is a headless system, I always had to reboot the Raspberry when it lost the connection. So I needed some logs first! I asked an LLM for help in setting up a permanent log configuration, since the default settings on the Pi is that the logs don’t survive a reboot. That part went pleasantly smoothly: the LLM gave me the right configuration and within minutes the logging was set to persistent. Great!
Debugging
The real test came the next day, when the connection problems returned. I dug into the logs with journalctl — which gave me .. quite a lot of system messages. Since I didn’t even know exactly what to look for, I pasted the log into ChatGPT and Mistral (to cross validate), asking them to verify whether the problem was Wi-Fi related. This worked brilliantly. Both models quickly identified the relevant log entries, explained the according error codes, and gave me a clear picture of what was happening. For diagnosis, this was perfect: dumping in logs and letting the AI highlight what matters saved me a lot of time.
Fixing
But when it came to fixing the issue, things took a turn. The AI began guiding me in circles: “try this,” “try that” — even though it already knew from the same chat that I had done those steps already. Even worse: it started recommending drivers from GitHub repos: one didn’t exist anymore, another driver was for a completely different chipset! Had I followed that blindly, I would have ended up breaking my network setup. Only after I challenged the model, it admit confirmed that its advice didn’t fit my hardware and would break the connection — reliably! It also continued recommending disabling drivers etc. without even testing the effects … pretty dangerous on a headless without a fallback plan.
I sense a pattern here …
Unfortunately, this wasn’t the first time I experienced such a behavior and to me, it seems like a pattern: LLMs can be incredibly helpful in diagnosing problems, but dangerously unreliable when it comes to fixing them. They analyze logs well, explain status codes instantly (adapter changed to status=4 = ???) and give you clarity. But if you use them as a guide for actual solutions on the command line, chances are dangerously high that they can mislead you with outdated, wrong, or even harmful instructions — delivered with full confidence.
My takeaway
Use AI for understanding & diagnosing, but not for blindly applying fixes — not even if they are suggested in full confidence! Always, always double-check what you execute! LLMs can be a powerful companion, but they’re a double-edged sword. Use them wisely.
Leave a Reply