My cluster has been showing my raspberrypi node as “Ready” but according to the description of the node, the last log was “NodeNotReady” all debug guides say look for any pressure, like disk, pid, or so on, but there is no pressure, no absence of network. Here is the logs of my pi and pi status: https://pastebin.com/UULz6Hcy My pods are stuck in unknown (except jellyfin which is awaiting another node to come on): https://pastebin.com/vw2masAC A description of one of my pods if that helps: https://pastebin.com/s5W03s0E
also i already tried re-installing k3s
These two jumped out at me:
Warning FreeDiskSpaceFailed 35m (x286 over 24h) kubelet Failed to garbage collect required amount of images. Attempted to free 12087084646 bytes, but only found 253243 bytes eligible to free.and
Warning InvalidDiskCapacity 21m/14m kubelet invalid capacity 0 on image filesystemSounds like maybe a disk space issue on your nodes?
Thats weird, because I clearly have free space:
spiderunderurbed@raspberrypi:~ $ df -h / Filesystem Size Used Avail Use% Mounted on /dev/mmcblk0p2 235G 184G 39G 83% / spiderunderurbed@raspberrypi:~ $ df -h . Filesystem Size Used Avail Use% Mounted on /dev/mmcblk0p2 235G 184G 39G 83% /home spiderunderurbed@raspberrypi:~ $any idea of things I can try to fix/debug this?
Sorry, I don’t have as much experience with node management and config, so I’m not sure past that. Aside from focusing on where and how your images are being stored? That’s probably where I’d start first.
/dev/mmcblk0p2
Is that a memory card?
Back in the days when I was using memory cards with my raspi, they had a habit of going to hell, every few months, or sometimes a year.
Now I have a M.2 SSD in one of these beautiful (and painfully expensive) external cases, and no problem anymore.