https://blog.alekc.org/tags/etcd/Recent content in etcd on Alekc's BlogHugo -- gohugo.io©{year}, All Rights ReservedSat, 03 Sep 2022 23:00:00 +0000https://blog.alekc.org/posts/fixing-etcddatabasehighfragmentationratio-prometheus-alert/Sat, 03 Sep 2022 23:00:00 +0000https://blog.alekc.org/posts/fixing-etcddatabasehighfragmentationratio-prometheus-alert/If you receive a following error in alertmanager 1etcd cluster &#34;kube-etcd&#34;: database size in use on instance xxx.xxx.xxx.xxx:2381 2is 49.55% of the actual allocated disk space, please run defragmentation 3(e.g. etcdctl defrag) to retrieve the unused fragmented disk space. it can be easily solved by running a following command: 1kubectl exec $(kubectl get pods --selector=component=etcd -A -o name \ 2| head -n 1) -n kube-system -- etcdctl defrag --cluster \ 3--cacert /etc/kubernetes/pki/etcd/ca.https://blog.alekc.org/posts/2021/recovery-of-etcd-failing-node/Thu, 14 Jan 2021 16:40:59 +0000https://blog.alekc.org/posts/2021/recovery-of-etcd-failing-node/<p>One of my etcd nodes in my home k8s cluster has been failing with following message:</p> <div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span>2021-01-14 11:16:09.233458 I | embed: listening for peers on 192.168.0.33:2380 <span class="ln">2</span>raft2021/01/14 11:16:09 tocommit(29492601) is out of range [lastIndex(29492469)]. Was the raft log corrupted, truncated, or lost? <span class="ln">3</span>panic: tocommit(29492601) is out of range [lastIndex(29492469)]. Was the raft log corrupted, truncated, or lost? </code></pre></div><p>These are steps I took to fix it:</p>