Tag: operation


etcd 故障排查之 `snapshotting is taking more than x seconds to finish`

查看 etcd 运行日志,如果看到如下日志: snapshotting is taking more than x seconds to finish … 当发送一个snapshot超过30s并且在1Gbps(千兆)网络环境下使用时间超过一定时间时,etcd就会打印这个日志进行告警。

继续阅读

etcd 故障排查之 `request ignored (cluster ID mismatch)`

查看 etcd 运行日志,如果看到如下日志: request ignored (cluster ID mismatch) 是因为 cluster token 不一致,新成员接收到请求后会报出这个warning。查看官方问答,解释如下: Every new etcd cluster

继续阅读

etcd 故障排查之 `failed to send out heartbeat on time`

查看 etcd 运行日志,如果看到如下日志: 08:52:05.164847 W | etcdserver: failed to send out heartbeat on time 说明 etcd 发送心跳有问题了,查看官方问答,解释如下: etcd uses a leader

继续阅读

etcd 故障排查之 `etcdserver apply entries took too long`

经常去看看 etcd 运行日志,如果 etcd 负载比较高,或者节点规格较差,往往能看到如下类似日志: 08:52:05.164847 W | etcdserver: apply entries took too long [140.696147ms for 1 entries] 0

继续阅读

etcd 故障排查之 `the clock difference against peer xxx is too high [xxxs > 1s]`

协助排查 etcd 的一个问题,出现如下日志: 2018-05-16 12:38:59.796724 W | rafthttp: the clock difference against peer e7e21c67737845ce is too high [3.370772704s &g

继续阅读
Bingo Huang