Why doesn't etcd start? #15837

xiaofanshifu · 2023-05-06T16:08:07Z

xiaofanshifu
May 6, 2023

Why doesn't etcd start?

v3.5.2

v3.5.8

Which configuration do I need to adjust?

jmhbnz · 2023-05-06T17:13:39Z

jmhbnz
May 6, 2023
Maintainer

Hi @thissuper - The error message couldn't find local name says to me that there is maybe an issue with finding the member name in the initial-cluster config. I notice you have that specified almost dictionary style with separate lines like:

initial-cluster:
  etcd01=<url:port>,
  etcd02=<url:port>,
  etcd03=<url:port>

Would you mind trying it as one string, i.e:

initial-cluster: etcd01=<url:port>,etcd02=<url:port>,etcd03=<url:port>

If that doesn't help please paste the configuration as text values, it's very difficult to try and replicate anything when they are screenshots. Thanks! 🙏🏻

1 reply

xiaofanshifu May 7, 2023
Author

The configuration was changed, but I ran into another problem.🙏🏻

[root@etcd03 ~]# cat /etc/etcd/etcd.yml
# [Member]
name: 'etcd03'
data-dir: '/var/lib/etcd'
listen-peer-urls: http://10.17.35.83:2380
listen-client-urls: http://10.17.35.83:2379
#[Clustering]
initial-advertise-peer-urls: http://10.17.35.83:2380
advertise-client-urls: http://10.17.35.83:2379
initial-cluster: etcd01=http://10.17.35.81:2380,etcd02=http://10.17.35.82:2380,etcd03=http://10.17.35.83:2380
initial-cluster-token: 'etcd-cluster'
initial-cluster-state: 'new'
[root@etcd03 ~]#
[root@etcd03 ~]# ssh -v -p 2380 10.17.35.82
OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 58: Applying options for *
debug1: Connecting to 10.17.35.82 [10.17.35.82] port 2380.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: identity file /root/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.4
debug1: ssh_exchange_identification: HTTP/1.1 400 Bad Request


debug1: ssh_exchange_identification: Content-Type: text/plain; charset=utf-8


debug1: ssh_exchange_identification: Connection: close


debug1: ssh_exchange_identification:


ssh_exchange_identification: Connection closed by remote host
[root@etcd03 ~]#

etcd03 log

{"level":"warn","ts":"2023-05-07T10:14:09.269949+0800","caller":"embed/config.go:673","msg":"Running http and grpc server on single port. This is not recommended for production."}
{"level":"info","ts":"2023-05-07T10:14:09.270504+0800","caller":"etcdmain/config.go:350","msg":"loaded server configuration, other configuration command line flags and environment variables will be ignored if provided","path":"/etc/etcd/etcd.yml"}
{"level":"info","ts":"2023-05-07T10:14:09.270556+0800","caller":"etcdmain/etcd.go:73","msg":"Running: ","args":["/usr/local/bin/etcd","--config-file=/etc/etcd/etcd.yml"]}
{"level":"info","ts":"2023-05-07T10:14:09.270668+0800","caller":"etcdmain/etcd.go:116","msg":"server has been already initialized","data-dir":"/var/lib/etcd","dir-type":"member"}
{"level":"warn","ts":"2023-05-07T10:14:09.270754+0800","caller":"embed/config.go:673","msg":"Running http and grpc server on single port. This is not recommended for production."}
{"level":"info","ts":"2023-05-07T10:14:09.2708+0800","caller":"embed/etcd.go:127","msg":"configuring peer listeners","listen-peer-urls":["http://10.17.35.83:2380"]}
{"level":"info","ts":"2023-05-07T10:14:09.270983+0800","caller":"embed/etcd.go:135","msg":"configuring client listeners","listen-client-urls":["http://10.17.35.83:2379"]}
{"level":"info","ts":"2023-05-07T10:14:09.271353+0800","caller":"embed/etcd.go:309","msg":"starting an etcd server","etcd-version":"3.5.8","git-sha":"217d183e5","go-version":"go1.19.8","go-os":"linux","go-arch":"amd64","max-cpu-set":4,"max-cpu-available":4,"member-initialized":true,"name":"etcd03","data-dir":"/var/lib/etcd","wal-dir":"","wal-dir-dedicated":"","member-dir":"/var/lib/etcd/member","force-new-cluster":false,"heartbeat-interval":"100ms","election-timeout":"1s","initial-election-tick-advance":true,"snapshot-count":100000,"max-wals":5,"max-snapshots":5,"snapshot-catchup-entries":5000,"initial-advertise-peer-urls":["http://10.17.35.83:2380"],"listen-peer-urls":["http://10.17.35.83:2380"],"advertise-client-urls":["http://10.17.35.83:2379"],"listen-client-urls":["http://10.17.35.83:2379"],"listen-metrics-urls":[],"cors":["*"],"host-whitelist":["*"],"initial-cluster":"","initial-cluster-state":"new","initial-cluster-token":"","quota-backend-bytes":2147483648,"max-request-bytes":1572864,"max-concurrent-streams":4294967295,"pre-vote":true,"initial-corrupt-check":false,"corrupt-check-time-interval":"0s","compact-check-time-enabled":false,"compact-check-time-interval":"1m0s","auto-compaction-mode":"","auto-compaction-retention":"0s","auto-compaction-interval":"0s","discovery-url":"","discovery-proxy":"","downgrade-check-interval":"5s"}
{"level":"info","ts":"2023-05-07T10:14:09.274028+0800","caller":"etcdserver/backend.go:81","msg":"opened backend db","path":"/var/lib/etcd/member/snap/db","took":"1.88674ms"}
{"level":"info","ts":"2023-05-07T10:14:09.274548+0800","caller":"etcdserver/server.go:530","msg":"No snapshot found. Recovering WAL from scratch!"}
{"level":"info","ts":"2023-05-07T10:14:09.277953+0800","caller":"etcdserver/raft.go:530","msg":"restarting local member","cluster-id":"d20920a0fe99cde","local-member-id":"81c47c2838f502bb","commit-index":13}
{"level":"info","ts":"2023-05-07T10:14:09.278176+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"81c47c2838f502bb switched to configuration voters=()"}
{"level":"info","ts":"2023-05-07T10:14:09.278409+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"81c47c2838f502bb became follower at term 3"}
{"level":"info","ts":"2023-05-07T10:14:09.278588+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"newRaft 81c47c2838f502bb [peers: [], term: 3, commit: 13, applied: 0, lastindex: 13, lastterm: 3]"}
{"level":"warn","ts":"2023-05-07T10:14:09.279835+0800","caller":"auth/store.go:1238","msg":"simple token is not cryptographically signed"}
{"level":"info","ts":"2023-05-07T10:14:09.281027+0800","caller":"mvcc/kvstore.go:393","msg":"kvstore restored","current-rev":1}
{"level":"info","ts":"2023-05-07T10:14:09.281881+0800","caller":"etcdserver/quota.go:94","msg":"enabled backend quota with default value","quota-name":"v3-applier","quota-size-bytes":2147483648,"quota-size":"2.1 GB"}
{"level":"info","ts":"2023-05-07T10:14:09.282782+0800","caller":"etcdserver/server.go:854","msg":"starting etcd server","local-member-id":"81c47c2838f502bb","local-server-version":"3.5.8","cluster-version":"to_be_decided"}
{"level":"info","ts":"2023-05-07T10:14:09.282951+0800","caller":"etcdserver/server.go:754","msg":"starting initial election tick advance","election-ticks":10}
{"level":"info","ts":"2023-05-07T10:14:09.283987+0800","caller":"fileutil/purge.go:44","msg":"started to purge file","dir":"/var/lib/etcd/member/snap","suffix":"snap.db","max":5,"interval":"30s"}
{"level":"info","ts":"2023-05-07T10:14:09.284152+0800","caller":"fileutil/purge.go:44","msg":"started to purge file","dir":"/var/lib/etcd/member/snap","suffix":"snap","max":5,"interval":"30s"}
{"level":"info","ts":"2023-05-07T10:14:09.284211+0800","caller":"fileutil/purge.go:44","msg":"started to purge file","dir":"/var/lib/etcd/member/wal","suffix":"wal","max":5,"interval":"30s"}
{"level":"info","ts":"2023-05-07T10:14:09.28435+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"81c47c2838f502bb switched to configuration voters=(6439254626089771960)"}
{"level":"info","ts":"2023-05-07T10:14:09.284651+0800","caller":"membership/cluster.go:421","msg":"added member","cluster-id":"d20920a0fe99cde","local-member-id":"81c47c2838f502bb","added-peer-id":"595cd3f73e5f23b8","added-peer-peer-urls":["http://10.17.35.82:2380"]}
{"level":"info","ts":"2023-05-07T10:14:09.284797+0800","caller":"rafthttp/peer.go:133","msg":"starting remote peer","remote-peer-id":"595cd3f73e5f23b8"}
{"level":"info","ts":"2023-05-07T10:14:09.284902+0800","caller":"rafthttp/pipeline.go:72","msg":"started HTTP pipelining with remote peer","local-member-id":"81c47c2838f502bb","remote-peer-id":"595cd3f73e5f23b8"}
{"level":"info","ts":"2023-05-07T10:14:09.285907+0800","caller":"rafthttp/stream.go:169","msg":"started stream writer with remote peer","local-member-id":"81c47c2838f502bb","remote-peer-id":"595cd3f73e5f23b8"}
{"level":"info","ts":"2023-05-07T10:14:09.287901+0800","caller":"rafthttp/stream.go:169","msg":"started stream writer with remote peer","local-member-id":"81c47c2838f502bb","remote-peer-id":"595cd3f73e5f23b8"}
{"level":"info","ts":"2023-05-07T10:14:09.290903+0800","caller":"rafthttp/peer.go:137","msg":"started remote peer","remote-peer-id":"595cd3f73e5f23b8"}
{"level":"info","ts":"2023-05-07T10:14:09.291032+0800","caller":"rafthttp/transport.go:317","msg":"added remote peer","local-member-id":"81c47c2838f502bb","remote-peer-id":"595cd3f73e5f23b8","remote-peer-urls":["http://10.17.35.82:2380"]}
{"level":"info","ts":"2023-05-07T10:14:09.291179+0800","caller":"rafthttp/stream.go:395","msg":"started stream reader with remote peer","stream-reader-type":"stream Message","local-member-id":"81c47c2838f502bb","remote-peer-id":"595cd3f73e5f23b8"}
{"level":"info","ts":"2023-05-07T10:14:09.291313+0800","caller":"embed/etcd.go:278","msg":"now serving peer/client/metrics","local-member-id":"81c47c2838f502bb","initial-advertise-peer-urls":["http://10.17.35.83:2380"],"listen-peer-urls":["http://10.17.35.83:2380"],"advertise-client-urls":["http://10.17.35.83:2379"],"listen-client-urls":["http://10.17.35.83:2379"],"listen-metrics-urls":[]}
{"level":"info","ts":"2023-05-07T10:14:09.291562+0800","caller":"embed/etcd.go:597","msg":"serving peer traffic","address":"10.17.35.83:2380"}
{"level":"info","ts":"2023-05-07T10:14:09.291601+0800","caller":"embed/etcd.go:569","msg":"cmux::serve","address":"10.17.35.83:2380"}
{"level":"info","ts":"2023-05-07T10:14:09.291753+0800","caller":"rafthttp/stream.go:395","msg":"started stream reader with remote peer","stream-reader-type":"stream MsgApp v2","local-member-id":"81c47c2838f502bb","remote-peer-id":"595cd3f73e5f23b8"}
{"level":"info","ts":"2023-05-07T10:14:09.291954+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"81c47c2838f502bb switched to configuration voters=(6439254626089771960 9350735238524109499)"}
{"level":"info","ts":"2023-05-07T10:14:09.292107+0800","caller":"membership/cluster.go:421","msg":"added member","cluster-id":"d20920a0fe99cde","local-member-id":"81c47c2838f502bb","added-peer-id":"81c47c2838f502bb","added-peer-peer-urls":["http://10.17.35.83:2380"]}
{"level":"info","ts":"2023-05-07T10:14:09.292347+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"81c47c2838f502bb switched to configuration voters=(6439254626089771960 9350735238524109499 12206494669637913953)"}
{"level":"info","ts":"2023-05-07T10:14:09.292467+0800","caller":"membership/cluster.go:421","msg":"added member","cluster-id":"d20920a0fe99cde","local-member-id":"81c47c2838f502bb","added-peer-id":"a9662e3ab0778d61","added-peer-peer-urls":["http://10.17.35.81:2380"]}
{"level":"info","ts":"2023-05-07T10:14:09.292523+0800","caller":"rafthttp/peer.go:133","msg":"starting remote peer","remote-peer-id":"a9662e3ab0778d61"}
{"level":"info","ts":"2023-05-07T10:14:09.292597+0800","caller":"rafthttp/pipeline.go:72","msg":"started HTTP pipelining with remote peer","local-member-id":"81c47c2838f502bb","remote-peer-id":"a9662e3ab0778d61"}
{"level":"info","ts":"2023-05-07T10:14:09.293311+0800","caller":"rafthttp/stream.go:169","msg":"started stream writer with remote peer","local-member-id":"81c47c2838f502bb","remote-peer-id":"a9662e3ab0778d61"}
{"level":"info","ts":"2023-05-07T10:14:09.293423+0800","caller":"rafthttp/stream.go:169","msg":"started stream writer with remote peer","local-member-id":"81c47c2838f502bb","remote-peer-id":"a9662e3ab0778d61"}
{"level":"info","ts":"2023-05-07T10:14:09.294544+0800","caller":"rafthttp/peer.go:137","msg":"started remote peer","remote-peer-id":"a9662e3ab0778d61"}
{"level":"info","ts":"2023-05-07T10:14:09.294614+0800","caller":"rafthttp/stream.go:395","msg":"started stream reader with remote peer","stream-reader-type":"stream Message","local-member-id":"81c47c2838f502bb","remote-peer-id":"a9662e3ab0778d61"}
{"level":"info","ts":"2023-05-07T10:14:09.29467+0800","caller":"rafthttp/stream.go:395","msg":"started stream reader with remote peer","stream-reader-type":"stream MsgApp v2","local-member-id":"81c47c2838f502bb","remote-peer-id":"a9662e3ab0778d61"}
{"level":"info","ts":"2023-05-07T10:14:09.294648+0800","caller":"rafthttp/transport.go:317","msg":"added remote peer","local-member-id":"81c47c2838f502bb","remote-peer-id":"a9662e3ab0778d61","remote-peer-urls":["http://10.17.35.81:2380"]}
{"level":"info","ts":"2023-05-07T10:14:09.295352+0800","caller":"rafthttp/peer_status.go:53","msg":"peer became active","peer-id":"595cd3f73e5f23b8"}
{"level":"info","ts":"2023-05-07T10:14:09.295413+0800","caller":"rafthttp/stream.go:412","msg":"established TCP streaming connection with remote peer","stream-reader-type":"stream MsgApp v2","local-member-id":"81c47c2838f502bb","remote-peer-id":"595cd3f73e5f23b8"}
{"level":"info","ts":"2023-05-07T10:14:09.295459+0800","caller":"rafthttp/stream.go:412","msg":"established TCP streaming connection with remote peer","stream-reader-type":"stream Message","local-member-id":"81c47c2838f502bb","remote-peer-id":"595cd3f73e5f23b8"}
{"level":"info","ts":"2023-05-07T10:14:09.295393+0800","caller":"membership/cluster.go:584","msg":"set initial cluster version","cluster-id":"d20920a0fe99cde","local-member-id":"81c47c2838f502bb","cluster-version":"3.0"}
{"level":"info","ts":"2023-05-07T10:14:09.295683+0800","caller":"api/capability.go:75","msg":"enabled capabilities for version","cluster-version":"3.0"}
{"level":"info","ts":"2023-05-07T10:14:09.296052+0800","caller":"membership/cluster.go:576","msg":"updated cluster version","cluster-id":"d20920a0fe99cde","local-member-id":"81c47c2838f502bb","from":"3.0","to":"3.5"}
{"level":"info","ts":"2023-05-07T10:14:09.296799+0800","caller":"rafthttp/peer_status.go:53","msg":"peer became active","peer-id":"a9662e3ab0778d61"}
{"level":"info","ts":"2023-05-07T10:14:09.296942+0800","caller":"rafthttp/stream.go:412","msg":"established TCP streaming connection with remote peer","stream-reader-type":"stream MsgApp v2","local-member-id":"81c47c2838f502bb","remote-peer-id":"a9662e3ab0778d61"}
{"level":"info","ts":"2023-05-07T10:14:09.297032+0800","caller":"rafthttp/stream.go:412","msg":"established TCP streaming connection with remote peer","stream-reader-type":"stream Message","local-member-id":"81c47c2838f502bb","remote-peer-id":"a9662e3ab0778d61"}
{"level":"info","ts":"2023-05-07T10:14:09.333625+0800","caller":"etcdserver/server.go:777","msg":"initialized peer connections; fast-forwarding election ticks","local-member-id":"81c47c2838f502bb","forward-ticks":8,"forward-duration":"800ms","election-ticks":10,"election-timeout":"1s","active-remote-members":2}
{"level":"info","ts":"2023-05-07T10:14:09.343674+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"raft.node: 81c47c2838f502bb elected leader a9662e3ab0778d61 at term 3"}
{"level":"info","ts":"2023-05-07T10:14:09.348651+0800","caller":"embed/serve.go:103","msg":"ready to serve client requests"}
{"level":"info","ts":"2023-05-07T10:14:09.348688+0800","caller":"etcdserver/server.go:2062","msg":"published local member to cluster through raft","local-member-id":"81c47c2838f502bb","local-member-attributes":"{Name:etcd03 ClientURLs:[http://10.17.35.83:2379]}","request-path":"/0/members/81c47c2838f502bb/attributes","cluster-id":"d20920a0fe99cde","publish-timeout":"7s"}
{"level":"info","ts":"2023-05-07T10:14:09.349425+0800","caller":"etcdmain/main.go:44","msg":"notifying init daemon"}
{"level":"info","ts":"2023-05-07T10:14:09.349479+0800","caller":"etcdmain/main.go:50","msg":"successfully notified init daemon"}
{"level":"info","ts":"2023-05-07T10:14:09.351205+0800","caller":"rafthttp/stream.go:249","msg":"set message encoder","from":"81c47c2838f502bb","to":"595cd3f73e5f23b8","stream-type":"stream Message"}
{"level":"info","ts":"2023-05-07T10:14:09.351271+0800","caller":"rafthttp/stream.go:274","msg":"established TCP streaming connection with remote peer","stream-writer-type":"stream Message","local-member-id":"81c47c2838f502bb","remote-peer-id":"595cd3f73e5f23b8"}
{"level":"info","ts":"2023-05-07T10:14:09.351249+0800","caller":"embed/serve.go:187","msg":"serving client traffic insecurely; this is strongly discouraged!","traffic":"grpc+http","address":"10.17.35.83:2379"}
{"level":"info","ts":"2023-05-07T10:14:09.351555+0800","caller":"rafthttp/stream.go:249","msg":"set message encoder","from":"81c47c2838f502bb","to":"595cd3f73e5f23b8","stream-type":"stream MsgApp v2"}
{"level":"info","ts":"2023-05-07T10:14:09.351632+0800","caller":"rafthttp/stream.go:274","msg":"established TCP streaming connection with remote peer","stream-writer-type":"stream MsgApp v2","local-member-id":"81c47c2838f502bb","remote-peer-id":"595cd3f73e5f23b8"}
{"level":"info","ts":"2023-05-07T10:14:09.353964+0800","caller":"rafthttp/stream.go:249","msg":"set message encoder","from":"81c47c2838f502bb","to":"a9662e3ab0778d61","stream-type":"stream Message"}
{"level":"info","ts":"2023-05-07T10:14:09.354026+0800","caller":"rafthttp/stream.go:274","msg":"established TCP streaming connection with remote peer","stream-writer-type":"stream Message","local-member-id":"81c47c2838f502bb","remote-peer-id":"a9662e3ab0778d61"}
{"level":"info","ts":"2023-05-07T10:14:09.354116+0800","caller":"rafthttp/stream.go:249","msg":"set message encoder","from":"81c47c2838f502bb","to":"a9662e3ab0778d61","stream-type":"stream MsgApp v2"}
{"level":"info","ts":"2023-05-07T10:14:09.354169+0800","caller":"rafthttp/stream.go:274","msg":"established TCP streaming connection with remote peer","stream-writer-type":"stream MsgApp v2","local-member-id":"81c47c2838f502bb","remote-peer-id":"a9662e3ab0778d61"}
{"level":"warn","ts":"2023-05-07T10:14:32.910906+0800","caller":"rafthttp/stream.go:421","msg":"lost TCP streaming connection with remote peer","stream-reader-type":"stream MsgApp v2","local-member-id":"81c47c2838f502bb","remote-peer-id":"595cd3f73e5f23b8","error":"EOF"}
{"level":"warn","ts":"2023-05-07T10:14:32.911362+0800","caller":"rafthttp/stream.go:421","msg":"lost TCP streaming connection with remote peer","stream-reader-type":"stream Message","local-member-id":"81c47c2838f502bb","remote-peer-id":"595cd3f73e5f23b8","error":"EOF"}
{"level":"warn","ts":"2023-05-07T10:14:32.915298+0800","caller":"rafthttp/peer_status.go:66","msg":"peer became inactive (message send to peer failed)","peer-id":"595cd3f73e5f23b8","error":"failed to dial 595cd3f73e5f23b8 on stream Message (peer 595cd3f73e5f23b8 failed to find local node 81c47c2838f502bb)"}
{"level":"warn","ts":"2023-05-07T10:14:39.286998+0800","caller":"rafthttp/stream.go:194","msg":"lost TCP streaming connection with remote peer","stream-writer-type":"stream MsgApp v2","local-member-id":"81c47c2838f502bb","remote-peer-id":"595cd3f73e5f23b8"}
{"level":"warn","ts":"2023-05-07T10:14:39.289262+0800","caller":"rafthttp/stream.go:194","msg":"lost TCP streaming connection with remote peer","stream-writer-type":"stream Message","local-member-id":"81c47c2838f502bb","remote-peer-id":"595cd3f73e5f23b8"}
{"level":"warn","ts":"2023-05-07T10:14:44.293611+0800","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_SNAPSHOT","remote-peer-id":"595cd3f73e5f23b8","rtt":"500.702µs","error":"dial tcp 10.17.35.82:2380: connect: connection refused"}
{"level":"warn","ts":"2023-05-07T10:14:44.293632+0800","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_RAFT_MESSAGE","remote-peer-id":"595cd3f73e5f23b8","rtt":"787.602µs","error":"dial tcp 10.17.35.82:2380: connect: connection refused"}
{"level":"info","ts":"2023-05-07T10:14:46.161128+0800","caller":"rafthttp/stream.go:249","msg":"set message encoder","from":"81c47c2838f502bb","to":"595cd3f73e5f23b8","stream-type":"stream Message"}
{"level":"info","ts":"2023-05-07T10:14:46.161344+0800","caller":"rafthttp/peer_status.go:53","msg":"peer became active","peer-id":"595cd3f73e5f23b8"}
{"level":"info","ts":"2023-05-07T10:14:46.161389+0800","caller":"rafthttp/stream.go:274","msg":"established TCP streaming connection with remote peer","stream-writer-type":"stream Message","local-member-id":"81c47c2838f502bb","remote-peer-id":"595cd3f73e5f23b8"}
{"level":"info","ts":"2023-05-07T10:14:46.163042+0800","caller":"rafthttp/stream.go:249","msg":"set message encoder","from":"81c47c2838f502bb","to":"595cd3f73e5f23b8","stream-type":"stream MsgApp v2"}
{"level":"info","ts":"2023-05-07T10:14:46.163144+0800","caller":"rafthttp/stream.go:274","msg":"established TCP streaming connection with remote peer","stream-writer-type":"stream MsgApp v2","local-member-id":"81c47c2838f502bb","remote-peer-id":"595cd3f73e5f23b8"}
{"level":"info","ts":"2023-05-07T10:14:46.213191+0800","caller":"rafthttp/stream.go:412","msg":"established TCP streaming connection with remote peer","stream-reader-type":"stream Message","local-member-id":"81c47c2838f502bb","remote-peer-id":"595cd3f73e5f23b8"}
{"level":"info","ts":"2023-05-07T10:14:46.213191+0800","caller":"rafthttp/stream.go:412","msg":"established TCP streaming connection with remote peer","stream-reader-type":"stream MsgApp v2","local-member-id":"81c47c2838f502bb","remote-peer-id":"595cd3f73e5f23b8"}
{"level":"info","ts":"2023-05-07T10:14:50.0788+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"81c47c2838f502bb [term 3] received MsgTimeoutNow from a9662e3ab0778d61 and starts an election to get leadership."}
{"level":"info","ts":"2023-05-07T10:14:50.078919+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"81c47c2838f502bb is starting a new election at term 3"}
{"level":"info","ts":"2023-05-07T10:14:50.078958+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"81c47c2838f502bb became candidate at term 4"}
{"level":"info","ts":"2023-05-07T10:14:50.079035+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"81c47c2838f502bb received MsgVoteResp from 81c47c2838f502bb at term 4"}
{"level":"info","ts":"2023-05-07T10:14:50.07909+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"81c47c2838f502bb [logterm: 3, index: 15] sent MsgVote request to 595cd3f73e5f23b8 at term 4"}
{"level":"info","ts":"2023-05-07T10:14:50.079121+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"81c47c2838f502bb [logterm: 3, index: 15] sent MsgVote request to a9662e3ab0778d61 at term 4"}
{"level":"info","ts":"2023-05-07T10:14:50.079208+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"raft.node: 81c47c2838f502bb lost leader a9662e3ab0778d61 at term 4"}
{"level":"info","ts":"2023-05-07T10:14:50.081328+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"81c47c2838f502bb received MsgVoteResp from a9662e3ab0778d61 at term 4"}
{"level":"info","ts":"2023-05-07T10:14:50.081389+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"81c47c2838f502bb has received 2 MsgVoteResp votes and 0 vote rejections"}
{"level":"info","ts":"2023-05-07T10:14:50.081428+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"81c47c2838f502bb became leader at term 4"}
{"level":"info","ts":"2023-05-07T10:14:50.081557+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"raft.node: 81c47c2838f502bb elected leader 81c47c2838f502bb at term 4"}
{"level":"warn","ts":"2023-05-07T10:14:50.186823+0800","caller":"rafthttp/stream.go:421","msg":"lost TCP streaming connection with remote peer","stream-reader-type":"stream MsgApp v2","local-member-id":"81c47c2838f502bb","remote-peer-id":"a9662e3ab0778d61","error":"EOF"}
{"level":"warn","ts":"2023-05-07T10:14:50.188421+0800","caller":"rafthttp/stream.go:421","msg":"lost TCP streaming connection with remote peer","stream-reader-type":"stream Message","local-member-id":"81c47c2838f502bb","remote-peer-id":"a9662e3ab0778d61","error":"EOF"}
{"level":"warn","ts":"2023-05-07T10:14:50.19043+0800","caller":"rafthttp/peer_status.go:66","msg":"peer became inactive (message send to peer failed)","peer-id":"a9662e3ab0778d61","error":"failed to dial a9662e3ab0778d61 on stream MsgApp v2 (peer a9662e3ab0778d61 failed to find local node 81c47c2838f502bb)"}
{"level":"warn","ts":"2023-05-07T10:14:50.580344+0800","caller":"rafthttp/stream.go:223","msg":"lost TCP streaming connection with remote peer","stream-writer-type":"stream Message","local-member-id":"81c47c2838f502bb","remote-peer-id":"a9662e3ab0778d61"}
{"level":"warn","ts":"2023-05-07T10:14:54.089151+0800","caller":"etcdserver/cluster_util.go:288","msg":"failed to reach the peer URL","address":"http://10.17.35.81:2380/version","remote-member-id":"a9662e3ab0778d61","error":"Get \"http://10.17.35.81:2380/version\": dial tcp 10.17.35.81:2380: connect: connection refused"}
{"level":"warn","ts":"2023-05-07T10:14:54.089283+0800","caller":"etcdserver/cluster_util.go:155","msg":"failed to get version","remote-member-id":"a9662e3ab0778d61","error":"Get \"http://10.17.35.81:2380/version\": dial tcp 10.17.35.81:2380: connect: connection refused"}
{"level":"warn","ts":"2023-05-07T10:14:55.961324+0800","caller":"rafthttp/stream.go:194","msg":"lost TCP streaming connection with remote peer","stream-writer-type":"stream MsgApp v2","local-member-id":"81c47c2838f502bb","remote-peer-id":"a9662e3ab0778d61"}
{"level":"warn","ts":"2023-05-07T10:14:58.090672+0800","caller":"etcdserver/cluster_util.go:288","msg":"failed to reach the peer URL","address":"http://10.17.35.81:2380/version","remote-member-id":"a9662e3ab0778d61","error":"Get \"http://10.17.35.81:2380/version\": dial tcp 10.17.35.81:2380: connect: connection refused"}
{"level":"warn","ts":"2023-05-07T10:14:58.090817+0800","caller":"etcdserver/cluster_util.go:155","msg":"failed to get version","remote-member-id":"a9662e3ab0778d61","error":"Get \"http://10.17.35.81:2380/version\": dial tcp 10.17.35.81:2380: connect: connection refused"}
{"level":"warn","ts":"2023-05-07T10:15:02.092437+0800","caller":"etcdserver/cluster_util.go:288","msg":"failed to reach the peer URL","address":"http://10.17.35.81:2380/version","remote-member-id":"a9662e3ab0778d61","error":"Get \"http://10.17.35.81:2380/version\": dial tcp 10.17.35.81:2380: connect: connection refused"}
{"level":"warn","ts":"2023-05-07T10:15:02.092563+0800","caller":"etcdserver/cluster_util.go:155","msg":"failed to get version","remote-member-id":"a9662e3ab0778d61","error":"Get \"http://10.17.35.81:2380/version\": dial tcp 10.17.35.81:2380: connect: connection refused"}
{"level":"info","ts":"2023-05-07T10:15:02.560987+0800","caller":"rafthttp/stream.go:249","msg":"set message encoder","from":"81c47c2838f502bb","to":"a9662e3ab0778d61","stream-type":"stream Message"}
{"level":"info","ts":"2023-05-07T10:15:02.561091+0800","caller":"rafthttp/peer_status.go:53","msg":"peer became active","peer-id":"a9662e3ab0778d61"}
{"level":"info","ts":"2023-05-07T10:15:02.561126+0800","caller":"rafthttp/stream.go:274","msg":"established TCP streaming connection with remote peer","stream-writer-type":"stream Message","local-member-id":"81c47c2838f502bb","remote-peer-id":"a9662e3ab0778d61"}
{"level":"info","ts":"2023-05-07T10:15:02.561106+0800","caller":"rafthttp/stream.go:249","msg":"set message encoder","from":"81c47c2838f502bb","to":"a9662e3ab0778d61","stream-type":"stream MsgApp v2"}
{"level":"info","ts":"2023-05-07T10:15:02.561203+0800","caller":"rafthttp/stream.go:274","msg":"established TCP streaming connection with remote peer","stream-writer-type":"stream MsgApp v2","local-member-id":"81c47c2838f502bb","remote-peer-id":"a9662e3ab0778d61"}
{"level":"info","ts":"2023-05-07T10:15:02.589624+0800","caller":"rafthttp/stream.go:412","msg":"established TCP streaming connection with remote peer","stream-reader-type":"stream MsgApp v2","local-member-id":"81c47c2838f502bb","remote-peer-id":"a9662e3ab0778d61"}
{"level":"info","ts":"2023-05-07T10:15:02.590557+0800","caller":"rafthttp/stream.go:412","msg":"established TCP streaming connection with remote peer","stream-reader-type":"stream Message","local-member-id":"81c47c2838f502bb","remote-peer-id":"a9662e3ab0778d61"}

ahrtr · 2023-05-06T21:17:48Z

ahrtr
May 6, 2023
Maintainer

The reason is the name (e.g. etcd03) has a prefix space in the flag --initial-cluster, but there is no space in --name. Please try to remove the spaces in --initial-cluster.

I think it's a bug, etcd should trim the space automatically when parsing the the parameters, such as --initial-cluster and --name. Please feel free to raise a PR, and we can have more discussion under the PR,

etcd/client/pkg/types/urlsmap.go

Lines 101 to 103 in 249c0d7

    
           if i := strings.Index(key, "="); i >= 0 { 
        
           	key, value = key[:i], key[i+1:] 
        
           }

1 reply

jmhbnz May 6, 2023
Maintainer

I'm also thinking our example https://github.com/etcd-io/etcd/blob/main/etcd.conf.yml.sample could be clearer about how we expect the values for initial-cluster to be specified to avoid this. Will raise a quick pr to fix that once we get confirmation from the original poster.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Why doesn't etcd start? #15837

{{title}}

Replies: 2 comments 2 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

Select a reply

Why doesn't etcd start? #15837

xiaofanshifu May 6, 2023

Replies: 2 comments · 2 replies

jmhbnz May 6, 2023 Maintainer

xiaofanshifu May 7, 2023 Author

ahrtr May 6, 2023 Maintainer

jmhbnz May 6, 2023 Maintainer

xiaofanshifu
May 6, 2023

Replies: 2 comments 2 replies

jmhbnz
May 6, 2023
Maintainer

xiaofanshifu May 7, 2023
Author

ahrtr
May 6, 2023
Maintainer

jmhbnz May 6, 2023
Maintainer