By type
+ CONSUL_BIND=
+ [ -n ]
+ CONSUL_CLIENT=
+ [ -n ]
+ CONSUL_DATA_DIR=/consul/data
+ CONSUL_CONFIG_DIR=/consul/config
+ [ -n ]
+ [ a = - ]
+ [ agent = agent ]
+ shift
+ set -- consul agent -data-dir=/consul/data -config-dir=/consul/config -bootstrap-expect=3 -enable-script-checks -retry-join=dev-consul-server-0.consul-server -retry-join=dev-consul-server-1.consul-server -retry-join=dev-consul-server-2.consul-server -client=0.0.0.0 -server -ui
+ [ consul = consul ]
+ stat -c %u /consul/data
+ id -u consul
+ [ 100 != 100 ]
+ stat -c %u /consul/config
+ id -u consul
+ [ 100 != 100 ]
+ [ ! -z ]
+ exec consul agent -data-dir=/consul/data -config-dir=/consul/config -bootstrap-expect=3 -enable-script-checks -retry-join=dev-consul-server-0.consul-server -retry-join=dev-consul-server-1.consul-server -retry-join=dev-consul-server-2.consul-server -client=0.0.0.0 -server -ui
bootstrap_expect > 0: expecting 3 servers
==> Starting Consul agent...
==> Consul agent running!
Version: 'v1.0.6'
Node ID: 'cece5e1c-d581-d6d7-9a7a-03a4bf5634cf'
Node name: 'dev-consul-server-0'
Datacenter: 'dc1' (Segment: '')
Server: true (Bootstrap: false)
Client Addr: [0.0.0.0] (HTTP: 8500, HTTPS: -1, DNS: 8600)
Cluster Addr: 10.242.219.70 (LAN: 8301, WAN: 8302)
Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false
==> Log data will now stream in as it occurs:
2021/04/11 10:40:04 [INFO] raft: Initial configuration (index=0): []
2021/04/11 10:40:04 [INFO] raft: Node at 10.242.219.70:8300 [Follower] entering Follower state (Leader: "")
2021/04/11 10:40:04 [INFO] serf: EventMemberJoin: dev-consul-server-0.dc1 10.242.219.70
2021/04/11 10:40:04 [INFO] serf: EventMemberJoin: dev-consul-server-0 10.242.219.70
2021/04/11 10:40:04 [INFO] consul: Handled member-join event for server "dev-consul-server-0.dc1" in area "wan"
2021/04/11 10:40:04 [INFO] agent: Started DNS server 0.0.0.0:8600 (udp)
2021/04/11 10:40:04 [INFO] consul: Adding LAN server dev-consul-server-0 (Addr: tcp/10.242.219.70:8300) (DC: dc1)
2021/04/11 10:40:04 [INFO] agent: Started DNS server 0.0.0.0:8600 (tcp)
2021/04/11 10:40:04 [INFO] agent: Started HTTP server on [::]:8500 (tcp)
2021/04/11 10:40:04 [INFO] agent: started state syncer
2021/04/11 10:40:04 [INFO] agent: Retry join LAN is supported for: aliyun aws azure digitalocean gce os scaleway softlayer
2021/04/11 10:40:04 [INFO] agent: Joining LAN cluster...
2021/04/11 10:40:04 [INFO] agent: (LAN) joining: [dev-consul-server-0.consul-server dev-consul-server-1.consul-server dev-consul-server-2.consul-server]
2021/04/11 10:40:04 [WARN] memberlist: Failed to resolve dev-consul-server-0.consul-server: lookup dev-consul-server-0.consul-server on 10.96.0.10:53: no such host
2021/04/11 10:40:04 [WARN] memberlist: Failed to resolve dev-consul-server-1.consul-server: lookup dev-consul-server-1.consul-server on 10.96.0.10:53: no such host
2021/04/11 10:40:04 [WARN] memberlist: Failed to resolve dev-consul-server-2.consul-server: lookup dev-consul-server-2.consul-server on 10.96.0.10:53: no such host
2021/04/11 10:40:04 [INFO] agent: (LAN) joined: 0 Err: 3 error(s) occurred:
* Failed to resolve dev-consul-server-0.consul-server: lookup dev-consul-server-0.consul-server on 10.96.0.10:53: no such host
* Failed to resolve dev-consul-server-1.consul-server: lookup dev-consul-server-1.consul-server on 10.96.0.10:53: no such host
* Failed to resolve dev-consul-server-2.consul-server: lookup dev-consul-server-2.consul-server on 10.96.0.10:53: no such host
2021/04/11 10:40:04 [WARN] agent: Join LAN failed: , retrying in 30s
2021/04/11 10:40:11 [ERR] agent: failed to sync remote state: No cluster leader
2021/04/11 10:40:13 [WARN] raft: no known peers, aborting election
==> Failed to check for updates: Get https://checkpoint-api.hashicorp.com/v1/check/consul?arch=amd64&os=linux&signature=7e6fe5a0-5fea-e3ca-8b3f-91dc8f80a52d&version=1.0.6: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
2021/04/11 10:40:31 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 10:40:34 [INFO] agent: (LAN) joining: [dev-consul-server-0.consul-server dev-consul-server-1.consul-server dev-consul-server-2.consul-server]
2021/04/11 10:40:34 [WARN] memberlist: Failed to resolve dev-consul-server-1.consul-server: lookup dev-consul-server-1.consul-server on 10.96.0.10:53: no such host
2021/04/11 10:40:34 [WARN] memberlist: Failed to resolve dev-consul-server-2.consul-server: lookup dev-consul-server-2.consul-server on 10.96.0.10:53: no such host
2021/04/11 10:40:34 [INFO] agent: (LAN) joined: 1 Err:
2021/04/11 10:40:34 [INFO] agent: Join LAN completed. Synced with 1 initial agents
2021/04/11 10:40:37 [ERR] agent: Coordinate update error: No cluster leader
2021/04/11 10:40:45 [ERR] agent: failed to sync remote state: No cluster leader
2021/04/11 10:40:57 [INFO] serf: EventMemberJoin: dev-consul-server-1 10.242.150.7
2021/04/11 10:40:57 [INFO] consul: Adding LAN server dev-consul-server-1 (Addr: tcp/10.242.150.7:8300) (DC: dc1)
2021/04/11 10:40:57 [INFO] serf: EventMemberJoin: dev-consul-server-1.dc1 10.242.150.7
2021/04/11 10:40:57 [INFO] consul: Handled member-join event for server "dev-consul-server-1.dc1" in area "wan"
2021/04/11 10:41:00 [ERR] agent: Coordinate update error: No cluster leader
2021/04/11 10:41:17 [ERR] agent: failed to sync remote state: No cluster leader
2021/04/11 10:41:18 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.197.19
2021/04/11 10:41:18 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 10:41:21 [INFO] consul: Existing Raft peers reported by dev-consul-server-2, disabling bootstrap mode
2021/04/11 10:41:21 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.197.19
2021/04/11 10:41:21 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/11 10:41:25 [ERR] agent: Coordinate update error: No cluster leader
2021/04/11 10:41:27 [DEBUG] raft-net: 10.242.219.70:8300 accepted connection from: 10.242.197.19:44068
2021/04/11 10:41:27 [WARN] raft: Failed to get previous log: 1 log not found (last: 0)
2021/04/11 10:41:27 [INFO] consul: New leader elected: dev-consul-server-2
2021/04/11 10:41:27 [INFO] agent: Synced node info
2021/04/11 10:41:28 [DEBUG] raft-net: 10.242.219.70:8300 accepted connection from: 10.242.197.19:44090
2021/04/11 11:13:29 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:41656->10.242.197.6:8301: i/o timeout
2021/04/11 11:13:29 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:13:32 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:41714->10.242.197.6:8301: i/o timeout
2021/04/11 11:13:32 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:13:36 [INFO] memberlist: Marking dev-consul-7467bf7b46-bl49x as failed, suspect timeout reached (2 peer confirmations)
2021/04/11 11:13:36 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 11:13:36 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:41776->10.242.197.6:8301: i/o timeout
2021/04/11 11:13:36 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:13:37 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 11:13:50 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:42018->10.242.197.6:8301: i/o timeout
2021/04/11 11:13:50 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:13:53 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:42038->10.242.197.6:8301: i/o timeout
2021/04/11 11:13:53 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:13:55 [WARN] memberlist: Refuting a suspect message (from: dev-consul-7467bf7b46-bl49x)
2021/04/11 11:14:09 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:42308->10.242.197.6:8301: i/o timeout
2021/04/11 11:14:09 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:14:12 [WARN] memberlist: Refuting a suspect message (from: dev-consul-7467bf7b46-bl49x)
2021/04/11 11:16:26 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:44474->10.242.197.6:8301: i/o timeout
2021/04/11 11:16:26 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:16:29 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:44954->10.242.197.19:8301: i/o timeout
2021/04/11 11:16:29 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:16:31 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:45004->10.242.197.19:8301: i/o timeout
2021/04/11 11:16:31 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:16:33 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:44576->10.242.197.6:8301: i/o timeout
2021/04/11 11:16:33 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:16:35 [WARN] memberlist: Was able to connect to dev-consul-server-2 but other probes failed, network may be misconfigured
2021/04/11 11:16:39 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:45142->10.242.197.19:8301: i/o timeout
2021/04/11 11:16:39 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:16:41 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:44722->10.242.197.6:8301: i/o timeout
2021/04/11 11:16:41 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:16:49 [WARN] memberlist: Was able to connect to dev-consul-7467bf7b46-bl49x but other probes failed, network may be misconfigured
2021/04/11 11:16:50 [WARN] memberlist: Refuting a suspect message (from: dev-consul-7467bf7b46-bl49x)
2021/04/11 11:16:50 [WARN] memberlist: Was able to connect to dev-consul-server-2 but other probes failed, network may be misconfigured
2021/04/11 11:16:52 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:44914->10.242.197.6:8301: i/o timeout
2021/04/11 11:16:52 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:16:55 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:44958->10.242.197.6:8301: i/o timeout
2021/04/11 11:16:55 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:16:58 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:45436->10.242.197.19:8301: i/o timeout
2021/04/11 11:16:58 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:17:02 [WARN] memberlist: Was able to connect to dev-consul-7467bf7b46-bl49x but other probes failed, network may be misconfigured
2021/04/11 11:17:02 [WARN] memberlist: Refuting a suspect message (from: dev-consul-7467bf7b46-bl49x)
2021/04/11 11:17:06 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:45606->10.242.197.19:8301: i/o timeout
2021/04/11 11:17:06 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:17:10 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:45684->10.242.197.19:8301: i/o timeout
2021/04/11 11:17:10 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:17:15 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:45766->10.242.197.19:8301: i/o timeout
2021/04/11 11:17:20 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:45848->10.242.197.19:8301: i/o timeout
2021/04/11 11:17:20 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:17:21 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/04/11 11:17:21 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.197.19
2021/04/11 11:17:21 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:17:21 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.197.19
2021/04/11 11:17:21 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:17:23 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:45882->10.242.197.19:8301: i/o timeout
2021/04/11 11:17:23 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:17:27 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:45942->10.242.197.19:8301: i/o timeout
2021/04/11 11:17:27 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:17:29 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:45510->10.242.197.6:8301: i/o timeout
2021/04/11 11:17:29 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:17:30 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2)
2021/04/11 11:17:32 [WARN] memberlist: Was able to connect to dev-consul-7467bf7b46-bl49x but other probes failed, network may be misconfigured
2021/04/11 11:17:34 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:45630->10.242.197.6:8301: i/o timeout
2021/04/11 11:17:34 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:17:36 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:46108->10.242.197.19:8301: i/o timeout
2021/04/11 11:17:36 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:17:38 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:46114->10.242.197.19:8301: i/o timeout
2021/04/11 11:17:38 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:17:40 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:45730->10.242.197.6:8301: i/o timeout
2021/04/11 11:17:40 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:17:42 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:46200->10.242.197.19:8301: i/o timeout
2021/04/11 11:17:42 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:17:44 [WARN] memberlist: Was able to connect to dev-consul-7467bf7b46-bl49x but other probes failed, network may be misconfigured
2021/04/11 11:17:45 [INFO] memberlist: Marking dev-consul-7467bf7b46-bl49x as failed, suspect timeout reached (1 peer confirmations)
2021/04/11 11:17:45 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 11:17:46 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:45826->10.242.197.6:8301: i/o timeout
2021/04/11 11:17:46 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:17:47 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (1 peer confirmations)
2021/04/11 11:17:47 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.197.19
2021/04/11 11:17:47 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:17:48 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:46284->10.242.197.19:8301: i/o timeout
2021/04/11 11:17:48 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:17:49 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:50172->10.242.197.19:8302: i/o timeout
2021/04/11 11:17:49 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/11 11:17:55 [ERR] agent: Coordinate update error: No cluster leader
2021/04/11 11:17:59 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:50338->10.242.197.19:8302: i/o timeout
2021/04/11 11:17:59 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/11 11:18:02 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/11 11:18:04 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.197.19:8301
2021/04/11 11:18:05 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 11:18:05 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.197.19
2021/04/11 11:18:05 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:18:07 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:46610->10.242.197.19:8301: i/o timeout
2021/04/11 11:18:07 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:18:10 [WARN] memberlist: Was able to connect to dev-consul-7467bf7b46-bl49x but other probes failed, network may be misconfigured
2021/04/11 11:18:12 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:46706->10.242.197.19:8301: i/o timeout
2021/04/11 11:18:12 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:18:14 [WARN] raft: Heartbeat timeout from "10.242.197.19:8300" reached, starting election
2021/04/11 11:18:14 [INFO] raft: Node at 10.242.219.70:8300 [Candidate] entering Candidate state in term 3
2021/04/11 11:18:15 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:46760->10.242.197.19:8301: i/o timeout
2021/04/11 11:18:15 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:18:16 [DEBUG] raft-net: 10.242.219.70:8300 accepted connection from: 10.242.150.7:32906
2021/04/11 11:18:16 [INFO] raft: Duplicate RequestVote for same term: 3
2021/04/11 11:18:16 [ERR] memberlist: Push/Pull with dev-consul-server-2 failed: read tcp 10.242.219.70:46612->10.242.197.19:8301: i/o timeout
2021/04/11 11:18:18 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:46340->10.242.197.6:8301: i/o timeout
2021/04/11 11:18:18 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:18:18 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.197.19
2021/04/11 11:18:18 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:18:21 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:46412->10.242.197.6:8301: i/o timeout
2021/04/11 11:18:21 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:18:23 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:46442->10.242.197.6:8301: i/o timeout
2021/04/11 11:18:23 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:18:23 [INFO] memberlist: Marking dev-consul-7467bf7b46-bl49x as failed, suspect timeout reached (1 peer confirmations)
2021/04/11 11:18:23 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 11:18:23 [DEBUG] raft-net: 10.242.219.70:8300 accepted connection from: 10.242.197.19:49246
2021/04/11 11:18:23 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.197.19
2021/04/11 11:18:23 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:18:23 [WARN] memberlist: Refuting a suspect message (from: dev-consul-7467bf7b46-bl49x)
2021/04/11 11:18:24 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2.dc1)
2021/04/11 11:18:24 [WARN] raft: Election timeout reached, restarting election
2021/04/11 11:18:24 [INFO] raft: Node at 10.242.219.70:8300 [Candidate] entering Candidate state in term 4
2021/04/11 11:18:24 [INFO] raft: Election won. Tally: 2
2021/04/11 11:18:24 [INFO] raft: Node at 10.242.219.70:8300 [Leader] entering Leader state
2021/04/11 11:18:24 [INFO] raft: Added peer 93926a22-3b0f-30f5-53a3-a64ad4920efa, starting replication
2021/04/11 11:18:24 [INFO] raft: Added peer 68b413ff-82e9-adbb-03e8-30a9524f6f0c, starting replication
2021/04/11 11:18:24 [INFO] consul: cluster leadership acquired
2021/04/11 11:18:24 [INFO] consul: New leader elected: dev-consul-server-0
2021/04/11 11:18:24 [INFO] raft: pipelining replication to peer {Voter 68b413ff-82e9-adbb-03e8-30a9524f6f0c 10.242.150.7:8300}
2021/04/11 11:18:24 [INFO] consul: member 'dev-consul-7467bf7b46-bl49x' failed, marking health critical
2021/04/11 11:18:24 [ERR] raft: Failed to make RequestVote RPC to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:60920->10.242.197.19:8300: i/o timeout
2021/04/11 11:18:25 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 11:18:25 [INFO] consul: member 'dev-consul-7467bf7b46-bl49x' joined, marking health alive
2021/04/11 11:18:26 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.501983353s
2021/04/11 11:18:27 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:18:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:18:28 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:46540->10.242.197.6:8301: i/o timeout
2021/04/11 11:18:28 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:18:29 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500212619s
2021/04/11 11:18:29 [WARN] raft: Rejecting vote request from 10.242.197.19:8300 since we have a leader: 10.242.219.70:8300
2021/04/11 11:18:29 [INFO] raft: pipelining replication to peer {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}
2021/04/11 11:18:48 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:46882->10.242.197.6:8301: i/o timeout
2021/04/11 11:18:48 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:18:49 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:18:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:18:51 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500342081s
2021/04/11 11:18:51 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:47352->10.242.197.19:8301: i/o timeout
2021/04/11 11:18:51 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:18:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:18:53 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 4.949977839s
2021/04/11 11:18:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:18:54 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:51286->10.242.197.19:8302: i/o timeout
2021/04/11 11:18:54 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/11 11:18:54 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:47400->10.242.197.19:8301: i/o timeout
2021/04/11 11:18:54 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:18:56 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 7.437375557s
2021/04/11 11:18:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:18:57 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:47010->10.242.197.6:8301: i/o timeout
2021/04/11 11:18:57 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:18:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:18:58 [INFO] raft: aborting pipeline replication to peer {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}
2021/04/11 11:18:59 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: read tcp 10.242.219.70:32856->10.242.197.19:8300: i/o timeout
2021/04/11 11:18:59 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 11:18:59 [INFO] consul: member 'dev-consul-7467bf7b46-bl49x' failed, marking health critical
2021/04/11 11:19:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:19:00 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:47094->10.242.197.6:8301: i/o timeout
2021/04/11 11:19:00 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:19:01 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500358765s
2021/04/11 11:19:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:19:02 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:47558->10.242.197.19:8301: i/o timeout
2021/04/11 11:19:02 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:19:02 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (1 peer confirmations)
2021/04/11 11:19:02 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.197.19
2021/04/11 11:19:02 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:19:02 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/11 11:19:03 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 4.984484304s
2021/04/11 11:19:03 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/11 11:19:03 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.197.19:44026
2021/04/11 11:19:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:19:04 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:51446->10.242.197.19:8302: i/o timeout
2021/04/11 11:19:04 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/11 11:19:05 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-bl49x 10.242.197.6:8301
2021/04/11 11:19:06 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 7.40587781s
2021/04/11 11:19:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:19:06 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2)
2021/04/11 11:19:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:19:08 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:32872->10.242.197.19:8300: i/o timeout
2021/04/11 11:19:08 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:19:09 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: read tcp 10.242.219.70:33472->10.242.197.19:8300: i/o timeout
2021/04/11 11:19:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:19:10 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:19:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:19:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:19:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:19:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:19:18 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:33614->10.242.197.19:8300: i/o timeout
2021/04/11 11:19:18 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:19:19 [WARN] memberlist: Was able to connect to dev-consul-server-2.dc1 but other probes failed, network may be misconfigured
2021/04/11 11:19:19 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:19:20 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:19:21 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:19:21 [WARN] consul: error getting server health from "dev-consul-server-1": context deadline exceeded
2021/04/11 11:19:22 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/04/11 11:19:22 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.197.19
2021/04/11 11:19:22 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/11 11:19:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:19:22 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.501151747s
2021/04/11 11:19:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:19:24 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 4.987378351s
2021/04/11 11:19:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:19:27 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 7.470607541s
2021/04/11 11:19:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:19:28 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:33784->10.242.197.19:8300: i/o timeout
2021/04/11 11:19:28 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:19:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:19:30 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: read tcp 10.242.219.70:33656->10.242.197.19:8300: i/o timeout
2021/04/11 11:19:31 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:19:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:19:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:19:34 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.197.19:8302
2021/04/11 11:19:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:19:37 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2.dc1)
2021/04/11 11:19:37 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.197.19
2021/04/11 11:19:37 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/11 11:19:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:19:38 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:19:38 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:33952->10.242.197.19:8300: i/o timeout
2021/04/11 11:19:38 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:19:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:19:40 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.505961971s
2021/04/11 11:19:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:19:42 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:19:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:19:44 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/11 11:19:44 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.197.6:39176
2021/04/11 11:19:45 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:19:45 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.197.19:8301
2021/04/11 11:19:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:19:46 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:19:46 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.197.19
2021/04/11 11:19:46 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:19:46 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 11:19:46 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/11 11:19:46 [INFO] consul: member 'dev-consul-7467bf7b46-bl49x' joined, marking health alive
2021/04/11 11:19:47 [INFO] raft: pipelining replication to peer {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}
2021/04/11 11:22:15 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:50258->10.242.197.6:8301: i/o timeout
2021/04/11 11:22:15 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:22:25 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:22:25 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:50414->10.242.197.6:8301: i/o timeout
2021/04/11 11:22:27 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:22:27 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:50436->10.242.197.6:8301: i/o timeout
2021/04/11 11:22:27 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:22:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:22:28 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500310312s
2021/04/11 11:22:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:22:30 [WARN] memberlist: Was able to connect to dev-consul-server-2 but other probes failed, network may be misconfigured
2021/04/11 11:22:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:22:32 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:50526->10.242.197.6:8301: i/o timeout
2021/04/11 11:22:32 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:22:32 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500263371s
2021/04/11 11:22:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:22:34 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:50994->10.242.197.19:8301: i/o timeout
2021/04/11 11:22:34 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:22:35 [INFO] raft: aborting pipeline replication to peer {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}
2021/04/11 11:22:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:22:37 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:50582->10.242.197.6:8301: i/o timeout
2021/04/11 11:22:37 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:22:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:22:38 [INFO] memberlist: Marking dev-consul-7467bf7b46-bl49x as failed, suspect timeout reached (1 peer confirmations)
2021/04/11 11:22:38 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 11:22:38 [INFO] consul: member 'dev-consul-7467bf7b46-bl49x' failed, marking health critical
2021/04/11 11:22:39 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (1 peer confirmations)
2021/04/11 11:22:39 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.197.19
2021/04/11 11:22:39 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:22:39 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/11 11:22:39 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500142987s
2021/04/11 11:22:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:22:41 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:51078->10.242.197.19:8301: i/o timeout
2021/04/11 11:22:41 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:22:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:22:42 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 4.929291778s
2021/04/11 11:22:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:22:44 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:55020->10.242.197.19:8302: i/o timeout
2021/04/11 11:22:44 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 7.399751249s
2021/04/11 11:22:45 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:36974->10.242.197.19:8300: i/o timeout
2021/04/11 11:22:45 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:22:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:22:46 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-bl49x 10.242.197.6:8301
2021/04/11 11:22:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:22:48 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: read tcp 10.242.219.70:34002->10.242.197.19:8300: i/o timeout
2021/04/11 11:22:48 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:22:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:22:50 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2)
2021/04/11 11:22:52 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.197.19
2021/04/11 11:22:52 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:22:52 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/11 11:22:53 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:22:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:22:54 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:55198->10.242.197.19:8302: i/o timeout
2021/04/11 11:22:54 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:51332->10.242.197.19:8301: i/o timeout
2021/04/11 11:22:54 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:22:55 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:37142->10.242.197.19:8300: i/o timeout
2021/04/11 11:22:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:22:57 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.501104853s
2021/04/11 11:22:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:22:58 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:51418->10.242.197.19:8301: i/o timeout
2021/04/11 11:22:58 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:22:59 [ERR] memberlist: failed to receive: read tcp 10.242.219.70:8301->10.242.197.6:56210: i/o timeout from=10.242.197.6:56210
2021/04/11 11:22:59 [ERR] memberlist: Failed to send error: write tcp 10.242.219.70:8301->10.242.197.6:56210: i/o timeout from=10.242.197.6:56210
2021/04/11 11:22:59 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:55286->10.242.197.19:8302: i/o timeout
2021/04/11 11:22:59 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/11 11:22:59 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 4.967736127s
2021/04/11 11:23:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:23:00 [WARN] memberlist: Was able to connect to dev-consul-server-2 but other probes failed, network may be misconfigured
2021/04/11 11:23:02 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 7.452276374s
2021/04/11 11:23:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:23:02 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:51484->10.242.197.19:8301: i/o timeout
2021/04/11 11:23:02 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:23:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:23:05 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:51548->10.242.197.19:8301: i/o timeout
2021/04/11 11:23:05 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:23:05 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: read tcp 10.242.219.70:37192->10.242.197.19:8300: i/o timeout
2021/04/11 11:23:05 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (1 peer confirmations)
2021/04/11 11:23:05 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.197.19
2021/04/11 11:23:05 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:23:05 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/11 11:23:05 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:37310->10.242.197.19:8300: i/o timeout
2021/04/11 11:23:05 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:23:05 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:23:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:23:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:23:09 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.197.19
2021/04/11 11:23:09 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:23:09 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/11 11:23:09 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2.dc1)
2021/04/11 11:23:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:23:10 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:51650->10.242.197.19:8301: i/o timeout
2021/04/11 11:23:10 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:23:11 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 11:23:11 [INFO] consul: member 'dev-consul-7467bf7b46-bl49x' joined, marking health alive
2021/04/11 11:23:12 [INFO] raft: pipelining replication to peer {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}
2021/04/11 11:23:15 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:23:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:23:19 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:23:19 [WARN] consul: error getting server health from "dev-consul-server-1": context deadline exceeded
2021/04/11 11:23:25 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:23:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:23:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:23:28 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:51474->10.242.197.6:8301: i/o timeout
2021/04/11 11:23:28 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:23:30 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500566001s
2021/04/11 11:23:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:23:31 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:51944->10.242.197.19:8301: i/o timeout
2021/04/11 11:23:31 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:23:33 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:23:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:23:34 [WARN] memberlist: Was able to connect to dev-consul-7467bf7b46-bl49x but other probes failed, network may be misconfigured
2021/04/11 11:23:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:23:37 [WARN] memberlist: Was able to connect to dev-consul-server-2 but other probes failed, network may be misconfigured
2021/04/11 11:23:38 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:51642->10.242.197.6:8301: i/o timeout
2021/04/11 11:23:38 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:23:41 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:23:41 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:51660->10.242.197.6:8301: i/o timeout
2021/04/11 11:23:41 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:23:43 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:23:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:23:45 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:51756->10.242.197.6:8301: i/o timeout
2021/04/11 11:23:45 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:23:45 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.50030219s
2021/04/11 11:23:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:23:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:23:48 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:52242->10.242.197.19:8301: i/o timeout
2021/04/11 11:23:48 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:23:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:23:50 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:51836->10.242.197.6:8301: i/o timeout
2021/04/11 11:23:50 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:23:50 [INFO] memberlist: Marking dev-consul-7467bf7b46-bl49x as failed, suspect timeout reached (1 peer confirmations)
2021/04/11 11:23:50 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 11:23:50 [INFO] consul: member 'dev-consul-7467bf7b46-bl49x' failed, marking health critical
2021/04/11 11:23:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:23:52 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:52336->10.242.197.19:8301: i/o timeout
2021/04/11 11:23:52 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:23:53 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500382629s
2021/04/11 11:23:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:23:54 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:52376->10.242.197.19:8301: i/o timeout
2021/04/11 11:23:54 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:23:55 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 4.991469785s
2021/04/11 11:23:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:23:56 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:52390->10.242.197.19:8301: i/o timeout
2021/04/11 11:23:56 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:23:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:23:58 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:52434->10.242.197.19:8301: i/o timeout
2021/04/11 11:23:58 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:23:58 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 7.455452284s
2021/04/11 11:23:59 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:56302->10.242.197.19:8302: i/o timeout
2021/04/11 11:23:59 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/11 11:23:59 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: read tcp 10.242.219.70:37488->10.242.197.19:8300: i/o timeout
2021/04/11 11:23:59 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (1 peer confirmations)
2021/04/11 11:23:59 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.197.19
2021/04/11 11:23:59 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:23:59 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/11 11:24:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:00 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:24:00 [INFO] raft: aborting pipeline replication to peer {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}
2021/04/11 11:24:01 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:24:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:03 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500222705s
2021/04/11 11:24:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:04 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 11:24:04 [INFO] consul: member 'dev-consul-7467bf7b46-bl49x' joined, marking health alive
2021/04/11 11:24:05 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 4.973808395s
2021/04/11 11:24:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:06 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:52118->10.242.197.6:8301: i/o timeout
2021/04/11 11:24:06 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:24:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:08 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:52154->10.242.197.6:8301: i/o timeout
2021/04/11 11:24:08 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:24:08 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 7.451370232s
2021/04/11 11:24:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:10 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: read tcp 10.242.219.70:38414->10.242.197.19:8300: i/o timeout
2021/04/11 11:24:11 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:38416->10.242.197.19:8300: i/o timeout
2021/04/11 11:24:11 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:24:11 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:24:11 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:52212->10.242.197.6:8301: i/o timeout
2021/04/11 11:24:11 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:24:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:13 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:52246->10.242.197.6:8301: i/o timeout
2021/04/11 11:24:13 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:24:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:14 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:56558->10.242.197.19:8302: i/o timeout
2021/04/11 11:24:14 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/11 11:24:15 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:52284->10.242.197.6:8301: i/o timeout
2021/04/11 11:24:15 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:24:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:16 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2.dc1)
2021/04/11 11:24:17 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:52298->10.242.197.6:8301: i/o timeout
2021/04/11 11:24:17 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:24:17 [INFO] memberlist: Marking dev-consul-7467bf7b46-bl49x as failed, suspect timeout reached (1 peer confirmations)
2021/04/11 11:24:17 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 11:24:17 [INFO] consul: member 'dev-consul-7467bf7b46-bl49x' failed, marking health critical
2021/04/11 11:24:19 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:24:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:20 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.197.19
2021/04/11 11:24:20 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:24:20 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/11 11:24:21 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:38590->10.242.197.19:8300: i/o timeout
2021/04/11 11:24:21 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: read tcp 10.242.219.70:38594->10.242.197.19:8300: i/o timeout
2021/04/11 11:24:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:22 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:52850->10.242.197.19:8301: i/o timeout
2021/04/11 11:24:22 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:24:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:25 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:52900->10.242.197.19:8301: i/o timeout
2021/04/11 11:24:25 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:24:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:26 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-bl49x 10.242.197.6:8301
2021/04/11 11:24:27 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:52918->10.242.197.19:8301: i/o timeout
2021/04/11 11:24:27 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:24:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:29 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:56808->10.242.197.19:8302: i/o timeout
2021/04/11 11:24:29 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/11 11:24:29 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:52968->10.242.197.19:8301: i/o timeout
2021/04/11 11:24:29 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:24:29 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2)
2021/04/11 11:24:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:31 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:38760->10.242.197.19:8300: i/o timeout
2021/04/11 11:24:31 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: read tcp 10.242.219.70:38772->10.242.197.19:8300: i/o timeout
2021/04/11 11:24:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:32 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:53010->10.242.197.19:8301: i/o timeout
2021/04/11 11:24:32 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:24:33 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (1 peer confirmations)
2021/04/11 11:24:33 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.197.19
2021/04/11 11:24:33 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:24:33 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/11 11:24:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:34 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:53066->10.242.197.19:8301: i/o timeout
2021/04/11 11:24:34 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:24:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:39 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:24:39 [WARN] memberlist: Was able to connect to dev-consul-server-2.dc1 but other probes failed, network may be misconfigured
2021/04/11 11:24:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:40 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500282864s
2021/04/11 11:24:41 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:38936->10.242.197.19:8300: i/o timeout
2021/04/11 11:24:41 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:24:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:42 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:24:43 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:24:44 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:24:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:46 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500385567s
2021/04/11 11:24:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:48 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 4.981952155s
2021/04/11 11:24:48 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.197.19
2021/04/11 11:24:48 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:24:48 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/11 11:24:49 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:57148->10.242.197.19:8302: i/o timeout
2021/04/11 11:24:49 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/11 11:24:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:50 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:53336->10.242.197.19:8301: i/o timeout
2021/04/11 11:24:50 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:24:51 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:39100->10.242.197.19:8300: i/o timeout
2021/04/11 11:24:51 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500616556s
2021/04/11 11:24:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:52 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:53358->10.242.197.19:8301: i/o timeout
2021/04/11 11:24:52 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:24:53 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 4.926615409s
2021/04/11 11:24:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:54 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:53372->10.242.197.19:8301: i/o timeout
2021/04/11 11:24:54 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:24:54 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.197.19
2021/04/11 11:24:54 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:24:54 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/11 11:24:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:56 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 7.365718107s
2021/04/11 11:24:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:24:58 [ERR] memberlist: failed to receive: read tcp 10.242.219.70:8301->10.242.197.6:57306: i/o timeout from=10.242.197.6:57306
2021/04/11 11:24:58 [ERR] memberlist: Failed to send error: write tcp 10.242.219.70:8301->10.242.197.6:57306: i/o timeout from=10.242.197.6:57306
2021/04/11 11:24:59 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:57320->10.242.197.19:8302: i/o timeout
2021/04/11 11:24:59 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/04/11 11:24:59 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.197.19
2021/04/11 11:24:59 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/11 11:24:59 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/11 11:24:59 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:24:59 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 11:24:59 [INFO] consul: member 'dev-consul-7467bf7b46-bl49x' joined, marking health alive
2021/04/11 11:24:59 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.197.19
2021/04/11 11:24:59 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/11 11:25:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:00 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:25:01 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:39274->10.242.197.19:8300: i/o timeout
2021/04/11 11:25:01 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:25:01 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2)
2021/04/11 11:25:01 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:25:02 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.197.19
2021/04/11 11:25:02 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:25:02 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/11 11:25:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:05 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500260459s
2021/04/11 11:25:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:07 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:53138->10.242.197.6:8301: i/o timeout
2021/04/11 11:25:07 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:25:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:08 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 4.976459076s
2021/04/11 11:25:09 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:53640->10.242.197.19:8301: i/o timeout
2021/04/11 11:25:09 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:25:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:11 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:38962->10.242.197.19:8300: i/o timeout
2021/04/11 11:25:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:13 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:53710->10.242.197.19:8301: i/o timeout
2021/04/11 11:25:13 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:25:14 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500251976s
2021/04/11 11:25:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:14 [WARN] raft: Rejecting vote request from 10.242.197.19:8300 since we have a leader: 10.242.219.70:8300
2021/04/11 11:25:17 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:25:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:18 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:53336->10.242.197.6:8301: i/o timeout
2021/04/11 11:25:18 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:25:18 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.50033962s
2021/04/11 11:25:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:21 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:39442->10.242.197.19:8300: i/o timeout
2021/04/11 11:25:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:22 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:53404->10.242.197.6:8301: i/o timeout
2021/04/11 11:25:22 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:25:23 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500402394s
2021/04/11 11:25:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:25 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 11:25:25 [INFO] consul: member 'dev-consul-7467bf7b46-bl49x' failed, marking health critical
2021/04/11 11:25:25 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:53870->10.242.197.19:8301: i/o timeout
2021/04/11 11:25:26 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 4.974243521s
2021/04/11 11:25:26 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:53458->10.242.197.6:8301: i/o timeout
2021/04/11 11:25:26 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:25:27 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:25:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:28 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:53928->10.242.197.19:8301: i/o timeout
2021/04/11 11:25:28 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:25:28 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 7.417970371s
2021/04/11 11:25:28 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/11 11:25:28 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.197.6:46118
2021/04/11 11:25:29 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:57816->10.242.197.19:8302: i/o timeout
2021/04/11 11:25:29 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/11 11:25:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:31 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:54014->10.242.197.19:8301: i/o timeout
2021/04/11 11:25:31 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:25:32 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: read tcp 10.242.219.70:39632->10.242.197.19:8300: i/o timeout
2021/04/11 11:25:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:32 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:39784->10.242.197.19:8300: i/o timeout
2021/04/11 11:25:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:34 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:54036->10.242.197.19:8301: i/o timeout
2021/04/11 11:25:34 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:25:35 [ERR] memberlist: failed to receive: read tcp 10.242.219.70:8301->10.242.197.6:57562: i/o timeout from=10.242.197.6:57562
2021/04/11 11:25:35 [ERR] memberlist: Failed to send error: write tcp 10.242.219.70:8301->10.242.197.6:57562: i/o timeout from=10.242.197.6:57562
2021/04/11 11:25:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:36 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:54092->10.242.197.19:8301: i/o timeout
2021/04/11 11:25:36 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:25:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:39 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:54144->10.242.197.19:8301: i/o timeout
2021/04/11 11:25:39 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:25:39 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (1 peer confirmations)
2021/04/11 11:25:39 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.197.19
2021/04/11 11:25:39 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:25:39 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/11 11:25:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:41 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:54178->10.242.197.19:8301: i/o timeout
2021/04/11 11:25:41 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:25:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:42 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: read tcp 10.242.219.70:39962->10.242.197.19:8300: i/o timeout
2021/04/11 11:25:43 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:25:43 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:39988->10.242.197.19:8300: i/o timeout
2021/04/11 11:25:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:44 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:58076->10.242.197.19:8302: i/o timeout
2021/04/11 11:25:44 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/11 11:25:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:46 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:25:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:53 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: read tcp 10.242.219.70:40160->10.242.197.19:8300: i/o timeout
2021/04/11 11:25:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:54 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:58254->10.242.197.19:8302: i/o timeout
2021/04/11 11:25:54 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/11 11:25:54 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:25:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:25:56 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:40200->10.242.197.19:8300: i/o timeout
2021/04/11 11:25:57 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/04/11 11:25:57 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.197.19
2021/04/11 11:25:57 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/11 11:25:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:00 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.197.19
2021/04/11 11:26:00 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/11 11:26:01 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:26:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:04 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: read tcp 10.242.219.70:40344->10.242.197.19:8300: i/o timeout
2021/04/11 11:26:05 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:26:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:06 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.197.19:8301
2021/04/11 11:26:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:10 [ERR] memberlist: failed to receive: read tcp 10.242.219.70:8302->10.242.197.19:36418: i/o timeout from=10.242.197.19:36418
2021/04/11 11:26:10 [ERR] memberlist: Failed to send error: write tcp 10.242.219.70:8302->10.242.197.19:36418: i/o timeout from=10.242.197.19:36418
2021/04/11 11:26:11 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:40452->10.242.197.19:8300: i/o timeout
2021/04/11 11:26:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:14 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:58588->10.242.197.19:8302: i/o timeout
2021/04/11 11:26:14 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/11 11:26:15 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: read tcp 10.242.219.70:40520->10.242.197.19:8300: i/o timeout
2021/04/11 11:26:15 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 11:26:15 [INFO] consul: member 'dev-consul-7467bf7b46-bl49x' joined, marking health alive
2021/04/11 11:26:15 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:26:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:17 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:54336->10.242.197.6:8301: i/o timeout
2021/04/11 11:26:17 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:26:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:19 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:54374->10.242.197.6:8301: i/o timeout
2021/04/11 11:26:19 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:26:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:21 [INFO] memberlist: Marking dev-consul-7467bf7b46-bl49x as failed, suspect timeout reached (0 peer confirmations)
2021/04/11 11:26:21 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 11:26:21 [INFO] consul: member 'dev-consul-7467bf7b46-bl49x' failed, marking health critical
2021/04/11 11:26:21 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:54416->10.242.197.6:8301: i/o timeout
2021/04/11 11:26:21 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:26:21 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:26:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:24 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:58758->10.242.197.19:8302: i/o timeout
2021/04/11 11:26:24 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/11 11:26:25 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: read tcp 10.242.219.70:40704->10.242.197.19:8300: i/o timeout
2021/04/11 11:26:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:26 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:26:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:31 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:40802->10.242.197.19:8300: i/o timeout
2021/04/11 11:26:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:33 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/11 11:26:33 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/11 11:26:34 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:58924->10.242.197.19:8302: i/o timeout
2021/04/11 11:26:34 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/11 11:26:35 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:26:35 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/11 11:26:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:36 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: read tcp 10.242.219.70:40876->10.242.197.19:8300: i/o timeout
2021/04/11 11:26:37 [ERR] memberlist: Push/Pull with dev-consul-server-2.dc1 failed: read tcp 10.242.219.70:58514->10.242.197.19:8302: i/o timeout
2021/04/11 11:26:38 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:26:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:41 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:26:42 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/04/11 11:26:42 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.197.19
2021/04/11 11:26:42 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/11 11:26:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:46 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.197.19:8301
2021/04/11 11:26:48 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: read tcp 10.242.219.70:41066->10.242.197.19:8300: i/o timeout
2021/04/11 11:26:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:49 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:26:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:51 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:41128->10.242.197.19:8300: i/o timeout
2021/04/11 11:26:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:52 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/11 11:26:52 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.197.19:50630
2021/04/11 11:26:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:26:59 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: read tcp 10.242.219.70:41266->10.242.197.19:8300: i/o timeout
2021/04/11 11:27:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:01 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:27:02 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:27:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:04 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:27:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:06 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500352747s
2021/04/11 11:27:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:08 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 4.953712863s
2021/04/11 11:27:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:11 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 7.446932385s
2021/04/11 11:27:12 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:41460->10.242.197.19:8300: i/o timeout
2021/04/11 11:27:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:12 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2.dc1)
2021/04/11 11:27:13 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:27:13 [WARN] memberlist: Refuting a suspect message (from: dev-consul-7467bf7b46-bl49x)
2021/04/11 11:27:14 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:27:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:14 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.197.19
2021/04/11 11:27:14 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:27:14 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/11 11:27:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:16 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500344871s
2021/04/11 11:27:17 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:55772->10.242.197.19:8301: i/o timeout
2021/04/11 11:27:17 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:27:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:19 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:55810->10.242.197.19:8301: i/o timeout
2021/04/11 11:27:19 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:27:19 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.501267569s
2021/04/11 11:27:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:21 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:55856->10.242.197.19:8301: i/o timeout
2021/04/11 11:27:21 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:27:21 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (0 peer confirmations)
2021/04/11 11:27:21 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.197.19
2021/04/11 11:27:21 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:27:21 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/11 11:27:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:22 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:27:22 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:27:23 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500302405s
2021/04/11 11:27:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:26 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 4.919787923s
2021/04/11 11:27:26 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-bl49x 10.242.197.6:8301
2021/04/11 11:27:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:28 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 7.374864495s
2021/04/11 11:27:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:32 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: read tcp 10.242.219.70:41452->10.242.197.19:8300: i/o timeout
2021/04/11 11:27:32 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:41790->10.242.197.19:8300: i/o timeout
2021/04/11 11:27:32 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:27:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:38 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.197.19
2021/04/11 11:27:38 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/11 11:27:39 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:27:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:41 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.197.19
2021/04/11 11:27:41 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:27:41 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/11 11:27:41 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 11:27:41 [INFO] consul: member 'dev-consul-7467bf7b46-bl49x' joined, marking health alive
2021/04/11 11:27:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:46 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:55806->10.242.197.6:8301: i/o timeout
2021/04/11 11:27:46 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:27:46 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500326787s
2021/04/11 11:27:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:49 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:56264->10.242.197.19:8301: i/o timeout
2021/04/11 11:27:49 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:27:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:52 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:56340->10.242.197.19:8301: i/o timeout
2021/04/11 11:27:52 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:27:52 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:42126->10.242.197.19:8300: i/o timeout
2021/04/11 11:27:52 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500301556s
2021/04/11 11:27:53 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:27:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:55 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 4.988140835s
2021/04/11 11:27:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:56 [WARN] memberlist: Was able to connect to dev-consul-7467bf7b46-bl49x but other probes failed, network may be misconfigured
2021/04/11 11:27:57 [ERR] memberlist: failed to receive: read tcp 10.242.219.70:8301->10.242.197.19:57322: i/o timeout from=10.242.197.19:57322
2021/04/11 11:27:57 [ERR] memberlist: Failed to send error: write tcp 10.242.219.70:8301->10.242.197.19:57322: i/o timeout from=10.242.197.19:57322
2021/04/11 11:27:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:27:58 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (1 peer confirmations)
2021/04/11 11:27:58 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.197.19
2021/04/11 11:27:58 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:27:58 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/11 11:27:59 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:27:59 [WARN] memberlist: Was able to connect to dev-consul-server-2 but other probes failed, network may be misconfigured
2021/04/11 11:27:59 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.197.19
2021/04/11 11:27:59 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:27:59 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/11 11:28:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:28:01 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:56054->10.242.197.6:8301: i/o timeout
2021/04/11 11:28:01 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:28:01 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500249713s
2021/04/11 11:28:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:28:04 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 4.990069471s
2021/04/11 11:28:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:28:04 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:56064->10.242.197.6:8301: i/o timeout
2021/04/11 11:28:04 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:28:04 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 11:28:04 [INFO] consul: member 'dev-consul-7467bf7b46-bl49x' failed, marking health critical
2021/04/11 11:28:04 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 11:28:04 [INFO] consul: member 'dev-consul-7467bf7b46-bl49x' joined, marking health alive
2021/04/11 11:28:07 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:28:07 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/11 11:28:08 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500286533s
2021/04/11 11:28:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:28:08 [WARN] memberlist: Was able to connect to dev-consul-server-2 but other probes failed, network may be misconfigured
2021/04/11 11:28:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:28:10 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 4.972859258s
2021/04/11 11:28:11 [WARN] memberlist: Was able to connect to dev-consul-server-2 but other probes failed, network may be misconfigured
2021/04/11 11:28:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:28:12 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:56236->10.242.197.6:8301: i/o timeout
2021/04/11 11:28:12 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:28:12 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:42454->10.242.197.19:8300: i/o timeout
2021/04/11 11:28:12 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 7.460074551s
2021/04/11 11:28:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:28:14 [WARN] memberlist: Was able to connect to dev-consul-server-2.dc1 but other probes failed, network may be misconfigured
2021/04/11 11:28:15 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:56748->10.242.197.19:8301: i/o timeout
2021/04/11 11:28:15 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:28:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:28:17 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:56310->10.242.197.6:8301: i/o timeout
2021/04/11 11:28:17 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:28:17 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500414928s
2021/04/11 11:28:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:28:18 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.197.19
2021/04/11 11:28:18 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:28:18 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/11 11:28:19 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:56804->10.242.197.19:8301: i/o timeout
2021/04/11 11:28:19 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:28:20 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 4.931355439s
2021/04/11 11:28:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:28:21 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:56384->10.242.197.6:8301: i/o timeout
2021/04/11 11:28:21 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:28:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:28:22 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 7.430474377s
2021/04/11 11:28:22 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:28:23 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:56420->10.242.197.6:8301: i/o timeout
2021/04/11 11:28:23 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:28:23 [INFO] memberlist: Marking dev-consul-7467bf7b46-bl49x as failed, suspect timeout reached (1 peer confirmations)
2021/04/11 11:28:23 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 11:28:23 [INFO] consul: member 'dev-consul-7467bf7b46-bl49x' failed, marking health critical
2021/04/11 11:28:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:28:24 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:60738->10.242.197.19:8302: i/o timeout
2021/04/11 11:28:24 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/11 11:28:25 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:56452->10.242.197.6:8301: i/o timeout
2021/04/11 11:28:25 [INFO] memberlist: Suspect dev-consul-7467bf7b46-bl49x has failed, no acks received
2021/04/11 11:28:25 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: read tcp 10.242.219.70:41966->10.242.197.19:8300: i/o timeout
2021/04/11 11:28:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:28:26 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:28:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:28:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:28:31 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-qc8pz 10.242.150.36
2021/04/11 11:28:31 [INFO] consul: member 'dev-consul-7467bf7b46-qc8pz' joined, marking health alive
2021/04/11 11:28:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:28:32 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: read tcp 10.242.219.70:42800->10.242.197.19:8300: i/o timeout
2021/04/11 11:28:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:28:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:28:36 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: read tcp 10.242.219.70:42858->10.242.197.19:8300: i/o timeout
2021/04/11 11:28:37 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:28:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:28:39 [ERR] memberlist: Failed fallback ping: read tcp 10.242.219.70:32776->10.242.197.19:8302: i/o timeout
2021/04/11 11:28:39 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/11 11:28:41 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:28:41 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:28:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:28:42 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:28:43 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:28:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:28:44 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.504435677s
2021/04/11 11:28:44 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:28:45 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2.dc1)
2021/04/11 11:28:45 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.197.19
2021/04/11 11:28:45 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:28:45 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/11 11:28:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:28:47 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500624152s
2021/04/11 11:28:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:28:50 [WARN] memberlist: Refuting a suspect message (from: dev-consul-7467bf7b46-bl49x)
2021/04/11 11:28:50 [WARN] memberlist: Was able to connect to dev-consul-server-2 but other probes failed, network may be misconfigured
2021/04/11 11:28:50 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 11:28:50 [INFO] consul: member 'dev-consul-7467bf7b46-bl49x' joined, marking health alive
2021/04/11 11:28:50 [INFO] raft: pipelining replication to peer {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}
2021/04/11 11:28:56 [INFO] serf: EventMemberLeave: dev-consul-7467bf7b46-bl49x 10.242.197.6
2021/04/11 11:28:56 [INFO] consul: member 'dev-consul-7467bf7b46-bl49x' left, deregistering
2021/04/11 11:28:57 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: EOF
2021/04/11 11:28:57 [INFO] raft: aborting pipeline replication to peer {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}
2021/04/11 11:28:57 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: dial tcp ->10.242.197.19:8300: getsockopt: connection refused
2021/04/11 11:28:57 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: dial tcp ->10.242.197.19:8300: getsockopt: connection refused
2021/04/11 11:28:57 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: dial tcp ->10.242.197.19:8300: getsockopt: connection refused
2021/04/11 11:28:57 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: dial tcp ->10.242.197.19:8300: getsockopt: connection refused
2021/04/11 11:28:57 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: dial tcp ->10.242.197.19:8300: getsockopt: connection refused
2021/04/11 11:28:57 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: dial tcp ->10.242.197.19:8300: getsockopt: connection refused
2021/04/11 11:28:58 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: dial tcp ->10.242.197.19:8300: getsockopt: connection refused
2021/04/11 11:28:58 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: dial tcp ->10.242.197.19:8300: getsockopt: connection refused
2021/04/11 11:28:58 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: stream closed
2021/04/11 11:28:58 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:28:58 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: dial tcp ->10.242.197.19:8300: getsockopt: connection refused
2021/04/11 11:28:59 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: dial tcp ->10.242.197.19:8300: getsockopt: connection refused
2021/04/11 11:28:59 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: dial tcp ->10.242.197.19:8300: getsockopt: connection refused
2021/04/11 11:28:59 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:28:59 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 2.500319854s
2021/04/11 11:28:59 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: dial tcp ->10.242.197.19:8300: getsockopt: connection refused
2021/04/11 11:29:00 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error getting client: failed to get conn: dial tcp ->10.242.197.19:8300: getsockopt: connection refused
2021/04/11 11:29:00 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: dial tcp ->10.242.197.19:8300: getsockopt: connection refused
2021/04/11 11:29:00 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: dial tcp ->10.242.197.19:8300: getsockopt: connection refused
2021/04/11 11:29:01 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:29:01 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/11 11:29:01 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: dial tcp ->10.242.197.19:8300: getsockopt: connection refused
2021/04/11 11:29:02 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 4.948260503s
2021/04/11 11:29:02 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/04/11 11:29:02 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.197.19
2021/04/11 11:29:02 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.197.19:8300) (DC: dc1)
2021/04/11 11:29:02 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/11 11:29:03 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:29:03 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:29:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:04 [WARN] raft: Failed to contact 93926a22-3b0f-30f5-53a3-a64ad4920efa in 7.447761531s
2021/04/11 11:29:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:12 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: dial tcp ->10.242.197.19:8300: i/o timeout
2021/04/11 11:29:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:12 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error getting client: failed to get conn: dial tcp ->10.242.197.19:8300: i/o timeout
2021/04/11 11:29:13 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: dial tcp ->10.242.197.19:8300: i/o timeout
2021/04/11 11:29:13 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:29:15 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:29:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:18 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:29:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:23 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: dial tcp ->10.242.197.19:8300: i/o timeout
2021/04/11 11:29:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:24 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error getting client: failed to get conn: dial tcp ->10.242.197.19:8300: i/o timeout
2021/04/11 11:29:24 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:29:27 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:29:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:28 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: dial tcp ->10.242.197.19:8300: i/o timeout
2021/04/11 11:29:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:32 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/11 11:29:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:34 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: dial tcp ->10.242.197.19:8300: i/o timeout
2021/04/11 11:29:36 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error getting client: failed to get conn: dial tcp ->10.242.197.19:8300: i/o timeout
2021/04/11 11:29:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:36 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:29:38 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:29:39 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:29:39 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/11 11:29:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:46 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: dial tcp ->10.242.197.19:8300: i/o timeout
2021/04/11 11:29:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:48 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error getting client: failed to get conn: dial tcp ->10.242.197.19:8300: i/o timeout
2021/04/11 11:29:48 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: dial tcp ->10.242.197.19:8300: i/o timeout
2021/04/11 11:29:49 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:29:51 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:29:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:29:58 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:29:59 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: dial tcp ->10.242.197.19:8300: i/o timeout
2021/04/11 11:30:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:30:00 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error getting client: failed to get conn: dial tcp ->10.242.197.19:8300: i/o timeout
2021/04/11 11:30:02 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/04/11 11:30:02 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.197.19
2021/04/11 11:30:02 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/11 11:30:03 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/11 11:30:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:30:05 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:30:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:30:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/11 11:30:08 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.197.54
2021/04/11 11:30:08 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.197.54:8300) (DC: dc1)
2021/04/11 11:30:08 [INFO] raft: Updating configuration with AddNonvoter (4578bcf6-0871-6c6a-06b9-edd6be698fa7, 10.242.197.54:8300) to [{Suffrage:Voter ID:93926a22-3b0f-30f5-53a3-a64ad4920efa Address:10.242.197.19:8300} {Suffrage:Voter ID:cece5e1c-d581-d6d7-9a7a-03a4bf5634cf Address:10.242.219.70:8300} {Suffrage:Voter ID:68b413ff-82e9-adbb-03e8-30a9524f6f0c Address:10.242.150.7:8300} {Suffrage:Nonvoter ID:4578bcf6-0871-6c6a-06b9-edd6be698fa7 Address:10.242.197.54:8300}]
2021/04/11 11:30:08 [INFO] raft: Added peer 4578bcf6-0871-6c6a-06b9-edd6be698fa7, starting replication
2021/04/11 11:30:08 [ERR] memberlist: Conflicting address for dev-consul-server-2.dc1. Mine: 10.242.197.19:8302 Theirs: 10.242.197.54:8302
2021/04/11 11:30:08 [WARN] serf: Name conflict for 'dev-consul-server-2.dc1' both 10.242.197.19:8302 and 10.242.197.54:8302 are claiming
2021/04/11 11:30:08 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/11 11:30:08 [WARN] raft: AppendEntries to {Nonvoter 4578bcf6-0871-6c6a-06b9-edd6be698fa7 10.242.197.54:8300} rejected, sending older logs (next: 1)
2021/04/11 11:30:08 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: dial tcp ->10.242.197.19:8300: i/o timeout
2021/04/11 11:30:08 [ERR] memberlist: Conflicting address for dev-consul-server-2.dc1. Mine: 10.242.197.19:8302 Theirs: 10.242.197.54:8302
2021/04/11 11:30:08 [WARN] serf: Name conflict for 'dev-consul-server-2.dc1' both 10.242.197.19:8302 and 10.242.197.54:8302 are claiming
2021/04/11 11:30:08 [ERR] memberlist: Conflicting address for dev-consul-server-2.dc1. Mine: 10.242.197.19:8302 Theirs: 10.242.197.54:8302
2021/04/11 11:30:08 [WARN] serf: Name conflict for 'dev-consul-server-2.dc1' both 10.242.197.19:8302 and 10.242.197.54:8302 are claiming
2021/04/11 11:30:09 [ERR] memberlist: Conflicting address for dev-consul-server-2.dc1. Mine: 10.242.197.19:8302 Theirs: 10.242.197.54:8302
2021/04/11 11:30:09 [WARN] serf: Name conflict for 'dev-consul-server-2.dc1' both 10.242.197.19:8302 and 10.242.197.54:8302 are claiming
2021/04/11 11:30:09 [ERR] memberlist: Conflicting address for dev-consul-server-2.dc1. Mine: 10.242.197.19:8302 Theirs: 10.242.197.54:8302
2021/04/11 11:30:09 [WARN] serf: Name conflict for 'dev-consul-server-2.dc1' both 10.242.197.19:8302 and 10.242.197.54:8302 are claiming
2021/04/11 11:30:09 [INFO] raft: pipelining replication to peer {Nonvoter 4578bcf6-0871-6c6a-06b9-edd6be698fa7 10.242.197.54:8300}
2021/04/11 11:30:12 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error getting client: failed to get conn: dial tcp ->10.242.197.19:8300: i/o timeout
2021/04/11 11:30:12 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/11 11:30:15 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: dial tcp ->10.242.197.19:8300: i/o timeout
2021/04/11 11:30:18 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:30:24 [INFO] autopilot: Promoting Server (ID: "4578bcf6-0871-6c6a-06b9-edd6be698fa7" Address: "10.242.197.54:8300") to voter
2021/04/11 11:30:24 [INFO] raft: Updating configuration with AddStaging (4578bcf6-0871-6c6a-06b9-edd6be698fa7, 10.242.197.54:8300) to [{Suffrage:Voter ID:93926a22-3b0f-30f5-53a3-a64ad4920efa Address:10.242.197.19:8300} {Suffrage:Voter ID:cece5e1c-d581-d6d7-9a7a-03a4bf5634cf Address:10.242.219.70:8300} {Suffrage:Voter ID:68b413ff-82e9-adbb-03e8-30a9524f6f0c Address:10.242.150.7:8300} {Suffrage:Voter ID:4578bcf6-0871-6c6a-06b9-edd6be698fa7 Address:10.242.197.54:8300}]
2021/04/11 11:30:24 [INFO] autopilot: Attempting removal of stale Server (ID: "93926a22-3b0f-30f5-53a3-a64ad4920efa" Address: "10.242.197.19:8300")
2021/04/11 11:30:24 [INFO] raft: Updating configuration with RemoveServer (93926a22-3b0f-30f5-53a3-a64ad4920efa, ) to [{Suffrage:Voter ID:cece5e1c-d581-d6d7-9a7a-03a4bf5634cf Address:10.242.219.70:8300} {Suffrage:Voter ID:68b413ff-82e9-adbb-03e8-30a9524f6f0c Address:10.242.150.7:8300} {Suffrage:Voter ID:4578bcf6-0871-6c6a-06b9-edd6be698fa7 Address:10.242.197.54:8300}]
2021/04/11 11:30:24 [INFO] raft: Removed peer 93926a22-3b0f-30f5-53a3-a64ad4920efa, stopping replication after 978
2021/04/11 11:30:27 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:30:28 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: dial tcp ->10.242.197.19:8300: i/o timeout
2021/04/11 11:30:37 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: dial tcp ->10.242.197.19:8300: i/o timeout
2021/04/11 11:30:37 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.197.19:8302
2021/04/11 11:30:39 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:30:48 [WARN] Unable to get address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa, using fallback address 10.242.197.19:8300: Could not find address for server id 93926a22-3b0f-30f5-53a3-a64ad4920efa
2021/04/11 11:30:49 [ERR] raft: Failed to AppendEntries to {Voter 93926a22-3b0f-30f5-53a3-a64ad4920efa 10.242.197.19:8300}: dial tcp ->10.242.197.19:8300: i/o timeout
2021/04/11 11:30:58 [ERR] raft: Failed to heartbeat to 10.242.197.19:8300: dial tcp ->10.242.197.19:8300: i/o timeout
2021/04/11 11:32:03 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.197.54
2021/04/11 11:32:03 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"