By type
+ CONSUL_BIND=
+ [ -n ]
+ CONSUL_CLIENT=
+ [ -n ]
+ CONSUL_DATA_DIR=/consul/data
+ CONSUL_CONFIG_DIR=/consul/config
+ [ -n ]
+ [ a = - ]
+ [ agent = agent ]
+ shift
+ set -- consul agent -data-dir=/consul/data -config-dir=/consul/config -bootstrap-expect=3 -enable-script-checks -retry-join=dev-consul-server-0.consul-server -retry-join=dev-consul-server-1.consul-server -retry-join=dev-consul-server-2.consul-server -client=0.0.0.0 -server -ui
+ [ consul = consul ]
+ stat -c %u /consul/data
+ id -u consul
+ [ 100 != 100 ]
+ stat -c %u /consul/config
+ id -u consul
+ [ 100 != 100 ]
+ [ ! -z ]
+ exec consul agent -data-dir=/consul/data -config-dir=/consul/config -bootstrap-expect=3 -enable-script-checks -retry-join=dev-consul-server-0.consul-server -retry-join=dev-consul-server-1.consul-server -retry-join=dev-consul-server-2.consul-server -client=0.0.0.0 -server -ui
bootstrap_expect > 0: expecting 3 servers
==> Starting Consul agent...
==> Consul agent running!
Version: 'v1.0.6'
Node ID: 'e524e1aa-4527-3cd0-d6fe-8da75eba4af0'
Node name: 'dev-consul-server-0'
Datacenter: 'dc1' (Segment: '')
Server: true (Bootstrap: false)
Client Addr: [0.0.0.0] (HTTP: 8500, HTTPS: -1, DNS: 8600)
Cluster Addr: 10.242.187.2 (LAN: 8301, WAN: 8302)
Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false
==> Log data will now stream in as it occurs:
2021/04/25 22:39:24 [INFO] raft: Initial configuration (index=0): []
2021/04/25 22:39:24 [INFO] raft: Node at 10.242.187.2:8300 [Follower] entering Follower state (Leader: "")
2021/04/25 22:39:24 [INFO] serf: EventMemberJoin: dev-consul-server-0.dc1 10.242.187.2
2021/04/25 22:39:24 [INFO] serf: EventMemberJoin: dev-consul-server-0 10.242.187.2
2021/04/25 22:39:24 [INFO] consul: Adding LAN server dev-consul-server-0 (Addr: tcp/10.242.187.2:8300) (DC: dc1)
2021/04/25 22:39:24 [INFO] consul: Handled member-join event for server "dev-consul-server-0.dc1" in area "wan"
2021/04/25 22:39:24 [INFO] agent: Started DNS server 0.0.0.0:8600 (tcp)
2021/04/25 22:39:24 [INFO] agent: Started DNS server 0.0.0.0:8600 (udp)
2021/04/25 22:39:24 [INFO] agent: Started HTTP server on [::]:8500 (tcp)
2021/04/25 22:39:24 [INFO] agent: started state syncer
2021/04/25 22:39:24 [INFO] agent: Retry join LAN is supported for: aliyun aws azure digitalocean gce os scaleway softlayer
2021/04/25 22:39:24 [INFO] agent: Joining LAN cluster...
2021/04/25 22:39:24 [INFO] agent: (LAN) joining: [dev-consul-server-0.consul-server dev-consul-server-1.consul-server dev-consul-server-2.consul-server]
2021/04/25 22:39:24 [WARN] memberlist: Failed to resolve dev-consul-server-0.consul-server: lookup dev-consul-server-0.consul-server on 10.96.0.10:53: no such host
2021/04/25 22:39:24 [WARN] memberlist: Failed to resolve dev-consul-server-1.consul-server: lookup dev-consul-server-1.consul-server on 10.96.0.10:53: no such host
2021/04/25 22:39:24 [WARN] memberlist: Failed to resolve dev-consul-server-2.consul-server: lookup dev-consul-server-2.consul-server on 10.96.0.10:53: no such host
2021/04/25 22:39:24 [INFO] agent: (LAN) joined: 0 Err: 3 error(s) occurred:
* Failed to resolve dev-consul-server-0.consul-server: lookup dev-consul-server-0.consul-server on 10.96.0.10:53: no such host
* Failed to resolve dev-consul-server-1.consul-server: lookup dev-consul-server-1.consul-server on 10.96.0.10:53: no such host
* Failed to resolve dev-consul-server-2.consul-server: lookup dev-consul-server-2.consul-server on 10.96.0.10:53: no such host
2021/04/25 22:39:24 [WARN] agent: Join LAN failed: , retrying in 30s
2021/04/25 22:39:31 [ERR] agent: failed to sync remote state: No cluster leader
2021/04/25 22:39:31 [WARN] raft: no known peers, aborting election
==> Failed to check for updates: Get https://checkpoint-api.hashicorp.com/v1/check/consul?arch=amd64&os=linux&signature=45ae42b2-bba6-204a-93fe-cd68272a4f30&version=1.0.6: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
2021/04/25 22:39:51 [ERR] agent: Coordinate update error: No cluster leader
2021/04/25 22:39:54 [INFO] serf: EventMemberJoin: dev-consul-server-1 10.242.32.71
2021/04/25 22:39:54 [INFO] consul: Adding LAN server dev-consul-server-1 (Addr: tcp/10.242.32.71:8300) (DC: dc1)
2021/04/25 22:39:54 [INFO] serf: EventMemberJoin: dev-consul-server-1.dc1 10.242.32.71
2021/04/25 22:39:54 [INFO] consul: Handled member-join event for server "dev-consul-server-1.dc1" in area "wan"
2021/04/25 22:39:54 [INFO] agent: (LAN) joining: [dev-consul-server-0.consul-server dev-consul-server-1.consul-server dev-consul-server-2.consul-server]
2021/04/25 22:39:54 [WARN] memberlist: Failed to resolve dev-consul-server-1.consul-server: lookup dev-consul-server-1.consul-server on 10.96.0.10:53: no such host
2021/04/25 22:39:54 [WARN] memberlist: Failed to resolve dev-consul-server-2.consul-server: lookup dev-consul-server-2.consul-server on 10.96.0.10:53: no such host
2021/04/25 22:39:54 [INFO] agent: (LAN) joined: 1 Err:
2021/04/25 22:39:54 [INFO] agent: Join LAN completed. Synced with 1 initial agents
2021/04/25 22:39:56 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 22:40:06 [ERR] agent: failed to sync remote state: No cluster leader
2021/04/25 22:40:13 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 22:40:13 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 22:40:13 [INFO] consul: Found expected number of peers, attempting bootstrap: 10.242.93.149:8300,10.242.187.2:8300,10.242.32.71:8300
2021/04/25 22:40:13 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 22:40:13 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 22:40:16 [WARN] raft: Heartbeat timeout from "" reached, starting election
2021/04/25 22:40:16 [INFO] raft: Node at 10.242.187.2:8300 [Candidate] entering Candidate state in term 2
2021/04/25 22:40:17 [INFO] raft: Election won. Tally: 2
2021/04/25 22:40:17 [INFO] raft: Node at 10.242.187.2:8300 [Leader] entering Leader state
2021/04/25 22:40:17 [INFO] raft: Added peer f66dacd5-df84-469e-7a9b-4103153d1fdc, starting replication
2021/04/25 22:40:17 [INFO] raft: Added peer 9fcb14a9-100e-2d75-3e7a-6fe772ff1f8f, starting replication
2021/04/25 22:40:17 [INFO] consul: cluster leadership acquired
2021/04/25 22:40:17 [INFO] consul: New leader elected: dev-consul-server-0
2021/04/25 22:40:17 [INFO] raft: pipelining replication to peer {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}
2021/04/25 22:40:17 [WARN] raft: AppendEntries to {Voter 9fcb14a9-100e-2d75-3e7a-6fe772ff1f8f 10.242.32.71:8300} rejected, sending older logs (next: 1)
2021/04/25 22:40:17 [INFO] raft: pipelining replication to peer {Voter 9fcb14a9-100e-2d75-3e7a-6fe772ff1f8f 10.242.32.71:8300}
2021/04/25 22:40:17 [INFO] consul: member 'dev-consul-server-0' joined, marking health alive
2021/04/25 22:40:18 [INFO] consul: member 'dev-consul-server-1' joined, marking health alive
2021/04/25 22:40:19 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' joined, marking health alive
2021/04/25 22:40:19 [INFO] agent: Synced node info
2021/04/25 22:40:19 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/25 22:45:14 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/25 22:45:16 [WARN] consul: error getting server health from "dev-consul-server-0": last request still outstanding
2021/04/25 22:49:26 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/25 22:49:26 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 22:49:26 [WARN] consul: error getting server health from "dev-consul-server-0": last request still outstanding
2021/04/25 22:58:48 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:41552->10.242.93.149:8301: i/o timeout
2021/04/25 22:58:48 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 22:59:06 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 22:59:07 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 22:59:20 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:47504->10.242.93.140:8301: i/o timeout
2021/04/25 22:59:20 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:00:30 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:00:30 [WARN] consul: error getting server health from "dev-consul-server-1": context deadline exceeded
2021/04/25 23:01:52 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:01:52 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:49916->10.242.93.140:8301: i/o timeout
2021/04/25 23:01:52 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:01:58 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:49992->10.242.93.140:8301: i/o timeout
2021/04/25 23:02:01 [INFO] memberlist: Marking dev-consul-7467bf7b46-ww995 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:02:01 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:02:01 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' failed, marking health critical
2021/04/25 23:02:02 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:02:02 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:50058->10.242.93.140:8301: i/o timeout
2021/04/25 23:02:02 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:02:03 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500243217s
2021/04/25 23:02:03 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:05 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:05 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:44570->10.242.93.149:8301: i/o timeout
2021/04/25 23:02:05 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:02:05 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.960141593s
2021/04/25 23:02:07 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:07 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:44622->10.242.93.149:8301: i/o timeout
2021/04/25 23:02:07 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:02:09 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:09 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:44634->10.242.93.149:8301: i/o timeout
2021/04/25 23:02:09 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:02:09 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500201088s
2021/04/25 23:02:11 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:11 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:44682->10.242.93.149:8301: i/o timeout
2021/04/25 23:02:11 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:02:12 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.969811537s
2021/04/25 23:02:13 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:13 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:44702->10.242.93.149:8301: i/o timeout
2021/04/25 23:02:13 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:02:14 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.442569321s
2021/04/25 23:02:15 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:15 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:02:15 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:02:15 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/25 23:02:15 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:44726->10.242.93.149:8301: i/o timeout
2021/04/25 23:02:15 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:02:17 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:17 [ERR] raft: Failed to pipeline AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: write tcp 10.242.187.2:59816->10.242.93.149:8300: use of closed network connection
2021/04/25 23:02:17 [INFO] raft: aborting pipeline replication to peer {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}
2021/04/25 23:02:17 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:02:18 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:59824->10.242.93.149:8300: i/o timeout
2021/04/25 23:02:18 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:02:19 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:19 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500178423s
2021/04/25 23:02:21 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:22 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.987755922s
2021/04/25 23:02:23 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:02:23 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.149:57732
2021/04/25 23:02:23 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:24 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:59212->10.242.93.149:8302: i/o timeout
2021/04/25 23:02:24 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:02:24 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:02:24 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.472144569s
2021/04/25 23:02:25 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:27 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:27 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:43472->10.242.93.149:8300: i/o timeout
2021/04/25 23:02:27 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:02:28 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:43492->10.242.93.149:8300: i/o timeout
2021/04/25 23:02:29 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:02:29 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:31 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:33 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:34 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:59356->10.242.93.149:8302: i/o timeout
2021/04/25 23:02:34 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:02:35 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:35 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:02:35 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' joined, marking health alive
2021/04/25 23:02:37 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:37 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:43612->10.242.93.149:8300: i/o timeout
2021/04/25 23:02:37 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:02:38 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:50556->10.242.93.140:8301: i/o timeout
2021/04/25 23:02:38 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:02:39 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:43638->10.242.93.149:8300: i/o timeout
2021/04/25 23:02:39 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:40 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:02:41 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:42 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:50622->10.242.93.140:8301: i/o timeout
2021/04/25 23:02:42 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:02:43 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:44 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:59498->10.242.93.149:8302: i/o timeout
2021/04/25 23:02:44 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:02:44 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/04/25 23:02:44 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:02:44 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:02:44 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:50636->10.242.93.140:8301: i/o timeout
2021/04/25 23:02:44 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:02:45 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:46 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:50656->10.242.93.140:8301: i/o timeout
2021/04/25 23:02:46 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:02:47 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:47 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:43752->10.242.93.149:8300: i/o timeout
2021/04/25 23:02:47 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:02:48 [INFO] memberlist: Marking dev-consul-7467bf7b46-ww995 as failed, suspect timeout reached (1 peer confirmations)
2021/04/25 23:02:48 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:02:48 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' failed, marking health critical
2021/04/25 23:02:48 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:50708->10.242.93.140:8301: i/o timeout
2021/04/25 23:02:48 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:02:48 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8301->10.242.93.140:44838: i/o timeout from=10.242.93.140:44838
2021/04/25 23:02:48 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8301->10.242.93.140:44838: i/o timeout from=10.242.93.140:44838
2021/04/25 23:02:49 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:50 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:43790->10.242.93.149:8300: i/o timeout
2021/04/25 23:02:51 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:02:51 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:53 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:54 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:59572->10.242.93.149:8302: i/o timeout
2021/04/25 23:02:54 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:02:54 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:02:55 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:57 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:57 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:43904->10.242.93.149:8300: i/o timeout
2021/04/25 23:02:57 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:02:59 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:02:59 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:02:59 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:03:01 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:43964->10.242.93.149:8300: i/o timeout
2021/04/25 23:03:02 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:03:02 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:03:02 [WARN] consul: error getting server health from "dev-consul-server-1": context deadline exceeded
2021/04/25 23:03:03 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:04 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:03:05 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:06 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:03:06 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.149:54798
2021/04/25 23:03:07 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:07 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:44040->10.242.93.149:8300: i/o timeout
2021/04/25 23:03:07 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:03:09 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:03:09 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:03:09 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/25 23:03:09 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:03:09 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:03:09 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:11 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:11 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:45534->10.242.93.149:8301: i/o timeout
2021/04/25 23:03:11 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:03:11 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500268626s
2021/04/25 23:03:13 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:13 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.96147368s
2021/04/25 23:03:14 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:45558->10.242.93.149:8301: i/o timeout
2021/04/25 23:03:14 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:03:15 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:16 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.449688605s
2021/04/25 23:03:16 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:45578->10.242.93.149:8301: i/o timeout
2021/04/25 23:03:16 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:03:17 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:17 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:44186->10.242.93.149:8300: i/o timeout
2021/04/25 23:03:19 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:19 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:45646->10.242.93.149:8301: i/o timeout
2021/04/25 23:03:19 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:03:19 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:44112->10.242.93.149:8300: i/o timeout
2021/04/25 23:03:21 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:21 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:45678->10.242.93.149:8301: i/o timeout
2021/04/25 23:03:21 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:03:22 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (1 peer confirmations)
2021/04/25 23:03:22 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:03:22 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:03:22 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/25 23:03:23 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:23 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:45692->10.242.93.149:8301: i/o timeout
2021/04/25 23:03:23 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:03:24 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:60070->10.242.93.149:8302: i/o timeout
2021/04/25 23:03:24 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:03:25 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:27 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:27 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:44326->10.242.93.149:8300: i/o timeout
2021/04/25 23:03:28 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:03:29 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:30 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:44364->10.242.93.149:8300: i/o timeout
2021/04/25 23:03:31 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:03:31 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:33 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:34 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:60212->10.242.93.149:8302: i/o timeout
2021/04/25 23:03:34 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:03:35 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:37 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:38 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:44476->10.242.93.149:8300: i/o timeout
2021/04/25 23:03:38 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:03:39 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:03:39 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:41 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:44530->10.242.93.149:8300: i/o timeout
2021/04/25 23:03:41 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:41 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:03:41 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:03:41 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:03:44 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:03:44 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/25 23:03:44 [WARN] consul: error getting server health from "dev-consul-server-1": context deadline exceeded
2021/04/25 23:03:45 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:47 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:48 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:44630->10.242.93.149:8300: i/o timeout
2021/04/25 23:03:49 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/04/25 23:03:49 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:03:49 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:03:49 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:49 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:60350->10.242.93.149:8302: i/o timeout
2021/04/25 23:03:49 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:03:50 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:03:51 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:51 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:44676->10.242.93.149:8300: i/o timeout
2021/04/25 23:03:52 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:03:53 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:55 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:57 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:03:59 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:00 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:44784->10.242.93.149:8300: i/o timeout
2021/04/25 23:04:01 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:02 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:44822->10.242.93.149:8300: i/o timeout
2021/04/25 23:04:02 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:04:03 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:03 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:04:05 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:07 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:09 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:11 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:12 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:44960->10.242.93.149:8300: i/o timeout
2021/04/25 23:04:13 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:13 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:44964->10.242.93.149:8300: i/o timeout
2021/04/25 23:04:14 [WARN] memberlist: Refuting a suspect message (from: dev-consul-7467bf7b46-ww995)
2021/04/25 23:04:14 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:04:15 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:17 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:17 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:04:19 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:04:19 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:19 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8302->10.242.93.149:46488: i/o timeout from=10.242.93.149:46488
2021/04/25 23:04:19 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8302->10.242.93.149:46488: i/o timeout from=10.242.93.149:46488
2021/04/25 23:04:21 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:23 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:24 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:45124->10.242.93.149:8300: i/o timeout
2021/04/25 23:04:25 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:25 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:04:27 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:27 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:45168->10.242.93.149:8300: i/o timeout
2021/04/25 23:04:29 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:31 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:33 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:35 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:35 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:45274->10.242.93.149:8300: i/o timeout
2021/04/25 23:04:37 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:04:37 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:38 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:04:39 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:04:39 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:41 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:43 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:44 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8302->10.242.93.149:46854: i/o timeout from=10.242.93.149:46854
2021/04/25 23:04:44 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8302->10.242.93.149:46854: i/o timeout from=10.242.93.149:46854
2021/04/25 23:04:45 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:04:45 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.140:44066
2021/04/25 23:04:45 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:47 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:45436->10.242.93.149:8300: i/o timeout
2021/04/25 23:04:47 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:48 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:45446->10.242.93.149:8300: i/o timeout
2021/04/25 23:04:49 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:49 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:04:51 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:53 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:55 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:57 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:58 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:04:59 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:04:59 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:04:59 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:45600->10.242.93.149:8300: i/o timeout
2021/04/25 23:05:01 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:02 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:05:03 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:05 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:07 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:08 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:45728->10.242.93.149:8300: i/o timeout
2021/04/25 23:05:09 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:11 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:12 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:45786->10.242.93.149:8300: i/o timeout
2021/04/25 23:05:13 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:15 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:17 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:18 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:05:18 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:05:19 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:21 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:23 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:25 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:25 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:05:25 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:05:28 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:05:28 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/25 23:05:28 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:45996->10.242.93.149:8300: i/o timeout
2021/04/25 23:05:28 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:46004->10.242.93.149:8300: i/o timeout
2021/04/25 23:05:29 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:31 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:05:31 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.149:55568
2021/04/25 23:05:31 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:33 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:35 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:37 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:38 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:05:39 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:05:39 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:39 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:05:41 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:43 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:45 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:47 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:48 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:46270->10.242.93.149:8300: i/o timeout
2021/04/25 23:05:49 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:49 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:46282->10.242.93.149:8300: i/o timeout
2021/04/25 23:05:51 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:53 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:55 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:57 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:05:59 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:05:59 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:00 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:06:01 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:03 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:05 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:07 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:07 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:06:07 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:06:09 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:06:09 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:46544->10.242.93.149:8300: i/o timeout
2021/04/25 23:06:10 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:06:10 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/25 23:06:10 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:46562->10.242.93.149:8300: i/o timeout
2021/04/25 23:06:11 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:13 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:15 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:17 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:19 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:06:19 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:19 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:06:21 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:21 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:06:23 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:25 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:27 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:29 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:29 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:46822->10.242.93.149:8300: i/o timeout
2021/04/25 23:06:31 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:31 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:46864->10.242.93.149:8300: i/o timeout
2021/04/25 23:06:33 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:35 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:37 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:39 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:39 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:06:41 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:42 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:06:43 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:45 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:47 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:49 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:49 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:06:49 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:06:49 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:47094->10.242.93.149:8300: i/o timeout
2021/04/25 23:06:52 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:06:52 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:47138->10.242.93.149:8300: i/o timeout
2021/04/25 23:06:53 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:55 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:57 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:59 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:06:59 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:06:59 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:07:01 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:03 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:03 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:07:05 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:07 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:09 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:07:09 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:09 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:47370->10.242.93.149:8300: i/o timeout
2021/04/25 23:07:11 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:13 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:13 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:47416->10.242.93.149:8300: i/o timeout
2021/04/25 23:07:15 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:17 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:19 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:20 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:07:21 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:23 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:24 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:07:25 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:27 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:29 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:30 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:47646->10.242.93.149:8300: i/o timeout
2021/04/25 23:07:31 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:31 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:07:31 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:07:34 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:47694->10.242.93.149:8300: i/o timeout
2021/04/25 23:07:34 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:07:34 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/25 23:07:35 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:37 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:39 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:07:39 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:40 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:07:41 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:43 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:45 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:07:45 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:47 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:49 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:50 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:47930->10.242.93.149:8300: i/o timeout
2021/04/25 23:07:51 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:53 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:55 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:47998->10.242.93.149:8300: i/o timeout
2021/04/25 23:07:55 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:07:55 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.149:56346
2021/04/25 23:07:55 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:57 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:07:59 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:00 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:08:01 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:03 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:05 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:06 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:08:07 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:09 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:08:09 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:10 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:48202->10.242.93.149:8300: i/o timeout
2021/04/25 23:08:11 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:13 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:13 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:08:13 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:08:16 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:48280->10.242.93.149:8300: i/o timeout
2021/04/25 23:08:16 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:08:16 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/25 23:08:16 [WARN] consul: error getting server health from "dev-consul-server-1": context deadline exceeded
2021/04/25 23:08:17 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:19 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:08:19 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:20 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:08:21 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:23 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:24 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-fdl7b 10.242.32.109
2021/04/25 23:08:24 [INFO] consul: member 'dev-consul-7467bf7b46-fdl7b' joined, marking health alive
2021/04/25 23:08:25 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:27 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:08:29 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:30 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:48500->10.242.93.149:8300: i/o timeout
2021/04/25 23:08:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:35 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:35 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:37 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:48646->10.242.93.149:8300: i/o timeout
2021/04/25 23:08:37 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:39 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:41 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:08:41 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:43 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:43 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8301->10.242.93.149:41604: i/o timeout from=10.242.93.149:41604
2021/04/25 23:08:43 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8301->10.242.93.149:41604: i/o timeout from=10.242.93.149:41604
2021/04/25 23:08:45 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:47 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:08:49 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:50 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/25 23:08:50 [WARN] consul: error getting server health from "dev-consul-server-1": context deadline exceeded
2021/04/25 23:08:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:50 [WARN] consul: error getting server health from "dev-consul-server-0": last request still outstanding
2021/04/25 23:08:51 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:48858->10.242.93.149:8300: i/o timeout
2021/04/25 23:08:51 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:53 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:55 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:08:55 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:08:55 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:08:57 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:48954->10.242.93.149:8300: i/o timeout
2021/04/25 23:08:59 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:08:59 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:01 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:09:01 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:03 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:05 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:07 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:08 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:09:09 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:09:09 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:11 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:49142->10.242.93.149:8300: i/o timeout
2021/04/25 23:09:11 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:13 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:15 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:17 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:18 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:49258->10.242.93.149:8300: i/o timeout
2021/04/25 23:09:19 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:21 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:21 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:09:23 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:25 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:27 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:29 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:09:29 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:29 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:09:31 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:31 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:49460->10.242.93.149:8300: i/o timeout
2021/04/25 23:09:33 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:35 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:37 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:38 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:09:38 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:09:39 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:49602->10.242.93.149:8300: i/o timeout
2021/04/25 23:09:41 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:09:42 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:09:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:43 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:45 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:47 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:49 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:50 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:09:51 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:51 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:49784->10.242.93.149:8300: i/o timeout
2021/04/25 23:09:53 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:55 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:57 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:09:59 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:00 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:49904->10.242.93.149:8300: i/o timeout
2021/04/25 23:10:01 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:02 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:10:03 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:05 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:07 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:09 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:10:09 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:10 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:10:10 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' joined, marking health alive
2021/04/25 23:10:11 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:11 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:10:12 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:50088->10.242.93.149:8300: i/o timeout
2021/04/25 23:10:13 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:15 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:15 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:57058->10.242.93.140:8301: i/o timeout
2021/04/25 23:10:15 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:10:17 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:18 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:10:18 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' failed, marking health critical
2021/04/25 23:10:18 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:57096->10.242.93.140:8301: i/o timeout
2021/04/25 23:10:18 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:10:19 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:21 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:10:21 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:10:21 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:50228->10.242.93.149:8300: i/o timeout
2021/04/25 23:10:22 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:10:22 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:10:23 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:25 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:25 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:10:25 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:10:25 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2)
2021/04/25 23:10:26 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2.dc1)
2021/04/25 23:10:27 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:27 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:10:27 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' joined, marking health alive
2021/04/25 23:10:27 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:10:27 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:10:27 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/25 23:10:28 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:51752->10.242.93.149:8301: i/o timeout
2021/04/25 23:10:28 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:10:29 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:31 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:32 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:50376->10.242.93.149:8300: i/o timeout
2021/04/25 23:10:32 [WARN] memberlist: Was able to connect to dev-consul-7467bf7b46-ww995 but other probes failed, network may be misconfigured
2021/04/25 23:10:34 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:10:34 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:10:34 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:10:34 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/25 23:10:35 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:35 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:57340->10.242.93.140:8301: i/o timeout
2021/04/25 23:10:35 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:10:37 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:37 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:10:37 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:10:37 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/25 23:10:39 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:39 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:51914->10.242.93.149:8301: i/o timeout
2021/04/25 23:10:39 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:10:41 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:43 [WARN] memberlist: Was able to connect to dev-consul-7467bf7b46-ww995 but other probes failed, network may be misconfigured
2021/04/25 23:10:43 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:10:43 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:10:43 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:10:43 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/25 23:10:43 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:10:43 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:10:43 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:10:43 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/25 23:10:44 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:51984->10.242.93.149:8301: i/o timeout
2021/04/25 23:10:44 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:10:44 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:10:45 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:47 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:47 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:52028->10.242.93.149:8301: i/o timeout
2021/04/25 23:10:47 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:10:49 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:49 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:57556->10.242.93.140:8301: i/o timeout
2021/04/25 23:10:49 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:10:50 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:10:50 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:10:50 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/25 23:10:51 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:51 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500244107s
2021/04/25 23:10:52 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:10:52 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:50682->10.242.93.149:8300: i/o timeout
2021/04/25 23:10:52 [INFO] raft: pipelining replication to peer {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}
2021/04/25 23:10:53 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:53 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:57632->10.242.93.140:8301: i/o timeout
2021/04/25 23:10:53 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:10:54 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.970553266s
2021/04/25 23:10:54 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:10:54 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' failed, marking health critical
2021/04/25 23:10:55 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:10:55 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:10:56 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:10:57 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2.dc1)
2021/04/25 23:10:57 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:10:58 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:10:59 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:11:00 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:11:00 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:11:00 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:11:01 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:11:01 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:11:02 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:11:02 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:11:03 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:11:03 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:11:04 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:11:04 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:11:05 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:11:05 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:11:06 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:11:07 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:11:07 [ERR] raft: Failed to pipeline AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: write tcp 10.242.187.2:50828->10.242.93.149:8300: use of closed network connection
2021/04/25 23:11:07 [INFO] raft: aborting pipeline replication to peer {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}
2021/04/25 23:11:07 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2)
2021/04/25 23:11:07 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:11:08 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:11:09 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:11:09 [WARN] memberlist: Was able to connect to dev-consul-server-2.dc1 but other probes failed, network may be misconfigured
2021/04/25 23:11:09 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.50031098s
2021/04/25 23:11:11 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:11:11 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8301->10.242.93.140:48856: i/o timeout from=10.242.93.140:48856
2021/04/25 23:11:11 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8301->10.242.93.140:48856: i/o timeout from=10.242.93.140:48856
2021/04/25 23:11:12 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.950269866s
2021/04/25 23:11:13 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:11:14 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:11:14 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:11:14 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/25 23:11:14 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.382109052s
2021/04/25 23:11:15 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:11:15 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' joined, marking health alive
2021/04/25 23:11:15 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:11:17 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:11:17 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:52480->10.242.93.149:8301: i/o timeout
2021/04/25 23:11:17 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:11:17 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:50532->10.242.93.149:8300: i/o timeout
2021/04/25 23:11:17 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500227217s
2021/04/25 23:11:19 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:11:20 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.952634481s
2021/04/25 23:11:21 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:11:21 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:58006->10.242.93.140:8301: i/o timeout
2021/04/25 23:11:21 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:11:21 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:11:21 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:11:21 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:11:21 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:11:21 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' failed, marking health critical
2021/04/25 23:11:21 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/25 23:11:22 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:11:22 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:11:22 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/25 23:11:23 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:11:23 [INFO] raft: pipelining replication to peer {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}
2021/04/25 23:11:28 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:11:29 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:11:29 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' joined, marking health alive
2021/04/25 23:12:35 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:53758->10.242.93.149:8301: i/o timeout
2021/04/25 23:12:35 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:12:36 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:12:42 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:12:42 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:53868->10.242.93.149:8301: i/o timeout
2021/04/25 23:12:43 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:12:44 [WARN] memberlist: Was able to connect to dev-consul-7467bf7b46-ww995 but other probes failed, network may be misconfigured
2021/04/25 23:12:45 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:12:45 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:59410->10.242.93.140:8301: i/o timeout
2021/04/25 23:12:45 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:12:47 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:12:49 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:12:49 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:53972->10.242.93.149:8301: i/o timeout
2021/04/25 23:12:49 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:12:49 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500264822s
2021/04/25 23:12:51 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:12:51 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' failed, marking health critical
2021/04/25 23:12:51 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:12:52 [ERR] raft: Failed to pipeline AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: write tcp 10.242.187.2:51204->10.242.93.149:8300: use of closed network connection
2021/04/25 23:12:52 [INFO] raft: aborting pipeline replication to peer {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}
2021/04/25 23:12:52 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:54024->10.242.93.149:8301: i/o timeout
2021/04/25 23:12:52 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:12:53 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:12:53 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:12:53 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:12:53 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/25 23:12:53 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:12:54 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500193824s
2021/04/25 23:12:55 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:12:57 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.989191011s
2021/04/25 23:12:57 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:12:59 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:12:59 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:40236->10.242.93.149:8302: i/o timeout
2021/04/25 23:12:59 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:12:59 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.418941487s
2021/04/25 23:13:01 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:51050->10.242.93.149:8300: i/o timeout
2021/04/25 23:13:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:02 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:52728->10.242.93.149:8300: i/o timeout
2021/04/25 23:13:02 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:13:02 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:13:03 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:03 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:13:03 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:13:04 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:13:04 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' joined, marking health alive
2021/04/25 23:13:06 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:13:07 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:07 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:59748->10.242.93.140:8301: i/o timeout
2021/04/25 23:13:07 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:13:09 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:09 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:40320->10.242.93.149:8302: i/o timeout
2021/04/25 23:13:09 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:13:10 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:13:10 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' failed, marking health critical
2021/04/25 23:13:10 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:59786->10.242.93.140:8301: i/o timeout
2021/04/25 23:13:10 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:13:12 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:52878->10.242.93.149:8300: i/o timeout
2021/04/25 23:13:12 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:13:12 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:13:12 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:52886->10.242.93.149:8300: i/o timeout
2021/04/25 23:13:13 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:13 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:13:15 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:16 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:13:16 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:13:16 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/25 23:13:17 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:18 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500342799s
2021/04/25 23:13:19 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:19 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:40540->10.242.93.149:8302: i/o timeout
2021/04/25 23:13:19 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:13:19 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:54418->10.242.93.149:8301: i/o timeout
2021/04/25 23:13:19 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:13:21 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:13:21 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:13:21 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/25 23:13:21 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500184783s
2021/04/25 23:13:21 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:22 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:53026->10.242.93.149:8300: i/o timeout
2021/04/25 23:13:22 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:13:22 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:54460->10.242.93.149:8301: i/o timeout
2021/04/25 23:13:22 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:13:23 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:23 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.947902266s
2021/04/25 23:13:25 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:26 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.441987002s
2021/04/25 23:13:27 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:29 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:29 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/04/25 23:13:29 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:13:29 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:13:29 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:53036->10.242.93.149:8300: i/o timeout
2021/04/25 23:13:30 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:13:31 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8301->10.242.93.149:46104: i/o timeout from=10.242.93.149:46104
2021/04/25 23:13:31 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8301->10.242.93.149:46104: i/o timeout from=10.242.93.149:46104
2021/04/25 23:13:31 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:32 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:53178->10.242.93.149:8300: i/o timeout
2021/04/25 23:13:32 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:13:33 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:34 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:40770->10.242.93.149:8302: i/o timeout
2021/04/25 23:13:34 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:13:35 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:37 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:39 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:40 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:53290->10.242.93.149:8300: i/o timeout
2021/04/25 23:13:40 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:13:41 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:42 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:53328->10.242.93.149:8300: i/o timeout
2021/04/25 23:13:42 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:13:43 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:13:43 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.149:58328
2021/04/25 23:13:43 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:45 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:45 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:13:45 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:13:46 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8302->10.242.93.149:52616: i/o timeout from=10.242.93.149:52616
2021/04/25 23:13:46 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8302->10.242.93.149:52616: i/o timeout from=10.242.93.149:52616
2021/04/25 23:13:48 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:13:49 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:13:49 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:50 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:53448->10.242.93.149:8300: i/o timeout
2021/04/25 23:13:51 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:51 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:13:52 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:53476->10.242.93.149:8300: i/o timeout
2021/04/25 23:13:52 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:13:53 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:55 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:57 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:13:59 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:01 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:01 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:53616->10.242.93.149:8300: i/o timeout
2021/04/25 23:14:02 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:14:02 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:53624->10.242.93.149:8300: i/o timeout
2021/04/25 23:14:02 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:14:03 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:05 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:07 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:09 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:14:09 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:11 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:12 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:53770->10.242.93.149:8300: i/o timeout
2021/04/25 23:14:12 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:53774->10.242.93.149:8300: i/o timeout
2021/04/25 23:14:13 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:14:13 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:13 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:14:15 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:17 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:19 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:21 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:23 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:53922->10.242.93.149:8300: i/o timeout
2021/04/25 23:14:23 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:23 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:53924->10.242.93.149:8300: i/o timeout
2021/04/25 23:14:24 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:14:24 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:14:25 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:25 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8301->10.242.93.140:51682: i/o timeout from=10.242.93.140:51682
2021/04/25 23:14:25 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8301->10.242.93.140:51682: i/o timeout from=10.242.93.140:51682
2021/04/25 23:14:27 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:27 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:14:27 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:14:29 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:14:30 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:14:31 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:33 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:34 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:54070->10.242.93.149:8300: i/o timeout
2021/04/25 23:14:34 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:54088->10.242.93.149:8300: i/o timeout
2021/04/25 23:14:35 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:14:35 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:37 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:14:37 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:38 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:14:38 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:14:38 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/25 23:14:39 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:40 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:55610->10.242.93.149:8301: i/o timeout
2021/04/25 23:14:40 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:14:41 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:43 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:43 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:14:43 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:14:43 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:14:43 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/25 23:14:44 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:55664->10.242.93.149:8301: i/o timeout
2021/04/25 23:14:44 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:14:45 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:54248->10.242.93.149:8300: i/o timeout
2021/04/25 23:14:45 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:46 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:14:46 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8302->10.242.93.149:52942: i/o timeout from=10.242.93.149:52942
2021/04/25 23:14:46 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8302->10.242.93.149:52942: i/o timeout from=10.242.93.149:52942
2021/04/25 23:14:47 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:54282->10.242.93.149:8300: i/o timeout
2021/04/25 23:14:47 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:49 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:51 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:52 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:14:53 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:55 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:56 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:54418->10.242.93.149:8300: i/o timeout
2021/04/25 23:14:57 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:14:58 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:14:59 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:15:01 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:15:02 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:54502->10.242.93.149:8300: i/o timeout
2021/04/25 23:15:03 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:15:04 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2)
2021/04/25 23:15:04 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:15:04 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:15:05 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:15:07 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:15:08 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2.dc1)
2021/04/25 23:15:08 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:54588->10.242.93.149:8300: i/o timeout
2021/04/25 23:15:09 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:15:09 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:15:09 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:15:09 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:15:10 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:15:10 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:15:10 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/25 23:15:12 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:15:13 [INFO] raft: pipelining replication to peer {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}
2021/04/25 23:15:13 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:15:13 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' joined, marking health alive
2021/04/25 23:15:19 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:56190->10.242.93.149:8301: i/o timeout
2021/04/25 23:15:19 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:15:26 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:15:27 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:15:32 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:33658->10.242.93.140:8301: i/o timeout
2021/04/25 23:15:32 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:15:37 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:33726->10.242.93.140:8301: i/o timeout
2021/04/25 23:15:38 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:15:38 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/25 23:15:39 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:15:40 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:33772->10.242.93.140:8301: i/o timeout
2021/04/25 23:15:40 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:15:44 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:16:00 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:16:00 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/25 23:16:00 [WARN] consul: error getting server health from "dev-consul-server-1": context deadline exceeded
2021/04/25 23:16:04 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:16:04 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:34114->10.242.93.140:8301: i/o timeout
2021/04/25 23:16:04 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:16:06 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:16:07 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:16:07 [INFO] memberlist: Marking dev-consul-7467bf7b46-ww995 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:16:07 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:16:07 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' failed, marking health critical
2021/04/25 23:16:07 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500257449s
2021/04/25 23:16:08 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:56898->10.242.93.149:8301: i/o timeout
2021/04/25 23:16:08 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:16:09 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:16:12 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:16:12 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:56974->10.242.93.149:8301: i/o timeout
2021/04/25 23:16:12 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:16:13 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:16:15 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:16:15 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:16:15 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/25 23:16:16 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500221239s
2021/04/25 23:16:16 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:16:16 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:57026->10.242.93.149:8301: i/o timeout
2021/04/25 23:16:16 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:16:17 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:16:18 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.979959753s
2021/04/25 23:16:19 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:16:19 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:43190->10.242.93.149:8302: i/o timeout
2021/04/25 23:16:19 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:16:20 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.43012778s
2021/04/25 23:16:21 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:16:23 [INFO] raft: aborting pipeline replication to peer {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}
2021/04/25 23:16:23 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:54802->10.242.93.149:8300: i/o timeout
2021/04/25 23:16:23 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:16:23 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:16:24 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:16:25 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:16:26 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.504049495s
2021/04/25 23:16:27 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:16:28 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.980858615s
2021/04/25 23:16:29 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:16:29 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:43334->10.242.93.149:8302: i/o timeout
2021/04/25 23:16:29 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:16:30 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:16:30 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.149:33552
2021/04/25 23:16:30 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.45467239s
2021/04/25 23:16:31 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:16:33 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:16:33 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:55828->10.242.93.149:8300: i/o timeout
2021/04/25 23:16:33 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:16:34 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:55838->10.242.93.149:8300: i/o timeout
2021/04/25 23:16:34 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:16:35 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:16:37 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:16:39 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:16:39 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:43412->10.242.93.149:8302: i/o timeout
2021/04/25 23:16:39 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:16:41 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:16:43 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:16:43 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:55980->10.242.93.149:8300: i/o timeout
2021/04/25 23:16:43 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:16:43 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:16:44 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:16:44 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.140:50430
2021/04/25 23:16:44 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:56004->10.242.93.149:8300: i/o timeout
2021/04/25 23:16:45 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:16:45 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:16:47 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:16:49 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:16:49 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/04/25 23:16:49 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:16:49 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:16:51 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:16:53 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:16:53 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:56124->10.242.93.149:8300: i/o timeout
2021/04/25 23:16:53 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:16:53 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:16:53 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:16:54 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:43702->10.242.93.149:8302: i/o timeout
2021/04/25 23:16:54 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:16:55 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:56168->10.242.93.149:8300: i/o timeout
2021/04/25 23:16:56 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:16:56 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/25 23:16:56 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:16:57 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:16:59 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:01 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:03 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:03 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:56270->10.242.93.149:8300: i/o timeout
2021/04/25 23:17:03 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:17:05 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:06 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:56324->10.242.93.149:8300: i/o timeout
2021/04/25 23:17:07 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:17:07 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:09 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:17:09 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:11 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:13 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:13 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:56420->10.242.93.149:8300: i/o timeout
2021/04/25 23:17:13 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:17:15 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:17 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:56484->10.242.93.149:8300: i/o timeout
2021/04/25 23:17:17 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:17 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:17:19 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:21 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:23 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:23 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:56562->10.242.93.149:8300: i/o timeout
2021/04/25 23:17:23 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:17:23 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:17:25 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:27 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:27 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:56630->10.242.93.149:8300: i/o timeout
2021/04/25 23:17:28 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:17:29 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:31 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:32 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:17:32 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:17:32 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/25 23:17:32 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:17:32 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:17:33 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:33 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:56706->10.242.93.149:8300: i/o timeout
2021/04/25 23:17:35 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:35 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:58176->10.242.93.149:8301: i/o timeout
2021/04/25 23:17:35 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:17:37 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:37 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:58196->10.242.93.149:8301: i/o timeout
2021/04/25 23:17:37 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:17:37 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:17:37 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:17:37 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:17:37 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/25 23:17:38 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:56788->10.242.93.149:8300: i/o timeout
2021/04/25 23:17:39 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:39 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:44360->10.242.93.149:8302: i/o timeout
2021/04/25 23:17:39 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:17:40 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:17:41 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:43 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:44 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:56860->10.242.93.149:8300: i/o timeout
2021/04/25 23:17:44 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:17:45 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:47 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:49 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:49 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:44436->10.242.93.149:8302: i/o timeout
2021/04/25 23:17:49 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:17:50 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:56956->10.242.93.149:8300: i/o timeout
2021/04/25 23:17:51 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:52 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:17:53 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:53 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8302->10.242.93.149:54862: i/o timeout from=10.242.93.149:54862
2021/04/25 23:17:53 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8302->10.242.93.149:54862: i/o timeout from=10.242.93.149:54862
2021/04/25 23:17:54 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:57028->10.242.93.149:8300: i/o timeout
2021/04/25 23:17:55 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:56 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:17:57 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:17:59 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:01 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:02 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:57142->10.242.93.149:8300: i/o timeout
2021/04/25 23:18:03 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:04 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:44730->10.242.93.149:8302: i/o timeout
2021/04/25 23:18:04 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:18:05 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:05 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:18:06 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:57200->10.242.93.149:8300: i/o timeout
2021/04/25 23:18:07 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:08 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:18:09 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/04/25 23:18:09 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:18:09 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:18:09 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:18:09 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:11 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:11 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:18:11 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.149:35344
2021/04/25 23:18:12 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:18:12 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:18:14 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:18:14 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/25 23:18:14 [WARN] consul: error getting server health from "dev-consul-server-1": context deadline exceeded
2021/04/25 23:18:14 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:44800->10.242.93.149:8302: i/o timeout
2021/04/25 23:18:14 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:18:15 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:15 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:57332->10.242.93.149:8300: i/o timeout
2021/04/25 23:18:17 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:18 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:57376->10.242.93.149:8300: i/o timeout
2021/04/25 23:18:19 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:21 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:21 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:18:23 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:23 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:18:25 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:27 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:29 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:31 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:31 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:57572->10.242.93.149:8300: i/o timeout
2021/04/25 23:18:32 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:18:33 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:33 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:57586->10.242.93.149:8300: i/o timeout
2021/04/25 23:18:35 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:37 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:39 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:41 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:42 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:18:43 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:44 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:18:45 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:47 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:49 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:51 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:52 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:57866->10.242.93.149:8300: i/o timeout
2021/04/25 23:18:53 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:53 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:18:53 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:18:54 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:57876->10.242.93.149:8300: i/o timeout
2021/04/25 23:18:56 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:18:57 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:18:59 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:01 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:03 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:03 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:19:04 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:19:05 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:07 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:09 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:19:09 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:11 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:12 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:19:13 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:13 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:58160->10.242.93.149:8300: i/o timeout
2021/04/25 23:19:14 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:58166->10.242.93.149:8300: i/o timeout
2021/04/25 23:19:15 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:17 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:19 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:21 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:23 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:24 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:19:24 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:19:25 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:27 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:29 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:31 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:33 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:34 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:58460->10.242.93.149:8300: i/o timeout
2021/04/25 23:19:34 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:58464->10.242.93.149:8300: i/o timeout
2021/04/25 23:19:35 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:35 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:19:35 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:19:38 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:19:39 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:41 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:43 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:44 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:19:45 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:45 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:19:45 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:19:45 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:19:47 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:49 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:51 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:53 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:54 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:46316->10.242.93.149:8302: i/o timeout
2021/04/25 23:19:54 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:19:54 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:58760->10.242.93.149:8300: i/o timeout
2021/04/25 23:19:55 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:55 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:58780->10.242.93.149:8300: i/o timeout
2021/04/25 23:19:57 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:19:59 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:01 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:03 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:04 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:46464->10.242.93.149:8302: i/o timeout
2021/04/25 23:20:04 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:20:05 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:20:05 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:06 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:20:07 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:09 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:11 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:13 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:14 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:46608->10.242.93.149:8302: i/o timeout
2021/04/25 23:20:14 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:20:15 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:59060->10.242.93.149:8300: i/o timeout
2021/04/25 23:20:15 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:16 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:59088->10.242.93.149:8300: i/o timeout
2021/04/25 23:20:17 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:20:17 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:20:18 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:20:19 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:21 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:22 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:20:23 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:24 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/04/25 23:20:24 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:20:24 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:20:25 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:20:25 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:25 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:20:25 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.140:52584
2021/04/25 23:20:27 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:27 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:20:29 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:29 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:46828->10.242.93.149:8302: i/o timeout
2021/04/25 23:20:29 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:20:31 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:33 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:35 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:59354->10.242.93.149:8300: i/o timeout
2021/04/25 23:20:35 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:37 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:37 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:59392->10.242.93.149:8300: i/o timeout
2021/04/25 23:20:38 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:20:38 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.149:35554
2021/04/25 23:20:39 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:20:39 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:41 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:43 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:45 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:45 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:20:47 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:49 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:20:49 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:51 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:53 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:55 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:55 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:59652->10.242.93.149:8300: i/o timeout
2021/04/25 23:20:57 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:20:57 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:20:57 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:20:59 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:59704->10.242.93.149:8300: i/o timeout
2021/04/25 23:21:00 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:21:01 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:02 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:21:03 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:05 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:05 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:21:07 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:09 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:10 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:21:11 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:13 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:15 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:15 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:59948->10.242.93.149:8300: i/o timeout
2021/04/25 23:21:17 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:19 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:20 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:60006->10.242.93.149:8300: i/o timeout
2021/04/25 23:21:21 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:23 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:25 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:26 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:21:27 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:29 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:31 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:21:31 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:33 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:35 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:36 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:60238->10.242.93.149:8300: i/o timeout
2021/04/25 23:21:37 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:39 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:39 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:21:39 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:21:41 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:60324->10.242.93.149:8300: i/o timeout
2021/04/25 23:21:42 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:21:42 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/25 23:21:43 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:45 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:46 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:21:47 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:49 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:51 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:52 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:21:53 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:55 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:56 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:60538->10.242.93.149:8300: i/o timeout
2021/04/25 23:21:57 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:21:59 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:01 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:02 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:60622->10.242.93.149:8300: i/o timeout
2021/04/25 23:22:03 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:05 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:06 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:22:07 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:09 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:22:09 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:11 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:12 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:22:12 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:22:13 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:15 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:16 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:60842->10.242.93.149:8300: i/o timeout
2021/04/25 23:22:17 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:19 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:21 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:21 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:22:21 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:22:22 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:60930->10.242.93.149:8300: i/o timeout
2021/04/25 23:22:24 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:22:25 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:26 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:22:27 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:29 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:31 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:33 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:33 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:22:35 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:36 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:32900->10.242.93.149:8300: i/o timeout
2021/04/25 23:22:37 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:39 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:41 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:43 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:43 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:32994->10.242.93.149:8300: i/o timeout
2021/04/25 23:22:45 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:45 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2)
2021/04/25 23:22:47 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:22:47 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:49 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:51 [DEBUG] raft-net: 10.242.187.2:8300 accepted connection from: 10.242.93.149:36674
2021/04/25 23:22:51 [WARN] raft: Rejecting vote request from 10.242.93.149:8300 since we have a leader: 10.242.187.2:8300
2021/04/25 23:22:51 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:22:51 [ERR] raft: peer {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300} has newer term, stopping replication
2021/04/25 23:22:51 [INFO] raft: Node at 10.242.187.2:8300 [Follower] entering Follower state (Leader: "")
2021/04/25 23:22:51 [INFO] raft: aborting pipeline replication to peer {Voter 9fcb14a9-100e-2d75-3e7a-6fe772ff1f8f 10.242.32.71:8300}
2021/04/25 23:22:51 [INFO] consul: cluster leadership lost
2021/04/25 23:22:51 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:22:51 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:22:51 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:22:51 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:22:52 [WARN] consul.coordinate: Batch update failed: node is not the leader
2021/04/25 23:22:52 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:22:55 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:22:57 [DEBUG] raft-net: 10.242.187.2:8300 accepted connection from: 10.242.32.71:36266
2021/04/25 23:22:57 [INFO] consul: New leader elected: dev-consul-server-1
2021/04/25 23:22:57 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:40138->10.242.93.140:8301: i/o timeout
2021/04/25 23:22:58 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:22:58 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:22:58 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:22:58 [DEBUG] raft-net: 10.242.187.2:8300 accepted connection from: 10.242.32.71:36316
2021/04/25 23:22:58 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:34646->10.242.93.149:8301: i/o timeout
2021/04/25 23:22:58 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:23:00 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:23:02 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:40172->10.242.93.140:8301: i/o timeout
2021/04/25 23:23:02 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:23:09 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:49168->10.242.93.149:8302: i/o timeout
2021/04/25 23:23:09 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:23:19 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:49310->10.242.93.149:8302: i/o timeout
2021/04/25 23:23:19 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:23:25 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:23:25 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.149:36664
2021/04/25 23:23:25 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:23:29 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:49454->10.242.93.149:8302: i/o timeout
2021/04/25 23:23:29 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:23:31 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:23:31 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.140:53536
2021/04/25 23:23:39 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:49598->10.242.93.149:8302: i/o timeout
2021/04/25 23:23:39 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:23:49 [WARN] memberlist: Was able to connect to dev-consul-server-2.dc1 but other probes failed, network may be misconfigured
2021/04/25 23:23:51 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:23:51 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:23:55 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:35472->10.242.93.149:8301: i/o timeout
2021/04/25 23:23:55 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:23:58 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:23:58 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:23:58 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:23:59 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/04/25 23:23:59 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:23:59 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:23:59 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:49888->10.242.93.149:8302: i/o timeout
2021/04/25 23:23:59 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:23:59 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:24:02 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:41096->10.242.93.140:8301: i/o timeout
2021/04/25 23:24:02 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:24:04 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:24:04 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:24:06 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:24:06 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:41148->10.242.93.140:8301: i/o timeout
2021/04/25 23:24:06 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:24:09 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:24:10 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:35664->10.242.93.149:8301: i/o timeout
2021/04/25 23:24:10 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:24:13 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:24:13 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:24:13 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:24:14 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:35740->10.242.93.149:8301: i/o timeout
2021/04/25 23:24:14 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:24:34 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8302->10.242.93.149:56976: i/o timeout from=10.242.93.149:56976
2021/04/25 23:24:34 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8302->10.242.93.149:56976: i/o timeout from=10.242.93.149:56976
2021/04/25 23:24:35 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:25:15 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:25:39 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:25:55 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:26:02 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:26:02 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:26:10 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2.dc1)
2021/04/25 23:26:10 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:26:10 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:26:11 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:26:11 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:37438->10.242.93.149:8301: i/o timeout
2021/04/25 23:26:11 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:26:15 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:37486->10.242.93.149:8301: i/o timeout
2021/04/25 23:26:15 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:26:18 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:26:18 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:26:18 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:43010->10.242.93.140:8301: i/o timeout
2021/04/25 23:26:18 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:26:19 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:26:34 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:52126->10.242.93.149:8302: i/o timeout
2021/04/25 23:26:34 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:26:44 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:52268->10.242.93.149:8302: i/o timeout
2021/04/25 23:26:44 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:26:46 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-0)
2021/04/25 23:26:46 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:26:46 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:26:50 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:26:50 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.140:55074
2021/04/25 23:26:51 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:38020->10.242.93.149:8301: i/o timeout
2021/04/25 23:26:51 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:26:52 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:26:52 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:26:52 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:26:54 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:38036->10.242.93.149:8301: i/o timeout
2021/04/25 23:26:54 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:26:56 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:26:59 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:52482->10.242.93.149:8302: i/o timeout
2021/04/25 23:26:59 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:26:59 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:43636->10.242.93.140:8301: i/o timeout
2021/04/25 23:26:59 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:27:02 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:43672->10.242.93.140:8301: i/o timeout
2021/04/25 23:27:02 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:27:04 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/04/25 23:27:04 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:27:04 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:27:05 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:27:05 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:27:05 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:43724->10.242.93.140:8301: i/o timeout
2021/04/25 23:27:05 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:27:08 [ERR] memberlist: Push/Pull with dev-consul-7467bf7b46-ww995 failed: read tcp 10.242.187.2:43620->10.242.93.140:8301: i/o timeout
2021/04/25 23:27:09 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:43782->10.242.93.140:8301: i/o timeout
2021/04/25 23:27:09 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:27:12 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:43820->10.242.93.140:8301: i/o timeout
2021/04/25 23:27:12 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:27:14 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:52710->10.242.93.149:8302: i/o timeout
2021/04/25 23:27:14 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:27:16 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:43890->10.242.93.140:8301: i/o timeout
2021/04/25 23:27:16 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:27:19 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:43934->10.242.93.140:8301: i/o timeout
2021/04/25 23:27:19 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:27:23 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:43994->10.242.93.140:8301: i/o timeout
2021/04/25 23:27:23 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:27:23 [INFO] memberlist: Marking dev-consul-7467bf7b46-ww995 as failed, suspect timeout reached (0 peer confirmations)
2021/04/25 23:27:23 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:27:24 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:52858->10.242.93.149:8302: i/o timeout
2021/04/25 23:27:24 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:27:30 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:27:30 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:27:32 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:38630->10.242.93.149:8301: i/o timeout
2021/04/25 23:27:32 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:27:35 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:27:35 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:27:35 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:27:35 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:27:36 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:38682->10.242.93.149:8301: i/o timeout
2021/04/25 23:27:36 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:27:39 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:53080->10.242.93.149:8302: i/o timeout
2021/04/25 23:27:39 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:27:44 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/04/25 23:27:44 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:27:44 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:27:49 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:27:49 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:53156->10.242.93.149:8302: i/o timeout
2021/04/25 23:27:49 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:27:50 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:44390->10.242.93.140:8301: i/o timeout
2021/04/25 23:27:50 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:27:51 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:27:51 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:27:55 [WARN] memberlist: Was able to connect to dev-consul-7467bf7b46-ww995 but other probes failed, network may be misconfigured
2021/04/25 23:27:58 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:27:58 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:44504->10.242.93.140:8301: i/o timeout
2021/04/25 23:27:58 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:27:58 [WARN] memberlist: Refuting a suspect message (from: dev-consul-7467bf7b46-ww995)
2021/04/25 23:27:59 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:27:59 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:28:00 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:29:02 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:39936->10.242.93.149:8301: i/o timeout
2021/04/25 23:29:02 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:29:06 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:45472->10.242.93.140:8301: i/o timeout
2021/04/25 23:29:06 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:29:09 [WARN] memberlist: Was able to connect to dev-consul-server-2 but other probes failed, network may be misconfigured
2021/04/25 23:29:10 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:45548->10.242.93.140:8301: i/o timeout
2021/04/25 23:29:10 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:29:10 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:29:11 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:29:11 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:29:15 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:29:18 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:29:18 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.140:55584
2021/04/25 23:29:24 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:54606->10.242.93.149:8302: i/o timeout
2021/04/25 23:29:24 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:29:34 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:54750->10.242.93.149:8302: i/o timeout
2021/04/25 23:29:34 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:29:35 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2.dc1)
2021/04/25 23:29:36 [WARN] raft: Rejecting vote request from 10.242.93.149:8300 since we have a leader: 10.242.32.71:8300
2021/04/25 23:29:37 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:29:37 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:29:40 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:29:42 [WARN] raft: Heartbeat timeout from "10.242.32.71:8300" reached, starting election
2021/04/25 23:29:42 [INFO] raft: Node at 10.242.187.2:8300 [Candidate] entering Candidate state in term 4
2021/04/25 23:29:42 [INFO] raft: Election won. Tally: 2
2021/04/25 23:29:42 [INFO] raft: Node at 10.242.187.2:8300 [Leader] entering Leader state
2021/04/25 23:29:42 [INFO] raft: Added peer f66dacd5-df84-469e-7a9b-4103153d1fdc, starting replication
2021/04/25 23:29:42 [INFO] raft: Added peer 9fcb14a9-100e-2d75-3e7a-6fe772ff1f8f, starting replication
2021/04/25 23:29:42 [INFO] consul: cluster leadership acquired
2021/04/25 23:29:42 [INFO] consul: New leader elected: dev-consul-server-0
2021/04/25 23:29:42 [INFO] raft: pipelining replication to peer {Voter 9fcb14a9-100e-2d75-3e7a-6fe772ff1f8f 10.242.32.71:8300}
2021/04/25 23:29:42 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' joined, marking health alive
2021/04/25 23:29:42 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/25 23:29:43 [WARN] raft: AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300} rejected, sending older logs (next: 1047)
2021/04/25 23:29:43 [INFO] raft: pipelining replication to peer {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}
2021/04/25 23:29:47 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:29:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:29:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:29:50 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500262493s
2021/04/25 23:29:51 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:40654->10.242.93.149:8301: i/o timeout
2021/04/25 23:29:51 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:29:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:29:55 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.50028718s
2021/04/25 23:29:55 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:46186->10.242.93.140:8301: i/o timeout
2021/04/25 23:29:55 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:29:55 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:29:55 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:29:55 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:29:55 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/25 23:29:55 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:29:55 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:29:55 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/25 23:29:55 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2)
2021/04/25 23:29:59 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:30:00 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:46278->10.242.93.140:8301: i/o timeout
2021/04/25 23:30:00 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:30:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:30:01 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:30:01 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' failed, marking health critical
2021/04/25 23:30:04 [WARN] memberlist: Was able to connect to dev-consul-server-2 but other probes failed, network may be misconfigured
2021/04/25 23:30:07 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:30:07 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' joined, marking health alive
2021/04/25 23:30:07 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:40884->10.242.93.149:8301: i/o timeout
2021/04/25 23:30:15 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:30:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:30:17 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500307454s
2021/04/25 23:30:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:30:19 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:41054->10.242.93.149:8301: i/o timeout
2021/04/25 23:30:19 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:30:20 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.971502562s
2021/04/25 23:30:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:30:22 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:46566->10.242.93.140:8301: i/o timeout
2021/04/25 23:30:22 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:30:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:30:22 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:30:22 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' failed, marking health critical
2021/04/25 23:30:22 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.456949593s
2021/04/25 23:30:23 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:30:23 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:30:23 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:30:23 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/25 23:30:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:30:25 [INFO] raft: aborting pipeline replication to peer {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}
2021/04/25 23:30:25 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:30:25 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:41110->10.242.93.149:8301: i/o timeout
2021/04/25 23:30:25 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:30:25 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:30:25 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:33196->10.242.93.149:8300: i/o timeout
2021/04/25 23:30:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:30:26 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:30:27 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500282199s
2021/04/25 23:30:29 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:55554->10.242.93.149:8302: i/o timeout
2021/04/25 23:30:29 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:30:29 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:30:30 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:30:30 [INFO] raft: pipelining replication to peer {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}
2021/04/25 23:30:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:30:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:30:32 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500336161s
2021/04/25 23:30:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:30:35 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.990187447s
2021/04/25 23:30:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:30:37 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:30:38 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:30:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:30:39 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:55698->10.242.93.149:8302: i/o timeout
2021/04/25 23:30:39 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:30:39 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:30:40 [INFO] raft: aborting pipeline replication to peer {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}
2021/04/25 23:30:40 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:30:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:30:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:30:42 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500206255s
2021/04/25 23:30:43 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:30:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:30:45 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500223356s
2021/04/25 23:30:45 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8301->10.242.93.140:59936: i/o timeout from=10.242.93.140:59936
2021/04/25 23:30:45 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8301->10.242.93.140:59936: i/o timeout from=10.242.93.140:59936
2021/04/25 23:30:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:30:47 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.918654204s
2021/04/25 23:30:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:30:49 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:55846->10.242.93.149:8302: i/o timeout
2021/04/25 23:30:49 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:30:50 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.406880236s
2021/04/25 23:30:50 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:40062->10.242.93.149:8300: i/o timeout
2021/04/25 23:30:50 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:30:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:30:50 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:30:50 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.140:57866
2021/04/25 23:30:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:30:53 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:39870->10.242.93.149:8300: i/o timeout
2021/04/25 23:30:54 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:30:54 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:30:54 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.149:41068
2021/04/25 23:30:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:30:56 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:30:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:30:58 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:30:58 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:30:58 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:30:58 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/25 23:30:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:30:59 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:55986->10.242.93.149:8302: i/o timeout
2021/04/25 23:30:59 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:31:00 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:40210->10.242.93.149:8300: i/o timeout
2021/04/25 23:31:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:00 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500188913s
2021/04/25 23:31:02 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:41690->10.242.93.149:8301: i/o timeout
2021/04/25 23:31:02 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:31:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:03 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.982219041s
2021/04/25 23:31:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:04 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:31:04 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:31:04 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/25 23:31:05 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:41700->10.242.93.149:8301: i/o timeout
2021/04/25 23:31:05 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:31:05 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:31:05 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.428403011s
2021/04/25 23:31:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:09 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:40256->10.242.93.149:8300: i/o timeout
2021/04/25 23:31:09 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:56066->10.242.93.149:8302: i/o timeout
2021/04/25 23:31:09 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:31:09 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:31:10 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:40354->10.242.93.149:8300: i/o timeout
2021/04/25 23:31:10 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:31:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:13 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:31:15 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500252223s
2021/04/25 23:31:15 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:31:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:17 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.982436447s
2021/04/25 23:31:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:20 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.451952479s
2021/04/25 23:31:20 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:40504->10.242.93.149:8300: i/o timeout
2021/04/25 23:31:20 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:31:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:23 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:40498->10.242.93.149:8300: i/o timeout
2021/04/25 23:31:24 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:31:24 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:56356->10.242.93.149:8302: i/o timeout
2021/04/25 23:31:24 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:31:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:30 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:40650->10.242.93.149:8300: i/o timeout
2021/04/25 23:31:30 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:31:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:34 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/04/25 23:31:34 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:31:34 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:31:34 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:40698->10.242.93.149:8300: i/o timeout
2021/04/25 23:31:34 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:56430->10.242.93.149:8302: i/o timeout
2021/04/25 23:31:34 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:31:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:35 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:31:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:40 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:40796->10.242.93.149:8300: i/o timeout
2021/04/25 23:31:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:40 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:31:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:45 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:40872->10.242.93.149:8300: i/o timeout
2021/04/25 23:31:45 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:31:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:50 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:40946->10.242.93.149:8300: i/o timeout
2021/04/25 23:31:51 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:31:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:52 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:31:52 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:31:55 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:31:55 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:41026->10.242.93.149:8300: i/o timeout
2021/04/25 23:31:56 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:31:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:31:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:01 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:41102->10.242.93.149:8300: i/o timeout
2021/04/25 23:32:01 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:32:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:06 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:41182->10.242.93.149:8300: i/o timeout
2021/04/25 23:32:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:07 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:32:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:11 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:41264->10.242.93.149:8300: i/o timeout
2021/04/25 23:32:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:12 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:32:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:15 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:32:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:17 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:32:17 [WARN] memberlist: Refuting a suspect message (from: dev-consul-7467bf7b46-ww995)
2021/04/25 23:32:17 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' joined, marking health alive
2021/04/25 23:32:17 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:41340->10.242.93.149:8300: i/o timeout
2021/04/25 23:32:18 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:32:18 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:32:18 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:32:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:19 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:48298->10.242.93.140:8301: i/o timeout
2021/04/25 23:32:19 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:32:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:22 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:41422->10.242.93.149:8300: i/o timeout
2021/04/25 23:32:23 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:32:23 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' failed, marking health critical
2021/04/25 23:32:23 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:48372->10.242.93.140:8301: i/o timeout
2021/04/25 23:32:23 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:32:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:25 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:32:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:28 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:41512->10.242.93.149:8300: i/o timeout
2021/04/25 23:32:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:29 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:32:29 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:57314->10.242.93.149:8302: i/o timeout
2021/04/25 23:32:29 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:32:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:31 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:32:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:33 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500202803s
2021/04/25 23:32:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:35 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:41612->10.242.93.149:8300: i/o timeout
2021/04/25 23:32:36 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.946964596s
2021/04/25 23:32:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:38 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.38750624s
2021/04/25 23:32:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:39 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:57456->10.242.93.149:8302: i/o timeout
2021/04/25 23:32:39 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:32:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:40 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:32:41 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:41666->10.242.93.149:8300: i/o timeout
2021/04/25 23:32:42 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:32:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:46 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:32:46 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.140:59180
2021/04/25 23:32:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:49 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:57534->10.242.93.149:8302: i/o timeout
2021/04/25 23:32:49 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:32:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:50 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:41824->10.242.93.149:8300: i/o timeout
2021/04/25 23:32:52 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:41862->10.242.93.149:8300: i/o timeout
2021/04/25 23:32:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:52 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:32:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:57 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8302->10.242.93.149:33776: i/o timeout from=10.242.93.149:33776
2021/04/25 23:32:57 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8302->10.242.93.149:33776: i/o timeout from=10.242.93.149:33776
2021/04/25 23:32:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:32:59 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/04/25 23:32:59 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:32:59 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:32:59 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:32:59 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' joined, marking health alive
2021/04/25 23:33:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:00 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:33:02 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:42008->10.242.93.149:8300: i/o timeout
2021/04/25 23:33:03 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:48954->10.242.93.140:8301: i/o timeout
2021/04/25 23:33:03 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:33:03 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:33:03 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:33:04 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:33:04 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2.dc1)
2021/04/25 23:33:04 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:33:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:05 [INFO] memberlist: Marking dev-consul-7467bf7b46-ww995 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:33:05 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:33:05 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' failed, marking health critical
2021/04/25 23:33:06 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:49002->10.242.93.140:8301: i/o timeout
2021/04/25 23:33:06 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:33:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:10 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:42120->10.242.93.149:8300: i/o timeout
2021/04/25 23:33:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:13 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:42162->10.242.93.149:8300: i/o timeout
2021/04/25 23:33:14 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:33:14 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:57894->10.242.93.149:8302: i/o timeout
2021/04/25 23:33:14 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:33:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:21 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:33:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:24 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:42312->10.242.93.149:8300: i/o timeout
2021/04/25 23:33:24 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:58038->10.242.93.149:8302: i/o timeout
2021/04/25 23:33:24 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:33:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:25 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:33:25 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:33:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:31 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:42430->10.242.93.149:8300: i/o timeout
2021/04/25 23:33:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:35 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:42482->10.242.93.149:8300: i/o timeout
2021/04/25 23:33:36 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:33:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:39 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:33:39 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:33:39 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:58328->10.242.93.149:8302: i/o timeout
2021/04/25 23:33:39 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:33:40 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:33:40 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.149:42490
2021/04/25 23:33:41 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:33:41 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:33:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:44 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/04/25 23:33:44 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:33:44 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:33:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:46 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:42646->10.242.93.149:8300: i/o timeout
2021/04/25 23:33:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:47 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:33:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:49 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:58480->10.242.93.149:8302: i/o timeout
2021/04/25 23:33:49 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:33:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:51 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:42724->10.242.93.149:8300: i/o timeout
2021/04/25 23:33:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:57 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:42812->10.242.93.149:8300: i/o timeout
2021/04/25 23:33:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:33:59 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:34:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:01 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:34:02 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:34:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:09 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:42968->10.242.93.149:8300: i/o timeout
2021/04/25 23:34:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:10 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:34:11 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:43018->10.242.93.149:8300: i/o timeout
2021/04/25 23:34:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:20 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:34:20 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:34:20 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:43140->10.242.93.149:8300: i/o timeout
2021/04/25 23:34:21 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:34:23 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:34:24 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:34:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:31 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:43302->10.242.93.149:8300: i/o timeout
2021/04/25 23:34:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:34 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:43324->10.242.93.149:8300: i/o timeout
2021/04/25 23:34:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:35 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:34:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:40 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:34:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:42 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:34:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:50 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:43558->10.242.93.149:8300: i/o timeout
2021/04/25 23:34:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:52 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:43596->10.242.93.149:8300: i/o timeout
2021/04/25 23:34:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:34:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:01 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:35:02 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:35:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:02 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:35:02 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:35:05 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:35:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:11 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:43870->10.242.93.149:8300: i/o timeout
2021/04/25 23:35:12 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:43882->10.242.93.149:8300: i/o timeout
2021/04/25 23:35:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:15 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:35:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:22 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:35:22 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:35:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:32 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:44164->10.242.93.149:8300: i/o timeout
2021/04/25 23:35:32 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:35:32 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:44172->10.242.93.149:8300: i/o timeout
2021/04/25 23:35:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:42 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:35:43 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:35:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:44 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:35:44 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:35:47 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:35:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:52 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:44460->10.242.93.149:8300: i/o timeout
2021/04/25 23:35:53 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:44464->10.242.93.149:8300: i/o timeout
2021/04/25 23:35:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:35:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:03 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:36:04 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:36:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:13 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:44754->10.242.93.149:8300: i/o timeout
2021/04/25 23:36:14 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:44760->10.242.93.149:8300: i/o timeout
2021/04/25 23:36:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:23 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:36:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:25 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:36:25 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:36:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:26 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:36:26 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:36:29 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:36:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:32 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:36:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:33 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:45038->10.242.93.149:8300: i/o timeout
2021/04/25 23:36:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:35 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:45068->10.242.93.149:8300: i/o timeout
2021/04/25 23:36:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:43 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:36:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:45 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:36:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:53 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:45336->10.242.93.149:8300: i/o timeout
2021/04/25 23:36:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:55 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:45380->10.242.93.149:8300: i/o timeout
2021/04/25 23:36:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:36:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:03 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:37:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:05 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:37:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:06 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:37:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:08 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:37:08 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:37:11 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:37:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:13 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:45626->10.242.93.149:8300: i/o timeout
2021/04/25 23:37:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:16 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:45686->10.242.93.149:8300: i/o timeout
2021/04/25 23:37:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:24 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:37:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:27 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:37:27 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.149:42836
2021/04/25 23:37:27 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:37:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:32 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:37:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:34 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:45916->10.242.93.149:8300: i/o timeout
2021/04/25 23:37:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:37 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:45984->10.242.93.149:8300: i/o timeout
2021/04/25 23:37:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:44 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:37:44 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:37:44 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:37:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:48 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:37:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:50 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:37:50 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:37:52 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:37:53 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:37:53 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.50022632s
2021/04/25 23:37:54 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:46208->10.242.93.149:8300: i/o timeout
2021/04/25 23:37:54 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:33784->10.242.93.149:8302: i/o timeout
2021/04/25 23:37:54 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:37:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:56 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.971388056s
2021/04/25 23:37:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:37:58 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.431542805s
2021/04/25 23:38:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:02 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:46292->10.242.93.149:8300: i/o timeout
2021/04/25 23:38:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:02 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:38:04 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:38:04 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:33858->10.242.93.149:8302: i/o timeout
2021/04/25 23:38:04 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:38:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:12 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:46498->10.242.93.149:8300: i/o timeout
2021/04/25 23:38:13 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:38:14 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:46512->10.242.93.149:8300: i/o timeout
2021/04/25 23:38:14 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:34076->10.242.93.149:8302: i/o timeout
2021/04/25 23:38:14 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:38:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:23 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:46644->10.242.93.149:8300: i/o timeout
2021/04/25 23:38:24 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:38:24 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/04/25 23:38:24 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:38:24 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:38:24 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:34222->10.242.93.149:8302: i/o timeout
2021/04/25 23:38:24 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:38:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:24 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:38:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:26 [WARN] memberlist: Refuting a dead message (from: dev-consul-7467bf7b46-ww995)
2021/04/25 23:38:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:34 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:46794->10.242.93.149:8300: i/o timeout
2021/04/25 23:38:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:34 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:46810->10.242.93.149:8300: i/o timeout
2021/04/25 23:38:34 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8301->10.242.93.140:33086: i/o timeout from=10.242.93.140:33086
2021/04/25 23:38:34 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8301->10.242.93.140:33086: i/o timeout from=10.242.93.140:33086
2021/04/25 23:38:34 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:38:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:42 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8302->10.242.93.149:34324: i/o timeout from=10.242.93.149:34324
2021/04/25 23:38:42 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8302->10.242.93.149:34324: i/o timeout from=10.242.93.149:34324
2021/04/25 23:38:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:44 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:38:44 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:46960->10.242.93.149:8300: i/o timeout
2021/04/25 23:38:45 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:38:46 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:38:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:50 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:38:50 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' joined, marking health alive
2021/04/25 23:38:52 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:38:52 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:38:52 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/25 23:38:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:38:52 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-fdl7b 10.242.32.109
2021/04/25 23:38:52 [INFO] consul: member 'dev-consul-7467bf7b46-fdl7b' failed, marking health critical
2021/04/25 23:38:52 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:38:52 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:38:52 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-fdl7b 10.242.32.109
2021/04/25 23:38:52 [INFO] consul: member 'dev-consul-7467bf7b46-fdl7b' joined, marking health alive
2021/04/25 23:38:53 [INFO] raft: pipelining replication to peer {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}
2021/04/25 23:38:55 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:38:55 [WARN] consul: error getting server health from "dev-consul-server-1": context deadline exceeded
2021/04/25 23:38:56 [WARN] memberlist: Was able to connect to dev-consul-7467bf7b46-ww995 but other probes failed, network may be misconfigured
2021/04/25 23:38:59 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:39:00 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:54130->10.242.93.140:8301: i/o timeout
2021/04/25 23:39:00 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:39:15 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:39:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:39:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:39:25 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:54484->10.242.93.140:8301: i/o timeout
2021/04/25 23:39:25 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:39:26 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:39:26 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' failed, marking health critical
2021/04/25 23:39:29 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:39:29 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' joined, marking health alive
2021/04/25 23:39:29 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:39:31 [WARN] memberlist: Was able to connect to dev-consul-server-2 but other probes failed, network may be misconfigured
2021/04/25 23:39:31 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:39:31 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/25 23:39:31 [WARN] consul: error getting server health from "dev-consul-server-1": context deadline exceeded
2021/04/25 23:39:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:39:33 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:49108->10.242.93.149:8301: i/o timeout
2021/04/25 23:39:33 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:39:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:39:35 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.50023005s
2021/04/25 23:39:35 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:39:35 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:39:35 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:39:35 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/25 23:39:36 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:54618->10.242.93.140:8301: i/o timeout
2021/04/25 23:39:36 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:39:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:39:37 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:39:37 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' failed, marking health critical
2021/04/25 23:39:37 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.981764637s
2021/04/25 23:39:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:39:39 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2)
2021/04/25 23:39:39 [WARN] memberlist: Was able to connect to dev-consul-server-2.dc1 but other probes failed, network may be misconfigured
2021/04/25 23:39:39 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:39:41 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500331766s
2021/04/25 23:39:41 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:39:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:39:44 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.99955631s
2021/04/25 23:39:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:39:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:39:47 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500209397s
2021/04/25 23:39:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:39:49 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:39:49 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:35468->10.242.93.149:8302: i/o timeout
2021/04/25 23:39:49 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:39:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:39:50 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:39:51 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:39:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:39:53 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500263006s
2021/04/25 23:39:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:39:55 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.975233748s
2021/04/25 23:39:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:39:58 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.390503136s
2021/04/25 23:39:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:39:59 [ERR] raft: Failed to pipeline AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: write tcp 10.242.187.2:47104->10.242.93.149:8300: use of closed network connection
2021/04/25 23:39:59 [INFO] raft: aborting pipeline replication to peer {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}
2021/04/25 23:39:59 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:39:59 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:35604->10.242.93.149:8302: i/o timeout
2021/04/25 23:39:59 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:40:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:01 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500265516s
2021/04/25 23:40:01 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:47134->10.242.93.149:8300: i/o timeout
2021/04/25 23:40:02 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:40:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:03 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.920276355s
2021/04/25 23:40:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:06 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.396011731s
2021/04/25 23:40:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:06 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:40:06 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:40:06 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:40:06 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.149:42902
2021/04/25 23:40:09 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:48188->10.242.93.149:8300: i/o timeout
2021/04/25 23:40:09 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:40:09 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:35754->10.242.93.149:8302: i/o timeout
2021/04/25 23:40:09 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:40:09 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:40:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:11 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:40:11 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' joined, marking health alive
2021/04/25 23:40:11 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:40:11 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:40:12 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/25 23:40:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:13 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:49692->10.242.93.149:8301: i/o timeout
2021/04/25 23:40:13 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:40:14 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/04/25 23:40:14 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:40:14 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:40:14 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500317328s
2021/04/25 23:40:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:16 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:55242->10.242.93.140:8301: i/o timeout
2021/04/25 23:40:16 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:40:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:16 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.975207869s
2021/04/25 23:40:17 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:40:17 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:40:17 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:40:17 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:40:17 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:40:17 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/25 23:40:17 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:40:18 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:40:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:19 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:48338->10.242.93.149:8300: i/o timeout
2021/04/25 23:40:19 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:40:19 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:40:19 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:49768->10.242.93.149:8301: i/o timeout
2021/04/25 23:40:19 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:40:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:20 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:40:20 [INFO] memberlist: Marking dev-consul-7467bf7b46-ww995 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:40:20 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:40:20 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' failed, marking health critical
2021/04/25 23:40:21 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500247633s
2021/04/25 23:40:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:23 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.950529506s
2021/04/25 23:40:24 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:35982->10.242.93.149:8302: i/o timeout
2021/04/25 23:40:24 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:40:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:25 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.372957197s
2021/04/25 23:40:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:29 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:48248->10.242.93.149:8300: i/o timeout
2021/04/25 23:40:29 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:40:29 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:48492->10.242.93.149:8300: i/o timeout
2021/04/25 23:40:29 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:40:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:34 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:36132->10.242.93.149:8302: i/o timeout
2021/04/25 23:40:34 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:40:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:39 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:48634->10.242.93.149:8300: i/o timeout
2021/04/25 23:40:39 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:40:39 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:48648->10.242.93.149:8300: i/o timeout
2021/04/25 23:40:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:40 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:40:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:44 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:36200->10.242.93.149:8302: i/o timeout
2021/04/25 23:40:44 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:40:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:48 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:40:48 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:40:49 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:48786->10.242.93.149:8300: i/o timeout
2021/04/25 23:40:49 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:40:50 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:48806->10.242.93.149:8300: i/o timeout
2021/04/25 23:40:51 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:40:51 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:40:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:54 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/04/25 23:40:54 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:40:54 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:40:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:40:59 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:48936->10.242.93.149:8300: i/o timeout
2021/04/25 23:40:59 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:40:59 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:36496->10.242.93.149:8302: i/o timeout
2021/04/25 23:40:59 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:41:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:00 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:41:01 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:48980->10.242.93.149:8300: i/o timeout
2021/04/25 23:41:02 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:41:02 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:41:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:09 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:49082->10.242.93.149:8300: i/o timeout
2021/04/25 23:41:09 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:41:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:12 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:49136->10.242.93.149:8300: i/o timeout
2021/04/25 23:41:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:13 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:41:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:16 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:41:16 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' joined, marking health alive
2021/04/25 23:41:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:19 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:56178->10.242.93.140:8301: i/o timeout
2021/04/25 23:41:19 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:41:19 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:49236->10.242.93.149:8300: i/o timeout
2021/04/25 23:41:20 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:41:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:21 [INFO] memberlist: Marking dev-consul-7467bf7b46-ww995 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:41:21 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:41:21 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' failed, marking health critical
2021/04/25 23:41:22 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:56190->10.242.93.140:8301: i/o timeout
2021/04/25 23:41:22 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:41:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:23 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:49286->10.242.93.149:8300: i/o timeout
2021/04/25 23:41:23 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:41:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:30 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:49398->10.242.93.149:8300: i/o timeout
2021/04/25 23:41:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:30 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:41:30 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:41:31 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:41:33 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:41:33 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:49440->10.242.93.149:8300: i/o timeout
2021/04/25 23:41:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:35 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:41:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:41 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:49570->10.242.93.149:8300: i/o timeout
2021/04/25 23:41:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:44 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:41:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:45 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:49616->10.242.93.149:8300: i/o timeout
2021/04/25 23:41:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:46 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:41:47 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8302->10.242.93.149:36058: i/o timeout from=10.242.93.149:36058
2021/04/25 23:41:47 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8302->10.242.93.149:36058: i/o timeout from=10.242.93.149:36058
2021/04/25 23:41:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:54 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:49734->10.242.93.149:8300: i/o timeout
2021/04/25 23:41:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:56 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:49790->10.242.93.149:8300: i/o timeout
2021/04/25 23:41:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:41:58 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:41:59 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:42:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:02 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:42:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:08 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:49960->10.242.93.149:8300: i/o timeout
2021/04/25 23:42:09 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:49964->10.242.93.149:8300: i/o timeout
2021/04/25 23:42:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:10 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:42:12 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:42:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:12 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:42:12 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:42:15 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:42:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:19 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:42:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:22 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:50158->10.242.93.149:8300: i/o timeout
2021/04/25 23:42:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:27 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:42:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:29 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:50266->10.242.93.149:8300: i/o timeout
2021/04/25 23:42:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:37 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:50390->10.242.93.149:8300: i/o timeout
2021/04/25 23:42:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:39 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:42:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:48 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:42:49 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:50562->10.242.93.149:8300: i/o timeout
2021/04/25 23:42:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:50 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:42:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:54 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:42:54 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:42:57 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:42:57 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/25 23:42:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:42:58 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:50700->10.242.93.149:8300: i/o timeout
2021/04/25 23:43:00 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:43:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:02 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:43:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:09 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:43:10 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:50858->10.242.93.149:8300: i/o timeout
2021/04/25 23:43:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:11 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:43:12 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:43:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:14 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500156837s
2021/04/25 23:43:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:16 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.982555433s
2021/04/25 23:43:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:19 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.452113938s
2021/04/25 23:43:20 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:43:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:22 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:51000->10.242.93.149:8300: i/o timeout
2021/04/25 23:43:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:23 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:43:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:30 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:51150->10.242.93.149:8300: i/o timeout
2021/04/25 23:43:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:30 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:43:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:33 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:51192->10.242.93.149:8300: i/o timeout
2021/04/25 23:43:34 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:43:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:39 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:43:39 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' joined, marking health alive
2021/04/25 23:43:40 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:43:40 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:43:40 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/25 23:43:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:41 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:52758->10.242.93.149:8301: i/o timeout
2021/04/25 23:43:41 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:43:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:42 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500340099s
2021/04/25 23:43:43 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:58286->10.242.93.140:8301: i/o timeout
2021/04/25 23:43:43 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:43:44 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:43:44 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' failed, marking health critical
2021/04/25 23:43:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:45 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.937945778s
2021/04/25 23:43:45 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:43:45 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:43:45 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:43:45 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/25 23:43:46 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:52810->10.242.93.149:8301: i/o timeout
2021/04/25 23:43:46 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:43:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:48 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:43:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:49 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500210695s
2021/04/25 23:43:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:50 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:51342->10.242.93.149:8300: i/o timeout
2021/04/25 23:43:52 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.946204894s
2021/04/25 23:43:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:54 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.406224016s
2021/04/25 23:43:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:58 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:51470->10.242.93.149:8300: i/o timeout
2021/04/25 23:43:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:43:58 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:44:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:00 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:44:01 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8302->10.242.93.149:36624: i/o timeout from=10.242.93.149:36624
2021/04/25 23:44:01 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8302->10.242.93.149:36624: i/o timeout from=10.242.93.149:36624
2021/04/25 23:44:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:02 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:44:02 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.149:45158
2021/04/25 23:44:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:08 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:51716->10.242.93.149:8300: i/o timeout
2021/04/25 23:44:09 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:44:09 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:44:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:10 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:51744->10.242.93.149:8300: i/o timeout
2021/04/25 23:44:11 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:44:11 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.140:33960
2021/04/25 23:44:11 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:44:11 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:44:13 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:44:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:19 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:51870->10.242.93.149:8300: i/o timeout
2021/04/25 23:44:20 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:44:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:21 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:44:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:30 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:52026->10.242.93.149:8300: i/o timeout
2021/04/25 23:44:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:31 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:44:31 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:52056->10.242.93.149:8300: i/o timeout
2021/04/25 23:44:32 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:44:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:41 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:52196->10.242.93.149:8300: i/o timeout
2021/04/25 23:44:41 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:44:41 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:44:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:51 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:52350->10.242.93.149:8300: i/o timeout
2021/04/25 23:44:51 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:52358->10.242.93.149:8300: i/o timeout
2021/04/25 23:44:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:52 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:44:52 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:44:53 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:44:55 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:44:55 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/25 23:44:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:44:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:01 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:45:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:03 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:52508->10.242.93.149:8300: i/o timeout
2021/04/25 23:45:03 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:45:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:11 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:52650->10.242.93.149:8300: i/o timeout
2021/04/25 23:45:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:13 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:52666->10.242.93.149:8300: i/o timeout
2021/04/25 23:45:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:15 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:45:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:19 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:45:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:21 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:45:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:25 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:52840->10.242.93.149:8300: i/o timeout
2021/04/25 23:45:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:27 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:45:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:31 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:52948->10.242.93.149:8300: i/o timeout
2021/04/25 23:45:32 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:45:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:34 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:45:34 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:45:37 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:53024->10.242.93.149:8300: i/o timeout
2021/04/25 23:45:37 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:45:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:40 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:45:42 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:45:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:50 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:53216->10.242.93.149:8300: i/o timeout
2021/04/25 23:45:52 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:53250->10.242.93.149:8300: i/o timeout
2021/04/25 23:45:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:56 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:45:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:45:59 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:46:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:02 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:46:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:06 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:53454->10.242.93.149:8300: i/o timeout
2021/04/25 23:46:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:12 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:53542->10.242.93.149:8300: i/o timeout
2021/04/25 23:46:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:16 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:46:16 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:46:17 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:46:19 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:46:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:22 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:46:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:27 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:53760->10.242.93.149:8300: i/o timeout
2021/04/25 23:46:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:32 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:53830->10.242.93.149:8300: i/o timeout
2021/04/25 23:46:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:38 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:46:39 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:46:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:42 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:46:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:48 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:54060->10.242.93.149:8300: i/o timeout
2021/04/25 23:46:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:52 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:54122->10.242.93.149:8300: i/o timeout
2021/04/25 23:46:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:46:58 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:46:58 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:47:00 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:47:01 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:47:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:03 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:47:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:10 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:54364->10.242.93.149:8300: i/o timeout
2021/04/25 23:47:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:13 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:54414->10.242.93.149:8300: i/o timeout
2021/04/25 23:47:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:19 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:47:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:21 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:47:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:23 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:47:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:31 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:54688->10.242.93.149:8300: i/o timeout
2021/04/25 23:47:32 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:47:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:33 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:54702->10.242.93.149:8300: i/o timeout
2021/04/25 23:47:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:40 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:47:40 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:47:41 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:47:43 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:47:43 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:47:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:51 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:54984->10.242.93.149:8300: i/o timeout
2021/04/25 23:47:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:53 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:54994->10.242.93.149:8300: i/o timeout
2021/04/25 23:47:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:47:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:02 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:48:03 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:48:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:12 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:55282->10.242.93.149:8300: i/o timeout
2021/04/25 23:48:13 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:55286->10.242.93.149:8300: i/o timeout
2021/04/25 23:48:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:22 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:48:22 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:48:23 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:48:24 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:48:25 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:48:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:29 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:48:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:33 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:55568->10.242.93.149:8300: i/o timeout
2021/04/25 23:48:34 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:55572->10.242.93.149:8300: i/o timeout
2021/04/25 23:48:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:39 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:48:39 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:48:39 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/25 23:48:39 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:48:39 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' joined, marking health alive
2021/04/25 23:48:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:41 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:34406->10.242.93.140:8301: i/o timeout
2021/04/25 23:48:41 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:48:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:45 [INFO] memberlist: Marking dev-consul-7467bf7b46-ww995 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:48:45 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:48:45 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:57160->10.242.93.149:8301: i/o timeout
2021/04/25 23:48:45 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:48:45 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' failed, marking health critical
2021/04/25 23:48:45 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:48:45 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:48:45 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:48:45 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/25 23:48:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:48 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:57214->10.242.93.149:8301: i/o timeout
2021/04/25 23:48:48 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:48:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:48 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8301->10.242.93.149:59536: i/o timeout from=10.242.93.149:59536
2021/04/25 23:48:48 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8301->10.242.93.149:59536: i/o timeout from=10.242.93.149:59536
2021/04/25 23:48:49 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8301->10.242.93.140:36598: i/o timeout from=10.242.93.140:36598
2021/04/25 23:48:49 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8301->10.242.93.140:36598: i/o timeout from=10.242.93.140:36598
2021/04/25 23:48:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:50 [ERR] memberlist: Push/Pull with dev-consul-7467bf7b46-ww995 failed: read tcp 10.242.187.2:34394->10.242.93.140:8301: i/o timeout
2021/04/25 23:48:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:54 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:55868->10.242.93.149:8300: i/o timeout
2021/04/25 23:48:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:54 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:55882->10.242.93.149:8300: i/o timeout
2021/04/25 23:48:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:48:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:02 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:49:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:04 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:49:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:04 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:49:04 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:49:05 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:49:07 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:49:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:14 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:56168->10.242.93.149:8300: i/o timeout
2021/04/25 23:49:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:15 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:56194->10.242.93.149:8300: i/o timeout
2021/04/25 23:49:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:24 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:49:26 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:49:26 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' joined, marking health alive
2021/04/25 23:49:26 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:49:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:28 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:35084->10.242.93.140:8301: i/o timeout
2021/04/25 23:49:28 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:49:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:31 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:35138->10.242.93.140:8301: i/o timeout
2021/04/25 23:49:31 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/25 23:49:32 [INFO] memberlist: Marking dev-consul-7467bf7b46-ww995 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:49:32 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/25 23:49:32 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' failed, marking health critical
2021/04/25 23:49:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:33 [INFO] serf: EventMemberFailed: dev-consul-server-1 10.242.32.71
2021/04/25 23:49:33 [INFO] consul: Removing LAN server dev-consul-server-1 (Addr: tcp/10.242.32.71:8300) (DC: dc1)
2021/04/25 23:49:33 [INFO] consul: member 'dev-consul-server-1' failed, marking health critical
2021/04/25 23:49:33 [WARN] Unable to get address for server id 9fcb14a9-100e-2d75-3e7a-6fe772ff1f8f, using fallback address 10.242.32.71:8300: Could not find address for server id 9fcb14a9-100e-2d75-3e7a-6fe772ff1f8f
2021/04/25 23:49:33 [INFO] serf: EventMemberJoin: dev-consul-server-1 10.242.32.71
2021/04/25 23:49:33 [INFO] consul: Adding LAN server dev-consul-server-1 (Addr: tcp/10.242.32.71:8300) (DC: dc1)
2021/04/25 23:49:33 [INFO] consul: member 'dev-consul-server-1' joined, marking health alive
2021/04/25 23:49:34 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:49:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:34 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:56462->10.242.93.149:8300: i/o timeout
2021/04/25 23:49:36 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500227048s
2021/04/25 23:49:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:37 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:49:37 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:49:37 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2)
2021/04/25 23:49:37 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:49:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:39 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500214886s
2021/04/25 23:49:39 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:49:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:41 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.986860365s
2021/04/25 23:49:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:44 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.46108914s
2021/04/25 23:49:44 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:49:44 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:49:44 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/25 23:49:44 [WARN] memberlist: Was able to connect to dev-consul-server-2.dc1 but other probes failed, network may be misconfigured
2021/04/25 23:49:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:46 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:58082->10.242.93.149:8301: i/o timeout
2021/04/25 23:49:46 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:49:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:49 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:58130->10.242.93.149:8301: i/o timeout
2021/04/25 23:49:49 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:49:49 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:49:49 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:49:49 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/25 23:49:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:51 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500258325s
2021/04/25 23:49:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:53 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.965775759s
2021/04/25 23:49:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:54 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:56500->10.242.93.149:8300: i/o timeout
2021/04/25 23:49:56 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.458552463s
2021/04/25 23:49:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:49:59 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:44392->10.242.93.149:8302: i/o timeout
2021/04/25 23:49:59 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:49:59 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:56760->10.242.93.149:8300: i/o timeout
2021/04/25 23:50:00 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:50:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:05 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:50:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:09 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:44470->10.242.93.149:8302: i/o timeout
2021/04/25 23:50:09 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:50:10 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:56992->10.242.93.149:8300: i/o timeout
2021/04/25 23:50:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:11 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:50:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:14 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/04/25 23:50:14 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:50:14 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:50:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:15 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:57068->10.242.93.149:8300: i/o timeout
2021/04/25 23:50:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:16 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:50:16 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:50:19 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:50:19 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:50:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:21 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:57166->10.242.93.149:8300: i/o timeout
2021/04/25 23:50:21 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:50:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:24 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:50:24 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.149:46562
2021/04/25 23:50:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:25 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:50:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:31 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:57320->10.242.93.149:8300: i/o timeout
2021/04/25 23:50:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:32 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:50:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:35 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:57364->10.242.93.149:8300: i/o timeout
2021/04/25 23:50:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:42 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:57474->10.242.93.149:8300: i/o timeout
2021/04/25 23:50:43 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:50:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:45 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:50:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:53 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:57616->10.242.93.149:8300: i/o timeout
2021/04/25 23:50:54 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:50:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:55 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:57658->10.242.93.149:8300: i/o timeout
2021/04/25 23:50:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:50:58 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:50:58 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:50:59 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:51:01 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:51:02 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:51:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:04 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:57774->10.242.93.149:8300: i/o timeout
2021/04/25 23:51:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:05 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:51:05 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:51:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:11 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:51:12 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500862773s
2021/04/25 23:51:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:15 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.958791294s
2021/04/25 23:51:15 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:57956->10.242.93.149:8300: i/o timeout
2021/04/25 23:51:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:17 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.412456614s
2021/04/25 23:51:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:21 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:57952->10.242.93.149:8300: i/o timeout
2021/04/25 23:51:21 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:51:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:26 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:51:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:31 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:58194->10.242.93.149:8300: i/o timeout
2021/04/25 23:51:32 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:51:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:36 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:58248->10.242.93.149:8300: i/o timeout
2021/04/25 23:51:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:39 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:51:39 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8302->10.242.93.149:38442: i/o timeout from=10.242.93.149:38442
2021/04/25 23:51:39 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8302->10.242.93.149:38442: i/o timeout from=10.242.93.149:38442
2021/04/25 23:51:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:40 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:51:40 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:51:42 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:58342->10.242.93.149:8300: i/o timeout
2021/04/25 23:51:43 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:51:43 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:51:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:46 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:51:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:53 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:58492->10.242.93.149:8300: i/o timeout
2021/04/25 23:51:53 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:51:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:56 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:58552->10.242.93.149:8300: i/o timeout
2021/04/25 23:51:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:51:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:03 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:58644->10.242.93.149:8300: i/o timeout
2021/04/25 23:52:04 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:52:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:06 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:52:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:14 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:58796->10.242.93.149:8300: i/o timeout
2021/04/25 23:52:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:15 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:52:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:16 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:58844->10.242.93.149:8300: i/o timeout
2021/04/25 23:52:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:19 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/25 23:52:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:22 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:52:22 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:52:25 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:58964->10.242.93.149:8300: i/o timeout
2021/04/25 23:52:25 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:52:25 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/25 23:52:26 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:52:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:26 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:52:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:36 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:59138->10.242.93.149:8300: i/o timeout
2021/04/25 23:52:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:36 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:59146->10.242.93.149:8300: i/o timeout
2021/04/25 23:52:38 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:52:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:47 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:52:48 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:59302->10.242.93.149:8300: i/o timeout
2021/04/25 23:52:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:49 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:52:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:51 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8301->10.242.93.149:60614: i/o timeout from=10.242.93.149:60614
2021/04/25 23:52:51 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8301->10.242.93.149:60614: i/o timeout from=10.242.93.149:60614
2021/04/25 23:52:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:57 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:59442->10.242.93.149:8300: i/o timeout
2021/04/25 23:52:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:52:59 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:52:59 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:59478->10.242.93.149:8300: i/o timeout
2021/04/25 23:53:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:03 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:53:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:04 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:53:04 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:53:06 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:53:06 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:53:06 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/25 23:53:07 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:53:08 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:32810->10.242.93.149:8301: i/o timeout
2021/04/25 23:53:08 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:53:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:11 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:32858->10.242.93.149:8301: i/o timeout
2021/04/25 23:53:11 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:53:12 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:53:12 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:53:12 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:53:12 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/25 23:53:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:13 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:59670->10.242.93.149:8300: i/o timeout
2021/04/25 23:53:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:17 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:59742->10.242.93.149:8300: i/o timeout
2021/04/25 23:53:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:18 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:53:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:27 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:53:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:28 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:59902->10.242.93.149:8300: i/o timeout
2021/04/25 23:53:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:32 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:53:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:36 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:53:37 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:60032->10.242.93.149:8300: i/o timeout
2021/04/25 23:53:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:39 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:53:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:46 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:53:46 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:53:47 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:53:49 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:53:49 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/25 23:53:49 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:60218->10.242.93.149:8300: i/o timeout
2021/04/25 23:53:50 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:53:50 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.149:47768
2021/04/25 23:53:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:53:57 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:60338->10.242.93.149:8300: i/o timeout
2021/04/25 23:53:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:00 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:54:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:08 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:54:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:10 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:60528->10.242.93.149:8300: i/o timeout
2021/04/25 23:54:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:18 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:60628->10.242.93.149:8300: i/o timeout
2021/04/25 23:54:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:21 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:54:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:28 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:54:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:28 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:54:28 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:54:31 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:54:31 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:60834->10.242.93.149:8300: i/o timeout
2021/04/25 23:54:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:38 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:60914->10.242.93.149:8300: i/o timeout
2021/04/25 23:54:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:42 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:54:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:46 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:54:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:48 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:54:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:52 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:32894->10.242.93.149:8300: i/o timeout
2021/04/25 23:54:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:54:58 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:32984->10.242.93.149:8300: i/o timeout
2021/04/25 23:55:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:02 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:55:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:03 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:55:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:08 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:55:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:10 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:55:10 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:55:13 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:55:13 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/25 23:55:13 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:33198->10.242.93.149:8300: i/o timeout
2021/04/25 23:55:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:18 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:33274->10.242.93.149:8300: i/o timeout
2021/04/25 23:55:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:25 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:55:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:29 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:55:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:35 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:33504->10.242.93.149:8300: i/o timeout
2021/04/25 23:55:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:39 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:33562->10.242.93.149:8300: i/o timeout
2021/04/25 23:55:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:46 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:55:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:49 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:55:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:52 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:55:52 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:55:55 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:55:56 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:33820->10.242.93.149:8300: i/o timeout
2021/04/25 23:55:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:55:59 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:33858->10.242.93.149:8300: i/o timeout
2021/04/25 23:56:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:07 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:56:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:09 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:56:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:17 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:34126->10.242.93.149:8300: i/o timeout
2021/04/25 23:56:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:19 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:34148->10.242.93.149:8300: i/o timeout
2021/04/25 23:56:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:28 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:56:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:29 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:56:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:32 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:56:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:34 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:56:34 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:56:37 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:56:38 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:34424->10.242.93.149:8300: i/o timeout
2021/04/25 23:56:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:39 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:56:39 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:56:39 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:56:39 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:56:39 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/25 23:56:39 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:34440->10.242.93.149:8300: i/o timeout
2021/04/25 23:56:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:42 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:35924->10.242.93.149:8301: i/o timeout
2021/04/25 23:56:42 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:56:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:46 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/04/25 23:56:46 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:56:46 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:56:46 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/25 23:56:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:49 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:56:49 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:50378->10.242.93.149:8302: i/o timeout
2021/04/25 23:56:49 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:56:50 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:56:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:56 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:56:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:56:59 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:34728->10.242.93.149:8300: i/o timeout
2021/04/25 23:56:59 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:50524->10.242.93.149:8302: i/o timeout
2021/04/25 23:56:59 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:57:00 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:34740->10.242.93.149:8300: i/o timeout
2021/04/25 23:57:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:10 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:57:10 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:57:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:14 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:50742->10.242.93.149:8302: i/o timeout
2021/04/25 23:57:14 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/25 23:57:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:16 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:57:16 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:57:19 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/04/25 23:57:19 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.93.149
2021/04/25 23:57:19 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/25 23:57:19 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:57:19 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:57:19 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.149:48588
2021/04/25 23:57:20 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:35030->10.242.93.149:8300: i/o timeout
2021/04/25 23:57:20 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:35034->10.242.93.149:8300: i/o timeout
2021/04/25 23:57:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:30 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:57:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:31 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:57:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:36 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:57:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:40 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:35324->10.242.93.149:8300: i/o timeout
2021/04/25 23:57:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:41 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:35352->10.242.93.149:8300: i/o timeout
2021/04/25 23:57:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:50 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:57:52 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/25 23:57:52 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:57:52 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/25 23:57:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:56 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:36996->10.242.93.149:8301: i/o timeout
2021/04/25 23:57:56 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:57:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:57 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/25 23:57:57 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/25 23:57:57 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/25 23:57:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:57:58 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:57:58 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:57:59 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:37016->10.242.93.149:8301: i/o timeout
2021/04/25 23:57:59 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/25 23:58:00 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:35620->10.242.93.149:8300: i/o timeout
2021/04/25 23:58:01 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:58:02 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:35648->10.242.93.149:8300: i/o timeout
2021/04/25 23:58:02 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:58:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:11 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:58:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:13 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:58:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:21 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:35928->10.242.93.149:8300: i/o timeout
2021/04/25 23:58:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:23 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:35950->10.242.93.149:8300: i/o timeout
2021/04/25 23:58:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:31 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:58:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:34 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:58:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:40 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:58:40 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:58:41 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:36224->10.242.93.149:8300: i/o timeout
2021/04/25 23:58:43 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:58:43 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/25 23:58:44 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:36246->10.242.93.149:8300: i/o timeout
2021/04/25 23:58:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:51 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:58:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:55 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:58:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:58:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:01 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:36518->10.242.93.149:8300: i/o timeout
2021/04/25 23:59:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:05 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:36560->10.242.93.149:8300: i/o timeout
2021/04/25 23:59:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:11 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:59:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:16 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:59:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:21 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:36802->10.242.93.149:8300: i/o timeout
2021/04/25 23:59:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:22 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/25 23:59:22 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/25 23:59:25 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/25 23:59:26 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:36866->10.242.93.149:8300: i/o timeout
2021/04/25 23:59:26 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8302->10.242.93.149:40582: i/o timeout from=10.242.93.149:40582
2021/04/25 23:59:26 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8302->10.242.93.149:40582: i/o timeout from=10.242.93.149:40582
2021/04/25 23:59:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:31 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:59:32 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/25 23:59:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:37 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:59:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:41 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:37098->10.242.93.149:8300: i/o timeout
2021/04/25 23:59:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:46 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/25 23:59:47 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:37174->10.242.93.149:8300: i/o timeout
2021/04/25 23:59:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:52 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/25 23:59:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/25 23:59:58 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:00:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:02 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:37390->10.242.93.149:8300: i/o timeout
2021/04/26 00:00:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:04 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/26 00:00:04 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/26 00:00:07 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/26 00:00:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:08 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:37478->10.242.93.149:8300: i/o timeout
2021/04/26 00:00:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:12 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:00:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:19 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:00:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:22 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:37678->10.242.93.149:8300: i/o timeout
2021/04/26 00:00:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:26 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/26 00:00:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:29 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:37778->10.242.93.149:8300: i/o timeout
2021/04/26 00:00:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:32 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/26 00:00:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:32 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:00:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:40 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:00:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:42 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:37976->10.242.93.149:8300: i/o timeout
2021/04/26 00:00:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:46 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/26 00:00:46 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/26 00:00:49 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/26 00:00:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:50 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:38082->10.242.93.149:8300: i/o timeout
2021/04/26 00:00:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:52 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:00:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:00:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:01 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:01:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:02 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:38264->10.242.93.149:8300: i/o timeout
2021/04/26 00:01:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:06 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/26 00:01:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:11 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:38402->10.242.93.149:8300: i/o timeout
2021/04/26 00:01:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:13 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:01:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:22 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:01:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:23 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:38554->10.242.93.149:8300: i/o timeout
2021/04/26 00:01:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:28 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/26 00:01:28 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/26 00:01:31 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/26 00:01:32 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:38696->10.242.93.149:8300: i/o timeout
2021/04/26 00:01:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:32 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/26 00:01:33 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:01:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:43 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:38850->10.242.93.149:8300: i/o timeout
2021/04/26 00:01:43 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:01:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:46 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/26 00:01:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:53 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:38990->10.242.93.149:8300: i/o timeout
2021/04/26 00:01:53 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:01:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:01:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:03 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:39140->10.242.93.149:8300: i/o timeout
2021/04/26 00:02:04 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:02:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:05 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.93.149
2021/04/26 00:02:05 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/26 00:02:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:10 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/26 00:02:10 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/26 00:02:13 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/26 00:02:13 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:02:14 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:55090->10.242.93.149:8302: i/o timeout
2021/04/26 00:02:14 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/26 00:02:14 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:39298->10.242.93.149:8300: i/o timeout
2021/04/26 00:02:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:23 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:39432->10.242.93.149:8300: i/o timeout
2021/04/26 00:02:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:25 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:02:26 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.93.149
2021/04/26 00:02:26 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-fdl7b 10.242.32.109
2021/04/26 00:02:26 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/26 00:02:26 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/04/26 00:02:26 [INFO] consul: member 'dev-consul-7467bf7b46-fdl7b' failed, marking health critical
2021/04/26 00:02:26 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-fdl7b 10.242.32.109
2021/04/26 00:02:26 [INFO] consul: member 'dev-consul-7467bf7b46-fdl7b' joined, marking health alive
2021/04/26 00:02:27 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/26 00:02:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:29 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:55312->10.242.93.149:8302: i/o timeout
2021/04/26 00:02:29 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/26 00:02:29 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:40960->10.242.93.149:8301: i/o timeout
2021/04/26 00:02:29 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/26 00:02:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:32 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:41016->10.242.93.149:8301: i/o timeout
2021/04/26 00:02:32 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/04/26 00:02:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:32 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.93.149
2021/04/26 00:02:32 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.93.149:8300) (DC: dc1)
2021/04/26 00:02:32 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/04/26 00:02:34 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:02:34 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/26 00:02:34 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' joined, marking health alive
2021/04/26 00:02:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:35 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:39606->10.242.93.149:8300: i/o timeout
2021/04/26 00:02:36 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:46580->10.242.93.140:8301: i/o timeout
2021/04/26 00:02:36 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/26 00:02:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:39 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:55392->10.242.93.149:8302: i/o timeout
2021/04/26 00:02:39 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/26 00:02:40 [INFO] memberlist: Marking dev-consul-7467bf7b46-ww995 as failed, suspect timeout reached (2 peer confirmations)
2021/04/26 00:02:40 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/26 00:02:40 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' failed, marking health critical
2021/04/26 00:02:40 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:46638->10.242.93.140:8301: i/o timeout
2021/04/26 00:02:40 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/26 00:02:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:44 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:39736->10.242.93.149:8300: i/o timeout
2021/04/26 00:02:44 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/26 00:02:44 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.93.149:49814
2021/04/26 00:02:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:46 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:02:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:49 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:55624->10.242.93.149:8302: i/o timeout
2021/04/26 00:02:49 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/26 00:02:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:52 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/26 00:02:52 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/26 00:02:54 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:02:55 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/26 00:02:55 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/26 00:02:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:56 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:39940->10.242.93.149:8300: i/o timeout
2021/04/26 00:02:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:02:59 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:55768->10.242.93.149:8302: i/o timeout
2021/04/26 00:02:59 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/26 00:03:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:04 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8302->10.242.93.149:41428: i/o timeout from=10.242.93.149:41428
2021/04/26 00:03:04 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8302->10.242.93.149:41428: i/o timeout from=10.242.93.149:41428
2021/04/26 00:03:04 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:40040->10.242.93.149:8300: i/o timeout
2021/04/26 00:03:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:07 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:03:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:09 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/04/26 00:03:09 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.93.149
2021/04/26 00:03:09 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/26 00:03:09 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:55914->10.242.93.149:8302: i/o timeout
2021/04/26 00:03:09 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/26 00:03:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:14 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:03:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:17 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:40248->10.242.93.149:8300: i/o timeout
2021/04/26 00:03:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:24 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:40336->10.242.93.149:8300: i/o timeout
2021/04/26 00:03:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:28 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:03:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:32 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/26 00:03:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:34 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/26 00:03:34 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/26 00:03:34 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:03:37 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/26 00:03:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:38 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:40548->10.242.93.149:8300: i/o timeout
2021/04/26 00:03:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:44 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:40636->10.242.93.149:8300: i/o timeout
2021/04/26 00:03:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:50 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:03:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:54 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8301->10.242.93.149:35260: i/o timeout from=10.242.93.149:35260
2021/04/26 00:03:54 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8301->10.242.93.149:35260: i/o timeout from=10.242.93.149:35260
2021/04/26 00:03:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:55 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:03:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:03:56 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/26 00:03:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:00 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:40856->10.242.93.149:8300: i/o timeout
2021/04/26 00:04:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:05 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:40928->10.242.93.149:8300: i/o timeout
2021/04/26 00:04:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:10 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:04:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:15 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:04:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:16 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/26 00:04:16 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/26 00:04:19 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/26 00:04:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:20 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:41160->10.242.93.149:8300: i/o timeout
2021/04/26 00:04:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:25 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/26 00:04:25 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' joined, marking health alive
2021/04/26 00:04:25 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:41220->10.242.93.149:8300: i/o timeout
2021/04/26 00:04:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:27 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:48198->10.242.93.140:8301: i/o timeout
2021/04/26 00:04:27 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/26 00:04:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:30 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:48208->10.242.93.140:8301: i/o timeout
2021/04/26 00:04:30 [INFO] memberlist: Suspect dev-consul-7467bf7b46-ww995 has failed, no acks received
2021/04/26 00:04:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:31 [INFO] memberlist: Marking dev-consul-7467bf7b46-ww995 as failed, suspect timeout reached (2 peer confirmations)
2021/04/26 00:04:31 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-ww995 10.242.93.140
2021/04/26 00:04:31 [INFO] consul: member 'dev-consul-7467bf7b46-ww995' failed, marking health critical
2021/04/26 00:04:31 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:04:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:32 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.93.149:8302
2021/04/26 00:04:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:35 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:04:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:36 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-ww995 10.242.93.140:8301
2021/04/26 00:04:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:41 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:41474->10.242.93.149:8300: i/o timeout
2021/04/26 00:04:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:45 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:41520->10.242.93.149:8300: i/o timeout
2021/04/26 00:04:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:52 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:04:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:55 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:04:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:04:58 [ERR] yamux: keepalive failed: i/o deadline reached
2021/04/26 00:04:58 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/04/26 00:05:01 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/26 00:05:01 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/26 00:05:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:02 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:41776->10.242.93.149:8300: i/o timeout
2021/04/26 00:05:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:05 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:41812->10.242.93.149:8300: i/o timeout
2021/04/26 00:05:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:13 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:05:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:16 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:05:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:18 [ERR] memberlist: failed to receive: read tcp 10.242.187.2:8301->10.242.93.149:35468: i/o timeout from=10.242.93.149:35468
2021/04/26 00:05:18 [ERR] memberlist: Failed to send error: write tcp 10.242.187.2:8301->10.242.93.149:35468: i/o timeout from=10.242.93.149:35468
2021/04/26 00:05:19 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:05:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:21 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:05:21 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2.dc1)
2021/04/26 00:05:21 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:05:21 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2)
2021/04/26 00:05:22 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:05:23 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/26 00:05:24 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500191505s
2021/04/26 00:05:24 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.93.149
2021/04/26 00:05:24 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/04/26 00:05:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:25 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:05:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:27 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500255421s
2021/04/26 00:05:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:29 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:05:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:30 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:05:31 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:05:31 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:42106->10.242.93.149:8300: i/o timeout
2021/04/26 00:05:31 [INFO] raft: pipelining replication to peer {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}
2021/04/26 00:05:32 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500184383s
2021/04/26 00:05:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:34 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:58024->10.242.93.149:8302: i/o timeout
2021/04/26 00:05:34 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/26 00:05:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:34 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.988145918s
2021/04/26 00:05:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:37 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.461561461s
2021/04/26 00:05:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:40 [ERR] raft: Failed to heartbeat to 10.242.93.149:8300: read tcp 10.242.187.2:42062->10.242.93.149:8300: i/o timeout
2021/04/26 00:05:41 [INFO] raft: aborting pipeline replication to peer {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}
2021/04/26 00:05:41 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:05:41 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:05:41 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:05:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:42 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:05:44 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.50036268s
2021/04/26 00:05:44 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:58168->10.242.93.149:8302: i/o timeout
2021/04/26 00:05:44 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/26 00:05:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:46 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 4.936967439s
2021/04/26 00:05:46 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.93.149:8301
2021/04/26 00:05:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:49 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 7.389786968s
2021/04/26 00:05:51 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:05:51 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/04/26 00:05:51 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/04/26 00:05:51 [ERR] raft: Failed to AppendEntries to {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}: read tcp 10.242.187.2:42484->10.242.93.149:8300: i/o timeout
2021/04/26 00:05:51 [WARN] Unable to get address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc, using fallback address 10.242.93.149:8300: Could not find address for server id f66dacd5-df84-469e-7a9b-4103153d1fdc
2021/04/26 00:05:51 [INFO] raft: pipelining replication to peer {Voter f66dacd5-df84-469e-7a9b-4103153d1fdc 10.242.93.149:8300}
2021/04/26 00:05:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/04/26 00:05:52 [WARN] raft: Failed to contact f66dacd5-df84-469e-7a9b-4103153d1fdc in 2.500239903s
2021/04/26 00:05:54 [ERR] memberlist: Failed fallback ping: read tcp 10.242.187.2:58246->10.242.93.149:8302: i/o timeout
2021/04/26 00:05:54 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/04/26 00:05:54 [WARN] consul: error getting server health from "dev-consul-serve