By type
+ CONSUL_BIND=
+ [ -n ]
+ CONSUL_CLIENT=
+ [ -n ]
+ CONSUL_DATA_DIR=/consul/data
+ CONSUL_CONFIG_DIR=/consul/config
+ [ -n ]
+ [ a = - ]
+ [ agent = agent ]
+ shift
+ set -- consul agent -data-dir=/consul/data -config-dir=/consul/config -bootstrap-expect=3 -enable-script-checks -retry-join=dev-consul-server-0.consul-server -retry-join=dev-consul-server-1.consul-server -retry-join=dev-consul-server-2.consul-server -client=0.0.0.0 -server -ui
+ [ consul = consul ]
+ stat -c %u /consul/data
+ id -u consul
+ [ 100 != 100 ]
+ stat -c %u /consul/config
+ id -u consul
+ [ 100 != 100 ]
+ [ ! -z ]
+ exec consul agent -data-dir=/consul/data -config-dir=/consul/config -bootstrap-expect=3 -enable-script-checks -retry-join=dev-consul-server-0.consul-server -retry-join=dev-consul-server-1.consul-server -retry-join=dev-consul-server-2.consul-server -client=0.0.0.0 -server -ui
bootstrap_expect > 0: expecting 3 servers
==> Starting Consul agent...
==> Consul agent running!
Version: 'v1.0.6'
Node ID: 'bc820de6-bd27-22d5-1da4-ff81b7c91007'
Node name: 'dev-consul-server-1'
Datacenter: 'dc1' (Segment: '')
Server: true (Bootstrap: false)
Client Addr: [0.0.0.0] (HTTP: 8500, HTTPS: -1, DNS: 8600)
Cluster Addr: 10.242.166.6 (LAN: 8301, WAN: 8302)
Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false
==> Log data will now stream in as it occurs:
2021/06/02 10:40:56 [INFO] raft: Initial configuration (index=0): []
2021/06/02 10:40:56 [INFO] raft: Node at 10.242.166.6:8300 [Follower] entering Follower state (Leader: "")
2021/06/02 10:40:56 [INFO] serf: EventMemberJoin: dev-consul-server-1.dc1 10.242.166.6
2021/06/02 10:40:56 [INFO] serf: EventMemberJoin: dev-consul-server-1 10.242.166.6
2021/06/02 10:40:56 [INFO] consul: Handled member-join event for server "dev-consul-server-1.dc1" in area "wan"
2021/06/02 10:40:56 [INFO] consul: Adding LAN server dev-consul-server-1 (Addr: tcp/10.242.166.6:8300) (DC: dc1)
2021/06/02 10:40:56 [INFO] agent: Started DNS server 0.0.0.0:8600 (tcp)
2021/06/02 10:40:56 [INFO] agent: Started DNS server 0.0.0.0:8600 (udp)
2021/06/02 10:40:56 [INFO] agent: Started HTTP server on [::]:8500 (tcp)
2021/06/02 10:40:56 [INFO] agent: started state syncer
2021/06/02 10:40:56 [INFO] agent: Retry join LAN is supported for: aliyun aws azure digitalocean gce os scaleway softlayer
2021/06/02 10:40:56 [INFO] agent: Joining LAN cluster...
2021/06/02 10:40:56 [INFO] agent: (LAN) joining: [dev-consul-server-0.consul-server dev-consul-server-1.consul-server dev-consul-server-2.consul-server]
2021/06/02 10:40:56 [INFO] serf: EventMemberJoin: dev-consul-server-0 10.242.68.10
2021/06/02 10:40:56 [INFO] consul: Adding LAN server dev-consul-server-0 (Addr: tcp/10.242.68.10:8300) (DC: dc1)
2021/06/02 10:40:56 [INFO] serf: EventMemberJoin: dev-consul-server-0.dc1 10.242.68.10
2021/06/02 10:40:56 [INFO] consul: Handled member-join event for server "dev-consul-server-0.dc1" in area "wan"
2021/06/02 10:40:56 [WARN] memberlist: Failed to resolve dev-consul-server-1.consul-server: lookup dev-consul-server-1.consul-server on 10.96.0.10:53: no such host
2021/06/02 10:40:56 [WARN] memberlist: Failed to resolve dev-consul-server-2.consul-server: lookup dev-consul-server-2.consul-server on 10.96.0.10:53: no such host
2021/06/02 10:40:56 [INFO] agent: (LAN) joined: 1 Err:
2021/06/02 10:40:56 [INFO] agent: Join LAN completed. Synced with 1 initial agents
2021/06/02 10:41:03 [WARN] raft: no known peers, aborting election
2021/06/02 10:41:04 [ERR] agent: failed to sync remote state: No cluster leader
==> Failed to check for updates: Get https://checkpoint-api.hashicorp.com/v1/check/consul?arch=amd64&os=linux&signature=c352a57e-764b-41c6-d9e4-a1d2ab8d9ffc&version=1.0.6: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
2021/06/02 10:41:26 [ERR] agent: Coordinate update error: No cluster leader
2021/06/02 10:41:34 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-fpx4p 10.242.120.133
2021/06/02 10:41:35 [ERR] agent: failed to sync remote state: No cluster leader
2021/06/02 10:41:55 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.0.7
2021/06/02 10:41:55 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.0.7:8300) (DC: dc1)
2021/06/02 10:41:55 [INFO] consul: Existing Raft peers reported by dev-consul-server-0, disabling bootstrap mode
2021/06/02 10:41:55 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.0.7
2021/06/02 10:41:55 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/06/02 10:42:02 [DEBUG] raft-net: 10.242.166.6:8300 accepted connection from: 10.242.68.10:42810
2021/06/02 10:42:02 [WARN] raft: Failed to get previous log: 1 log not found (last: 0)
2021/06/02 10:42:02 [INFO] consul: New leader elected: dev-consul-server-0
2021/06/02 10:42:03 [INFO] agent: Synced node info
2021/06/02 10:42:03 [DEBUG] raft-net: 10.242.166.6:8300 accepted connection from: 10.242.68.10:42820
2021/06/02 11:09:13 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:43892->10.242.68.10:8301: i/o timeout
2021/06/02 11:09:13 [INFO] memberlist: Suspect dev-consul-server-0 has failed, no acks received
2021/06/02 11:09:16 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:48372->10.242.68.10:8302: i/o timeout
2021/06/02 11:09:16 [INFO] memberlist: Suspect dev-consul-server-0.dc1 has failed, no acks received
2021/06/02 11:09:16 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:43896->10.242.68.10:8301: i/o timeout
2021/06/02 11:09:16 [INFO] memberlist: Suspect dev-consul-server-0 has failed, no acks received
2021/06/02 11:09:17 [INFO] serf: EventMemberFailed: dev-consul-server-0 10.242.68.10
2021/06/02 11:09:17 [INFO] consul: Removing LAN server dev-consul-server-0 (Addr: tcp/10.242.68.10:8300) (DC: dc1)
2021/06/02 11:09:24 [INFO] serf: EventMemberJoin: dev-consul-server-0 10.242.68.10
2021/06/02 11:09:24 [INFO] consul: Adding LAN server dev-consul-server-0 (Addr: tcp/10.242.68.10:8300) (DC: dc1)
2021/06/02 11:09:31 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:44118->10.242.68.10:8301: i/o timeout
2021/06/02 11:09:31 [INFO] memberlist: Suspect dev-consul-server-0 has failed, no acks received
2021/06/02 11:09:34 [INFO] memberlist: Marking dev-consul-server-0 as failed, suspect timeout reached (2 peer confirmations)
2021/06/02 11:09:34 [INFO] serf: EventMemberFailed: dev-consul-server-0 10.242.68.10
2021/06/02 11:09:34 [INFO] consul: Removing LAN server dev-consul-server-0 (Addr: tcp/10.242.68.10:8300) (DC: dc1)
2021/06/02 11:09:34 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:44152->10.242.68.10:8301: i/o timeout
2021/06/02 11:09:34 [INFO] memberlist: Suspect dev-consul-server-0 has failed, no acks received
2021/06/02 11:09:36 [WARN] raft: Heartbeat timeout from "10.242.68.10:8300" reached, starting election
2021/06/02 11:09:36 [INFO] raft: Node at 10.242.166.6:8300 [Candidate] entering Candidate state in term 3
2021/06/02 11:09:36 [WARN] Unable to get address for server id 7d1eb420-a521-1afe-1e5e-5eb8c91fa030, using fallback address 10.242.68.10:8300: Could not find address for server id 7d1eb420-a521-1afe-1e5e-5eb8c91fa030
2021/06/02 11:09:41 [INFO] serf: EventMemberJoin: dev-consul-server-0 10.242.68.10
2021/06/02 11:09:41 [INFO] consul: Adding LAN server dev-consul-server-0 (Addr: tcp/10.242.68.10:8300) (DC: dc1)
2021/06/02 11:09:41 [WARN] raft: Election timeout reached, restarting election
2021/06/02 11:09:41 [INFO] raft: Node at 10.242.166.6:8300 [Candidate] entering Candidate state in term 4
2021/06/02 11:09:50 [WARN] raft: Election timeout reached, restarting election
2021/06/02 11:09:50 [INFO] raft: Node at 10.242.166.6:8300 [Candidate] entering Candidate state in term 5
2021/06/02 11:09:52 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:44388->10.242.68.10:8301: i/o timeout
2021/06/02 11:09:52 [INFO] memberlist: Suspect dev-consul-server-0 has failed, no acks received
2021/06/02 11:09:54 [DEBUG] raft-net: 10.242.166.6:8300 accepted connection from: 10.242.0.7:40004
2021/06/02 11:09:55 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:44426->10.242.68.10:8301: i/o timeout
2021/06/02 11:09:55 [INFO] memberlist: Suspect dev-consul-server-0 has failed, no acks received
2021/06/02 11:09:55 [INFO] memberlist: Marking dev-consul-server-0 as failed, suspect timeout reached (2 peer confirmations)
2021/06/02 11:09:55 [INFO] serf: EventMemberFailed: dev-consul-server-0 10.242.68.10
2021/06/02 11:09:55 [INFO] consul: Removing LAN server dev-consul-server-0 (Addr: tcp/10.242.68.10:8300) (DC: dc1)
2021/06/02 11:09:56 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:48886->10.242.68.10:8302: i/o timeout
2021/06/02 11:09:56 [INFO] memberlist: Suspect dev-consul-server-0.dc1 has failed, no acks received
2021/06/02 11:09:59 [WARN] raft: Election timeout reached, restarting election
2021/06/02 11:09:59 [INFO] raft: Node at 10.242.166.6:8300 [Candidate] entering Candidate state in term 6
2021/06/02 11:09:59 [WARN] Unable to get address for server id 7d1eb420-a521-1afe-1e5e-5eb8c91fa030, using fallback address 10.242.68.10:8300: Could not find address for server id 7d1eb420-a521-1afe-1e5e-5eb8c91fa030
2021/06/02 11:10:00 [ERR] raft: Failed to make RequestVote RPC to {Voter 7d1eb420-a521-1afe-1e5e-5eb8c91fa030 10.242.68.10:8300}: read tcp 10.242.166.6:48248->10.242.68.10:8300: i/o timeout
2021/06/02 11:10:01 [ERR] agent: failed to sync remote state: No cluster leader
2021/06/02 11:10:01 [INFO] raft: Node at 10.242.166.6:8300 [Follower] entering Follower state (Leader: "")
2021/06/02 11:10:01 [WARN] raft: Failed to get previous log: 392 log not found (last: 391)
2021/06/02 11:10:01 [INFO] consul: New leader elected: dev-consul-server-2
2021/06/02 11:10:02 [DEBUG] raft-net: 10.242.166.6:8300 accepted connection from: 10.242.0.7:40090
2021/06/02 11:10:06 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:10:06 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:49138->10.242.68.10:8302: i/o timeout
2021/06/02 11:10:06 [INFO] memberlist: Suspect dev-consul-server-0.dc1 has failed, no acks received
2021/06/02 11:10:09 [ERR] raft: Failed to make RequestVote RPC to {Voter 7d1eb420-a521-1afe-1e5e-5eb8c91fa030 10.242.68.10:8300}: read tcp 10.242.166.6:48170->10.242.68.10:8300: i/o timeout
2021/06/02 11:10:21 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:49334->10.242.68.10:8302: i/o timeout
2021/06/02 11:10:21 [INFO] memberlist: Suspect dev-consul-server-0.dc1 has failed, no acks received
2021/06/02 11:10:25 [INFO] memberlist: Marking dev-consul-server-0.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/06/02 11:10:25 [INFO] serf: EventMemberFailed: dev-consul-server-0.dc1 10.242.68.10
2021/06/02 11:10:25 [INFO] consul: Handled member-failed event for server "dev-consul-server-0.dc1" in area "wan"
2021/06/02 11:10:31 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:49386->10.242.68.10:8302: i/o timeout
2021/06/02 11:10:31 [INFO] memberlist: Suspect dev-consul-server-0.dc1 has failed, no acks received
2021/06/02 11:10:37 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:10:37 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.68.10:42782
2021/06/02 11:10:56 [INFO] serf: attempting reconnect to dev-consul-server-0.dc1 10.242.68.10:8302
2021/06/02 11:11:26 [INFO] serf: attempting reconnect to dev-consul-server-0 10.242.68.10:8301
2021/06/02 11:11:56 [INFO] serf: attempting reconnect to dev-consul-server-0.dc1 10.242.68.10:8302
2021/06/02 11:12:09 [INFO] serf: EventMemberJoin: dev-consul-server-0 10.242.68.10
2021/06/02 11:12:09 [INFO] consul: Adding LAN server dev-consul-server-0 (Addr: tcp/10.242.68.10:8300) (DC: dc1)
2021/06/02 11:12:09 [INFO] serf: EventMemberJoin: dev-consul-server-0.dc1 10.242.68.10
2021/06/02 11:12:09 [INFO] consul: Handled member-join event for server "dev-consul-server-0.dc1" in area "wan"
2021/06/02 11:12:09 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-0.dc1)
2021/06/02 11:12:10 [WARN] raft: Rejecting vote request from 10.242.68.10:8300 since we have a leader: 10.242.0.7:8300
2021/06/02 11:13:25 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:47252->10.242.68.10:8301: i/o timeout
2021/06/02 11:13:25 [INFO] memberlist: Suspect dev-consul-server-0 has failed, no acks received
2021/06/02 11:13:28 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:47264->10.242.68.10:8301: i/o timeout
2021/06/02 11:13:28 [INFO] memberlist: Suspect dev-consul-server-0 has failed, no acks received
2021/06/02 11:13:29 [INFO] memberlist: Marking dev-consul-server-0 as failed, suspect timeout reached (2 peer confirmations)
2021/06/02 11:13:29 [INFO] serf: EventMemberFailed: dev-consul-server-0 10.242.68.10
2021/06/02 11:13:29 [INFO] consul: Removing LAN server dev-consul-server-0 (Addr: tcp/10.242.68.10:8300) (DC: dc1)
2021/06/02 11:13:36 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:51844->10.242.68.10:8302: i/o timeout
2021/06/02 11:13:36 [INFO] memberlist: Suspect dev-consul-server-0.dc1 has failed, no acks received
2021/06/02 11:13:46 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:51914->10.242.68.10:8302: i/o timeout
2021/06/02 11:13:46 [INFO] memberlist: Suspect dev-consul-server-0.dc1 has failed, no acks received
2021/06/02 11:14:01 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:52154->10.242.68.10:8302: i/o timeout
2021/06/02 11:14:01 [INFO] memberlist: Suspect dev-consul-server-0.dc1 has failed, no acks received
2021/06/02 11:14:05 [INFO] memberlist: Marking dev-consul-server-0.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/06/02 11:14:05 [INFO] serf: EventMemberFailed: dev-consul-server-0.dc1 10.242.68.10
2021/06/02 11:14:05 [INFO] consul: Handled member-failed event for server "dev-consul-server-0.dc1" in area "wan"
2021/06/02 11:14:09 [INFO] serf: attempting reconnect to dev-consul-server-0.dc1 10.242.68.10:8302
2021/06/02 11:16:06 [INFO] serf: attempting reconnect to dev-consul-server-0 10.242.68.10:8301
2021/06/02 11:16:36 [INFO] serf: EventMemberJoin: dev-consul-server-0 10.242.68.10
2021/06/02 11:16:36 [INFO] consul: Adding LAN server dev-consul-server-0 (Addr: tcp/10.242.68.10:8300) (DC: dc1)
2021/06/02 11:16:37 [INFO] serf: EventMemberJoin: dev-consul-server-0.dc1 10.242.68.10
2021/06/02 11:16:37 [INFO] consul: Handled member-join event for server "dev-consul-server-0.dc1" in area "wan"
2021/06/02 11:17:37 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:50520->10.242.68.10:8301: i/o timeout
2021/06/02 11:17:37 [INFO] memberlist: Suspect dev-consul-server-0 has failed, no acks received
2021/06/02 11:17:40 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:50554->10.242.68.10:8301: i/o timeout
2021/06/02 11:17:40 [INFO] memberlist: Suspect dev-consul-server-0 has failed, no acks received
2021/06/02 11:17:41 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:55026->10.242.68.10:8302: i/o timeout
2021/06/02 11:17:41 [INFO] memberlist: Suspect dev-consul-server-0.dc1 has failed, no acks received
2021/06/02 11:17:41 [INFO] serf: EventMemberFailed: dev-consul-server-0 10.242.68.10
2021/06/02 11:17:41 [INFO] consul: Removing LAN server dev-consul-server-0 (Addr: tcp/10.242.68.10:8300) (DC: dc1)
2021/06/02 11:17:46 [INFO] serf: attempting reconnect to dev-consul-server-0 10.242.68.10:8301
2021/06/02 11:17:56 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:55228->10.242.68.10:8302: i/o timeout
2021/06/02 11:17:56 [INFO] memberlist: Suspect dev-consul-server-0.dc1 has failed, no acks received
2021/06/02 11:18:06 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:55290->10.242.68.10:8302: i/o timeout
2021/06/02 11:18:06 [INFO] memberlist: Suspect dev-consul-server-0.dc1 has failed, no acks received
2021/06/02 11:18:11 [INFO] memberlist: Marking dev-consul-server-0.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/06/02 11:18:11 [INFO] serf: EventMemberFailed: dev-consul-server-0.dc1 10.242.68.10
2021/06/02 11:18:11 [INFO] consul: Handled member-failed event for server "dev-consul-server-0.dc1" in area "wan"
2021/06/02 11:18:17 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:18:39 [INFO] serf: attempting reconnect to dev-consul-server-0.dc1 10.242.68.10:8302
2021/06/02 11:18:56 [INFO] serf: attempting reconnect to dev-consul-server-0 10.242.68.10:8301
2021/06/02 11:19:36 [INFO] serf: attempting reconnect to dev-consul-server-0 10.242.68.10:8301
2021/06/02 11:19:39 [INFO] serf: attempting reconnect to dev-consul-server-0.dc1 10.242.68.10:8302
2021/06/02 11:20:46 [INFO] serf: attempting reconnect to dev-consul-server-0 10.242.68.10:8301
2021/06/02 11:22:26 [INFO] serf: attempting reconnect to dev-consul-server-0 10.242.68.10:8301
2021/06/02 11:22:39 [INFO] serf: attempting reconnect to dev-consul-server-0.dc1 10.242.68.10:8302
2021/06/02 11:23:06 [INFO] serf: attempting reconnect to dev-consul-server-0 10.242.68.10:8301
2021/06/02 11:23:22 [INFO] serf: EventMemberJoin: dev-consul-server-0 10.242.68.10
2021/06/02 11:23:22 [INFO] consul: Adding LAN server dev-consul-server-0 (Addr: tcp/10.242.68.10:8300) (DC: dc1)
2021/06/02 11:23:23 [INFO] serf: EventMemberJoin: dev-consul-server-0.dc1 10.242.68.10
2021/06/02 11:23:23 [INFO] consul: Handled member-join event for server "dev-consul-server-0.dc1" in area "wan"
2021/06/02 11:23:29 [INFO] memberlist: Suspect dev-consul-server-0 has failed, no acks received
2021/06/02 11:23:31 [INFO] memberlist: Suspect dev-consul-server-0 has failed, no acks received
2021/06/02 11:23:33 [INFO] serf: EventMemberFailed: dev-consul-server-0 10.242.68.10
2021/06/02 11:23:33 [INFO] consul: Removing LAN server dev-consul-server-0 (Addr: tcp/10.242.68.10:8300) (DC: dc1)
2021/06/02 11:23:43 [INFO] memberlist: Suspect dev-consul-server-0 has failed, no acks received
2021/06/02 11:23:53 [ERR] memberlist: Conflicting address for dev-consul-server-0. Mine: 10.242.68.10:8301 Theirs: 10.242.68.40:8301
2021/06/02 11:23:53 [WARN] serf: Name conflict for 'dev-consul-server-0' both 10.242.68.10:8301 and 10.242.68.40:8301 are claiming
2021/06/02 11:23:53 [ERR] memberlist: Conflicting address for dev-consul-server-0.dc1. Mine: 10.242.68.10:8302 Theirs: 10.242.68.40:8302
2021/06/02 11:23:53 [WARN] serf: Name conflict for 'dev-consul-server-0.dc1' both 10.242.68.10:8302 and 10.242.68.40:8302 are claiming
2021/06/02 11:23:53 [ERR] memberlist: Conflicting address for dev-consul-server-0. Mine: 10.242.68.10:8301 Theirs: 10.242.68.40:8301
2021/06/02 11:23:53 [WARN] serf: Name conflict for 'dev-consul-server-0' both 10.242.68.10:8301 and 10.242.68.40:8301 are claiming
2021/06/02 11:23:53 [ERR] memberlist: Conflicting address for dev-consul-server-0. Mine: 10.242.68.10:8301 Theirs: 10.242.68.40:8301
2021/06/02 11:23:53 [WARN] serf: Name conflict for 'dev-consul-server-0' both 10.242.68.10:8301 and 10.242.68.40:8301 are claiming
2021/06/02 11:23:53 [ERR] memberlist: Conflicting address for dev-consul-server-0.dc1. Mine: 10.242.68.10:8302 Theirs: 10.242.68.40:8302
2021/06/02 11:23:53 [WARN] serf: Name conflict for 'dev-consul-server-0.dc1' both 10.242.68.10:8302 and 10.242.68.40:8302 are claiming
2021/06/02 11:23:54 [ERR] memberlist: Conflicting address for dev-consul-server-0.dc1. Mine: 10.242.68.10:8302 Theirs: 10.242.68.40:8302
2021/06/02 11:23:54 [WARN] serf: Name conflict for 'dev-consul-server-0.dc1' both 10.242.68.10:8302 and 10.242.68.40:8302 are claiming
2021/06/02 11:24:04 [INFO] memberlist: Suspect dev-consul-server-0.dc1 has failed, no acks received
2021/06/02 11:24:33 [INFO] memberlist: Marking dev-consul-server-0.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/06/02 11:24:33 [INFO] serf: EventMemberFailed: dev-consul-server-0.dc1 10.242.68.10
2021/06/02 11:24:33 [INFO] consul: Handled member-failed event for server "dev-consul-server-0.dc1" in area "wan"
2021/06/02 11:24:33 [INFO] serf: EventMemberJoin: dev-consul-server-0 10.242.68.40
2021/06/02 11:24:33 [INFO] consul: Adding LAN server dev-consul-server-0 (Addr: tcp/10.242.68.40:8300) (DC: dc1)
2021/06/02 11:24:39 [INFO] serf: attempting reconnect to dev-consul-server-0.dc1 10.242.68.10:8302
2021/06/02 11:24:44 [INFO] memberlist: Suspect dev-consul-server-0.dc1 has failed, no acks received
2021/06/02 11:24:57 [ERR] memberlist: Conflicting address for dev-consul-server-0.dc1. Mine: 10.242.68.10:8302 Theirs: 10.242.68.40:8302
2021/06/02 11:24:57 [WARN] serf: Name conflict for 'dev-consul-server-0.dc1' both 10.242.68.10:8302 and 10.242.68.40:8302 are claiming
2021/06/02 11:24:57 [ERR] memberlist: Conflicting address for dev-consul-server-0.dc1. Mine: 10.242.68.10:8302 Theirs: 10.242.68.40:8302
2021/06/02 11:24:57 [WARN] serf: Name conflict for 'dev-consul-server-0.dc1' both 10.242.68.10:8302 and 10.242.68.40:8302 are claiming
2021/06/02 11:24:57 [ERR] memberlist: Conflicting address for dev-consul-server-0.dc1. Mine: 10.242.68.10:8302 Theirs: 10.242.68.40:8302
2021/06/02 11:24:57 [WARN] serf: Name conflict for 'dev-consul-server-0.dc1' both 10.242.68.10:8302 and 10.242.68.40:8302 are claiming
2021/06/02 11:25:57 [INFO] serf: EventMemberJoin: dev-consul-server-0.dc1 10.242.68.40
2021/06/02 11:25:57 [INFO] consul: Handled member-join event for server "dev-consul-server-0.dc1" in area "wan"
2021/06/02 11:26:07 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:54806->10.242.0.7:8301: i/o timeout
2021/06/02 11:26:07 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/06/02 11:26:10 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:54842->10.242.0.7:8301: i/o timeout
2021/06/02 11:26:10 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/06/02 11:26:11 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/06/02 11:26:11 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.0.7
2021/06/02 11:26:11 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.0.7:8300) (DC: dc1)
2021/06/02 11:26:16 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.0.7
2021/06/02 11:26:16 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.0.7:8300) (DC: dc1)
2021/06/02 11:26:17 [DEBUG] raft-net: 10.242.166.6:8300 accepted connection from: 10.242.68.40:38420
2021/06/02 11:26:17 [WARN] raft: Rejecting vote request from 10.242.68.40:8300 since we have a leader: 10.242.0.7:8300
2021/06/02 11:26:18 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:54960->10.242.0.7:8301: i/o timeout
2021/06/02 11:26:18 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/06/02 11:26:19 [WARN] raft: Heartbeat timeout from "10.242.0.7:8300" reached, starting election
2021/06/02 11:26:19 [INFO] raft: Node at 10.242.166.6:8300 [Candidate] entering Candidate state in term 8
2021/06/02 11:26:24 [INFO] raft: Election won. Tally: 2
2021/06/02 11:26:24 [INFO] raft: Node at 10.242.166.6:8300 [Leader] entering Leader state
2021/06/02 11:26:24 [INFO] consul: cluster leadership acquired
2021/06/02 11:26:24 [INFO] raft: Added peer 60597ff4-43fd-637b-127d-664e51804d7f, starting replication
2021/06/02 11:26:24 [INFO] raft: Added peer 54b018c9-9a7d-17e5-0fad-ecc6749d17b1, starting replication
2021/06/02 11:26:24 [INFO] consul: New leader elected: dev-consul-server-1
2021/06/02 11:26:24 [INFO] raft: pipelining replication to peer {Voter 54b018c9-9a7d-17e5-0fad-ecc6749d17b1 10.242.68.40:8300}
2021/06/02 11:26:24 [INFO] raft: pipelining replication to peer {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}
2021/06/02 11:26:27 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:28:11 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:28:15 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:56570->10.242.0.7:8301: i/o timeout
2021/06/02 11:28:15 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/06/02 11:28:19 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:28:19 [WARN] consul: error getting server health from "dev-consul-server-1": context deadline exceeded
2021/06/02 11:28:19 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/06/02 11:28:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:28:20 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:56642->10.242.0.7:8301: i/o timeout
2021/06/02 11:28:20 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/06/02 11:28:20 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 2.500253002s
2021/06/02 11:28:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:28:23 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 4.962886893s
2021/06/02 11:28:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:28:24 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.0.7
2021/06/02 11:28:24 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.0.7:8300) (DC: dc1)
2021/06/02 11:28:24 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/06/02 11:28:25 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 7.410125498s
2021/06/02 11:28:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:28:26 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:56330->10.242.0.7:8302: i/o timeout
2021/06/02 11:28:26 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 11:28:27 [INFO] raft: aborting pipeline replication to peer {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}
2021/06/02 11:28:27 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:28:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:28:29 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:41506->10.242.0.7:8300: i/o timeout
2021/06/02 11:28:29 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 2.501055344s
2021/06/02 11:28:30 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:28:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:28:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:28:32 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 4.925895105s
2021/06/02 11:28:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:28:34 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 7.3927346s
2021/06/02 11:28:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:28:36 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:56458->10.242.0.7:8302: i/o timeout
2021/06/02 11:28:36 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 11:28:37 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:43204->10.242.0.7:8300: i/o timeout
2021/06/02 11:28:37 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:28:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:28:40 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:43238->10.242.0.7:8300: i/o timeout
2021/06/02 11:28:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:28:40 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:28:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:28:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:28:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:28:46 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:56582->10.242.0.7:8302: i/o timeout
2021/06/02 11:28:46 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 11:28:47 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:43330->10.242.0.7:8300: i/o timeout
2021/06/02 11:28:47 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:28:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:28:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:28:50 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:43372->10.242.0.7:8300: i/o timeout
2021/06/02 11:28:51 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:28:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:28:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:28:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:28:56 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/06/02 11:28:56 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.0.7
2021/06/02 11:28:56 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/06/02 11:28:57 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:43450->10.242.0.7:8300: i/o timeout
2021/06/02 11:28:57 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:28:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:01 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:43508->10.242.0.7:8300: i/o timeout
2021/06/02 11:29:01 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:56770->10.242.0.7:8302: i/o timeout
2021/06/02 11:29:01 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 11:29:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:02 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:29:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:07 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:43572->10.242.0.7:8300: i/o timeout
2021/06/02 11:29:07 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:29:08 [ERR] memberlist: failed to receive: read tcp 10.242.166.6:8302->10.242.0.7:41544: i/o timeout from=10.242.0.7:41544
2021/06/02 11:29:08 [ERR] memberlist: Failed to send error: write tcp 10.242.166.6:8302->10.242.0.7:41544: i/o timeout from=10.242.0.7:41544
2021/06/02 11:29:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:09 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 11:29:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:12 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:43640->10.242.0.7:8300: i/o timeout
2021/06/02 11:29:12 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:29:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:17 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:43700->10.242.0.7:8300: i/o timeout
2021/06/02 11:29:17 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:29:18 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:29:18 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.0.7:40114
2021/06/02 11:29:18 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:29:18 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:29:19 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:29:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:22 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:43764->10.242.0.7:8300: i/o timeout
2021/06/02 11:29:23 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:29:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:27 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:43820->10.242.0.7:8300: i/o timeout
2021/06/02 11:29:27 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:29:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:33 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:43906->10.242.0.7:8300: i/o timeout
2021/06/02 11:29:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:34 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:29:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:37 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:43954->10.242.0.7:8300: i/o timeout
2021/06/02 11:29:38 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:29:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:44 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:44036->10.242.0.7:8300: i/o timeout
2021/06/02 11:29:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:46 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:29:48 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:44084->10.242.0.7:8300: i/o timeout
2021/06/02 11:29:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:48 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:29:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:56 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:44172->10.242.0.7:8300: i/o timeout
2021/06/02 11:29:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:29:58 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:29:58 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:29:58 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:44212->10.242.0.7:8300: i/o timeout
2021/06/02 11:29:58 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:29:59 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:30:01 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:30:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:08 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:44330->10.242.0.7:8300: i/o timeout
2021/06/02 11:30:09 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:44344->10.242.0.7:8300: i/o timeout
2021/06/02 11:30:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:12 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:30:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:12 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:30:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:16 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.0.7:8301
2021/06/02 11:30:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:22 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:44492->10.242.0.7:8300: i/o timeout
2021/06/02 11:30:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:22 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:44494->10.242.0.7:8300: i/o timeout
2021/06/02 11:30:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:27 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:30:27 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:30:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:37 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:44690->10.242.0.7:8300: i/o timeout
2021/06/02 11:30:37 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:44694->10.242.0.7:8300: i/o timeout
2021/06/02 11:30:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:40 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:30:40 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:30:43 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:30:43 [WARN] consul: error getting server health from "dev-consul-server-1": context deadline exceeded
2021/06/02 11:30:43 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/06/02 11:30:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:47 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:30:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:48 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:30:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:57 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:44984->10.242.0.7:8300: i/o timeout
2021/06/02 11:30:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:30:58 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:44998->10.242.0.7:8300: i/o timeout
2021/06/02 11:31:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:08 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:31:09 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 11:31:09 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:31:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:18 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:45330->10.242.0.7:8300: i/o timeout
2021/06/02 11:31:19 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:45346->10.242.0.7:8300: i/o timeout
2021/06/02 11:31:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:22 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:31:22 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:31:25 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:31:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:26 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.0.7:8301
2021/06/02 11:31:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:28 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:31:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:30 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:31:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:38 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:45586->10.242.0.7:8300: i/o timeout
2021/06/02 11:31:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:40 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:45616->10.242.0.7:8300: i/o timeout
2021/06/02 11:31:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:48 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:31:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:52 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:31:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:31:58 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:45830->10.242.0.7:8300: i/o timeout
2021/06/02 11:32:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:02 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:45872->10.242.0.7:8300: i/o timeout
2021/06/02 11:32:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:04 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:32:04 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:32:06 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.0.7:8301
2021/06/02 11:32:07 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:32:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:08 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:32:09 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 11:32:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:13 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:32:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:18 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:46080->10.242.0.7:8300: i/o timeout
2021/06/02 11:32:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:23 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:46136->10.242.0.7:8300: i/o timeout
2021/06/02 11:32:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:29 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:32:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:33 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:32:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:39 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:46350->10.242.0.7:8300: i/o timeout
2021/06/02 11:32:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:43 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:46404->10.242.0.7:8300: i/o timeout
2021/06/02 11:32:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:46 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:32:46 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:32:49 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:32:49 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:32:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:54 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:32:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:32:59 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:46594->10.242.0.7:8300: i/o timeout
2021/06/02 11:33:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:33:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:33:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:33:04 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:46656->10.242.0.7:8300: i/o timeout
2021/06/02 11:33:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:33:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:33:09 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 11:33:09 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:33:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:33:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:33:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:33:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:33:16 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:33:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:33:19 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:46842->10.242.0.7:8300: i/o timeout
2021/06/02 11:33:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:33:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:33:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:33:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:33:26 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:46920->10.242.0.7:8300: i/o timeout
2021/06/02 11:33:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:33:28 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:33:28 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:33:28 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.0.7
2021/06/02 11:33:28 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/06/02 11:33:28 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.0.7
2021/06/02 11:33:28 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.0.7:8300) (DC: dc1)
2021/06/02 11:33:28 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/06/02 11:33:31 [INFO] raft: pipelining replication to peer {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}
2021/06/02 11:34:25 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:35:25 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:35:45 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:35:45 [WARN] consul: error getting server health from "dev-consul-server-1": context deadline exceeded
2021/06/02 11:35:45 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/06/02 11:35:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:36:19 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:37:09 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:37:15 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:37:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:37:41 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:37:41 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/06/02 11:37:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:37:43 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 2.500160802s
2021/06/02 11:37:43 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:36284->10.242.0.7:8301: i/o timeout
2021/06/02 11:37:43 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/06/02 11:37:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:37:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:37:47 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/06/02 11:37:47 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.0.7
2021/06/02 11:37:47 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.0.7:8300) (DC: dc1)
2021/06/02 11:37:47 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/06/02 11:37:47 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:36328->10.242.0.7:8301: i/o timeout
2021/06/02 11:37:47 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/06/02 11:37:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:37:48 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:37:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:37:50 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 2.50016892s
2021/06/02 11:37:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:37:52 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 4.957120287s
2021/06/02 11:37:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:37:55 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 7.42168665s
2021/06/02 11:37:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:37:57 [INFO] raft: aborting pipeline replication to peer {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}
2021/06/02 11:37:57 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:37:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:37:58 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:47442->10.242.0.7:8300: i/o timeout
2021/06/02 11:37:59 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:38:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:38:00 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 2.500211902s
2021/06/02 11:38:01 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:36126->10.242.0.7:8302: i/o timeout
2021/06/02 11:38:01 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 11:38:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:38:02 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 4.913052412s
2021/06/02 11:38:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:38:05 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 7.409062716s
2021/06/02 11:38:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:38:07 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:51166->10.242.0.7:8300: i/o timeout
2021/06/02 11:38:07 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:38:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:38:08 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:38:08 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.0.7:57660
2021/06/02 11:38:09 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:51184->10.242.0.7:8300: i/o timeout
2021/06/02 11:38:09 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:38:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:38:10 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:38:10 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:38:11 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:36256->10.242.0.7:8302: i/o timeout
2021/06/02 11:38:11 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 11:38:13 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:38:13 [WARN] consul: error getting server health from "dev-consul-server-1": context deadline exceeded
2021/06/02 11:38:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:38:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:38:16 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.0.7:8301
2021/06/02 11:38:17 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:51300->10.242.0.7:8300: i/o timeout
2021/06/02 11:38:17 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:38:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:38:19 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:51326->10.242.0.7:8300: i/o timeout
2021/06/02 11:38:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:38:20 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:38:21 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:36394->10.242.0.7:8302: i/o timeout
2021/06/02 11:38:21 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 11:38:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:38:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:38:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:38:27 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:51440->10.242.0.7:8300: i/o timeout
2021/06/02 11:38:28 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:38:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:38:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:38:30 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:51472->10.242.0.7:8300: i/o timeout
2021/06/02 11:38:31 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:38:31 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/06/02 11:38:31 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.0.7
2021/06/02 11:38:31 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/06/02 11:38:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:38:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:38:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:38:36 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:36582->10.242.0.7:8302: i/o timeout
2021/06/02 11:38:36 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 11:38:38 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:51570->10.242.0.7:8300: i/o timeout
2021/06/02 11:38:38 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:38:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:38:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:38:41 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:51622->10.242.0.7:8300: i/o timeout
2021/06/02 11:38:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:38:42 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:38:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:38:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:38:46 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.0.7
2021/06/02 11:38:46 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.0.7:8300) (DC: dc1)
2021/06/02 11:38:46 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/06/02 11:38:46 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.0.7
2021/06/02 11:38:46 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/06/02 11:38:48 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:51700->10.242.0.7:8300: i/o timeout
2021/06/02 11:38:48 [INFO] raft: pipelining replication to peer {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}
2021/06/02 11:41:07 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:41:07 [WARN] consul: error getting server health from "dev-consul-server-1": context deadline exceeded
2021/06/02 11:41:07 [WARN] memberlist: Was able to connect to dev-consul-server-2 but other probes failed, network may be misconfigured
2021/06/02 11:41:08 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:39140->10.242.0.7:8301: i/o timeout
2021/06/02 11:41:08 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/06/02 11:41:09 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/06/02 11:41:09 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.0.7
2021/06/02 11:41:09 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.0.7:8300) (DC: dc1)
2021/06/02 11:41:09 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/06/02 11:41:10 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 2.50022234s
2021/06/02 11:41:11 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:41:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:41:13 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 5.000126375s
2021/06/02 11:41:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:41:15 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 7.478711389s
2021/06/02 11:41:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:41:16 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:38850->10.242.0.7:8302: i/o timeout
2021/06/02 11:41:16 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 11:41:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:41:18 [INFO] raft: aborting pipeline replication to peer {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}
2021/06/02 11:41:18 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:41:19 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:51860->10.242.0.7:8300: i/o timeout
2021/06/02 11:41:19 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:41:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:41:20 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 2.500188362s
2021/06/02 11:41:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:41:23 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 4.995923254s
2021/06/02 11:41:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:41:25 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 7.448319986s
2021/06/02 11:41:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:41:26 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:41:26 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:41:26 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:41:26 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.0.7:35490
2021/06/02 11:41:26 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:38926->10.242.0.7:8302: i/o timeout
2021/06/02 11:41:26 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 11:41:26 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.0.7:8301
2021/06/02 11:41:28 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:53980->10.242.0.7:8300: i/o timeout
2021/06/02 11:41:28 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:41:29 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:41:29 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:53990->10.242.0.7:8300: i/o timeout
2021/06/02 11:41:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:41:30 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:41:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:41:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:41:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:41:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:41:38 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:54112->10.242.0.7:8300: i/o timeout
2021/06/02 11:41:38 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:41:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:41:40 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:54136->10.242.0.7:8300: i/o timeout
2021/06/02 11:41:41 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:41:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:41:43 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/06/02 11:41:43 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.0.7
2021/06/02 11:41:43 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/06/02 11:41:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:41:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:41:46 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:39244->10.242.0.7:8302: i/o timeout
2021/06/02 11:41:46 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 11:41:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:41:48 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:54238->10.242.0.7:8300: i/o timeout
2021/06/02 11:41:48 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:41:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:41:51 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:54266->10.242.0.7:8300: i/o timeout
2021/06/02 11:41:51 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:41:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:41:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:41:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:41:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:41:58 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 11:41:58 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:54368->10.242.0.7:8300: i/o timeout
2021/06/02 11:41:58 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:42:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:01 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:54414->10.242.0.7:8300: i/o timeout
2021/06/02 11:42:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:03 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:42:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:08 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:42:08 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:42:08 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:54496->10.242.0.7:8300: i/o timeout
2021/06/02 11:42:08 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:42:11 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:42:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:13 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:54548->10.242.0.7:8300: i/o timeout
2021/06/02 11:42:13 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:42:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:18 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:54618->10.242.0.7:8300: i/o timeout
2021/06/02 11:42:18 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:42:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:23 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:54682->10.242.0.7:8300: i/o timeout
2021/06/02 11:42:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:24 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:42:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:28 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:54758->10.242.0.7:8300: i/o timeout
2021/06/02 11:42:29 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:42:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:34 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:54834->10.242.0.7:8300: i/o timeout
2021/06/02 11:42:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:36 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:42:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:39 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:54888->10.242.0.7:8300: i/o timeout
2021/06/02 11:42:39 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:42:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:46 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:54976->10.242.0.7:8300: i/o timeout
2021/06/02 11:42:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:48 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:42:49 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:55022->10.242.0.7:8300: i/o timeout
2021/06/02 11:42:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:50 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:42:50 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:42:51 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:42:53 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:42:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:42:58 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:55136->10.242.0.7:8300: i/o timeout
2021/06/02 11:43:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:01 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:55166->10.242.0.7:8300: i/o timeout
2021/06/02 11:43:01 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:43:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:03 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:43:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:06 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.0.7:8301
2021/06/02 11:43:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:11 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:55402->10.242.0.7:8300: i/o timeout
2021/06/02 11:43:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:13 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:55424->10.242.0.7:8300: i/o timeout
2021/06/02 11:43:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:17 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:43:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:18 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:43:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:27 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:55608->10.242.0.7:8300: i/o timeout
2021/06/02 11:43:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:28 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:55626->10.242.0.7:8300: i/o timeout
2021/06/02 11:43:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:32 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:43:32 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:43:35 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:43:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:38 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:43:39 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:43:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:48 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:55876->10.242.0.7:8300: i/o timeout
2021/06/02 11:43:49 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:55880->10.242.0.7:8300: i/o timeout
2021/06/02 11:43:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:43:58 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 11:43:59 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:43:59 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:44:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:09 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:56130->10.242.0.7:8300: i/o timeout
2021/06/02 11:44:09 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:56140->10.242.0.7:8300: i/o timeout
2021/06/02 11:44:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:14 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:44:14 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:44:17 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:44:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:19 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:44:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:20 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:44:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:29 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:56392->10.242.0.7:8300: i/o timeout
2021/06/02 11:44:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:30 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:56408->10.242.0.7:8300: i/o timeout
2021/06/02 11:44:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:39 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:44:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:41 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:44:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:46 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.0.7:8301
2021/06/02 11:44:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:49 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:56642->10.242.0.7:8300: i/o timeout
2021/06/02 11:44:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:51 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:56672->10.242.0.7:8300: i/o timeout
2021/06/02 11:44:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:44:56 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:44:56 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:44:59 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:44:59 [WARN] consul: error getting server health from "dev-consul-server-1": context deadline exceeded
2021/06/02 11:44:59 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/06/02 11:45:00 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:45:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:02 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:45:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:10 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:56898->10.242.0.7:8300: i/o timeout
2021/06/02 11:45:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:12 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:56926->10.242.0.7:8300: i/o timeout
2021/06/02 11:45:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:20 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:45:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:23 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:45:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:30 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:57152->10.242.0.7:8300: i/o timeout
2021/06/02 11:45:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:33 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:57198->10.242.0.7:8300: i/o timeout
2021/06/02 11:45:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:38 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:45:38 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:45:40 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:45:41 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:45:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:44 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:45:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:50 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:57406->10.242.0.7:8300: i/o timeout
2021/06/02 11:45:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:54 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:57450->10.242.0.7:8300: i/o timeout
2021/06/02 11:45:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:56 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.0.7:8301
2021/06/02 11:45:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:45:58 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 11:46:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:46:00 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:46:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:46:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:46:05 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:46:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:46:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:46:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:46:10 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:57768->10.242.0.7:8300: i/o timeout
2021/06/02 11:46:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:46:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:46:15 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:57828->10.242.0.7:8300: i/o timeout
2021/06/02 11:46:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:46:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:46:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:46:20 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:46:20 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:46:20 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.0.7
2021/06/02 11:46:20 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.0.7:8300) (DC: dc1)
2021/06/02 11:46:20 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/06/02 11:46:20 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.0.7
2021/06/02 11:46:20 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/06/02 11:46:20 [WARN] raft: Rejecting vote request from 10.242.0.7:8300 since we have a leader: 10.242.166.6:8300
2021/06/02 11:46:21 [INFO] raft: pipelining replication to peer {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}
2021/06/02 11:47:51 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:47:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:47:53 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:44780->10.242.0.7:8301: i/o timeout
2021/06/02 11:47:53 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/06/02 11:47:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:47:55 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/06/02 11:47:55 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.0.7
2021/06/02 11:47:55 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.0.7:8300) (DC: dc1)
2021/06/02 11:47:55 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/06/02 11:47:55 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:47:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:47:56 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:47:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:47:58 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 2.500279982s
2021/06/02 11:48:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:01 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 4.995360497s
2021/06/02 11:48:01 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:44494->10.242.0.7:8302: i/o timeout
2021/06/02 11:48:01 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 11:48:02 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:48:02 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:48:03 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:48:03 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 7.473254141s
2021/06/02 11:48:04 [INFO] raft: aborting pipeline replication to peer {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}
2021/06/02 11:48:04 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:48:05 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:48:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:06 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:58292->10.242.0.7:8300: i/o timeout
2021/06/02 11:48:07 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 2.50018046s
2021/06/02 11:48:07 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:48:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:09 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 4.985937866s
2021/06/02 11:48:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:11 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:44562->10.242.0.7:8302: i/o timeout
2021/06/02 11:48:11 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 11:48:11 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 7.442516308s
2021/06/02 11:48:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:14 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:59624->10.242.0.7:8300: i/o timeout
2021/06/02 11:48:14 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:48:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:17 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:59668->10.242.0.7:8300: i/o timeout
2021/06/02 11:48:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:18 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:48:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:24 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:59758->10.242.0.7:8300: i/o timeout
2021/06/02 11:48:24 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:48:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:26 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:44838->10.242.0.7:8302: i/o timeout
2021/06/02 11:48:26 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 11:48:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:28 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:59822->10.242.0.7:8300: i/o timeout
2021/06/02 11:48:28 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:48:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:31 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:48:31 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.0.7:40608
2021/06/02 11:48:31 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/06/02 11:48:31 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.0.7
2021/06/02 11:48:31 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/06/02 11:48:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:34 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:59900->10.242.0.7:8300: i/o timeout
2021/06/02 11:48:34 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:48:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:38 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:59984->10.242.0.7:8300: i/o timeout
2021/06/02 11:48:39 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:48:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:44 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:48:44 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:48:44 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:60056->10.242.0.7:8300: i/o timeout
2021/06/02 11:48:44 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:48:47 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:48:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:49 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:60124->10.242.0.7:8300: i/o timeout
2021/06/02 11:48:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:50 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 11:48:50 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:48:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:54 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:60180->10.242.0.7:8300: i/o timeout
2021/06/02 11:48:54 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:48:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:48:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:00 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:60272->10.242.0.7:8300: i/o timeout
2021/06/02 11:49:01 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:49:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:04 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:60312->10.242.0.7:8300: i/o timeout
2021/06/02 11:49:05 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:49:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:06 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.0.7:8301
2021/06/02 11:49:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:11 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:60508->10.242.0.7:8300: i/o timeout
2021/06/02 11:49:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:12 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:49:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:15 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:60540->10.242.0.7:8300: i/o timeout
2021/06/02 11:49:15 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:49:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:22 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:60638->10.242.0.7:8300: i/o timeout
2021/06/02 11:49:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:24 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:49:25 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:60676->10.242.0.7:8300: i/o timeout
2021/06/02 11:49:25 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:49:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:26 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:49:26 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:49:29 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:49:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:34 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:60790->10.242.0.7:8300: i/o timeout
2021/06/02 11:49:35 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:60826->10.242.0.7:8300: i/o timeout
2021/06/02 11:49:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:36 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:49:37 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:49:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:46 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:60960->10.242.0.7:8300: i/o timeout
2021/06/02 11:49:47 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:60972->10.242.0.7:8300: i/o timeout
2021/06/02 11:49:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:49 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:49:49 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:49:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:50 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 11:49:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:49:59 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:32898->10.242.0.7:8300: i/o timeout
2021/06/02 11:49:59 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:32902->10.242.0.7:8300: i/o timeout
2021/06/02 11:50:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:04 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:50:05 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:50:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:08 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:50:08 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:50:11 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:50:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:14 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:33088->10.242.0.7:8300: i/o timeout
2021/06/02 11:50:15 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:33096->10.242.0.7:8300: i/o timeout
2021/06/02 11:50:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:25 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:50:26 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:50:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:35 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:33346->10.242.0.7:8300: i/o timeout
2021/06/02 11:50:36 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:33354->10.242.0.7:8300: i/o timeout
2021/06/02 11:50:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:45 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:50:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:47 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:50:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:50 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:50:50 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:50:50 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 11:50:53 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:50:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:55 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:33606->10.242.0.7:8300: i/o timeout
2021/06/02 11:50:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:50:57 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:33620->10.242.0.7:8300: i/o timeout
2021/06/02 11:50:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:05 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:51:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:08 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:51:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:15 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:33852->10.242.0.7:8300: i/o timeout
2021/06/02 11:51:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:18 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:33890->10.242.0.7:8300: i/o timeout
2021/06/02 11:51:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:25 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:51:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:29 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:51:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:32 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:51:32 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:51:33 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:51:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:35 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:34106->10.242.0.7:8300: i/o timeout
2021/06/02 11:51:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:39 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:34152->10.242.0.7:8300: i/o timeout
2021/06/02 11:51:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:46 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:51:46 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.0.7:8301
2021/06/02 11:51:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:50 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:51:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:56 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:34360->10.242.0.7:8300: i/o timeout
2021/06/02 11:51:57 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.0.7
2021/06/02 11:51:57 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/06/02 11:51:57 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.0.7
2021/06/02 11:51:57 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.0.7:8300) (DC: dc1)
2021/06/02 11:51:57 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/06/02 11:51:58 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2)
2021/06/02 11:51:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:51:59 [WARN] raft: Rejecting vote request from 10.242.0.7:8300 since we have a leader: 10.242.166.6:8300
2021/06/02 11:52:06 [INFO] raft: pipelining replication to peer {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}
2021/06/02 11:53:21 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:53:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:53:22 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 2.500366475s
2021/06/02 11:53:23 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:49444->10.242.0.7:8301: i/o timeout
2021/06/02 11:53:23 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/06/02 11:53:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:53:25 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 4.944427993s
2021/06/02 11:53:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:53:26 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:49474->10.242.0.7:8301: i/o timeout
2021/06/02 11:53:26 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/06/02 11:53:27 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/06/02 11:53:27 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.0.7
2021/06/02 11:53:27 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.0.7:8300) (DC: dc1)
2021/06/02 11:53:27 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/06/02 11:53:27 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 7.435238128s
2021/06/02 11:53:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:53:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:53:30 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:34856->10.242.0.7:8300: i/o timeout
2021/06/02 11:53:30 [INFO] raft: aborting pipeline replication to peer {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}
2021/06/02 11:53:30 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:53:31 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:53:31 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:49162->10.242.0.7:8302: i/o timeout
2021/06/02 11:53:31 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 11:53:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:53:32 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 2.500332111s
2021/06/02 11:53:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:53:35 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 4.94363832s
2021/06/02 11:53:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:53:37 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 7.401162579s
2021/06/02 11:53:37 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:53:37 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.0.7:43572
2021/06/02 11:53:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:53:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:53:40 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:36006->10.242.0.7:8300: i/o timeout
2021/06/02 11:53:40 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:53:41 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:36018->10.242.0.7:8300: i/o timeout
2021/06/02 11:53:41 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:49294->10.242.0.7:8302: i/o timeout
2021/06/02 11:53:41 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 11:53:42 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:53:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:53:42 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:53:42 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:53:45 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:53:45 [WARN] consul: error getting server health from "dev-consul-server-1": context deadline exceeded
2021/06/02 11:53:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:53:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:53:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:53:50 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:36138->10.242.0.7:8300: i/o timeout
2021/06/02 11:53:50 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:53:51 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:49428->10.242.0.7:8302: i/o timeout
2021/06/02 11:53:51 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 11:53:52 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:36170->10.242.0.7:8300: i/o timeout
2021/06/02 11:53:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:53:52 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:53:52 [ERR] memberlist: Push/Pull with dev-consul-server-2.dc1 failed: read tcp 10.242.166.6:49066->10.242.0.7:8302: i/o timeout
2021/06/02 11:53:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:53:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:53:56 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.0.7:8301
2021/06/02 11:53:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:00 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:36272->10.242.0.7:8300: i/o timeout
2021/06/02 11:54:00 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:54:01 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/06/02 11:54:01 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:49496->10.242.0.7:8302: i/o timeout
2021/06/02 11:54:01 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 11:54:01 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.0.7
2021/06/02 11:54:01 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/06/02 11:54:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:02 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:36308->10.242.0.7:8300: i/o timeout
2021/06/02 11:54:03 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:54:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:10 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:36410->10.242.0.7:8300: i/o timeout
2021/06/02 11:54:10 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:54:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:13 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:36454->10.242.0.7:8300: i/o timeout
2021/06/02 11:54:13 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:54:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:20 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:36542->10.242.0.7:8300: i/o timeout
2021/06/02 11:54:20 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:54:20 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 11:54:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:23 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:36586->10.242.0.7:8300: i/o timeout
2021/06/02 11:54:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:24 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:54:24 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:54:24 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:54:27 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:54:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:30 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:36678->10.242.0.7:8300: i/o timeout
2021/06/02 11:54:30 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:54:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:34 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:36728->10.242.0.7:8300: i/o timeout
2021/06/02 11:54:35 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:54:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:36 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.0.7:8301
2021/06/02 11:54:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:40 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:36814->10.242.0.7:8300: i/o timeout
2021/06/02 11:54:41 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:54:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:45 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:36880->10.242.0.7:8300: i/o timeout
2021/06/02 11:54:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:47 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:54:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:51 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:36944->10.242.0.7:8300: i/o timeout
2021/06/02 11:54:51 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:54:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:57 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:37022->10.242.0.7:8300: i/o timeout
2021/06/02 11:54:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:54:59 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:55:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:01 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:37094->10.242.0.7:8300: i/o timeout
2021/06/02 11:55:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:03 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:55:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:06 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:55:06 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:55:09 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:55:09 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:37182->10.242.0.7:8300: i/o timeout
2021/06/02 11:55:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:12 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:55:13 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:37332->10.242.0.7:8300: i/o timeout
2021/06/02 11:55:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:15 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:55:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:22 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:37452->10.242.0.7:8300: i/o timeout
2021/06/02 11:55:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:25 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:37494->10.242.0.7:8300: i/o timeout
2021/06/02 11:55:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:28 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:55:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:30 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:55:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:38 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:37660->10.242.0.7:8300: i/o timeout
2021/06/02 11:55:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:40 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:37686->10.242.0.7:8300: i/o timeout
2021/06/02 11:55:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:46 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.0.7:8301
2021/06/02 11:55:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:48 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:55:48 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:55:49 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:55:50 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 11:55:50 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:55:51 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:55:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:55:59 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:37926->10.242.0.7:8300: i/o timeout
2021/06/02 11:56:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:00 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:37954->10.242.0.7:8300: i/o timeout
2021/06/02 11:56:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:10 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:56:11 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:56:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:20 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:38192->10.242.0.7:8300: i/o timeout
2021/06/02 11:56:21 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:38204->10.242.0.7:8300: i/o timeout
2021/06/02 11:56:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:30 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:56:30 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:56:31 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:56:31 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:56:33 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:56:33 [WARN] consul: error getting server health from "dev-consul-server-1": context deadline exceeded
2021/06/02 11:56:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:41 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:38472->10.242.0.7:8300: i/o timeout
2021/06/02 11:56:41 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:38474->10.242.0.7:8300: i/o timeout
2021/06/02 11:56:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:51 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:56:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:52 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:56:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:56:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:01 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:38742->10.242.0.7:8300: i/o timeout
2021/06/02 11:57:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:02 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:38746->10.242.0.7:8300: i/o timeout
2021/06/02 11:57:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:11 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:57:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:12 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:57:12 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:57:13 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:57:15 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:57:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:21 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:38996->10.242.0.7:8300: i/o timeout
2021/06/02 11:57:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:23 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:39010->10.242.0.7:8300: i/o timeout
2021/06/02 11:57:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:32 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:57:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:34 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:57:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:42 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:39254->10.242.0.7:8300: i/o timeout
2021/06/02 11:57:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:44 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:39272->10.242.0.7:8300: i/o timeout
2021/06/02 11:57:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:52 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:57:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:57:54 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:57:54 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:57:55 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:57:56 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.0.7:8301
2021/06/02 11:57:57 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:57:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:58:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:58:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:58:02 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:39508->10.242.0.7:8300: i/o timeout
2021/06/02 11:58:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:58:05 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:39544->10.242.0.7:8300: i/o timeout
2021/06/02 11:58:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:58:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:58:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:58:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:58:12 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:58:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:58:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:58:16 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:58:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:58:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:58:20 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 11:58:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:58:22 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:39870->10.242.0.7:8300: i/o timeout
2021/06/02 11:58:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:58:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:58:26 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:39912->10.242.0.7:8300: i/o timeout
2021/06/02 11:58:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:58:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:58:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:58:32 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:58:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:58:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:58:36 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 11:58:36 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 11:58:37 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 11:58:39 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 11:58:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:58:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:58:42 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:40136->10.242.0.7:8300: i/o timeout
2021/06/02 11:58:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 11:58:45 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2)
2021/06/02 11:58:45 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.0.7
2021/06/02 11:58:45 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.0.7:8300) (DC: dc1)
2021/06/02 11:58:45 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/06/02 11:58:45 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.0.7
2021/06/02 11:58:45 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-1.dc1)
2021/06/02 11:58:45 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/06/02 11:58:45 [WARN] raft: Rejecting vote request from 10.242.0.7:8300 since we have a leader: 10.242.166.6:8300
2021/06/02 11:58:53 [INFO] raft: pipelining replication to peer {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}
2021/06/02 12:00:15 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:00:15 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:55146->10.242.0.7:8301: i/o timeout
2021/06/02 12:00:15 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/06/02 12:00:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:00:16 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 2.500223953s
2021/06/02 12:00:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:00:19 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 4.954323473s
2021/06/02 12:00:19 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/06/02 12:00:19 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.0.7
2021/06/02 12:00:19 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.0.7:8300) (DC: dc1)
2021/06/02 12:00:19 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/06/02 12:00:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:00:20 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:55220->10.242.0.7:8301: i/o timeout
2021/06/02 12:00:20 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/06/02 12:00:21 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 7.378919329s
2021/06/02 12:00:21 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:54840->10.242.0.7:8302: i/o timeout
2021/06/02 12:00:21 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 12:00:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:00:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:00:24 [ERR] raft: Failed to pipeline AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: write tcp 10.242.166.6:40192->10.242.0.7:8300: use of closed network connection
2021/06/02 12:00:24 [INFO] raft: aborting pipeline replication to peer {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}
2021/06/02 12:00:24 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:00:24 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:40502->10.242.0.7:8300: i/o timeout
2021/06/02 12:00:25 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:00:25 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.0.7:46620
2021/06/02 12:00:25 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:00:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:00:26 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 2.500209247s
2021/06/02 12:00:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:00:29 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 4.969104519s
2021/06/02 12:00:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:00:31 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 7.447649742s
2021/06/02 12:00:31 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:54976->10.242.0.7:8302: i/o timeout
2021/06/02 12:00:31 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 12:00:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:00:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:00:34 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:41734->10.242.0.7:8300: i/o timeout
2021/06/02 12:00:34 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:00:35 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:41754->10.242.0.7:8300: i/o timeout
2021/06/02 12:00:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:00:36 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:00:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:00:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:00:41 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:55032->10.242.0.7:8302: i/o timeout
2021/06/02 12:00:41 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 12:00:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:00:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:00:44 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:41860->10.242.0.7:8300: i/o timeout
2021/06/02 12:00:44 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:00:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:00:46 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:41898->10.242.0.7:8300: i/o timeout
2021/06/02 12:00:47 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:00:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:00:48 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:00:48 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:00:51 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:00:51 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/06/02 12:00:51 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.0.7
2021/06/02 12:00:51 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/06/02 12:00:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:00:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:00:54 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:41990->10.242.0.7:8300: i/o timeout
2021/06/02 12:00:54 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:00:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:00:56 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:55296->10.242.0.7:8302: i/o timeout
2021/06/02 12:00:56 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 12:00:57 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:42030->10.242.0.7:8300: i/o timeout
2021/06/02 12:00:57 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:00:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:04 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:42124->10.242.0.7:8300: i/o timeout
2021/06/02 12:01:04 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:01:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:07 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:42170->10.242.0.7:8300: i/o timeout
2021/06/02 12:01:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:08 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:01:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:14 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:42346->10.242.0.7:8300: i/o timeout
2021/06/02 12:01:14 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:01:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:18 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:42408->10.242.0.7:8300: i/o timeout
2021/06/02 12:01:18 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:01:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:24 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:42474->10.242.0.7:8300: i/o timeout
2021/06/02 12:01:24 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:01:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:29 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:42544->10.242.0.7:8300: i/o timeout
2021/06/02 12:01:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:30 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:01:30 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:01:30 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:01:33 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:01:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:34 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:42610->10.242.0.7:8300: i/o timeout
2021/06/02 12:01:34 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:01:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:40 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:42686->10.242.0.7:8300: i/o timeout
2021/06/02 12:01:41 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:01:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:44 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:42744->10.242.0.7:8300: i/o timeout
2021/06/02 12:01:45 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 12:01:45 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:01:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:51 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:42840->10.242.0.7:8300: i/o timeout
2021/06/02 12:01:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:53 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:01:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:55 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:42888->10.242.0.7:8300: i/o timeout
2021/06/02 12:01:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:01:56 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:01:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:03 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:42984->10.242.0.7:8300: i/o timeout
2021/06/02 12:02:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:06 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.0.7:8301
2021/06/02 12:02:06 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:43030->10.242.0.7:8300: i/o timeout
2021/06/02 12:02:06 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:02:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:09 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:02:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:12 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:02:12 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:02:15 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:02:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:16 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:43156->10.242.0.7:8300: i/o timeout
2021/06/02 12:02:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:19 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:43186->10.242.0.7:8300: i/o timeout
2021/06/02 12:02:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:23 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:02:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:24 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:02:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:33 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:43364->10.242.0.7:8300: i/o timeout
2021/06/02 12:02:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:34 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:43380->10.242.0.7:8300: i/o timeout
2021/06/02 12:02:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:44 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:02:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:44 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:02:45 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 12:02:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:54 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:43620->10.242.0.7:8300: i/o timeout
2021/06/02 12:02:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:02:54 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:02:54 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:02:54 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:43634->10.242.0.7:8300: i/o timeout
2021/06/02 12:02:57 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:02:57 [WARN] consul: error getting server health from "dev-consul-server-1": context deadline exceeded
2021/06/02 12:02:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:05 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:03:05 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:03:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:15 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:43898->10.242.0.7:8300: i/o timeout
2021/06/02 12:03:15 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:43900->10.242.0.7:8300: i/o timeout
2021/06/02 12:03:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:25 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:03:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:26 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:03:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:35 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:44166->10.242.0.7:8300: i/o timeout
2021/06/02 12:03:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:36 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:03:36 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:03:36 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:44172->10.242.0.7:8300: i/o timeout
2021/06/02 12:03:39 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:03:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:45 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 12:03:45 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:03:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:47 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:03:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:55 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:44430->10.242.0.7:8300: i/o timeout
2021/06/02 12:03:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:03:57 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:44444->10.242.0.7:8300: i/o timeout
2021/06/02 12:03:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:04:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:04:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:04:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:04:05 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:04:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:04:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:04:08 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:04:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:04:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:04:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:04:15 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:44784->10.242.0.7:8300: i/o timeout
2021/06/02 12:04:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:04:16 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.0.7:8301
2021/06/02 12:04:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:04:18 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:04:18 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:04:18 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:44814->10.242.0.7:8300: i/o timeout
2021/06/02 12:04:21 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:04:21 [WARN] consul: error getting server health from "dev-consul-server-1": context deadline exceeded
2021/06/02 12:04:21 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/06/02 12:04:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:04:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:04:26 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:04:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:04:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:04:29 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:04:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:04:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:04:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:04:36 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:45032->10.242.0.7:8300: i/o timeout
2021/06/02 12:04:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:04:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:04:39 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:45082->10.242.0.7:8300: i/o timeout
2021/06/02 12:04:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:04:40 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.0.7
2021/06/02 12:04:40 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/06/02 12:04:40 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.0.7
2021/06/02 12:04:40 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.0.7:8300) (DC: dc1)
2021/06/02 12:04:40 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/06/02 12:04:40 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-2.dc1)
2021/06/02 12:04:40 [WARN] raft: Rejecting vote request from 10.242.0.7:8300 since we have a leader: 10.242.166.6:8300
2021/06/02 12:04:46 [INFO] raft: pipelining replication to peer {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}
2021/06/02 12:06:01 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:06:01 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:60136->10.242.0.7:8301: i/o timeout
2021/06/02 12:06:01 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/06/02 12:06:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:04 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:60170->10.242.0.7:8301: i/o timeout
2021/06/02 12:06:04 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/06/02 12:06:05 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:06:05 [WARN] consul: error getting server health from "dev-consul-server-1": context deadline exceeded
2021/06/02 12:06:05 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/06/02 12:06:05 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.0.7
2021/06/02 12:06:05 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.0.7:8300) (DC: dc1)
2021/06/02 12:06:05 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/06/02 12:06:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:07 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 2.500248719s
2021/06/02 12:06:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:10 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 4.952802898s
2021/06/02 12:06:11 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:59882->10.242.0.7:8302: i/o timeout
2021/06/02 12:06:11 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 12:06:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:12 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 7.404467163s
2021/06/02 12:06:12 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:45546->10.242.0.7:8300: i/o timeout
2021/06/02 12:06:13 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:06:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:15 [INFO] raft: aborting pipeline replication to peer {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}
2021/06/02 12:06:15 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:06:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:17 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 2.50025872s
2021/06/02 12:06:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:20 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 4.97407545s
2021/06/02 12:06:20 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:06:20 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.0.7:49390
2021/06/02 12:06:21 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:60024->10.242.0.7:8302: i/o timeout
2021/06/02 12:06:21 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 12:06:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:22 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 7.460509138s
2021/06/02 12:06:23 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:46772->10.242.0.7:8300: i/o timeout
2021/06/02 12:06:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:24 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:06:25 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:46794->10.242.0.7:8300: i/o timeout
2021/06/02 12:06:25 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:06:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:30 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:06:30 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:06:33 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:06:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:34 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:46920->10.242.0.7:8300: i/o timeout
2021/06/02 12:06:35 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:06:35 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:46936->10.242.0.7:8300: i/o timeout
2021/06/02 12:06:35 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:06:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:36 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:60214->10.242.0.7:8302: i/o timeout
2021/06/02 12:06:36 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 12:06:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:38 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/06/02 12:06:38 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.0.7
2021/06/02 12:06:38 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/06/02 12:06:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:45 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:47058->10.242.0.7:8300: i/o timeout
2021/06/02 12:06:45 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:47066->10.242.0.7:8300: i/o timeout
2021/06/02 12:06:45 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:06:45 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:06:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:46 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:60286->10.242.0.7:8302: i/o timeout
2021/06/02 12:06:46 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 12:06:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:55 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:47196->10.242.0.7:8300: i/o timeout
2021/06/02 12:06:55 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:06:55 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:47198->10.242.0.7:8300: i/o timeout
2021/06/02 12:06:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:06:56 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.0.7:8301
2021/06/02 12:06:56 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:06:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:05 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:47324->10.242.0.7:8300: i/o timeout
2021/06/02 12:07:05 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:07:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:06 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:47346->10.242.0.7:8300: i/o timeout
2021/06/02 12:07:07 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:07:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:12 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:07:12 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:07:15 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:07:15 [WARN] consul: error getting server health from "dev-consul-server-1": context deadline exceeded
2021/06/02 12:07:15 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 12:07:15 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:47550->10.242.0.7:8300: i/o timeout
2021/06/02 12:07:15 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:07:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:17 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:47578->10.242.0.7:8300: i/o timeout
2021/06/02 12:07:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:18 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:07:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:25 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:47688->10.242.0.7:8300: i/o timeout
2021/06/02 12:07:25 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:07:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:28 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:47730->10.242.0.7:8300: i/o timeout
2021/06/02 12:07:29 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:07:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:35 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:47818->10.242.0.7:8300: i/o timeout
2021/06/02 12:07:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:36 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:07:36 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.0.7:8301
2021/06/02 12:07:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:39 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:47872->10.242.0.7:8300: i/o timeout
2021/06/02 12:07:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:42 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:07:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:46 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:47950->10.242.0.7:8300: i/o timeout
2021/06/02 12:07:47 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:07:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:52 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:48030->10.242.0.7:8300: i/o timeout
2021/06/02 12:07:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:07:54 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:07:54 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:07:55 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:07:57 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:07:57 [WARN] consul: error getting server health from "dev-consul-server-0": context deadline exceeded
2021/06/02 12:07:57 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:48100->10.242.0.7:8300: i/o timeout
2021/06/02 12:07:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:00 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:08:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:05 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:48200->10.242.0.7:8300: i/o timeout
2021/06/02 12:08:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:10 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:48266->10.242.0.7:8300: i/o timeout
2021/06/02 12:08:11 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:08:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:15 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 12:08:15 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:08:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:21 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:48406->10.242.0.7:8300: i/o timeout
2021/06/02 12:08:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:25 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:48458->10.242.0.7:8300: i/o timeout
2021/06/02 12:08:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:32 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:08:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:35 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:08:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:36 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:08:36 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:08:39 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:08:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:42 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:48668->10.242.0.7:8300: i/o timeout
2021/06/02 12:08:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:45 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:48714->10.242.0.7:8300: i/o timeout
2021/06/02 12:08:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:46 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.0.7:8301
2021/06/02 12:08:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:53 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:08:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:08:56 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:08:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:09:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:09:00 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.0.7
2021/06/02 12:09:00 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.0.7:8300) (DC: dc1)
2021/06/02 12:09:00 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/06/02 12:09:00 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.0.7
2021/06/02 12:09:00 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/06/02 12:09:01 [WARN] raft: Rejecting vote request from 10.242.0.7:8300 since we have a leader: 10.242.166.6:8300
2021/06/02 12:09:01 [INFO] raft: pipelining replication to peer {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}
2021/06/02 12:15:17 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:39662->10.242.0.7:8301: i/o timeout
2021/06/02 12:15:17 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/06/02 12:15:19 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:15:19 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 2.500176263s
2021/06/02 12:15:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:15:21 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 4.915487218s
2021/06/02 12:15:21 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:39336->10.242.0.7:8302: i/o timeout
2021/06/02 12:15:21 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 12:15:21 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:39732->10.242.0.7:8301: i/o timeout
2021/06/02 12:15:21 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/06/02 12:15:21 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/06/02 12:15:21 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.0.7
2021/06/02 12:15:21 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.0.7:8300) (DC: dc1)
2021/06/02 12:15:21 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/06/02 12:15:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:15:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:15:24 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 7.411888575s
2021/06/02 12:15:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:15:26 [INFO] raft: aborting pipeline replication to peer {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}
2021/06/02 12:15:26 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:15:27 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:48934->10.242.0.7:8300: i/o timeout
2021/06/02 12:15:27 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:15:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:15:29 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 2.500202193s
2021/06/02 12:15:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:15:31 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 4.982203597s
2021/06/02 12:15:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:15:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:15:34 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 7.44009565s
2021/06/02 12:15:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:15:36 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:39522->10.242.0.7:8302: i/o timeout
2021/06/02 12:15:36 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 12:15:36 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:54498->10.242.0.7:8300: i/o timeout
2021/06/02 12:15:36 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:15:37 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:54508->10.242.0.7:8300: i/o timeout
2021/06/02 12:15:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:15:38 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:15:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:15:40 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:15:40 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.0.7:51990
2021/06/02 12:15:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:15:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:15:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:15:46 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:39652->10.242.0.7:8302: i/o timeout
2021/06/02 12:15:46 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 12:15:46 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:54626->10.242.0.7:8300: i/o timeout
2021/06/02 12:15:46 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:15:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:15:48 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:15:48 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:15:48 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:54646->10.242.0.7:8300: i/o timeout
2021/06/02 12:15:49 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:15:51 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:15:51 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/06/02 12:15:51 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.0.7
2021/06/02 12:15:51 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/06/02 12:15:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:15:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:15:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:15:56 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:39782->10.242.0.7:8302: i/o timeout
2021/06/02 12:15:56 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 12:15:56 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:54748->10.242.0.7:8300: i/o timeout
2021/06/02 12:15:56 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:15:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:15:59 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:54782->10.242.0.7:8300: i/o timeout
2021/06/02 12:16:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:00 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:16:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:06 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:54884->10.242.0.7:8300: i/o timeout
2021/06/02 12:16:06 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:16:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:10 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:54924->10.242.0.7:8300: i/o timeout
2021/06/02 12:16:10 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:16:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:15 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 12:16:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:16 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:55116->10.242.0.7:8300: i/o timeout
2021/06/02 12:16:17 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:16:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:20 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:55160->10.242.0.7:8300: i/o timeout
2021/06/02 12:16:21 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:16:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:27 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:55238->10.242.0.7:8300: i/o timeout
2021/06/02 12:16:27 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:16:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:30 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:16:30 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:16:31 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:55312->10.242.0.7:8300: i/o timeout
2021/06/02 12:16:32 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:16:33 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:16:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:37 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:55370->10.242.0.7:8300: i/o timeout
2021/06/02 12:16:37 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:16:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:42 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:55450->10.242.0.7:8300: i/o timeout
2021/06/02 12:16:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:44 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:16:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:47 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:55506->10.242.0.7:8300: i/o timeout
2021/06/02 12:16:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:48 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:16:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:54 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:55584->10.242.0.7:8300: i/o timeout
2021/06/02 12:16:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:56 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:16:56 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.0.7:8301
2021/06/02 12:16:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:16:58 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:55650->10.242.0.7:8300: i/o timeout
2021/06/02 12:16:59 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:17:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:06 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:55738->10.242.0.7:8300: i/o timeout
2021/06/02 12:17:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:09 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:55782->10.242.0.7:8300: i/o timeout
2021/06/02 12:17:09 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:17:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:12 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:17:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:12 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:17:12 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:17:15 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:17:15 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 12:17:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:19 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:55910->10.242.0.7:8300: i/o timeout
2021/06/02 12:17:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:22 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:55950->10.242.0.7:8300: i/o timeout
2021/06/02 12:17:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:25 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:17:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:27 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:17:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:35 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:56118->10.242.0.7:8300: i/o timeout
2021/06/02 12:17:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:37 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:56138->10.242.0.7:8300: i/o timeout
2021/06/02 12:17:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:46 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:17:47 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:17:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:17:54 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:17:54 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:17:56 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:56376->10.242.0.7:8300: i/o timeout
2021/06/02 12:17:57 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:17:57 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:56386->10.242.0.7:8300: i/o timeout
2021/06/02 12:17:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:07 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:18:07 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:18:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:15 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 12:18:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:17 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:56638->10.242.0.7:8300: i/o timeout
2021/06/02 12:18:17 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:56642->10.242.0.7:8300: i/o timeout
2021/06/02 12:18:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:27 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:18:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:28 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:18:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:36 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:18:36 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:18:37 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:56900->10.242.0.7:8300: i/o timeout
2021/06/02 12:18:38 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:56920->10.242.0.7:8300: i/o timeout
2021/06/02 12:18:39 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:18:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:48 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:18:49 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:18:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:18:58 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:57162->10.242.0.7:8300: i/o timeout
2021/06/02 12:18:59 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:57174->10.242.0.7:8300: i/o timeout
2021/06/02 12:19:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:08 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:19:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:10 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:19:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:15 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 12:19:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:18 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:19:18 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:19:18 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:57512->10.242.0.7:8300: i/o timeout
2021/06/02 12:19:20 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:57534->10.242.0.7:8300: i/o timeout
2021/06/02 12:19:21 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:19:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:28 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:19:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:31 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:19:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:38 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:57784->10.242.0.7:8300: i/o timeout
2021/06/02 12:19:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:41 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:57824->10.242.0.7:8300: i/o timeout
2021/06/02 12:19:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:48 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:19:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:52 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:19:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:19:58 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:58046->10.242.0.7:8300: i/o timeout
2021/06/02 12:20:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:20:00 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:20:00 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:20:02 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:58094->10.242.0.7:8300: i/o timeout
2021/06/02 12:20:03 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:20:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:20:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:20:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:20:08 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.0.7
2021/06/02 12:20:08 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.0.7:8300) (DC: dc1)
2021/06/02 12:20:08 [INFO] consul: member 'dev-consul-server-2' joined, marking health alive
2021/06/02 12:20:08 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.0.7
2021/06/02 12:20:08 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan"
2021/06/02 12:20:09 [WARN] raft: Rejecting vote request from 10.242.0.7:8300 since we have a leader: 10.242.166.6:8300
2021/06/02 12:20:09 [INFO] raft: pipelining replication to peer {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}
2021/06/02 12:21:25 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:21:25 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:44820->10.242.0.7:8301: i/o timeout
2021/06/02 12:21:25 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received
2021/06/02 12:21:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:21:27 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 2.500231015s
2021/06/02 12:21:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:21:28 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations)
2021/06/02 12:21:28 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.0.7
2021/06/02 12:21:28 [INFO] consul: Removing LAN server dev-consul-server-2 (Addr: tcp/10.242.0.7:8300) (DC: dc1)
2021/06/02 12:21:28 [INFO] consul: member 'dev-consul-server-2' failed, marking health critical
2021/06/02 12:21:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:21:30 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 4.993985328s
2021/06/02 12:21:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:21:32 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 7.446922791s
2021/06/02 12:21:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:21:34 [INFO] raft: aborting pipeline replication to peer {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}
2021/06/02 12:21:34 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:21:35 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:58528->10.242.0.7:8300: i/o timeout
2021/06/02 12:21:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:21:36 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:21:36 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:44570->10.242.0.7:8302: i/o timeout
2021/06/02 12:21:36 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 12:21:36 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 2.500174087s
2021/06/02 12:21:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:21:39 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 4.999867053s
2021/06/02 12:21:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:21:41 [WARN] raft: Failed to contact 60597ff4-43fd-637b-127d-664e51804d7f in 7.41191837s
2021/06/02 12:21:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:21:42 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:21:42 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:21:44 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:59630->10.242.0.7:8300: i/o timeout
2021/06/02 12:21:44 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:21:45 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:21:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:21:46 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:59666->10.242.0.7:8300: i/o timeout
2021/06/02 12:21:46 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:44640->10.242.0.7:8302: i/o timeout
2021/06/02 12:21:46 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 12:21:47 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:21:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:21:48 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:21:48 [ERR] consul.rpc: multiplex conn accept failed: keepalive timeout from=10.242.0.7:60890
2021/06/02 12:21:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:21:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:21:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:21:54 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:59758->10.242.0.7:8300: i/o timeout
2021/06/02 12:21:54 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:21:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:21:56 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:44846->10.242.0.7:8302: i/o timeout
2021/06/02 12:21:56 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 12:21:57 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:59812->10.242.0.7:8300: i/o timeout
2021/06/02 12:21:57 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:21:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:03 [INFO] memberlist: Marking dev-consul-server-2.dc1 as failed, suspect timeout reached (0 peer confirmations)
2021/06/02 12:22:03 [INFO] serf: EventMemberFailed: dev-consul-server-2.dc1 10.242.0.7
2021/06/02 12:22:03 [INFO] consul: Handled member-failed event for server "dev-consul-server-2.dc1" in area "wan"
2021/06/02 12:22:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:04 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:59906->10.242.0.7:8300: i/o timeout
2021/06/02 12:22:04 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:22:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:06 [ERR] memberlist: Failed fallback ping: read tcp 10.242.166.6:45084->10.242.0.7:8302: i/o timeout
2021/06/02 12:22:06 [INFO] memberlist: Suspect dev-consul-server-2.dc1 has failed, no acks received
2021/06/02 12:22:07 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:59952->10.242.0.7:8300: i/o timeout
2021/06/02 12:22:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:08 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:22:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:14 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:60144->10.242.0.7:8300: i/o timeout
2021/06/02 12:22:14 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:22:15 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 12:22:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:18 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:60208->10.242.0.7:8300: i/o timeout
2021/06/02 12:22:19 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:22:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:24 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:22:24 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:22:24 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:60272->10.242.0.7:8300: i/o timeout
2021/06/02 12:22:24 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:22:27 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:22:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:29 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:60346->10.242.0.7:8300: i/o timeout
2021/06/02 12:22:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:30 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:22:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:34 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:60410->10.242.0.7:8300: i/o timeout
2021/06/02 12:22:34 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:22:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:40 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:60494->10.242.0.7:8300: i/o timeout
2021/06/02 12:22:41 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:22:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:44 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:60542->10.242.0.7:8300: i/o timeout
2021/06/02 12:22:45 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:22:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:51 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:60638->10.242.0.7:8300: i/o timeout
2021/06/02 12:22:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:53 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:22:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:55 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:60680->10.242.0.7:8300: i/o timeout
2021/06/02 12:22:55 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:22:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:22:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:03 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:60782->10.242.0.7:8300: i/o timeout
2021/06/02 12:23:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:05 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:23:05 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:60814->10.242.0.7:8300: i/o timeout
2021/06/02 12:23:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:06 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:23:06 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:23:07 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:23:09 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:23:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:15 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:60924->10.242.0.7:8300: i/o timeout
2021/06/02 12:23:15 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 12:23:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:17 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:60956->10.242.0.7:8300: i/o timeout
2021/06/02 12:23:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:18 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:23:19 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:23:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:28 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:32880->10.242.0.7:8300: i/o timeout
2021/06/02 12:23:29 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:32888->10.242.0.7:8300: i/o timeout
2021/06/02 12:23:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:34 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:23:34 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:23:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:44 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:33066->10.242.0.7:8300: i/o timeout
2021/06/02 12:23:44 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:33072->10.242.0.7:8300: i/o timeout
2021/06/02 12:23:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:48 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:23:48 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:23:51 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:23:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:54 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:23:55 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:23:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:23:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:04 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:33328->10.242.0.7:8300: i/o timeout
2021/06/02 12:24:05 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:33332->10.242.0.7:8300: i/o timeout
2021/06/02 12:24:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:14 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:15 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:24:16 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:24:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:25 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:33582->10.242.0.7:8300: i/o timeout
2021/06/02 12:24:26 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:33596->10.242.0.7:8300: i/o timeout
2021/06/02 12:24:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:30 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:24:30 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:24:33 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:24:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:35 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:24:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:36 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:24:36 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.0.7:8301
2021/06/02 12:24:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:45 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:33840->10.242.0.7:8300: i/o timeout
2021/06/02 12:24:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:46 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:33864->10.242.0.7:8300: i/o timeout
2021/06/02 12:24:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:55 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:24:56 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:24:57 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:24:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:05 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:34096->10.242.0.7:8300: i/o timeout
2021/06/02 12:25:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:07 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:34122->10.242.0.7:8300: i/o timeout
2021/06/02 12:25:08 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:10 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:12 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:12 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:25:12 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:25:15 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:25:15 [INFO] serf: attempting reconnect to dev-consul-server-2.dc1 10.242.0.7:8302
2021/06/02 12:25:15 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:25:16 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:18 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:18 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:25:20 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:22 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:24 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:25 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:34456->10.242.0.7:8300: i/o timeout
2021/06/02 12:25:26 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:28 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:28 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:34500->10.242.0.7:8300: i/o timeout
2021/06/02 12:25:30 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:32 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:34 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:36 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:36 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:25:38 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:39 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:25:40 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:42 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:44 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:46 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:46 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:34706->10.242.0.7:8300: i/o timeout
2021/06/02 12:25:46 [INFO] serf: attempting reconnect to dev-consul-server-2 10.242.0.7:8301
2021/06/02 12:25:48 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:49 [ERR] raft: Failed to heartbeat to 10.242.0.7:8300: read tcp 10.242.166.6:34748->10.242.0.7:8300: i/o timeout
2021/06/02 12:25:50 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:52 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:54 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:25:54 [ERR] yamux: keepalive failed: i/o deadline reached
2021/06/02 12:25:54 [WARN] consul: error getting server health from "dev-consul-server-2": rpc error making call: EOF
2021/06/02 12:25:56 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:25:57 [WARN] consul: error getting server health from "dev-consul-server-2": context deadline exceeded
2021/06/02 12:25:58 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:26:00 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:26:00 [WARN] Unable to get address for server id 60597ff4-43fd-637b-127d-664e51804d7f, using fallback address 10.242.0.7:8300: Could not find address for server id 60597ff4-43fd-637b-127d-664e51804d7f
2021/06/02 12:26:02 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:26:04 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:26:06 [WARN] consul: error getting server health from "dev-consul-server-2": last request still outstanding
2021/06/02 12:26:06 [ERR] raft: Failed to AppendEntries to {Voter 60597ff4-43fd-637b-127d-664e51804d7f 10.242.0.7:8300}: read tcp 10.242.166.6:34956->10.242.0.7:8300: i/o timeout