By type
+ CONSUL_BIND= + [ -n ] + CONSUL_CLIENT= + [ -n ] + CONSUL_DATA_DIR=/consul/data + CONSUL_CONFIG_DIR=/consul/config + [ -n ] + [ a = - ] + [ agent = agent ] + shift + set -- consul agent -data-dir=/consul/data -config-dir=/consul/config -bootstrap-expect=3 -enable-script-checks -retry-join=dev-consul-server-0.consul-server -retry-join=dev-consul-server-1.consul-server -retry-join=dev-consul-server-2.consul-server -client=0.0.0.0 -server -ui + [ consul = consul ] + stat -c %u /consul/data + id -u consul + [ 100 != 100 ] + stat -c %u /consul/config + id -u consul + [ 100 != 100 ] + [ ! -z ] + exec consul agent -data-dir=/consul/data -config-dir=/consul/config -bootstrap-expect=3 -enable-script-checks -retry-join=dev-consul-server-0.consul-server -retry-join=dev-consul-server-1.consul-server -retry-join=dev-consul-server-2.consul-server -client=0.0.0.0 -server -ui bootstrap_expect > 0: expecting 3 servers ==> Starting Consul agent... ==> Consul agent running! Version: 'v1.0.6' Node ID: '4b16da66-7a8a-3eae-065c-181a00688338' Node name: 'dev-consul-server-2' Datacenter: 'dc1' (Segment: '
') Server: true (Bootstrap: false) Client Addr: [0.0.0.0] (HTTP: 8500, HTTPS: -1, DNS: 8600) Cluster Addr: 10.242.169.136 (LAN: 8301, WAN: 8302) Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false ==> Log data will now stream in as it occurs: 2021/05/07 22:38:28 [INFO] raft: Initial configuration (index=0): [] 2021/05/07 22:38:28 [INFO] raft: Node at 10.242.169.136:8300 [Follower] entering Follower state (Leader: "") 2021/05/07 22:38:28 [INFO] serf: EventMemberJoin: dev-consul-server-2.dc1 10.242.169.136 2021/05/07 22:38:28 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.169.136 2021/05/07 22:38:28 [INFO] consul: Adding LAN server dev-consul-server-2 (Addr: tcp/10.242.169.136:8300) (DC: dc1) 2021/05/07 22:38:28 [INFO] agent: Started DNS server 0.0.0.0:8600 (udp) 2021/05/07 22:38:28 [INFO] consul: Handled member-join event for server "dev-consul-server-2.dc1" in area "wan" 2021/05/07 22:38:28 [INFO] agent: Started DNS server 0.0.0.0:8600 (tcp) 2021/05/07 22:38:28 [INFO] agent: Started HTTP server on [::]:8500 (tcp) 2021/05/07 22:38:28 [INFO] agent: started state syncer 2021/05/07 22:38:28 [INFO] agent: Retry join LAN is supported for: aliyun aws azure digitalocean gce os scaleway softlayer 2021/05/07 22:38:28 [INFO] agent: Joining LAN cluster... 2021/05/07 22:38:28 [INFO] agent: (LAN) joining: [dev-consul-server-0.consul-server dev-consul-server-1.consul-server dev-consul-server-2.consul-server] 2021/05/07 22:38:28 [INFO] serf: EventMemberJoin: dev-consul-server-1 10.242.251.82 2021/05/07 22:38:28 [INFO] serf: EventMemberJoin: dev-consul-server-0 10.242.134.70 2021/05/07 22:38:28 [INFO] consul: Adding LAN server dev-consul-server-1 (Addr: tcp/10.242.251.82:8300) (DC: dc1) 2021/05/07 22:38:28 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-pgbp7 10.242.251.74 2021/05/07 22:38:28 [INFO] consul: Found expected number of peers, attempting bootstrap: 10.242.169.136:8300,10.242.251.82:8300,10.242.134.70:8300 2021/05/07 22:38:28 [INFO] consul: Adding LAN server dev-consul-server-0 (Addr: tcp/10.242.134.70:8300) (DC: dc1) 2021/05/07 22:38:28 [INFO] serf: EventMemberJoin: dev-consul-server-1.dc1 10.242.251.82 2021/05/07 22:38:28 [INFO] serf: EventMemberJoin: dev-consul-server-0.dc1 10.242.134.70 2021/05/07 22:38:28 [INFO] consul: Handled member-join event for server "dev-consul-server-1.dc1" in area "wan" 2021/05/07 22:38:28 [INFO] consul: Handled member-join event for server "dev-consul-server-0.dc1" in area "wan" 2021/05/07 22:38:28 [WARN] memberlist: Failed to resolve dev-consul-server-2.consul-server: lookup dev-consul-server-2.consul-server on 10.96.0.10:53: no such host 2021/05/07 22:38:28 [INFO] agent: (LAN) joined: 2 Err: 2021/05/07 22:38:28 [INFO] agent: Join LAN completed. Synced with 2 initial agents 2021/05/07 22:38:28 [DEBUG] raft-net: 10.242.169.136:8300 accepted connection from: 10.242.134.70:41418 2021/05/07 22:38:28 [INFO] consul: New leader elected: dev-consul-server-0 2021/05/07 22:38:28 [INFO] agent: Synced node info 2021/05/07 22:38:29 [DEBUG] raft-net: 10.242.169.136:8300 accepted connection from: 10.242.134.70:41428 ==> Failed to check for updates: Get https://checkpoint-api.hashicorp.com/v1/check/consul?arch=amd64&os=linux&signature=0464a505-62b6-4466-ef3f-79896f84589c&version=1.0.6: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/07 22:52:55 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:55816->10.242.251.82:8301: i/o timeout 2021/05/07 22:52:55 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/07 22:52:58 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:53758->10.242.251.74:8301: i/o timeout 2021/05/07 22:52:58 [INFO] memberlist: Suspect dev-consul-7467bf7b46-pgbp7 has failed, no acks received 2021/05/07 22:53:01 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:56070->10.242.251.82:8301: i/o timeout 2021/05/07 22:53:01 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/07 22:53:05 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:56390->10.242.251.82:8301: i/o timeout 2021/05/07 22:53:05 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/07 22:53:08 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:54320->10.242.251.74:8301: i/o timeout 2021/05/07 22:53:08 [INFO] memberlist: Suspect dev-consul-7467bf7b46-pgbp7 has failed, no acks received 2021/05/07 22:53:10 [INFO] serf: EventMemberFailed: dev-consul-server-1 10.242.251.82 2021/05/07 22:53:10 [INFO] consul: Removing LAN server dev-consul-server-1 (Addr: tcp/10.242.251.82:8300) (DC: dc1) 2021/05/07 22:53:11 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:56448->10.242.251.82:8301: i/o timeout 2021/05/07 22:53:11 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/07 22:53:13 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:51418->10.242.251.82:8302: i/o timeout 2021/05/07 22:53:13 [INFO] memberlist: Suspect dev-consul-server-1.dc1 has failed, no acks received 2021/05/07 22:53:14 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:54390->10.242.251.74:8301: i/o timeout 2021/05/07 22:53:14 [INFO] memberlist: Suspect dev-consul-7467bf7b46-pgbp7 has failed, no acks received 2021/05/07 22:53:17 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:54460->10.242.251.74:8301: i/o timeout 2021/05/07 22:53:17 [INFO] memberlist: Suspect dev-consul-7467bf7b46-pgbp7 has failed, no acks received 2021/05/07 22:53:17 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-pgbp7 10.242.251.74 2021/05/07 22:53:19 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:54472->10.242.251.74:8301: i/o timeout 2021/05/07 22:53:19 [INFO] memberlist: Suspect dev-consul-7467bf7b46-pgbp7 has failed, no acks received 2021/05/07 22:53:23 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:51544->10.242.251.82:8302: i/o timeout 2021/05/07 22:53:23 [INFO] memberlist: Suspect dev-consul-server-1.dc1 has failed, no acks received 2021/05/07 22:53:28 [INFO] serf: attempting reconnect to dev-consul-server-1 10.242.251.82:8301 2021/05/07 22:53:33 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:51622->10.242.251.82:8302: i/o timeout 2021/05/07 22:53:33 [INFO] memberlist: Suspect dev-consul-server-1.dc1 has failed, no acks received 2021/05/07 22:53:37 [INFO] serf: EventMemberJoin: dev-consul-server-1 10.242.251.82 2021/05/07 22:53:37 [INFO] consul: Adding LAN server dev-consul-server-1 (Addr: tcp/10.242.251.82:8300) (DC: dc1) 2021/05/07 22:53:39 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:56832->10.242.251.82:8301: i/o timeout 2021/05/07 22:53:39 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/07 22:53:41 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:56846->10.242.251.82:8301: i/o timeout 2021/05/07 22:53:41 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/07 22:53:43 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:56886->10.242.251.82:8301: i/o timeout 2021/05/07 22:53:43 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/07 22:53:43 [INFO] memberlist: Marking dev-consul-server-1.dc1 as failed, suspect timeout reached (0 peer confirmations) 2021/05/07 22:53:43 [INFO] serf: EventMemberFailed: dev-consul-server-1.dc1 10.242.251.82 2021/05/07 22:53:43 [INFO] consul: Handled member-failed event for server "dev-consul-server-1.dc1" in area "wan" 2021/05/07 22:53:45 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:56912->10.242.251.82:8301: i/o timeout 2021/05/07 22:53:45 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/07 22:53:48 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:51878->10.242.251.82:8302: i/o timeout 2021/05/07 22:53:48 [INFO] memberlist: Suspect dev-consul-server-1.dc1 has failed, no acks received 2021/05/07 22:53:48 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:56954->10.242.251.82:8301: i/o timeout 2021/05/07 22:53:48 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/07 22:53:50 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:57190->10.242.251.82:8301: i/o timeout 2021/05/07 22:53:50 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/07 22:53:50 [INFO] memberlist: Marking dev-consul-server-1 as failed, suspect timeout reached (1 peer confirmations) 2021/05/07 22:53:50 [INFO] serf: EventMemberFailed: dev-consul-server-1 10.242.251.82 2021/05/07 22:53:50 [INFO] consul: Removing LAN server dev-consul-server-1 (Addr: tcp/10.242.251.82:8300) (DC: dc1) 2021/05/07 22:53:52 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:57276->10.242.251.82:8301: i/o timeout 2021/05/07 22:53:52 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/07 22:54:07 [ERR] memberlist: failed to receive: read tcp 10.242.169.136:8302->10.242.251.82:57138: i/o timeout from=10.242.251.82:57138 2021/05/07 22:54:07 [ERR] memberlist: Failed to send error: write tcp 10.242.169.136:8302->10.242.251.82:57138: i/o timeout from=10.242.251.82:57138 2021/05/07 22:54:08 [INFO] serf: attempting reconnect to dev-consul-server-1 10.242.251.82:8301 2021/05/07 22:54:25 [ERR] memberlist: failed to receive: read tcp 10.242.169.136:8302->10.242.251.82:57172: i/o timeout from=10.242.251.82:57172 2021/05/07 22:54:25 [ERR] memberlist: Failed to send error: write tcp 10.242.169.136:8302->10.242.251.82:57172: i/o timeout from=10.242.251.82:57172 2021/05/07 22:54:28 [INFO] serf: attempting reconnect to dev-consul-server-1.dc1 10.242.251.82:8302 2021/05/07 22:54:48 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-pgbp7 10.242.251.74:8301 2021/05/07 22:54:48 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-pgbp7 10.242.251.74 2021/05/07 22:54:49 [INFO] serf: EventMemberJoin: dev-consul-server-1 10.242.251.82 2021/05/07 22:54:49 [INFO] consul: Adding LAN server dev-consul-server-1 (Addr: tcp/10.242.251.82:8300) (DC: dc1) 2021/05/07 22:54:50 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:56180->10.242.251.74:8301: i/o timeout 2021/05/07 22:54:50 [INFO] memberlist: Suspect dev-consul-7467bf7b46-pgbp7 has failed, no acks received 2021/05/07 22:54:52 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:58292->10.242.251.82:8301: i/o timeout 2021/05/07 22:54:52 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/07 22:54:55 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:58360->10.242.251.82:8301: i/o timeout 2021/05/07 22:54:55 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/07 22:54:57 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:56286->10.242.251.74:8301: i/o timeout 2021/05/07 22:54:57 [INFO] memberlist: Suspect dev-consul-7467bf7b46-pgbp7 has failed, no acks received 2021/05/07 22:54:59 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:58402->10.242.251.82:8301: i/o timeout 2021/05/07 22:54:59 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/07 22:55:01 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:56336->10.242.251.74:8301: i/o timeout 2021/05/07 22:55:01 [INFO] memberlist: Suspect dev-consul-7467bf7b46-pgbp7 has failed, no acks received 2021/05/07 22:55:01 [INFO] memberlist: Marking dev-consul-7467bf7b46-pgbp7 as failed, suspect timeout reached (1 peer confirmations) 2021/05/07 22:55:01 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-pgbp7 10.242.251.74 2021/05/07 22:55:02 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:58448->10.242.251.82:8301: i/o timeout 2021/05/07 22:55:02 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/07 22:55:03 [INFO] memberlist: Marking dev-consul-server-1 as failed, suspect timeout reached (1 peer confirmations) 2021/05/07 22:55:03 [INFO] serf: EventMemberFailed: dev-consul-server-1 10.242.251.82 2021/05/07 22:55:03 [INFO] consul: Removing LAN server dev-consul-server-1 (Addr: tcp/10.242.251.82:8300) (DC: dc1) 2021/05/07 22:55:04 [ERR] memberlist: Failed fallback ping: read tcp 10.242.169.136:58492->10.242.251.82:8301: i/o timeout 2021/05/07 22:55:04 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/07 22:55:28 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-pgbp7 10.242.251.74:8301 2021/05/07 22:56:08 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-pgbp7 10.242.251.74:8301 2021/05/07 22:56:28 [INFO] serf: attempting reconnect to dev-consul-server-1.dc1 10.242.251.82:8302 2021/05/07 22:56:48 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-pgbp7 10.242.251.74:8301 2021/05/07 22:57:28 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-pgbp7 10.242.251.74:8301 2021/05/07 22:57:58 [INFO] serf: attempting reconnect to dev-consul-server-1.dc1 10.242.251.82:8302 2021/05/07 22:58:08 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-pgbp7 10.242.251.74:8301 2021/05/07 22:58:48 [INFO] serf: attempting reconnect to dev-consul-server-1 10.242.251.82:8301 2021/05/07 22:58:52 [ERR] yamux: keepalive failed: i/o deadline reached 2021/05/07 22:58:57 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-87mjc 10.242.134.93 2021/05/07 22:59:28 [INFO] serf: attempting reconnect to dev-consul-server-1.dc1 10.242.251.82:8302 2021/05/07 22:59:28 [INFO] serf: attempting reconnect to dev-consul-server-1 10.242.251.82:8301 2021/05/07 23:00:08 [INFO] serf: attempting reconnect to dev-consul-7467bf7b46-pgbp7 10.242.251.74:8301 2021/05/07 23:00:30 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-pgbp7 10.242.251.74 2021/05/07 23:00:30 [DEBUG] raft-net: 10.242.169.136:8300 accepted connection from: 10.242.251.82:51518 2021/05/07 23:00:30 [WARN] raft: Rejecting vote request from 10.242.251.82:8300 since we have a leader: 10.242.134.70:8300 2021/05/07 23:00:30 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-1.dc1) 2021/05/07 23:00:30 [INFO] serf: EventMemberJoin: dev-consul-server-1 10.242.251.82 2021/05/07 23:00:30 [INFO] consul: Adding LAN server dev-consul-server-1 (Addr: tcp/10.242.251.82:8300) (DC: dc1) 2021/05/07 23:00:30 [INFO] serf: EventMemberJoin: dev-consul-server-1.dc1 10.242.251.82 2021/05/07 23:00:30 [INFO] consul: Handled member-join event for server "dev-consul-server-1.dc1" in area "wan" 2021/05/07 23:00:31 [WARN] memberlist: Refuting a suspect message (from: dev-consul-server-1) 2021/05/07 23:00:31 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-pgbp7 10.242.251.74 2021/05/07 23:00:31 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-pgbp7 10.242.251.74 2021/05/07 23:00:33 [INFO] serf: EventMemberLeave: dev-consul-7467bf7b46-pgbp7 10.242.251.74 2021/05/07 23:00:36 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/07 23:00:38 [INFO] serf: EventMemberFailed: dev-consul-server-1 10.242.251.82 2021/05/07 23:00:38 [INFO] consul: Removing LAN server dev-consul-server-1 (Addr: tcp/10.242.251.82:8300) (DC: dc1) 2021/05/07 23:00:39 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/07 23:00:48 [INFO] memberlist: Suspect dev-consul-server-1.dc1 has failed, no acks received 2021/05/07 23:01:15 [INFO] memberlist: Marking dev-consul-server-1.dc1 as failed, suspect timeout reached (0 peer confirmations) 2021/05/07 23:01:15 [INFO] serf: EventMemberFailed: dev-consul-server-1.dc1 10.242.251.82 2021/05/07 23:01:15 [INFO] consul: Handled member-failed event for server "dev-consul-server-1.dc1" in area "wan" 2021/05/07 23:01:18 [INFO] serf: attempting reconnect to dev-consul-server-1 10.242.251.82:8301 2021/05/07 23:01:20 [ERR] memberlist: Conflicting address for dev-consul-server-1.dc1. Mine: 10.242.251.82:8302 Theirs: 10.242.251.121:8302 2021/05/07 23:01:20 [WARN] serf: Name conflict for 'dev-consul-server-1.dc1' both 10.242.251.82:8302 and 10.242.251.121:8302 are claiming 2021/05/07 23:01:20 [INFO] serf: EventMemberJoin: dev-consul-server-1 10.242.251.121 2021/05/07 23:01:20 [INFO] consul: Adding LAN server dev-consul-server-1 (Addr: tcp/10.242.251.121:8300) (DC: dc1) 2021/05/07 23:01:20 [ERR] memberlist: Conflicting address for dev-consul-server-1.dc1. Mine: 10.242.251.82:8302 Theirs: 10.242.251.121:8302 2021/05/07 23:01:20 [WARN] serf: Name conflict for 'dev-consul-server-1.dc1' both 10.242.251.82:8302 and 10.242.251.121:8302 are claiming 2021/05/07 23:01:20 [ERR] memberlist: Conflicting address for dev-consul-server-1.dc1. Mine: 10.242.251.82:8302 Theirs: 10.242.251.121:8302 2021/05/07 23:01:20 [WARN] serf: Name conflict for 'dev-consul-server-1.dc1' both 10.242.251.82:8302 and 10.242.251.121:8302 are claiming 2021/05/07 23:01:21 [ERR] memberlist: Conflicting address for dev-consul-server-1.dc1. Mine: 10.242.251.82:8302 Theirs: 10.242.251.121:8302 2021/05/07 23:01:21 [WARN] serf: Name conflict for 'dev-consul-server-1.dc1' both 10.242.251.82:8302 and 10.242.251.121:8302 are claiming 2021/05/07 23:01:21 [ERR] memberlist: Conflicting address for dev-consul-server-1.dc1. Mine: 10.242.251.82:8302 Theirs: 10.242.251.121:8302 2021/05/07 23:01:21 [WARN] serf: Name conflict for 'dev-consul-server-1.dc1' both 10.242.251.82:8302 and 10.242.251.121:8302 are claiming 2021/05/07 23:01:21 [INFO] memberlist: Suspect dev-consul-server-1.dc1 has failed, no acks received 2021/05/07 23:01:28 [INFO] serf: attempting reconnect to dev-consul-server-1.dc1 10.242.251.82:8302 2021/05/07 23:02:24 [INFO] serf: EventMemberJoin: dev-consul-server-1.dc1 10.242.251.121 2021/05/07 23:02:24 [INFO] consul: Handled member-join event for server "dev-consul-server-1.dc1" in area "wan"