By type
+ CONSUL_BIND= + [ -n ] + CONSUL_CLIENT= + [ -n ] + CONSUL_DATA_DIR=/consul/data + CONSUL_CONFIG_DIR=/consul/config + [ -n ] + [ a = - ] + [ agent = agent ] + shift + set -- consul agent -data-dir=/consul/data -config-dir=/consul/config -client 0.0.0.0 -enable-script-checks -retry-join consul-server + [ consul = consul ] + stat -c %u /consul/data + id -u consul + [ 100 != 100 ] + stat -c %u /consul/config + id -u consul + [ 100 != 100 ] + [ ! -z ] + exec consul agent -data-dir=/consul/data -config-dir=/consul/config -client 0.0.0.0 -enable-script-checks -retry-join consul-server ==> Starting Consul agent... ==> Consul agent running! Version: 'v1.0.6' Node ID: 'f2d168dc-7b9b-289f-0bc3-3ba86c2a768f' Node name: 'dev-consul-7467bf7b46-z9mlm' Datacenter: 'dc1' (Segment: '') Server: false (Bootstrap: false) Client Addr: [0.0.0.0] (HTTP: 8500, HTTPS: -1, DNS: 8600) Cluster Addr: 10.242.1.149 (LAN: 8301, WAN: 8302) Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false ==> Log data will now stream in as it occurs: 2021/05/21 11:23:09 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-z9mlm 10.242.1.149 2021/05/21 11:23:09 [WARN] Service name "Health Check: Application Authorization Framework" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] Service name "Health Check: Secret Management Service (sms)" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] Service name "A&AI Synapse Data Routing Service" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] agent: check "data-router-process" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/21 11:23:09 [WARN] Service name "A&AI Model Loader" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] agent: check "model-loader-process" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/21 11:23:09 [WARN] Service name "A&AI Search Data Service" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] agent: check "elasticsearch-write-health" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/21 11:23:09 [WARN] agent: check "search-data-service-availability" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/21 11:23:09 [WARN] agent: check "search-data-service-api" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/21 11:23:09 [WARN] Service name "Active and Available Inventory" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] Service name "A&AI UI Backend Service" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] agent: check "sparky-be-process" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/21 11:23:09 [WARN] Service name "Health Check: APPC - Dgbuilder" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] Service name "Health Check: APPC" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] agent: check "appc-dbhost-healthcheck" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/21 11:23:09 [WARN] Service name "Health Check: APPC-SDN-CTL-DB-01" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] Service name "Health Check: APPC-SDN-CTL-DB-02" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] Service name "Health Check: APPC - SDN Host" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] Service name "Health Check: CLAMP" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] Service name "Health Check: CLAMP - MariaDb" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] agent: check "clamp-mariadb" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/21 11:23:09 [WARN] Service name "Health Check: ONAP CLI" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] Service name "Health Check: Log - Elastic Search" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] Service name "Health Check: Log - Kibana" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] Service name "Health Check: Log - Log Stash" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] Service name "Health Check: DMaaP" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] agent: check "mr-zookeeper" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/21 11:23:09 [WARN] agent: check "mr-kafka" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/21 11:23:09 [WARN] Service name "Health Check: MSB" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] Service name "Health Check: MULTICLOUD" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] Service name "Health Check: Policy" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] agent: check "Policy-mariadb-healthcheck" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/21 11:23:09 [WARN] Service name "Health Check: SDC" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] agent: check "sdc-fe-healthcheck" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/21 11:23:09 [WARN] agent: check "sdc-be-healthcheck" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/21 11:23:09 [WARN] agent: check "sdc-titan-healthcheck" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/21 11:23:09 [WARN] agent: check "sdc-cs-healthcheck" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/21 11:23:09 [WARN] Service name "Health Check: SDNC ODL Cluster" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] agent: check "sdnc-odl-cluster-healthcheck" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/21 11:23:09 [WARN] Service name "Health Check: SDNC - DB Host" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] agent: check "sdnc-dbhost-healthcheck" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/21 11:23:09 [WARN] Service name "Health Check: SDNC - DGBuilder" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] Service name "Health Check: SDNC" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] Service name "Health Check: SDNC-SDN-CTL-DB-01" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] Service name "Health Check: SDNC-SDN-CTL-DB-02" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] Service name "Health Check: SDNC - SDN Host" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] Service name "Health Check: SO" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] agent: check "so-api-healthcheck" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/21 11:23:09 [WARN] agent: check "so-camunda-healthcheck" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/21 11:23:09 [WARN] agent: check "so-jra-healthcheck" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/21 11:23:09 [WARN] Service name "Health Check: SO - MariaDb" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] agent: check "so-mariadb" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/21 11:23:09 [WARN] Service name "Health Check: VFC" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] Service name "Health Check: VID" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/21 11:23:09 [WARN] agent: check "vid-mariadb" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/21 11:23:09 [INFO] agent: Started DNS server 0.0.0.0:8600 (udp) 2021/05/21 11:23:09 [INFO] agent: Started DNS server 0.0.0.0:8600 (tcp) 2021/05/21 11:23:09 [INFO] agent: Started HTTP server on [::]:8500 (tcp) 2021/05/21 11:23:09 [INFO] agent: started state syncer 2021/05/21 11:23:09 [WARN] manager: No servers available 2021/05/21 11:23:09 [ERR] agent: failed to sync remote state: No known Consul servers 2021/05/21 11:23:09 [INFO] agent: Retry join LAN is supported for: aliyun aws azure digitalocean gce os scaleway softlayer 2021/05/21 11:23:09 [INFO] agent: Joining LAN cluster... 2021/05/21 11:23:09 [INFO] agent: (LAN) joining: [consul-server] 2021/05/21 11:23:09 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.142.201 2021/05/21 11:23:09 [INFO] serf: EventMemberJoin: dev-consul-server-0 10.242.1.135 2021/05/21 11:23:09 [INFO] consul: adding server dev-consul-server-2 (Addr: tcp/10.242.142.201:8300) (DC: dc1) 2021/05/21 11:23:09 [INFO] consul: adding server dev-consul-server-0 (Addr: tcp/10.242.1.135:8300) (DC: dc1) 2021/05/21 11:23:09 [INFO] agent: (LAN) joined: 2 Err:
2021/05/21 11:23:09 [INFO] agent: Join LAN completed. Synced with 2 initial agents 2021/05/21 11:23:09 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:10 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:11 [WARN] agent: Check "sdc-catalog-healthcheck" HTTP request failed: Get https://sdc-be:8443/asdc/v1/catalog/services: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:23:11 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:11 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:11 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:23:12 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:12 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:23:12 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:23:12 [WARN] agent: Check "mr-kafka" is now warning 2021/05/21 11:23:12 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:12 [INFO] agent: Synced service "Health Check: APPC - SDN Host" 2021/05/21 11:23:13 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-22bkc 10.242.129.72 2021/05/21 11:23:13 [INFO] agent: Synced service "Health Check: SDNC-SDN-CTL-DB-01" 2021/05/21 11:23:13 [INFO] agent: Synced service "Health Check: Secret Management Service (sms)" 2021/05/21 11:23:13 [INFO] agent: Synced service "A&AI UI Backend Service" 2021/05/21 11:23:13 [INFO] agent: Synced service "Health Check: APPC" 2021/05/21 11:23:13 [INFO] agent: Synced service "Health Check: SDNC-SDN-CTL-DB-02" 2021/05/21 11:23:13 [INFO] agent: Synced service "Health Check: Application Authorization Framework" 2021/05/21 11:23:13 [INFO] agent: Synced service "A&AI Synapse Data Routing Service" 2021/05/21 11:23:13 [INFO] agent: Synced service "Health Check: APPC - Dgbuilder" 2021/05/21 11:23:13 [INFO] agent: Synced service "Health Check: DMaaP" 2021/05/21 11:23:13 [INFO] agent: Synced service "Health Check: MSB" 2021/05/21 11:23:13 [INFO] agent: Synced service "Health Check: Policy" 2021/05/21 11:23:13 [INFO] agent: Synced service "Health Check: SDNC ODL Cluster" 2021/05/21 11:23:13 [INFO] agent: Synced service "Health Check: SDNC - DGBuilder" 2021/05/21 11:23:13 [INFO] agent: Synced service "Health Check: SO" 2021/05/21 11:23:13 [INFO] agent: Synced service "Health Check: VID" 2021/05/21 11:23:13 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:13 [INFO] agent: Synced service "A&AI Search Data Service" 2021/05/21 11:23:13 [INFO] agent: Synced service "Health Check: APPC-SDN-CTL-DB-01" 2021/05/21 11:23:13 [INFO] agent: Synced service "Health Check: APPC-SDN-CTL-DB-02" 2021/05/21 11:23:13 [INFO] agent: Synced service "Health Check: ONAP CLI" 2021/05/21 11:23:14 [INFO] agent: Synced service "Health Check: Log - Elastic Search" 2021/05/21 11:23:14 [INFO] agent: Synced service "Health Check: SDC" 2021/05/21 11:23:14 [INFO] agent: Synced service "Health Check: SDNC - SDN Host" 2021/05/21 11:23:14 [INFO] agent: Synced service "Health Check: SO - MariaDb" 2021/05/21 11:23:14 [INFO] agent: Synced service "Health Check: VFC" 2021/05/21 11:23:14 [INFO] agent: Synced service "Health Check: CLAMP" 2021/05/21 11:23:14 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:23:14 [INFO] agent: Synced service "Health Check: Log - Kibana" 2021/05/21 11:23:14 [INFO] agent: Synced service "Health Check: MULTICLOUD" 2021/05/21 11:23:14 [INFO] agent: Synced service "Health Check: SDNC" 2021/05/21 11:23:14 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: net/http: request canceled (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:23:14 [INFO] agent: Synced service "A&AI Model Loader" 2021/05/21 11:23:14 [INFO] agent: Synced service "Health Check: Log - Log Stash" 2021/05/21 11:23:14 [INFO] agent: Synced service "Health Check: SDNC - DB Host" 2021/05/21 11:23:14 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:23:14 [INFO] agent: Synced service "Active and Available Inventory" 2021/05/21 11:23:14 [INFO] agent: Synced service "Health Check: CLAMP - MariaDb" 2021/05/21 11:23:14 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:15 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:23:15 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:15 [INFO] agent: Synced check "clamp-mariadb" 2021/05/21 11:23:15 [INFO] agent: Synced check "vfc-vnfres" 2021/05/21 11:23:15 [INFO] agent: Synced check "aai-resources" 2021/05/21 11:23:15 [INFO] agent: Synced check "vid-mariadb" 2021/05/21 11:23:15 [INFO] agent: Synced check "aai-traversal" 2021/05/21 11:23:15 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:15 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:15 [INFO] agent: Synced check "msb-discovery" 2021/05/21 11:23:15 [INFO] agent: Synced check "vfc-nslcm" 2021/05/21 11:23:15 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:23:15 [INFO] agent: Synced check "appc-dbhost-healthcheck" 2021/05/21 11:23:15 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:15 [INFO] agent: Synced check "sdnc-odl-cluster-healthcheck" 2021/05/21 11:23:15 [INFO] agent: Synced check "aaf-sms-health" 2021/05/21 11:23:16 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:16 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:16 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:16 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:23:16 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:23:17 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:17 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:23:17 [INFO] agent: Synced check "vfc-ztevnfmdriver" 2021/05/21 11:23:17 [INFO] agent: Synced check "multicloud-vio" 2021/05/21 11:23:17 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:23:17 [INFO] agent: Synced check "so-mariadb" 2021/05/21 11:23:17 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:17 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:17 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:17 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:17 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:17 [ERR] memberlist: Failed fallback ping: read tcp 10.242.1.149:48028->10.242.129.72:8301: i/o timeout 2021/05/21 11:23:17 [INFO] memberlist: Suspect dev-consul-7467bf7b46-22bkc has failed, no acks received 2021/05/21 11:23:17 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:23:17 [INFO] agent: Synced check "vfc-gvnfmdriver" 2021/05/21 11:23:18 [INFO] memberlist: Marking dev-consul-7467bf7b46-22bkc as failed, suspect timeout reached (2 peer confirmations) 2021/05/21 11:23:18 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-22bkc 10.242.129.72 2021/05/21 11:23:18 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:18 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:23:18 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:19 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:19 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:23:19 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:23:19 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:23:19 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:19 [INFO] agent: Synced check "mr-zookeeper" 2021/05/21 11:23:19 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:19 [INFO] agent: Synced check "sdnc-dbhost-healthcheck" 2021/05/21 11:23:19 [INFO] agent: Synced check "Policy-mariadb-healthcheck" 2021/05/21 11:23:19 [INFO] agent: Synced check "msb-consul" 2021/05/21 11:23:19 [INFO] agent: Synced check "vfc-vnfmgr" 2021/05/21 11:23:19 [ERR] memberlist: Failed fallback ping: read tcp 10.242.1.149:48044->10.242.129.72:8301: i/o timeout 2021/05/21 11:23:19 [INFO] memberlist: Suspect dev-consul-7467bf7b46-22bkc has failed, no acks received 2021/05/21 11:23:19 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:20 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:20 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:20 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:20 [INFO] agent: Synced check "vfc-vnflcm" 2021/05/21 11:23:20 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:23:20 [INFO] agent: Synced check "search-data-service-api" 2021/05/21 11:23:20 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:21 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:21 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:23:21 [INFO] agent: Synced check "data-router-process" 2021/05/21 11:23:22 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:23:22 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:22 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:23:22 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:22 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:22 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:22 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:23:23 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:23:23 [INFO] agent: Synced check "aai-service" 2021/05/21 11:23:23 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:23 [INFO] agent: Synced check "dmaap" 2021/05/21 11:23:23 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:23:23 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:23 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:23:23 [INFO] agent: Synced check "sparky-be-process" 2021/05/21 11:23:23 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:23:24 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:24 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: net/http: request canceled (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:23:24 [INFO] agent: Synced check "model-loader-process" 2021/05/21 11:23:24 [INFO] agent: Synced check "multicloud-pike" 2021/05/21 11:23:24 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:23:24 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:23:24 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:24 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:35318->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:23:25 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:23:25 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:23:25 [INFO] agent: Synced check "aaf-service" 2021/05/21 11:23:25 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:25 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:23:26 [INFO] agent: Synced check "search-data-service-availability" 2021/05/21 11:23:26 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:26 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:23:26 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:26 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:27 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:27 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:23:27 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:27 [WARN] agent: Check "sdc-catalog-healthcheck" HTTP request failed: Get https://sdc-be:8443/asdc/v1/catalog/services: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:23:27 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:23:28 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:23:29 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:23:29 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:23:29 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:29 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:29 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:29 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:30 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:23:30 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:30 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:23:30 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:30 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:30 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:31 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:31 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving ==> Failed to check for updates: Get https://checkpoint-api.hashicorp.com/v1/check/consul?arch=amd64&os=linux&signature=d9ebeb71-04b9-dfc8-36bc-25a6e96a66ab&version=1.0.6: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:23:32 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:32 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:23:32 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:32 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:32 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-22bkc 10.242.129.72 2021/05/21 11:23:32 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:32 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:33 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:23:33 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:33 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:23:34 [INFO] serf: EventMemberJoin: dev-consul-server-1 10.242.129.75 2021/05/21 11:23:34 [INFO] consul: adding server dev-consul-server-1 (Addr: tcp/10.242.129.75:8300) (DC: dc1) 2021/05/21 11:23:34 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:23:34 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:23:34 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:34 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:34 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:34 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:34 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:23:34 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:23:35 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:35 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:23:35 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:35 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:35 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:23:35 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:35 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:23:36 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:36 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:23:36 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:36 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:23:37 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:37 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:37 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:23:37 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:37 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:37 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:38 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:23:38 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:38 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:23:38 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:38 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:23:38 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:23:39 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:23:39 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:23:39 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:39 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:23:39 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:23:39 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:23:39 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:39 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:35582->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:23:40 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:40 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:41 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:23:41 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:23:41 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:41 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:42 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:42 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:23:42 [WARN] agent: Check "mr-kafka" is now warning 2021/05/21 11:23:43 [WARN] agent: Check "elasticsearch-write-health" is now warning 2021/05/21 11:23:43 [WARN] agent: Check "sdc-catalog-healthcheck" HTTP request failed: Get https://sdc-be:8443/asdc/v1/catalog/services: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:23:44 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:44 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:44 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:23:44 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:44 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:23:45 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:23:45 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:23:45 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:23:45 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:45 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:23:45 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:45 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:45 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:23:45 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:46 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:46 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:46 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:46 [ERR] memberlist: Failed fallback ping: read tcp 10.242.1.149:54230->10.242.129.75:8301: i/o timeout 2021/05/21 11:23:46 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/21 11:23:46 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:23:47 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:47 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:23:47 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:47 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:47 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:47 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:47 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:48 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:48 [INFO] agent: Synced check "elasticsearch-write-health" 2021/05/21 11:23:48 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:23:49 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:23:49 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:49 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:49 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:49 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:23:49 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:23:49 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:49 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:23:49 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:23:50 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:50 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:50 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:50 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:50 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:23:50 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:51 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:51 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:23:52 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:52 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:52 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:23:52 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:23:52 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:52 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:53 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:53 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:53 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:23:54 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:23:54 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:23:54 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:54 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:54 [ERR] memberlist: Failed fallback ping: read tcp 10.242.1.149:54362->10.242.129.75:8301: i/o timeout 2021/05/21 11:23:54 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/21 11:23:54 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:35830->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:23:55 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:23:55 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:23:55 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:56 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:56 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:57 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:23:58 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:23:58 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:23:58 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:58 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:23:58 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:23:58 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:58 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:23:58 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:23:58 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:23:58 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:23:59 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:23:59 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:23:59 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:23:59 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:24:00 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:00 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:24:02 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:02 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:24:03 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:24:03 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:03 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:03 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:03 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:03 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:03 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:03 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:03 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:03 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:03 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:03 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:03 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:03 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:03 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:04 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:24:04 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:04 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:04 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:04 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:04 [WARN] agent: Check "sdc-catalog-healthcheck" HTTP request failed: Get https://sdc-be:8443/asdc/v1/catalog/services: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:24:05 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:05 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:05 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:24:05 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:05 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:24:05 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:05 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:24:06 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:06 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:24:06 [ERR] memberlist: Failed fallback ping: read tcp 10.242.1.149:54550->10.242.129.75:8301: i/o timeout 2021/05/21 11:24:06 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/21 11:24:07 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:07 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:07 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:08 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:08 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:08 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:24:08 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:08 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:08 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:24:08 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:24:08 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:24:08 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:24:08 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:24:08 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:24:08 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:08 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:24:09 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:24:09 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:09 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:24:09 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:24:09 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:09 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:36052->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:24:10 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:24:10 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:10 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:10 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:24:10 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:24:11 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:24:11 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:24:11 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:11 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:24:11 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:12 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:12 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:24:12 [WARN] agent: Check "mr-kafka" is now warning 2021/05/21 11:24:12 [ERR] memberlist: Failed fallback ping: read tcp 10.242.1.149:48932->10.242.129.72:8301: i/o timeout 2021/05/21 11:24:12 [INFO] memberlist: Suspect dev-consul-7467bf7b46-22bkc has failed, no acks received 2021/05/21 11:24:13 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:24:14 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:24:16 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-22bkc 10.242.129.72 2021/05/21 11:24:16 [WARN] memberlist: Was able to connect to dev-consul-server-1 but other probes failed, network may be misconfigured 2021/05/21 11:24:18 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:18 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:18 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:18 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:18 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:18 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:18 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:18 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:18 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:18 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:18 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:18 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:18 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:18 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:18 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:24:18 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:18 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:18 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:18 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:24:18 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:24:18 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:24:18 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:24:18 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:24:18 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:24:19 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:19 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:19 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:19 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:24:19 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:24:19 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:19 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:24:19 [ERR] memberlist: Failed fallback ping: read tcp 10.242.1.149:54732->10.242.129.75:8301: i/o timeout 2021/05/21 11:24:19 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/21 11:24:20 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:20 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:20 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:20 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:20 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: net/http: request canceled (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:24:20 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:24:20 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:21 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:21 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:24:21 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:24:21 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:24:21 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:24:21 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:24:22 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:22 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:22 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:24:22 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:22 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:22 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:24:23 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:23 [ERR] memberlist: Failed fallback ping: read tcp 10.242.1.149:54834->10.242.129.75:8301: i/o timeout 2021/05/21 11:24:23 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/21 11:24:23 [INFO] memberlist: Marking dev-consul-server-1 as failed, suspect timeout reached (2 peer confirmations) 2021/05/21 11:24:23 [INFO] serf: EventMemberFailed: dev-consul-server-1 10.242.129.75 2021/05/21 11:24:23 [INFO] consul: removing server dev-consul-server-1 (Addr: tcp/10.242.129.75:8300) (DC: dc1) 2021/05/21 11:24:23 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:23 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:24:24 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:24:24 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:24 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:24 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:36306->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:24:25 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:25 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:24:26 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:24:26 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:24:26 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:26 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:27 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:24:28 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:28 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:28 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:24:28 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:24:28 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:28 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:24:28 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:24:28 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:24:28 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:24:28 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:24:29 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:24:29 [INFO] serf: EventMemberJoin: dev-consul-server-1 10.242.129.75 2021/05/21 11:24:29 [INFO] consul: adding server dev-consul-server-1 (Addr: tcp/10.242.129.75:8300) (DC: dc1) 2021/05/21 11:24:29 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:24:29 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:24:30 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:30 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-22bkc 10.242.129.72 2021/05/21 11:24:31 [WARN] memberlist: Was able to connect to dev-consul-server-1 but other probes failed, network may be misconfigured 2021/05/21 11:24:32 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:32 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:24:32 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:24:32 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:24:32 [ERR] memberlist: Failed fallback ping: read tcp 10.242.1.149:49246->10.242.129.72:8301: i/o timeout 2021/05/21 11:24:32 [INFO] memberlist: Suspect dev-consul-7467bf7b46-22bkc has failed, no acks received 2021/05/21 11:24:33 [WARN] memberlist: Refuting a suspect message (from: dev-consul-7467bf7b46-22bkc) 2021/05/21 11:24:33 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:33 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:33 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:33 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:33 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:33 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:33 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:33 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:33 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:33 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:33 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:33 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:33 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:33 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:33 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:24:34 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:34 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:34 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:34 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:34 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:24:35 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:35 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:35 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:35 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:24:35 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:24:35 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:36 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:36 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:24:36 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:24:37 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:24:37 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:37 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:37 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:24:37 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:38 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:38 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:38 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:38 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:38 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:24:38 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:24:38 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:24:38 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:24:38 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:24:38 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:38 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:24:38 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:24:38 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:24:39 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:24:39 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:39 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:24:39 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:24:39 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:39 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:36564->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:24:40 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:40 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:41 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:24:41 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:42 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:42 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:24:42 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:42 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:24:42 [WARN] agent: Check "mr-kafka" is now warning 2021/05/21 11:24:43 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:24:43 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:24:43 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:24:44 [INFO] serf: EventMemberLeave: dev-consul-7467bf7b46-22bkc 10.242.129.72 2021/05/21 11:24:44 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:24:45 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:24:45 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/21 11:24:47 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:24:48 [WARN] agent: Check "elasticsearch-write-health" is now warning 2021/05/21 11:24:48 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:48 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:48 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:48 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:48 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:48 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:48 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:48 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:48 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:48 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:48 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:48 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:48 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:48 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:48 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:48 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:48 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:48 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:24:48 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:24:48 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:24:48 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:24:48 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:24:48 [INFO] memberlist: Suspect dev-consul-server-1 has failed, no acks received 2021/05/21 11:24:48 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:24:48 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:24:49 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:49 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:49 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:49 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:24:49 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:24:49 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:49 [INFO] memberlist: Marking dev-consul-server-1 as failed, suspect timeout reached (2 peer confirmations) 2021/05/21 11:24:49 [INFO] serf: EventMemberFailed: dev-consul-server-1 10.242.129.75 2021/05/21 11:24:49 [INFO] consul: removing server dev-consul-server-1 (Addr: tcp/10.242.129.75:8300) (DC: dc1) 2021/05/21 11:24:49 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:24:50 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:50 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:50 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:50 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:50 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:24:50 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:24:50 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:51 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:51 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:24:52 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:52 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:52 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:24:52 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:52 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:53 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:24:53 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:53 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:53 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:24:53 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:24:54 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:24:54 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:54 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:24:54 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:24:54 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:54 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:36870->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:24:55 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:55 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:24:56 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:24:56 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:57 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:58 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:24:58 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:24:58 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:58 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:58 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:24:58 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:24:58 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:24:58 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:24:58 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:24:58 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:24:59 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:24:59 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:24:59 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:24:59 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:24:59 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:25:00 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:01 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:25:02 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:02 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:25:03 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:03 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:03 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:03 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:03 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:03 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:03 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:03 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:03 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:03 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:03 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:03 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:03 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:03 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:04 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:04 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:04 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:04 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:04 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:25:05 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:05 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:05 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:05 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:25:05 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:25:05 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:05 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:25:05 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:25:06 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:06 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:25:07 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:25:07 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:07 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:07 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:08 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:08 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:08 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:08 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:08 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:25:08 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:25:08 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:25:08 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:25:08 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:25:08 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:08 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:25:09 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:25:09 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:25:09 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:25:09 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:25:09 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:09 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:25:09 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:25:09 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:25:09 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:09 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:37164->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:25:09 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:25:10 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:10 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:11 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:25:11 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:12 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:12 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:12 [WARN] agent: Check "mr-kafka" is now warning 2021/05/21 11:25:12 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:25:14 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:25:15 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:25:15 [ERR] memberlist: Conflicting address for dev-consul-server-1. Mine: 10.242.129.75:8301 Theirs: 10.242.129.99:8301 2021/05/21 11:25:15 [WARN] serf: Name conflict for 'dev-consul-server-1' both 10.242.129.75:8301 and 10.242.129.99:8301 are claiming 2021/05/21 11:25:15 [ERR] memberlist: Conflicting address for dev-consul-server-1. Mine: 10.242.129.75:8301 Theirs: 10.242.129.99:8301 2021/05/21 11:25:15 [WARN] serf: Name conflict for 'dev-consul-server-1' both 10.242.129.75:8301 and 10.242.129.99:8301 are claiming 2021/05/21 11:25:16 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:25:16 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:25:17 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:25:18 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:25:18 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:18 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:18 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:18 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:18 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:18 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:18 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:18 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:18 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:18 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:18 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:18 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:18 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:18 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:18 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:18 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:18 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:18 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:25:18 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:25:18 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:25:18 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:25:18 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:25:19 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:25:19 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:25:19 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:19 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:19 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:19 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:25:19 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:25:19 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:19 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:25:20 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:20 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:20 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:20 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:20 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:25:20 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:25:20 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:25:20 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:21 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:21 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:25:22 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:22 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:22 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:22 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:25:22 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:23 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:23 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:23 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:25:24 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:25:24 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:24 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:24 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:37454->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:25:25 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:25:25 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:25 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:25:26 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:27 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:27 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:25:27 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:25:27 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:25:28 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:28 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:28 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:28 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:25:28 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:25:28 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:25:28 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:25:28 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:25:29 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:25:29 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:25:29 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:25:29 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:25:29 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:25:30 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:25:30 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:31 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:25:31 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:25:32 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:32 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:25:33 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:25:33 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:33 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:33 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:33 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:33 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:33 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:33 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:33 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:33 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:33 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:33 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:33 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:33 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:33 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:34 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:34 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:34 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:34 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:34 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:25:35 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:35 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:35 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:35 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:25:35 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:25:35 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:36 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:36 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:25:37 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:37 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:37 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:38 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:38 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:25:38 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:25:38 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:38 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:38 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:38 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:38 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:25:38 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:25:38 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:25:38 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:25:38 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:25:38 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:25:39 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:25:39 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:25:39 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:25:39 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:39 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:25:39 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:25:39 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:39 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:37724->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:25:40 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:25:40 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:40 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:41 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:25:41 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:25:41 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:42 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:42 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:42 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:25:42 [WARN] agent: Check "mr-kafka" is now warning 2021/05/21 11:25:42 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:25:43 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:25:46 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:25:47 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:25:48 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:48 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:48 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:48 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:48 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:48 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:48 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:48 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:48 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:48 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:48 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:48 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:48 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:48 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:48 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:48 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:48 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:48 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:25:48 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:25:48 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:25:48 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:25:48 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:25:49 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:25:49 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:25:49 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:25:49 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:49 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:49 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:49 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:25:49 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:25:49 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:49 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:25:49 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:25:49 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:25:50 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:50 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:50 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:50 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:50 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:25:50 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:25:50 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:51 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:25:51 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:51 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:25:52 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:52 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:52 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:52 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:25:52 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:53 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:25:53 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:53 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:53 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:25:54 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:25:54 [WARN] agent: Check "elasticsearch-write-health" is now warning 2021/05/21 11:25:54 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:54 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:54 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:37988->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:25:55 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:56 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:57 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:57 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:25:57 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:25:58 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:25:58 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:58 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:58 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:25:58 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:25:58 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:25:58 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:25:58 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:25:58 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:25:59 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:25:59 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:25:59 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:25:59 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:25:59 [INFO] agent: Synced check "policy-nexus-local-status" 2021/05/21 11:26:00 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:00 [INFO] agent: Synced check "multicloud-windriver" 2021/05/21 11:26:00 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:26:00 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:26:01 [INFO] agent: Synced check "vfc-ztesdncdriver" 2021/05/21 11:26:02 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:26:02 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:26:02 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:02 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:26:03 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:26:03 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:03 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:03 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:03 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:03 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:03 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:03 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:03 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:03 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:03 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:03 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:03 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:03 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:03 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:04 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:26:04 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:04 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:04 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:04 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:05 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:26:05 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:05 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:05 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:26:05 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:05 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:26:05 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:26:05 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:06 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:06 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:26:07 [INFO] serf: EventMemberJoin: dev-consul-server-1 10.242.129.99 2021/05/21 11:26:07 [INFO] consul: adding server dev-consul-server-1 (Addr: tcp/10.242.129.99:8300) (DC: dc1) 2021/05/21 11:26:07 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:07 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:07 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:08 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:08 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:08 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:08 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:08 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:08 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:26:08 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:26:08 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:26:08 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:26:08 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:26:09 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:26:09 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:26:09 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:26:09 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:26:09 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:09 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:09 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:38212->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:26:09 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:26:09 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:26:10 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:10 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:11 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:26:11 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:26:11 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:12 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:12 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:12 [WARN] agent: Check "mr-kafka" is now warning 2021/05/21 11:26:12 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:26:13 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:26:13 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:26:13 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:26:13 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:26:15 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:26:16 [INFO] agent: Synced check "appc-sdnctldb01" 2021/05/21 11:26:18 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:26:18 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:18 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:18 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:18 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:18 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:18 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:18 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:18 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:18 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:18 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:18 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:18 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:18 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:18 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:18 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:18 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:18 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:18 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:26:18 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:26:18 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:26:18 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:26:18 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:26:19 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:26:19 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:26:19 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:26:19 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:19 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:19 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:19 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:19 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:26:19 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:26:20 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:26:20 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:20 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:20 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:20 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:20 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:26:20 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:20 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:26:21 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:26:21 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:21 [INFO] agent: Synced check "framework" 2021/05/21 11:26:21 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:26:22 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:22 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:22 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:22 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:26:22 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:26:22 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:26:22 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:23 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:23 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:24 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:26:24 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:26:24 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:26:24 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:24 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:24 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:38440->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:26:25 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:25 [INFO] agent: Synced check "elasticsearch" 2021/05/21 11:26:26 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:26:26 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:27 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:28 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:26:28 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:28 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:28 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:28 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:26:28 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:26:28 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:26:28 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:26:28 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:26:29 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:26:29 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:26:29 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:26:29 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:26:29 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:26:29 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:26:30 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:31 [INFO] agent: Synced check "sdc-fe-healthcheck" 2021/05/21 11:26:31 [INFO] agent: Synced check "aai-service" 2021/05/21 11:26:32 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:32 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:26:33 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:26:33 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:26:33 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:33 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:33 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:33 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:33 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:33 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:33 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:33 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:33 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:33 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:33 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:33 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:33 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:33 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:34 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:26:34 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:34 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:34 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:34 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:35 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:26:35 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:26:35 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:26:35 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:35 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:35 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:35 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:26:35 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:35 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:26:36 [INFO] agent: Synced check "log-logstash-internal-server-plugin-info" 2021/05/21 11:26:36 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:36 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:26:37 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:26:37 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:37 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:26:37 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:37 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:38 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:38 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:38 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:38 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:38 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:38 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:26:38 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:26:38 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:26:39 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:26:39 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:26:39 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:26:39 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:26:39 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:26:39 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:26:39 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:39 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:39 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:38664->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:26:39 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:26:39 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:26:40 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:40 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:41 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:42 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:42 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:42 [WARN] agent: Check "mr-kafka" is now warning 2021/05/21 11:26:43 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:26:43 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:26:44 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:26:44 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:26:45 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:26:45 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:26:46 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:26:48 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:26:48 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:48 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:48 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:48 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:48 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:48 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:48 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:48 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:48 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:48 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:48 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:48 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:48 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:48 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:48 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:48 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:48 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:49 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:26:49 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:26:49 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:26:49 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:26:49 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:26:49 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:26:49 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:26:49 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:49 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:49 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:49 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:49 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:26:49 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:26:50 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:26:50 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:26:50 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:50 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:50 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:50 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:50 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:26:50 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:50 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:26:51 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:26:51 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:51 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:26:52 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:52 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:52 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:52 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:53 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:26:53 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:26:53 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:53 [INFO] agent: Synced check "so-jra-healthcheck" 2021/05/21 11:26:53 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:54 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:26:54 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:26:54 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:54 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:54 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:38880->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:26:55 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:55 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:26:55 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:26:56 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:57 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:26:57 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:58 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:26:58 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:58 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:58 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:26:59 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:26:59 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:26:59 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:26:59 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:26:59 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:26:59 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:26:59 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:26:59 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:26:59 [WARN] agent: Check "elasticsearch-write-health" is now warning 2021/05/21 11:26:59 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:26:59 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:27:00 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:01 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:27:01 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:27:02 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:03 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:27:03 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:03 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:03 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:03 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:03 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:03 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:03 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:03 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:03 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:03 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:03 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:03 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:03 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:03 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:04 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:04 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:04 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:04 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:05 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:05 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:27:05 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:05 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:05 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:27:05 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:05 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:27:06 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:27:06 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:06 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:27:06 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:27:06 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:27:07 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:27:07 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:07 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:07 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:08 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:27:08 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:08 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:08 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:08 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:08 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:09 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:27:09 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:27:09 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:27:09 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:27:09 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:27:09 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:27:09 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:27:09 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:27:09 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:09 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:27:09 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:27:09 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:09 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:39102->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:27:09 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:27:09 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:27:10 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:10 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:10 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:27:10 [INFO] agent: Synced check "sdnc-dgbuilder" 2021/05/21 11:27:11 [INFO] agent: Synced check "vfc-nokiav2vnfmdriver" 2021/05/21 11:27:11 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:12 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:12 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:12 [WARN] agent: Check "mr-kafka" is now warning 2021/05/21 11:27:13 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:27:13 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:27:15 [INFO] agent: Synced check "multicloud-starlingx" 2021/05/21 11:27:17 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:27:17 [INFO] agent: Synced check "aai-resources" 2021/05/21 11:27:17 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:27:17 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:27:17 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:27:18 [INFO] agent: Synced check "sdnc-sdnhost" 2021/05/21 11:27:18 [INFO] agent: Synced check "drools" 2021/05/21 11:27:18 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:18 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:18 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:18 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:18 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:18 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:18 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:18 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:18 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:18 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:18 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:18 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:18 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:18 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:18 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:18 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:18 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:19 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:27:19 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:27:19 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:27:19 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:27:19 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:27:19 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:27:19 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:19 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:19 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:19 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:27:19 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:27:19 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:19 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:27:19 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:27:20 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:20 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:27:20 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:20 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:20 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:20 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:27:20 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:20 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:27:21 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:21 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:27:21 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:27:22 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:27:22 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:22 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:22 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:23 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:23 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:27:23 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:27:23 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:23 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:24 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:27:24 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:27:24 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:24 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:39322->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:27:24 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:25 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:27:25 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:26 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:27 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:28 [INFO] agent: Synced check "log-logstash-internal-server-os-info" 2021/05/21 11:27:28 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:27:28 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:27:28 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:27:28 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:28 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:28 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:29 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:27:29 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:27:29 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:27:29 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:27:29 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:27:29 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:27:29 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:27:29 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:27:30 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:27:30 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:27:30 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:32 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:32 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:27:33 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:27:33 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:27:33 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:27:33 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:33 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:33 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:33 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:33 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:33 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:33 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:33 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:33 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:33 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:33 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:33 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:33 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:33 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:34 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:34 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:34 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:34 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:35 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:35 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:35 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:27:35 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:35 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:27:35 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:35 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:27:36 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:36 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:27:37 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:37 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:38 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:38 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:27:38 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:39 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:39 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:39 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:39 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:39 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:27:39 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:27:39 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:27:39 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:27:39 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:27:39 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:27:39 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:27:39 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:27:39 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:39 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:27:39 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:27:39 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:27:39 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:27:39 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:39544->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:27:39 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:40 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:27:40 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:27:40 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:40 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:41 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:27:41 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:27:41 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:42 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:42 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:42 [WARN] agent: Check "mr-kafka" is now warning 2021/05/21 11:27:43 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:27:43 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:27:43 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:27:48 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:48 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:48 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:48 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:48 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:48 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:48 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:48 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:48 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:48 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:48 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:48 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:48 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:48 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:49 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:49 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:49 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:49 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:27:49 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:27:49 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:27:49 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:27:49 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:27:49 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:27:49 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:49 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:49 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:49 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:27:49 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:27:49 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:49 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:27:50 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:27:50 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:27:50 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:50 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:50 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:27:50 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:50 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:50 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:27:50 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:27:50 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:27:50 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:50 [INFO] agent: Synced check "odl-api-healthcheck" 2021/05/21 11:27:51 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:27:51 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:51 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:27:52 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:27:52 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:52 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:52 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:53 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:53 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:27:53 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:54 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:54 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:27:54 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:27:54 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:27:54 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:27:54 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:54 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:39760->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:27:54 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:55 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:27:55 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:56 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:57 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:57 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:27:57 [INFO] agent: Synced check "so-api-healthcheck" 2021/05/21 11:27:58 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:27:59 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:59 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:59 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:27:59 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:27:59 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:27:59 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:27:59 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:27:59 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:27:59 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:27:59 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:28:00 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:28:00 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:28:00 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:01 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:28:01 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:28:02 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:02 [INFO] agent: Synced check "log-logstash-internal-server-process-stat" 2021/05/21 11:28:03 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:28:03 [INFO] agent: Synced check "sdc-titan-healthcheck" 2021/05/21 11:28:03 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:28:03 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:03 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:03 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:03 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:03 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:03 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:03 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:03 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:03 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:03 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:03 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:03 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:03 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:03 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:04 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:04 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:04 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:04 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:05 [INFO] agent: Synced check "sdc-catalog-healthcheck" 2021/05/21 11:28:05 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:05 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:05 [WARN] agent: Check "elasticsearch-write-health" is now warning 2021/05/21 11:28:05 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:28:05 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:28:05 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:05 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:28:05 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:28:05 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:05 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:28:06 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:28:06 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:06 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:28:07 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:07 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:08 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:08 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:09 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:09 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:09 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:09 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:09 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:28:09 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:28:09 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:28:09 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:28:09 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:28:09 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:28:09 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:28:09 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:09 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:28:09 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:28:09 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:39968->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:28:09 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:10 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:28:10 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:28:10 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:28:10 [INFO] agent: Synced check "log-logstash-internal-server-node-stat" 2021/05/21 11:28:10 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:10 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:11 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:28:11 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:12 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:12 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:12 [WARN] agent: Check "mr-kafka" is now warning 2021/05/21 11:28:12 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:28:12 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:28:13 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:28:13 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:28:13 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:28:14 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:28:15 [INFO] agent: Synced check "appc-sdnhost" 2021/05/21 11:28:16 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:28:17 [INFO] agent: Synced check "sdc-be-healthcheck" 2021/05/21 11:28:17 [INFO] agent: Synced check "msb-eag" 2021/05/21 11:28:18 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:18 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:18 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:18 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:18 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:18 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:18 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:18 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:18 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:18 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:18 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:18 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:18 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:18 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:19 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:19 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:19 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:19 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:28:19 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:28:19 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:28:19 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:28:19 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:28:19 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:19 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:19 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:19 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:19 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:28:19 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:28:20 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:28:20 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:28:20 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:20 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:20 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:28:20 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:20 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:20 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:28:20 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:21 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:28:21 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:28:21 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:21 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:28:21 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:28:22 [INFO] agent: Synced check "log-logstash-internal-server-gi" 2021/05/21 11:28:22 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:22 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:22 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:23 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:23 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:28:23 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:23 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:28:23 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:28:24 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:24 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:28:24 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:28:24 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:24 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:40188->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:28:24 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:25 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:28:25 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:26 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:28:26 [INFO] agent: Synced check "clamp-health" 2021/05/21 11:28:26 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:27 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:28:27 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:27 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:28:28 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:28:29 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:29 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:29 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:29 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:28:29 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:28:29 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:28:29 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:28:29 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:28:29 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:28:29 [INFO] agent: Synced check "vfc-catalog" 2021/05/21 11:28:29 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:28:29 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:28:30 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:28:30 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:28:30 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:32 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:33 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:28:33 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:33 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:33 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:33 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:33 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:33 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:33 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:33 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:33 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:33 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:33 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:33 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:33 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:33 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:34 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:34 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:34 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:34 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:34 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:28:34 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:28:35 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:35 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:35 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:28:35 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:35 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:28:35 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:36 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:28:36 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:28:36 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:36 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:28:37 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:28:37 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:37 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:37 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:28:38 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:38 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:28:38 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:39 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:39 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:39 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:39 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:39 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:28:39 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:28:39 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:28:39 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:28:39 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:28:39 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:28:39 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:28:39 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:39 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:28:39 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:28:39 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:40424->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:28:39 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:40 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:28:40 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:28:40 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:40 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:41 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:42 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:28:42 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:42 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:42 [WARN] agent: Check "mr-kafka" is now warning 2021/05/21 11:28:43 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:28:43 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:28:43 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:28:45 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:28:45 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:28:45 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:28:47 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:28:48 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:48 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:48 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:48 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:48 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:48 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:48 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:48 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:48 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:48 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:48 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:48 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:48 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:48 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:49 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:49 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:49 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:49 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:28:49 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:28:49 [INFO] agent: Synced check "log-logstash-tcp" 2021/05/21 11:28:49 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:28:49 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:28:49 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:28:49 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:28:49 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:49 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:49 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:49 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:49 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:28:49 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:28:50 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:28:50 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:28:50 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:50 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:50 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:50 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:28:50 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:50 [INFO] agent: Synced check "vid-server" 2021/05/21 11:28:50 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:28:50 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:51 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:28:51 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:51 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:28:52 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:52 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:52 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:53 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:53 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:28:53 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:28:53 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:53 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:28:54 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:54 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:28:54 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:28:54 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:54 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:40758->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:28:54 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:55 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:55 [INFO] agent: Synced check "sdnctldb01" 2021/05/21 11:28:56 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:28:56 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:28:56 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:57 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:58 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:28:58 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:28:58 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:28:59 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:59 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:59 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:28:59 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:28:59 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:28:59 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:28:59 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:28:59 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:28:59 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:28:59 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:28:59 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:29:00 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:29:00 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:29:00 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:00 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:29:01 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:29:02 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:03 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:29:03 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:03 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:03 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:03 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:03 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:03 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:03 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:03 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:03 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:03 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:03 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:03 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:03 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:03 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:04 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:04 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:04 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:04 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:04 [INFO] agent: Synced check "log-elasticsearch-server" 2021/05/21 11:29:05 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:05 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:05 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:29:05 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:05 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:29:05 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:06 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:29:06 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:06 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:29:07 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:07 [INFO] agent: Synced check "sdnctldb02" 2021/05/21 11:29:07 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:07 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:29:07 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:29:08 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:08 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:09 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:09 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:09 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:09 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:09 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:29:09 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:29:09 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:29:09 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:29:09 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:29:09 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:29:09 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:29:09 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:29:09 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:29:09 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:09 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:29:09 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:41020->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:29:09 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:09 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:29:10 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:29:10 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:29:10 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:29:10 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:10 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:10 [WARN] agent: Check "elasticsearch-write-health" is now warning 2021/05/21 11:29:11 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:29:11 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:12 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:12 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:12 [WARN] agent: Check "mr-kafka" is now warning 2021/05/21 11:29:13 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:29:13 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:29:14 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:29:15 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:29:17 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:29:18 [INFO] agent: Synced check "msb-iag" 2021/05/21 11:29:18 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:29:18 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:29:18 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:18 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:18 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:18 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:18 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:18 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:18 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:18 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:18 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:18 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:18 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:18 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:18 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:18 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:19 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:19 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:19 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:19 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:29:19 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:29:19 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:29:19 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:29:19 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:29:19 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:19 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:19 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:19 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:20 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:29:20 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:29:20 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:29:20 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:29:20 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:29:20 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:20 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:20 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:20 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:20 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:29:20 [INFO] agent: Synced check "log-elasticsearch-tcp" 2021/05/21 11:29:20 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:29:20 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:21 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:29:21 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:21 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:29:22 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:22 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:22 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:29:22 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:23 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:23 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:23 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:29:24 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:24 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:29:24 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:29:24 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:24 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:41530->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:29:24 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:25 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:29:25 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:25 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:29:26 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:27 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:28 [INFO] agent: Synced check "policy-nexus-internal-healthcheck" 2021/05/21 11:29:28 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:29:29 [INFO] agent: Synced check "vfc-workflowengineactiviti" 2021/05/21 11:29:29 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:29 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:29 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:29 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:29:29 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:29:29 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:29:29 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:29:29 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:29:29 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:29:29 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:29:30 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:29:30 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:29:30 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:29:30 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:29:30 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:29:30 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:31 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:29:31 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:29:32 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:33 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:29:33 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:29:33 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:29:33 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:33 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:33 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:33 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:33 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:33 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:33 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:33 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:33 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:33 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:33 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:33 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:33 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:34 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:34 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:34 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:34 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:34 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:35 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:35 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:35 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:35 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:29:35 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:29:35 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:36 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:29:36 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:36 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:29:37 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:37 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:38 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:38 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:39 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:39 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:39 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:39 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:39 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:29:39 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:29:39 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:29:39 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:29:39 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:29:39 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:29:39 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:29:39 [INFO] agent: Synced check "brmsgw-tcp" 2021/05/21 11:29:39 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:39 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:41820->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:29:39 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:40 [INFO] agent: Synced check "vfc-workflow" 2021/05/21 11:29:40 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:29:40 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:29:40 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:29:40 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:29:40 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:40 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:40 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:29:40 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:29:41 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:29:41 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:29:41 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:42 [INFO] agent: Synced check "vfc-jujudriver" 2021/05/21 11:29:42 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:29:42 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:42 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:42 [WARN] agent: Check "mr-kafka" is now warning 2021/05/21 11:29:43 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:29:43 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:29:44 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:29:46 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:29:47 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:29:48 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:48 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:48 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:48 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:48 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:48 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:48 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:48 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:48 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:48 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:48 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:48 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:48 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:49 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:49 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:49 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:49 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:49 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:29:49 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:29:49 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:29:49 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:29:49 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:29:49 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:29:49 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:49 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:49 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:49 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:50 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:29:50 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:29:50 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:29:50 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:29:50 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:50 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:50 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:50 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:50 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:29:50 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:29:50 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:51 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:29:51 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:51 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:29:51 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:29:51 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:29:52 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:52 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:52 [INFO] agent: Synced check "log-kibana-server" 2021/05/21 11:29:52 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:53 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:53 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:29:53 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:53 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:29:53 [INFO] agent: Synced check "vfc-nokiavnfmdriver" 2021/05/21 11:29:54 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:54 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:29:54 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:29:54 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:54 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:42130->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:29:54 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:55 [WARN] agent: Check "dmaap" HTTP request failed: Get http://message-router:3904/topics: net/http: request canceled (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:29:55 [INFO] agent: Synced check "dmaap" 2021/05/21 11:29:55 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:55 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:29:55 [INFO] agent: Synced check "appc-dgbuilder" 2021/05/21 11:29:56 [INFO] agent: Synced check "policy-nexus-internal-metrics" 2021/05/21 11:29:56 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:57 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:57 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:29:57 [INFO] agent: Synced check "log-logstash-internal-server-node-info" 2021/05/21 11:29:57 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:29:58 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:29:59 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:59 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:59 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:29:59 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:29:59 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:29:59 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:29:59 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:29:59 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:30:00 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:30:00 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:30:00 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:30:00 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:30:00 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:02 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:30:02 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:02 [INFO] agent: Synced check "vfc-resmgr" 2021/05/21 11:30:02 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:30:02 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:30:03 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:30:03 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:30:03 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:03 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:03 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:03 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:03 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:03 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:03 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:03 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:03 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:03 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:04 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:04 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:04 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:04 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:04 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:30:04 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:04 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:04 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:04 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:05 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:05 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:30:05 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:05 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:05 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:30:05 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:30:05 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:06 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:30:06 [INFO] agent: Synced check "so-jra-healthcheck" 2021/05/21 11:30:06 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:06 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:30:06 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:30:07 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:07 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:08 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:09 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:30:09 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:30:09 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:09 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:09 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:09 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:09 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:09 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:30:09 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:30:09 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:30:09 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:30:09 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:30:09 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:09 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:42420->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:30:09 [INFO] agent: Synced check "pdp" 2021/05/21 11:30:09 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:10 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:30:10 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:30:10 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:30:10 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:30:10 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:10 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:11 [INFO] agent: Synced check "log-logstash-internal-server-jvm-stat" 2021/05/21 11:30:12 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:12 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:12 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:12 [WARN] agent: Check "mr-kafka" is now warning 2021/05/21 11:30:13 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:30:13 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:30:13 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:30:13 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:30:13 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:30:13 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:30:15 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:30:15 [INFO] agent: Synced check "pap" 2021/05/21 11:30:16 [WARN] agent: Check "elasticsearch-write-health" is now warning 2021/05/21 11:30:17 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:30:18 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:30:18 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:18 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:18 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:19 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:19 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:19 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:19 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:19 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:19 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:19 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:19 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:19 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:19 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:19 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:19 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:30:19 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:30:19 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:19 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:19 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:19 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:30:19 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:30:19 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:30:19 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:30:19 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:19 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:19 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:19 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:20 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:30:20 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:30:20 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:30:20 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:30:20 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:20 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:20 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:20 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:20 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:30:20 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:30:20 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:21 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:30:21 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:30:21 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:21 [INFO] agent: Synced check "sdc-cs-healthcheck" 2021/05/21 11:30:21 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:30:22 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:22 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:22 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:23 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:23 [INFO] agent: Synced check "log-logstash-internal-server-os-stat" 2021/05/21 11:30:23 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:30:24 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:30:24 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:24 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:24 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:30:24 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:24 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:30:24 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:30:24 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:42722->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:30:25 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:25 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:26 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:30:26 [INFO] agent: Synced check "appc-sdnctldb02" 2021/05/21 11:30:26 [WARN] agent: Check "dmaap" HTTP request failed: Get http://message-router:3904/topics: net/http: request canceled (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:30:27 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:27 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:28 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:30:28 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:30:29 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:30:29 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:29 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:29 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:29 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:30:29 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:30:29 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:30:29 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:30:29 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:30:29 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:30:30 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:30:30 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:30:30 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:30:30 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:30:30 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:30 [INFO] agent: Synced check "log-logstash-internal-server-jvm-info" 2021/05/21 11:30:30 [INFO] agent: Synced check "so-camunda-healthcheck" 2021/05/21 11:30:32 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:33 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:30:34 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:34 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:34 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:34 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:34 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:34 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:34 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:34 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:34 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:34 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:34 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:34 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:34 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:34 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:30:34 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:34 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:34 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:34 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:34 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:35 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:30:35 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:35 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:35 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:35 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:30:35 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:30:35 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:30:35 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:30:36 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:36 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:30:36 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:36 [INFO] agent: Synced check "vfc-emsdriver" 2021/05/21 11:30:36 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:30:37 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:30:37 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:30:37 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:37 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:38 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:39 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:30:39 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:30:39 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:39 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:39 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:39 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:39 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:39 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:30:39 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:30:39 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:30:39 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:30:39 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:30:39 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:30:39 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:39 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:43034->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:30:40 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:40 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:30:40 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:30:40 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:30:40 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:30:40 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:40 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:42 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:42 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:42 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:42 [WARN] agent: Check "mr-kafka" is now warning 2021/05/21 11:30:43 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:30:43 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:30:45 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:30:45 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:30:46 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:30:46 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:30:48 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:30:49 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:49 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:49 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:49 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:49 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:49 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:49 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:49 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:49 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:49 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:49 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:49 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:49 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:49 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:49 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:30:49 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:49 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:49 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:49 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:30:49 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:30:49 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:30:49 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:30:49 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:49 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:49 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:49 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:50 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:30:50 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:30:50 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:30:50 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:50 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:50 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:30:50 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:30:50 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:50 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:50 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:30:50 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:30:50 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:30:51 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:51 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:30:51 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:30:51 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:51 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:30:52 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:52 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:52 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:53 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:53 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:30:53 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:30:54 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:30:54 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:54 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:54 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:30:54 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:54 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:43388->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:30:55 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:55 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:57 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:57 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:57 [WARN] agent: Check "dmaap" HTTP request failed: Get http://message-router:3904/topics: net/http: request canceled (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:30:57 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:30:57 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:30:58 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:30:59 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:30:59 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:30:59 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:59 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:59 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:30:59 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:30:59 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:30:59 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:30:59 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:31:00 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:31:00 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:31:00 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:31:00 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:31:00 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:01 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:31:01 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:31:01 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:31:02 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:04 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:31:04 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:04 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:04 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:04 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:04 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:04 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:04 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:04 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:04 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:04 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:04 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:04 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:04 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:04 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:04 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:04 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:04 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:04 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:05 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:05 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:05 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:05 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:31:05 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:31:06 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:31:06 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:06 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:31:06 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:06 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:31:07 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:31:07 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:07 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:08 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:08 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:31:08 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:31:09 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:31:09 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:31:09 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:09 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:09 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:31:09 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:09 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:09 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:09 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:31:09 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:31:09 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:31:09 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:31:09 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:31:09 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:09 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:43680->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:31:10 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:10 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:31:10 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:31:10 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:31:10 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:31:10 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:31:10 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:10 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:12 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:12 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:12 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:12 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:31:12 [WARN] agent: Check "mr-kafka" is now warning 2021/05/21 11:31:13 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:31:14 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:31:17 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:31:17 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:31:19 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:19 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:19 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:19 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:19 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:19 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:19 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:19 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:19 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:19 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:19 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:19 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:19 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:19 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:19 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:19 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:19 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:31:19 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:19 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:31:19 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:31:19 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:31:19 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:31:19 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:19 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:19 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:19 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:19 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:31:19 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:31:20 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:31:20 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:31:20 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:20 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:20 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:20 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:20 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:31:20 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:31:20 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:31:20 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:31:21 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:21 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:31:21 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:31:21 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:21 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:31:21 [WARN] agent: Check "elasticsearch-write-health" is now warning 2021/05/21 11:31:22 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:31:22 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:22 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:22 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:23 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:31:23 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:23 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:31:24 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:31:24 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:31:24 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:24 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:24 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:31:24 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:24 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:44160->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:31:25 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:25 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:31:25 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:27 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:27 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:28 [WARN] agent: Check "dmaap" HTTP request failed: Get http://message-router:3904/topics: net/http: request canceled (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:31:28 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:31:29 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:29 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:29 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:31:29 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:29 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:31:29 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:31:29 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:31:29 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:31:30 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:31:30 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:31:30 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:30 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:31:30 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:31:30 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:31:30 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:31:32 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:31:32 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:33 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:31:33 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:31:34 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:34 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:34 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:34 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:34 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:34 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:34 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:34 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:34 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:34 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:34 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:34 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:34 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:34 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:34 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:31:34 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:31:34 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:34 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:34 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:34 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:35 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:35 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:35 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:35 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:31:35 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:31:36 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:36 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:31:36 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:36 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:31:37 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:37 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:38 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:31:38 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:39 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:31:39 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:31:39 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:39 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:39 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:39 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:31:39 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:31:39 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:39 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:39 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:31:39 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:31:39 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:31:39 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:31:39 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:39 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:44752->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:31:40 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:40 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:31:40 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:31:40 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:40 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:31:40 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:31:40 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:41 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:31:41 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:31:41 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:31:42 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:42 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:42 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:42 [WARN] agent: Check "mr-kafka" is now warning 2021/05/21 11:31:43 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:31:43 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:31:44 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:31:45 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:31:49 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:49 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:49 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:49 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:49 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:49 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:49 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:49 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:49 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:49 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:49 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:49 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:49 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:49 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:49 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:31:49 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:49 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:49 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:49 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:31:49 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:31:49 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:31:49 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:31:49 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:31:49 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:49 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:49 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:49 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:49 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:31:50 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:31:50 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:31:50 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:50 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:50 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:50 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:50 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:31:50 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:31:50 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:31:50 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:31:51 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:51 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:31:51 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:51 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:31:52 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:52 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:52 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:52 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:31:52 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:31:53 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:54 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:31:54 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:31:54 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:31:54 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:31:54 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:54 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:54 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:31:54 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:54 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:45166->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:31:55 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:55 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:31:55 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:56 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:31:57 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:57 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:31:57 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:58 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:31:59 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:59 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:59 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:31:59 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:31:59 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:31:59 [WARN] agent: Check "dmaap" HTTP request failed: Get http://message-router:3904/topics: net/http: request canceled (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:31:59 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:31:59 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:31:59 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:32:00 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:32:00 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:32:00 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:00 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:32:00 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:32:02 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:03 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:32:03 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:32:04 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:04 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:04 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:04 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:04 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:04 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:04 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:04 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:04 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:04 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:04 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:04 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:04 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:04 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:04 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:32:04 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:04 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:04 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:04 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:05 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:32:05 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:32:05 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:05 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:05 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:05 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:32:05 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:32:05 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:32:06 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:06 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:06 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:32:06 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:32:07 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:07 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:32:07 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:08 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:09 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:32:09 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:09 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:09 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:09 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:32:09 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:32:09 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:09 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:09 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:32:09 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:32:09 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:32:09 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:32:09 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:09 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:45408->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:32:10 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:32:10 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:10 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:32:10 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:32:10 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:10 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:10 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:32:10 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:32:11 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:32:12 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:12 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:12 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:12 [WARN] agent: Check "mr-kafka" is now warning 2021/05/21 11:32:13 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:32:13 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:32:14 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:32:14 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:32:14 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:32:16 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:32:18 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:32:19 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:19 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:19 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:19 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:19 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:19 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:19 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:19 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:19 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:19 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:19 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:19 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:19 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:19 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:19 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:19 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:19 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:19 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:32:19 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:32:19 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:32:19 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:32:19 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:32:19 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:19 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:19 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:19 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:20 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:20 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:20 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:32:20 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:32:20 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:20 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:20 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:32:20 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:32:20 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:32:20 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:32:21 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:21 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:32:21 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:21 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:32:21 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:32:21 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:32:22 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:22 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:22 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:23 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:24 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:32:24 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:32:24 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:24 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:32:24 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:24 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:24 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:45660->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:32:25 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:25 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:25 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:32:25 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:32:26 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:32:27 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:32:27 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:32:27 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:27 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:27 [WARN] agent: Check "elasticsearch-write-health" is now warning 2021/05/21 11:32:28 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:32:29 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:32:29 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:29 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:29 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:29 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:32:29 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:32:29 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:32:29 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:32:29 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:32:29 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:32:30 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:32:30 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:32:30 [WARN] agent: Check "dmaap" HTTP request failed: Get http://message-router:3904/topics: net/http: request canceled (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:32:30 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:30 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:32:30 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:32:32 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:34 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:34 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:34 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:34 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:34 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:34 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:34 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:34 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:34 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:34 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:34 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:34 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:34 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:34 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:34 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:32:34 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:34 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:34 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:34 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:35 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:35 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:35 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:35 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:32:35 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:32:36 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:36 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:36 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:32:36 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:32:36 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:32:36 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:32:37 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:32:37 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:37 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:32:37 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:38 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:32:38 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:39 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:32:39 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:39 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:39 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:39 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:32:39 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:39 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:32:39 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:32:39 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:39 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:32:39 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:32:39 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:32:39 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:39 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:45914->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:32:40 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:40 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:32:40 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:32:40 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:40 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:40 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:32:40 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:32:40 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:32:42 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:32:42 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:42 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:42 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:42 [WARN] agent: Check "mr-kafka" is now warning 2021/05/21 11:32:43 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:32:43 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:32:44 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:32:45 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:32:47 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:32:47 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:32:49 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:49 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:49 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:49 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:49 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:49 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:49 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:49 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:49 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:49 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:49 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:49 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:49 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:49 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:49 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:32:49 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:49 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:49 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:49 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:32:49 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:32:49 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:32:49 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:32:49 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:32:49 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:49 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:49 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:49 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:50 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:50 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:50 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:32:50 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:32:50 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:50 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:50 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:32:50 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:32:50 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:32:50 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:32:51 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:51 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:51 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:32:51 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:32:51 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:32:52 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:52 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:52 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:53 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:53 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:32:53 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:32:54 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:32:54 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:32:54 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:54 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:54 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:32:54 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:54 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:46168->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:32:55 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:55 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:57 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:57 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:58 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:32:58 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:32:58 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:32:58 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:32:59 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:32:59 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:59 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:59 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:32:59 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:32:59 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:32:59 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:32:59 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:32:59 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:33:00 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:33:00 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:33:00 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:33:00 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:00 [INFO] agent: Synced check "dmaap" 2021/05/21 11:33:00 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:33:00 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:33:01 [INFO] agent: Synced check "so-api-healthcheck" 2021/05/21 11:33:01 [INFO] agent: Synced check "aai-service" 2021/05/21 11:33:01 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:33:02 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:33:02 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:04 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:04 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:04 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:04 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:04 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:04 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:04 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:04 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:04 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:04 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:04 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:04 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:04 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:04 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:04 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:33:04 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:04 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:04 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:04 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:05 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:05 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:05 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:05 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:33:05 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:33:06 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:06 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:06 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:33:06 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:33:07 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:07 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:08 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:09 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:33:09 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:33:09 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:09 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:09 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:09 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:33:09 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:09 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:09 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:33:09 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:33:09 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:33:09 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:33:09 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:33:09 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:09 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:33:09 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:33:09 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:46410->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:33:09 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:33:10 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:10 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:33:10 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:33:10 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:10 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:10 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:33:10 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:33:11 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:33:12 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:12 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:12 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:12 [WARN] agent: Check "mr-kafka" is now warning 2021/05/21 11:33:13 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:33:13 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:33:14 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:33:14 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:33:15 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:33:17 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:33:19 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:19 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:19 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:19 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:19 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:19 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:19 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:19 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:19 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:19 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:19 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:19 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:19 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:19 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:19 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:19 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:19 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:19 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:33:19 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:33:19 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:33:19 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:33:19 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:33:19 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:19 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:19 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:19 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:20 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:20 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:20 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:33:20 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:33:20 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:20 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:20 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:33:20 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:33:20 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:33:20 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:33:20 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:33:20 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:33:21 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:21 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:21 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:33:21 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:33:22 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:33:22 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:22 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:22 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:23 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:24 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:33:24 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:33:24 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:24 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:24 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:33:24 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:33:24 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:24 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:46666->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:33:25 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:25 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:33:25 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:25 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:33:27 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:27 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:28 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:33:29 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:29 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:29 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:29 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:33:29 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:33:29 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:33:29 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:33:29 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:33:30 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:33:30 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:33:30 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:33:30 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:30 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:33:30 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:33:31 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:33:31 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:33:31 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:33:32 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:32 [WARN] agent: Check "elasticsearch-write-health" is now warning 2021/05/21 11:33:33 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:33:33 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:33:34 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:34 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:34 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:34 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:34 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:34 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:34 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:34 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:34 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:34 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:34 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:34 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:34 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:34 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:34 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:33:34 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:34 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:34 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:34 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:35 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:35 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:35 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:35 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:33:35 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:33:35 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:33:36 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:36 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:36 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:33:36 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:33:37 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:38 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:38 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:38 [INFO] agent: Synced check "so-jra-healthcheck" 2021/05/21 11:33:39 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:33:39 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:39 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:39 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:39 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:33:39 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:39 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:39 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:33:39 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:33:39 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:33:39 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:33:39 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:33:39 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:39 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:46900->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:33:40 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:40 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:33:40 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:33:40 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:40 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:41 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:33:41 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:33:41 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:33:41 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:33:42 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:42 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:42 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:42 [WARN] agent: Check "mr-kafka" is now warning 2021/05/21 11:33:42 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:33:42 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:33:43 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/21 11:33:44 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:33:44 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:33:46 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:33:46 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:33:47 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:33:49 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:49 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:49 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:49 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:49 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:49 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:49 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:49 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:49 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:49 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:49 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:49 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:49 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:49 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:49 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:33:49 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:49 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:49 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:49 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/21 11:33:49 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/21 11:33:49 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/21 11:33:49 [WARN] agent: Check "so-mariadb" is now warning 2021/05/21 11:33:49 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/21 11:33:49 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:49 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:49 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:49 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:50 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:50 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:50 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/21 11:33:50 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/21 11:33:50 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:50 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:50 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/21 11:33:50 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/21 11:33:51 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/21 11:33:51 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/21 11:33:51 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:51 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:51 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/21 11:33:51 [WARN] agent: Check "data-router-process" is now warning 2021/05/21 11:33:52 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:52 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:53 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:53 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:53 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/21 11:33:54 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/21 11:33:54 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/21 11:33:54 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:54 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:54 [WARN] agent: Check "model-loader-process" is now warning 2021/05/21 11:33:54 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/21 11:33:54 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:54 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.1.149:47156->10.96.239.37:9009: read: connection reset by peer 2021/05/21 11:33:55 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:55 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/21 11:33:55 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:57 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/21 11:33:57 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:57 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/21 11:33:57 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/21 11:33:57 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request cancele