By type
+ CONSUL_BIND= + [ -n ] + CONSUL_CLIENT= + [ -n ] + CONSUL_DATA_DIR=/consul/data + CONSUL_CONFIG_DIR=/consul/config + [ -n ] + [ a = - ] + [ agent = agent ] + shift + set -- consul agent -data-dir=/consul/data -config-dir=/consul/config -client 0.0.0.0 -enable-script-checks -retry-join consul-server + [ consul = consul ] + stat -c %u /consul/data + id -u consul + [ 100 != 100 ] + stat -c %u /consul/config + id -u consul + [ 100 != 100 ] + [ ! -z ] + exec consul agent -data-dir=/consul/data -config-dir=/consul/config -client 0.0.0.0 -enable-script-checks -retry-join consul-server ==> Starting Consul agent... ==> Consul agent running! Version: 'v1.0.6' Node ID: 'b0f3d442-8201-bd39-1b8e-37df197a9010' Node name: 'dev-consul-7467bf7b46-2wbhv' Datacenter: 'dc1' (Segment: '') Server: false (Bootstrap: false) Client Addr: [0.0.0.0] (HTTP: 8500, HTTPS: -1, DNS: 8600) Cluster Addr: 10.242.168.100 (LAN: 8301, WAN: 8302) Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false ==> Log data will now stream in as it occurs: 2021/05/10 00:04:57 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-2wbhv 10.242.168.100 2021/05/10 00:04:57 [WARN] Service name "Health Check: Application Authorization Framework" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] Service name "Health Check: Secret Management Service (sms)" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] Service name "A&AI Synapse Data Routing Service" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] agent: check "data-router-process" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/10 00:04:57 [WARN] Service name "A&AI Model Loader" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] agent: check "model-loader-process" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/10 00:04:57 [WARN] Service name "A&AI Search Data Service" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] agent: check "elasticsearch-write-health" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/10 00:04:57 [WARN] agent: check "search-data-service-availability" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/10 00:04:57 [WARN] agent: check "search-data-service-api" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/10 00:04:57 [WARN] Service name "Active and Available Inventory" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] Service name "A&AI UI Backend Service" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] agent: check "sparky-be-process" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/10 00:04:57 [WARN] Service name "Health Check: APPC - Dgbuilder" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] Service name "Health Check: APPC" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] manager: No servers available 2021/05/10 00:04:57 [WARN] agent: check "appc-dbhost-healthcheck" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/10 00:04:57 [WARN] Service name "Health Check: APPC-SDN-CTL-DB-01" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] Service name "Health Check: APPC-SDN-CTL-DB-02" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] Service name "Health Check: APPC - SDN Host" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] Service name "Health Check: CLAMP" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] Service name "Health Check: CLAMP - MariaDb" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] agent: check "clamp-mariadb" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/10 00:04:57 [WARN] Service name "Health Check: ONAP CLI" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] Service name "Health Check: Log - Elastic Search" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] Service name "Health Check: Log - Kibana" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] Service name "Health Check: Log - Log Stash" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] Service name "Health Check: DMaaP" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] agent: check "mr-zookeeper" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/10 00:04:57 [WARN] agent: check "mr-kafka" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/10 00:04:57 [WARN] Service name "Health Check: MSB" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] Service name "Health Check: MULTICLOUD" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] Service name "Health Check: Policy" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] agent: check "Policy-mariadb-healthcheck" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/10 00:04:57 [WARN] Service name "Health Check: SDC" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] agent: check "sdc-fe-healthcheck" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/10 00:04:57 [WARN] agent: check "sdc-be-healthcheck" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/10 00:04:57 [WARN] agent: check "sdc-titan-healthcheck" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/10 00:04:57 [WARN] agent: check "sdc-cs-healthcheck" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/10 00:04:57 [WARN] Service name "Health Check: SDNC ODL Cluster" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] agent: check "sdnc-odl-cluster-healthcheck" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/10 00:04:57 [WARN] Service name "Health Check: SDNC - DB Host" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] agent: check "sdnc-dbhost-healthcheck" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/10 00:04:57 [WARN] Service name "Health Check: SDNC - DGBuilder" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] Service name "Health Check: SDNC" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] Service name "Health Check: SDNC-SDN-CTL-DB-01" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] Service name "Health Check: SDNC-SDN-CTL-DB-02" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] Service name "Health Check: SDNC - SDN Host" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] Service name "Health Check: SO" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] agent: check "so-api-healthcheck" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/10 00:04:57 [WARN] agent: check "so-camunda-healthcheck" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/10 00:04:57 [WARN] agent: check "so-jra-healthcheck" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/10 00:04:57 [WARN] Service name "Health Check: SO - MariaDb" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] agent: check "so-mariadb" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/10 00:04:57 [WARN] Service name "Health Check: VFC" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] Service name "Health Check: VID" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes. 2021/05/10 00:04:57 [WARN] agent: check "vid-mariadb" has the 'script' field, which has been deprecated and replaced with the 'args' field. See https://www.consul.io/docs/agent/checks.html 2021/05/10 00:04:57 [INFO] agent: Started DNS server 0.0.0.0:8600 (udp) 2021/05/10 00:04:57 [INFO] agent: Started DNS server 0.0.0.0:8600 (tcp) 2021/05/10 00:04:57 [INFO] agent: Started HTTP server on [::]:8500 (tcp) 2021/05/10 00:04:57 [INFO] agent: started state syncer 2021/05/10 00:04:57 [INFO] agent: Retry join LAN is supported for: aliyun aws azure digitalocean gce os scaleway softlayer 2021/05/10 00:04:57 [INFO] agent: Joining LAN cluster... 2021/05/10 00:04:57 [INFO] agent: (LAN) joining: [consul-server] 2021/05/10 00:04:57 [ERR] agent: failed to sync remote state: No known Consul servers 2021/05/10 00:04:57 [INFO] serf: EventMemberJoin: dev-consul-server-0 10.242.61.70 2021/05/10 00:04:57 [INFO] serf: EventMemberJoin: dev-consul-server-1 10.242.210.71 2021/05/10 00:04:57 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.163.157 2021/05/10 00:04:57 [INFO] consul: adding server dev-consul-server-0 (Addr: tcp/10.242.61.70:8300) (DC: dc1) 2021/05/10 00:04:57 [INFO] consul: adding server dev-consul-server-1 (Addr: tcp/10.242.210.71:8300) (DC: dc1) 2021/05/10 00:04:57 [INFO] consul: adding server dev-consul-server-2 (Addr: tcp/10.242.163.157:8300) (DC: dc1) 2021/05/10 00:04:57 [INFO] agent: (LAN) joined: 2 Err:
2021/05/10 00:04:57 [INFO] agent: Join LAN completed. Synced with 2 initial agents 2021/05/10 00:04:57 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:04:57 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:04:58 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:04:58 [WARN] agent: Check "multicloud-pike" HTTP request failed: Head http://multicloud-pike:9007/api/multicloud-pike/v0/swagger.json: net/http: request canceled (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: CLAMP" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: CLAMP - MariaDb" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: ONAP CLI" 2021/05/10 00:04:59 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: Policy" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: APPC - Dgbuilder" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: APPC-SDN-CTL-DB-02" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: SDNC-SDN-CTL-DB-01" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: VFC" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: Secret Management Service (sms)" 2021/05/10 00:04:59 [INFO] agent: Synced service "Active and Available Inventory" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: MULTICLOUD" 2021/05/10 00:04:59 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:04:59 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:04:59 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: SDC" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: Application Authorization Framework" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: SDNC - SDN Host" 2021/05/10 00:04:59 [INFO] agent: Synced service "A&AI Synapse Data Routing Service" 2021/05/10 00:04:59 [INFO] agent: Synced service "A&AI Model Loader" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: DMaaP" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: SDNC - DGBuilder" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: SDNC-SDN-CTL-DB-02" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: VID" 2021/05/10 00:04:59 [INFO] agent: Synced service "A&AI UI Backend Service" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: APPC" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: Log - Log Stash" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: SDNC ODL Cluster" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: SDNC" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: SO" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: APPC-SDN-CTL-DB-01" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: Log - Elastic Search" 2021/05/10 00:04:59 [INFO] agent: Synced service "A&AI Search Data Service" 2021/05/10 00:04:59 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: APPC - SDN Host" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: Log - Kibana" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: MSB" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: SDNC - DB Host" 2021/05/10 00:04:59 [INFO] agent: Synced service "Health Check: SO - MariaDb" 2021/05/10 00:05:00 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:00 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:05:00 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:00 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:05:00 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:00 [WARN] memberlist: Was able to connect to dev-consul-server-2 but other probes failed, network may be misconfigured 2021/05/10 00:05:00 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:01 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:01 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:05:01 [INFO] agent: Synced check "vfc-ztevnfmdriver" 2021/05/10 00:05:01 [INFO] agent: Synced check "vfc-hwvnfmdriver" 2021/05/10 00:05:01 [INFO] agent: Synced check "aai-traversal" 2021/05/10 00:05:01 [INFO] agent: Synced check "sdnc-dbhost-healthcheck" 2021/05/10 00:05:01 [INFO] agent: Synced check "vfc-vnfres" 2021/05/10 00:05:01 [INFO] agent: Synced check "Policy-mariadb-healthcheck" 2021/05/10 00:05:01 [INFO] agent: Synced check "aai-service" 2021/05/10 00:05:01 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:01 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:01 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:02 [INFO] agent: Synced check "msb-discovery" 2021/05/10 00:05:02 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: net/http: request canceled (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:02 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:05:02 [INFO] agent: Synced check "appc-dbhost-healthcheck" 2021/05/10 00:05:02 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:02 [INFO] agent: Synced check "vfc-vnfmgr" 2021/05/10 00:05:02 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:02 [INFO] agent: Synced check "sdnc-odl-cluster-healthcheck" 2021/05/10 00:05:02 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:03 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving ==> Failed to check for updates: Get https://checkpoint-api.hashicorp.com/v1/check/consul?arch=amd64&os=linux&signature=6e885c8b-c395-78b3-29b1-2677f980270f&version=1.0.6: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:03 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:03 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:05:03 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:03 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:05:03 [INFO] agent: Synced check "so-mariadb" 2021/05/10 00:05:03 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:05:03 [INFO] agent: Synced check "model-loader-process" 2021/05/10 00:05:03 [WARN] agent: Check "elasticsearch-write-health" is now warning 2021/05/10 00:05:03 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:05:03 [INFO] agent: Synced check "elasticsearch-write-health" 2021/05/10 00:05:04 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:04 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:04 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:05:04 [INFO] agent: Synced check "search-data-service-api" 2021/05/10 00:05:04 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:04 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:05:04 [INFO] agent: Synced check "clamp-mariadb" 2021/05/10 00:05:04 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations) 2021/05/10 00:05:04 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.163.157 2021/05/10 00:05:04 [INFO] consul: removing server dev-consul-server-2 (Addr: tcp/10.242.163.157:8300) (DC: dc1) 2021/05/10 00:05:04 [ERR] memberlist: Failed fallback ping: read tcp 10.242.168.100:46422->10.242.163.157:8301: i/o timeout 2021/05/10 00:05:04 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received 2021/05/10 00:05:04 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:04 [INFO] agent: Synced check "aaf-service" 2021/05/10 00:05:04 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:05:04 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:05:04 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:05 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:05:05 [INFO] agent: Synced check "sparky-be-process" 2021/05/10 00:05:05 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:05 [INFO] agent: Synced check "aaf-sms-health" 2021/05/10 00:05:05 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:05 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:05:06 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:06 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: net/http: request canceled (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:06 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:05:06 [INFO] agent: Synced check "mr-zookeeper" 2021/05/10 00:05:07 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:07 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:07 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:07 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:07 [INFO] agent: Synced check "multicloud-vio" 2021/05/10 00:05:07 [WARN] Check "so-api-healthcheck": Timed out (1s) running check 2021/05/10 00:05:07 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:05:07 [WARN] agent: Check "dmaap" HTTP request failed: Get http://message-router:3904/topics: net/http: request canceled (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:07 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:07 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:07 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:07 [WARN] agent: Check "mr-kafka" is now warning 2021/05/10 00:05:07 [INFO] agent: Synced check "mr-kafka" 2021/05/10 00:05:09 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.163.157 2021/05/10 00:05:09 [INFO] consul: adding server dev-consul-server-2 (Addr: tcp/10.242.163.157:8300) (DC: dc1) 2021/05/10 00:05:09 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:09 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:10 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:10 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:05:10 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:05:10 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:10 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:05:10 [WARN] Check "so-camunda-healthcheck": Timed out (1s) running check 2021/05/10 00:05:11 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:11 [INFO] agent: Synced check "msb-consul" 2021/05/10 00:05:11 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:11 [INFO] agent: Synced check "vfc-gvnfmdriver" 2021/05/10 00:05:11 [INFO] agent: Synced check "vfc-vnflcm" 2021/05/10 00:05:11 [ERR] memberlist: Failed fallback ping: read tcp 10.242.168.100:46514->10.242.163.157:8301: i/o timeout 2021/05/10 00:05:11 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received 2021/05/10 00:05:11 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:12 [INFO] agent: Synced check "vfc-nslcm" 2021/05/10 00:05:12 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:05:12 [INFO] agent: Synced check "data-router-process" 2021/05/10 00:05:12 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:12 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:05:12 [INFO] agent: Synced check "aai-resources" 2021/05/10 00:05:12 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:12 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:13 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:13 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:13 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:05:14 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:14 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:05:14 [INFO] agent: Synced check "multicloud-pike" 2021/05/10 00:05:14 [INFO] agent: Synced check "search-data-service-availability" 2021/05/10 00:05:14 [ERR] memberlist: Failed fallback ping: read tcp 10.242.168.100:46582->10.242.163.157:8301: i/o timeout 2021/05/10 00:05:14 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received 2021/05/10 00:05:14 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:14 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:14 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:05:14 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:05:15 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:15 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:15 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:15 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:16 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:05:16 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:16 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:05:16 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:16 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:16 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:16 [WARN] Check "so-jra-healthcheck": Timed out (1s) running check 2021/05/10 00:05:16 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:16 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:05:17 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:17 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:17 [ERR] memberlist: Failed fallback ping: read tcp 10.242.168.100:46624->10.242.163.157:8301: i/o timeout 2021/05/10 00:05:17 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received 2021/05/10 00:05:17 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:17 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:05:17 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:17 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:18 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:05:18 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: net/http: request canceled (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:18 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:18 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations) 2021/05/10 00:05:18 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.163.157 2021/05/10 00:05:18 [INFO] consul: removing server dev-consul-server-2 (Addr: tcp/10.242.163.157:8300) (DC: dc1) 2021/05/10 00:05:18 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:18 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:18 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:05:19 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:19 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:05:19 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:19 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:19 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:19 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:05:20 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:20 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:05:20 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:05:20 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:05:21 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:05:21 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:05:21 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:22 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:22 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:05:22 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:22 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:22 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: net/http: request canceled (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:22 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:22 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:23 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:23 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:05:24 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:24 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:24 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:05:24 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-xplxm 10.242.163.138 2021/05/10 00:05:25 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:25 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:25 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:25 [ERR] memberlist: Failed fallback ping: read tcp 10.242.168.100:36702->10.242.163.138:8301: i/o timeout 2021/05/10 00:05:25 [INFO] memberlist: Suspect dev-consul-7467bf7b46-xplxm has failed, no acks received 2021/05/10 00:05:25 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:05:25 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:25 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:05:26 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:26 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:26 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:26 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:05:27 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:05:27 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:05:27 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:05:27 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:27 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:27 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:27 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:05:27 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:28 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.163.157 2021/05/10 00:05:28 [INFO] consul: adding server dev-consul-server-2 (Addr: tcp/10.242.163.157:8300) (DC: dc1) 2021/05/10 00:05:28 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:05:28 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:29 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:29 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:29 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:29 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:30 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:05:30 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:30 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:05:30 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:31 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:31 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:05:31 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:31 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:32 [WARN] Check "so-camunda-healthcheck": Timed out (1s) running check 2021/05/10 00:05:32 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:32 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:05:32 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:32 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:32 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:32 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:05:32 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:33 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:33 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:33 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:33 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:33 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:05:33 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:05:34 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:34 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:34 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:05:34 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: net/http: request canceled (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:34 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:34 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:05:34 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:34 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:05:35 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:35 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:05:35 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:36 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:36 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:36 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:05:36 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:05:37 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:05:37 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:37 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:37 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:37 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:05:37 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:37 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:37 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:37 [WARN] agent: Check "mr-kafka" is now warning 2021/05/10 00:05:38 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:05:38 [WARN] Check "so-jra-healthcheck": Timed out (1s) running check 2021/05/10 00:05:38 [INFO] agent: Synced check "dmaap" 2021/05/10 00:05:38 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: net/http: request canceled (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:38 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:05:39 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:39 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:39 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:40 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:40 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:05:40 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:05:40 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:41 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:41 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:41 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:42 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:05:42 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:42 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:05:42 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:42 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:43 [WARN] Check "so-camunda-healthcheck": Timed out (1s) running check 2021/05/10 00:05:43 [ERR] memberlist: Failed fallback ping: read tcp 10.242.168.100:37036->10.242.163.138:8301: i/o timeout 2021/05/10 00:05:43 [INFO] memberlist: Suspect dev-consul-7467bf7b46-xplxm has failed, no acks received 2021/05/10 00:05:43 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:05:43 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:43 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:05:44 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:44 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:05:44 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:44 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:45 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:45 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:45 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-xplxm 10.242.163.138 2021/05/10 00:05:45 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:45 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:46 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:46 [WARN] agent: Check "multicloud-pike" HTTP request failed: Head http://multicloud-pike:9007/api/multicloud-pike/v0/swagger.json: net/http: request canceled (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:46 [INFO] agent: Synced check "multicloud-pike" 2021/05/10 00:05:46 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:46 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:05:46 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:46 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:47 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:05:47 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-xplxm 10.242.163.138 2021/05/10 00:05:47 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:47 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:47 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:05:47 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:47 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:05:47 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:48 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:05:48 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:48 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:48 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:48 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:48 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:48 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:05:49 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:05:49 [WARN] Check "so-jra-healthcheck": Timed out (1s) running check 2021/05/10 00:05:49 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:49 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:05:49 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:05:49 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:49 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:49 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:49 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:49 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:05:50 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:50 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:05:50 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: net/http: request canceled (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:50 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:05:50 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:05:51 [ERR] memberlist: Failed fallback ping: read tcp 10.242.168.100:37166->10.242.163.138:8301: i/o timeout 2021/05/10 00:05:51 [INFO] memberlist: Suspect dev-consul-7467bf7b46-xplxm has failed, no acks received 2021/05/10 00:05:51 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:52 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:52 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:52 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:05:52 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:52 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:52 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:53 [INFO] serf: EventMemberFailed: dev-consul-7467bf7b46-xplxm 10.242.163.138 2021/05/10 00:05:53 [WARN] memberlist: Refuting a suspect message (from: dev-consul-7467bf7b46-xplxm) 2021/05/10 00:05:53 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:05:54 [WARN] Check "so-camunda-healthcheck": Timed out (1s) running check 2021/05/10 00:05:54 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:54 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:54 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:05:54 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:05:54 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: net/http: request canceled (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:55 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:05:55 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:55 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:55 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:56 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:56 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:56 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:56 [INFO] serf: EventMemberJoin: dev-consul-7467bf7b46-xplxm 10.242.163.138 2021/05/10 00:05:56 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:57 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:05:57 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:05:57 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:57 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:57 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:57 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:05:57 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:58 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:58 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:05:59 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:05:59 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:05:59 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:05:59 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:59 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:05:59 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:00 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:06:00 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:06:00 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:00 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:06:00 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:01 [INFO] agent: Synced check "multicloud-pike" 2021/05/10 00:06:01 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:01 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:06:01 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:06:01 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:01 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:02 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:06:02 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:02 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:02 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:03 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:03 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:03 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:03 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:06:03 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:06:04 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:04 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:04 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:06:04 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:06:04 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:06:04 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:06:04 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:04 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:06:04 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:04 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:06:05 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:05 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:06:05 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:05 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:06:05 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:06:05 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:06:06 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:06 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:07 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:06:07 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:07 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:07 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:07 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:06:07 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:07 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:07 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:08 [WARN] agent: Check "mr-kafka" is now warning 2021/05/10 00:06:09 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:09 [WARN] agent: Check "elasticsearch-write-health" is now warning 2021/05/10 00:06:09 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:06:09 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:06:09 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:09 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:34386->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:06:09 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:06:10 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:06:10 [INFO] serf: EventMemberLeave: dev-consul-7467bf7b46-xplxm 10.242.163.138 2021/05/10 00:06:10 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:10 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:06:10 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:10 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:06:11 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:06:11 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:11 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:06:11 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:11 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:12 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:06:12 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:12 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:06:12 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:12 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:13 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:14 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:06:14 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:14 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:06:14 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:06:14 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:14 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:15 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:15 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:15 [INFO] memberlist: Suspect dev-consul-server-2 has failed, no acks received 2021/05/10 00:06:15 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:16 [INFO] memberlist: Marking dev-consul-server-2 as failed, suspect timeout reached (2 peer confirmations) 2021/05/10 00:06:16 [INFO] serf: EventMemberFailed: dev-consul-server-2 10.242.163.157 2021/05/10 00:06:16 [INFO] consul: removing server dev-consul-server-2 (Addr: tcp/10.242.163.157:8300) (DC: dc1) 2021/05/10 00:06:16 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:16 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:06:16 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:16 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:16 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:06:16 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:17 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:06:17 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:06:17 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:17 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:17 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:06:17 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:17 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:18 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:18 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:18 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:18 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:06:19 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:19 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:06:19 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:19 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:06:19 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:06:19 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:19 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:19 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:06:20 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:20 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:06:20 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:06:20 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:06:20 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:06:20 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:06:20 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:06:20 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:06:21 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:06:21 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:06:21 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:22 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:06:22 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:22 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:22 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:06:22 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:22 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:22 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:24 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:06:24 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:24 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:06:24 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:24 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:06:24 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:34726->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:06:25 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:25 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:25 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:26 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:26 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:26 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:26 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:27 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:06:27 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:06:27 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:06:27 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:27 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:27 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:27 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:06:27 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:06:27 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:28 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:29 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:06:29 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:06:29 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:29 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:29 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:30 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:06:30 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:30 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:06:30 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:31 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:06:31 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:31 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:31 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:06:32 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:32 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:06:32 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:32 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:06:32 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:32 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:33 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:06:33 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:06:33 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:33 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:33 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:34 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:06:34 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:06:34 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:34 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:34 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:06:34 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:34 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:06:34 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:06:34 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:34 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:06:35 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:35 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:06:35 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:35 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:06:36 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:36 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:06:36 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:06:36 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:37 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:06:37 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:37 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:37 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:37 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:06:37 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:06:37 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:37 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:37 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:38 [WARN] agent: Check "mr-kafka" is now warning 2021/05/10 00:06:38 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:06:39 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:39 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:06:39 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:06:39 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:39 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:35040->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:06:40 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:40 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:06:40 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:40 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:06:41 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:41 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:41 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:42 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:06:42 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:42 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:06:42 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:42 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:42 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:06:43 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:06:43 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:06:43 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:44 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:06:44 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:06:44 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:44 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:06:44 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:06:44 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:44 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:45 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:45 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:45 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:46 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:46 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:06:46 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:46 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:47 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:47 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:06:47 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:47 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:47 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:06:47 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:47 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:48 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:48 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:48 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:49 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:06:49 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:49 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:06:49 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:06:49 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:49 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:06:49 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:06:49 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:06:49 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:49 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:50 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:06:50 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:50 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:06:50 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:06:50 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:06:50 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:06:51 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:52 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:06:52 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:52 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:06:52 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:52 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:06:52 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:52 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:53 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:53 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:06:53 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:06:54 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:06:54 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:06:54 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:54 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:54 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:06:54 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:06:54 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:35322->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:06:55 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:06:55 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:55 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:55 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:56 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:56 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:56 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:56 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:57 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:06:57 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:06:57 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:57 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:57 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:57 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:06:57 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:58 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:59 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:06:59 [INFO] serf: EventMemberJoin: dev-consul-server-2 10.242.163.183 2021/05/10 00:06:59 [INFO] consul: adding server dev-consul-server-2 (Addr: tcp/10.242.163.183:8300) (DC: dc1) 2021/05/10 00:06:59 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:06:59 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:59 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:06:59 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:06:59 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:00 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:07:00 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:00 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:07:00 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:07:00 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:01 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:01 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:07:01 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:02 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:02 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:02 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:07:02 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:02 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:03 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:03 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:03 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:04 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:07:04 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:07:04 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:04 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:04 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:04 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:07:04 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:07:04 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:04 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:07:05 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:07:05 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:07:05 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:05 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:07:05 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:07:05 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:07:05 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:05 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:07:06 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:07:06 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:06 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:07 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:07:07 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:07 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:07 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:07 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:07:07 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:07 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:08 [WARN] agent: Check "mr-kafka" is now warning 2021/05/10 00:07:08 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:08 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:07:08 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:07:09 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:09 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:07:09 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:09 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:07:09 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:35594->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:07:09 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:07:10 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:10 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:10 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:07:10 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:07:11 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:11 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:11 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:07:11 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:12 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:07:12 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:12 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:07:12 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:12 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:13 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:14 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:07:14 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:14 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:07:14 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:14 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:15 [WARN] agent: Check "elasticsearch-write-health" is now warning 2021/05/10 00:07:15 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:07:15 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:07:15 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:15 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:15 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:15 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:07:16 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:07:16 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:16 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:07:16 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:16 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:17 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:17 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:07:17 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:07:17 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:17 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:17 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:07:17 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:17 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:18 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:18 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:18 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:19 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:07:19 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:19 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:19 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:07:19 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:19 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:07:19 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:07:19 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:20 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:07:20 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:20 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:07:20 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:07:20 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:07:20 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:07:21 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:07:21 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:22 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:22 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:22 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:22 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:07:22 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:07:22 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:23 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:24 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:07:24 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:24 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:07:24 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:24 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:07:24 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:07:24 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:35870->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:07:25 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:07:25 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:25 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:25 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:25 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:07:26 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:26 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:26 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:26 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:07:26 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:27 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:07:27 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:07:27 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:07:27 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:27 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:27 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:27 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:07:27 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:28 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:07:28 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:29 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:29 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:29 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:29 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:07:29 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:07:30 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:30 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:07:30 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:07:30 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:31 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:07:31 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:07:31 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:31 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:32 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:32 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:32 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:07:32 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:32 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:33 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:33 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:33 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:07:33 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:34 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:07:34 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:07:34 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:34 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:34 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:34 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:07:34 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:07:34 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:35 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:07:35 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:35 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:07:35 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:07:35 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:35 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:07:36 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:36 [INFO] agent: Synced check "vfc-ztesdncdriver" 2021/05/10 00:07:36 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:37 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:07:37 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:07:37 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:37 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:37 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:37 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:07:37 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:37 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:07:38 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:38 [WARN] agent: Check "mr-kafka" is now warning 2021/05/10 00:07:38 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:38 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:07:39 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:07:39 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:39 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:39 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:36128->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:07:39 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:07:39 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:07:40 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:07:40 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:07:40 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:40 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:40 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:07:40 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:07:41 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:41 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:41 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:07:41 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:42 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:42 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:07:42 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:07:42 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:42 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:43 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:43 [INFO] agent: Synced check "pap" 2021/05/10 00:07:44 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:07:44 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:44 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:07:44 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:07:44 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:44 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:45 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:07:45 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:45 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:45 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:46 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:46 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:07:46 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:46 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:47 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:47 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:07:47 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:07:47 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:47 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:47 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:07:47 [INFO] agent: Synced check "log-logstash-internal-server-plugin-info" 2021/05/10 00:07:47 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:47 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:48 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:48 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:48 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:07:48 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:49 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:07:49 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:07:49 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:49 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:49 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:07:49 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:49 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:49 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:07:49 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:07:50 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:07:50 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:07:50 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:50 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:07:50 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:07:50 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:07:50 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:07:51 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:52 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:52 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:52 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:52 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:07:53 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:53 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:53 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:07:54 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:07:54 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:54 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:54 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:07:54 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:36386->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:07:55 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:07:55 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:55 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:55 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:55 [INFO] agent: Synced check "sdc-be-healthcheck" 2021/05/10 00:07:55 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:07:56 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:56 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:07:56 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:56 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:07:56 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:56 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:57 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:07:57 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:57 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:07:57 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:57 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:07:57 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:57 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:07:57 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:58 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:59 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:59 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:59 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:07:59 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:08:00 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:08:00 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:08:00 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:08:00 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:00 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:08:00 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:08:00 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:01 [INFO] agent: Synced check "log-logstash-internal-server-node-stat" 2021/05/10 00:08:01 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:08:01 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:08:01 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:01 [INFO] agent: Synced check "so-jra-healthcheck" 2021/05/10 00:08:01 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:02 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:02 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:02 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:08:02 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:02 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:03 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:08:03 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:03 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:03 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:04 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:08:04 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:08:04 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:04 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:04 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:04 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:08:04 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:08:04 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:05 [INFO] agent: Synced check "vfc-workflow" 2021/05/10 00:08:05 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:05 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:08:05 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:08:05 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:08:05 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:05 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:08:06 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:06 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:08:06 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:07 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:08:07 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:07 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:07 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:07 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:07 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:08:08 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:08 [WARN] agent: Check "mr-kafka" is now warning 2021/05/10 00:08:08 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:09 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:08:09 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:09 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:36652->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:08:09 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:10 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:08:10 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:08:10 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:10 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:10 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:08:10 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:08:10 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:08:11 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:08:11 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:11 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:11 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:12 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:08:12 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:08:12 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:08:12 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:12 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:08:12 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:08:12 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:12 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:13 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:08:13 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:14 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:08:14 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:14 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:08:14 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:14 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:15 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:08:15 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:15 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:15 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:16 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:16 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:08:16 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:16 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:17 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:17 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:08:17 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:17 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:17 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:08:17 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:17 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:08:18 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:18 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:18 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:18 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:19 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:08:19 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:19 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:08:19 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:19 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:08:19 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:19 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:20 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:08:20 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:08:20 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:20 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:08:20 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:08:20 [WARN] agent: Check "elasticsearch-write-health" is now warning 2021/05/10 00:08:20 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:08:20 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:08:20 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:08:21 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:21 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:08:22 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:08:22 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:22 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:22 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:22 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:08:23 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:23 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:23 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:08:24 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:08:24 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:24 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:24 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:08:24 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:36934->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:08:25 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:08:25 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:08:25 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:25 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:25 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:26 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:26 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:26 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:27 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:27 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:08:27 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:27 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:08:27 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:27 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:27 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:27 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:08:28 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:08:28 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:08:28 [INFO] agent: Synced check "appc-sdnctldb01" 2021/05/10 00:08:28 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:08:28 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:29 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:08:29 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:29 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:29 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:30 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:08:30 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:08:30 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:30 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:08:30 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:08:30 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:31 [INFO] agent: Synced check "vfc-nokiavnfmdriver" 2021/05/10 00:08:31 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:08:31 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:31 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:32 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:32 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:32 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:32 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:08:32 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:08:33 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:33 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:08:33 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:33 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:34 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:34 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:08:34 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:08:34 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:34 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:08:34 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:34 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:34 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:08:34 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:08:34 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:35 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:35 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:08:35 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:08:35 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:08:35 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:08:35 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:35 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:08:36 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:36 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:37 [INFO] agent: Synced check "log-logstash-internal-server-os-stat" 2021/05/10 00:08:37 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:08:37 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:37 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:37 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:37 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:37 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:08:38 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:38 [WARN] agent: Check "mr-kafka" is now warning 2021/05/10 00:08:38 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:39 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:39 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:08:39 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:37162->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:08:39 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:40 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:08:40 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:08:40 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:40 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:40 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:08:40 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:08:41 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:08:41 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:41 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:42 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:42 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:42 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:08:42 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:42 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:08:42 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:43 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:08:43 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:44 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:08:44 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:08:44 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:08:44 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:44 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:08:44 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:08:44 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:44 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:45 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:08:45 [WARN] agent: Check "aaf-service" is now critical 2021/05/10 00:08:45 [INFO] agent: Synced check "aaf-service" 2021/05/10 00:08:45 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:08:45 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:45 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:08:45 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:45 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:46 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:46 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:08:46 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:46 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:47 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:47 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:08:47 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:47 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:47 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:48 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:08:48 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:48 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:48 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:49 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:49 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:08:49 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:49 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:49 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:08:49 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:49 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:50 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:50 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:08:50 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:08:50 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:08:50 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:08:50 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:08:50 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:08:50 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:08:50 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:08:51 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:08:51 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:52 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:52 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:52 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:52 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:08:53 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:53 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:54 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:08:54 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:54 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:54 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:08:54 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:37422->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:08:54 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:08:55 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:08:55 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:55 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:55 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:55 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:08:56 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:08:56 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:56 [INFO] agent: Synced check "multicloud-starlingx" 2021/05/10 00:08:56 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:56 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:57 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:57 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:08:57 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:08:57 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:57 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:08:57 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:57 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:57 [INFO] agent: Synced check "policy-nexus-internal-healthcheck" 2021/05/10 00:08:57 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:58 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:08:59 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:59 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:59 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:08:59 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:00 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:09:00 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:09:00 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:09:00 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:09:00 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:00 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:09:00 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:09:00 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:01 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:09:01 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:01 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:09:01 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:01 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:09:02 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:02 [INFO] agent: Synced check "vfc-workflowengineactiviti" 2021/05/10 00:09:02 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:02 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:02 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:09:03 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:03 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:03 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:04 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:04 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:09:04 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:04 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:09:04 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:04 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:04 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:09:04 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:09:04 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:05 [WARN] agent: Check "aaf-service" is now critical 2021/05/10 00:09:05 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:05 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:09:05 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:09:05 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:05 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:09:05 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:09:05 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:09:06 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:09:06 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:06 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:07 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:09:07 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:09:07 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:09:07 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:07 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:07 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:08 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:08 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:09:08 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:08 [WARN] agent: Check "mr-kafka" is now warning 2021/05/10 00:09:08 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:09 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:09 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:37654->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:09:09 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:10 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:09:10 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:09:10 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:10 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:10 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:09:10 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:09:11 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:11 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:12 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:12 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:12 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:09:12 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:12 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:09:12 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:12 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:09:12 [INFO] agent: Synced check "framework" 2021/05/10 00:09:13 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:09:14 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:14 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:09:14 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:14 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:09:14 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:14 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:15 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:15 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:15 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:09:15 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:16 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:09:16 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:09:16 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:16 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:16 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:09:16 [INFO] agent: Synced check "log-logstash-internal-server-jvm-stat" 2021/05/10 00:09:16 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:16 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:09:17 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:09:17 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:17 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:09:17 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:17 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:09:17 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:18 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:18 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:18 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:09:18 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:09:18 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:18 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:19 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:19 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:09:19 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:19 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:19 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:09:19 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:19 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:20 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:20 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:09:20 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:09:20 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:09:20 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:09:20 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:09:20 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:09:20 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:09:21 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:22 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:22 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:22 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:22 [INFO] agent: Synced check "msb-iag" 2021/05/10 00:09:22 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:09:23 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:23 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:23 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:09:23 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:09:24 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:09:24 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:24 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:24 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:09:24 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:37908->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:09:25 [WARN] agent: Check "aaf-service" is now critical 2021/05/10 00:09:25 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:25 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:25 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:25 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:09:26 [WARN] agent: Check "elasticsearch-write-health" is now warning 2021/05/10 00:09:26 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:26 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:26 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:27 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:27 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:09:27 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:27 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:09:27 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:27 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:27 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:09:28 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:28 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:09:28 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:09:29 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:29 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:09:29 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:09:29 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:29 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:29 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:30 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:09:30 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:09:30 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:30 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:09:30 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:09:30 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:31 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:09:31 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:31 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:32 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:32 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:09:32 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:09:32 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:32 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:32 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:09:33 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:33 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:33 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:09:33 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:34 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:34 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:09:34 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:34 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:09:34 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:34 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:34 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:09:34 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:09:34 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:09:34 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:35 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:35 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:09:35 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:09:35 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:09:35 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:35 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:09:36 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:36 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:37 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:09:37 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:37 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:37 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:38 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:38 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:09:38 [WARN] agent: Check "mr-kafka" is now warning 2021/05/10 00:09:38 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:38 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:38 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:09:39 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:09:39 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:09:39 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:39 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:38164->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:09:39 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:40 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:09:40 [INFO] agent: Synced check "sdc-cs-healthcheck" 2021/05/10 00:09:40 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:09:40 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:09:40 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:40 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:40 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:09:40 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:09:41 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:41 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:42 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:42 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:42 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:09:42 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:42 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:09:42 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:43 [INFO] agent: Synced check "sdnc-sdnhost" 2021/05/10 00:09:44 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:44 [INFO] agent: Synced check "appc-sdnhost" 2021/05/10 00:09:44 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:09:44 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:44 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:09:44 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:44 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:44 [INFO] agent: Synced check "clamp-health" 2021/05/10 00:09:45 [WARN] agent: Check "aaf-service" is now critical 2021/05/10 00:09:45 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:09:45 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:45 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:45 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:09:45 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:09:45 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:46 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:09:46 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:46 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:46 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:47 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:47 [INFO] agent: Synced check "drools" 2021/05/10 00:09:47 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:09:47 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:47 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:48 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:48 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:48 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:09:48 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:09:48 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:09:48 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:48 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:49 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:49 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:09:49 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:49 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:49 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:09:49 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:09:49 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:49 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:09:49 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:50 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:09:50 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:50 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:09:50 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:09:50 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:09:50 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:09:50 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:09:50 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:09:50 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:09:51 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:09:51 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:52 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:52 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:52 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:52 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:09:53 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:53 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:54 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:09:54 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:54 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:54 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:09:54 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:38454->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:09:55 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:09:55 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:55 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:55 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:55 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:09:56 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:56 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:56 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:56 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:09:57 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:57 [INFO] agent: Synced check "vfc-resmgr" 2021/05/10 00:09:57 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:09:57 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:57 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:09:57 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:57 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:58 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:58 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:09:59 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:59 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:59 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:09:59 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:00 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:10:00 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:10:00 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:00 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:10:00 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:10:00 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:00 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:10:01 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:10:01 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:10:01 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:10:01 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:01 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:02 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:10:02 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:02 [INFO] agent: Synced check "sdnc-dgbuilder" 2021/05/10 00:10:02 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:02 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:02 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:10:03 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:03 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:03 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:04 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:04 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:10:04 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:10:04 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:04 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:10:04 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:10:04 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:04 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:04 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:10:04 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:10:04 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:05 [WARN] agent: Check "aaf-service" is now critical 2021/05/10 00:10:05 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:05 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:10:05 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:10:05 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:10:05 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:05 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:10:05 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:10:06 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:06 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:07 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:10:07 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:07 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:07 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:07 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:10:08 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:08 [WARN] agent: Check "mr-kafka" is now warning 2021/05/10 00:10:08 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:08 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:10:08 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:09 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:09 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:38688->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:10:09 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:10 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:10:10 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:10:10 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:10 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:10 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:10:10 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:10:11 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:10:11 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:11 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:11 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:10:12 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:12 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:10:12 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:12 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:10:12 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:12 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:10:12 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:13 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:10:14 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:14 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:10:14 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:14 [INFO] agent: Synced check "appc-sdnctldb02" 2021/05/10 00:10:14 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:10:14 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:14 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:15 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:15 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:15 [INFO] agent: Synced check "vfc-emsdriver" 2021/05/10 00:10:15 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:10:15 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:16 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:10:16 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:16 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:16 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:17 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:17 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:10:17 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:10:17 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:17 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:18 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:18 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:18 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:10:18 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:18 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:18 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:10:19 [INFO] agent: Synced check "log-logstash-internal-server-process-stat" 2021/05/10 00:10:19 [INFO] agent: Synced check "aai-resources" 2021/05/10 00:10:19 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:19 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:10:19 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:19 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:19 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:10:19 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:19 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:20 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:20 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:10:20 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:10:20 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:10:20 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:10:20 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:10:20 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:10:20 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:10:20 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:10:20 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:10:21 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:10:21 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:22 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:22 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:22 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:22 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:10:22 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:10:23 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:23 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:23 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:10:24 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:10:24 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:10:24 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:24 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:24 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:38946->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:10:24 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:10:25 [WARN] agent: Check "aaf-service" is now critical 2021/05/10 00:10:25 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:25 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:25 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:25 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:10:26 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:26 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:26 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:27 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:27 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:10:27 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:10:27 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:27 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:10:27 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:27 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:28 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:28 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:10:29 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:29 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:10:29 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:29 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:29 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:30 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:10:30 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:10:30 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:30 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:10:30 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:10:30 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:31 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:10:31 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:31 [WARN] agent: Check "elasticsearch-write-health" is now warning 2021/05/10 00:10:31 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:32 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:32 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:32 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:32 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:10:33 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:33 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:10:33 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:33 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:33 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:10:34 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:10:34 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:34 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:10:34 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:34 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:10:34 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:34 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:34 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:10:34 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:10:35 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:35 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:10:35 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:35 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:10:35 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:10:35 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:10:35 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:35 [INFO] agent: Synced check "so-api-healthcheck" 2021/05/10 00:10:36 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:10:36 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:10:36 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:10:36 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:36 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:37 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:10:37 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:37 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:37 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:37 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:10:38 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:38 [WARN] agent: Check "mr-kafka" is now warning 2021/05/10 00:10:38 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:38 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:10:38 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:39 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:39 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:39174->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:10:40 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:40 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:40 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:10:40 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:10:40 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:40 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:10:40 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:10:40 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:10:41 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:41 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:42 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:42 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:42 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:10:42 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:42 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:42 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:10:43 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:10:44 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:44 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:10:44 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:44 [INFO] agent: Synced check "policy-nexus-internal-metrics" 2021/05/10 00:10:44 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:44 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:44 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:10:44 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:10:45 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:10:45 [WARN] agent: Check "aaf-service" is now critical 2021/05/10 00:10:45 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:45 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:45 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:46 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:10:46 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:10:46 [INFO] agent: Synced check "sdnctldb01" 2021/05/10 00:10:46 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:10:46 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:46 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:46 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:47 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:47 [INFO] agent: Synced check "pdp" 2021/05/10 00:10:47 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:10:47 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:47 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:47 [INFO] agent: Synced check "log-logstash-internal-server-os-info" 2021/05/10 00:10:48 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:48 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:48 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:10:48 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:48 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:49 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:49 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:10:49 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:10:49 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:49 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:50 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:50 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:50 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:10:50 [INFO] agent: Synced check "vfc-catalog" 2021/05/10 00:10:50 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:50 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:10:50 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:10:50 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:10:50 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:10:50 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:10:50 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:10:50 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:10:51 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:10:51 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:52 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:10:52 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:10:52 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:52 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:52 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:52 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:10:53 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:53 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:53 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:10:54 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:10:54 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:54 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:54 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:39432->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:10:54 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:10:55 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:55 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:55 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:55 [INFO] agent: Synced check "elasticsearch" 2021/05/10 00:10:55 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:10:56 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:10:56 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:10:56 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:56 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:56 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:57 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:57 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:10:57 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:10:57 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:57 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:10:57 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:57 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:58 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:58 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:10:58 [INFO] agent: Synced check "sdc-titan-healthcheck" 2021/05/10 00:10:59 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:59 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:10:59 [INFO] agent: Synced check "log-logstash-internal-server-node-info" 2021/05/10 00:10:59 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:10:59 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:00 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:00 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:11:00 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:11:00 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:00 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:11:00 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:11:01 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:01 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:11:01 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:01 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:02 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:02 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:02 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:02 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:11:02 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:11:03 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:03 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:03 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:04 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:04 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:11:04 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:04 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:11:04 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:04 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:04 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:11:05 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:05 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:11:05 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:05 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:11:05 [WARN] agent: Check "aaf-service" is now critical 2021/05/10 00:11:05 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:11:05 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:11:05 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:11:05 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:06 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:11:06 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:06 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:11:06 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:07 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:11:07 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:11:07 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:07 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:07 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:08 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:08 [WARN] agent: Check "mr-kafka" is now warning 2021/05/10 00:11:08 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:08 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:08 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:11:08 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:11:08 [INFO] agent: Synced check "appc-dgbuilder" 2021/05/10 00:11:08 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:11:08 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:11:09 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:09 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:11:09 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:39672->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:11:10 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:10 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:10 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:10 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:11:10 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:11:10 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:11:10 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:11:11 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:11 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:12 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:12 [INFO] agent: Synced check "vfc-nokiav2vnfmdriver" 2021/05/10 00:11:12 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:12 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:11:12 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:12 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:12 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:11:13 [INFO] agent: Synced check "sdc-fe-healthcheck" 2021/05/10 00:11:13 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:11:14 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:14 [INFO] agent: Synced check "log-kibana-server" 2021/05/10 00:11:14 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:11:14 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:14 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:14 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:14 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:11:15 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:11:15 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:15 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:16 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:16 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:11:16 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:11:16 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:16 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:16 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:17 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:17 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:11:17 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:17 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:17 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:11:18 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:18 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:18 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:11:18 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:11:18 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:18 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:19 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:11:19 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:19 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:11:19 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:19 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:20 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:20 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:11:20 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:20 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:20 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:11:20 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:11:20 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:11:20 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:11:20 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:11:20 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:11:20 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:11:21 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:11:22 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:22 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:22 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:22 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:22 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:11:23 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:23 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:24 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:11:24 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:11:24 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:11:24 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:24 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:24 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:39920->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:11:24 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:11:24 [INFO] agent: Synced check "log-logstash-internal-server-gi" 2021/05/10 00:11:24 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:11:25 [WARN] agent: Check "aaf-service" is now critical 2021/05/10 00:11:25 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:25 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:11:25 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:25 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:26 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:11:26 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:26 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:26 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:27 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:27 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:11:27 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:27 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:11:27 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:27 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:28 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:28 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:11:29 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:11:29 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:29 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:11:29 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:29 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:30 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:30 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:11:30 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:30 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:11:30 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:11:30 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:11:30 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:11:31 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:31 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:11:31 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:11:31 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:31 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:32 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:32 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:32 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:32 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:11:33 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:33 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:33 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:34 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:34 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:11:34 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:34 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:11:34 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:34 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:34 [INFO] agent: Synced check "vid-server" 2021/05/10 00:11:35 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:11:35 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:35 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:11:35 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:35 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:11:35 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:11:35 [INFO] agent: Synced check "multicloud-windriver" 2021/05/10 00:11:35 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:11:35 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:35 [INFO] agent: Synced check "log-elasticsearch-server" 2021/05/10 00:11:35 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:11:36 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:11:36 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:37 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:37 [WARN] agent: Check "elasticsearch-write-health" is now warning 2021/05/10 00:11:37 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:11:37 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:11:37 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:37 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:37 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:38 [WARN] agent: Check "mr-kafka" is now warning 2021/05/10 00:11:38 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:38 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:38 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:38 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:11:39 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:39 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:40152->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:11:40 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:11:40 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:40 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:11:40 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:11:40 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:11:40 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:40 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:40 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:11:40 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:11:40 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:11:41 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:11:41 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:11:41 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:41 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:41 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:11:42 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:42 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:42 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:11:42 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:42 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:42 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:11:44 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:44 [INFO] agent: Synced check "aai-service" 2021/05/10 00:11:44 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:11:44 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:44 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:44 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:45 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:11:45 [WARN] agent: Check "aaf-service" is now critical 2021/05/10 00:11:45 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:45 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:46 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:46 [INFO] agent: Synced check "log-logstash-internal-server-jvm-info" 2021/05/10 00:11:46 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:11:46 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:11:46 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:46 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:46 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:46 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:11:47 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:11:47 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:47 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:11:47 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:47 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:48 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:48 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:48 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:11:48 [INFO] agent: Synced check "vfc-jujudriver" 2021/05/10 00:11:48 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:48 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:49 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:49 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:11:49 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:49 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:50 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:50 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:50 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:11:50 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:50 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:11:50 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:11:50 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:11:50 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:11:50 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:11:50 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:11:51 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:11:51 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:11:51 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:11:52 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:52 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:11:52 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:52 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:52 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:52 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:11:53 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:53 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:53 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:11:54 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:11:54 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:54 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:54 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:40404->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:11:55 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:11:55 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:55 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:55 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:56 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:11:56 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:11:56 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:11:56 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:56 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:56 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:57 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:57 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:11:57 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:57 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:11:57 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:11:57 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:57 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:57 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:11:58 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:58 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:11:58 [INFO] agent: Synced check "brmsgw-tcp" 2021/05/10 00:11:59 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:59 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:11:59 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:00 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:00 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:00 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:12:00 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:12:00 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:12:01 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:12:01 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:01 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:12:01 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:01 [INFO] agent: Synced check "sdc-catalog-healthcheck" 2021/05/10 00:12:01 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:02 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:12:02 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:12:02 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:02 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:02 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:02 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:12:03 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:03 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:12:03 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:12:03 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:03 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:04 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:04 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:12:04 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:04 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:12:04 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:04 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:05 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:12:05 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:05 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:12:05 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:05 [WARN] agent: Check "aaf-service" is now critical 2021/05/10 00:12:05 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:12:05 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:12:05 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:12:05 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:06 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:12:06 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:07 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:07 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:12:07 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:07 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:07 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:08 [WARN] agent: Check "mr-kafka" is now warning 2021/05/10 00:12:08 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:08 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:08 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:08 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:12:08 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:12:09 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:12:09 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:09 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:40644->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:12:10 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:10 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:10 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:10 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:12:11 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:12:11 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:12:11 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:12:11 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:11 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:12 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:12 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:12:12 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:12:12 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:12 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:12:12 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:12 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:12 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:12:13 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:12:13 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:12:13 [INFO] agent: Synced check "odl-api-healthcheck" 2021/05/10 00:12:13 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:12:14 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:14 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:12:14 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:12:14 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:15 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:15 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:15 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:12:15 [INFO] agent: Synced check "so-jra-healthcheck" 2021/05/10 00:12:15 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:15 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:16 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:16 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:12:16 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:12:16 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:16 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:16 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:17 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:17 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:12:17 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:17 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:18 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:18 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:18 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:12:18 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:18 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:19 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:12:19 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:19 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:12:19 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:19 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:19 [INFO] agent: Synced check "log-elasticsearch-tcp" 2021/05/10 00:12:19 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:12:20 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:20 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:20 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:12:20 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:20 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:12:20 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:12:20 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:12:20 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:12:21 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:12:21 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:12:21 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:12:21 [INFO] agent: Synced check "sdnctldb02" 2021/05/10 00:12:22 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:22 [INFO] agent: Synced check "msb-eag" 2021/05/10 00:12:22 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:22 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:22 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:22 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:12:23 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:23 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:23 [INFO] agent: Synced check "policy-nexus-local-status" 2021/05/10 00:12:24 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:12:24 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:12:24 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:12:24 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:24 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:24 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:40896->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:12:25 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:12:25 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:12:25 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:12:25 [WARN] agent: Check "aaf-service" is now critical 2021/05/10 00:12:25 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:25 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:25 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:25 [INFO] agent: Synced check "so-camunda-healthcheck" 2021/05/10 00:12:26 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:12:26 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:26 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:26 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:27 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:27 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:12:27 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:27 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:12:27 [INFO] agent: Synced check "log-logstash-tcp" 2021/05/10 00:12:27 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:27 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:28 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:28 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:12:28 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:12:28 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:12:29 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:29 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:12:30 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:30 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:30 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:30 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:30 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:12:31 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:12:31 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:31 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:12:31 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:12:31 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:12:31 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:12:31 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:31 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:32 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:32 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:32 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:32 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:12:33 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:33 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:33 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:34 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:34 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:12:34 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:34 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:12:34 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:34 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:35 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:12:35 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:35 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:12:35 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:12:35 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:12:35 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:12:35 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:35 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:12:35 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:12:35 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:12:35 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:36 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:12:36 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:12:36 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:37 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:37 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:12:37 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:37 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:37 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:38 [WARN] agent: Check "mr-kafka" is now warning 2021/05/10 00:12:38 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:38 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:38 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:38 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:12:39 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:39 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:41160->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:12:40 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:40 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:40 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:41 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:12:41 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:12:41 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:12:41 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:12:41 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:12:41 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:41 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:41 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:12:42 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:42 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:42 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:12:42 [WARN] agent: Check "elasticsearch-write-health" is now warning 2021/05/10 00:12:42 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:42 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:43 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:12:44 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:44 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:12:44 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:12:44 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:12:44 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:45 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:45 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:45 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:12:45 [WARN] agent: Check "aaf-service" is now critical 2021/05/10 00:12:45 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:12:45 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:45 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:46 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:12:46 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:46 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:12:46 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:12:46 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:46 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:12:46 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:46 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:47 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:12:47 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:47 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:12:47 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:47 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:48 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:48 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:48 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:12:48 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:48 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:49 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:49 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:12:49 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:49 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:50 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:50 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:50 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:12:50 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:50 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:12:50 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:12:50 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:12:51 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:12:51 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:12:51 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:12:51 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:12:51 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:12:52 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:52 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:52 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:52 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:52 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:12:53 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:12:53 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:53 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:54 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:12:54 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:54 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:54 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:41438->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:12:55 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:12:55 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:55 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:55 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:56 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:56 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:12:56 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:56 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:57 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:12:57 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:57 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:12:57 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:12:57 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:12:57 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:57 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:12:57 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:57 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:58 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:12:58 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:12:58 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:12:59 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:00 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:00 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:00 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:00 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:13:00 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:13:00 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:01 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:13:01 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:01 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:13:01 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:13:01 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:13:01 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:13:01 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:01 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:13:01 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:02 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:02 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:02 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:03 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:13:03 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:03 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:03 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:03 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:13:04 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:04 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:13:04 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:04 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:13:04 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:04 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:05 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:05 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:13:05 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:13:05 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:05 [WARN] agent: Check "aaf-service" is now critical 2021/05/10 00:13:05 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:13:05 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:13:05 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:13:05 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:06 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:13:06 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:07 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:07 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:13:07 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:07 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:13:07 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:07 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:08 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:13:08 [WARN] agent: Check "mr-kafka" is now warning 2021/05/10 00:13:08 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:13:08 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:08 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:08 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:08 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:13:09 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:13:09 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:09 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:41676->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:13:10 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:10 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:10 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:11 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:13:11 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:13:11 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:13:11 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:13:11 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:11 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:12 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:12 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:12 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:13:12 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:12 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:13 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:13:13 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:13:14 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:14 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:13:14 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:14 [INFO] agent: Synced check "multicloud-starlingx" 2021/05/10 00:13:14 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:13:15 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:15 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:15 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:13:15 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:15 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:16 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:16 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:13:16 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:13:16 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:13:16 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:16 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:13:16 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:16 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:17 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:17 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:13:17 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:13:17 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:17 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:18 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:18 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:18 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:13:18 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:18 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:19 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:13:19 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:13:19 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:19 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:13:19 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:19 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:20 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:20 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:20 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:13:20 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:13:20 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:20 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:13:20 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:13:20 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:13:21 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:13:21 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:13:21 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:13:21 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:13:22 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:22 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:22 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:22 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:23 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:13:23 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:23 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:23 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:13:23 [INFO] agent: Synced check "aai-resources" 2021/05/10 00:13:24 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:24 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:24 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:13:24 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:41922->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:13:25 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:13:25 [WARN] agent: Check "aaf-service" is now critical 2021/05/10 00:13:25 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:25 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:25 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:25 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:13:26 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:26 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:26 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:26 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:13:27 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:27 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:27 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:13:27 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:13:27 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:27 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:28 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:28 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:13:29 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:29 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:13:30 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:30 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:30 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:13:30 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:13:30 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:30 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:31 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:13:31 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:31 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:13:31 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:13:31 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:13:31 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:13:31 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:13:31 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:31 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:32 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:13:32 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:32 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:13:32 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:32 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:33 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:13:33 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:33 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:33 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:13:33 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:34 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:34 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:13:34 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:34 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:34 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:34 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:13:35 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:35 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:13:35 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:13:35 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:35 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:13:35 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:13:35 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:13:35 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:36 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:36 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:13:37 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:13:37 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:37 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:37 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:13:37 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:37 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:38 [WARN] agent: Check "mr-kafka" is now warning 2021/05/10 00:13:38 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:38 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:38 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:38 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:13:39 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:13:39 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:39 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:42162->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:13:40 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:40 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:40 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:41 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:13:41 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:13:41 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:13:41 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:13:41 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:13:41 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:13:41 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:41 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:42 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:42 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:13:42 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:42 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:13:42 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:42 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:43 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:13:44 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:44 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:44 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:13:45 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:45 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:45 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:13:45 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:13:45 [WARN] agent: Check "aaf-service" is now critical 2021/05/10 00:13:45 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:45 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:46 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:46 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:13:46 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:46 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:46 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:13:46 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:47 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:47 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:47 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:13:47 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:48 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:13:48 [WARN] agent: Check "elasticsearch-write-health" is now warning 2021/05/10 00:13:48 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:48 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:48 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:13:48 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:13:48 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:13:48 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:48 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:49 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:49 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:13:49 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:49 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:49 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:13:50 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:50 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:50 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:13:50 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:50 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:13:50 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:13:50 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:13:51 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:13:51 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:13:51 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:13:51 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:13:52 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:13:52 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:52 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:13:52 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:52 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:52 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:53 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:13:53 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:53 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:13:53 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:54 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:13:54 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:54 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:54 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:42418->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:13:55 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:13:55 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:13:55 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:55 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:55 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:56 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:56 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:56 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:56 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:13:57 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:57 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:57 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:13:57 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:13:57 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:57 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:58 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:13:58 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:13:59 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:13:59 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:00 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:00 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:00 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:00 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:01 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:01 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:14:01 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:14:01 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:14:01 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:14:01 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:14:01 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:14:01 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:01 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:01 [INFO] agent: Synced check "so-api-healthcheck" 2021/05/10 00:14:02 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:02 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:02 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:03 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:14:03 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:14:03 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:14:03 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:03 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:03 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:04 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:14:04 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:14:04 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:04 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:14:04 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:14:04 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:04 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:04 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:14:04 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:05 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:05 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:14:05 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:14:05 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:05 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:14:05 [WARN] agent: Check "aaf-service" is now critical 2021/05/10 00:14:05 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:14:05 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:14:05 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:14:05 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:06 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:06 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:14:07 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:07 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:07 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:14:07 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:07 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:08 [WARN] agent: Check "mr-kafka" is now warning 2021/05/10 00:14:08 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:08 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:08 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:08 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:14:09 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:09 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:42650->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:14:10 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:14:10 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:10 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:10 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:11 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:14:11 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:14:11 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:14:11 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:14:11 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:14:11 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:11 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:12 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:12 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:12 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:14:12 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:12 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:13 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:14:14 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:14:14 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:14 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:14:14 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:14:14 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:15 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:15 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:15 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:14:15 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:14:15 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:15 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:16 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:16 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:14:16 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:16 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:16 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:16 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:14:17 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:14:17 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:17 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:17 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:14:17 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:18 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:18 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:18 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:14:18 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:18 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:19 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:19 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:14:19 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:19 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:20 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:20 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:20 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:14:20 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:14:20 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:14:20 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:20 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:14:20 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:14:20 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:14:21 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:14:21 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:14:21 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:14:21 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:14:21 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:14:21 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:14:22 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:22 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:22 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:22 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:23 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:14:23 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:23 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:24 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:14:24 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:24 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:24 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:42908->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:14:25 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:14:25 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:14:25 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:14:25 [WARN] agent: Check "aaf-service" is now critical 2021/05/10 00:14:25 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:25 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:25 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:26 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:14:26 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:26 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:26 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:26 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:14:27 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:14:27 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:27 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:27 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:14:27 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:14:27 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:27 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:28 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:28 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:14:29 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:30 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:30 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:30 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:30 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:31 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:31 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:14:31 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:14:31 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:14:31 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:14:31 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:14:31 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:31 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:32 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:14:32 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:32 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:32 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:33 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:14:33 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:14:33 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:33 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:33 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:34 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:34 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:14:34 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:34 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:34 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:14:34 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:35 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:35 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:14:35 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:35 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:14:35 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:14:35 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:14:35 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:14:35 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:36 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:14:36 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:14:36 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:14:36 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:14:36 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:37 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:14:37 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:37 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:14:37 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:14:37 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:37 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:37 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:37 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:14:38 [WARN] agent: Check "mr-kafka" is now warning 2021/05/10 00:14:38 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:38 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:38 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:38 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:14:39 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:39 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:43140->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:14:40 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:40 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:40 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:41 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:14:41 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:14:41 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:14:41 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:14:41 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:41 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:42 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:42 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:42 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:14:42 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:42 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:43 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:14:43 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:14:43 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:14:44 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:44 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:44 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:14:45 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:45 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:45 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:14:45 [WARN] agent: Check "aaf-service" is now critical 2021/05/10 00:14:45 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:45 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:46 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:46 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:14:46 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:46 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:46 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:47 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:14:47 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:14:47 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:14:47 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:47 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:47 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:14:47 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:48 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:14:48 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:48 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:48 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:14:48 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:48 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:49 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:14:49 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:49 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:49 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:14:49 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:50 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:50 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:50 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:50 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:14:50 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:14:50 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:14:50 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:14:51 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:14:51 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:14:51 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:14:51 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:14:52 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:52 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:14:52 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:14:52 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:52 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:52 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:53 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:14:53 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:53 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:53 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:14:53 [WARN] agent: Check "elasticsearch-write-health" is now warning 2021/05/10 00:14:54 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:14:54 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:54 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:14:54 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:54 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:43392->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:14:55 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:14:55 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:55 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:55 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:56 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:56 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:56 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:57 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:14:57 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:57 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:57 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:14:57 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:14:57 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:57 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:58 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:14:58 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:14:58 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:58 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:14:59 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:14:59 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:14:59 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:15:00 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:00 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:00 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:00 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:01 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:01 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:15:01 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:15:01 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:15:01 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:01 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:15:01 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:15:01 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:02 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:02 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:02 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:03 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:15:03 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:03 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:03 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:04 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:04 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:04 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:15:04 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:04 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:15:04 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:05 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:15:05 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:05 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:15:05 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:15:05 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:05 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:15:05 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:15:05 [WARN] agent: Check "aaf-service" is now critical 2021/05/10 00:15:05 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:15:05 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:15:05 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:06 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:07 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:15:07 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:07 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:07 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:07 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:07 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:15:08 [WARN] agent: Check "mr-kafka" is now warning 2021/05/10 00:15:08 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:08 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:15:08 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:08 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:08 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:15:08 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:15:09 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:15:09 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:15:09 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:15:09 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:09 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:43632->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:15:10 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:15:10 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:10 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:10 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:11 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:15:11 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:15:11 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:11 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:15:11 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:15:11 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:12 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:12 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:12 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:15:12 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:13 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:13 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:15:14 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:14 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:14 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:15:15 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:15 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:15 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:15:15 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:15:15 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:15 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:16 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:15:16 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:16 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:15:16 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:16 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:16 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:17 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:15:17 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:17 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:17 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:15:17 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:18 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:18 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:18 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:15:18 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:18 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:19 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:19 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:19 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:15:19 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:20 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:15:20 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:20 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:15:20 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:20 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:20 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:15:20 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:15:20 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:15:20 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:15:21 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:15:21 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:15:21 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:15:21 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:15:21 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:15:21 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:15:22 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:22 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:22 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:22 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:23 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:15:23 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:23 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:24 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:15:24 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:15:24 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:24 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:24 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:15:24 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:43886->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:15:25 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:15:25 [WARN] agent: Check "aaf-service" is now critical 2021/05/10 00:15:25 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:15:25 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:25 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:25 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:26 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:26 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:26 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:27 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:15:27 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:15:27 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:27 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:27 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:15:27 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:15:27 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:28 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:28 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:28 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:15:29 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head http://multicloud-windriver:9005/api/multicloud-titaniumcloud/v1/swagger.json: dial tcp: lookup multicloud-windriver on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:30 [WARN] agent: Check "vfc-nokiav2vnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-v2vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:30 [WARN] agent: Check "elasticsearch" HTTP request failed: Get http://aai-elasticsearch:9200/_cat/indices?v: dial tcp: lookup aai-elasticsearch on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:30 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:30 [WARN] agent: Check "appc-sdnhost" HTTP request failed: Head http://appc-sdnhost:8282/apidoc/explorer/index.html: dial tcp: lookup appc-sdnhost on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:31 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:15:31 [WARN] agent: Check "vfc-ztesdncdriver" socket connection failed: dial tcp: lookup vfc-zte-sdnc-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:31 [WARN] agent: Check "sdnc-sdnhost" HTTP request failed: Head http://sdnc:8282/apidoc/explorer/index.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:15:31 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:15:31 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:15:31 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:15:31 [WARN] agent: Check "sdnc-dgbuilder" HTTP request failed: Head http://sdnc-dgbuilder:3000/: EOF 2021/05/10 00:15:31 [WARN] agent: Check "policy-nexus-internal-metrics" HTTP request failed: Get http://nexus:8081/nexus/internal/metrics: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:31 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:15:31 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:15:31 [WARN] agent: Check "log-logstash-internal-server-jvm-info" HTTP request failed: Get http://log-ls-http:9600/_node/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:32 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:15:32 [WARN] agent: Check "pdp" HTTP request failed: Get https://pdp:8081/pdp/test: dial tcp: lookup pdp on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:32 [WARN] agent: Check "vfc-emsdriver" socket connection failed: dial tcp: lookup vfc-ems-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:32 [WARN] agent: Check "log-logstash-internal-server-plugin-info" HTTP request failed: Get http://log-ls-http:9600/_node/plugins?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:33 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:15:33 [WARN] agent: Check "log-elasticsearch-server" HTTP request failed: Get http://log-es:9200/_cluster/health?pretty: dial tcp: lookup log-es on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:33 [WARN] agent: Check "log-logstash-internal-server-process-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/process?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:33 [WARN] agent: Check "log-logstash-internal-server-node-info" HTTP request failed: Get http://log-ls-http:9600/_node/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:34 [WARN] agent: Check "vfc-catalog" socket connection failed: dial tcp: lookup vfc-catalog on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:34 [WARN] agent: Check "policy-nexus-local-status" HTTP request failed: Get http://nexus:8081/nexus/service/local/status: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:34 [WARN] agent: Check "model-loader-process" is now warning 2021/05/10 00:15:34 [WARN] agent: Check "log-logstash-tcp" socket connection failed: dial tcp: lookup log-ls on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:34 [WARN] agent: Check "sdnctldb01" socket connection failed: dial tcp: lookup sdnc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:34 [WARN] agent: Check "so-mariadb" is now warning 2021/05/10 00:15:35 [WARN] agent: Check "log-logstash-internal-server-os-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:35 [WARN] agent: Check "clamp-mariadb" is now warning 2021/05/10 00:15:35 [WARN] agent: Check "vfc-workflowengineactiviti" socket connection failed: dial tcp: lookup vfc-workflow-engine on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:35 [WARN] agent: Check "search-data-service-api" is now warning 2021/05/10 00:15:35 [WARN] agent: Check "sparky-be-process" is now warning 2021/05/10 00:15:35 [WARN] agent: Check "appc-dgbuilder" HTTP request failed: Head http://appc-dgbuilder:3000/: EOF 2021/05/10 00:15:35 [WARN] agent: Check "sdc-catalog-healthcheck" is now critical 2021/05/10 00:15:35 [WARN] agent: Check "sdnctldb02" socket connection failed: dial tcp: lookup sdnc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:36 [INFO] agent: Synced check "aai-service" 2021/05/10 00:15:36 [WARN] agent: Check "appc-sdnctldb02" socket connection failed: dial tcp: lookup appc-sdnctldb02 on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:37 [WARN] agent: Check "vfc-nokiavnfmdriver" socket connection failed: dial tcp: lookup vfc-nokia-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:37 [WARN] agent: Check "so-camunda-healthcheck" is now critical 2021/05/10 00:15:37 [WARN] agent: Check "msb-eag" HTTP request failed: Head http://msb-eag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:15:37 [WARN] agent: Check "drools" HTTP request failed: Get https://drools:6969/healthcheck: dial tcp: lookup drools on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:37 [WARN] agent: Check "log-logstash-internal-server-jvm-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats/jvm?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:37 [WARN] agent: Check "brmsgw-tcp" socket connection failed: dial tcp: lookup brmsgw on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:37 [WARN] agent: Check "mr-zookeeper" is now warning 2021/05/10 00:15:38 [WARN] Check "sdc-be-healthcheck": Timed out (1s) running check 2021/05/10 00:15:38 [WARN] agent: Check "mr-kafka" is now warning 2021/05/10 00:15:38 [WARN] agent: Check "framework" HTTP request failed: Head http://framework:9001/api/multicloud/v0/swagger.json: dial tcp: lookup framework on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:38 [WARN] agent: Check "pap" HTTP request failed: Get https://pap:9091/pap/test: dial tcp: lookup pap on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:38 [WARN] agent: Check "appc-sdnctldb01" socket connection failed: dial tcp: lookup appc-sdnctldb01 on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:38 [WARN] agent: Check "vid-mariadb" is now warning 2021/05/10 00:15:39 [WARN] agent: Check "log-logstash-internal-server-gi" HTTP request failed: Get http://log-ls-http:9600/?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:39 [WARN] agent: Check "multicloud-starlingx" HTTP request failed: Head http://multicloud-starlingx:9009/api/multicloud-starlingx/v0/swagger.json: read tcp 10.242.168.100:44116->10.96.80.214:9009: read: connection reset by peer 2021/05/10 00:15:40 [WARN] agent: Check "odl-api-healthcheck" HTTP request failed: Post http://sdnc:8282/restconf/operations/SLI-API:healthcheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:15:40 [WARN] agent: Check "clamp-health" HTTP request failed: Get http://clamp:8080/restservices/clds/v1/clds/healthcheck: dial tcp: lookup clamp on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:40 [WARN] agent: Check "msb-iag" HTTP request failed: Head http://msb-iag:80/iui/microservices/default.html: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:15:40 [WARN] agent: Check "log-logstash-internal-server-os-info" HTTP request failed: Get http://log-ls-http:9600/_node/os?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:40 [WARN] agent: Check "log-elasticsearch-tcp" socket connection failed: dial tcp: lookup log-es-tcp on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:41 [WARN] agent: Check "Policy-mariadb-healthcheck" is now warning 2021/05/10 00:15:41 [WARN] agent: Check "sdnc-dbhost-healthcheck" is now warning 2021/05/10 00:15:41 [WARN] agent: Check "policy-nexus-internal-healthcheck" HTTP request failed: Get http://nexus:8081/nexus/internal/healthcheck: dial tcp: lookup nexus on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:41 [WARN] agent: Check "vid-server" HTTP request failed: Get http://vid:8080/vid/healthCheck: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 2021/05/10 00:15:41 [WARN] agent: Check "log-logstash-internal-server-node-stat" HTTP request failed: Get http://log-ls-http:9600/_node/stats?pretty: dial tcp: lookup log-ls-http on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:41 [WARN] agent: Check "so-jra-healthcheck" is now critical 2021/05/10 00:15:41 [WARN] agent: Check "so-api-healthcheck" is now critical 2021/05/10 00:15:42 [WARN] Check "sdc-cs-healthcheck": Timed out (1s) running check 2021/05/10 00:15:42 [WARN] Check "sdc-fe-healthcheck": Timed out (1s) running check 2021/05/10 00:15:42 [WARN] agent: Check "vfc-jujudriver" socket connection failed: dial tcp: lookup vfc-juju-vnfm-driver on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:42 [WARN] agent: Check "vfc-workflow" socket connection failed: dial tcp: lookup vfc-workflow on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:42 [WARN] agent: Check "data-router-process" is now warning 2021/05/10 00:15:43 [WARN] agent: Check "log-kibana-server" HTTP request failed: Head http://log-kibana:5601/status: dial tcp: lookup log-kibana on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:43 [WARN] agent: Check "vfc-resmgr" socket connection failed: dial tcp: lookup vfc-resmgr on 10.96.0.10:53: server misbehaving 2021/05/10 00:15:43 [WARN] agent: Check "appc-dbhost-healthcheck" is now warning 2021/05/10 00:15:43 [WARN] Check "sdc-titan-healthcheck": Timed out (1s) running check 2021/05/10 00:15:44 [WARN] agent: Check "multicloud-windriver" HTTP request failed: Head h